Personalization of Interactive Multimedia Services: a Research and Development Perspective

Personalization of Interactive Multimedia Services: A Research and Development Perspective

Edited by

José Juan Pazos Arias, Carlos Delgado Kloos and Martín López Nores

Contents

1 Preface ...... 3

Part I Legal and Technological Foundations

2 Legal Framework for Personalization-Based Business Models Teresa Rodríguez de las Heras Ballell ...... 9 3 Technological Platforms, Convergence and Adaptation of Interactive Contents Fabio Paternò ...... 23 4 Filtering Techniques for Selection of Contents and Products Qing Li, Yuanzhu Peter Chen, Zhangxi Lin...... 41 5 Personalization and Contextualization of Media Marcus Specht, Andreas Zimmermann and Tim de Jong ...... 55 6 Contextualized Recommender Systems: Data Model and Recommendation Process Wolfgang Woerndl and Johann Schlichter ...... 65 7 Enhanced Media Descriptions for the Automatic Adaptation of Audiovisual Content Retrieval Iván Cantador, Fernando López, Jesús Bescós, Pablo Castells, and José M. Martínez ...... 77 8 A Semantics-Based Approach Beyond Traditional Personalization Paradigms Yolanda Blanco-Fernández, Alberto Gil-Solla and Manuel Ramos-Cabrer...... 89

Part II Applications

9 PROGNOSIS: An Adaptive System for Virtual Technical Documentation Nikolaos Lagos and Rossitza M. Setchi ...... 105 Contents 1

10 Investigating the Applicability of Content-Based Filtering on Digital Television Advertisements George Lekakos ...... 119 11 The Role of Communities in E-Commerce Rodolfo Carpintier Santana ...... 129 12 Personalized Education Rosa M. Carro, Estefanía Martín ...... 135 13 New Trends for Personalised T-Learning Marta Rey-López, Ana Fernández-Vilas, Rebeca P. Díaz-Redondo ...... 149 14 M-Learning: Personalized and Pervasive Education Anytime, Anywhere and for Everybody Mario Muñoz Organero, Elena Yndurain, Carlos Delgado Kloos ...... 163 15 Self-Adaptive Portals: E-Government Case Study Ljiljana Stojanovic and Nenad Stojanovic ...... 183 16 Use Cases and Challenges of Personalized Healthcare Mikhail Simonov ...... 197 17 Context-Aware Personalization for Services in the Smart Home: A Semantic Approach for OSGi services Rebeca P. Díaz-Redondo, Ana Fernández-Vilas, Manuel Ramos-Cabrer ...... 207

18 New Perspectives to Lifecycle Management of Context-Aware and Personalised Services Bharat Bhushan, Stephan Steglich, Christian Räck and Bernd Mrohs ...... 223 19 The Role of Service Discovery in Context-Aware, Pervasive Computing Applications Carlos García-Rubio and Celeste Campo ...... 233 20 Personalizing Pedestrian Location Services through Ontologies and Rules Vassileios Tsetsos, Vassilis Papataxiarhis and Stathes Hadjiefthymiades ...... 245

Part III Open Problems and Current Research Trends

21 Conclusions and Future Trends José Juan Pazos Arias, Carlos Delgado Kloos and Martín López Nores ...... 259

1 Preface

During the last few years, we have witnessed the deployment of new digital communication technologies such as satellite, terrestrial or cable forms of Digital TV broadcasting, 3G mobile networks, xDSL, domotic networks, WiMax, ..., which make up a wealthy scenario that will be further enhanced with the ongoing developments of TV broadcasting networks for mobile devices through standards like DVB-H in Europe, DMB in South Korea or MediaFLO in North America. In this new scenario, the users can benefit from an ever growing bandwidth at lower pricer, which promotes the development of new communication services. We may say that we are living a process similar to what has been happening with informatic equipment, characterized by a progressive increase of memory and computational power at decreasing cost, which favored the access to computers to massive sectors of society. By means of diverse initiatives, public administrations are striving to exploit this situation of unnegligible advantages and opportunities, pursuing the goal of achieving universal and ubiquitous access to the Information Society, thus reducing the effects of the digital divide that draws a clear distinction between those who make frequent use of the information and communication technologies and those who do not know these technologies or find severe difficulties in their use. >From these facts, it would be expectable to see a rapid penetration of technology in many different domains of applications, including entertainment, news, education and training, administration, electronic commerce, healthcare, etc. Certainly, the offerings have augmented much, with a clear example in the number of TV channels that can be already received in homes. However, the penetration of information services has been noticeably much slower than expected, which is mainly due to the problem of information overload: frequently, the users are faces with such an overwhelming amount of information that it is very difficult for them to make decisions or remain informed about a given topic. This poses enormous challenges not only to the users themselves, but also to the market forces that would expect to get revenues from the contents and services they develop, because their presence is quickly diluted in a vast disorganized amount of pieces of information and services of a similar nature. A similar situation started to be noticeable during the mid 1990s with the exponential growth of the Internet, that made the users fell disoriented among the myriad of web contents available through their PCs. This context gave birth to a set of tools commonly referred to as search engines (e.g. Google and Yahoo), that would retrieve relevant web pages in response to user-entered queries. That model proved effective and successful, with millions of people using them to find pieces of information and services in the World Wide Web. However, the advent of new access devices different from the PC (DTV set-top boxes, mobile and smart phones, PDAs, media players, car navigators, etc) introduces a new range of consumption habits and usage contexts that render the search engine 4 paradigm insufficient. In this broader picture, it is no longer realistic to think that the users will bother to purposefully visit a given site, enter a query with the details of what they want, and then select the contents they might be interested in from among those in a list. The reasons may have to do with the users adopting an inactive or predominantly passive role (e.g. while driving or watching TV in the living room); the devices providing no accessible means to enter text (as it happens with remote controls); only a downstream communication channel being available (as in broadcasting environments); or users feeling uneasy with the interfaces provided. To tackle these issues, a large body of research is being devoted nowadays to the design and provision of personalized information services, with a new paradigm of recommender systems proactively selecting the particular pieces of information and functionality that match the preferences, interests and needs of each individual user at any given moment. In this book, we have gathered contributions of experts from academia and industry worldwide, in order to provide a research and development perspective on personalized information services, to help the reader understand their goals, the major steps of their evolution, the functionalities they can deliver to the users and the main challenges they still face. The contributed chapter have been arranged into three parts: • Part I is devoted to presenting the legal and technological foundations involved with the development of personalized information services. – To begin with, it is noticeable that personalization is typically opposed to privacy, since its effectiveness depends on accumulating information about the users, while there is a body of legislation to restrict the management of personal information by commercial or administrative entities. In Chapter 2, Teresa Rodríguez de las Heras Ballell provides an in-depth (though accessible) discussion of the most significant legal issues arising from personalization strategies, together with a description of personalization-based business models and legal mechanisms to implement personalization. – As mentioned above, personalization arose in the context of the Internet as accessed through personal computers, but its scope is now growing much to become a multi-device experience. In Chapter 3, Fabio Paternò analyzes the aspects related to the provision of personalized services over various technological platforms, addressing important questions of convergence, adaptation of interactive contents and migration of user interfaces, with a comprehensive list of the many dimensions of the related research. – Personalization is achieved by matching characterizations of the contents and services available with the information stored in profiles that capture the preferences, interests, likings and needs of the users. This process is commonly referred to as filtering. In Chapter 4, Qing Li provides insight into the existing approaches to do so, which lie at the core of most recommender systems regardless of particular application domains. – One of the features that is concentrating most efforts in personalization research is context awareness, which aims at acquiring information about the physical and social situation in which computational devices are embedded to maximize the value of the information delivered to the user. In Chapter 5, Marcus Specht, Andreas Zimmermann and Tim de Jong explain the phases of information acquisition, representation and management in context- aware systems, with illustrative examples in ubiquitous computing for learning and information delivery. – Traditionally, the research and development efforts involved with filtering techniques paid little attention to the advances in context awareness, and vice versa. In Chapter 6, Wolfgang Woerndl and Johann Schlichter provide a comprehensive overview about the state-of-the-art 1 Preface 5

in contextualized recommender systems, which introduce context as a new dimension of the user profiles to drive the selection of contents and services. – To enable automatic selection of contents and services, it is necessary to define standards for the characterization of resources, considering their format, length, topics, functionality, etc. Also, normalization is needed in what concerns the data structures and fields to store and manage information about users and context. In Chapter 7, Jesús Bescós et al. enumerate solutions available for these tasks, building upon the expressive capabilities of the MPEG standards. – To finish with the technological foundations of personalization, in Chapter 8, Yolanda Blanco-Fernández et al. look at the internals of the filtering techniques, explaining the ongoing shift from syntactic reasoning processes (predominant until very recently) to semantic reasoning ones. The semantic approach yields more powerful means to discover relationships between user preferences, contexts, pieces of content and services, which in turn results in greater personalization quality. • Part II provides an overview of the functionalities available nowadays in diverse areas of application. – The first studies on personalized information services arose in the field of adaptive content retrieval, aimed at providing the users with pieces of content tailored to their evolving interests and needs. Recent advances in this line can be found in Chapter 9 by Nikolaos Lagos and Rossitza M. Setchi, instantiated in a system that deliver personalized technical documentation taking into account user and context characteristics. – Personalized e-commerce services have been around on the Internet for some years. In Chapter 10, Rodolfo Carpintier Santana analyzes those development in the light of the ongoing shift to a new way of conceiving the World Wide Web around communities of users (the so-called Web 2.0). As electronic commerce also evolves to embrace new technological platforms, Chapter 11 by George Lekakos reports the results of experiments with personalized advertising in Digital TV. – Personalized learning systems developed hand in hand with adaptive content retrieval from the late 1990s, with the common goal of delivering the best suited educational activities to the users. Notwithstanding, the same goal is pursued in very different ways depending on the devices employed to interact with the learning material. In this regard, Chapter 12 by Rosa M. Carro explores the perspective of learning through Internet-enabled personal computers (e- learning); Chapter 13 by Marta Rey-López et al. focuses on Digital TV settings (t-learning); and Chapter 14 by Mario Muñoz Organero et al. finally deals with the learning opportunities enabled by mobile devices (m-learning). – The need for personalization in e-government arises from the fact that users, especially those with little web experience, often get lost in the information space of a portal and need specific hints that are usually provided in an administrative building. In Chapter 15, Ljiljana and Nenad Stojanovic present a novel approach to create personalized e-government portals harnessing the power of semantic reasoning technologies. – Personalized healthcare services aim at supporting the users’ well-being when they are healthy and their medical treatments when they are sick, taking into account the individual’s unique biological, social and cultural characteristics. To help realize the wide range of technologies and applications involved, Mikhail Simonov describes in Chapter 16 a number of use cases of personalized healthcare: prevention, alert service, care and nutrition, food, informed consent, and home living and elderly. – Personalized is also envisaged to play a role in the networked home of the future, which may become the central point for all of a user’s interactions with information services and devices. 6

In Chapter 17, Rebeca P. Díaz-Redondo et al. look at the most relevant issues involved with personalization in domotic environments, with a semantic approach to orchestrating the many appliances that may be installed in a house. – One of the fields that is concentrating more attention nowadays is that of ubiquitous computing (also referred to as “pervasive computing” or “ambient intelligence”), which consists of embedding context-aware information processing into everyday objects and activities, also out of the home. To gain insight into the development and deployment of ubiquitous and personalized services, Chapter 18 by Bharat Bhushan et al. explains relevant issues of lifecycle management, whereas Chapter 19 by Carlos García-Rubio and Celeste Campo looks at the important tasks of service discovery. Finally, in Chapter 20, Vassileios Tsetsos et al. present an application of these technologies plus a core of semantic modeling and reasoning in pedestrian wayfinding and location-based services. • To finish, Part III of the book provides a discussion of the most important open problems and challenges in personalization research overall, identified jointly by the editors and some of the chapter authors. Overall, the technical level of the book makes it suitable for third-year or posterior undergraduate courses on information systems, and also for graduate ones. Notwithstanding, we believe it should be appealing for any one involved with the design and development of information services, in any of the many applications and technological platforms considered.

Hope you will have a nice read!

The editors

José Juan Pazos Arias Carlos Delgado Kloos Martín López Nores Part I

Legal and Technological Foundations

2 Legal Framework for Personalization-Based Business Models

Teresa Rodríguez de las Heras Ballell

Universidad Carlos III de Madrid [email protected]

Summary. Unlike mainstream belief, Internet does not embody a global marketplace providing uniform, standardized and massively produced goods and services intended to attend a purported universal and undistin- guished demand. Contrarily, Internet offers an unthinkable range of personalizing possibilities. Even more, success in Internet appears today not to repose on the “dimensions” of the demand yet, but to depend on the “precision” of the offer. In contrast with its rival mass media largely assuming passive users, Internet does not only encourage user interaction, but it is indeed nourished by contents, data, information inferred from behaviors, searches, responses performed by its own users acting as prosumers (neologism coming from a combination of consumer and producer). Companies have become aware of the extraordinary power of such an input data flow to model access profiles, to tailor information items with regard to access devices or applications, to recommend goods and services according to users’ interests and preferences, to place ads closely connected to search results, emails texts or prior consultations, to set up specialized business platforms (e-marketplaces) suited to members’ needs. Such strategies, fuelled by sensitive personal data and behavioral information, are enough to evoke certain legal concerns. Business models based on personalization strategies, recommender systems or context-aware services encounter delicate privacy issues. Assessment of such privacy concerns entails the prior understanding of legal infrastructure enabling personalization within electronic relationships framework. The aim of this paper is to explore the most noteworthy legal issues arising from personalization strategies. The first step is to gain insight of how legal instruments help business to manage personalization in so-called “age of access”, where products and services are defined indeed not as close items but as open “set of uses” (functionalities) in accordance to agreed license terms. License agreements embody an appealing inherent paradox as far as they symbolize the struggle for combining the standard form contract logic of massive trade with the most demanding requirements for personalizing. Contractual terms of use are a key legal tool to rule users browsing web site, to govern participants accessing e-marketplaces, to tailor access profiles to contents, to conduct recommendation processes. Several legal concerns are aroused by use licenses as personalization managing devices; among them, certain challenging issues regarding privacy. A thin line separates reasonable personalizing strategies from unlawful invasion of users’ privacy. Accordingly, the paper next recreates several scenarios illustrating the unsettled relationship between personalization and privacy protection and traces key principles and rules to manage this tough conflict.

2.1 Personalization in the Digital Economy

The advent of digital world, settled on the infrastructure provided by Internet, has turned information into the life blood of global economy that struggles to trigger its metamorphosis to knowledge. Information processing embodies the keys of the digital economy’s success, but at the same time conceals all the risks attributed to new technologies. 10 Teresa Rodríguez de las Heras Ballell

Unlike mainstream belief, Internet does not embody a global even space (marketspace) providing uniform, standardized and massively produced goods and services intended to attend a purported universal and undis- tinguished demand. Contrarily, Internet offers an unthinkable range of personalizing possibilities. Even more, success in Internet appears today not to repose on the "dimensions" of the demand yet, but to depend on the “precision” of the offer. Collecting, managing and transferring information are the critical tasks supporting the provision of value-added services and the devising of successful personalization-based business strategies. Companies have become aware of the extraordinary power of such data flow to model access profiles, to tailor information items with regard to access devices or applications, to recommend goods and services according to users’ interests and preferences, to place ads closely connected to search results, emails texts or prior consultations, to set up specialized business platforms (e-marketplaces) suited to members’ needs. Nonethe- less, information manipulation arouses, at the same time, delicate privacy issues. Accordingly, such strategies, launched by personalization-based business models, recommender systems or context-aware service providers, as for fuelled by sensitive personal data and behavioral information, are enough to evoke certain legal concerns. Most representative actors of digital economy, such as Google, have embarked on a personalizing strategies race. Google, the world’s biggest search engine, collects data from users’ searches to provide the best possible search for the user, improve its value-added services, sharpen ads’ target, tailor contents according to users’ preferences and develop new services and products in users’ benefit. In the wild new world of Internet, where individuals fight not to be drowned in the flood of information lacking of credibility references, personalizing strategies, recommender systems and context-aware services are unquestionably the most value-added business offers. Nevertheless, recent declarations of intent to offer new sophisticated personalized services have been received with mistrust and certain alarmism from diverse opposing constituencies. Google mission statement condensed in the sentence “the goal is to enable users to be able to ask the question such as “What shall I do tomorrow?” and “What job shall I take?” has aroused the most catastrophist concerns of an Orwellian vision. Concerns that have been aggravated by Google bid for DoubleClick, a company that helps to place and track online advertisements through sophisticated targeting capabilities. To reinforced privacy worries, complaints from its nearest competitors about the risk of the deal to damage fair competition have joined. Meanwhile, European data protection and competition watchdogs, in contrast with other alarming voices, have been nonetheless more cautious. On the one hand, Article 29 Data Protection Working Party has expressly revealed its concern just as regards information storage period policy ruling Goggle services. On the other hand, European Commission is investigating eventual antitrust considerations of envisaged Google-DoubleClick merger to decide to clear the deal or to open an in-depth probe. No privacy inquiry is nevertheless hinted thereupon. Even if it is undeniable that delicate issues are involved, perception of personalization strategies seems to be too influenced by alarmist visions on the scope of privacy right and the threats that new technologies impose thereupon. Hence, a careful analysis, firstly, of the scope and constituent elements of Data Protection and Privacy regulations and, secondly, of the extent of personalization and the mechanisms supporting such business strategies is due in order to readjust the discussion framework. Against such a backdrop, this chapter aims at exploring the most noteworthy legal concerns arising from personalization-based business strategies in the digital space. The starting hypothesis is that, insofar credibility and attention are the scarcest resources in the digital economy, recommender systems and trusted third parties provide extremely appealing and welcome value-added services. Personalization-based business models create value. At the same time, such strategies may imply risks to be efficiently manage. Despite the mainstream perception, any information collecting and processing do not entail the encroaching on privacy spheres insofar as the principle of consent is regarded. A thin line separates reasonable personalizing strategies from unlawful invasion of users’ privacy, but in fact it is a matter of limits. Privacy does not mean anonymity but control on personal data. Section 2.2 of this chapter is devoted to define and distinguish these blurred notions of personalization, privacy and anonymity. On the foundations of such distinction, Section 2.3 aims to trace the guidelines of data protection and privacy regulation. In conformity with such legal background, personalizing and recommending business strategies must be assessed. The first step is to gain insight of how companies implement such strategies considering the diversity of activities nourishing digital economy. Section 2.4 does then explore the range of business models competing in the market in order to find out how personalization aims are devised. From such a market overview, it is inferred that license agreements are the main legal devices applied by business to manage personalization in so-called “age of access”, where products and services are de- 2 Legal Framework for Personalization-Based Business Models 11

fined indeed not as close items but as open “set of uses” (functionalities) in accordance to agreed license terms. Contractual terms of use are key legal tool to rule users browsing web site, to govern participants accessing e-marketplaces, to tailor access profiles to contents, to conduct recommendation processes. Basics of license agreements, most challenging issues arising from the formation process and proposals of guidelines to ensure valid and enforceable contracts are tackled in Section 2.5 of the chapter. An underlying conclusion permeates the whole reasoning. New technologies play as regards the matter in hand two and paradoxically opposing roles. From the perspective of a harming action, network effects turn new technologies into pernicious multiplier of damages or potential victims. Such an undesirable risk is aggravated by the difficulties commonly associated to digital space in order to achieve an efficient and enforceable regulation thereof. Internet, a new space lacking of centralized supervisor and seasoned by a feasible anonymity and a natural delocalization, is then perceived as a dangerous receipt for impunity and anarchy. Regulation, either initiatives on international-basis, domestic legislation or self-regulation (codes of conducts), is nevertheless possible and viable. Uncertainty provoked by delocalization in digital space may be overcome by redefining territory-based connecting factors to decide the law to apply to situations taking place in a new space. Verifying such available solutions renders a calming effect that leads us to the other face of new technologies. From the perspective of control possibilities, new technologies enable unimaginable capabilities of monitoring, tracking and ruling conducts. Everything in the digital space is represented by traceable data, behaviors, movements, actions, searches, decisions (transactional data). Although these controlling functions may be exploited in an Orwellian manner1 (Echelon and Carnivore are patent evidences), new technologies truly hold a critical and powerful position within available regulation techniques in the digital space. Technological architecture assists the enforcement process of law and contract in the new space. Technology is indeed an allied actor to traditional regulating mechanisms (market rules, social norms and legislation). Access to an e-marketplace may be controlled by implementing passwords, electronic signatures or biometric identification devices; complying with digital auction rules may be ensured by designing an interface allowing only the option among technically available actions; eventual copyright infringements may be, not only discouraged but also, and above all, pre- vented by rendering technically impossible the implementation of certain violating actions; the enforcement of a software license may be achieved by deactivating or suspending the software in case that the user breaches contractual obligations incumbent on him or her; even privacy itself may be protected with the assistance of a software supporting the automatic compliance of the privacy level preselected by the user (P3P, the Platform for Privacy Preferences). Hence, rather than assuming an avoidable and permanent danger of privacy under new technologies’ reign, privacy enhancing technologies offer a totally different perspective over the problem.

2.2 Personalization, Privacy and Anonymity

Personalization is a value-creating business strategy. Assuming such a premise, the immediate question is why it does then arouse so controversial issues. Business models based on personalized services and recommender systems need to collect and process sensitive data, and they are indeed fuelled by wide-ranging information concerning users’ habits and preferences supplied by searching records, browsing behavior or profiling devised by invisible data-capturing devices (“cookies”). In a very simplistic manner, it may be upheld that collecting too much information for personalizing purposes may imply an unlawful invasion of privacy, whereas an absolute anonymity prevents business from providing personalized services. Such an alternative, in the way it was depicted, leads to an inefficient and undesirable result. Rethinking the involved concepts renders the discussion more reasonable. Privacy, rather than anonymity, has to be interpreted as a right to control our own data and any information related to our preferences, behaviors and interests. It is indeed a question of limits; it is a matter of control. Further precision in the definition of the three concepts concerned may be helpful to unravel the tricky interrelation among them. Firstly, personalization does not name a unique and homogeneous business strategy, but encompasses diverse business models ranging from internet access providers, servers, browsers or searchers to content 1 Rodotà [2004] suggests very skillfully that such a society under surveillance evokes the “Panopticon” of Jeremy Bentham rather than the “Big Brother” of George Orwell. 12 Teresa Rodríguez de las Heras Ballell providers, a large number of varied on-line retail shops or business-to-business e-marketplaces. As detailed below under Section 2.4, wide-ranging formulas competing in the market share personalization as business mission. In addition to a variety of customizing techniques, a clear line sharply distinguishes customizing models from recommender systems. Even if both business strategies are based in tailoring the service or the product being aware of user’s needs, surroundings circumstances or location, the latter provide a value-added service, the recommendation. Whereas the former merely implies the provision of the user with well-tailored information, the latter entails the enhancement of the supply of information with the act of recommending. Recommendation value depends on the reliability of recommender. Information may be true or wrong, accurate or inexact; nevertheless qualifying a recommendation requires using a wider range of grey. Provider’s position is different in each case. Both strategies run on data collected by diverse devices in the course of business relationships. Customization demands a large amount of detailed data about the user, recommendation commonly requires crossing data of a cluster to identify similar users’ profiles. Secondly, privacy is a broader concept than personal data protection [Ballesteros Moffa, 2005, Bergadamo et al., 2005, Kuner, 2007, Herrán Ortiz, 2003, Lisi, 2003, Masimini, 2002]. Even if collecting and processing personal data may involve the encroaching on the data subject’s private sphere, other kind of privacy invading actions does not always concern the manipulation of personal data but the breach of security systems, confidentiality standards or unsolicited commercial communications rules. Albeit the mainstream use of “privacy policy” in Internet, the right to respect for the private and family life, home and communications and the right to the protection of the personal data are separately enshrined by, among other texts, the Charter of Funda- mental Rights of European Union in Article 7 and 8 respectively. Such an independent treatment endorses the distinction upheld above, but its relevance dilutes when specific legislation on protecting privacy is analyzed. From the perspective of the privacy infringement, both risks are nonetheless tackled jointly by regulation applicable to business in digital space. Thus, within the European market, Directive 2002/58/EC of the European Parliament and of the Council, dated on 12th July, concerning the processing of personal data and the protection of privacy in the electronic communications sector (hereinafter, Directive on privacy and electronic communications)2 approaches the problem from the view of the risks created by electronic communications, either entailing the invasion of privacy or an unlawful manipulation of personal data. Any attempt to abstractly define the concept of privacy is a cumbersome task that will surely lead to an open notion of blurred outlines. Addressing the meaning of privacy from the perspective of the potential dan- gers that it must face in the digital environment guides to a more useful and pragmatic result. A realistic and perceptive look at the rapid evolvement of electronic communications has enabled to identify certain scenarios of risk. Firstly, a breach of the security of the network may imply serious risks for users’ privacy, therefore service providers must take appropriate measures to safeguard the security of their services and inform users and subscribers of any special risks thereby and of measures they can take to protect the security of their communications. Secondly, unauthorized access to communications may involve a grave attack to the confidentiality of both contents and any data related to such communications, accordingly, measures should be taken to ensure that confidentiality remains guaranteed in the course of business practice and during the data storage period. Thirdly, certain devices such as so-called spyware, web bugs, hidden identifiers and other similar ones can enter the users’ terminal equipment without their knowledge, gain access to information stored thereupon or trace users’ activities with severe impact on the private sphere of users; consequently, the use of such devices should be allowed only for legitimate purposes and upon the knowledge of the users concerned. Fourthly, traffic data processed and stored by providers contain information on the private life of natural persons and concern the right to respect for their correspondence or concern the legitimate interests of legal persons; hence, they should be erased or made anonymous when they are no longer needed for the purpose of the transmission of a communication of other legitimate purposes (billing, marketing). Fifthly, location data giving the geographic position of the terminal equipment of the mobile user in order to enable the transmission of communications or provide value-added services such as individualized traffic information and guidance to drivers run on data concerning the private sphere of users, so the processing of such data for providing certain services should require the user’s consent. To put an end to this long list of potential risk situations on privacy, it might well be worth alluding to unsolicited communications for direct marketing purposes (automatic calling machines, telefaxes, e-mails or SMS messages) that may represent an additional threat to users’ privacy.

2 O.J. L 201, dated on 31st July 2002. 2 Legal Framework for Personalization-Based Business Models 13

Despite of the first impression that such a lengthy relation of menacing situations may arouse, privacy and personal data protection do not always prevail over any other right, interest or public policy principle in the digital space and, accordingly, certain limits may be imposed thereupon provided that a legitimate purpose is to achieved. In that regard, it is not a question of prohibition, but of balance. Privacy is then a right to control one’s sensitive data. In conformity with such perspective, regulation on protecting privacy and personal data does not intend to ban every collecting or processing action above-mentioned, but establishing the rules allowing users to control their own data instead (see infra Section 2.3). Consistency of such reasoning leads us to affirm that privacy does not mean a right of full anonymity. In other words, privacy and anonymity are to reconcile with other public policy objectives. On the one hand, the possibility of remaining anonymous is essential if the fundamental rights to privacy and freedom of expression are to be maintained in the digital space. On the other hand, the ability to participate in the digital world without revealing one’s identity challenges the viability and the enforceability of other key areas of public policy such as the fight against illegal and harmful content, financial fraud, or copyright infringements3.

2.3 Privacy and Personal Data Protection Keys

Prior to go into the analysis of the personalization-based business models and the potential risks on privacy arising therefrom, it might be well worth tracing the basics of regulation on protecting privacy and personal data with special reference to electronic communications. Such an overview is expected to provide the further discussion with a benchmark to test compliance level of business. The first obstacle that regulation on privacy protection in Internet must face is the territoriality problem. Decentralized structure and a-national character are the most distinctive features of a new space where traditional principles determining applicable law on territory-basis appear to become impracticable. Drafting and agreeing an international standard on personal data and privacy protection would be truly the most convincing and efficient solution. Nevertheless, albeit debate is pointing at such a harmonizing process, existing regulation initiatives still rest on a less ambitious stage. Certain legal instruments, adopted by diverse formulating agen- cies (Organization for Economic and Cooperation Development (OECD); Council of Europe, United Nations General Assembly), have achieved to enshrine widely accepted common core principles on protecting privacy that might facilitate the progressive rapprochement on international-basis. Despite the reasonability of such common approach and its suitability for tackling problems arising from and in the new space, uniformity aim on privacy issues on Internet has been intensely marked by a strong confrontation between protecting legal models, in particular the one staged by the laxer model championed by United States and the stricter one advo- cated by Europe. In the interests of achieving better understanding between the two legal systems and ensuring competitiveness in the digital economy, a bilateral agreement, the Safe Harbor Agreement, was at last reached in a middle ground form. The safe harbor, approved by the EU Commission Decision4 in 2000, is an important way for US companies to avoid experiencing interruptions in their business dealings with the EU or facing prosecution by European authorities under European privacy laws. Certifying to the safe harbor will assure that EU organizations know that a company provides "adequate" privacy protection, as defined by the European Directives. With such a compromise as a backdrop, a mainstream understanding on worldwide privacy protection is conceivable whose keys are to be traced below. Regulation on privacy issues on Internet turn into a complex mechanism containing legal initiative of diverse geographical scope complemented by self-regulation and assisted by standards on privacy-enhancing technologies. Back to the analytical look at privacy and personal data regulation, three main components will be traced. First of all, scope of protection is to be delimited. In this regard, the meaning and extent of personal data notion hold a central position. Secondly, basic operating principles on privacy protection are to be related. Thirdly, diverse available techniques to enhance privacy and rule personal data issues in the course of business are identified. 3 Recommendation 3/97, “Anonymity on the Internet”, adopted by the Working Party on the Protection of Individuals with regard to the Processing of Personal Data on 3rd December 1997, XV D/5022/97 final. 4 EU Commission Decision 2000/520/EC, dated on 26th July 2000, published in O. J. L 215/7 of 25.8.2000. 14 Teresa Rodríguez de las Heras Ballell

2.3.1 Personal Data

A wide notion of personal data has been upheld, in particular by European lawmaker, in order to ensure a satisfactory scope of protection covering thereby all information which may be linked to an individual5. Legislative process distilled therefrom the current definition of personal data contained in Directive 95/46/EC with the following wording: “Personal data shall mean any information relating to an identified or identifiable natural person (...)”. Four main defining elements build the definition: “any information”, “relating to”, “an identified or identifiable”, and “natural person”. The term “any information” clearly signals the willingness of the legislator to call for a wide interpretation of personal data including information of any nature —descriptive or containing opinions or assessments—; of any sort —concerning private and family life or working relations, economic activities, and social behaviours— ; expressed in any format and stored in any medium. Information can be deemed to “relate” to an individual when it is about that individual. Establishing that link is crucial to define the scope of protection. Such a link may emerge in three possible situations: when the information is about an individual (content); when the data are used or likely to be used with the purpose to evaluate or treat in a certain way or influence the status or behaviour of an individual (purpose); when the data are likely to have an impact on a certain person’s rights and interests, taking into account all the circumstances surrounding the precise case. Information becomes sensitive for privacy purposes as far as it is related to a natural person that is identified or may be identifiable by means of certain identifiers taking into account all the factors at stake. While identification through the name is the most common occurrence in practice, other identifiers may be successfully used to single someone out. IP and e-mail addresses, or log-ins have been considered data relating to an identifiable person under certain circumstances, although such an assertion expounded in general and absolute terms is not undisputed [Guerrero Picó, 2006]. Thus, in case that identification of the data subject is not intended by the processor, appropriate state-of-the-art technical and organizational measures to prevent identification should be taken to protect the data against identification. Moreover, data must be related to a natural person. Such condition, apart from raising appealing questions regarding the treatment as personal data of information about legal persons, deceased persons or unborn children, evokes a challenging matter rooted in the idea of man-machine relationship. Certain sorts of data collected in the course of the service provision are truly related to the computer, terminal equipment or access system. In addition to pure personal data (such as contact data supplied by an application form), traffic data are a valuable source to monitor user’s actions, interests and on-line behaviour, but they are purportedly identifying the machine not the individual. Ascertaining that triggers an exciting reflection about man-machine interaction. Electronic devices are today becoming true prosthesis of human being; artificial parts that, on permanent and proximity basis amplify, complement or assist individual’s actions in the digital space. These kinds of data, un- less dissociated from a person’s identity, may conduct to devise a profile including users’ interests, preferences, behaviours and private environment. Traffic data and location data are valuable assets for commercial purposes and highly prized input to fuel public control processes.

2.3.2 Principles Relating to Data Quality

Under the general premise of data minimization, data collecting, processing6 and transfer are subject to the following principles:

Fair and lawful data collecting means. • 5 COM (90) 314 ﬁnal, dated on 13th September 1990, p. 19 (Commentary on Article 2). 6 Article 2 (b) of Personal Data Directive deﬁnes “processing of personal data” for the purposes of the Di- rective as “any operation or set of operations which is performed upon personal data, whether or not by automatic means, such as collection, recoding, organization, storage, adaptation or alteration, retrieval, con- sultation, use, disclosure by transmission, dissemination or otherwise making available, alignment or combination, blocking, erasure or destruction”. 2 Legal Framework for Personalization-Based Business Models 15

Specified, explicit and legitimate purposes, prohibiting any further data processing in a way incompatible • with those purposes or for different purposes. Adequacy, relevancy and minimization of data in relation to the purposes for which they are collected • and/or further processed. Accuracy and up-to-date state of data. • Complying with such principles implies two further conditions. On the one hand, an extensive duty to inform the subject concerned about all abovementioned particulars of the data manipulation; on the other hand, and presuming the latter condition of transparency is fulfilled, the knowledge or consent of the data subject to ensure that the processing carried out is lawful. The data subject’s consent shall mean “any freely given specific and informed indication of his wishes by which the data subject signifies his agreement to personal data relating to him processed” 7. Since the consent principle underpin the whole legal protecting system, invisible devices capturing data in Internet (spyware, adware, keystroke loggers, Javascripts, cookies, and so on) arouse serious privacy concerns. Technical functions for facilitating network operation and client-server transactions or legitimate purposes for providing value-added services within the framework of a commercial relationship initiated by the user can be proved. Accordingly, a lawful use of such devices must be based on a transparency policy intending to duly inform the user thereabout and give the opportunity to refuse their installation or deactivate their capturing functions.

2.3.3 Techniques: Licenses and Private-Enhancing Technologies

Uncertainty about applicable law in Internet has encouraged the emergence of two phenomena to alleviate distorting effect of delocalization: self-regulation and self-help. Self-regulation has shaped the form of codes of conducts and the form of license agreements. By means of those self-regulation instruments, companies adopt a privacy policy intended for ruling its activity in relation to users’ privacy and ensuring the full knowledge of the user about personal data processing applied by the company. As a powerful complement of regulation, companies and users may install self-help technical measures to support enforcement or privacy-enhancing technologies to improve protection level on personal data.

2.4 Personalization-Based Business Models

Digital economy business models can be typiﬁed in conformity with three basic strategies corresponding to three functional and activity layers. Such three-layer structure8 captures a complex conception of the digital space assuming a reciprocal complementing interaction among the three constituent parts: infrastructure, content and community. Taxonomy of models proposed to clear the tangled business scene is based on identifying on which layer functions performed and activities carried on by companies are located.

7 Article 2 (h), Directive 95/46/EC of the European Parliament and of the Council, dated on 24th October 1995, on the protection of individuals with regard to the processing of personal data and on the free movement of such data, published in O.J. L 281/31, dated on 23rd November 1995 (hereinafter, Personal Data Directive). 8 A structure composed of three layers is not unknown within scientific literature, the parts selected to make the complex mechanism up do nevertheless differs from those proposed therein. In this regard, Benkler [2000] defines the whole communication system in accordance to three variables or layers that are strongly interconnected but independent. Following the so-called “layer approach”, in every communication three layers entwine: a physical layer (technical infrastructure composed of access systems, networks and nodes), a logical layer (software operating network functioning) and content layer (data and information). Such approach has been rescued and redefined by Lessig [2002]. 16 Teresa Rodríguez de las Heras Ballell

2.4.1 A Three-Layer Structure to Understand Digital Business Models

The infrastructure layer embraces all the functions, products and services rendering network functioning possible from a technical perspective. Access providers, hosting services providers or caching services providers are the major figures playing in this instrumental stage. Insofar as their function is basically instrumental, their activity is fuelled by data not by contents, since technical services providers are devoid of ability to control and capacity to actually know the information that they transmit (mere conduit), host (hosting) or automatically and temporarily store (caching) at the request of a recipient of the service. The content layer accommodates the largest and best known Internet services covering the major part of content providers (websites, portal) and wide ranging sorts of online businesses. Content is the blood flowing through networks and nodes of the digital space’s infrastructure. Hence, content is intended to encompass any kind of information, product or services supplied on Internet or offered and provided by means thereof. Business creativity has been able to recreate on the digital stage every brick-and-mortar strategy and even open new spaces for business specifically devised for the digital world. Any attempt to describe and enumerate all on-line initiatives will suffer from an unmanageable casuism ruining any useful classification. Access to contents, supply of information, provision of services, sales and deliver of goods perfectly the fit business models competing at the content layer. Community is the most meaningful term to capture the essence of the third layer constituting the digital space. Virtual community creating models are the most sophisticated and promising business strategies. In fact, the ability to add value by exploiting community features on Internet opens challenging opportunities for social and commercial purposes. Creating communities is in itself a powerful personalizing project. A community is based on three pillars [Harvard Law Review Association, 1999]: unity of purpose emerging from a many-to-many interaction; sense of membership distilled in an enter/exit policy; and common rules governing relationships among members. Despite their origin as mechanism of social aggregation, virtual communities are skilful strategies for “networking and building business relationships” [Kannan et al., 2000]. The so-called “community factor” pervades a wide range of Internet-based relational models [Dai and Kauffman, 2002] from the most basic social networks shaped as chats, fora, bulletin boards or newsletter to the extremely sophisticated collaborative commerce projects distilled in e-marketplaces, virtual auctions, e-procurement platforms, virtual alliances or highly integral virtual business communities [Rodríguez de las Heras Ballell, 2006a]. The key factor of such networking models lies in their ability to engender, propagate and consolidate trust in market relationships. Assuming that credibility is a scarce resource in digital economy, trust-creating strategies are massively demanded by users as critical value-added services to guide their browsing and surfing.

2.4.2 Virtual Communities and Trusted Third Parties

The third category of digital business models just proposed, belonging to the layer named “community”, arouses certainly the most suggestive issues. Exploring community-based strategies leads us to the appealing concept of Trusted Third Party that distinctly vertebrates digital economy functioning and stems from the challenging versions of social-networking. Hence, theses community-creating business models are worth further analyzing to identify personalizing strategies. Three approaches are required to gain full insight of structure and functioning of community-based business models: technological, business and legal approach. From a technological perspective, a community-based business model must be necessarily built as a closed electronic environment. Unlike an open electronic environment, a closed one provides controlling capacities over membership, access, exit, members’ behaviour, content quality, security standard, data and information flow. Within the borderless space of Internet, worldwide open electronic environment par excellence, or beyond it, spaces can be closed either by physical means (virtual private networks) or by using passwords, user names or other sorts of identification mechanisms. Physical or logic closure provides control and control reduces uncertainty, insecurity and privacy risks. From a business perspective, virtual business communities would symbolise an efficient strategy of coopetition, a neologism stemming from a fruitful combination of cooperation and competition. It would be quite 2 Legal Framework for Personalization-Based Business Models 17 reasonable to define them as the digital version of traditional fairs and market places9. Coopetition strategies are pushed by important efficiency gains achieved by the application of new technologies (electronic communications effect, electronic brokerage effect and electronic integration effect [Malone et al., 1987]). Any form of multilateral electronic platform devised for trading purposes acts as an intermediary in economic sense repairing market failures by carrying on its inherent functions: aggregation, credibility, many-to- many interaction, centralized management, synergies and economies of scales. Most powerful strengths associated to community-based business models are then the gains obtained from multilateral interaction managed by a centralized supervisor who, by exerting its controlling functions over the market and its participants, generates credibility and trust. E-marketplace, trading platform or virtual business community manager performs roles of a Trusted Third Party. When it controls the access to the closed environment, firstly, by selecting suitable members among applicants, and secondly, implementing a reliable access control mechanism using passwords, digital signatures or any other identification procedures, the community manager grants a sort of trust mark in benefit of the admitted participants. Third persons may rely on such trust mark insofar as the manager credibility would be solidly underpinned by expertise, trustworthiness and reliability. Provided that offering, marketing and trading in the market space is ruled by the Rules Book drafted by the manager and expressly accepted by participants, the marketplace is an area of certainty regarding applicable regulation and common rules of conduct. Additionally, to the extent that the manager is performing its supervising and sanctioning functions, market participants rely on the compliance of Rules Book and the effective discouragement of deviating actions within the trading scenario. Exploiting community features entails the promotion of trust in transactions and allocates Trusted Third Party’s roles on the e-marketplace’s manager, the Internet forum’s moderator, the on-line auction’s auctioneer or the virtual community’s administrator. From a legal perspective, community-shaped spaces under a central manager’s control are self-regulated environment. Structure, organization and relationships within the market are contract-based. Such a condition insulates community against the disconcerting effects arising from delocalization. Applicable legislation and internal rules are agreed by parties, provided that they are not consumers, otherwise compulsory law may annuls certain provisions derogating from protecting principles in weakest parties’ interest. Apart from those consumer-biased exceptions, business-to-business communities are underpinned by two sorts of contract-based relationships. Within the so-called intersystem dimension, communities in operation compete in the market to attract new participants. Applications of those interested in joining the community are submitted to a scrutiny prior to decide their admission or refuse their incorporation. In case of favourable result, applicants enter into a membership contract regulating their access to the system, the use of common services and their behaviour therein. Once inside the market, each competing or cooperating relationships among members are based on a contract of the pertinent nature (sales contract, hiring agreement, software licence, technology transfer, and so on). Thus, the community assembles two kinds of relationships: vertical agreements between the admitted applicants and the manager and horizontal agreements among the market participants.

2.4.3 The Logic of Personalization-Based Business Models

All business models above described are apt to implement personalizing strategies. Nevertheless, it would be wrong to think that any data manipulation or each attempt to adapt the service or the product to the user entails the launch of personalization initiative. Upon an analytical view, personalization results from completing three successive stages: controlling, profiling and personalizing. Personalization assumes prior controlling functions over data and relevant information; otherwise, personalizing is not feasible. Merely controlling data does nevertheless not imply implementing a personalization- based strategy. Profiling abilities are decisive therefor. Comparing new collected data with existing profiles and crossing available information cobble the way to personalization. Should this three-stage process be placed on the business models’ taxonomy, meaningful results would be reached. Technical intermediaries, providing infrastructure and related services, collect and store in the course of their business personal data, traffic data and location data. Such information allows them to adapt their services 9 “Although it is Internet´s technology that makes business-to-business exchanges possible, the exchanges themselves are primarily business applications and not technological innovations” [Sculley and Woods, 2001] 18 Teresa Rodríguez de las Heras Ballell to context, technical characteristics of the accessing device, language used by the terminal equipment, installed operating system and even surrounding circumstances or user’s mood. In case that collected and processed data are related to an identified or identifiable person, provider will abide by personal data processing obligations. Their duties over collected data will be lighter if a dissociation process is applied to successfully break the connecting link between data and identifiable users. Technical services providers do then control data but do not carry on real personalization strategies. They merely provide alternative versions in conformity with certain predefined requirements. To the extent that technical intermediaries carried on controlling task over traffic and location data, certain obligations are imposed thereupon. For the purpose of criminal investigations or public defence policy, providers must retain location and traffic data during a predetermined period. Content providers of any kind are able to reach the second stage. Building profiles means a powerful strategy to sharpen the target. Personalizing access, tailoring contents, customizing online shop or providing selected value-added services are successful strategies to catch and lock-in users in the digital space rescuing them from the overwhelming sea of anonymity represented by the Web. Personalizing access, contents, products or value-added services requires, firstly, the capacity to control and process data and, secondly, the devising of profiles related to identified or identifiable persons. The whole personal data burdens are loaded on content providers. Accordingly, transparency duties, legitimate purpose principle and express consent rule must guide providers’ activity in their relations with users/subscribers/clients. If the three parts composing personal data regulation are assembled, it is quite easy to design an acceptable standard of conduct for content providers performing personalizing functions. Firstly, any invisible device intended for collecting data from users’ terminals must be avoided. Clear and full information regarding the use of those devices have to be supplied to users and an easy way to refuse their installation must be offered to users any time. Secondly, users have to be informed by the legitimate purposes of data collection (marketing, customization, security, value-added services provision, and so on). Thirdly, a reliable mechanism entitling the user to give a free consent must be enabled. The debate regarding opt-in/opt-out standard captures the concern of ensuring a real, well-informed and free consent giving. Whereas opt-in standard to obtain consent is stricter and shields users from unconscious supply of personal data, opt-out standard is endorsed by more flexibility and agility aims. The compliance vehicle of all expounded rules is the so-called “terms of use” included in the use licence agreement predisposed by provider. Further analysis thereof is traced in the next part of this chapter. Community-creating and -managing business models perform personalizing strategies in a very similar way to content providers. Nevertheless, unlike the latter, community-based business enjoy additional abilities to skilfully manage personalization. On the one hand, the extent and the depth of the knowledge about their members’ interests, preferences and behaviour is highly larger than the ones never reached by content providing structures. Creating a community is indeed fuelled by managing precise data about its members. On the other hand, due to the closed nature of community-based business models, they benefit from an express and conscious acceptance by clicking or logging of any member concerning privacy policy, terms of use and rules of conduct. A self-regulated and contract-based environment enables a full and rigorous compliance of personal data regulations. Moreover, social- or business-networking enables to implement recommender-based marketing strategies through so-called “opinion leaders” and “social/business interactions”. In order to safeguard their privacy, users are entitled to opt out of sharing their information. Social-networking do not obviously offer anonymity, personal data and information related to interests, preferences or hobbies are the sap of the vascular system of a community. By means of membership agreements and common policies, community equips its members with controlling devices over their data.

2.5 Legal Mechanisms to Implement Personalizing Strategies

The specific aim of this section of the chapter is to gain insight of how legal instruments help business to manage personalization. In a new space marked by the delocalization effect over traditional applicable law rules, agreements have reached a leading position to rule and manage private interests. Agreements’ power as 2 Legal Framework for Personalization-Based Business Models 19 regulation source is notably ample in business-to-business transactions where autonomy principle pervades. Hence, a B2B relationships’ environment will be assumed to test our reasoning. Prior to further enquire into legal issues arising from the use of license agreements to rule privacy matters, a suggestive consideration may be posed. The advent of digital world has entailed the passing from the age of property to the so-called “age of access” [Rifkin, 2000]. Gaining access is worthier than possessing. This change of paradigm means that products and services are defined indeed not as close items but as open “set of uses” (functionalities) in accordance to agreed license terms [Rodríguez de las Heras Ballell, 2006b]. The real value of the contract does not stem from the product or the service concerned in itself but from what one is entitled to do with or in respect of it. A music file is not a close and unique asset; on the contrary, it embodies a range of “products” depending on the available uses agreed by the parties (copy, reproduce, rent, transmit, publicly display, modify, distribute, and so on). It may be upheld that the contract is indeed the product. License agreements, site licenses or simply terms of use include provisions concerning collecting, processing and transferring data that fuel personalizing strategies. Privacy issues do nevertheless use to be separately dealt with under the so-called privacy policy. The acceptance of the privacy policy is anyway an express provision of the license agreement. License agreements play three functions for privacy purposes: identifying the nature of collected data, fully informing of the legitimate purposes and obtaining express consent from the data subject. Despite the overwhelming success of license agreements in the business praxis, certain legal concerns have been evoked. From both procedural and substantial perspective, the validity and the enforceability of browse-agreements has been riddled with doubts. License agreements embody an appealing inherent paradox as far as they symbolize the struggle for combining the standard form contract logic of massive trade with the most demanding requirements for personalizing. In the line of standard form contract logic, users face an unavoidable alternative (take-it or leave-it) that challenges the validity of consent as well as the fairness of contractual provisions [Rodríguez de las Heras Ballell, 2006c]. As far as the consent-giving process is concerned, site licenses have inherited from their predecessor —the so-called shrink-wrap licences [Einhorn, 1998, Lemley, 1995, Baker, 1997, Puhala, 1985, Kaufman, 1989, Mi- nassian, 1997]— the controversy [Lemley, 1995, Classen, 1996, Harrison, 1999] that permeated their validity and enforceability [Kaufman, 1989, Pitet, 1997, Das, 2002], and furthermore they have had to bear the concerns arising from web agreements. In the digital world, license agreements had adopted three main formats: click-agreements, logging- agreements and browse-wrap agreements. Whereas click-agreements duly fit the way of operating business in opened electronic environment, such as a wide range of transactions entered into with online shops; accepting by logging is associated to closed spaces as e-marketplaces, electronic platform and virtual business communities. In any case, consent-giving procedures are express, well-known (clicking, logging), and satisfactorily prevent users from unconscious actions. Browse-wrap agreements involve sensitive questions as far as the act of acceptance is concerned. Every website we visit has placed at the bottom of the page a link entitled “terms of use”, “legal notice” or similar expression. Under such a link, a license agreement, including the privacy policy, is hiding. All the precautions intended to ensure full knowledge, opportunity to review and free and informed consent are here undermined. The location of the link is not the most suitable, users are not expected to review the provisions and acceptance is not express but presumed from the mere use of the web. They are indeed named “click-free agreements”. Responding to an unfavorable treatment by case law10, all the suggestions are addressed to eliminate existing risks and adopt precautionary measures to ensure the obtaining of free and valid consent —better location of the link, due notice about the existence of the terms, real opportunity to review, warning about the assenting effect of mere use [Kunz et al., 2001]. As regards substantial perspective, a quick look at the license agreements drafted by the most famous online providers conveys quite illuminating ideas about how providers manage personalization and privacy. Reading,

10 A highly irregular case law proves the intense controversy embedded in the admissibility discourse of browse-wrap agreements: Register.com v. Verio Inc., 126 FSupp2d 238 (SDNY 2000); Ticketmaster v. Tick- ets.com, 2000 USDist LEXIS 4553 (CDCal March 27, 2000); Pollstar v. Gigmania Ltd., 170 FSupp2d 974 (EDCal October 17, 2000); Specht v. Netscape, 2002 USApp LEXIS 20714 (2d Cir. October 1, 2002). [Raysman and Brown, 2002, Kunkel, 2002, Grossman, 2001, Hayes, 2003] 20 Teresa Rodríguez de las Heras Ballell for instance, Google services privacy policy, at least four categories of provisions are remarkable for privacy purposes. Firstly, the kind of data collected and processed in the course of the service performance (IP, date and hour, cookie ID, URL, enquire, log) and the envisaged use thereof. Secondly, the purposes that are expected to satisfy by the data processing (improving services, offering value-added services, preventing from spam, storing backup, interacting users, marketing). Thirdly, the remainder that its data transfer policy is framed by applicable law and as far as advertisers are involved, data are facilitated in an aggregated manner. Finally, the provision concerning express consent-giving procedures —combining opt-in and opt-out standards— to cover other purposes differing from those established in the privacy policy.

2.6 Balancing Personalization Strategies and Privacy Concerns

Personalizing ability has become a powerful competitive advantage in the digital economy. Contrasting with the mainstream perception of a global, homogeneous and uniform Web, personalized spaces are emerging around virtual communities, social-networking, or simply customized relationships. Companies have become aware of the intense appeal of personalization-based strategies, recommender systems and context-aware services to make their offers different from competing products and services supplied by their rivals and provide users with appreciated value-added services suited for deploying a lock-in effect. Accordingly, many of them have embarked on ambitious projects to model access profiles, tailor information items, recommend goods and services within social-networking, sharpen ads’ target, or devise specialized business platforms (e-marketplaces). Effectiveness and competitiveness gains are expected to be high. Personalizing implies the full knowledge of users’ interests, preferences and wishes by collecting, processing and manipulating a wide range of data concerning personal details, behavior, location, previous actions, and social-/business-networking membership. Therefore, personalization-based strategies, as for fuelled by sensitive personal data, transactional and behavioral information, do inevitably arouse certain legal concerns. According to the hypothesis proposed in this chapter, a successful personalization-based strategy entails the fulfillment of three successive phases culminating in the personalizing aim: control, profile and personalize. Just mentioned phases do not always process personal data in the sense provided by personal data and privacy regulations and, besides, any personal, traffic and location data processing does not always mean the performance of personalizing strategies. A more careful analysis of business scene is then due. Firstly, basics of regulations on protecting personal data and privacy have been traced. Briefly summarizing, three main constituent parts underpin the legal building on privacy protection: a wide concept of personal data as related to an identified or identifiable natural person; a principle of data minimization in accordance to legitimate and specific purposes; and a predominant position of data subject’s consent. Assembling those three elements, an useful benchmark is built to test business policies concerning manipulation of data and devising of personalization-oriented strategies (opt-in/opt-out standards, use of cookies, consent-giving procedures, nature of data collected and processed in the course of trade, data retention period). Secondly, by exploring in depth business models competing in the digital space, taxonomy has been proposed in accordance to the layers composing the Web: physical layer, content layer and community layer. Such a three-layer vision allows identifying three categories of business models: technical intermediaries, content providers and community creators (Trusted Thirds Parties). All of them attempt to adapt their services to achieve an improvement of offer, fidelize constituencies and distinguish own services from competition. Nevertheless, adaptation does not mean personalization. Real personalizing strategies find in content providers and particularly in trading platforms, e-marketplaces, virtual communities and business-networking models the fittest spaces to develop. Thirdly, licence agreements have been identified as challenging contractual mechanisms to implement personalizing strategies. Under the format of click-agreements, log-in agreements or browse-wrap agreements, on-line businesses include provisions ruling personal data and privacy matters. Through the so-called terms of use and privacy policies, they expect to successfully manage privacy issues along the fine line separating lawful personalizing initiatives from undesirable invasions of users’ private sphere. They achieve such purpose when, complying with privacy and personal data protection standards, they apply dissociation process over data to untie them from personal identities, duly inform users about collecting and processing procedures, give them a 2 Legal Framework for Personalization-Based Business Models 21 real and effective opportunity to refuse the installation or the functioning of capturing devices and obtain free consent of users to devote collected information to fulfil legitimate purposes.

References

D. C. Baker. NOTE. ProCD v. Zeidenberg: Comercial reality, ﬂexibility in contract formation, and notions of manifested assent in the arena of shrinkwrap licences. Nortwestern University Law Review, 92:379–433, 1997. L. A. Ballesteros Moffa. La privacidad electrónica. Internet en el centro de protección. Tirant lo Blanch, 2005. Y. Benkler. From consumers to users: Shifting the deeper structures of regulation toward sustainable commons and user access. Federal Communication Law Journal, 52(3):561–579, 2000. F. Bergadamo, A. Montelero, G. Ruffo, and G. Sartor. Privacy digitale. Giuristi e informatici a confronto. G. Giappichelli Editore, 2005. H. W. Classen. Fundamentals of software licensing. IDEA, The Journal of Law and Technology, 37(1):1–85, 1996. Q. Dai and R. J. Kauffman. B2B e-commerce revisited: Leading perspectives on the key issues and research directions. Electronic Markets, 12(2):67–83, 2002. K. M. Das. Forum-selection clauses in consumer clickwrap and browsewrap agreements and the “reasonably communicated” test. Washington Law Review, 77:481–510, 2002. D. A. Einhorn. Shrink-wrap licenses: the debate continues. IDEA, The Journal of Law and Technology, 38(3): 383–401, 1998. M. Grossman. Forming contracts online. The Miami Herald, December 3rd, 2001. M. C. Guerrero Picó. El Impacto de Internet en el Derecho Fundamental a la Protección de Datos de Carácter Personal. Aranzadi, 2006. Z. M. Harrison. Just click here: Article 2B’s failure to guarantee adequate manifestation of assent in click-wrap contracts. Fordham Intellectual Property, Media & Entertainment Law Journal, 8:907–946, 1999. Harvard Law Review Association. Developments in the law: The law of cyberspace. Harvard Law Review, 112 (7):1574–1704, 1999. D. L. Hayes. The Enforceability of Shrinkwrap License Agreements On-Line and Off-Line. Fenwick & West, 2003. A. I. Herrán Ortiz. El derecho a la protección de datos personales en la sociedad de la información. Universidad de Deusto, 2003. P. Kannan, P. K. Kannan, A.-M. Chang, and A. B. Whiston. Handbook on Electronic Commerce, chapter The Internet Information Market: The Emerging Role of Intermediaries. Springer, 2000. P. M. Kaufman. The enforceability of state “shrink-wrap” license statutes in light of Vault Corp. v. Quaid Software, Ltd. Cornell Law Review, 74:222–244, 1989. C. Kuner. European Data Protection Law: Corporate compliance and regulation. Oxford University Press, 2007. R. G. Kunkel. Recent developments in shrinkwrap, clickwrap and browsewrap licenses in the United States. http://www.murdoch.edu.au/elaw/issues/v9n3/kunkel93_text.html, 2002. C. L. Kunz, M. F. del Duca, H. Thayer, and J. Debrow. Click-through agreements: Strategies for avoiding disputes on validity of assent. Business Lawyer, 57(1):401–420, 2001. M. A. Lemley. Intellectual property and shrink-wrap licenses. Southern California Law Review, 68:1239–1294, 1995. L. Lessig. The architecture of innovation. Duke Law Journal, 51(6):1781–1801, 2002. A. Lisi, editor. La privacy in Internet. Simone, 2003. T. W. Malone, J. Yates, and R. I. Benjamin. Electronic markets and electronic hierarchies. Communications of the ACM, 30(6):484–497, 1987. M. Masimini. Il diritto alla privacy. Motivi ed aspetti di una tutela problematica. Arcipelago, 2002. A. Minassian. The death of copyright: Enforceability of shrinkwrap licensing agreements. UCLA Law Review, 45:569–609, 1997. 22 Teresa Rodríguez de las Heras Ballell

C. L. Pitet. The problem with “money now, terms later”: ProCD, Inc. v. Zeidenberg and the enforceability of “shrink-wrap” software licenses. Loyola Los Angeles Law Review, 31:325–352, 1997. K. Puhala. The protection of computer software through shrink-wrap license agreements. Washington & Lee Law Review, 42:1347–1380, 1985. R. Raysman and P. Brown. Clarifying the rules for clickwrap and browsewrap agreements. New York Law Journal, 228(93), 2002. J. Rifkin. La era del acceso. La revolución de la nueva economía. Paidós (Estado y Sociedad), 2000. S. Rodotà. Tecnología y derechos fundamentales. Datospersonales.org, 8, 2004. T. Rodríguez de las Heras Ballell. El régimen jurídico de los Mercados Electrónicos Cerrados (e- marketplaces). Marcial Pons, 2006a. T. Rodríguez de las Heras Ballell. La responsabilidad por software defectuoso en la contratación mercantil. Revista Aranzadi de Derecho y Nuevas Tecnologías, 10:83–110, 2006b. T. Rodríguez de las Heras Ballell. Nueva Lex Mercatoria y contratos internacionales, chapter Las condiciones de uso de los sitios web y los browse-wrap agreements. Ediciones Jurídicas Gustavo Ibáñez, 2006c. A. B. Sculley and W. A. Woods. B2B exchanges. The Killer Application in the Business-to-Business Internet Revolution. Harper Business, 2001. 3 Technological Platforms, Convergence and Adaptation of Interactive Contents

Fabio Paternò

Istituto di Scienza e Tecnologie dell’Informazione “Alessandro Faedo” Area della Ricerca CNR di Pisa, Via G. Moruzzi 1, 56124 PISA, Italy [email protected]

Summary. Our lives are becoming a multi-device experience where people are surrounded by different types of devices (e.g. cell phones, PDAs, desktop computers, digital television sets, intelligent watches, and so on) through which they can connect to networks in different ways. Most of them are mobile personal devices carried by users moving freely about different environments populated by various other devices. Such environments raise many issues for designers and developers, such as the need to obtain interactive contents able to adapt to the interaction resources of the available devices. The objective of this book chapter is to allow readers to gain knowledge in methods and tools for the design of multi-device interactive services that can support designers and developers in addressing a number of the issues raised by multi-device environments.

3.1 Introduction

Multi-device interactive services are acquiring an increasing importance, however they raise a number of issues with regard to obtaining adaptation to the changing interaction resources. Indeed, the diversity in features of the potential devices, such as different screen size, interaction capabilities, processing and power supply, can make a user interface developed for a desktop unsuitable for a PDA and vice versa. For example, an interface layout designed for a desktop platform does not ﬁt in the smaller screen of a PDA, or a graphic interface running on a desktop system must be transformed to a voice interface when the application migrates to a car. Thus, a user interface needs to adapt to the different features of the target platform taking into account usability principles. This chapter ﬁrst provides an introduction of the main requirements that approaches to interactive content should satisfy. Next, there is a discussion of why and how model-based approaches can be useful for this purpose followed by a description of authoring environments that can be helpful for this purpose. Then, the possible solutions for adaptation at run-time are discussed with an indication of the more promising approaches. The last part is dedicated to migratory interactive services that are acquiring an increasing interest. Lastly, some conclusions are drawn along with indications of interesting possible further developments in the area.

3.2 Requirements in Device Adaptation

The device choice has an influence on the possible tasks to accomplish and how the structure of such tasks can vary in terms of possible secondary tasks, inter-task temporal relations, and content required depending on the device. One issue is that one-fits-all does not work. This means that it is not meaningful to try to support all tasks for all platforms. The big push for cross-platform design comes with the advent of mobile devices, which people can use when they are on the move. This means that there is a clear distinction regarding what it is meaningful to do with a desktop and with a mobile device. For example, people can use a desktop to compare airfares 24 Fabio Paternò and make reservations, while the mobile device can be used to check the real-time status of a particular flight. Likewise, the desktop is well-suited to reading a movie review and/or watching a trailer but not to purchasing a cinema ticket to avoid the line. There are many tasks that are not suitable at all for a mobile device. For example, currently in Europe with the advent of UMTS there is an interest in proposing football matches over phones. This seems senseless even with the last generation mobile phones, which have large displays and better connectivity. People like watching football matches but generally only when they are comfortable sitting and watching a large display in front of them. This allows them to appreciate how the footballers play, the details of the action, the tactics adopted and so on. Few people would ever do this through a mobile phone, even if it is technically possible, because of the small display and discomfort of using it on the move. Also considering that a football match lasts 90 minutes, this would be a terrible experience. What they could appreciate with a mobile device is a different task: receiving real-time updates regarding the match score. Another useful option is to have just the vocal description of the match that can be followed while driving the car or on the move. Thus, they could do this in parallel with other activities. In general, when a multi-platform application is considered, it is important to understand what type of tasks can actually be performed in each available platform. There are various possibilities: The same task can be performed on multiple platforms in the same manner (there may be only some • changes in attributes of the user interface objects from platform to platform). This is the case of tasks whose presentation remains mostly unchanged on different platforms: an example is when in a museum application textual links are provided to access general information about the museum (how to reach, timetable, etc). Same task on multiple platforms but with different user interface objects. An example of this case is high- • lighted in Fig. 3.1. In both systems users can select a geographical area (e.g. London/South East, De- von/Cornwall, etc). However, while in the desktop system a large, coloured interactive map of the museum is at the users’ disposal, in the phone, because of its limited capabilities, a text link is available for each area.

Fig. 3.1. Example of same task and different user interface objects.

Same task on multiple platforms but with different domain objects. This means that during the performance • of the same task different sets of domain objects are manipulated. An example of this is presentations of different information on a desktop system and a mobile phone for a museum application: while in the 3 Technological Platforms, Convergence and Adaptation of Interactive Contents 25

desktop system it is possible to access a wider set of domain elements (title, image type, description, author, material, and date of creation), the mobile interface supports access to only an image (which is a low resolution image just to give users a rough idea of what the work of art is), along with the indications of the title and associated museum section. Same task on multiple platforms but with different task decomposition. This means that the task is sub- • divided differently, with different sets of sub-tasks, depending on the platform. An example of this possibility is displayed in Fig. 3.2 that shows how differently the task access work of art is supported in a desktop and in a mobile device. In the desktop system, the users can accomplish additional sub-tasks, which are not supported in other systems. An example concerns the possibility of reading reviews of a particular work of art, which is a lengthy information-processing task that users can perform satisfactorily when sitting in front of a desktop computer, but which is not appropriate with handheld devices.

Fig. 3.2. Example of same main task and different task descompostition.

Same task on multiple platforms but with different temporal constraints. In this case the difference is in the • temporal relationships among the subtasks. With reference to the museum application, consider the case of users who wish to electronically reserve their tickets for a particular visit in order to avoid queues. In both systems they have to provide personal information. However, while in the desktop system they are free to choose the order to follow for ﬁlling in the various ﬁelds, within the phone application they are constrained by the mobile interface to follow a sequential order. 26 Fabio Paternò

Dependencies among tasks performed on different platforms. An example of this can be found when the • users have to reserve their flight tickets. Through the desktop system users can access, compare and contrast the different options about the best time for the flight ticket. Once they have selected their preferences and entered personal data, the system automatically enables their mobile phone to access real-time data regarding their flight, which ca be useful for example to know whether it is on time (see Fig. 3.3). Capturing this type of task relationships is particularly important when there is some task relevant to only a particular platform and that affects the performance of another task through a different platform. A typical situation occurs when users physically visit the museum and simultaneously annotate the most interesting works of art on the PDA. When they arrive home they would appreciate being able to receive information regarding such works first during their access to the museum web site through a desktop system.

Fig. 3.3. Example of dependencies among asks performed through different platforms.

3.3 Model-based Approaches

In this section we provide an overview of the results that can be obtained through model-based approaches when multi-device interfaces, even using different modalities, are considered, and will link up the discussion to projects currently underway. Indeed, as Myers et al. [2000] indicated while discussing the future of user interface tools, the wide platform variability encourages a return to the study of some techniques for device- independent user interface specification, ... Then, the system might choose appropriate interaction techniques taking all of these into account. The basic idea is that instead of having separate applications for each device 3 Technological Platforms, Convergence and Adaptation of Interactive Contents 27 that exchange only basic data, there is some abstract description and then an environment that is able to suggest a design for a specific device that adapts to its features and possible contexts of use. Thus, a key aspect is to be able to have different views on interactive systems, each view associated with a different abstraction level. With the support of tools, XML-based languages and transformations, it is possible to move from one level to another and tailor a description for one abstraction level to one more refined for the target interaction platform. The model-based community has long discussed such possible description levels (see for example [Szekely, 1996]). It is possible to have various viewpoints on an interactive system. Such viewpoints differ for the abstraction levels (to what extent the details are considered) and the focus (whether the task or the user interface is considered). Such abstraction levels are:

Task and object model. At this level, the logical activities that need to be performed in order to reach the • users’ goals are considered. Often they are represented hierarchically along with indications of the temporal relations among them and their associated attributes. The objects that have to be manipulated in order to perform tasks can be identified as well. Abstract user interface. In this case the focus shifts to the user interface supporting task performance. • Only the logical structure is considered, in a modality-independent manner, thereby avoiding low-level details. Interaction objects are described in terms of their semantics through interactors [Paternò and Leonardi, 1994]. Thus, it is possible to indicate, for example, that at a given point there is a need for a selection object without indicating whether the selection is performed graphically or vocally or through a gesture or some other modality. Concrete user interface. At this point, each abstract interactor is replaced with a concrete interaction • object that depends on the type of platform and media available and has a number of attributes that define more concretely how it should be perceived by the user. Final user interface. At this level the concrete interface is translated into an interface implemented by a • specific software environment (e.g. XHTML, Java, ...).

To better understand such abstraction levels we can consider an example of a task: making a hotel reservation. This task can be decomposed into selecting arrival and departure dates and other subtasks. At the abstract user interface level we need to identify the interaction objects needed to support such tasks. For example, for easily specifying arrival and departure days we need selection interaction objects. When we move on to the concrete user interface, we need to consider the specific interaction objects supported. So, in a desktop interface, selection can be supported by a graphical list object. This choice is more effective than others because the list supports a single selection from a potentially long list of elements. The final user interface is the result of these choices and others involving attributes such as the type and size of the font, the colours, and decoration images that, for example, can show the list in the form of a calendar. Many transformations are possible among these four levels for each interaction platform considered: from higher level descriptions to more concrete ones or vice versa or between the same level of abstraction but for different type of platforms or even any combination of them. Consequently, a wide variety of situations can be addressed. More generally, the possibility of linking aspects related to user interface elements to more semantic aspects opens up the possibility of intelligent tools that can help in the design, evaluation and run-time execution. The main issue underlying the last generation of model-based approaches is the design of multi-device interfaces. In current practise the design of multi-platform applications is often obtained through the development of several versions of the same applications, one for each platform considered. Then, such versions can at most exchange data. This solution with no tool support is rather limited, because it implies high implementation and maintenance costs. Thus, there is a need for authoring environments able to support the development of multi-device interfaces by providing design suggestions taking into account the specific features of the devices at hand. LiquidUI is an authoring environment whose main goal is to reduce the time to develop user interfaces for multiple devices. It is based on the User Interface Markup Language (UIML) [Abrams et al., 1999], a declarative language that then can be transformed in Java, HTML, and WML through specific rendering software. A UIML program, with its generic vocabulary, is specific to a family of devices (such as the desktop family, the PDA family, the WAP family). There is a transformation algorithm for each family of devices. For example, 28 Fabio Paternò using a generic vocabulary for desktop applications, the developer can write a program in UIML once and have it rendered for Java or HTML. Another approach is UsiXML [Limbourg and Vanderdonckt, 2004], the semantics of the UsiXML models are based on meta-models expressed in terms of UML class diagrams, from which the XML schema definition are derived. Right now, there is no automation between the initial definition of the semantics and their derivation into XML schemas. Only a systematic method is used for each new release. UsiXML aims to address the development of user interfaces for multiple contexts of use and has the advantage of providing a graphical syntax for a majority of constituent models. However, UsiXML renderers are still at the development stage. TERESA citepP24 is intended to provide a complete semi-automatic environment supporting a number of transformations useful for designers to build and analyse their design at different abstraction levels, including the task level, and consequently generate the concrete user interface for a specific type of platform. Currently, the tool supports generation of user interface implementations in XHTML, XHTML mobile device, VoiceXML, multimodal user interfaces in X+V and Java for the digital TV. The tool is able to support different level of automations ranging from completely automatic solutions to highly interactive solutions where designers can tailor or even radically change the solutions proposed by the tool. The last version of the tool supports different entry-points, so designers can start with a high-level task models but they can also start with the abstract user interface level in cases where only a part of the related design process needs to be supported. With the TERESA tool, at each abstraction level the designer is in the position of modifying the representations while the tool keeps maintaining forward and backward the relationships with the other levels thanks to a number of automatic features that have been implemented (e.g. the possibility of links between abstract interaction objects and the corresponding tasks in the task model so that designers can immediately identify their relations). This is useful for designers to maintain a unique overall picture of the system, with an increased consistence among the user interfaces generated for the different devices and consequent improved usability for end-users. Even recent W3C standards, such as XForms (XForms, 2004), have introduced the use of abstractions similar to those considered in the model-based community to address new heterogeneous environments.

3.4 Authoring Multi-device User Interfaces

The concepts discussed in the previous section can be incorporated in authoring environments for multi-device interfaces able to deal with a variety of platforms with different modalities (such as graphical and vocal interfaces, digital TV, tilt-based interaction, ...). Examples of such tools will be discussed (e .g. Multimodal TERESA). Once we have identified the tasks that are meaningful to support for each platform then we have to identify how they performance can vary according to the target platform. The idea is to first identify potential interactors in terms of their semantics (how they can change the state of the application) and then to generate corresponding interfaces depending on the platform features. All devices belonging to a given platform will receive an interface with consistent implementation. An abstract user interface is structured into presentations and connections indicating how it is possible to move from one presentation to another. Each presentation is structured into interactors and composition operators. We have defined a number of composition operators, which aim to capture communication effects that often designers want to achieve when they structure their user interfaces. The purpose of the composition operators is to indicate how to put together interactors. Each composition operator is associated with a communication goal. Depending on such goals, different implementation techniques will be used to support the composition operator. Figure 3.4 shows an example of a Web page taken from a frequently accessed Web site. We can note how the designer used various techniques to highlight groups of related interface elements. On the top there are elements that are ordered according to the potential user interest. Some elements are grouped using implementation techniques such as same background, same structure, bullets and so on. There are elements that are related to the rest of the Web site, such as the search element. Other elements are highlighted using large image and fonts because they are considered important. In general, the composition operators can involve several interactors or even compositions of interactors. In addition, their definition is modality-independent. They are: 3 Technological Platforms, Convergence and Adaptation of Interactive Contents 29

Grouping (G): indicates a set of interface elements logically connected to each other. • Relation (R): highlights a relation (usually one-to-many) among some elements; one element has some • effects on a set of elements. Ordering (O): some kind of ordering among a set of elements can be highlighted. • Hierarchy (H): different levels of importance can be deﬁned among a set of elements. • There are different types of interaction elements depending on the type of task supported. We have selection elements (to select between a set of elements), edit (to edit an object), control (to trigger an event within the user interface, which can be useful to activate either a functionality or the transition to a new presentation). There are different types of only_output elements (text, object, description, feedback) depending on the type of output the application provides to the user: a textual one, an object, a description, or a feedback about a particular state of the user interface.

Fig. 3.4. Web page with indication of some associated communication goals.

The Multimodal TERESA authoring environment allows designers and developers to start from two possible points: the task model description or the abstract interface description. In both cases they have to specify the target platform (in the current tool version either multimodal desktop or multimodal PDA). If they start with the task model then the tool automatically generates the corresponding abstract interface. As it can be seen in Fig. 3.5, the main area is mainly divided into four parts: the top-left dedicated to the list of presentations composing the user interface, the bottom-left indicating the connections defining how it is possible to move from one presentation to another, the top-right indicating the abstract description of the currently selected presentation and the bottom-right part displays the description of the possible concrete implementation of the currently selected element in the abstract part. The concrete part has three tabbed panes, one for the concrete graphical attributes, one for the concrete vocal attributes and one to specify how to compose the multimodal attributes. For example, if we consider the single selection interactor used to indicate the time of a cinema reservation, then the tool as a first suggestion for an implementation in a graphical+vocal desktop interface would propose that the input be equivalent (either graphical or vocal) and the prompt and feedback both redundant. Then, in the vocal section there would be an indication of the corresponding label, and the associated vocal message and feedback, in addition to the definition of the possible choice elements. The graphical part indicates the interaction technique for implementation (e.g. a radio-button), and the corresponding label and elements. The 30 Fabio Paternò tool keeps information in the graphical and vocal parts consistent. So, if the designer indicates five possible choice elements in the vocal part, then this is indicated when the graphical part is accessed as well. Likewise, in the case of a text output, if the corresponding multimodal property is complementarity, then different texts can be specified for vocal and graphical rendering, while if the multimodal attribute is redundant, then the text modified in either part will be updated for the other one as well.

Fig. 3.5. The MultiModal TERESA Environment.

3.5 Run-time adaptation to the device

This section is dedicated to run-time support for multi-device environments. Different types of solution and associated software architectures will be ﬁrst introduced. Issues and solutions for automatic transformation from desktop interfaces to different platforms (especially mobile ones) will be discussed, showing how presentation, navigation and content can be transformed and the usability issues to address in this process. I will show examples of results that can be obtained by tools provided by main software companies such as Google, Nokia, Microsoft, Opera, along with research results from various groups (including mine). The increasing availability of mobile devices has stimulated interest in tools for adapting the large amount of existing Web applications originally developed for desktop systems into versions that are accessible and usable for mobile devices. This adaptation process implies transforming various aspects:

the presentations, the perceivable aspects, including choice of media and interaction techniques, layout, • graphical attributes, etc; the dynamic behaviour, including navigation structure, dynamic activation and deactivation of interaction • techniques; the content, including text, labels, images. • In carrying out this adaptation it is important to consider the main characteristics of the target platform, in this section the mobile device will be mainly considered. By platform we mean a group of devices that share similar characteristics (such as the desktop, the mobile, the vocal, ...). For example, one aspect to consider is that often mobile devices have no pointing device, thus users have to navigate through 5-way keys, which 3 Technological Platforms, Convergence and Adaptation of Interactive Contents 31 allow them to go left, right, up, down and select the current element. There are also softkeys, which are used to activate commands, but their number and purpose vary depending on the device. In addition, text input is slow and users often have to pay to access the information, and thus prefer short sessions. Regarding the devices, the need for describing their features derives from the increasing availability of various types of interactive devices. Thus, in order to have applications able to adapt to their features there should be a way to represent them. This is particularly important in the area of mobile phones, in which the possible characteristics are the most variable. The generic Composite Capabilities/Preference Profiles (CC/PP) framework (www.w3.org/2001/di/) provides a mechanism through which a mobile user agent —a client, such as a browser, that performs rendering within a mobile device— can transmit information about the mobile device. It is based on RDF and aims to provide a firm foundation for UAProf. The user agent profile (UAProf; www.openmobilealliance.org/release_program/uap_v20.html) is an application of the CC/PP framework. It includes device hardware and software characteristics, information about the network the device is connected to, and other attributes. It is possible ot identify a device through the header of HTTP requests. All the devices complying with UAProf have a CC/PP description of their characteristics in a repository server, which can be queried for knowing them. The description of the devices is in the Resource Description Framework (RDF), language XML-based. When a mobile device sends a request, ity also informs about the URL where its profile is through a specific field in the request called X-Wap-Profile. For example, the x_wap_profile for a Nokia N9500 is http://nds1.nds.nokia.com/uaprof/N9500r100.xml. In this area another proposal is WURFL (Wireless Universal Resource File) [Passani, 2006], which is an XML configuration file which contains information about capabilities and features of several wireless devices. The main scope of the file is to collect as much information as we can about all the existing wireless devices that access WAP pages so that developers will be able to build better applications and better services for the users. This proposal aims to support Web applications for mobile devices. The goal is programmatically abstract away devices differences, avoid that we need to modify applications whenever a new device ships, avoid that we need to track new devices that ship (particularly those in uninteresting markets). The basic idea is a global database of all devices and their capabilities. Starting from the assumption that browsers are different, but they also have many features in common with one another; browsers/devices coming from the same manufacturer are most often an evolution of the same hardware/software. In other words, differences between, for example, a Nokia 7110 and a Nokia 6210 are minimal; devices from different manufacturers may run the same software. WURFL has created a compact, small, and easy to update matrix. The WURFL is based on the concept of family of devices. All devices are descendent of a generic device, but they may also descend from more specialized families (such as those who use the same browser). Its goal is to overcome some limitations of the UAProf standard developed in the OMA. Uaprof seems to rely too much on someone else’s setting up the infrastructure to request profiles. There are cases of manufacturers just associating the profile of a different phones into a new one. The WURFL can be installed in any site and does not need to access device profiles from a repository on the net. In general there are various approaches to authoring multi-device interfaces: Device-specific authoring which means that a specific version for each target platform is developed sepa- • rtely. One example is the Amazon Web site, which has a separate version for mobile devices. This approach is clearly expensive in terms of time and effort. Multiple-device authoring is similar to the previous one but there is a single application that has separate • parts for different platforms. An example is the use of CSS depending on the media. Single authoring. In this case, only one version of the application is built, which is then adapted to various • target platforms. There are two main possibilities for this purpose: either to include the authors’ hints or to specify an abstract description, which is then refined according to the target platform. Interesting contributions in this area are PIMA [Banavar et al., 2004] and SUPPLE [Gajos et al., 2005]. Automatic re-authoring. Here, there is an automatic transformation of a version for a given platform, • usually the desktop, into a version for the target platform. Recently, particular attention has been paid to the latter category, with the aim of obtaining a solution that does not require particular effort in terms of time but is still able to produce meaningful results. Various solutions have been proposed in this category, a first distinction can be made depending on where the re- authoring process occurs: the client device, the application server or an intermediate proxy server. The last 32 Fabio Paternò solution seems particularly interesting because performing the transformation on the client device can raise performance issues with limited capabilities devices, while a solution on the application server would require duplication of installations in all the applications of interest. The feasibility of proxy-based solutions is also shown by its widespread use in tools, such as Google for mobile devices (www.google.com/xhtml) [Kamvar and Baluja, 2006], which converts the Web pages identified by the search engine into versions adapted for mobile devices. Indeed, depending on user agent in http request Google search redirects to www.google.com/xhtml in the case of mobile devices. This version (see Fig. 3.6) has radio buttons instead of tabs to access the various sections. There is no advertisement in the XHTML version, the pieces of text for each link are smallers in the XHTML version, the results for the XHTML version do not contain link cached or similar pages and don’t indicate the size of the page, and the user can access only the previous and next result page.

Fig. 3.6. The two user interfaces for the Google search engine.

In addition, we think that effective solutions for transforming desktop Web sites for mobile access should be based on semantic aspects. Unfortunately, the semantic Web so far has mainly focused on the data semantics through the use of ontologies and languages that allow for more intelligent processing. It would also be important to consider the semantics of interaction, which is related to the tasks to support in order to reach the users’ goals. Several solutions for automatic re-authoring from desktop-to-mobile have been proposed in recent years. The simplest one just proposes resizing the elements according to the size of the target screen. However, they often generate unusable results with unreadable elements and presentation structures unsuitable for the mobile device. Some solutions such as AvantGo (www.avantgo.com) translate elements and images into other formats, and compress and convert images to match the device characteristics but suffer from similar limitations as the former approach. Thus, research work has focused on transformations able to go further, to modify both the content and structure originally designed for desktop systems to make them suitable for display on small screens. Even in this case various possibilities have been explored. The most common transformation supported by current mobile devices is into a single column (the narrow solution): the order of the content follows that of the mark-up ﬁle starting from the top, the images are scaled to the size of the screen, and the text is always visible and the content compacted without blank spaces. It eliminates scrolling in one dimension, though it greatly increases the amount of scrolling in the other dimension. For example Opera SSR (Small Screen Ren- dering, www.opera.com/products/smartphone/ smallscreen/) uses a remote server to pre-process Web pages before sending them to a mobile device, Web content is compressed to reduce the size of data transfers (see Fig. 3.7). In general, in this solution content requiring a good deal of space such as maps and tables can become 3 Technological Platforms, Convergence and Adaptation of Interactive Contents 33 unreadable; and often it is difﬁcult to understand that the corresponding desktop page has changed because the initial part of several desktop pages is indistinguishable.

Fig. 3.7. The example of transformation with Opera SSR.

Various approaches have considered the application of information visualization [Spence, 2001] techniques to address these issues. Fish-eye representations have been considered, for example Fishnet [Baudisch et al., 2004], which is a fisheye Web browser that shows a focus region at a readable scale, while spatially com- pressing page content outside the focus region. However, generating fish-eye representations in mobile devices can require excessive processing. Overview + detail splits a Web page into multiple sections and provides an overview page with links to these sections. The overview page can be either a thumbnail image, or a text summary of the Web page. Within this approach various solutions have been proposed. Smartview [Milic-Frayling and Sommerer, 2002] is a thumbnail view of the original Web page in zoom-out, fitting the screen horizontally. The approach partitions the page in logical regions; when one is selected, content is presented inside the screen space in a detailed view. In Gateway [MacKay et al., 2004] the detailed view uses a focus-plus-context technique, enlarging the selected region above the detailed view. Summary Thumbnails [Lam and Baudisch, 2005] uses the same thumbnail approach but the texts are summarized enabling good legibility (fonts are enlarged to a legible size and characters are cropped from right to left until the sentence fits in the available area). The main issue with this type of approach is that it works well in some cases, less in others because they mainly focus on the transformation of specific elements (for example Summary Thumbnail mainly works on text snip- pets). Another contribution in this area is MiniMap [Roto et al., 2006] a browser for Nokia 6600 mobile phones 34 Fabio Paternò developed at Nokia Research. It removes the need for horizontal scrolling to read text and provides enough contextual information to give an idea of the page structure and the current location without destroying the original page layout. The user interface is organised in such a way that text size should not exceed the screen space and provides an overview+detail representation. The overview is given by an area dedicated to showing where the current mobile page is located in the original desktop page. However, this solution is effective only with mobile devices with relatively large screens. We believe that model-based approaches [Paternò, 1999, Szekely, 1996] can provide a more general solution to adaptation to the interaction device. They are based on the use of logical descriptions that capture the main semantic aspects of the user interface and hide low-level details. Some first studies of how to apply them in this context have already been proposed [Bandelloni et al., 2004, Eisenstein et al., 2001, Florins and Vander- donckt, 2004]. These interesting works were useful to provide solutions to specific issues raised by supporting interactions of mobile users, but they did not address the issue of providing a general solution for taking Web sites originally developed for desktop systems and dynamically transforming them into accessible and usable versions for mobile devices while users are accessing them. Another interesting application of model-based approaches in this area is PUC [Nichols et al., 2002] which dynamically generates user interfaces for mobile devices able to control a domestic appliance starting with its logical description. A novel solution [Bandelloni et al., 2004] for a different domain (Web applications), which is characterised by wider variability in terms of content and tasks to support, is based on the use of a migration/proxy server able to support both adaptation and state preservation across multiple devices.

3.6 Migratory User Interfaces

This section discusses how mobile users can be supported in multi-device environments. To this end, distributed and migratory interfaces are introduced. One important aspect of ubiquitous environments is to provide users with the possibility to freely move about and naturally continue the interaction with the available applications through a variety of interactive devices (i.e. cell phones, PDAs, desktop computers, digital television sets, intelligent watches, and so on). Indeed, in such environments one big potential source of frustration is that people have to start their session over again from the beginning at each interaction device change. Migratory interactive services can overcome this limitation and support continuous task performance. This implies that interactive applications be able to follow users and adapt to the changing context of use while preserving their state. Migratory interfaces are interfaces that can transfer among different devices, and thus allow the users to continue their tasks. This definition highlights important concepts: task performance continuity, device adaptation and interface usability. Task performance continuity means that when migration occurs users do not have to restart the application on the new device, but they can continue their task from the same point where they left off, without having to re-enter the same data and go through the same long series of interactions to get to the presentation they were accessing on the previous device. General solutions for migratory interactive services can be obtained by means of addressing three aspects: adapt and preserve the state of the software application parts dedicated to interacting with end users; support mechanisms for application logic reconfiguration; and define suitably flexible mechanisms from the underlying network layers. There are many applications that can benefit from migratory interfaces. In general, applications that require time to be completed (such as games, business applications) or applications that have some rigid deadline and thus need to be completed wherever the user is (e.g. online auctions). Other applications that can benefit from this flexible reconfiguration support are those that have to provide users with continuous support during the whole day through different devices (for example, in the assisted living domain). ICrafter [Ponnekanti et al., 2001] is a solution to generate adaptive interfaces for accessing services in interactive spaces. It generates interfaces that adapt to different devices starting with XML-based descriptions of the service that must be supported. However, ICrafter is limited to creating support for controlling interactive workspaces by generating UI for services obtained by dynamic composition of elementary ones and does not provide support for continuity of task performance across different devices. Aura [Garlan et al., 2002] provides support for migration but it is obtained by changing the application depending on the resources available in 3 Technological Platforms, Convergence and Adaptation of Interactive Contents 35

Fig. 3.8. A migration example scenario. the device in question, while there is a need for solutions able to generate interfaces of the same application that adapt to the interaction resources available. Bharat and Cardelli [1995] addressed the migration of entire applications (which is problematic with limited-resource devices and different CPU architectures or operating systems), while we focus on the migration of the UI part of a software application. Kozuch and Satyanarayanan [2002] identified a solution for migration based on the encapsulation of all volatile execution state of a virtual machine. However, their solution mainly supports migration of applications among desktop or laptop systems by copying the application with the current state in a virtual machine and then copy the virtual machine in the target device. This solution does not address the support of different interaction platforms supporting different interaction resources and modalities, with the consequent ability to adapt to them. Chung and Dewan [1996] proposed a specific solution for migration of applications shared among several users. When migration is triggered the environment starts a new copy of the application process in the target system and replay the saved sequence of input events to the copy to ensure that the process will get the state where it left off. This solution does not consider migration across platforms and consequently does not support run-time generation of a new version of the UI for a different platform. A discussion of some high-level requirements for software architectures in multi-device environments is proposed in [Balme et al., 2004] without presenting a detailed software architecture and implementation indicating a concrete solutions at these issues. In this section, we also introduce a specific architectural solution, based on a migration/proxy server, able to support migration of user interfaces associated with applications hosted by different content servers. At the HHIS Laboratory of ISTI-CNR, Ban- delloni et al. [2004] found a solution based on the use of pre-computed interfaces for different platforms that are dynamically activated, which then evolved in a new solution [R.Bandelloni et al., 2007], implemented in an engineered prototype, supporting also migration through different modalities (such as voice). Especially in heterogeneous environments (namely, environments in which different types of devices exist), the concept of migratory user interfaces raises a number of design issues that should be appropriately analysed and addressed in the attempt to identifying an effective migration architecture/process. A suitable framework for migration should consider at least the dimensions described hereafter. Such dimensions are:

Activation type: how the migration is triggered. The simplest case is on demand, in which the user actively • selects when and how to migrate. Otherwise, in automatic migration, it is the system that activates the device change (depending on e.g. mobile device battery consumption level, device proximity, etc). Type of migration: This dimension analyses the extent of migration, as there are cases in which only a • portion of the interactive application should be migrated. A number of migration types can be identiﬁed: Total migration allows basically the user to change the device used to interact with the application. 36 Fabio Paternò

– Partial migration is the ability to migrate a portion of the UI (the remaining portion remains in the source device). – In the distributing migration the user interface is totally distributed over two or more devices after migration. – The aggregating migration performs the inverse process: the interface of multiple source devices are grouped in the user interface of a single target device. – The multiple migration occurs when both the source and the target of the migration process are multiple devices. Number/combinations of migration modalities. This dimension analyses the modalities involved in the • migration process. Mono-modality means that the devices involved in the migration adopt the same modality interaction. Trans-modality means that the user can migrate changing the interface modality. An example of migration from graphical interface to vocal interface is the case of users navigating the Web through a PDA or Desktop PC and afterwards migrate the application to a mobile phone supporting only vocal interaction. Lastly, with multi-modality the migratory interface contemporaneously supports two or more interaction modalities at least in one device involved in the migration. Work in this area often has mainly focused on graphical interfaces, investigating how to change the graphical representations depending on the size of the screens available. Type of interface activated. This dimension specifies how the user interface is generated in order to be • rendered on the target device(s). With precomputed user interfaces the UI has been produced in advance for each type of device. Thus, at runtime there is no need for further adaptation to the device but only for adaptation of the state of the UI. On the contrary, if a runtime generation of user interfaces is considered, the migration engine generates the UI according to the features of the target device when migration occurs. In an intermediate approach the migration engine adapts dynamically to the different devices using some templates previously created. Granularity of adaptation. The adaptation process can be affected at various levels: the entire application • can be changed depending on the new context or the UI components (presentation, navigation, content). Migration time. The device change that characterise migration can occur in different temporal evolutions: • – Continuous: The user interface migration occurs immediately after its triggering. – Postponed: The request of migration can be triggered at any time but it actually occurs after some time. This can happen when the target device is not immediately available (for example when migrating from the desktop in the office to the vocal device in the car, which has to be first turned on by the user when he enters the car). How the UI is Adapted. Several strategies can be identified regarding how to adapt user interfaces after a • migration process occurs: – Conservation. This strategy maintains the arrangement and the presentation of each object of the user interface: one possible example is the simple scaling of the user interface to different screen sizes. – Rearrangement. In this case, all the UI objects are kept during the migration but they are rearranged according to some techniques (e.g. using different layout strategies). – Increase. When the UI migrates from one device with limited resources to one offering more capabilities, the UI might be improved accordingly, by providing users with more features. – Reduction. This technique is the opposite of increase and it can be applied when the UI migrates from desktop to mobile device because some activities that can be performed on the desktop might result unsuitable on a mobile device. – Simplification. In this case, all the user interface objects are kept during the migration but their representation is simplified, for example, different resolutions are used for figures or figures are substituted with textual descriptions. – Enhancement. This technique represents the opposite of simplification (e.g. a textual description might be substituted with multimedia information). The impact of migration on tasks depends on how the user interface is adapted because reduction and • increase can produce some change on the range of tasks supported by each device. Differently, conservation and rearrangement do not produce any effect on the set of tasks supported. Then some possible cases are: (i) after a partial or distributing migration some tasks can be performed on two or more devices in the 3 Technological Platforms, Convergence and Adaptation of Interactive Contents 37

same manner (task redundancy), which means, for instance, that the decomposition of the different tasks into subtasks is unchanged, as well as the temporal relationships (sequencing, concurrency, etc) occurring among them; (ii) after a partial or distributing migration a part of a task can be supported on one device and the other part/s is/are available on different devices (task complementarity). Additional cases are when the number of task supported (iii) increases (task increase) or (iv) decreases (task decrease) after migration. Obviously, a final case might be identified when the migration has no impact on tasks, as they remain substantially unchanged. Context model. During adaptation of the UI the migration process can consider the context in terms of • description of device, user and environment. Generally, the context dimension that is most taken into account with the aim of producing usable UI is the device and its properties, together with the surrounding environment. Context reasoning. A context modelling process begins by identifying the context of interest. This depends • on the specific task which should be associated with the context of interest. The context of interest can be a primitive context, which can directly be captured by employing a sensor; or it can be a higher-level context, which is a result of manipulation of several primitive contexts. If a context of interest is a higher-level context, a reasoning scheme is inadvertently required, which can be either a logic-based reasoning scheme or a probabilistic reasoning scheme. A logic-based reasoning scheme considers a primitive context as a factual data while a probabilistic reasoning scheme does not. Depending on the nature of the sensed data available and the way the data are manipulated, ignorance can be classified as follows: – Incompleteness refers to the fact that some vital data about the real-world situation to be reasoned about is missing; the available data, however, are considered to be accurate. – Imprecision refers to inexact data from sensors. Inexact data arises due, partly, to the physical limitations of the sensing elements employed. Different sensors have different resolution, accuracy, and sensing range. Besides, the performance of physical sensors can be influenced by external factors such as surrounding noise or temperature. – Uncertainty refers to the absence of knowledge about the reliability of the data sources —this knowledge might be information about the parameters listed above to determine (characterise) the degree of imprecision incorporated in sensory data. Implementation environment. The migration process can involve different types of applications. Probably • due to their diffusion, the most recurrently considered applications are web-based systems (static/dynamic pages), but also other applications (Java, Microsoft .NET, etc) can be considered. Architecture. With regard to the architecture of the migration support environment there are different • strategies, for example: proxy-based, in which there is an intelligent unit managing all migration requests and sending all data to target devices; and peer to peer, where the devices directly communicate and nego- tiate the migration parameters.

3.7 Conclusions

This chapter has discussed a number of issues related to technological platforms, convergence and adaptation of interactive contents and it has described the state of art in this area. Particular attention has been dedicated to solutions for adaptation to the device. Migratory interactive services has been described along with an indication of the relevant design dimensions. This is an area that is acquiring an increasing importance, with a strong technological push. One important research area for the next years is dedicated to making the transformation rules driving adaptation accessible and modiﬁable by end users in order to allow their customization for speciﬁc preferences.

References

Abrams, M., Phanouriou, C., Batongbacal, A., Williams, S., and Shuster, J. (1999). UIML: An appliance- independent XML user interface language. In 8th International World Wide Web Conference, Toronto, Canada. 38 Fabio Paternò

Balme, L., Demeure, A., Barralon, N., Coutaz, J., and Calvary, G. (2004). CAMELEON-RT: a software architecture reference model for distributed, migratable, and plastic user interfaces. Lecture Notes in Computer Science, 3295:291–302. Banavar, G., Bergman, L. D., Gaeremynck, Y., Soroker, D., and Sussman, J. B. (2004). Tooling and system support for authoring multi-device applications. Journal of Systems and Software, 69(3):227–242. Bandelloni, R., S.Berti, and Paternò, F. (2004). Mixed-initiative, trans-modal interface migration. Lecture Notes in Computer Science, 3160:216–227. Baudisch, P., Lee, B., and Hanna, L. (2004). Fishnet, a ﬁsheye web browser with search term popouts: A comparative evaluation with overview and linear view. In International Working Conference on Advanced Visual Interfaces, Gallipoli, Italy. Bharat, K. A. and Cardelli, L. (1995). Migratory applications. In 8th ACM Symposium on User Interface Software and Technology, Pittsburgh, Pennsylvania (USA). Chung, G. and Dewan, P. (1996). A mechanism for supporting client migration in a shared window system. In 9th ACM Symposium on User Interface Software and Technology, Seattle, Washington (USA). Eisenstein, J., Vanderdonckt, J., and Puerta, A. R. (2001). Applying model-based techniques to the development of UIs for mobile computers. In International Conference on Intelligent User Interfaces, Santa Fe, New Mexico (USA). Florins, M. and Vanderdonckt, J. (2004). Graceful degradation of user interfaces as a design method for multi- platform systems. In International Conference on Intelligent User Interfaces, Funchal, Madeira (Portugal). Gajos, K., Christianson, D., Hoffmann, R., Shaked, T., Henning, K., Long, J. J., and Weld, D. S. (2005). Fast and robust interface generation for ubiquitous applications. Lecture Notes in Computer Science, 3660:37–55. Garlan, D., Siewiorek, D., Smailagic, A., and Steenkiste, P. (2002). Project Aura: Toward distraction-free pervasive computing. IEEE Pervasive Computing, 21(2):22–31. Kamvar, M. and Baluja, S. (2006). A large scale study of wireless search behavior: Google Mobile search. In SIGCHI Conference on Human Factors in Computing Systems, Montréal, Canada. Kozuch, M. and Satyanarayanan, M. (2002). Internet Suspend/Resume. In 4th IEEE Workshop on Mobile Computing Systems and Applications, New York (USA). Lam, H. and Baudisch, P. (2005). Summary thumbnails: Readable overviews for small screen web browsers. In SIGCHI Conference on Human Factors in Computing Systems, Portland, Oregon (USA). Limbourg, Q. and Vanderdonckt, J. (2004). UsiXML: A user interface description language supporting multiple levels of independence. In Workshop on Engineering Advanced Web Applications, in conjunction with the 4th International Conference on Web Engineering, Munich, Germany. MacKay, B., Watters, C., and Duffy, J. (2004). Web page transformation when switching devices. Lecture Notes in Computer Science, 3160:228–239. Milic-Frayling, N. and Sommerer, R. (2002). Smartview: Enhanced document viewer for mobile devices. Technical report, Microsoft Research, Cambridge (UK). Myers, B., Hudson, S., and Pausch, R. (2000). Past, present, future of user interface tools. ACM Transactions on Computer-Human Interaction, 7(1):3–38. Nichols, J., Myers, B. A., Higgins, M., Hughes, J., Harris, T. K., Rosenfeld, R., and Pignol, M. (2002). Gener- ating remote control interfaces for complex appliances. In 15th ACM Symposium on User Interface Software and Technology, Paris, France. Passani, L. (2006). The Wireless Universal Resource File. http://wurﬂ.sourceforge.net/. Paternò, F. (1999). Model-based Design and Evaluation of Interactive Applications. Spriger-Verlag. Paternò, F. and Leonardi, A. (1994). A semantics-based approach to the design and implementation of interaction objects. Computer Graphics Forum, 13(3):195–204. Ponnekanti, S. R., Lee, B., Fox, A., Hanrahan, P., and Winograd, T. (2001). ICrafter: A service framework for ubiquitous computing environments. Lecture Notes in Computer Science, 2201:56–75. R.Bandelloni, Paternò, F., Santoro, C., and Scorcia, A. (2007). Web user interface migration through different modalities with dynamic device discovery. In 2nd International Workshop on Adaptation and Evolution in Web Systems Engineering, Como, Italy. Roto, V., Popescu, A., Koivisto, A., and Vartiainen, E. (2006). Minimap: a web page visualization method for mobile phones. In SIGCHI Conference on Human Factors in Computing Systems, Montréal, Canada. 3 Technological Platforms, Convergence and Adaptation of Interactive Contents 39

Spence, R. (2001). Information Visualization. ACM Press. Szekely, P. (1996). Retrospective and challenges for model-based interface development. In 2nd International Workshop on Computer-Aided Design of User Interfaces, Namur, Belgium.

4 Filtering Techniques for Selection of Contents and Products

Qing Li1, Yuanzhu Peter Chen2, and Zhangxi Lin3

1 School of Economic Information Engineering, Southwestern University of Finance and Economics, China. [email protected] 2 Department of Computer Science, Memorial University of Newfoundland, Canada. [email protected] 3 The Rawls College of Business Administration, Texas Tech University, USA [email protected]

Summary. With the development of e-commerce and the proliferation of easily accessible information, filtering systems (also known as recommender systems) have become a popular technique to prune large information spaces so that users are directed to products that best meet their needs and preferences. In this chapter, we present an overview of the field of filtering techniques and describe the current filtering methods that are usually classified as content-based, collaborative, and hybrid filtering approaches. We also analyze various limitations of current filtering techniques and discuss the challenges for building a large-scale and practical filtering system. In particular, content-based filtering methods, which select the appropriate information for a user by matching user preferences, are first introduced. Second, we look into the memory-based and model-based collaborative filtering approaches to elaborate on the collaborative mechanism, which utilizes the opinions of other users to make recommendations for personalized information filtering. Third, hybrid filtering techniques are introduced, which combines the above filtering methods. At last, we address the challenges for building a large-scale and commercial recommender system. These challenges include scalability of filtering algorithms, data sparsity, effectiveness of recommender systems and so forth.

4.1 Introduction

Information filtering, also known as information recommendation, has an obvious appeal in an environment where the amount of on-line information vastly outstrips any individual’s capability to survey. In our daily life, critique experts perform information filtering on our behalves. For instance, we often check ranking lists for bestselling movies and listen to movie critics. However, such manually abstracted information is not responsive and comprehensive enough compared to the volumous raw information generated everyday. Thus, information filtering technologies offer a powerful solution to this problem. Information filtering has found its way into the activities in specific areas of current social life. For example, filtering systems can continuously provide financial investors with relevant information, such as news, financial reports, and so forth, for better analysis and decision making. With the help of a filtering system, a novice can obtain suggestions from a professional in a community network. In reality, some of the on-line vendors have already incorporated more or less filtering capabilities into their commercial promotion such as Amazon, Yahoo! and CDNow. Information recommendation focuses on providing an individual with tailored information. Different from search engines which target at immediate information acquisition, information recommendation is more of a long-term information delivery service. Products, including books, movies, music, and so forth, are often selected and recommended to users automatically according to their preferences by such systems. Techniques applied in information filtering are generally categorized as content-based filtering (CBF), collaborative filtering (CF), and hybrid filtering (HF) according to their operation mechanism [Li and Kim, 2003]. 42 Qing Li, Yuanzhu Peter Chen, and Zhangxi Lin

In particular, content-based filtering utilizes the summarized content information of products for individual users. In contrast, collaborative filtering applies preference induction based on the similarities among multiple users. Hybrid filtering utilizes content-based and collaborative filtering for more flexible and accurate recommendation. In this chapter, we present a categorized overview of the field of information filtering and analyze various limitations of these filtering techniques. First, content-based filtering methods, which select the appropriate information for a user by matching the user preferences, are introduced in Section 4.2. Second, we look into the memory-based and model-based collaborative filtering approaches to elaborate on the collaborative mechanism in Section 4.3. In Section 4.4, hybrid filtering techniques are introduced as extension to the above approaches. To conclude, we address the challenges for building a large-scale and commercial recommender system.

4.2 Content-Based Filtering

Content-based filtering selects the appropriate information for a user by matching the user preference against an information repository (Figure 4.1). Typically, in a content-based filtering system, we construct and maintain a profile for each user to model the user’s preferences. It can be generated explicitly or implicitly. For instance, the system may have a user explicitly describe his/her preferences, or it builds a user profiles based on the user’s browsing behaviors (e.g. clicks and reading time) [Wittenburg et al., 1995]. On the other hand, the system searches the information repository to recommend products by matching the user profiles against products in the repository. Although details differ from system to system, content-based filtering systems share in common a means for creating user profiles, and a means for comparing products to user profiles for recommendation.

User Interface

Interested Documents

A Browsed Documents

B Matched documents

User Profile

Information Repository

Fig. 4.1. Content-based ﬁltering.

4.2.1 Construction of User Proﬁles

A content-based filtering system usually processes textual products such as news articles and web pages. For non-textual ones (e.g. multimedia files) the textual descriptions of these items in the file metadata are utilized. For instance, a music filtering system typically utilizes the textual descriptions such as a music genre, piece 4 Filtering Techniques for Selection of Contents and Products 43 title, artist name, and album title to build user profiles. A user profile can be explicitly customized by a user or implicity constructed by the system based on the user’s behaviors. In user customization, a user is provided with an interfaces as a step in registration, where possibly check boxes are used for a user to select from a known set of attributes, e.g. name of favorite singers and genre of favorite books or music. Alternatively, the interface may allow a user to enter texts in an edit box to express his or her interests, e.g. scenario of interested movies, name of musician. Obviously, user customization puts a big burden on end users. Because not many users would like to make this effort at the registration. In addition, users are reluctant to revise their profiles frequently as their preferences change. Due to the limitation of manual user customization, automatic construction of user profiles is usually adopted in content-based filtering systems. In this case, the system first selects interested products based on a user’s browsing behaviors. Second, a user profile, typically represented as a set of keywords, is generated based on the textual contents of interested items. For instance, the Fab system [Balabanovic and Shoham, 1997], a web page recommender, represents a web page preferred by the users with the 100 most “important” words therein. A user profiles is generated as a set of words using these summary words of the preferred pages. The “importance” of a word can be determined by a weighting metric. The most accepted one is the normalized TF-IDF metric [Baeza-Yates and Ribeiro-Neto, 1999]. The weight of term i in document j is denoted by wi,j :

(1 α)fi,j N wi,j = α + − log , (4.1) „ maxifi,j « × ni where N is total number of documents that can be recommended to a user, ni is the number of documents containing term i, and α is a weighting factor usually set to 0.5. fi,j is the frequency of word i in document j; N maxi fi,j is the maximum frequency among all words that appear in document dj ; and the factor log is a ni variant of the inverse document frequency, an indication of the rareness of term i in the collection of documents to be recommended.

4.2.2 Selection of Recommended Products

Once a user profile is constructed, we should compare the profile against an information repository to select relevant products. There are several techniques to evaluate the similarity between a user profile and a product using Boolean methods [Anick et al., 1989, Lee, 1994, Verhoeff et al., 1961], vector-space methods [Salton and Buckley, 1988], probabilistic models [Robertson and Sparck Jones, 1976], neural networks [Kim and Raghavan, 2000, Wilkinson and Hingston, 1991], fuzzy set models [Ogawa et al., 1991], etc. The most accepted is the vector-space approach using cosine similarity. As mentioned above, a user profile u and a product p can be represented as TF-IDF vectors u =< w1,u, w2,u, ...wM,u > and p =< w1,p, w2,p, ...wM,p > where M is the total number of wordsWin the system. Essentially, a user profileW or a product is a document, and each component of the vector can be calculated by Equation 4.1. The cosine similarity between a profile and an item is denoted by s(u, p):

M wi,u wi,p s(u, p) = i=1 × . (4.2) PM 2 M 2 i=1 wi,u i=1 wi,p q q For example, if a user has read a large numberP of news articlesP related to financial risk investment, then a content-based recommender will provide other relevant articles to him/her. By processing the articles read by this user, a user profile can be constructed. Intuitively, an article interested by the user would have a high frequency for the finance-related terms, which is reflected by this user’s profile. Consequently, a recommender system using the above cosine similarity measure will assign a high similarity value to the articles containing more finance-related terms.

4.2.3 Limitations

Although content-based ﬁltering has proven to be effective in recommending textual items relevant to a topic, it also has its limitations [Adomavicius and Tuzhilin, 2005]: 44 Qing Li, Yuanzhu Peter Chen, and Zhangxi Lin

Content-based filtering more than often provides recommendation in a literal sense, because all the infor- • mation is selected and recommended based on textual contents. Even though a product was essentially useful, it might be under-valued because of the ambiguous and thus misleading appearance of the textual contents. In addition, it is indistinguishable in quality of the recommended products. This is because the term vector for a product simply captures the frequency of each term in an article, and a poorly worded article can well have an equal or even higher similarity value than a finely written one. Content-based filtering generally works well with sufficient textual information. However, other multimedia • files such as images, audio and video streams are not applicable if the metadata do not have enough textual information. A new user of such a system must face a “slow start” because he/she has to rate relatively large number • of items before a recommender can induce his/her preferences and can serve him/her with reliable recommendation. Content-based filtering usually focuses on recommending the most similar to users. However, in some cases • the most similar product is effective redundant, such as two different news articles describing the same event. For example, Daily-learn [Billsus and Pazzani, 2000] discards items not only if they are different from a user’s preferences but also overly similar to the previously recommended ones.

4.3 Collaborative Filtering

Unlike content-based filtering, collaborative filtering is a technology wherein peer opinions are employed to predict the interests of others. The idea of collaborative filtering has been around for about 20 years. Gold- berg and his colleagues first applied collaborative filtering to recommender systems [Goldberg et al., 1992, Terry, 1993]. There, binary preference indicators and custom texts were applied to identify like-minded users manually. Subsequently, GroupLens [Resnick et al., 1994] and Ringo [Shardanand and Maes, 1995], developed independently, were proposed for automated prediction. Collaborative filtering has prevailed not only in research but also in the industry. There are plenty of collaborative recommender systems designed for different purposes. These include the MovieLens system for movies, Jeter system for jokes [Gupta et al., 1999], Flycasting for online radio [Hauver and French, 2001], VERSIFI (formerly AdaptiveInfo.com) for news, etc. Most collaborative filtering techniques falls into one of two categories: memory-based and model-based methods. Memory-based methods store massive raw user ratings in a database which can be uploaded into computer memory at once to compute preference similarities among users. Here, a user rating corresponds to a point in a metric space. In contrast, model-based algorithms build abstract models upon raw ratings and recommend products using these light-weight models. Both approaches have been shown to be effective in collaborative filtering.

4.3.1 Memory-Based Collaborative Filtering

Memory-based collaborative filtering techniques have been deployed widely at many commercial web sites because not only are they simple and conceptually intuitive, but also they are deemed sufficiently accurate for many real-world applications. One of the earliest memory-based collaborative filtering systems [Shardanand and Maes, 1995] is devised by the GroupLens Research Lab, University of Minnesota. It adopts the Pearson correlation coefficient method to compute the similarities among users which are later used to group the like- minded users. Based on the opinions of like-minded users about a certain product, the recommender can infer the preference of a user who does not have a rating for this product. Generally, a memory-based collaborative filtering algorithm consists of two steps:

1. Calculate user similarities to ﬁnd the nearest neighbors of the target user in the metric space. One of the most accepted metrics to compute similarity is the Pearson Correlation Coefﬁcient. Consider two users x and y and the intersection of their rated products contains m elements, i.e. they share m co-rated products. Let Rx,i (respectively Ry,i) denote the rating on product i by user x (respectively user y), which is typically in 1, 2, 3...5 . As a shorthand, we use R¯x (respectively R¯y) to denote the average rating value { } 4 Filtering Techniques for Selection of Contents and Products 45

of user x (respectively user y) over all the products. The similarity between user x and y is denoted by Sp(x, y):

m cov(x, y) ((Rx,i R¯x) (Ry,i R¯y)) Sp(x, y) = = i=1 − × − , (4.3) σxσy Pm ¯ 2 m ¯ 2 i=1(Rx,i Rx) i=1(Ry,i Ry) q − q − 2. Predict a user’s rating on a yet-to-rat productPbased on the ratings ofPthe above selected near neighbors. The prediction of user x on product i, denoted by Px,i, in the simplest form could be computed as the average ratings of these neighbors on this product. However, the most common approach [Shardanand and Maes, 1995] is based on a weighted average of rating deviations of like-minded users. Formally,

n (Ry,i R¯y) Sp(x, y) ˆ y=1 Px,i = Rx + n − × , (4.4) P Sp(x, y) y=1 | | P 0 where n is the number of the selected nearest neighbors, and Rˆx is user x s average rating value over all products that he/she has rated so far. The following is a simple example of applying the above memory-based algorithm for recommendations. Four users have rated a subset of six pieces of music using values in 1, 2, ..., 5 as shown in Table 4.1. { }

Table 4.1. User-product table. Kate Luke Mike Nancy Music 1 1 4 2 Music 2 5 2 4 Music 3 3 5 Music 4 2 5 Music 5 4 1 Music 6 2 5

The higher the score, the more preference the user. The recommendation problem is to estimate the users’ ratings for the unrated products in the table, i.e. to ﬁll in the empty cells. To predict Kate’s rating on music 6, we ﬁrst compute the similarity between Kate and Luke using Equation 4.3.

¯ (1+5+2+4) RKate = 4 = 3 (4+2+5+1) RLuke = 4 = 3 Sp(Kate, Luke) = = (1−3)×(4−3)+(5−3)×(2−3)+(2−3)×(5−3)+(4−3)×(1−3) √(1−3)2+(5−3)2+(2−3)2+(4−3)2×√(4−3)2+(2−3)2+(5−3)2+(1−3)2 = 0.8 − This means that Kate and Luke have dissimilar appreciation in music. Similarly, we can compute the metric between Kate and Mike, which is 1.0. This indicates that they have a very similar taste in music. Note that there is no need to compute the value for Kate and Nancy since they have no co-rated music pieces and Nancy dose not have a reference rating for music 6. Next, we select the 2-nearest neighbors of Kate using these similarity values, i.e. Luke and Mike. Using the rating of Mike and that of Luke for music 6, we obtain an estimated rating of Kate for this piece:

¯ ¯ (RLuke,6−RLuke)×Sp(Kate,Luke)+(RMike,6−RMike)×Sp(Kate,Mike) P , = Rˆ + Kate i Kate |Sp(Kate,Luke)|+|Sp(Kate,Mike)| (2−3)×(−0.8)+(5−3)×1 = 3 + |1|+|−0.8| = 4.56 Therefore, the recommender can recommend music 6 to Kate since she is likely to appreciate it based on the prediction score 4.56. 46 Qing Li, Yuanzhu Peter Chen, and Zhangxi Lin

There are a series of further research on memory-based collaborative filtering with some adjustments. For instance, a variety of similarity metrics are adopted instead of the original Pearson correlation, such as the mean-squared difference [Shardanand and Maes, 1995], vector similarity [Breese et al., 1998] or probabilistic similarity [Pennock et al., 2000]. Although details of these techniques are different, all of them share the same intuition that users tend to have similar opinions on a product if they have similar preferences on other products. These memory-based filtering techniques are referred to as user-based approaches. In contrast, item-based collaborative filtering, e.g. Sarwar et al. [2001] assume that a user would also be interested in products similar to the ones that he/she has high ratings for. For instance, if a user has bought many action movies before, he/she usually has a strong intention to buy more action movies. Compared to user-based approaches, item- based methods show more advantages from a theoretical point of view. First, human beings can be emotional and may change their preferences and interests over time or under different circumstances. On the contrary, the features of merchandises are fairly stable. Therefore, the similarity between items often involves a higher system overhead. Second, the construction of user profiles is a little difficult and usually adds the burden of users to input. Furthermore, users are sometimes reluctant to reveal their private information through user profiles.

4.3.2 Model-Based Collaborative Filtering

In contrast to memory-based collaborative filtering, a model-based method constructs an abstract model for underlying user preferences from which predictions are inferred. Several approaches have been proposed and studied in previous work [Popescul et al., 2001, Huang et al., 2004, Hofmann, 2003, Jin et al., 2003b, Nas- raoui and Pavuluri, 2004]. All of these share the same idea of clustering, i.e. partitioning users or products into groups of similar tastes or attributes for recommendations. Examples include the Bayesian clustering (BC) model [Breese et al., 1998], Bayesian network (BN) model [Breese et al., 1998], and Aspect model [Hofmann, 2003]. The basic idea of the BC model is to assume that ratings are observations of the multinomial mixture model with parameters, and the model is estimated using the Expectation Maximization (EM) algorithm. The BN model aims at capturing item dependency, where each item is a node and dependency structure is learned from observables. The Aspect model is based on a latent cause model that introduces the notion of user communities or groups of items. With the successful application of the aspect model, various probabilistic models have been developed by researchers to complement its shortcomings. The flexible mixture model [Si and Jin, 2003] extends clustering algorithms for collaborative filtering by clustering both users and items simultaneously without assuming that each user and item should only belong to a single cluster. The preference-based graphic model [Jin et al., 2003a] addresses the behaviorial distinction between users. That is, users with similar interests in items may have very different rating habits; some users tend to assign a higher rating to all items than other users. Popescul et al. [2001] extend the Aspect model by incorporating the content information of items in order to improve the quality of recommendation and to overcome rating data sparsity. Hofmann [2004] further extends the Aspect model to Probabilistic Latent Semantic Analysis (PLSA), which supports numerical ratings instead of categorical ratings (e.g. binary ratings). Pennock et al. [2000] propose a method combining both memory-based and model-based approaches and demonstrate that this combined approach provide better recommendations than pure memory-based or model- based collaborative approaches. Yu et al. [2002] also propose a probabilistic approach to collaborative filtering combining both approaches. In particular, it first learns a probabilistic model for each user’s preferences using active learning. Next, a mixture model of user profiles is applied for recommendations.

4.3.3 Limitations

Since collaborative ﬁltering uses other users’ ratings, it can deals with any kind of content for universal recommendation, even those that are quite different from what have been rated in nature. However, collaborative ﬁltering has its own limitations :

Sparsity problem. In any recommender system, the number of ratings collected is usually very small • compared to the number of predictions to make. Effective prediction of ratings from a small number of examples is critical in this sense. One solution to this problem is to adopt user proﬁle information or item 4 Filtering Techniques for Selection of Contents and Products 47

attributes when calculating user or item similarities. Alternatively, the sparsity problem can be addressed by applying an associative retrieval framework and related spreading activation algorithms to explore transitive associations among consumers through their past transactions and feedback [Huang et al., 2004]. Cold start problem. It is hard for pure collaborative ﬁltering to recommend a new item where there is no • historical data and hence no user opinions about it. Music 5 in Table 4.2 is an example. The same holds for new users to the system.

Table 4.2. User bias problem. Item Jack Oliver Peter Tom Genre Music 1 5 4 3 Rock Music 2 4 4 3 Rock Music 3 4 3 Rock Music 4 4 3 Country Music 5 Rock Music 6 4 Rock

Ignorance problem. Collaborative filtering can be ignorant to certain hidden information (e.g. item cat- • egory) that could be used for better system performance. For instance in Table 4.2, collaborative filtering only utilities the rating columns but ignores the genre columns in practice. collaborative recommenders are based on the parts of ratings from users which may mislead the inferences made by systems. Here, Music 3 and 4 have the same ratings, and thus have the same chance to be recommended to user Jack by item-based filtering. However, if the system knows that Music 1,2 and 3 fall in the rock category and Music 4 in the country category, it can recommend Music 3 to Jack with a higher priority, provided that he is known to better appreciate rock music.

4.4 Hybrid Filtering

Although collaborative filtering is able to improve the quality of content-based recommendations based on the ratings of different users, it completely ignores other information that may be available from the semantic contents. On the other hand, content-based filtering does not suffer from this problem. As such, a hybrid approach which combines content-based and collaborative filtering helps to alleviate this problem. Hybrid recommender systems generally are sub-categorized into three different models. The first is the simple combination model, which combines results from the collaborative and content- based filters as shown in Fig. 4.2. For example, ProfBuilder [Wasfi, 1998] recommends web pages using both content-based and collaborative filters, each of which creates a separate recommendation list. Claypool et al. [1999] describe a hybrid approach for online news delivery, combining the two predictions using an adaptive weighted average. As the number of users accessing an item increases, so does the weight of the collaborative component. However, Claypool et al. do not clearly specify how to determine the weights of collaborative and content-based components as the system continues to operate. The second is the sequential combination model as shown in Fig. 4.3. In this model, user profiles are first constructed by a content-based filtering algorithm. Next, a collaborative filtering algorithm is applied to make predictions based on these user profiles, such as RAAP [Delgado et al., 1998] and Fab filtering [Balabanovic and Shoham, 1997]. RAAP uses a content-based filtering technique to help a user to classify domain specific information found on the Web, and also recommends URLs to other users with similar interests. To determine the similarity of interests among users, a scalable Pearson correlation algorithm based on the web page categories is used. The Fab system also uses content-based techniques rather than user ratings to create user profiles. Hence the quality of predictions is completely dependent on the content-based filtering techniques, and inaccurate profiles can cause inaccurate correlations among users, thus yielding poor predictions. 48 Qing Li, Yuanzhu Peter Chen, and Zhangxi Lin

Rating Collaborative Matrix Filter Combination

Products Content-based Filter

Fig. 4.2. Simple combination.

Content- User Collaborative Products based Filter Profiles Filter

Fig. 4.3. Sequential combination.

The third is the synthetic combination model, in which both semantic contents and user ratings are applied to make recommendations; these include the probabilistic model by Popescul et al. [2001], Ripper system by Basu et al. [1998], and the music recommender by Li et al. [2004, 2007]. Ripper is a machine learning system using a combination of content information and rating data in an effort to produce better recommendations. Good et al. [1999] combine personal information filtering agents and user ratings to make recommendations. Popescul et al. [2001] provide a probabilistic model for unified collaborative and content-based recommendations. Li et al. [2004, 2007] present a music recommender system based on an item-based probabilistic model, where items are classified into groups and predictions are made for users considering the Gaussian distribution of user ratings. Audio features and user ratings are utilized together for classification and recommendation.

Rating Matrix Combination Filter Products

Fig. 4.4. Synthetic combination.

4.5 Challenges for the Current Filtering Techniques

Recommender systems discussed so far can be extended in several ways to improve the scalability, data sparsity tolerance, and effectiveness of recommender systems. Such more advanced recommender systems can provide better recommendation capabilities. Here, we review some preliminary efforts in these directions and also identify various future research opportunities.

4.5.1 Scalability

Existing recommender systems can deal with thousands of consumers within a reasonable amount time, but the demand of modern e-commerce systems is to handle millions of consumers. On the other hand, more and more 4 Filtering Techniques for Selection of Contents and Products 49 products are available for consumers to select at an exponentially increasing speed. A robust and successful recommender system should have the ability to process large number of recommended items for numerous consumers. One possible approach to the scalability problem is to divide products or users into different groups. Sarwar et al. [2002] partition the users in a system using a clustering algorithm and threat each partition as subsystem. In particular, it first executes a clustering algorithm to create user groups. Next, for a given user, the selection of its nearest neighbors is restricted to groups that are close to this user. As a further extension, Xue et al. [2005] introduces a smoothing-based method where smoothed clusters are used for neighbor selection. Traditional filtering algorithms usually are implemented in one or several central servers which play a management role to collect information from individual users and make recommendations. However, ideas from the community of distributed computing can be borrowed for building more scalable filters. Han et al. [2004a,b] provide a distributed filtering algorithm, PipeCF, using a peer-to-peer (P2P) overlay based on Distributed Hash Tables (DHTs). However, the performance of DHTs under the dynamic conditions common for P2P systems is unknown. Tveit [2001] proposes a P2P collaborative system for mobile commerce based on Gnutella. Peers are represented as software assistant agents interfacing with a mobile customer. When a peer receives a query rating vector, it calculates the similarity to the rating vector carried by the message and caches the message at the node. If the similarity is greater than a threshold, the cached voting vector is sent back to the peer which had issued the query rating vector. Otherwise, the query rating vector is broadcast further to its neighbors. It is noticed that this system has the communication overhead as any P2P network with an unstructured overlay. Kim et al. [2008] provide a distributed collaborative filtering algorithm which is based on the Gnutella schema. It applies a self-organizing protocol using friend lists to create shortcuts among peers, which provides a logical network on top of Gnutella’s unstructured overlay. This feature makes it suitable work over any straightforward use of the Gnutella schema such as Tveit.

4.5.2 Data Sparsity

Data sparsity refers to the difficulty that most users rate only a small number of products and hence a very sparse user-item matrix is available. Consequently, the accuracy of the method is often quite poor. In addition, because of data sparsity, a recommender system may have insufficient ratings to make a recommendation. In particular, for memory-based filtering algorithms, user similarity is only defined between customers who have rated at least two products in common. Many pairs of customers have no correlation at all. In practice, many commercial recommender systems are used to evaluate large product sets (e.g. Amazon.com recommends books and CDnow recommends music albums). In these systems, even active customers may have rated well under 1% of the products (1% of 2 million books is 20,000 books — a large set on which to have an opinion). Accordingly, the system may not be able to provide a particular user with a non-trivial portion of all items in the system. To overcome data sparsity, Brand [2003] and Sarwar et al. [2000] apply the Singular Value Decompo- sition (SVD) technique to reduce the dimensionality of the customer-product ratings matrix. Others rely on the estimation of missing ratings based on the obtained ones. Xue et al. [2005] introduce a smoothing-based method to estimate unrated products. Hu and Lu [2006] also apply a hybrid predictive algorithm with smoothing. Their algorithm uses item-based methods to provide the basis for data smoothing and builds a prediction model based on both users’ and items’ aspects for better robustness against sparsity and for higher prediction accuracy. Wang et al. [2006] reformulate the memory-based collaborative filtering problem under a generative probabilistic framework, treating individual user-item ratings as predictors of missing ratings. The final rating is estimated by fusing predictions from three different sources. The complete model is more robust against data sparsity because different types of ratings are used in concert while additional ratings from similar users towards similar items are employed as a background model to smooth the predictions.

4.5.3 Effectiveness of Recommendation

The problem of developing good metrics to measure the effectiveness of recommendation has been extensively addressed in the recommender systems literature. Some examples of this work include [Herlocker et al., 1999, 50 Qing Li, Yuanzhu Peter Chen, and Zhangxi Lin

2004, Yang and Padmanabhan, 2001]. In most of the recommender systems literature, the performance evaluation of recommendation algorithms is usually done in terms of coverage and accuracy. The coverage measure is the percentage of items for which a recommender system is capable of providing predictions. The accuracy measures can take two forms. The statistical accuracy metric compares the estimated ratings against the actual ratings including Mean Absolute Error (MAE), root mean squared error, and correlation between predictions and ratings. Alternatively, it may also determine how well a recommender system can make predictions of high-relevance items. In this sense, an accuracy measure includes classical information retrieval measures of precision, recall, F-measure (a harmonic mean of precision and recall) in recommender systems [Adomavicius and Tuzhilin, 2005]. Despite their popularity, these empirical evaluation measures have some limitations. These measures are typically performed on test data that the users chose to rate. However, items that users choose to rate are likely to constitute a skewed sample, e.g. users may rate mostly the items that they are interested in. Understand- ably, it is expensive and time-consuming to conduct controlled experiments with users in the recommender systems settings. In addition, although crucial for measuring the accuracy of recommendations, the technical measures mentioned earlier often do not adequately capture “usefulness” and “quality” of recommendations. For example, as Wade [2003] observes for a supermarket application, recommending obvious items (such as milk or bread) that the consumer will buy anyway will produce high accuracy rates; however, it will not be very innovative to the consumer. Developing and studying the measures that would remedy the limitations described here indicates an interesting and important research direction.

4.5.4 Further Investigation

Despite the efforts in the fields mentioned above, other important issues have been explored include user privacy, trustworthiness and context-aware recommendation. One of user concerns to use recommender systems freely and comfortably is user privacy. Users are usually • reluctant to disclose their private information such as purchase, reading, browsing records. However, most current filtering algorithms need to obtain user private information for further analysis and recommendation services. Some work [Canny, 2002, Polat and Du, 2003] has studied on how to protect user privacy in recommender systems. Current filtering techniques assume that user ratings are trustable and treat all users equally. However, some • authors [Kautz et al., 1997, Zacharia et al., 2000, Massa and Avesani, 2004, O’Donovan and Smyth, 2005] argue that the opinions of experts should be more emphasized than that of novices. Context-aware recommendation [Brunato and Battiti, 2003, Byung Kwon, 2003, Lemire et al., 2005, Chen, • 2005, Yu et al., 2006] study on how to apply contextual information to improve a system performance and provide on-demand services in a certain circumstance. For instance, a context-aware restaurant recommender system provides a traveller with recommended restaurants which are close to his/her current location.

4.6 Conclusion

Research in recommender systems has made signiﬁcant progresses over last decades when numerous content- based, collaborative, and hybrid methods have been proposed and several commercial systems have been developed. However, despite all of these advances, current recommender systems still deserve further improvements to make them more effective in a broader range of applications.

References

G. Adomavicius and A. Tuzhilin. Toward the next generation of recommender systems: a survey of the state-of- the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6):734–749, 2005. 4 Filtering Techniques for Selection of Contents and Products 51

P. G. Anick, J. D. Brennan, R. A. Flynn, D. R. Hanssen, B. Alvey, and J. M. Robbins. A direct manipulation interface for Boolean information retrieval via natural language query. In 13th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Cambridge, Massachusetts (USA), 1989. R. Baeza-Yates and B. Ribeiro-Neto. Modern information retrieval. Addison-Wesley, 1999. M. Balabanovic and Y. Shoham. Fab: Content-based, collaborative recommendation. Communications of the ACM, 40(3):66–72, 1997. C. Basu, H. Hirsh, and W. Cohen. Recommendation as classification: Using social and content-based information in recommendation. In 15th National Conference on Artificial Intelligence, Madison, Wisconsin (USA), 1998. D. Billsus and M. J. Pazzani. User modeling for adaptive news access. User Modeling and User-Adapted Interaction, 10(2):147–180, 2000. M. Brand. Fast online SVD revisions for lightweight recommender systems. In 3rd SIAM International Con- ference on Data Mining, San Francisco, California (USA), 2003. J. S. Breese, D. Heckerman, and C. Kadie. Empirical analysis of predictive algorithms for collaborative filtering. In 14th Conference on Uncertainty in Artificial Intelligence, Madison, Wisconsin (USA), 1998. M. Brunato and R. Battiti. PILGRIM: A location broker and mobility-aware recommendation system. In 1st IEEE International Conference on Pervasive Computing and Communications, Dallas, Texas (USA), 2003. O. Byung Kwon. “i know what you need to buy”: Context-aware multimedia-based recommendation system. Expert Systems With Applications, 25(3):387–400, 2003. J. Canny. Collaborative filtering with privacy. In IEEE Symposium on Security and Privacy, Oakland, California (USA), 2002. A. Chen. Context-aware collaborative filtering system: Predicting the user’s preference in the ubiquitous computing environment. In Extended Abstracts of the Conference on Human Factors in Computing Systems, Portland, Oregon (USA), 2005. M. Claypool, A. Gokhale, T. Miranda, P. Murnikov, D. Netes, and M. Sartin. Combining content-based and collaborative filters in an online newspaper. In ACM SIGIR Workshop on Recommender Systems, Berkeley, California (USA), 1999. J. Delgado, N. Ishii, and T. Ura. Content-based collaborative information filtering. Lecture Notes in Computer Science, 1435:206–215, 1998. D. Goldberg, D. Nichols, B. M. Oki, and D. Terry. Using collaborative filtering to weave an information tapestry. Communications of the ACM, 35(12):61–70, 1992. N. Good, J. B. Schafer, J. A. Konstan, A. Borchers, B. M. Sarwar, J. L. Herlocker, and J. T. Riedl. Combining collaborative filtering with personal agents for better recommendations. In 16th National Conference on Artificial Intelligence, Orlando, Florida (USA), 1999. D. Gupta, M. Digiovanni, H. Narita, and K. Goldberg. Jester 2.0: A new linear-time collaborative filtering algorithm applied to jokes. In Workshop on Recommender Systems, in conjunction with ACM SIGIR, Berkeley, California (USA), 1999. P. Han, B. Xie, F. Yang, and R. Shen. A scalable P2P recommender system based on distributed collaborative filtering. Expert Systems with Applications, 27(2):203–210, 2004a. P. Han, B. Xie, F. Yang, J. Wang, and R. Shen. A novel distributed collaborative filtering algorithm and its implementation on P2P overlay network. In 8th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, Sydney, Australia, 2004b. D. B. Hauver and J. C. French. Flycasting: using collaborative filtering to generate a playlist for online radio. In 1st International Conference on Web Delivering of Music, Florence, Italy, 2001. J. L. Herlocker, J. A. Konstan, and J. T. Riedl. An algorithmic framework for performing collaborative filtering. In 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Berkeley, California (USA), 1999. J. L. Herlocker, J. A. Konstan, L. G. Terveen, and J. T. Riedl. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems, 22(1):5–53, 2004. 52 Qing Li, Yuanzhu Peter Chen, and Zhangxi Lin

T. Hofmann. Collaborative filtering via Gaussian probabilistic latent semantic analysis. In 26th Annual Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval, Toronto, Canada, 2003. T. Hofmann. Latent semantic models for collaborative filtering. ACM Transactions on Information Systems, 22 (1):89–115, 2004. R. Hu and Y. Lu. A hybrid user and item-based collaborative filtering with smoothing on sparse data. In 16th International Conference on Artificial Reality and Telexistence, Hangzhou, China, 2006. Z. Huang, H. Chen, and D. Zeng. Applying associative retrieval techniques to alleviate the sparsity problem in collaborative filtering. ACM Transactions on Information Systems, 22(1):116–142, 2004. R. Jin, L. Si, and C. X. Zhai. Preference-based graphic models for collaborative filtering. In 19th Conference on Uncertainty in Artificial Intelligence, Acapulco, Mexico, 2003a. R. Jin, L. Si, C. X. Zhai, and J. Callan. Collaborative filtering with decoupled models for preferences and ratings. In 12th ACM International Conference on Information and Knowledge Management, New Orleans, Louisiana (USA), 2003b. H. Kautz, B. Selman, and M. Shah. Referral Web: Combining social networks and collaborative filtering. Communications of the ACM, 40(3):63–65, 1997. B. M. Kim, Q. Li, A. E. Howe, and Y. P. Chen. Collaborative web agent based on friend networks. Applied Artificial Intelligence, 22, 2008. M. Kim and V. V. Raghavan. Adaptive concept-based retrieval using a neural network. In ACM SIGIR Workshop on Mathematical/Formal Methods in Information Retrieval, Athens, Greece, 2000. J. H. Lee. Ranking documents in thesaurus-based Boolean retrieval systems. Information Processing and Management, 30(1):79–91, 1994. D. Lemire, H. Boley, S. McGrath, and M. Ball. Collaborative filtering and inference rules for context-aware learning object recommendation. International Journal of Interactive Technology and Smart Education, 2 (3):179–188, 2005. Q. Li and B. M. Kim. Clustering approach for hybrid recommender system. In IEEE/WIC International Conference on Web Intelligence, Halifax, Canada, 2003. Q. Li, B. M. Kim, D. H. Guan, and D. Whan Oh. A music recommender based on audio features. In 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Sheffield, United Kingdom, 2004. Q. Li, S. H. Myaeng, and B. M. Kim. A probabilistic music recommender considering user opinions and audio features. Information Processing and Management, 43(2):473–487, 2007. P. Massa and P. Avesani. Trust-aware collaborative filtering for recommender systems. In International Con- ference on Cooperative Information Systems, Agia Napa, Cyprus, 2004. O. Nasraoui and M. Pavuluri. A context ultra-sensitive approach to high quality web recommendations based on web usage mining and neural network committees. In 3rd International Workshop on Web Dynamics, in conjunction with WWW, New York, USA, 2004. J. O’Donovan and B. Smyth. Trust in recommender systems. In 10th International Conference on Intelligent User Interfaces, San Diego, California (USA), 2005. Y. Ogawa, T. Morita, and K. Kobayashi. A fuzzy document retrieval system using the keyword connection matrix and a learning method. Fuzzy Sets and Systems, 39(2):163–179, 1991. D. M. Pennock, E. Horvitz, S. Lawrence, and C. L. Giles. Collaborative filtering by personality diagnosis: A hybrid memory-and model-based approach. In 16th Conference on Uncertainty in Artificial Intelligence, Stanford, California (USA), 2000. H. Polat and W. Du. Privacy-preserving collaborative filtering. International Journal of Electronic Commerce, 9(4):9–35, 2003. A. Popescul, L. H. Ungar, D. M. Pennock, and S. Lawrence. Probabilistic models for unified collaborative and content-based recommendation in sparse-data environments. In 17th Conference on Uncertainty in Artificial Intelligence, Seattle, Washington (USA), 2001. P. Resnick, N. Iacovou, M. Suchak, P. Bergstrom, and J. T. Riedl. GroupLens: an open architecture for collaborative filtering of netnews. In ACM Conference on Computer-Supported Cooperative Work, Chapel Hill, North Carolina (USA), 1994. 4 Filtering Techniques for Selection of Contents and Products 53

S. E. Robertson and K. Sparck Jones. Relevance weighting of search terms. Journal of the American Society for Information Science, 27(3):129–46, 1976. G. Salton and C. Buckley. Term-weighting approaches in automatic text retrieval. Information Processing and Management, 24(5):513–523, 1988. B. M. Sarwar, G. Karypis, J. A. Konstan, and J. T. Riedl. Application of dimensionality reduction in recommender system-a case study. In International Workshop on Web Mining for E-Commerce – Challenges and Opportunities, Boston, Massachusetts (USA), 2000. B. M. Sarwar, G. Karypis, J. A. Konstan, and J. Reidl. Item-based collaborative filtering recommendation algorithms. In 10th International World Wide Web Conference, Hong Kong, China, 2001. B. M. Sarwar, G. Karypis, J. A. Konstan, and J. T. Riedl. Recommender systems for large-scale e-commerce: Scalable neighborhood formation using clustering. In 5th International Conference on Computer and Infor- mation Technology, Shanghai, China, 2002. U. Shardanand and P. Maes. Social information filtering: Algorithms for automating “word of mouth”. In SIGCHI Conference on Human factors in Computing Systems, Denver, Colorado (USA), 1995. L. Si and R. Jin. Flexible mixture model for collaborative filtering. In 12th International Conference on Machine Learning (ICML), Washington, D.C. (USA), 2003. D. B. Terry. A tour through Tapestry. In Proceedings of the Conference on Organizational Computing Systems, Milpitas, California (USA), 1993. A. Tveit. Peer-to-peer based recommendations for mobile commerce. In 1st International Workshop on Mobile Commerce, Rome, Italy, 2001. J. Verhoeff, W. Goffman, and J. Belzer. Inefficiency of the use of Boolean functions for information retrieval systems. Communications of the ACM, 4(12):557–558, 1961. W. Wade. A grocery cart that holds bread, butter and preferences. New York Times, 2003. J. Wang, A. P. de Vries, and M. J. T. Reinders. Unifying user-based and item-based collaborative filtering approaches by similarity fusion. In 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, Washington (USA), 2006. A. M. A. Wasfi. Collecting user access patterns for building user profiles and collaborative filtering. In 4th International Conference on Intelligent User Interfaces, Los Angeles, California (USA), 1998. R. Wilkinson and P. Hingston. Using the cosine measure in a neural network for document retrieval. In 14th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Chicago, Illinois (USA), 1991. K. Wittenburg, D. Das, W. Hill, and L. Stead. Group asynchronous browsing on the World Wide Web. In 4th International World Wide Web Conference, Boston, Massachusetts (USA), 1995. G. R. Xue, C. Lin, Q. Yang, W. S. Xi, H. J. Zeng, Y. Yu, and Z. Chen. Scalable collaborative filtering using cluster-based smoothing. In 28th Annual International ACM SIGIR Conference on Research and Develop- ment in Information Retrieval, Salvador, Brazil, 2005. Y. Yang and B. Padmanabhan. On evaluating online personalization. In Workshop on Information Technology and Systems, Montréal, Canada, 2001. K. Yu, X. Xu, J. Tao, M. Ester, and H. P. Kriegel. Instance selection techniques for memory-based collaborative filtering. In 2nd SIAM International Conference on Data Mining, Arlington, Virginia (USA), 2002. Z. Yu, X. Zhou, D. Zhang, C. Y. Chin, X. Wang, and J. Men. Supporting context-aware media recommendations for smart phones. IEEE Pervasive Computing, 5(3):68–75, 2006. G. Zacharia, A. Moukas, and P. Maes. Collaborative reputation mechanisms for electronic marketplaces. De- cision Support Systems, 29(4):371–388, 2000.

5 Personalization and Contextualization of Media

Marcus Specht1, Andreas Zimmermann2 and Tim de Jong1

1 Educational Technology Expertise Centre, Open Universiteit Nederlands Valkenburgerweg 177, 6419 AT Heerlen {marcus.specht,tim.dejong}@ou.nl 2 Fraunhofer-Institute for Applied Information Technology FIT Schloss Birlinghoven, 53754 Sankt Augustin, Germany Andreas.Zimmermann@ﬁt.fraunhofer.de

Summary. Personalization allows users to obtain information that is adapted to their needs, goals, knowledge, interests or other characteristics. User models deliver the main parameters for selecting and adapting information presentation to the individual user. Nevertheless the individual user is always in a current context and contextualization of information systems is an important research topic in several areas of research like context-aware systems, ubiquitous computing, or applications areas contextualized learning support. Contextu- alization complements personalization so that environmental states or the context of use can also be taken into account. The paper will show how personalization and contextualization approaches are related in different phases of the adaptation process and how the approaches can be used in a complementary way for personalized and contextualized information systems. The analysis will guide the reader through the phases of information acquisition in adaptive information systems, the representation of personal and contextual information, and the selection adaptive methods applicable based on personal proﬁles and on context information available. Furthermore application examples will be described which integrate personalization and contextualization in ubiquitous computing for learning and information delivery. The paper will present an integrated approach for describing and planning personalized and contextualized information systems which allow the individual to become a digital nomad making use of ubiquitous computing facilities while still having a personal information space. The strengths and weaknesses of today’s approaches will be discussed.

5.1 Introduction

In the last decade personalization has been an issue of discussions in different research fields [Kobsa and Wahlster, 1989, Brusilovsky, 1996, Kobsa, 1996, Kim, 2002]. Recently also for a variety of products personalization is an essential added value and more and more personalized services allow for personalized user portals in the Web 2.0. In the past personalization was mostly interpreted as an adaptation of a machine service to a human user based on a profile of a user. Examples for adaptive methods and personalization can be found in different research areas as intelligent tutoring systems [Carbonell, 1970, Clancey, 1987, Anderson et al., 1989], adaptive user interfaces [Carroll and Carrithers, 1984, Brusilovsky et al., 1995], adaptive hypermedia [Brusilovsky, 1996], intelligent multimedia or intelligent agents [Feiner and McKeown, 1995, Rickel and Johnson, 1997]. In the last years new research approaches like context-aware computing [Dey and Abowd, 1999, Gross and Specht, 2001, Oppermann and Specht, 2000, Schilit et al., 1994, Zimmermann et al., 2005] extended the idea of personalization for not only adapting to a personal profile of a user (preferences, knowledge, or cognitive characteristics) but also the context of use [Oppermann and Specht, 2000] became more and more important. In the effort to create added value in ubiquitous environments, the current context of use becomes more important as the computer disappears. From our point of view, it is not enough to supply content or services that 56 Marcus Specht, Andreas Zimmermann and Tim de Jong consider single environmental or user characteristics; we need to identify approaches for the integration and interpretation of different sensing components for modeling the user and the context more appropriately. The combination of context sensor data with user modeling appears to be one of the key challenges for personalized information systems of the future. Jameson [2001] describes how modeling the user and modeling the context of the user can both lead to more valid assumptions about the current needs and goals of a user. In this sense, delivering information that is to the point has two main facets: Personalization allows users to obtain information that is adapted to their needs, goals, knowledge, interests • or other characteristics. User models deliver the main parameters for selecting and adapting information presentation to the individual user. Contextualization complements personalization so that environmental states or the context of use can also • be taken into account for adaptation. Furthermore the combination of approaches for user modeling and user model inference can deliver much more valid data about users when they are combined with contextual information. Personalization and contextualization can be seen as specialized forms of adaptation. At the core of adaptive systems are adaptive methods, which take an adaptation mean as a certain user characteristic or a part of the current user context and adapt different adaptation targets to this adaptation mean [Leutner, 1992]. From the background of user modeling and user adapted interaction different phases of adaptation are often differentiated ranging from user model acquisition, user model representation, information inference, adaptation, and evaluation of adaptation. In this chapter we will outline some works from the background of adaptive systems based on user modeling and show how this work is related to context-aware systems. As one result it can be seen that personalization and contextualization can be seen as complementary concepts, especially in newer approaches of ubiquitous computing for contextualized services [Zimmermann et al., 2005]. In the following we will also give some examples for the relation of personalization and contextualization and apply our analysis in the field of contextualized media for learning. Contextualized media for learning is a relatively new research area that combines the latest developments in ubiquitous and context aware computing with pedagogical approaches relevant to structure more situated and context aware learning support. Searching for different backgrounds of mobile and contextualized learning authors have identified the relations between existing educational paradigms and new classes of mobile applications for education [Naismith et al., 2004]. Furthermore best practices of mobile learning applications have been identified and discussed in focused workshops [Stone et al., 2002, Tatar et al., 2003]. We consider this field as an ideal evaluation area for the integration of personalization and contextualization as it also strongly integrates theoretical backgrounds about cognition in context and therefore emphasis the differences between personalization and contextualization and added values of the complementary use in educational artefacts.

5.2 Adaptation: Personalization and Contextualisation

In adaptive educational hypermedia a variety of research work about questions on how to adapt media to individuals and groups of learners has been done [Leutner, 1992, Brusilovsky, 1996, Weber and Specht, 1997, de Bra and Calvi, 1998, Specht, 1998]. From our point of view an adaptation process in contextualized learning applications needs to consider four key questions: 1. Why does the system adapt? 2. How does the system gather the information to adapt to? 3. What parts or components of the learning process are adapted? 4. What information does the system use for adaptation? Answering these four questions requires a certain amount of adaptation knowledge. The quality of the adaptation knowledge is a crucial factor for making the personalization and contextualization of applications a benefit rather than a hindrance [Bellotti and Edwards, 2001]. Basically, the idea of adaptation stems out of system dynamics in which self regulative systems have to adapt to changing conditions in an environment or internal and external feedback loops. The application of 5 Personalization and Contextualization of Media 57 adaptation in computer systems tries to mainly adapt the interaction and the facilities of a system to the changing needs of a user and the differences between users. So on the one hand changing parameters in an environment and associated conditions can trigger the execution of an adaptation. Many characteristics might be taken into account as catalysts for an adaptation process. They can be clustered into three main categories: Inter-individual differences address varieties among several users along manifold dimensions. Physi- • ological characteristics like disabilities are of major concern for application designers if they want to have their system accepted by a large community. The consideration of user preferences like language, colour schemes, modality of interaction, menu options or security properties, and numberless other personal favourite preferences is one of the most popular sources of adaptation and can be reused in different applications. An important variable for the content adaptation is the awareness of the user’s interests and disinterests. The user’s interest is a confounded variable that filters perception and the acceptance of adaptation. Additionally, a broad spectrum of personality characteristics like emotions, self-confidence, motivation, beliefs, which are difficult to assess automatically have an impact on the adaptation process and its evaluation. The same holds for the user’s level of factual, general and domain-specific knowledge (e.g. beginners or experts), which is a valuable source of adaptive operations [Specht and Oppermann, 1998]. Intra-individual differences consider the evolution and further development of a single user, as well as • the task over time. A static system falls short of changing user requirements as the user’s activities and goals evolve. In an extreme case users are overstrained by the system in the beginning and perceive the same system as cumbersome and restricted as the user’s expertise increases. In the same manner, the need for a higher flexibility of computer systems is pushed by the changing of the tasks to be accomplished with such a system. Environmental differences result from the mobility of computing devices, applications and people, which • leads to highly dynamic computing environments. Unlike desktop applications, which rely on a carefully configured and largely static set of resources, ubiquitous computing applications are subject to changes in available resources such as network connectivity and input and output devices. Moreover, they are frequently required to cooperate spontaneously and opportunistically with previously unknown software services in order to accomplish tasks on behalf of users. Thus, the environment surrounding an application and its user is a major source to justify adaptation operations.

Contextualized learning applications mainly focus on the pedagogical models behind the adaptation [Pask, 1964, Frank, 1965, Salomon, 1975]. Classical educational hypermedia system mainly adapted according for compensation of knowledge deficits, ergonomic reasons, or adaptations to learning styles for an easier introduction into a topic. Location-based services are an example for the type of adaptation mainly to environmental differences. In those applications the individual possibility for encoding and decoding information is one interesting aspect for better understanding artifacts in context. Furthermore the authentic collaboration on a topic is another example in principle all differences from inter-, intra-, and environmental differences play a role for best learning support. As for example co learners can be selected according to the learning task of an individual, his/her level of the domain, and the current location of persons or the availability of local communication facilities. Applications from the field of contextualised learning can make use of context in different ways and the role context plays in these applications is important. Four roles of context can be identified: Filter. The context serves as a basis of the preparation of a selection of choices. The usage of context for • decision support configures queries carried out on possibly large data and information sets and generates a specific subset from this set. Trigger. The context initiates an activity based on the occurrence of some significant event, i.e. changes in • the context. The usage of context as a trigger can cause various actions, depending on the application. The context as a trigger either activates or deactivates an action. Annotation. The context is used for the generation of metadata in order to tag this information to physical • or virtual objects. The captured context can be matched against a filter context in a later retrieval. Reflection. The context serves as a pure information source. This usage of context comprises the display • and recording of context information as well as the exploitation of context information for the parameterised execution of actions (e.g. reconfiguration). 58 Marcus Specht, Andreas Zimmermann and Tim de Jong

An obvious observation of these four roles of context clarifies that a single contextualised learning application can make use of context in more than one way. The context of such an application takes over several roles either in parallel, temporally interlinked or in a combination of both. Each role of the context might emphasise another part of the context description. Some contextualised learning applications allow the learner to take over control and play the role of the context. Then, the user pretends to be in a certain context [Brown, 1998] and filters, triggers, annotates or reflects through the manual specification of context information.

5.3 Contextualisation complements Personalisation

Inter- and intra-individual differences particularly refer to adaptation effects based on changes in characteristics of users. Jameson [2003] defines all systems that automatically perform an adaptation to the individual user in some nontrivial way as user-adaptive systems. User-adaptive systems emphasise dialog systems and demand for a detailed, explicit model of the user to adapt the behaviour of the system. User modelling components are constructed in such a way that they are tightly intertwined with the application. Prominent functions for user-adaptive systems are product recommendation, interface adaptation, learner support, or information retrieval [Kröner, 2001]. The e-commerce sector prefers the more popular term personalization focusing on systems that tailor products, services and information to better cater to each individual user. Their primary driver for using personalization is the belief that they can establish a stronger relationship with a customer through treating them individually (see Amazon3 or BroadVision4 as examples). If now environmental differences are integrated with inter- and intra-individual differences into a coherent whole, the notion of context becomes apparent. The notion of context varies across many different research areas. In general, context is understood as a knowledge type that facilitates a selective and focused processing of information. In context-aware computing a variety of notions and interpretations has developed over the years. Zimmermann et. al distinguish between definitions by synonym or definitions by example which mainly name and describe certain context parameters as location, identity, time, temperature, noise, as well as beliefs, desires, and commitments and intentions [Zimmermann et al., 2007]. Furthermore they introduce an operational definition of context describing following main categories of context information:

Individuality context includes information about objects and users in the real world as well as information • about groups and the attributes or properties the members have in common. Time context: from simple points in time to ranges, intervals and a complete history of entities. • Location contexts are divided into quantitative and qualitative location models, which allow to work with • absolute and relative positions. Activity context reﬂects the entities goals, tasks, and actions. • Relations context captures the relation an entity has established to other entities, and describes social, • functional, and compositional relationships.

5.4 Acquisition of Context Information

The realization of adaptation mechanisms in contextualized learning applications requires the coding of knowledge for observing the learners’ context. Traditionally and factually, the learners’ interaction with the application is the most important source of adaptivity. In here lies the biggest inter-individual and intra-individual varieties and in here lies the biggest need for adaptation during the usage of an application. Generally, over a certain time period the application needs to detect and record relevant indicators regarding context information, implicitly or explicitly [Jameson, 2003]. The knowledge of how to acquire context includes the selection of the appropriate acquisition method to collect information about learners to adapt to. These methods supply information that is provided explicitly by the user, implicitly through observations or from external sources:

3 http://www.amazon.com 4 http://www.broadvision.com 5 Personalization and Contextualization of Media 59

Questionnaires or forms. Questionnaires and forms present an effective means to capture information • explicitly provided by the learner. The learner fills in questionnaires and delivers a variety of valuable information to the application. This sort of input events oftentimes enables the application to learn about the learner’s properties, preferences, interests, task, etc. because they are explicitly specified by the learner. Tests. Like questionnaires and forms, tests are filled in by the learners in order to make information avail- • able to the contextualized learning application. In general, tests serve as means of the determination of the learner’s knowledge level, competence, and expertise. In addition, third party evaluation, assessments and results of tests (e.g. through tutors in learning systems) can be supplied to the application. Sensors. As additional sources of information and technical components, sensors measure physical or • chemical properties of the learners and their environments. Sensors quantify temperature, humidity, pressure, sound, lightness, magnetism, acceleration, force, and many more other properties of the environment. Furthermore, sensors measure the learner’s blood pressure, body temperature or eye movements as well as the location of the learner in- and outdoor. Queries. Furthermore, there exists a multiplicity of external sources like databases, pools or external appli- • cations that manage valuable information (e.g. learner profiles, weather forecasts). Information cannot be gained exclusively through direct interaction of the learner with the application or through observation of the learner. Additional queries to external information sources may augment the application with a variety of valuable information.

The careful selection of appropriate acquisition methods requires a lot of knowledge because one acquisition method might deliver valuable information for one domain, but might be useless in another. The determination of the learner’s knowledge exemplifies possible difficulties in the selection process: using a questionnaire the learner in fact specifies her knowledge explicitly. However, this input may be afflicted with several drawbacks because the learner might have limited self-assessment capabilities and may overes- timate her knowledge. On the other hand, if the system uses sensors for the observation of the learner, the determination of her knowledge might result from weak interpretation methods like “the longer the learner accesses a webpage the more knowledge she will gain”. Any implementation of the acquisition procedure will always be subjective and fail in covering the entire spectrum of learner personalities.

5.5 Targets of Adaptation

Targets of adaptation are parts of the application that is adapted by the adaptive method. An adaptation of these application parts may be immediately visible or noticeable to the learner or the adaptation effects may be hidden from the learner and display their impact at a later point in time. Contextualized learning applications consider ﬁve properties or parts of the system that can be tailored to the context and the learner: 1. Human-computer interaction, i.e. the modality needed to enter commands or data, and receive information and services. 2. Information presentation, i.e. the methods and coding required for receiving and displaying content (front-end). 3. Functionality, i.e. the features needed to perform tasks (back-end) and the ability of the application to solve a single task or a set of tasks. 4. Information and service selection, i.e. the information content, density and depth as well as the functionality and complexity of necessary services. 5. Knowledge base, i.e. the collection, organization, and retrieval of the knowledge about and the model of the learner, the context and the application. Examples for adaptation targets taken from the domain of contextualized learning application can be the pace of the instruction [Tennyson and Christensen, 1988, Leutner, 1992] that can be modiﬁed based on diagnostic modules embedded in the learning process or adaptation of content presentations, the sequencing of contents and others. Taking into account contextual information enables new interesting applications as the selection of information based on locations or even the streaming of media to a mobile user in changing contexts. 60 Marcus Specht, Andreas Zimmermann and Tim de Jong 5.6 The Adaptation Process in Contextualized Learning Applications

The way the system reacts or adapts is based on model assumptions on user needs, heuristics or ontological models of the application domain. In most adaptive educational hypermedia applications a learner model is the basis for the adaptation of the previously given parameters of the learning process. Nevertheless there are a several examples where the adaptation takes place not only to the learner knowledge, preferences, interests, cognitive capabilities, but also to tasks and learner goals. In contextualized learning the information used for adaptation is extended by the environmental parameters. The inference methods of the adaptive system can gain precision from the additional information of environmental sensors. The adaptation process controls the way how the application behavior is adapted according to changes in the context. Dieterich et al. [1993] assume four phases of one cycle of the adaptation process: Initiating the process cycle, proposal of alternatives, deciding on one alternative and the execution of an adaptation. In the particular case of contextualized learning applications, this four-phase process needs to be framed by two extensions: identiﬁcation of the need for adaptation and retaining the adaptation after it has been executed. In summary, the six phases of the adaptation process cycle can be described as follows:

Identify. Before an adaptation process cycle can be initiated, the need for an adaptation has to be identified • based on changes in the context. Initiate. If the need for adaptation is sufficient either the user or the system initiates the adaptation process • cycle. Propose. After the process cycle is initiated, one or more alternatives for the adaptation method need to be • proposed. Decide. During the decision phase one alternative of the set of adaptation methods proposed in the preced- • ing step is selected. Execute. Then, the selected adaptation method is performed using a set of adaptation operations that are • conducted by either the user or the system. Retain. After the execution of the selected adaptation method, the adaptation process cycle finishes with • the retaining of the adaptation for measuring the success of the process cycle or for a later inspection.

As basic knowledge-based systems, rule-based systems present a popular way to deterministically express adaptation knowledge because of their simple implementation. The interpretation of rules leads to decisions that determine the role of the context during the course of the adaptation process cycle. As a basic classiﬁcation schema the introduced questions can describe a variety of adaptation methods, an overview can be seen in Table 5.1.

5.7 Personalized and Contextualized Media Applied in Learning Support

Situated learning as introduced by Wenger and Lave [1991] states the importance of knowledge acquisition in a cultural context and the integration in a community of practice. Learning in this sense must not only be planned structured by a curriculum but also by the tasks and learning situations and the interaction with the social environment of the learner. This is often contrasted with the classroom-based learning where most knowledge is out of context and presented de-contextualized. On the one hand the process of contextualization and de- contextualization might be important for abstraction and generalization of knowledge on the other hand in the sense of cognitive apprenticeship [Collins et al., 1989] it is reasonable to guide the learner towards appropriate levels and context of knowledge coming from an authentic learning situation. From a constructivist point of view not only knowledge is always contextualized and but also the construction of knowledge, e.g. learning is always situated within its application and the community of practice [Mandl et al., 1995]. Stein defines four central elements of situated learning where the content emphasizes higher order thinking rather than acquisition of facts, the context for embedding the learning process in the social, psychological, and material environment in which the learner is situated, the community of practice that enables reflection and knowledge construction, and the participation in a process of reflecting, interpreting and nego- tiating meaning [Stein, 2003]. From the perspective of situated learning several requirements for new learning 5 Personalization and Contextualization of Media 61

Adaptive methods What is adapted? To which features? Why? Learning goal Learner Didactical reasons Content Preferences Preference model • Teaching method • Usage • Compensation of deficits • • Previous knowledge • Reduction of deficits Teaching style • Professional background • • Ergonomic reasons Media selection Knowledge • • Sequence Interests Efficiency • • Time constraints Goals • Effectiveness • • Help Task • Acceptance • • Complexity • Presentation • ... • Hiding • Dimming • Annotation • Extensions in contextualized learning Presentation Context sensors Situation 3D sound User location Authenticity of learning situa- • Augmented reality displays • Time • tions • Distribution to different con- • Light Situated collaboration • texts • Noise • Active construction of knowl- • Other user’s locations • edge •

Table 5.1. An overview for classiﬁcation of adaptive methods.

tools can be stated like: use authentic problems, allow multiple perspectives, enable learning with peers and social interaction within communities, enable active construction and reflection about knowledge. A shift towards a new tradition of online learning is described by Herrington et al. [2002]. Moreover the idea of situated learning is also closely related to the ideas of blended learning and learning on demand especially in educational systems for adults and at the workplace [Oppermann and Specht, 2006]. An important point that is not taken into account by a lot of new approaches for delivering personalized learning on demand is the aspect that the need (demand) for knowledge and learning arises in a working context with the motivation for solving specific problems or understanding problem situations. This notion of personalized learning on demand in the workplace exemplifies the potential of contextualized learning in the workplace. Learners that identify a problem in a certain working situation are highly motivated for learning and acquiring knowledge for problem solving. They have a complex problem situation as a demand, which can be used for delivering learning content adapted to their situation. Furthermore not only the delivery of content into a certain context or practice is needed but also interaction facilities must be provided which allow an appropriate interaction and cooperation with educational systems. The contextualization of the learning on demand can not only be seen from the point of view of an actual problem or learning situation but also in a longer lasting process of learning activities that are integrated. Different learning activities are combined in blended learning approaches where the preparation for a task, updates on base knowledge, then the application in an actual working situation and the documentation of problem solutions and the reflection about one’s activities evaluates that process. In the literature for mobile and contextualized learning support different examples can be found for utilizing those context parameters to enable learning. Several projects have looked at how to use and enrich contents with contextual metadata [EQUATOR Project, 2003, Specht and Kravcik, 2006]. Mostly interesting new approaches 62 Marcus Specht, Andreas Zimmermann and Tim de Jong in context-aware systems see the main strength in combining different context parameters for user support. Even more new approaches tend to combine different forms of metadata about learning objects and media to allow for flexible and contextualized learning support. In the MACE project the combination of various types of content, usage, social and contextual metadata enables users to develop multiple perspectives and navigation paths that effectively lead to experience multiplication for the learner himself [Stefaner et al., 2007]. Identity context is also often combined with other forms of context. One specific example of such a combination is given by Ogata and Yano [2004b] who present CLUE, a system for learning English in real-world situations. CLUE uses (i) a learner profile to adapt the learning content to the learner’s interest and (ii) location information to link objects/locations to suitable English expressions, i.e. appropriate learning content. Likewise, Bo [2002] combines a user profile and user position, to facilitate personalised and location-based information delivery. AwarePhone [Bardram and Hansen, 2004] uses several context-parameters at the same time. First of all, location is used to locate fellow employees within the hospital. Second, a calendar artefact is used to capture and share time context and also indicate the activity of a user at a certain moment. The activity is furthermore given by a shared status message. The combination of these three context parameters lead to what the writers call “context-mediated social awareness”. Environmental context information is used in several systems, most notably QueryLens [Konomi, 2002] which focuses on information sharing using smart objects. Moreover, the TANGO system presented by Ogata and Yano [2004a] and the Musex system [Yatani et al., 2004] detect objects in the vicinity by using RFID tags. Moop [Mattila and Fordel, 2005] couples a GPS location to observations/information gathered in the field for later analysis in the classroom. Wallop [Farnham et al., 2004] allows its users to discover social relationships and provides social awareness by storing and analysing social context information; to derive the social context communication patterns system interactions and co-occurrence information were analysed.

5.8 Conclusion

The chapter has illustrated the common principles of personalized and contextualized media with the concept of adaptive methods and their parameters. As adaptation mean a system can focus on intra-individual, inter- individual differences or environmental changes and differences. It has been shown that both the personalization of systems to inter- and intra-individual differences as also the contextualization to environmental changes can be used in a complementary way to create new forms of adaptation with effective learning support. In the adaptation process personalization and contextualization differ mostly in the form triggers for adaptive methods are used. In user-adaptive systems mostly users take decisions or even sometimes identify the adaptive method to be instantiated or applied. In context adaptive systems the available information resources for adaptation become highly complex and that leads on the one hand to a need for more transparent user interfaces for contextualization interfaces as also the need for more semiautomatic processes of information aggregation and validation for adaptive contextualized support. In that sense it also becomes important that contextual information can not only be used for filtering and triggering adaptation but can play a role for annotation and reflection for supporting learning in the context of use. In the last part several examples from the field of mobile and ubiquitous learning support have been presented that use context parameters for a more effective learning and build on personalized learning support.

References

J. R. Anderson, F. G. Conrad, and A. T. Corbett. Skill acquisition and the LISP tutor. Cognitive Science, 13: 467–505, 1989. J. E. Bardram and T. R. Hansen. The AWARE architecture: Supporting context-mediated social awareness in mobile cooperation. In ACM Conference on Computer Supported Cooperative Work, Chicago, Illinois (USA), 2004. V. Bellotti and W. K. Edwards. Intelligibility and accountability: Human considerations in context aware systems. Human Computer Interaction, 16(2-4):193–212, 2001. 5 Personalization and Contextualization of Media 63

G. Bo. MOBILEARN: Project final report. http://www.mobilearn.org/download/results/ /pub- lic_deliverables/MOBIlearn_Final%20Report.zip, 2002. P. J. Brown. Triggering information by context. Personal Technologies, 2(1):1–9, 1998. P. Brusilovsky. Methods and techniques of adaptive hypermedia. User Modeling and User-Adapted Interaction, 6(2-3):87–129, 1996. P. Brusilovsky, M. Specht, and G. Weber. Towards adaptive learning environments. In Herausforderungen eines globalen Informationsverbundes für die Informatik (GISI), Zürich, Switzerland, 1995. J. R. Carbonell. AI in CAI: An artificial intelligence approach to computer-aided instruction. IEEE Transactions on Man-Machine Systems, 11(4):190–202, 1970. J. M. Carroll and C. Carrithers. Training wheels in a user interface. Communications of the ACM, 27(8): 800–806, 1984. W. J. Clancey. Knowledge based tutoring: The GUIDON program. MIT Press, 1987. A. Collins, J. S. Brown, and S. E. Newman. Knowing, Learning and Instruction, chapter Cognitive Appren- ticeship: Teaching the Craft of Reading, Writing, and Mathematics. Lawrence Erlbaum Associates, 1989. P. de Bra and L. Calvi. The New Review of Hypermedia and Multimedia, volume 1, chapter AHA! An open Adaptive Hypermedia Architecture. Taylor Graham Publishers, 1998. A. K. Dey and G. D. Abowd, editors. Towards a Better Understanding of Context and Context-Awareness, Karlsruhe, Germany, 1999. H. Dieterich, U. Malinowski, T. Kühme, and M. Schneider-Hufschmidt. Adaptive User Interfaces, chapter State of the Art in Adaptive User Interfaces. North-Holland, 1993. EQUATOR Project. The EQUATOR interdisciplinary research collaboration. http://www.equator.ac.uk/, 2003. S. Farnham, S. U. Kelly, W. Portnoy, and J. L. K. Schwartz. Wallop: designing social software for co-located social networks. In 37th Annual Hawaii International Conference on System Sciences, Big Island, Hawaii (USA), 2004. S. K. Feiner and K. R. McKeown. Intelligent Multimedia Interfaces, chapter Automating the Generation of Coordinated Multimedia Explanations. MIT Press, 1995. H. Frank. Lehrautomaten für die einzel- und gruppenschulung. In Lehrmaschinen in kybernetischer und pädagogischer, Sicht, Germany, 1965. T. Gross and M. Specht. Awareness in context-aware information systems. In Mensch und Computer, Bad Honnef, Germany, 2001. J. Herrington, R. Olivier, T. Herrington, and H. Sparrow. Towards a new tradition of online instruction: Using situated learning theory to design web-based units. In 17th Annual ASCILITE Conference, Auckland, New Zealand, 2002. A. Jameson. Modelling both the context and the user. Personal Technologies, 5(1):29–33, 2001. A. Jameson. Systems that adapt to their users: An integrative overview. In 9th International Conference on User Modelling, Johnstown, Philadelphia (USA), 2003. W. Kim. Personalization: Definition, status, and challenges ahead. Journal of Object Technology, 1(1):29–40, 2002. A. Kobsa. Generic user modeling systems. User Modeling and User-Adapted Interaction, 11(1):49–63, 1996. A. Kobsa and W. Wahlster. User Models in Dialog Systems. Springer Verlag, 1989. S. I. Konomi. QueryLens: Beyond ID-based information access. In 4th International Conference on Ubiquitous Computing, Göteborg, Sweden, 2002. A. Kröner. Adaptive Layout of Dynamic Web Pages. Akademische Verlagsgesellschaft Aka GmbH, 2001. D. Leutner. Adaptive Lehrsysteme; Instruktionspsychologische Grundlagen und experimentelle Analysen. Beltz, 1992. H. Mandl, H. Gruber, and A. Renkl. Information und Lernen mit Multimedia, chapter Situiertes Lernen in multimedialen Lernumgebungen. Psychologie Verlags Union, 1995. P. Mattila and T. Fordel. MOOP - using m-learning environment in primary schools. In 4th World Conference on mLearning, Cape Town, South Africa, 2005. L. Naismith, P. Lonsdale, G. Vavoula, and M. Sharples. Literature review in mobile technologies and learning. Technical Report NESTA Futurelab Series, report 11, University of Birmingham, 2004. 64 Marcus Specht, Andreas Zimmermann and Tim de Jong

H. Ogata and Y. Yano. Context-aware support for computer-supported ubiquitous learning. In 2nd IEEE International Workshop on Wireless and Mobile Technologies in Education, Taoyuan, Taiwan, 2004a. H. Ogata and Y. Yano. Knowledge awareness map for computer-supported ubiquitous language-learning. In 2nd IEEE International Workshop on Wireless and Mobile Technologies in Education, Taoyuan, Taiwan, 2004b. R. Oppermann and M. Specht. A context-sensitive nomadic exhibition guide. In 2nd International Symposium on Handheld and Ubiquituous Computing, Bristol, United Kingdom, 2000. R. Oppermann and M. Specht. Engaged Learning with Emerging Technologies, chapter Situated Learning in the Process of Work. Springer Verlag, 2006. G. Pask. Teaching machines and programming, chapter The adaptive teaching system. Pergamon Press, 1964. J. Rickel and W. L. Johnson. Integrating pedagogical capabilities in a virtual environment agent. In 1st Inter- national Conference on Autonomous Agents, Marina del Rey, California (USA), 1997. G. Salomon. Adaptiver Unterricht, chapter Heuristische Modelle für die Gewinnung von Interaktionshypothe- sen. Kösel, 1975. B. N. Schilit, N. I. Adams, and R. Want. Context-aware computing applications. In Workshop on Mobile Computing Systems and Applications, Santa Cruz, California (USA), 1994. M. Specht. Adaptive Methoden in computerbasierten Lehr/Lernsystemen. University of Trier, 1998. M. Specht and M. Kravcik. Authoring of learning objects in context. International Journal on E-Learning, 5 (1):25–33, 2006. M. Specht and R. Oppermann. The New Review of Hypermedia and Multimedia, volume 4, chapter ACE, Adaptive courseware environment. Taylor Graham Publishers, 1998. M. Stefaner, E. Dalla Vecchia, M. Condotta, M. Wolpers, M. Specht, M. Apelt, and E. Duval. MACE - enriching architectural learning objects for experience multiplication. Lecture Notes in Computer Science, 4753:322– 336, 2007. D. Stein. Situated learning in adult education. http://ericacve.org/docs/situated195.htm, 2003. A. Stone, G. Alsop, J. Briggs, and C. Tompsett. M-learning and e-learning: a review of work undertaken by the learning technology research group. In European Workshop on Mobile and Contextual Learning, London, United Kingdom, 2002. D. Tatar, D. Roschelle, P. Vahey, and W. R. Penuel. Handhelds go to school: Lessons learned. Computer, 36 (9):30–37, 2003. R. D. Tennyson and D. L. Christensen. Instructional Designs for microcomputer courseware, chapter MAIS: An intelligent learning System. Lawrence Erlbaum Associates, 1988. G. Weber and M. Specht. User modeling and adaptive navigation support in WWW-based tutoring systems. In 6th International Conference on User Modeling, Chia Laguna, Italy, 1997. E. Wenger and J. Lave. Situated Learning: Legitimate peripheral participation. Cambridge University Press, 1991. K. Yatani, M. Sugimoto, and F. Kusunoki. Musex: a system for supporting children’s collaborative learning in a museum with PDAs. In 2nd IEEE International Workshop on Wireless and Mobile Technologies in Education, Taoyuan, Taiwan, 2004. A. Zimmermann, A. Lorenz, and M. Specht. Personalization and context-management. User Modeling and User-Adapted Interaction, 15(3-4):275–302, 2005. A. Zimmermann, A. Lorenz, and R. Oppermann. An operational deﬁnition of context. In 6th International and Interdisciplinary Conference on Modeling and Using Context, Roskilde, Denmark, 2007. 6 Contextualized Recommender Systems: Data Model and Recommendation Process

Wolfgang Woerndl and Johann Schlichter

Technische Universität München Boltzmannstr. 3, 85748 Garching b. München, Germany {woerndl,schlicht}@in.tum.de

Summary. Recommender systems ﬁlter information items according to users’ interests and preferences. Sys- tems thereby utilize explicit ratings of items or implicitly observed user behaviour. In addition, it is important to tailor information access with regard to users’ current context, especially in mobile and ubiquitous application domains. Although the integration of context into recommender systems has not been investigated thoroughly so far, several approaches and applications do exist. We give a comprehensive overview about the state-of-the-art of contextualized recommender systems in this chapter. First we explain how context can be modeled, generated and managed. Then, we discuss how to introduce context into the data model of recommender systems. We argue to consider context as an additional dimension to the traditional user-item ratings matrix of collaborative ﬁltering. Context attributes can be utilized in different ways in various recommendation processes. We provide an overview of principal approaches that exist in the research literature. The discussion is illustrated by several application examples. We conclude with a summary and an outlook about future research prospects.

6.1 Introduction

The sheer volume of data that exists in our networked society today makes it difficult for users to find and access relevant information. Personalization of content is a technique to reduce the omnipresent information overload. Thereby, information is customized according to users’ needs and preferences. Often, recommender systems using collaborative or content-based filtering are applied. In a mobile or ubiquitous scenario, information personalization is even more important, because of the limitations of mobile devices regarding displays, input capabilities, bandwidth etc. For example, users cannot browse through many Web search results on the small screen of a mobile device in contrast to working on a desktop PC. It would be more desirable to pre-filter results to adapt to the user’s current needs, if possible. Another promising and possibly complementary approach is to utilize context. For instance, a user travelling with a PDA or a smartphone needs access to current train schedules, the weather report at her destination or a recommendation for a restaurant in her vicinity, that is currently open and caters to the users preferences [Schwinger et al., 2005]. Hence while on the one hand personalized and context-adapted information access is important in mobile settings or scenarios with non-standard user interfaces, on the other hand obtaining relevant information is even more difficult. A lot of work has been done with regard to context-aware applications and personalization research in general. For example, Baldauf et al. [2007] present a recent overview on context-aware systems, however without mentioning context-aware recommender systems. Adomavicius and Tuzhilin [2005] outline the state- of-the-art of recommender systems and also dedicate one chapter to context-awareness where they mainly explain their multi-dimensional recommender approach [Adomavicius et al., 2005]. Other papers consider some kind of context attributes as part of a user and/or item model, but these systems cannot easily be applied to other application areas and it is difficult to derive general principles for contextualized recommender systems. The integration of context into recommender systems has not been investigated thoroughly up to now [Anand and 66 Wolfgang Woerndl and Johann Schlichter

Mobasher, 2005], but several approaches and applications do exist. The goal of this chapter is to present an overview over concepts and principles of contextualized recommender systems. Firstly, we explain how context can be modelled and acquired (Section 6.2). Then, we discuss in Section 6.3 how to introduce context into the data model of recommender systems. We argue to consider context as an additional dimension to the traditional user-item ratings matrix of collaborative filtering. Context attributes can be utilized in different ways in various recommendation processes including collaborative filtering, context- based filtering and hybrid recommenders (Section 6.4). We provide an overview of some principal approaches we have developed in our group [Woerndl and Schlichter, 2007] and also ideas available in research literature. The discussion is illustrated by application examples where possible. We finally conclude with a brief summary in Section 6.5.

6.2 Context: Deﬁnition, Model and Acquisition

In a recent survey on web personalization, Anand and Mobasher [2005] list context-awareness as an issue that has to be addressed appropriately if personalization systems are to find wider acceptance. They state that “context as a concept has rarely been incorporated into personalization research”. Furthermore, they identify that “one of the reasons for this is that it is hard to arrive at a consensus of what defines context let alone modelling the concept”. Therefore, we will first define what we mean by context, and also present how (high level) context can be modelled and acquired for example in mobile scenarios we are working on.

6.2.1 Context Deﬁnition and Modelling

In our work we follow the context definition from Dey et al. [2001]: “Context is any information that can be used to characterize the situation of entities (i.e. whether a person, place or subject) that are considered relevant to the interaction between a user and an application, including the user and the application themselves”. That means, context is very dynamic and transient. For example in a mobile application domain, a context model could include location, movement, lighting condition or current availability of network bandwidth of mobile devices. As far as context modelling is concerned, several approaches for representing context information have been developed [Strang and Linnhoff-Popien, 2004, Baldauf et al., 2007]. Key-value pairs can be used as a simple data structure. Markup scheme models consist of hierarchically organized markup tags with attributes and content. An example is W3C’s standard Composite Capabilities/Preference Profile (CC/PP) [Klyne et al., 2004] for representing device capabilities. Object oriented or graphical models, e.g. using the Unified Mod- elling Language (UML), provide more powerful means to represent context types and describing attributes. In addition, ontologies are a very promising instrument for modelling contextual information due to their high and formal expressiveness and the possibilities for applying ontology reasoning techniques. Ontology-based models can thereby represent complex relationships between (context) concepts.

6.2.2 Acquisition of High Level Context from Sensor Data

Most of the context is available as low level sensor data that is unsuitable to be utilized in recommender or personalization systems. These applications need higher level context. For example, instead of exploiting GPS coordinates directly, it is more desirable to be able to use names of places or nearby landmarks. Therefore, we explain below how to acquire high level context from sensor data.

Process Model

Most approaches in the literature are based on a similar process for the generation of high level context, even if they specialize in different ways. For example, [Himberg et al., 2003] show that data mining techniques are 6 Contextualized Recommender Systems: Data Model and Recommendation Process 67 suitable for identifying high level context, and [Korpipää et al., 2003] use Naïve Bayes networks to classify context. The following process model consists of the following four tasks and is similar to what is used in other approaches [Woerndl et al., 2007a]:

1. Acquisition of raw data. 2. Preprocessing: aggregation and feature extraction. 3. Interpreting the data. 4. Utilization in applications.

Note that this is not a one-time process but an iterative activity to generate context. The first step is to acquire raw data from sensors, e.g. for location, acceleration, temperature, sound/voice, etc. For each sensor, a different software component has to be developed. Thereby, it is important to consider measurement ranges and error rates to get useful results. Thus, the acquisition of raw data allows for a perception of the physical environment and establishes the foundation for the other tasks. The second step is preprocessing of the (raw) data. This step varies significantly in the existing literature, but some kind of preprocessing is done in about every approach. One reason is that the sheer amount of data usually is huge after acquiring raw data, and it cannot be handled directly, especially in our domain of mobile devices. Therefore, aggregation and feature extraction has to be applied. One example is to use sliding windows to extract features from sensor data streams. Thereby, an algorithm examines only a subset of the whole data set at one time, and calculates characteristics such as mean values or standard deviation. The extraction window is then moved along the data stream. The size and overlap of the windows have an influence on the necessary memory and processing requirements, which is especially an issue for mobile devices. The third step in the process model is interpretation which realizes the transition from lower-level data to context information that is useful in applications. The interpretation may take several steps, dependent on the used methods, e.g. supervised or unsupervised learning algorithms. After interpreting the data, the fourth step is to utilize the generated high level context in applications, or to visualize results for users.

Implementation Example

We have designed and implemented software components that realize the model explained above [Woerndl et al., 2007a]. The architecture consists of a client-side part, running on a PDA or smartphone. After acquisition, the data is transferred to a server-based application for the preprocessing and interpretation. Users can control the context acquisition via an interface on the client. The client also allows for entering annotations for a supervised training algorithm. We have tested the feasibility of our approach by experimenting with several sensors, including an acceleration sensor to infer the activity a user is doing, for example “running” or “sitting”. Figure 6.1 shows the visualization of the acceleration sensor information using real world data. The aggregated values of the three axis of the acceleration sensor are plotted at the top, and the inferred and actual user activity are displayed at the bottom of the screen. We have not done a formal study regarding the accuracy of the results yet, but our tests indicate that the algorithm is performing reasonably well and can be used to develop an application which monitors, stores and analyzes user activities. Some problems emerged when activities are somewhat related. For example, it is difﬁcult to distinguish between “sitting” and “sitting in a car”. Therefore at timestamp 950, the algorithm ﬁrst inferred that the user is driving, even though that was not the case.

6.3 Context in Recommender Systems Data Model

After explaining what we mean by context, and how (high level) context can be acquired, we now focus on the integration of context into recommender systems. The discussion is split into ﬁrst the data model of recommender systems (Section 6.3) and then the various recommender system processes (Section 6.4). 68 Wolfgang Woerndl and Johann Schlichter

Fig. 6.1. Inferring user activity from an acceleration sensor.

6.3.1 Traditional Data Model

A recommender system tries to predict the relevance of information items for a user. This is traditionally based on information about the user, meta data associated with items and/or implicit or explicit ratings for items made by a group of users. The standard data model can be summarized as follows [Anand and Mobasher, 2005]. Given is a set of n items I = ij : 1 j n, and a set of m users U = uk : 1 k m. Each user is described by a t-dimensional vector of attrib≤ utes,≤ which is also called (user) profile.≤So the≤ profile contains attribute-value pairs such as “language = EN”. The items have associated meta data, in form of a s-dimensional vector (item description). For example, an item may be a particular restaurant (in a restaurant guide application), and the item description may contain “location = (longitude, latitude)”, and also the price range and opening hours of a restaurant to be utilized in a tourist guide. Thus items and users are technically modelled in the same way with respect to data structures. In addition, often users’ ratings of items are used. Technically, the rating is a function r : U I S that maps a user instance (and therefore the values of the user profile attribute values) and an item×instance−→ (or the values of the corresponding item meta data attribute values) to some appropriate mapping result-space S, e.g. a rating value in [0, 1]. In other words, a rating ri,s expresses how much a user uj likes an item is. Ratings can be obtained using either implicit or explicit user profiling methods. An example for explicit ratings would be filling out a Web form by a user to express which type of restaurants she liked. Implicit ratings can be acquired by observing the user, for example the consideration of eating at a particular place results in a positive rating, without any additional user input. The goal of the personalization process is to recommend items ij to an active user ua that would be of interest to the user. How this is done varies among the different recommendation techniques and is covered in Section 6.4 of this chapter. Since recommenders usually operate on the user-item matrix with ratings as elements, one major problem is the sparseness in this matrix, e.g. there are often large numbers of users and items but only few ratings that can be exploited. Another drawback is the cold start problem because recommendations for new users or items are hard to generate, depending on the used technique. 6 Contextualized Recommender Systems: Data Model and Recommendation Process 69

6.3.2 Integrating Context

The question discussed in this section is how to incorporate context information into the data model of recommender systems.

Context Versus User Proﬁle

A first idea would be to incorporate context information as part of the user profile. However, our assumption is that it is important to distinguish between user profile and context attributes and treat them as separate entities in the recommender data model. While the user profile is rather static and somewhat longer lasting, context is highly dynamic and transient. Depending on the application area, user models can also be highly dynamic, but from our point of view a user profile contains information such as preferences, interests or knowledge (e.g. the user’ss favourite type of restaurant), whereas the context model describes the current environment in which the user operates in (e.g. her current location). Profile information can be implicitly observed or explicitly provided by the user. The profile is stored permanently on a profile/identity server and can be manually modified by the user if needed, for example if user preferences or interests change. On the other hand, context information is neither stored permanently nor manually entered by the user. In our model, the context is captured and analyzed when a user wants to generate a recommendation or implicitly or explicitly submits a rating for an item. In the latter case, the captured context model is stored together with the rating to be used later. However, it is possible to transfer (inferred, high level) context information to the profile. For instance in the acceleration sensor example from Section 6.2.2, the current activity information such as “sitting in a car” at one particular moment is context, while derived information such as “the user drives from A to B every working day” could be part of a user profile.

User-Item-Context Matrix

As far as the context model is concerned (Section 6.2.1), most available contextualized recommender systems only use a rather simple model so far, such as attribute value pairs. More formally, the context C can be described as a vector of context attributes, analogous to the user proﬁle and the item meta data [Chen, 2005]. The similarity or identity of context values is not as clear as with users or items and it may depend on the actual application domain. For example, for a tourist guide the same context might be constituted by a range of GPS coordinates. The integration of context extends the domain of the rating function to U I C. In other words, context adds another dimension to the item-user matrix (Figure 6.2), as introduced by×Adoma× vicius et al. [2005]. This is also of importance with regard to the sparseness of ratings, because R may only be deﬁned for a small part of possible contexts, e.g. ratings may be valid in one particular context only. Which context attributes are actually modelled and used in systems is largely dependent on the requirements of the application domains.

Spreading Activation Framework

As an alternative to the explained user-item-context matrix, Kovács and Ueno [2006] present CASAN nets, which are an extension of the classical associative spreading activation network formalism. In CASAN nets, content nodes are not just connected by plain directed weighted links but also have associated link type and context nodes. Their spreading activation algorithm incorporates mechanisms to handle these two new functions of nodes. However, it is not clear how these CASAN nets can really be exploited in a practical recommendation process.

6.4 Context in Recommender Systems Process

After explaining alternatives for integrating context into the data model of recommender systems, we will now cover how recommender system processes can be improved by utilizing context. The discussion is structured 70 Wolfgang Woerndl and Johann Schlichter

Fig. 6.2. The user-item-context matrix. according to the most important types of recommenders: collaborative ﬁltering (Section 6.4.1), context-based recommender (Section 6.4.2) and hybrid approaches (Section 6.4.3).

6.4.1 Collaborative Filtering

In comparison to non-collaborative, individual recommender techniques, traditional collaborative ﬁltering (CF) does take other users into account and tries to automate the process of word of mouth. This is usually done by recommending items to an active user that have been rated highly by users who are similar to the active user. Two basic steps are needed to generate recommendations: neighbourhood creation and ratings prediction. Neighbourhood creation means selecting users that are similar to the active user. In most approaches, similar users are users that have rated items in a similar way as the active user in the past. This user-user similarity usually is calculated by using Pearson correlation, cosine-based approach or other likewise metrics [Adomavi- cius and Tuzhilin, 2005]. Subsequently, a set S of n users who are most similar to the active user is computed. For predicting the ratings of some items for the active user, the mean average of ratings given by the users in S for an item is determined. This average can be weighted according to the actual user similarity of a user to the active user and also adjusted if some users are always giving higher-than-average (or lower) ratings. Finally, a set of items with the highest predicted ratings is presented to the active user as recommendations.

Contextualized Neighbourhood Creation and Ratings Prediction

When introducing context into CF, the main idea is to weigh ratings in both algorithms according to context similarity. For the user-user metrics the deviation of ratings of users in comparison to the ratings of the active users is computed. Thereby no additional information has an inﬂuence while considering the context of ratings appears to be an important factor. For example, if one user A rated restaurants when having lunch, and another user B rated the restaurant when dining in the evening, the current time noon or evening when the active user asks for a recommendation may play a signiﬁcant role when determining the neighbourhood of similar users. Context can be incorporated by weighing the ratings according to context similarity with the current context. Therefore a contextualized CF algorithm should store a snapshot of the current context with every rating made. As for the ratings prediction, the option is also to apply a higher weight to ratings that were made in a similar context in comparison to the current context of the active user, analog to applying weights to user similarity. For instance, we assume A, B as the set of users who are similar to the active user. A user A rated an item { } as 0.2 on a scale from 0 to 1 in a context c1, the other user gave a rating 0.8 in context c2. A contextualized 6 Contextualized Recommender Systems: Data Model and Recommendation Process 71

CF algorithm would compare the current context of the active user to c1 and c2 and then compute the predicted rating according to the context similarity, e.g. 0.7 if the current context of the actice user is similar to c2. Chen presents a context-aware collaborative filtering system based on past experiences of users [Chen, 2005]. The scenario is ubiquitous computing to assist users in different situations. To incorporate context, the approach weighs the current context of the active user against the context of each rating with respect to their relevance to locate ratings given in a similar context. Thereby, Chen uses a modified Pearson correlation coefficient to measure the correlation between context variables such as location, temperature and time [Chen, 2005]. One major problem of this approach is the availability of ratings in comparable contexts. Collaborative filtering needs a sufficient number of ratings to produce satisfying results. This sparseness problem is an issue with CF in general, but this problem is aggravated when integrating context information. In addition, contextualized CF needs a context similarity function as we outlined above, which may be difficult to formalize. One possible solution is to move from simple context models to context ontologies (Section 6.2.1). Thereby, the relationship between objects in the context model can be utilized to improve recommender systems. For example in a mobile recommender: if there are not enough ratings for a particular location, e.g. a street, the ontological information that “street” is part of “town” could be used to increase the available of relevant ratings in a certain context. Another general solution to cope with the disadvantages including the new item and new user problems of CF is to combine CF with other recommendation techniques to form a hybrid recommender. Contextualized hybrid recommenders are covered in Section 6.4.3. The approach by Herlocker and Konstan [2001] was one of the first examples of a contextualized recommender system. They argue that adding content-independent context can improve collaborative filtering. The context is knowledge about a user’s task which consists of a list of example items. Therefore, this approach is still two-dimensional, i.e. it uses only user and item dimensions, and does not explicitly model any additional contextual information.

Social Context

Context does not always have to be physical context such as location and environment conditions. Equally important is the social context [Woerndl and Groh, 2007], e.g. the social network of a user, buddy lists, past interactions with other users etc. It seems obvious that social context is important for collaborative filtering, because it does resemble to the word of mouth principle of CF. For example in social groups, group members and opinion leaders have great impact on the advice seeking and advice evaluation process of other group members in domains related to users’ tastes where personal opinions figure to play a major role. To use social models in recommender system, Groh and Ehmig [2007] replace the neighbourhood creation based on ratings similarity in collaborative filtering by a social neighbourhood of users, e.g. all friends or all friends plus friends of friends. They demonstrate that their social recommender outperforms traditional CF in their scenario of recommending clubs. In this approach, ratings are not used at all for the neighbourhood creation. Another option is to weigh the ratings with a social context similarity score which should be computed at the time a rating is made, to account for changing social environments of users.

6.4.2 Content-Based Recommender Systems

In content-based recommender systems the approach is to generate recommendations based on the analysis of items previously rated by the active user and generating a user profile based on the meta data of these items. The profile is then used to predict a rating for previously unseen items [Anand and Mobasher, 2005]. In other words, content-based techniques are individual and do not take other users into consideration. Only one row of the user-item matrix is used, the ratings of the active user. Matching between items can be done in different ways, for example by performing text analysis to find similar items or by identifying and applying rules. In general, we need a function F that maps (U I) I, i.e. the user-item matrix with a user’s past ratings and a particular item, to a result space S, e.g. [0, 1].× For×example, if user A liked Chinese restaurants in the past, F would produce a high value for new restaurants that have “Chinese” in their item description. Moving to a contextual recommender, this function has to consider the context, i.e. F : (U I) I C S. For × × × −→ 72 Wolfgang Woerndl and Johann Schlichter example, if user A previously liked Chinese restaurants only for dinner, recommendations may change when she is looking for a quick lunch. There are different approaches to actually realize a contextualized content-based recommender. The most prominent example for a contextual content-based recommender system is the multi-dimensional approach by Adomavicius et al. (see below). Other options include rule-based systems (see below) or statistics-based approaches. An example for the latter is the mobile opportunistic planning study by Horvitz et al. [2007].

Multi-Dimensional Recommender Systems

Adomavicius et al. [2005] enhance the user-item data model to a multidimensional setting where the additional dimensions constitute different context attributes such as location and time. In general, the recommendation process can then be generalized to a multidimensional utility function u : D1 D2 . . . Dn R. Then, the recommendation problem is deﬁned by selecting certain “what” dimensions× and× certain× “for −whom→ ” dimensions, and recommending the “what” tuples for each “for whom” tuple that maximizes the utility function u [Adomavicius and Tuzhilin, 2005]. Adomavicius et al. also propose a reduction-based approach with the goal to reduce the dimensions of the utility function. For example, if a person wants to see a movie on a Saturday night, the approach would only consider ratings of movies seen on weekends and thereby eliminate the “time of the week” dimension. Thereby, the n-dimensional cube of ratings is projected to the user-item matrix and any standard algorithm including CF can be applied [Adomavicius and Tuzhilin, 2005].

Point-of-Interest Trigger

The PoiAppRecommender is an example for a content-based context-aware recommender using rules or triggers that we have designed and implemented. This recommender is integrated in a play.tools framework which supports the development of innovative, mobile applications for devices such as smartphones [Woerndl et al., 2007b]. The realized acquisition of high level context from sensors is also integrated in this framework and can be used for personalized applications. Part of the framework is a deployment server where developers of mobile applications can register their services and users can browse and search for relevant and interesting gadgets. Users can access the deployment server and download client modules on their mobile devices. One problem for users is to ﬁnd interesting and with regard to their current context relevant applications. The PoiAppRecommender does not recommend points-of-interest (POIs) but recommends mobile applications based on POIs in the vicinity of the user using triggers. A software developer or administrator can select among types of points-of-interest (such as restaurant, museum or train station) and specify within which radius of an actual POI an application is recommended. This is done when registering the application with the mentioned deployment server. When making a recommendation, the system then retrieves the current user position (using a GPS-enabled mobile device), determines POIs in the vicinity and generates a recommendation based on this context information. For example, an administrator can specify that her mobile train table application shall be recommended when the user is near a train station. After applying the trigger rules, our approach uses collaborative ﬁltering to rank found items according to user ratings of applications in a second step. User ratings are collected implicitly by automatically recording when a user installs an application within our framework on her mobile device. It is also optionally possible for users to explicitly rate applications after usage. The ratings are stored together with context information (time, location, used device, ...) to capture the situation when a rating was made as motivated above.

6.4.3 Hybrid Recommender

Hybrid recommender systems combine different approaches with the goal of utilizing the strengths of one algorithm while avoiding its weaknesses by applying a second approach [Burke, 2002]. Combination strategies are: weighted (the scores of several recommendation techniques are combined), cascading (one recommender 6 Contextualized Recommender Systems: Data Model and Recommendation Process 73 reﬁnes the recommendations given by another), switching (switching between appropriate recommendation techniques), mixed (system presents results some several recommenders at the same time), feature combination (features from different recommendation data sources are thrown together into a single recommendation algorithm) or feature augmentation (the output from one technique is used as an input feature to another).

Contextualized Hybrid Recommender Systems

When integrating context, one option is to combine several algorithms to one hybrid context-aware recommender system. In principle, all of the above combination strategies could be used, but it is particularly promising to use a hybrid recommender to reduce the mentioned complexity of the user-item-context matrix (Sec- tion 6.3.2). Thereby, a ﬁrst algorithm RS 1 would operate on two dimensions and compute an intermediary result set of items by taking only user and context attributes into account. In a second step, RS 2 further ﬁlters and ranks the results by considering the third dimension, for example by utilizing item meta data. Thus, the used combination strategy is cascading or feature augmentation. Figure 6.2 illustrates this principle.

Fig. 6.3. Hybrid contextual recommender.

Recommending Mobile Applications

This principle of combining several recommenders into one context-sensitive system has been realized in the already mentioned play.tools framework for the development of mobile applications (Section 6.4.2) [Woerndl et al., 2007b]. First of all, a switching strategy is used to select among different available recommender modules. Users can thereby choose between the following content-based and collaborative ﬁltering components:

LocationAppRecommender. • CFAppRecommender. • PoiAppRecommender (see Section 6.4.2). • The recommender components itself are also context-sensitive and hybrid. The LocationAppRecommender selects mobile applications that were used in a similar context (most importantly, a nearby location) by other users in a first step. In a second step, the intermediary results are ranked according to ratings (positive/negative), i.e. taking other users into account. Thus, this recommender is an implementation of a cascading hybrid recommender. The first step of the LocationAppRecommender can be seen as a contextualized collaborative recommender where the similarity of users is calculated according to context similarity only, and not according to previously made user ratings. Therefore, this recommender is in principle related to the social recommender explained in Section 6.4.1 [Woerndl and Groh, 2007]. The CFAppRecommender applies collaborative filtering in our domain of mobile applications using the Taste library [Lemire and Maclachlan, 2005]. Taste provides a set of components from which one can construct a customized recommender system. The CFAppRecommender analyzes the item-user data without considering 74 Wolfgang Woerndl and Johann Schlichter context in the first step and filters the intermediary results, i.e. recommended mobile applications, according to context attributes such as devices capabilities (display size and input capabilities) and currently available network bandwidth in the second step.

6.5 Conclusion

In this chapter, we have identiﬁed some general ideas for utilizing context to improve recommender systems. First, we have discussed how to integrate context in the data model of recommender systems. In the main part of the chapter, we have explained contextualized recommender processes, grouped according to the different recommendation techniques that are mainly available today. We have also illustrated the ideas by outlining context-aware recommender applications including a hybrid recommender system which recommends mobile applications to users derived from what other users have installed in a similar physical context (location, currently used type of device, etc). When evaluating contextual recommender systems, a problem is the availability of relevant data sets. Ex- isting data sets (e.g. from the GroupLens project1) are not associated with contextual information. Furthermore, since context is very dynamic it is hard to judge when a predicted recommendation is actually good a bad. A rating given for an item in a certain context may not be valid in another context. Therefore, it seems that an evaluation of context-aware recommender systems should not only involve computed accuracy metrics [Herlocker et al., 2004], but also some kind of user studies. Contextual recommender systems seem to be especially promising in very dynamic environments. This includes mobile and ubiquitous scenarios as motivated in the introduction. Thereby, user interfaces are crucial and developing innovative user interfaces for contextualized personalization applications is a major future research issues. This eminently applies for non-standard application domains. One example is ad-hoc networking of cars, where information like rating of points-of-interest in the vicinity can be exchanged between vehicles and utilized for collaborative, context-aware applications [Woerndl and Eigner, 2007].

References

G. Adomavicius and A. Tuzhilin. Towards the next generation of recommender systems: A survey of the state- of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6), June 2005. G. Adomavicius, R. Sankaranarayanan, S. Sen, and A. Tuzhilin. Incorporating contextual information in recommender system using a multidimensional approach. ACM Transactions on Information Systems, 23(1): 103–145, 2005. S. S. Anand and B. Mobasher. Intelligent techniques for web personalization. Lecture Notes in Artiﬁcial Intelligence, 3169:1–36, 2005. M. Baldauf, S. Dushdar, and F. Rosenberg. Survey on context-aware systems. International Journal of Ad Hoc and Ubiquitous Computing, 2(4):263–277, 2007. R. Burke. Hybrid recommender systems: Survey and experiments. User Modeling and User-Adapted Interac- tion, 12(4):331–370, 2002. A. Chen. Context-aware collaborative ﬁltering system. In 1st International Workshop on Location- and Context- Awareness, pages 244–253, Oberpfaffenhofen, Germany, 2005. A. K. Dey, G. D. Abowd, and D. Salber. Conceptual framework and a toolkit for supporting the rapid prototyp- ing of context-aware-applications. Human-Computer Interaction, 16(2-4):97–166, 2001. G. Groh and C. Ehmig. Improving recommender systems via social networks. In 1st International Conference on Supporting Group Work, Sanibel Island, Florida (USA), 2007. J. L. Herlocker and J. A. Konstan. Content-independent task-focused recommendation. IEEE Internet Com- puting, 5(6):40–47, 2001.

1 http://www.grouplens.org/ 6 Contextualized Recommender Systems: Data Model and Recommendation Process 75

J. L. Herlocker, J. A. Konstan, L. G. Terveen, and J. T. Riedl. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems, 22(1):5–53, 2004. J. Himberg, J. A. Flanagan, and J. Mäntyjärvi. Towards context awareness using symbol clustering map. In 4th International Workshop on Self-Organizing Maps, pages 249–254, Kitakyushu, Japan, 2003. E. Horvitz, P. Koch, and M. Subramani. Mobile opportunistic planning: Methods and models. Lecture Notes in Artificial Intelligence, 4511:228–237, 2007. G. Klyne, F. Reynolds, C. Woodrow, H. Ohto, J. Hjelm, M. H. Butler, and L. Tran. Composite Capa- bility/Preference Profiles (CC/PP): Structure and Vocabularies 1.0, W3C Recommendation, 2004. URL http://www.w3.org/TR/CCPP-struct-vocab/. P. Korpipää, M. Koskinen, J. Peltola, S. M. Mäkelä, and T. Seppänen. Bayesian approach to sensor-based context awareness. Personal and Ubiquitous Computing, 7(2):113–124, 2003. A. I. Kovács and H. Ueno. Recommending in context: A spreading activation model that is independent of the type of recommender system and its contents. In 2nd International Workshop on Web Personalisation, Recommender Systems and Intelligent User Interfaces, Dublin, Ireland, 2006. D. Lemire and A. Maclachlan. Slope one predictors for online rating-based collaborative filtering. In 5th SIAM International Conference on Data Mining, Newport Beach, California (USA), 2005. SIAM. W. Schwinger, C. Grün, B. Pröll, W. Retschitzegger, and A. Schauerhuber. Context-awareness in mobile tourism guides - a comprehensive survey. Technical report, Technical University of Wien, 2005. T. Strang and C. Linnhoff-Popien. Context modeling survey. In 1st International Workshop on Advanced Context Modelling, Reasoning and Management, pages 34–41, Nottingham, United Kingdom, 2004. W. Woerndl and R. Eigner. Collaborative, context-aware applications for inter-networked cars. In 5th Interna- tional Workshop on Distributed and Mobile Collaboration, Paris, France, 2007. W. Woerndl and G. Groh. Utilizing physical and social context to improve recommender systems. In Workshop on Web Personalization and Recommender Systems, in conjunction with the IEEE/WIC/ACM International Conference on Web Intelligence, Silicon Valley, California (USA), 2007. W. Woerndl and J. Schlichter. Introducing context into recommender systems. In Workshop on Recommender Systems in e-Commerce, in conjunction with the 22nd Conference on Artificial Intelligence, Vancouver, Canada, 2007. W. Woerndl, C. Schueller, and T. Rottach. Generating high level context from sensor data for mobile applications. In IADIS International Conference on Wireless Applications and Computing, Lisbon, Portugal, 2007a. W. Woerndl, C. Schueller, and R. Wojtech. Hybrid recommender system for context-aware recommendations of mobile applications. In 3rd IEEE International Workshop on Web Personalisation, Recommender Systems and Intelligent User Interfaces, Istanbul, Turkey, 2007b.

7 Enhanced Media Descriptions for the Automatic Adaptation of Audiovisual Content Retrieval

Iván Cantador, Fernando López, Jesús Bescós, Pablo Castells, and José M. Martínez

Escuela Politécnica Superior, Universidad Autónoma de Madrid (Spain) {ivan.cantador,f.lopezDELIT,J.Bescos,pablo.castells,JoseM.Martinez}@uam.es

Summary. The continuous growth of the audiovisual content body worldwide poses new challenges to content retrieval and delivery technologies, calling for new solutions that cope with the current scale and complexity of content corpora, technological infrastructures, and user audiences. This chapter focuses on a set of initiatives and achievements addressing problems that involve the effective selection of content items and their automatic adjustment to ﬁt a range of support infrastructures. Starting out from a proper description of multimedia content, the adaptive effects discussed here comprise low to high-level adaptation methods, from media transcoding and transmoding, scalable bit-stream modiﬁcation, or video skimming and summarization, to the ranking of content units according to background user interests in different scenarios (e.g. presence vs. absence of an explicit user query, single vs. multiple users, etc).

7.1 Introduction

The continuous growth of audiovisual content in digital format available worldwide poses new challenges to content retrieval technologies, calling for new solutions that cope with the current scale and complexity of content corpora, technological infrastructures, and user audiences. Two major problem areas to be addressed in facing such challenges involve (i) the effective selection of content items when the scale of the retrieval space surpasses the capacity of users to query the corpus, or even browse search result lists; and (ii) the automatic adjustment of multimedia content to fit a wide variety of support infrastructures (terminals, networks, codecs, players, etc), while making the most of the available delivery channel capabilities. Addressing such problems implies work at the level of the identification, representation, dynamic analysis, and effective introduction of the contextual conditions that are relevant for the content retrieval and delivery processes, in order to best suit the situation at hand, in a way that optimizes the effectiveness in terms of user satisfaction. This involves building up a system awareness of dynamic conditions such as user interests and preferences of different kinds (high-level and low-level, broad and specific, explicit and implicit, content- related, source-related, etc), device and network capabilities (screen resolution, network bandwidth, etc). A proper description of multimedia content itself is needed, ranging from signal-level (colour, bitrate, etc) to syntactic (objects, motion, visual attention, etc) and semantic-level descriptions (topics, domain concepts, events, semantic relations). This chapter focuses on a set of initiatives and achievements addressing such problems, resulting in different forms of personalized retrieval and dynamic adaptation (previous or simultaneously to delivery, not covered here) of multimedia content. The comprehensive view on multimedia adaptation provided here comprises low to high-level adaptation methods from the ranking of content units according to background user interests in different scenarios (e.g. presence vs. absence of an explicit user query, single vs. multiple users, etc) to media adaptation techniques to different usage scenarios (terminals, networks, user preferences, etc). The chapter is structured as follows. The next section addresses the different aspects of content that need to be explicitly modelled in order to enable the adaptation techniques discussed thereafter. Section 7.3 focuses 78 Iván Cantador, Fernando López, Jesús Bescós, Pablo Castells, and José M. Martínez on the personalization of content retrieval, the different sides and problems involved, and describes specific proposals in a semantic-based approach. After this, Section 7.4 the adaptation of content at signal-level, that is, customizing it in a content-agnostic way to the different terminals, networks and user preferences (from the media presentation point of view). Finally, some concluding remarks are provided in Section 7.5.

7.2 Content Modelling

Content access and delivery involve different processes or phases, such as query, selection (ﬁltering), linking (recommendation), semantic adaptation, and signal-level adaptation (these last two steps could be combined in one), each involving different technologies. The automatic customization problem can thus be addressed at the different phases, motivating diverse requirements and approaches for each step of the chain. A common key aspect to all of them is the need for appropriate descriptions of the content to which the adaptation strategies apply, at the proper level and providing the relevant details needed to handle the content pieces, as required by the adaptive operations. Considering the speciﬁcs of multimedia content, the following aspects need to be taken into account to this respect:

Representation model: The syntax and semantics associated to the bitstream that represents the signal • used to deliver and present the content. Description model: The syntax and semantics associated to what is present in the content from a signal- • level point of view (e.g. colour, shape, texture, motion for visual content) and from a mid-level point of view (generic objects and events), as well as the descriptive information required from an archival point of view. High-level semantics model: The syntax and semantics associated to the high-level interpretation and • meaning of what is present in the content, generally linked to a particular application domain.

The requirements and available technologies related to each of the above description levels are discussed in the next sections.

7.2.1 Content Representation

There are different speciﬁcations for the structuring of the bitstream used to represent media content. Although proprietary formats were in use in the past, nowadays most applications work with standardized formats deﬁn- ing both the syntax and the semantics of the bitstream, from a purely signal point of view, and mainly describing a compressed representation of the media, with a good subjective quality even in the presence of heavy compression. Most of the wide-spread standards for audiovisual content belong to the JPEG family for images (the well- known JPEG format available in all digital cameras, and the new JPEG2000 providing much better quality for the same bitrate), and to the MPEG family for video and audio: from MPEG-2 used in DVDs and Digital Television, and MP3 for audio on the Internet, to heavily improved versions in the form of MPEG-4 Advanced Audio Coding (AAC) and MPEG-4 Advanced Video Coding (AVC) also known as H.264.

7.2.2 Content Description

Besides the content representation of the signal, it is required to have a description of the content, that is, metadata (data about the content essence). These descriptions range from the classical archival data (title, author, release date, abstract, keywords, etc) to content-based descriptions of what is present in the content (e.g., a goal in a soccer video, nudity in a ﬁlm, an interview by the anchorman in a news program). There are several standards for archival metadata (e.g., EXIF for images, ID3 for audio ﬁles, Dublin Core for generic digital content), whilst only a small number of standards, mostly broadcasters-oriented, deal to some extent with content-based metadata (e.g., SMTPE Metadata Dictionary, EBU P/Meta, TV-Anytime). 7 Enhanced Media Descriptions for Audiovisual Content Retrieval 79

The MPEG-7 standard [Manjunath et al., 2003] is the most comprehensive multimedia content description specification, covering both archival metadata as well a large number of metadata for content-based description. MPEG-7 provides description tools for low-level audiovisual descriptions (e.g. colour, shape, audio spectrum), mid-level (e.g. face description, musical timbre), structural level (e.g. shot and scene decomposition, musical movements), and even tools for semantic and linguistic description. The main drawback of the MPEG-7 specifications is their complexity, mainly due to the fact that MPEG-7 was aimed to be a generic standard, not focused on a particular application domain. In order to help reducing this complexity, and following previous MPEG standards, profiles (subset of tools) have been defined for different applications domains.

7.2.3 Usage Context Description

By usage context we mean the terminal capabilities, network conditions and media related user preferences (e.g. preference for an image slide rather than a video, or a travelling video instead of an image that does not ﬁt in the screen) that are active in each media consumption session. All this information is required in order to make decisions about what kind of adaptation should be performed over a media for a speciﬁc session. There are several standards dealing with some of these descriptions (e.g., UAProf, CC/PP), but the Digital Item Adaptation (DIA) part of MPEG-21 [Burnett et al., 2006, Vetro and Timmerer, 2005] is the one covering the more complete set of such descriptions, as well as additional tools for media adaptation techniques.

7.2.4 High-level Semantics

The descriptions discussed above convey a lot of useful information for handling and reformatting content at the signal-level, as described later in Section 7.4. However, from the point of view of cognitive user needs, higher-level information is required. At the end of the consumption chain, users of multimedia content are concerned with the information they are going to get by viewing a specific document, which is a major source of relevant input for personalizing the choice of documents to be delivered, or fragments to be viewed within. High-level semantics aim to describe what objects appear in a scene (people, cars, buildings, trees, roads), what type of scene is displayed (tennis, beach, restaurant, indoors/outdoors), what happens in it (a person enters the room, walks, sits down, smiles, eats), and so forth. The type of entities, events, and subjects that may appear in a multimedia document are virtually anything, which makes it hard to provide a general framework supporting the needed descriptions in a general way. While it is indeed impossible to model the world as a whole (although some attempts have been made [Lenat and Guha, 1990]), a partial and feasible approach is possible for restricted domains, as supported by ontology- based knowledge technologies [Staab and Studer, 2004]. Ontology-driven representations, and in particular the ones supported by W3C standards such as RDFS [Brickley and Guha, 2004] and OWL [McGuinness and van Harmelen, 2004], have nice properties such as being formal, non-ambiguous, rich (in direct proportion to the human effort invested), and enabling automatic inference based on Description Logics [Baader et al., 2003]. In the techniques discussed in the next section, the proposed representation to describe the meanings within content consists of a list of domain concepts that appear or happen in the content [Castells et al., 2007]. Concepts are associated to content by semantic annotation links, which may include time stamps when concepts occur in specific content segments, and one or several weight values indicating the strength of the link. The weights can reflect the importance of the concept in the content, the certainty that the concept actually occurs (e.g. when it has been recognized by an automatic content analysis technique), or any other relevant numeric measures. Ontology-based annotations can be created manually, or obtained through automatic means [Dasiopoulou et al., 2005], e.g. by extracting concepts from manual textual annotations, spotting words or known sounds in audio tracks, detecting objects and events in visual scenes, or by coordinating several of these approaches. This representation goes beyond the simpler and currently dominating forms of free annotation, commonly consisting of plain string keywords or arbitrary text sentences. It provides a unified, unambiguous representation of the semantic space for annotation, hence a solid ground upon which powerful adaptation techniques can be devised, capable of making sense of the high-level semantic descriptions. Additionally, all the aforementioned facilities supported by ontology-based technologies are available to the advantage of the development of personalization strategies. 80 Iván Cantador, Fernando López, Jesús Bescós, Pablo Castells, and José M. Martínez 7.3 Personalized Content Retrieval

In general terms, personalizing the selection of content for user access involves knowing something about the user beyond her last single request, and taking advantage of this knowledge in order to improve the system response to the actual user needs, in terms of the utility of the retrieved content for each individual user. Room for such improvement exists increasingly often, to varying degrees, in common retrieval scenarios, either because the request is vague, in the sense that too much content matches it for the user to process (e.g. as happens most of the time in large-scale content spaces), or because there is no explicit request (e.g. the user is not in- stantly aware of new content of interest). In addition to the requirements in the scope of content description, the problems to be addressed to realize this view can be related to the areas of representation, elicitation, and exploitation of user preferences.

7.3.1 User Preference Modelling

Modelling user preferences is central to content personalization technologies, as it forms the basis on which the system may automate decisions on behalf of users. User profiles for multimedia retrieval provide a logical view of known or predicted user particularities and needs with respect to content features and formats, in a suitable form for use by the algorithms that implement the automatic adaptation strategies. A variety of structures and paradigms have been used in research and industry for the representation of relevant user information in this context. Among standardization efforts, the MPEG-21 DIA stands out as one of the most exhaustive specifications covering, among other more media-related features, information such as demographic data, general preferences, audiovisual presentation preferences, disabilities, focus of attention, etc. The Multimedia Descrip- tion Schemes in MPEG-7 provides additional structures for storing usage history data [Manjunath et al., 2003]. MPEG-21 describes the user information which, along with content descriptions in MPEG-7, provides a suitable basis for multimedia-specific content adaptation in the delivery phase, as will be described in Section 7.4. Related to content selection, the usage environment data described in MPEG-21 is generally not enough (or not relevant) to make far-reaching predictions of what content choice a user might enjoy. To this respect, it is more important to know what the content is about, and what subjects, concepts or objects the user is interested in. A common approach to model this kind of preference is the vector-space model, in which user interests are represented as a vector of weights defining the intensity of interest for different things. The space of such things, in terms of which user interests are described, can correspond to elements of diverse nature, such as individual documents, categories from a taxonomy, or even plain words [Chirita et al., 2005, Jeh and Widom, 2003, Micarelli and Sciarrone, 2004]. In our research, we have explored the potential of enhanced representations of meanings as the foundation of this preference space, beyond prior simple approaches, as a means to enable improvements in the reach and accuracy of personalization [Cantador et al., 2008, Gauch et al., 2004]. More specifically, we have developed ontology-based preference models, where the preference space is based on domain ontologies [Staab and Studer, 2004], in such a way that user interests are represented by a weight for each domain concept in the ontology. These weights can be derived from the observation of user activity (queries and accessed content) over a period of time, and the repeated occurrence of concepts related to the objects involved in this activity. This is of course a problem on its own; the reader is referred to e.g. [Cantador et al., 2008, Mylonas et al., 2008] for further discussion. The advantage of using ontologies as the representational grounding for the semantic space lies in the precise and detailed information that is made available for the system e.g. to match user preferences to content descriptions, enabling substan-tially more elaborate strategies than simpler representations support. For instance, the system can derive new user preferences from the initial ones based on the extra knowledge supplied by an ontology, using the formal inference mechanisms supported by ontology standards. This is further explained when we discuss the use for context modelling and content filtering in the next sections. An alternative (or complementary) view on the personalized recommendation of content is the social one, in which users are not just analyzed in isolation, but in comparison to other users. A well-known approach in this direction is the so-called collaborative filtering strategy, which is based on the general assumption that users with common traits may enjoy the same or related content [Adomavicius and Tuzhilin, 2005]. This principle raises the problem of measuring the similarity between users, which can be done based on a comparison of the 7 Enhanced Media Descriptions for Audiovisual Content Retrieval 81 profile information (e.g. semantic preferences, demographic information), or the history of common choices. The similarity of picked contents can be in turn measured by functions other than straight equality, where again, a rich semantic representation of content enables the detection of indirect similarities between items, that would not be found by checking for plain coincidence. For in-stance, a user who liked the films Interiors and Annie Hall, and a user who liked Broadway Danny Rose and Manhattan would be considered candidates for a possible affinity based on indirect evidence, since they both enjoyed films directed by Woody Allen, even if they did not watch the same films. This is possible if a domain ontology on cinema is available to the system, where movies, directors, actors and so on are defined as interrelated domain concepts and individuals.

7.3.2 Context Modelling

In order to properly customize content, knowledge about the context in which the content is sought and consumed by a particular user at a given time, is relevant as well. Research in this area has commonly considered user preferences and terminal and network capabilities (which we discuss later in this chapter) as part of the context model. Beyond this, other elements addressed in the literature include the rendering context (e.g. noise, illumination), location, time, meteorological data, etc. Besides the user preference information described in the previous section, in our research on content retrieval adaptation we consider three additional contextual dimensions, namely (i) the dynamic user focus, consisting of a weighted, semantically coherent set of domain concepts that have been involved, in some way or other, in an ongoing user session [Vallet et al., 2007]; (ii) the semantic context of meanings, defined as the domain concepts closely related (through semantic relations explicitly defined in an ontology) to a given set of concepts [Mylonas et al., 2008]; and (iii) the social context of a user, consisting of the sets of users related to her in different possible ways [Cantador and Castells, 2006]. The semantic context of a concept is given by the concepts around it in the semantic network defined by a domain ontology, based on the relations that interlink the concepts. Semantic paths provide a basis to define distance measures between concepts, upon which we build fuzzy contextual supersets for a given set of concepts [Vallet et al., 2007]. This step is key in our approach to handle the social context, in which further similarities between users are found by comparing the semantic context of their preferences [Cantador and Castells, 2006]. This is applied in a similar way in order to take advantage of the live user focus to contextualize persistent user preferences, and enable a more accurate personalization of retrieval results [Mylonas et al., 2008], as will be described in the next section.

7.3.3 Content Filtering

Based on the relevant knowledge about user interests that a system is able to capture and elaborate on, different techniques can be used to personalize search and retrieval results, which come into play at different points of a retrieval system. For instance, user preferences have been used to reformulate user queries, by expanding or refining the information in the query (e.g. adding, reweighing, or even disambiguating terms), using information from the user profile and history [Rocchio, 1971]. Preferences can also be applied on the search result after the query has been answered. For example, the results can be re-ranked by a complementary similarity measure between documents and user preferences [Chirita et al., 2005, Micarelli and Sciarrone, 2004]; clustering techniques have also been proposed to group results in sets of categories, ranking higher the more relevant categories by user preference [Liu et al., 2004]. A long series of variations of the popular PageRank algorithm for Web search have been researched as well, which use different initialization strategies based on user preferences, in such a way that the resulting PageRank value is biased towards individual user interests [Jeh and Widom, 2003]. In scenarios where information (preferences, history) of a large number of users is centrally available to the personalization system, it is also possible to apply collaborative filtering techniques in a way that thousands of users benefit from each other’s experience without even getting ever acquainted, as discussed earlier. We have explored the potential for improvement that can be gained in such different strategies by building on the semantic enhancements for the representation of user interests described in Section 7.3.1. To begin with, ontologies enable the automatic extension of user preference reach by means of inference steps. For instance, 82 Iván Cantador, Fernando López, Jesús Bescós, Pablo Castells, and José M. Martínez a preference for pets would automatically expand to cats and dogs. On a more complex path, a preference for award-winning films can be defined in an ontology language like OWL by a restriction class defined as the movies that have won some award. The user can have his preference fulfilled by the system inferring that, say, an Oscar is a subclass of award, and a movie that won some Oscar is an instance of the preferred class by the user. Our primary application of these principles has been brought to a basic content filtering approach, which re-ranks search results by comparing user preferences and content metadata in the ontology-based vector space as described in earlier sections. This comparison is achieved by a similarity measure that compares these two concept vectors (user preference and content semantics) based on the cosine function. This measure is then combined with the relevance score returned by the search engine (which is taken here as a black box) for each content item and the query being answered, thus introducing a personalized bias in the ranking [Cantador et al., 2008]. This approach is further elaborated by contextualizing the preference model before it is matched to content descriptions, based on the live user focus introduced in the previous section. In our approach, putting preferences in context means to activate only those persistent preferences that are semantically close to what is going on in the session and what the user has in mind at that time. This matches the way human preferences work in real life where, for instance, bringing up a long-term (e.g. professional) user interest for pharmaceuticals when a user is searching for a good movie in the weekend could be generally out of place. This feature is addressed as follows. During a retrieval session, the system collects all the concepts involved in user queries and metadata of the content se-lected by the user. A vector of weighted concepts (where weights decay with time) is built in this way, which is taken to be representative of the short-term user attention, or at least to be related to it. Then, the contextualization mechanism starts by estimating the semantic distance between each concept in the preference set and the live focus set, which is achieved by scanning the number and length of paths connecting them through the semantic network defined in the ontology. This is achieved by computing the extended fuzzy semantic context of user preferences and focus, and their fuzzy intersection. The resulting degrees of membership of concepts to this fuzzy set is combined with the original preference weight in the long-term user profile to produce a contextualized preference vector, where the weight of preferences that are semantically unrelated or far from the current user focus will be close to zero, thus achieving the desired effect [Mylonas et al., 2008, Vallet et al., 2007]. The semantic context modelling and user profile extension techniques have also been applied in our research to elaborate on the collaborative recommendation approach, as introduced in Section 7.3.1. Besides the extension of similarity functions based on indirect semantic comparisons discussed earlier in this chapter, we have explored a step beyond this by further refining the model with the notion of semantic layers within user preferences, as we explain next. In typical collaborative filtering approaches, the comparison between users is performed globally, in such a way that partial, but strong and useful similarities might be missed. For instance, two people may have a highly coincident taste in cinema, but a very divergent one in sports. Their opinions on movies could be highly valuable for each other, but risk to be ignored by many recommender systems, because the global similarity between the users might be low. We thus contend for the distinction of different layers within the interests of users, as a useful refinement to produce better recommendations. This idea is achieved by analyzing the structure of the domain ontology, the weighted links between users and ontology concepts (as defined by preferences), the links between concepts and contents (annotations), and the links (explicit ratings) between content and users. Based on this rich interrelation within and across the three spaces (users, concepts, content), we develop strategies of coordinated clustering which produce focused recommendations based on partial but cohesive similarities. Our approach finds groups of interests shared by users, and communities of interest among users. Users who share interests of a specific concept cluster are connected in the corresponding community, where their preference weights determine the degree of membership to that cluster. This enables focused recommendations in the different communities [Cantador and Castells, 2006]. At the same time, this approach tackles the common sparsity problems of the collaborative filtering approach in large-scale retrieval spaces (i.e. user preference information being scarce when an item is new, a user is new, or user ratings are thinly spread over a huge number of items [Adomavicius and Tuzhilin, 2005]), by finding indirect similarities among users and content items. Note that links between users (contacts) 7 Enhanced Media Descriptions for Audiovisual Content Retrieval 83 and between documents (e.g. hyperlinks or other cross-references) could also be considered as part of the available information to further enhance the analysis, which we foresee as future research.

7.4 Content Adaptation

Once a speciﬁc content item is picked by the user from a list of choices retrieved by the system according to user queries, preference, or any of the strategies discussed in the previous section, the time comes to actually deliver the content itself in the most suitable form. At this point, the development of new access networks providing multimedia capabilities, and a wide and growing range of terminals, makes the adaptation of content a major issue for future multimedia services. Content adaptation is the main objective of a set of technologies that can be grouped under the umbrella of Universal Multimedia Access (UMA) [Vetro et al., 2003]. This means the capability of accessing to rich multimedia content through any client terminal and network. In this way content adaptation bridges content authors and content consumers in a world of increasing multimedia diversity. In order to perform content adaptation it is necessary to have the description of the content and the description of the usage environment. To enhance the user’s experience [Pereira and Burnett, 2003], not only terminal and network characteristics and conditions should be taken into account when adapting, but also user preferences and disabilities, as well as environmental conditions. All this information imposes some constraints to the content to be obtained after adaptation.

7.4.1 Content Adaptation Taxonomy

According to the level of understanding applied to the media, multimedia adaptation can be performed in two different ways:

Signal level adaptation, committed to transcoding media resources without knowledge of the meaning of • the content. Semantic level adaptation, which modiﬁes the media assuming that there exists some knowledge about • the meaning of the content.

According to the information used to decide the adaptation to perform, multimedia adaptation can be based on three different sources:

Usage environment driven adaptation. The terminal, network and user preferences are taken into account • to decide the adaptation to perform to the multimedia content. Semantic driven adaptation evaluates the content of the media to select the more relevant parts. This kind • of adaptation has been mainly used to perform content summarization. Perception driven adaptation performs transformations according to some user preferred sensation, or • assists a user with a certain perception limitation or condition. The user perception of the content will be different from its original version, for example, to address the needs of users with visual handicaps (e.g. colour blind deﬁciencies, or speciﬁc preferences) in terms of visual temperature sensations. This class of customization operations considers all types of adaptations related to some human perceptual preferences, characteristics or handicaps.

According to the way to group media, adaptation can be performed at two different levels:

Media level adaptation, committed to adapt a unique resource, named media resource, and usually stored • as a file. Content to be adapted at this level can also be represented as a stream of bytes. Typically, a standardization body defines the whole format of this single modality media. System level adaptation (also named multimedia level), committed to adapt one or more resources grouped • as a compound resource, named system resource (e.g. a web page, or a SMIL file). This system resource can also convey metadata (descriptors). Within the system level compositions, we can identify three main kinds of adaptation: (i) structure level adaptation adapts a resource that is the union of (or has references to) other 84 Iván Cantador, Fernando López, Jesús Bescós, Pablo Castells, and José M. Martínez

media resources (e.g. DI, HTML); (ii) layout level adaptation changes the arrangements of the constituent elements in the scene (e.g. HTML, SMIL); (iii) synchronization level adaptation modiﬁes the timeline of the constituent resources (e.g. MPEG ﬁles summarization, or SMIL image serialization). For example, an MPEG-21 Digital Item Declaration [Burnett et al., 2006] is capable of conveying media resources and descriptors grouped as a system resource named Digital Item (DI). During the adaptation, the number of resources and descriptor elements may change.

These two adaptation levels (media and system) can be performed in a signal or semantic way. Conversely, signal and semantic level adaptations can be performed in a media or system level. Another dimension to consider in an adaptation framework are the different adaptation approaches, that can be also applied with content understanding (semantic level adaptation) or without it (blind or signal level adaptation); all these approaches are expected to coexist [Vetro, 2003]: Transcoding. This is the most frequent adaptation, where the media parameters are changed in order to fit • the constraints imposed by the usage environment. Scaling, colour reduction, coding standard, etc. Transmoding. This adaptation implies a change of modality of the original media in order to enable it to • be presented in the selected scenario. Text to audio, video to slideshow or image to video are among this new generation of content adaptation. Scalable content truncation. Scalable formats are based on the co-existence in the same file or stream of • different versions of the same content at different temporal, spatial and quality resolutions. Efficient tools for management of such content are provided by bitstream description tools within MPEG-21 DIA [Vetro and Timmerer, 2005]. Summarization. Video and audio skimming (creating excerpts of the temporal content, as for video trail- • ers) are the most common approaches to summarization, although there are other proposals based on additional transmoding (e.g., a video poster consisting of the composition into one poster image of selected keyframes of the summarised video). The presentation of information provides different options to summarization, whilst the common ground are the content analysis technologies for the identification of the relevant parts of the content: keyframes or relevant video segments of a video, chorus of a song, main theme of a symphony, etc.

7.4.2 Content Adaptation Engine

Content adaptation is performed by a module of the complete content retrieval system, that is usually named Content Adaptation Engine. It aims to provide the user the best experience when accessing the requested content within the current usage environment. A Content Adaptation Engine includes two main functionalities: deciding the adaptation to perform, and managing the decided adaptation process. In the simplest cases an Adaptation Engine is only able to perform one adaptation (e.g., reducing the spatial resolution of an image to a half in each dimension) and therefore no decision phase is required. The next complexity step comes up when the Content Adaptation Engine can choose between different values for certain parameters of the adaptation (e.g. reducing the spatial resolution in a predefined range). Additional complexity appears when a Content Adaptation Engine is able to perform two (or more) adaptation processes, each with its own parameters (e.g. for achieving a target file size, spatial resolution or bits per pixel can be reduced). In this case, the decision is not only parameter-based; instead, a combination of adaptation processes and their parameters can achieve the target adaptation. According to the identified functionality, a generic Content Adaptation Engine can be modelled to include two main submodules: a Decision Module and a set of Content Adaptation Tools (CATs). Each CAT is able to perform a particular adaptation. The different available CATs may diverge in the adaptation approach (e.g. transcoding, transmoding, etc), the range of parameters values, the supported input and output formats, the performance (in terms of processing requirements, quality, etc), and so forth. Hence, there is a need for describing the CAT adaptation capabilities [Valdés and Martínez, 2006a], so that the Decision Module is able to incorporate also this description when making a decision primarily based on the aforementioned content and usage environment descriptions. 7 Enhanced Media Descriptions for Audiovisual Content Retrieval 85

Several approaches have been proposed to perform content adaptation [Jannach et al., 2006, Magalhaes and Pereira, 2004, Martínez et al., 2005, Tseng et al., 2004, Vetro and Timmerer, 2005]. We have developed a Content Adaptation Engine, named CAIN (for Content Adaptation Integrator) [Martínez et al., 2005], aimed to the integration of different content adaptation approaches, that is, different CATs [Valdés and Martínez, 2006b]. In CAIN the Decision Module uses Constraints Programming [López and Martínez, 2007] for selecting the CAT that is best suited to the optional and mandatory constraints imposed, respectively, by terminal and network characteristics and user preferences [López et al., 2006]. It should be noted that [Jannach et al., 2006] proposes the use of a scheduling algorithm to ﬁnd a chain of elementary adaptation operations which transform the media accordingly, whilst the CAIN framework considers CATs that perform several elementary adaptations. The Decision Module selects only one CAT from the available ones (we are evaluating to extend our solution in order to allow the concatenation of CATs in the future). The adaptation methods are constructed upon the foundations provided by MPEG description standards: content descriptions are based on MPEG-7 MDS [Manjunath et al., 2003] and MPEG-21 DIA BSD [Burnett et al., 2006], whilst the context descriptions are based on a subset of MPEG-21 DIA Usage Environment Descriptions tools [Burnett et al., 2006].

7.5 Conclusions

Automatic adaptation is a major issue in modern multimedia content delivery environments and infrastructures. In this chapter we have discussed a set of adaptation techniques that address this need, spanning across the whole content retrieval and delivery cycle, from the selection and choice of content units, based on high-level descriptions of the meanings within, to the actual delivery of content streams, adapted to the available conditions at the consumer end-point. The main innovations in our proposed approaches for personalized retrieval focus on the potential for improvements enabled by working on (i) the representation of semantics and (ii) the consideration of the retrieval user context, where in our model so far the latter includes semantic contexts, the social environment, and the user focus. The ontology-based approach to semantics representation has its own trade-offs, the main ones being the limited availability of ontologies, and the development cost and formalization problems involved in deﬁn- ing very detailed ones. However, the proposed personalization techniques are tolerant to incomplete knowledge, which means that they make the most of high-quality semantic information whenever it is available, and they degrade gracefully to the performance of any standard technique based on simpler representations (e.g. keywords, documents) with which our techniques can be combined, as the completeness of ontological knowledge decreases for the domain area at hand. The proposed techniques for content adaptation are characterized by the capability to both decide on the most appropriate adaptation strategy for the dynamic situation in process, and actually carry out and manage the selected adaptation approach. The adaptation of content is built upon the expressive capabilities of the MPEG standards. Content and access adaptation together address the challenges raised by the new order of magnitude in the scale of current content environments, and the heterogeneous and dynamic nature of delivery platforms and user audiences, in order to deliver the subjective content service quality that an enhanced user experience requires. In order to reach this point, automatic content understanding is currently the most challenging research issue in content analysis in order to be able to perform semantic based adaptation based on what is present in the content.

7.6 Acknowledgements

This research was partially supported by the European Commission under contracts FP6-001765 aceMedia and FP6-027685 MESH. The expressed content is the view of the authors but not necessarily the view of the aceMedia or MESH projects as a whole. Thanks are due to the members of the NETS and GTI research groups at EPS-UAM who have collaborated in these and other research projects, where the experiences on multimedia content customization reported here have been acquired: Miriam Fernández, Víctor Fernández- Carbajales, Javier Molina, Víctor Valdés, and David Vallet. 86 Iván Cantador, Fernando López, Jesús Bescós, Pablo Castells, and José M. Martínez References

G. Adomavicius and A. Tuzhilin. Toward the next generation of recommender systems: A survey of the state- of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6):734–749, 2005. F. Baader, D. Calvanese, D. L. McGuinness, D. Nardi, and P. F. Patel-Schneider. The Description Logic Hand- book: Theory, Implementation, Applications. Cambridge University Press, 2003. D. Brickley and R. V. Guha. RDF Vocabulary Description Language 1.0: RDF Schema. W3C Recommenda- tion, February 2004. I. S. Burnett, F. Pereira, R. Van de Walle, and R. Koenen. The MPEG-21 Book. Wiley & Sons, 2006. I. Cantador and P. Castells. Multilayered semantic social network modelling by ontology-based user profiles clustering: Application to collaborative filtering. In 15th International Conference on Knowledge Engineer- ing and Knowledge Management, Podebrady, Czech Republic, 2006. I. Cantador, M. Fernández, D. Vallet, P. Castells, J. Picault, and M. Ribière. Advances in Semantic Media Adaptation and Personalization, chapter A Multi-Purpose Ontology-Based Approach for Personalised Con- tent Filtering and Retrieval. Springer Verlag, 2008. P. Castells, M. Fernández, and D. Vallet. An adaptation of the vector-space model for ontology-based information retrieval. IEEE Transactions on Knowledge and Data Engineering, 19(2):261–272, 2007. P. A. Chirita, W. Nejdl, R. Paiu, and C. Kohlschütter. Using ODP metadata to personalize search. In 28th International ACM SIGIR Conference on Research and Development in information Retrieval, Salvador, Brazil, 2005. S. Dasiopoulou, V. Mezaris, I. Kompatsiaris, V. K. Papastathis, and M. G. Strintzis. Knowledge-assisted semantic video object detection. IEEE Transactions on Circuits and Systems for Video, 15(10):1210–1224, 2005. S. Gauch, J. Chaffee, and A. Pretschner. Ontology-based personalized search and browsing. Web Intelligence and Agent Systems, 1(3-4):219–234, 2004. D. Jannach, K. Leopold, C. Timmerer, and H. Hellwagner. A knowledge-based framework for multimedia adaptation. International Journal on Applied Intelligence, 24(2):109–125, 2006. G. Jeh and J. Widom. Scaling personalized web search. In 12th International World-Wide Web Conference, Budapest, Hungary, 2003. D. Lenat and R. V. Guha. Building Large Knowledge-Based Systems: Representation and Inference in the Cyc Project. Addison-Wesley, 1990. F. Liu, C. Yu, and W. Meng. Personalized web search for improving retrieval effectiveness. IEEE Transactions on Knowledge and Data Engineering, 16(1):28–40, 2004. F. López and J. M. Martínez. Multimedia content adaptation modelled as a constraints matching problem with optimisation. In Workshop on Image Analysis for Multimedia Interactive Services, Santorini, Greece, 2007. F. López, J. M. Martínez, and V. Valdés. Multimedia content adaptation within the CAIN framework via constraints satisfaction and optimization. In 4th International Workshop on Adaptative Multimedia Retrieval, Geneva, Switzerland, 2006. J. Magalhaes and F. Pereira. Using MPEG standards for multimedia customization. Signal Processing: Image Communications, 19(5):437–456, 2004. B. S. Manjunath, P. Salembier, and T. Sikora. Introduction to MPEG-7 Multimedia Content Description Inter- face. Wiley & Sons, 2003. J. M. Martínez, V. Valdés, J. Bescós, and L. Herranz. Introducing CAIN: A meta-data-driven content adaptation manager integrating heterogeneous content adaptation tools. In Workshop on Image Analysis for Multimedia Interactive Services, Montreux, Switzerland, 2005. D. L. McGuinness and F. van Harmelen. OWL Web Ontology Language Overview. W3C Recommendation, February 2004. A. Micarelli and F. Sciarrone. Anatomy and empirical evaluation of an adaptive web-based information filtering system. User Modeling and User-Adapted Interaction, 14(2-3):159–200, 2004. P. Mylonas, D. Vallet, P. Castells, M. Fernández, and Y. Avrithis. Personalized information retrieval based on context and ontological knowledge. Knowledge Engineering Review, 23(1), 2008. 7 Enhanced Media Descriptions for Audiovisual Content Retrieval 87

F. Pereira and I. Burnett. Universal multimedia experiences for tomorrow. IEEE Signal Processing Magazine, 20(2):63–73, 2003. J. Rocchio. The SMART Retrieval System: Experiments in Automatic Document Processing, chapter Relevance feedback information retrieval. Prentice-Hall, 1971. S. Staab and R. Studer, editors. Handbook on Ontologies. Springer Verlag, 2004. B. L. Tseng, C. Y. Lin, and J. R. Smith. Using MPEG-7 and MPEG-21 for personalizing video. IEEE Multi- media, 11(1):42–53, 2004. V. Valdés and J. M. Martínez. Content adaptation capabilities description tool for supporting extensibility in the CAIN framework. In International Workshop on Multimedia Content Representation, Classiﬁcation and Security, Istambul, Turkey, 2006a. V. Valdés and J. M. Martínez. Content adaptation tools in the CAIN framework. In 9th International Workshop Visual Content Processing and Representation, Sardinia, Italy, 2006b. D. Vallet, P. Castells, M. Fernández, P. Mylonas, and Y. Avrithis. Personalized content retrieval in context using ontological knowledge. IEEE Transactions on Circuits and Systems for Video Technology, 17(3):336–346, 2007. A. Vetro. Transcoding, scalable coding and standardized metadata. In 8th International Workshop on Visual Content Processing and Representation, Madrid, Spain, 2003. A. Vetro and C. Timmerer. Digital item adaptation: Overview of standardization and research activities. IEEE Transactions on Multimedia, 7(3):418–426, 2005. A. Vetro, C. Christopoulos, and T. Ebrahimi. Special issue on universal multimedia access. IEEE Signal Processing Magazine, 20(2), 2003.

8 A Semantics-Based Approach Beyond Traditional Personalization Paradigms

Yolanda Blanco-Fernández, Alberto Gil-Solla and Manuel Ramos-Cabrer

Department of Telematics Engineering, University of Vigo, 36310, Spain {yolanda,agil,mramos}@det.uvigo.es

Summary. The so-called recommender systems have become assistance tools indispensable to the users in domains where the information overload hampers manual searching processes. In literature, diverse personalization paradigms have been proposed to match the personal preferences of each user against the available items. All these paradigms are laid down on a common substratum that uses syntactic matching techniques, which greatly limit the quality of the offered recommendations due to their inflexible nature. To fight these limitations, this paper explores a novel approach based on reasoning about the semantics of both the users’ preferences and considered items, by resorting to less rigid inference mechanisms borrowed from the Semantic Web. Our reasoning-based approach shape enhanced personalization strategies that migrate from the traditional syntactic proposals to the inference of semantic relationships between the users’ interests and the available items. Thanks to the exploitation of semantics, the discovered relationships provide the recommender system with additional knowledge about the user’s preferences, thus leading to more accurate suggestion processes. The flexibility and strength of our approach enable its application in multiple domains and personalization tools. Specifically, for testing and validation purposes, our semantics-based strategies have been incorporated and evaluated in several tools bound to the field of Digital TV, where viewers are exposed to overwhelming amounts of audiovisual contents and challenged by the plethora of interactive functionality provided by the current digital receivers. The achieved results have revealed both the scalable nature and the computational feasibility of our domain-independent personalization approach based on semantic reasoning.

8.1 Introduction

Since the middle of nineties, recommender systems have been gaining an increasing interest as tools that help the users by selecting automatically items of interest in domains where the overload of information is noticeable [Adomavicius and Tuzhilin, 2005]. For that purpose, such personalization tools employ matching techniques that compare the available items with the users’ preferences, which are previously modeled in personal profiles. As Qing Li et al. describe elsewhere in this book, the most used strategies can be grouped into three main categories, named content-based filtering, collaborative filtering and hybrid approaches. Broadly speaking, a content-based filtering approach seeks to recommend similar items to those the user liked in the past, whereas the collaborative approaches select items for a given user that similar users also appreciated, as we detail next.

Content-based filtering relies on an ability to accurately represent recommendable items in terms of a • suitable set of content attributes, and to represent user profile information as an analogous feature set. In this strategy, the user’s profile effectively delimits a region of the item-space from which all future suggestions will be drawn, in such a way that the relevance of a given content item to a specific target user is proportional to the similarity of this item to her profile (see [Hammond et al., 1995, Smyth and Cotter, 1999]). To measure these similarity values, traditional approaches resort to syntactic metrics that 90 Yolanda Blanco-Fernández, Alberto Gil-Solla and Manuel Ramos-Cabrer

only permit to detect resemblance among items sharing identical attributes, thus leading to overspecialized recommendations that suffer from a limited diversity. This is particularly problematic for new users since their recommendations will be based on a reduced set of items represented in their immature profiles. So, items relevant to a user, but bear little or no resemblance to the items in her profile, will never be suggested in the future. Collaborative filtering. As explained in [Goldberg et al., 1992, Maltz and Ehrlich, 1995], the basic idea • in the collaborative paradigm is to move beyond the experience of an individual user’s profile, and instead draw on the experiences of a community of users. Typically, each target user is associated to a set of nearest- neighbor users by comparing their respective profiles. To delimit the user’s neighborhood, collaborative techniques look for correlation between users in terms of the ratings they assigned to the items contained in their profiles. From this it follows that two users are neighbors when they have rated common items in their profiles by assigning them similar ratings. Once the user’s neighborhood has been created, collaborative approaches predict her interest in the items rated by those neighbors. As a result, the items with highest ratings in the neighbors’ profiles are finally suggested to the target user. Collaborative approaches also suffer from severe drawbacks, such as the so-called sparsity problem and latency concerns. As for the first limitation, note that as the number of available items increases, it is unlikely that two users rate the same items in their profiles, thus hampering the selection of the user’s neighbors. Besides, in this situation, the creation of the user’s neighborhood (based on computing correlations between vectors containing the users’ ratings) becomes too demanding in computational terms, resulting in substantial scalability problems. The second-mentioned limitation is related to the fact that collaborative filtering is not suitable for suggesting new items because only those already rated by other users can be recommended. So, if a new item is added to the content database in the recommender system, there can be a significant delay before this item will be considered for recommendation (see [Adomavicius and Tuzhilin, 2005] for details). Hybrid approaches. Bearing in mind the internals of content-based and collaborative filtering techniques, • researchers tended to define hybrid approaches where both techniques complement each other perfectly, allowing to exploit their advantages and mitigate their respective weaknesses. As Burke described in [Burke, 2002], the most of the hybrid recommender systems adopt the so-called “collaboration via content” paradigm by Pazzani [Pazzani, 1999], based on computing the similarity between two users’s preferences by considering both the content descriptions (just like in content-based approaches), and their respective ratings (considered in collaborative filtering). This way, Pazzani fights the sparsity problem by detecting that two users have similar preferences even when there does not exist overlap between the items contained in their profiles. However, in order to measure similarity in this case, it is necessary that these items have common attributes, hence the fact that Pazzani’s approach is still limited by the syntactic metrics used in the traditional content-based techniques. In this paper, we describe how to overcome this kind of limitations —derived from the purely syntactic techniques adopted by existing personalization paradigms— by exploiting the semantics of the application domain of each recommender system. To this aim, our approach resorts to reasoning mechanisms bound to the so-called Semantic Web, in which information is given a well-defined meaning, better enabling computers and people to work in cooperation. To achieve this goal, three main elements are required in the Semantic Web: firstly, metadata to annotate the available Web resources; secondly, a domain ontology where these semantic annotations are formalized; and finally, some reasoning techniques to infer semantic relationships among the annotated resources. Specifically, the personalization approach we describe in this paper proposes new matching techniques based on inferring semantic relationships between the user’s preferences and the available items, which would go unnoticed in traditional syntactic mechanisms. These relationships allow the recommender system to discover a huge amount of knowledge about the user’s interests, thus enabling to select more accurately items of interest for her. To illustrate synergies existing between semantic reasoning and recommender systems, our domain-independent approach defines two strategies —using content-based and collaborative filtering, respectively– that can be either separately applied or mixed in an advanced hybrid technique. On the one hand, our content-based approach mitigates the overspecialization of the traditional sugges- • tions by using a new similarity metrics that reasons about the semantics of the compared items. Thanks 8 A Semantics-Based Approach Beyond Traditional Personalization Paradigms 91

to the flexibility enabled by semantics, our metrics measures resemblance between items that do not share identical attributes, thus endowing the offered recommendation with a rich diversity. Regarding the collaborative strategy, our reasoning capabilites are exploited during both the process of • neighborhood formation for each user, and the selection of her personalized recommendations. This way, it is possible to: (i) detect similarity between users’ profiles that do not contain common items, and (ii) suggest items that have not been rated by any user in the system, thus alleviating the adverse effects of both the sparsity problem and the latency concerns present in current collaborative approaches. This paper is organized as follows: Section 8.2 focuses on the two key components of our reasoning framework: the ontology where the semantic annotations of the items available in the personalization tool are formalized, and the techniques employed to model the users’ preferences. Next, Sections 8.3 and 8.4 describe in detail how our content-based and collaborative filtering strategies mitigate the intrinsic drawbacks of the traditional paradigms. Then, Section 8.5 highlights aspects related to the experimental evaluation of our strategies in different personalization system for Digital TV, and it discusses concerns about scalability and computational viability of our approach. Finally, Section 8.6 concludes the paper and points out possible lines of further research.

8.2 The Reasoning Framework 8.2.1 The Domain Ontology In the field of the Semantic Web, an ontology is a formal specification of a conceptualization, that is, an abstract and simplified view of the world that we wish to represent, described in a language that is equipped with a formal semantics [Berners-Lee et al., 2001]. An ontology characterizes that semantics in terms of concepts and their relationships, which are represented by classes and properties, respectively. Both entities are hierarchically organized in the conceptualization, which is populated by including specific instances of both classes and properties. As an example, note that in the context of a recommender system, instances of classes represent the available items and their attributes, whereas instances of properties link the items and attributes each other. As a sample of this formalization process, we depict in Fig. 8.1 a brief excerpt from an ontology for the domain of television. Here it is possible to identify several instances that identify specific TV programs, which belong to a hierarchy of classes referred to different genres (e.g. Fiction, Sports, Music, Leisure). As we commented before, the attributes of these TV contents (e.g. cast, intented audience, topics) are also identified by hierarchically organized classes, and are related to each program by means of labeled properties (e.g. hasActor, hasIntendedAudience, isAbout). To define these semantic annotations it is possible to reuse existing collection of metadata that describe in detail the specific domain of application. For instance, in the TV field, the TV- Anytime metadata specification [TV-Anytime Forum, 2001] provides very detailed description about generic audiovisual contents, thus greatly simplifying the formalization process. Ontologies have become the cornerstone in the Semantic Web due to two reasons. On the one hand, as these conceptualization represent formally a specific domain, they permit to employ inference processes to discover new knowledge from the formalized information. On the other hand, ontologies facilitate automated knowledge sharing, by allowing an easy reusing between users and software agents. To this aim, several standard implementation languages have been developed in the Semantic Web. The former proposals were RDF [Beckett, 2004] and RDFS [Brickley and Guha, 2004], which added a formal semantics to the purely syntactic specifications provided in XML. Next, DAML [DAML, 2000] and OIL [Fensel et al., 2001] arose, which have been finally fused and standardized by W3C as OWL [McGuinness and van Harmelen, 2004]. Nowadays, OWL is the most expressive language in which three sub-levels have been defined (Lite, DL and Full). In this regard, note that the language used to implement the ontology required in our reasoning approach depends on the knowledge and expressive necessities of each application domain and each recommender system.

8.2.2 The User Modeling Technique To reason about the user’s preferences, our approach needs a formal representation including semantic descriptions of the items which are interesting and unappealing to him/her (named positive and negative preferences, 92 Yolanda Blanco-Fernández, Alberto Gil-Solla and Manuel Ramos-Cabrer

[4] [3] Aboutyour M.Schumacher Idols F1race Formula1 Interviews

NON-FICTION Documentaries CONTENTS [2] Motor Sports SPORTS Tom FICTION Comedy TheMoneyPit Hanks

CONTENTS Kartracing

¡ ¡ ¡ ¡ ¡ ¡ ¡ ¢ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¢ ¡ ¡ ¡ ¡

[2]

Karting

’

You veGot

Mail Jacky

[3]

Blues&Soul

Woo

Children

[5]

Total

MUSIC

Snowboarding

Age Age

CONTENTS

Snowboarding

8-10 5-7

JazzinLive

[1]

Winter

[1]

Cookingwith Magic Skiingfor [5] Sports

Madeat Circus

Home&Garden Karlos circus Kids

Home Clown

Skiing

Cookery Do-It-Yourself Gardening Circus Sport

LEISURE Properties Classes CONTENTS [1]HasIntendedAudience [4]HasInterviewee SubClassOf [2]HasActor [5]IsAbout InstanceOf Instances [3]HasCommentator

Fig. 8.1. A brief extract from an ontology about the TV domain. respectively). These descriptions permit the recommender system to learn new knowledge about the user’s interests, which will be efficiently employed during the reasoning-based recommendation process. This requirement prevents the current user modeling proposals from being applied in our approach: Some existing works define too simple user models, which only contain flat lists of key words (e.g. at- • tributes) or ratings referred to each item defined in the user’s profile [Shardanand, 1994, Kobsa et al., 2001, Hill et al., 1995]. These proposals provide little knowledge about the user’s preferences, and therefore hamper the application of advanced inference processes like those aimed in our approach. Other more sophisticated proposals take advantage of the hierarchical structures defined in an ontology • to model the user’s preferences [Middleton, 2003, Ziegler et al., 2004]. The profiles defined in these approaches do not contain the specific items the user (dis)liked in the past, but the categories under which these items are classified into a hierarchy previously defined. The main drawback of this approach is that only explores the hierarchical structure of the domain and misses the semantic descriptions of items, which are specially useful for user modeling tasks (and for later reasoning processes applied on them). Bearing in mind that the descriptions required in our reasoning processes are already defined in the domain ontology, we propose to model the user’s preferences by reusing the knowledge formalized in it. The resulting models are named ontology-profiles and store the interest of the user in: (i) the attributes of the items which are (un)interesting for him/her, and (ii) the hierarchy of categories under which these items are classified in the ontology. Our user modeling proposal has two main advantages for a recommender system: On the one hand, the formal representation of the user’s profile allows the system to reason and compare • effectively her preferences with the available items, thus favoring more accurate personalization processes. On the other hand, even though our modeling approach provides the system with a very detailed model • of the user’s interests, it does not require that the classes, properties and instances that identify these preferences in the domain ontology to be stored in each profile. Therefore, our ontology-profiles significantly reduce the storage capabilities needed in a reasoning-based recommender system. 8 A Semantics-Based Approach Beyond Traditional Personalization Paradigms 93

To achieve this last goal, we use the domain ontology as a common knowledge repository, in such a way that only two elements are defined in the user’s profile: unique references (denoted by IDs) to identify in the domain ontology the items the user (dis)liked, and her specific level of interest in each one of them. The mentioned references permit to locate in the ontology the items defined in the user’s profile and to query their semantic descriptions (i.e. attributes and hierarchical categories) over the conceptualization, as shown in Fig. 8.2 for a recommender system in TV domain. In this regard, note that our modeling technique does not consider a flat list of attributes referred to the user’s preferences, but it exploits the structure of the domain ontology and the relationships existing among these attributes in order to learn knowledge about her interests and use it during the personalization process.

ProfilePX ProfilePY

DOI indexes DOI indexes ID2 ID1

DOI DOI indexes indexes ID1 ID3

Actor

ID2 ID1 Director Topic Instancesinthe domainontology Alert Intended ID Audience 3

Cookery Drama Adventure

Tourism Hierarchyofclasses LEISURE inthedomainontology FICTION CONTENTS CONTENTS Gardening TV CONTENTS

InstanceOf SubClassOf Properties Fig. 8.2. Our ontology-based approach for modeling user in a TV recommender system.

In order to measure the user’s level of interest in each item identified in her profile, we have defined the so-called DOI indexes (Degree Of Interest, in the range [-1,1]), which can be either explicitly stated by the user or automatically inferred by the recommender system from the relevance feedback provided after recommendations. Specifically, the DOI computed for each item is also used to set the ratings corresponding to its attributes and to the categories under which the item is classified in the ontology. The attributes of a given item inherit its DOI index, in such a way that if an attribute is related to several items, we calculate its DOI index by averaging their respective levels of interest. Regarding the computation of the DOI index of the classes, we firstly compute the DOI of the lowest classes in the ontology hierarchy as the average value of the 94 Yolanda Blanco-Fernández, Alberto Gil-Solla and Manuel Ramos-Cabrer

DOI indexes assigned to the items in the profile belonging to those classes. Then, we propagate these values through the hierarchy until reaching its root class. For that purpose, we adopt the approach proposed by Ziegler et al. [2004], that leads to higher DOI indexes for the superclasses closer to the leaf class whose value is being propagated, and lower ones for the classes which are closer to the root of the hierarchy. In addition, the higher the DOI of a given class and the lower the number of its siblings, the higher the DOI index propagated to its superclass. As a class can be superclass to multiple classes, every time its DOI is updated, we add the indexes of all of its subclasses defined in the profile, and so compute its final DOI index.

8.3 Beyond Traditional Content-based Strategies

In order to fight the limitations of the traditional content-based filtering approaches, it is necessary to employ metrics that match the user’s preferences against the available items in a flexible and accurate way. To achieve this flexibility, our semantics-based approach proposes a novel metrics that reasons about the semantics of the compared items, instead of using syntactic matching mechanisms.

8.3.1 Our Reasoning-based Approach

Thanks to the reasoning, our semantic similarity metrics detects that two items are similar if they are semantically related, even when their respective attributes are different. In this regard, note that we consider two kinds of relationships which lead to the two components identified in the proposed metrics (named hierarchical and inferential semantic similarity). Both components are finally weighted and added to obtain the final similarity value. Hierarchical semantic similarity. In literature, it is possible to find numerous similarity metrics which • measure resemblance by looking at the hierarchical relationships established in a taxonomy [Resnik, 1999, Lin, 1998, Rada et al., 1989]. According to these proposals, the hierarchical similarity measured between two items depends on the existence and the depth1 of a class which is ancestor of both of them in a hierarchy. This way, the deeper the lowest common ancestor (henceforth LCA) of the two items in the taxonomy, the higher the value of hierarchical semantic similarity between them, since both items are strongly related to each other. Inferential similarity. As explained by Ganesan et al. [2003], “taxonomy-based approaches do not ac- • curately capture similarity in certain domains, such as when the data is sparse or when there are known relationships between the compared items”. For that reason, we extend the existing similarity metrics by mixing the hierarchical relationships included in the domain ontology, with other associations hidden behind the properties explicitly defined in it. Specifically, two items are similar if they share some attributes (named union instances), or if they have different attributes belonging to the same class in some hierarchy (named union class). Thanks to the two components of our semantic similarity metrics, our content-based filtering diversifies the recommendations by offering both items classified in the ontology under classes hierarchically related to those the user liked in the past, and items that share attributes (identical or sibling) with her positive preferences, as we detail in the following example.

8.3.2 A Sample Scenario

Consider a family made up of a couple and two children (6 and 9 years-old, respectively), who have viewed some of the TV programs that are depicted in Fig. 8.1 along with their TV-Anytime attributes. Speciﬁcally, parents enjoyed a documentary about snowboarding and a romance comedy starring Tom Hanks, whereas kids liked a Circus show intended for 8-10 years-old children. 1 The depth of a class in a taxonomy is deﬁned as the number of hierarchical links between it and the root node. 8 A Semantics-Based Approach Beyond Traditional Personalization Paradigms 95

In this scenario, the syntactic metrics used in traditional content-based TV recommender systems would • suggest programs that share attributes with the preferences deﬁned in the family’s proﬁle, thus leading to overspecialized recommendations. Out of all available TV contents in Fig. 8.1, these approaches would select other Circus program for children, and a comedy starring Tom Hanks for their parents (also suitable for kids). On the contrary, a TV recommender system using our enhanced content-based strategy resorts to reasoning • to select programs which: (i) are appealing to all members of the family, and (ii) are not excessively similar to those they already know. In this scenario, we assume that the parents could be interested in their children learn some type of winter sport, in view of their interest in snowboarding. For that reason, we suggest Skiing for Kids, whose relevance is detected by the relationships discovered between this program and the family’s preferences, as shown in Fig. 8.3.

Unionclass= {WinterSports} Unionclass= {Children}

Total Skiingfor Magic Snowboarding Kids circus

LCA = {LeisureContents}

Fig. 8.3. Some relationships between TV programs discovered by semantic reasoning.

On the one hand, Skiing for Kids is related to Total Snowboarding because both programs are about winter sports. This permits our inferential similarity metrics to detect a union class between them2, which increases the relevance of Skiing for Kids for our target family. On the other hand, Skiing for Kids is also related to Magic Circus from two points of view. Firstly, both programs are classiﬁed under the Leisure Contents category in the TV ontology, hence the fact that our approach increases the value of hierarchical similarity between them. Secondly, as shown in the area marked by dotted lines in Fig. 8.1, Magic Circus and Skiing for kids are intended for different age groups belonging to Children collective. This fact allows to detect a union class between both contents, thus increasing their inferential similarity.

Lastly, note that the program suggested by our approach not only diversiﬁes the offered recommendation, but also it does match the preferences of all the member of the family: Skiing for Kids would be appealing to the parents because it is about winter sports, and it would also be interesting for children because belongs to their favorite category (Leisure Contents).

8.4 Beyond Traditional Collaborative Strategies

8.4.1 Our Reasoning-based Approach

To fight the limitations of the existing collaborative filtering approaches, we employ semantic reasoning processes both in the phase of neighborhood formation, and in the prediction of the interest of the user in a generic item (not necessarily rated in profiles available in the recommender system). In order to form the user’s neighborhood, we extend the “collaboration via content” paradigm of Pazzani [1999], so that we reason about the semantics of the users’ preferences, instead of using only their content descriptions. This way, in order to form a user’s neighborhood, our approach proceeds as follows:

2 The union class Winter Sports is shown in an area marked by lines in Fig. 8.1. 96 Yolanda Blanco-Fernández, Alberto Gil-Solla and Manuel Ramos-Cabrer

1. Firstly, we propose a taxonomy-based approach to create a rating vector for the user. Instead of including her levels of interest in the available items, our vector contains her ratings in relation to the categories under which these items are classified in the ontology. Actually, the rating vector of a user only includes the DOI indexes of the classes most significant for the personalization process, that is, the classes which are most appealing and most unappealing to him/her (identified by DOI indexes close to 1 and to -1, respectively). We select these categories by using a threshold ψ, in such a way that the (absolute value of) DOI indexes of all the classes defined in the user’s rating vector must exceed the established threshold3. 2. Once the user’s rating vector has been created, we select those users who have rated items belonging to the most of classes defined in it. Next, we create their respective rating vectors by including their DOI indexes in these classes. 3. Finally, we compare their preferences by computing the Pearson-r correlation [Middleton, 2003] between their respective rating vectors. The N users with the highest correlation values w.r.t. the considered user are finally included in her neighborhood. In this regard, note that the hierarchical organization of the categories included in the user’s rating vector fights the sparsity problem of the traditional collaborative approaches, as it allows to detect that the preferences of two users are similar even when their respective profiles do not contain either items or identical attributes. In our approach, it is only necessary that the categories of the considered items share a common ancestor in the hierarchy defined in the domain ontology. Once the user’s neighbors have been identified, it is necessary to predict the user’s level of interest in the considered item. In contrast with current collaborative approaches —which only consider the contribution of the neighbors who have already rated this item— we explore the full neighborhood of the user, by exploiting the benefits from our reasoning mechanisms. This way, if a neighbor has rated the considered item, the prediction is based on the level of interest defined in her profile; otherwise, we predict this level by measuring the semantic similarity between her preferences and the target item. This process of neighbors’ ratings estimation is especially beneficial to our collaborative phase from two points of view: On the one hand, it enables to suggest (without unnecessary delays) items which are completely novel for • all the users in the system. Therefore, we eliminate the latency problem of the traditional approaches, in which an item must be rated by many users before it can be suggested. This improvement is especially important in some domains, such as the TV field, where a large number of programs are continuously appearing in and disappearing from the broadcast database, staying in the recommender system for a short time. On the other hand, the estimation of neighbors’ ratings increases the accuracy of our hybrid strategy w.r.t. • the traditional approaches, by selecting accurately many items appealing to the user which would go unnoticed in the existing collaborative techniques. Our approach discovers these items because predicts in advance that they are also interesting for the user’s nearest neighbors, as we will see in the following example.

8.4.2 A Sample Scenario

Suppose three users who independently watch TV, whom have interested the programs depicted in Table 8.1. In this example, we explore the recommendations that a collaborative approach would suggest to the user U2 by considering the preferences of the remaining viewers in the system.

U1’s preferences U2’s preferences U3’s preferences Home & Garden Made at Home Made at Home Jazz in Live Blues & Soul Blues & Soul Cooking with Karlos About your Idols Table 8.1. TV programs appealing to the viewers registered in the recommender system.

3 By its very nature, ψ must take values close to 1 (which is the maximum value of the DOI indexes). 8 A Semantics-Based Approach Beyond Traditional Personalization Paradigms 97

The traditional collaborative approaches would only suggest to U2 programs which her neighbors already • know. Specifically, these approaches would select U3 as U2’s neighbor, in view of the overlap between the programs defined in their profiles (Made at Home and Blues & Soul in Table 8.1). Consequently, U2 is suggested the program About your Idols defined in U3’s profile. On the contrary, our reasoning-based collaborative approach is able to suggest programs which are com- • pletely novel for all the users in the system. Suppose the F1 race depicted at the top of Fig. 8.1, which is commented by the retired driver Michael Schumacher. In contrast with traditional proposals, our taxonomy-based approach selects both U1 and U3 as U2’s neighbors: although some of them have not viewed exactly the same programs, they like the same categories in the TV ontology (specifically, Leisure Contents and Music Contents, as shown in the first two rows in Fig. 8.4, respectively). As U2’s neighbors do not know F1 race, our collaborative strategy estimates their ratings in this program and discovers that it is appealing to both viewers. This process involves the relationships inferred between F1 race and both neighbors’ preferences, which are represented at the bottom of Fig. 8.4. On the one hand, we predict that the neighbor U1 is interested in F1 race since she likes the motor sports (as shown in upper right corner of Fig. 8.1, Kart racing and F1 race belong to the subcategories Karting and Formula1, respectively). On the other hand, it is also to be expected that F1 race is appealing to the neighbor U3, because it involves the driver Michael Schumacher whom she is interested (as shown in Table 8.1, U3 has viewed an interview to this driver in About your Idols).

LCA = {LeisureContents}

Cookingwith Home&Garden Karlos

LCA = {LeisureContents} Madeat LCA = {LeisureContents} Home

LCA = {MusicContents}

Blues&Soul JazzinLive

LCA = {MotorSports/Sports} Unioninstance= {M.Schumacher}

Kartracing F1race Aboutyour Idols

Fig. 8.4. Some relationships between TV programs discovered by semantic reasoning.

Finally, according to the discovered interest of U2’s neighbors in F1 race, our collaborative approach suggests this program to the viewer.

In accordance with the described example, it is clear that our reasoning-based approach exploits the semantics the users’ preferences with the goal of alleviating greatly the negative effects of unresolved limitations in collaborative systems, such as sparsity problem and delays in recommendation of new items. 98 Yolanda Blanco-Fernández, Alberto Gil-Solla and Manuel Ramos-Cabrer 8.5 Evaluation and Concerns about Computational Viability and Scalability

Although our semantics-based approach is generic enough to be used in multiple domains and personalization tools, our testing and validation tasks have been carried out in the specific field of the Digital TV. As proof of the flexibility of our proposal, next we describe the diverse tools with which we had corroborate the potential of the reasoning capabilities of our strategies in this context.

AVATAR. Nowadays, TV viewers are exposed to overwhelming amounts of information, and challenged • by the plethora of interactive functionality provided by the current digital receivers. As there are hundreds of channels with an abundance of programs available and large amounts of material that can be retrieved from digital video archives and satellite streams, it is likely that the contents that are appealing to the viewer go unnoticed. As a result, TV viewers waste a lot of time browsing the available contents or end up watching a very limited number of channels. To fight these drawbacks, we take advantage of the personalization capabilities provided by a TV recommender named AVATAR, which sifts through the myriad of TV programs available in the digital stream and selects those that match the viewers’ preferences by resorting to a two-phase hybrid approach combining the content-based and collaborative filtering strategies described in this paper. The experimental evaluation of the AVATAR system (see [Blanco-Fernández et al., 2007] for details) showed that our reasoning-based personalization approach increases the average recall and precision4 achieved in the traditional syntactic proposals. Likewise, this evaluation permitted also to validate empirically the benefits of the process of neighbors’ ratings estimation used in our collaborative strategy. Such process yielded increments in the recommendations accuracy w.r.t. traditional approaches. From these results, it follows that: (i) our estimated ratings are accurate and reliable, and (ii) the estimation process is a relevant contribution in the collaborative strategy, since it allows to discover many programs appealing to the users, which would go unnoticed in traditional collaborative systems. ATLAS. We have recently integrated our reasoning-based strategies as a new component of ATLAS, a • platform introduced in [Pazos-Arias et al., 2006] to provide educational courses. In this context, it is not a suitable approach to leave the discovery of courses in a pull model where it is the user, on her own initiative, who starts looking for interesting material by navigating through the Electronic Programming Guides (EPGs). While leaving place for that possibility, we implemented an alternative model based on recommending courses that the user may find interesting. The recommender works by matching descriptions of the courses available with ontology-profiles that, in addition to the information described in Section 8.2.2, contain a record of previous learning activities5. The results of some preliminary experiments (see [Pazos- Arias et al., 2008] for the details) confirmed that our strategy performs better than other existing approaches in what concerns the users’ satisfaction with the recommended courses. MiSpot. Lastly, we are also working in the integration of our recommendation techniques into the MiSpot • system [López-Nores et al., 2007], a technological framework that enables new and more effective forms of publicity in the IDTV field, both for in-home and mobile TV settings. MiSpot combines two basic ideas: non-invasiveness, to ensure that publicity does not interfere with the viewers’ enjoyment of the audiovisual contents; and interactivity, to deliver online trading services through the TV. In this context, a personalization engine with the kind of abilities discussed in this paper comes as a crucial element to turn advertising into a source of useful information for each individual viewer and, thereby, maximize the revenues of publicity.

Even though the scenarios employed in our empirical evaluations are realistic enough (with about 400 users), several reasons lead to think that our approach is also computationally feasible in more complex scenarios with a greater number of users and items:

4 In our experiments, recall is defined as the percentage of programs appealing to the viewer that are suggested by the strategy. Precision is the percentage of the suggested programs that are interesting for the viewer. 5 The metadata for the learning-specific features was borrowed from the SCORM specifications [Advanced Distributed Learning (ADL), 2006]. 8 A Semantics-Based Approach Beyond Traditional Personalization Paradigms 99

The first one is related to the fact that a recommender system has no real-time requirements. The users are • not continuously checking their recommendations, but they log into the system with the goal of receiving a personalized suggestion when it is available. As the system knows the user’s preferences and the available items in advance, the computational complexity of our approach does not cause a unacceptable delay in the delivery of recommendations. Secondly, note that the maintenance tasks of the domain ontology and the computational cost related to • our reasoning-based algorithms prevent us from running our strategies in a device with limited computational power. For that reason, we employ centralized servers that are supplied with substantial storage and processing capabilities, in such a way that the computational feasibility of our reasoning-based approach is guaranteed. Lastly, we summarize some measures aimed at improving the scalability of the proposed semantics-based • approach: – The first one is related to our user modeling approach. As the proposed technique models the user’s preferences from a common knowledge repository (the domain ontology), it is not necessary to store in each profile the classes, properties and instances referred to the user’s preferences. Consequently, our approach alleviates the storage capabilities needed in the server where the reasoning-strategies are executed. – Secondly, note that both the hierarchical approach employed to create the rating vectors in the collaborative strategy, and the ψ threshold used to reduce their sizes, also bring benefit in terms of scalability: On the one hand, as new items arrive to the existing collaborative systems, the size of the users’ · vectors increases and therefore, the cost of computing correlations greatly rises. In contrast with this, as the number of available items is higher in our reasoning approach, our rating vectors do not necessarily increase in size. This is due to the fact that many new items can belong to the same hierarchical classes in the ontology, thus reducing the computational complexity of our neighborhood formation process. On the other hand, the computational benefits of our collaborative approach are closely related to · the values of the ψ threshold we use. So, if we decrease ψ, the computational cost increases because the user’s rating vector would include more components. In this case, the offered recommendations would be more accurate as the collaborative strategy knows more information about the user’s preferences. On the contrary, the higher ψ, the shorter the user’s rating vector, thus alleviating the computational cost of our collaborative approach in exchange for (possibly) reducing the quality of the suggestions. For that reason, it is necessary to get a trade-off between the recommendations accuracy and the computational complexity of the personalization strategy. – Finally, to prevent the measure of semantic similarity from becoming too demanding in computational terms, we compute off-line many parameters that can be reused as new users log into the system. For instance, for each pair of items, we identify in advance their LCA, their respective depths, and the common instances they share (both union instances and instances of union classes). In conclusion, the adopted measures shape a personalization approach enhanced by semantic reasoning, whose computational feasibility permits an extended use in numerous domains and personalization tools.

8.6 Conclusions and Further Work

In this paper, we present a reasoning-based personalization approach which overcomes unresolved limitations of the traditional recommendation strategies: Firstly, the proposed content-based strategy leads to diverse enhanced recommendations by removing the • noticeable overspecialized nature of the traditional suggestions. Instead of offering items with the same attributes as those deﬁned in the user’s proﬁle, our reasoning-based approach suggests items which are semantically related to her preferences, thus enhancing the diversity of the recommendations. These semantic relationships —discovered from an ontology where the application domain is formalized— provide additional knowledge about the user’s interests and, therefore favor more accurate personalization processes. 100 Yolanda Blanco-Fernández, Alberto Gil-Solla and Manuel Ramos-Cabrer

Secondly, the proposed collaborative strategy allows to select a user’s neighbors even when the data about • their preferences are very sparse. To this aim, we take advantage of the knowledge represented in the domain ontology and the semantic relationships which can be inferred from it. Our collaborative strategy permits also to suggest (without unnecessary delays) items which are completely • novel for all the users in the recommender system. Instead of waiting until these new items are rated by a significant number of users, our approach predicts accurately their ratings and uses them during the elaboration of the collaborative recommendations. In accordance with the sample examples described in the paper, our approach is flexible enough to be used both in a single-user model of recommendation and in a scenario where the target audience is a group of users using simultaneously the offered suggestions (e.g. a family viewing TV together, a group of friends listening music together, tourists who plan to spend holidays in a group and are looking for a destination). In the first case, the proposed strategies focus on each independent user, whereas in the second scenario our reasoning approach guarantees that the offered suggestions match the preferences of all the group members. As our reasoning-based strategies are executed in a centralized server which stores all the users’ preferences, it is necessary to clarify some concerns related to their privacy. Firstly, we suppose that this server is reliable and will never reveal information about the users’ personal preferences. Also, even though the collaborative recommendations offered to a user are based on her neighbors’ preferences, she never knows their identities, thus protecting their privacy. We conclude this section by highlighting the flexibility of our reasoning approach, which is not joined to an exclusive domain and therefore can be reused in multiple personalization applications and systems. In proof of its flexibility, our approach has been incorporated into diverse personalization tools in the field of Digital TV, such as a TV recommender system, a t-learning platform, and a tool that delivers personalized advertisements. Our experimental evaluation revealed: (i) enhanced accurate recommendations, (ii) significant increments of recall and precision w.r.t. traditional syntactic approaches, which are devoid of our reasoning capabilities, and (iii) computational viability thanks to refinements aimed at reducing the cost and improving the scalability of our approach. Regarding future work, we plan to refine our personalization strategies by combining semantic reasoning capabilities with information about the user’s context (e.g. mood, geographical and temporal localization). Consequently, this research line would lead to intelligent tools that could adapt the offered suggestions to the particular situation of each user at any moment.

References

G. Adomavicius and A. Tuzhilin. Towards the next generation of recommender systems: A survey of the state- of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6):739–749, 2005. Advanced Distributed Learning (ADL). Sharable Content Object Reference Model (SCORM). http://www.adlnet.org, 2006. D. Beckett. RDF syntax specification. http://www.w3.org/TR/rdf-syntax-grammar, 2004. T. Berners-Lee, J. Hendler, and O. Lassila. The Semantic Web: A new form of web content that is meaningful to computers will unleash a revolution of new possibilities. The Scientific American, 279(5):34–43, 2001. Y. Blanco-Fernández, J. J. Pazos-Arias, A. Gil-Solla, M. Ramos-Cabrer, M. López-Nores, and J. Bermejo- Muñoz. AVATAR: Enhancing the personalized television by semantic inference. International Journal of Pattern Recognition and Artificial Intelligence, 21(2):397–422, 2007. D. Brickley and R. Guha. RDF vocabulary description language 1.0: RDF Schema. http://www.w3.org/TR/rdf- schema, 2004. R. Burke. Hybrid recommender systems: Survey and experiments. User Modeling and User-Adapted Interac- tion, 12(4):331–370, 2002. DAML. The DARPA Agent Markup Language. http://www.daml.org, 2000. D. Fensel, F. van Harmelen, I. Horrocks, D. McGuinness, and P. Patel-Schneider. SOIL: An ontology infrastructure for the Semantic Web. IEEE Intelligent Systems, 16(2):38–45, 2001. 8 A Semantics-Based Approach Beyond Traditional Personalization Paradigms 101

P. Ganesan, H. Garcia-Molina, and J. Widom. Exploiting hierarchical domain structure to compute similarity. ACM Transactions on Information Systems, 21(1):64–93, 2003. D. Goldberg, D. Nichols, B. M. Oki, and D. Terry. Using collaborative filtering to weave an information tapestry. Communications of the ACM, 35(12):61–70, 1992. K. J. Hammond, R. Burke, and S. L. Lytinen. A case-based approach to knowledge navigation. In International Joint Conference on Artificial Intelligence, Montréal, Canada, 1995. W. Hill, L. Stead, M. Rosenstein, and G. Furnas. Recommending and evaluating choices in a virtual community of use. In International Conference on Human Factors in Computing Systems, Denver, Colorado (USA), 1995. A. Kobsa, J. Koenemann, and W. Pohl. Personalized hypermedia presentation techniques for improving online customer relationships. Knowledge Engineering Review, 16(2):111–155, 2001. D. Lin. An information-theoretic definition of similarity. In 15th International Conference on Machine Learn- ing, Madison, Wisconsin (USA), 1998. M. López-Nores, J. J. Pazos-Arias, Y. Blanco-Fernández, J. García-Duque, R. Tubío-Pardavila, and M. Rey- López. E-Business and Telecommunication Networks - Selected papers from ICETE 2007, chapter The MiSPOT system: Personalized publicity and marketing over interactive Digital TV. Springer, 2007. D. Maltz and K. Ehrlich. Pointing the way: Active collaborative filtering. In ACM Conference on Human Factors in Computing Systems, Denver, Colorado (USA), 1995. D. L. McGuinness and F. van Harmelen. OWL Web Ontology Language Overview. W3C Recommendation, February 2004. S. Middleton. Capturing knowledge of user preferences with recommender systems. PhD thesis, Department of Electronic and Computer Science, University of Southampton, 2003. J. J. Pazos-Arias, M. López-Nores, J. García-Duque, A. Gil-Solla, M. Ramos-Cabrer, Y. Blanco-Fernández, R. R. Díaz-Redondo, and A. Fernández-Vilas. ATLAS: A framework to provide multiuser and distributed t-learning services over MHP. Software–Practice & Experience, 36(8):845–869, 2006. J. J. Pazos-Arias, M. López-Nores, J. García-Duque, R. P. Díaz-Redondo, Y. Blanco-Fernández, M. Ramos- Cabrer, A. Gil-Solla, and A. Fernández-Vilas. Provision of distance learning services over interactive Digital TV with MHP. Computers & Education, 50(3):927–949, 2008. M. Pazzani. A framework for collaborative, content-based and demographic filtering. Artificial Intelligence Review, 13(5):393–408, 1999. R. Rada, H. Mili, E. Bicknell, and M. Blettner. Development and application of a metric on semantic nets. IEEE Transactions on Systems, Man, and Cybernetics, 19(1):17–30, 1989. P. Resnik. Semantic similarity in a taxonomy: An information-based measure and its application to problems of ambiguity in natural language. Journal of Artificial Intelligence Research, 11(4):95–130, 1999. U. Shardanand. Social information filtering for music recommendation. PhD thesis, Program in Media Arts and Sciences, Massachusetts Institute of Technology, 1994. B. Smyth and P. Cotter. Surfing the digital wave: Generating personalised TV listings using collaborative, case-based recommendation. Lecture Notes in Computer Science, 1650:561–571, 1999. TV-Anytime Forum. TV-Anytime specification series S-3 on metadata. http://www.tv-anytime.org, 2001. C. N. Ziegler, L. Schmidt-Thieme, and G. Lausen. Exploiting semantic product descriptions for recommender systems. In 2nd ACM SIGIR Semantic Web and Information Retrieval Workshop, Sheffield, United Kingdom, 2004.

Part II

Applications

9 PROGNOSIS: An Adaptive System for Virtual Technical Documentation

Nikolaos Lagos and Rossitza M. Setchi

Knowledge Engineering Systems Group, University of Cardiff Cardiff School of Engineering, CF24 3AA, United Kingdom {lagosn,setchi}@Cardiff.ac.uk

Summary. User-tailored delivery has been one of the main themes for several years in ﬁelds such user interfaces design, human-computer interaction and web-based systems. However, personalisation (i.e. the ability to deliver information that most closely corresponds to an individual’s proﬁle) has been largely based on reasoning techniques that utilise solely user characteristics as a resource. The adaptation process has been therefore considered as independent of the time, place and intention of the user. The research reported in this chapter presents PROGNOSIS, an adaptive virtual technical documentation system that integrates the above aspects within the domain of product support. That is achieved by modelling the structure, relations and attributes of the product, task, context and documentation knowledge. In that model a Virtual Documentat (VD) is presented as an aggregation of Information Objects (IOs) and In- formation Object Clusters (IOCs). The description of VD is followed by an analysis of the relation between IOs, IOCs and domain knowledge. The research builds on the ontology-based representation of technical documentation knowledge and explores the synergy between product support, problem solving and knowledge engineering.

9.1 Introduction

A PRoduct Support System (PRSS) aims to alleviate the lack of knowledge of the user in a particular subject or situation related to a supported product by providing user-tailored information just-in-time and just-in- place. Researchers in this field have utilised the Internet and digital technologies as enablers of achieving the “just-in-time” and “just-in-place” requirements. User-tailored delivery on the other hand has been one of the main themes for several years in fields such user interfaces design, human-computer interaction and web-based systems. However, personalisation (i.e. the ability to deliver information that most closely corresponds to an individual’s profile) has been largely based on reasoning techniques that utilise solely user characteristics as a resource. The adaptation process has been therefore considered as independent of the time, place and intention of the user. Recently context has received attention as a way of representing different situations in a universe of discourse. It has been mainly used in mobile applications where location is a very important aspect towards achieving personalisation. Attempts to use other context information as well have increased over the last few years [Baldauf et al., 2007]. The increasingly widespread use of context in different applications indicates the trend towards adaptive information delivery. However, to date the integration between domain and context knowledge in the field of support systems has been limited to the representation of user characteristics. The main assumptions underlying this work are two. First, in addition to user specific data, context should include other data used to characterise a situation such as time. Second, domain and context knowledge can be integrated within a PRSS and utilised to generate context-tailored information. The objective is to transform product support systems into a medium that not only offers just-in-time support but also enables users to improve their skills by acquiring context-specific information. 106 Nikolaos Lagos and Rossitza M. Setchi 9.2 Background

9.2.1 Product Support Systems and Professional Learning

Pham et al. [1999] define product support as everything necessary to allow the continued use of a product. It takes various forms, ranging from conventional paper-based technical manuals to more advanced interactive electronic technical manuals (IETMs), intelligent product manuals (IPMs) and electronic performance support systems (EPSSs). An IETM is defined as a technical manual authored in digital form and designed for electronic display by means of an automated authoring system [Jorgensen and Fuller, 1998]. An IPM is referred to as a computerised interactive product support system that uses product life-cycle information, expert knowledge and hypermedia to provide just-in-time support to the user [Pham et al., 1999]. Finally, an EPSS is regarded as an integrated, readily available set of tools that help individuals to do their job and increase their productivity [Bezanson, 1995]. The development of such systems is based on the following key principles: adopt the perspective of the performers of the job; facilitate incremental learning, or learning while carrying out a task; allow performers to control their own training; and give them the ability to access information and knowledge at the time they need it. Translated in technical terms, these key principles required task-specific access to information and knowledge; just-in-time, on-the-job training; learning controlled by the worker and accommodating different levels of knowledge and learning styles. Subsequently, the idea “to provide the right information to the right person at the right time” [Lindsey and Flowers, 1995] included research into user [Jokela, 2004] and performance- cantered [Marion, 2002] design, pedagogical models [Bradley and Oliver, 2002], the use of knowledge engineering techniques for representing product support knowledge [Setchi et al., 2006] and personalisation [Pham and Setchi, 2003]. In addition, recent studies suggest that there is a noticeable shift in responsibility for learning from organizations to individuals [Garofano and Salas, 2005]. Employees are expected to take greater personal responsibility to ensure that their skills are current. This trend has made it more difficult for individuals to base their careers on established paths [Bosley et al., 2007] and has led to the development of a work-based learning philosophy which embraces independent self-managed study. This review shows that there is a demand for flexible, tailored and just-in-time delivery of information that can be accessed quickly, widely and cost-effectively by employees. Organizations have to tailor their approaches based on the subject and the individual.

9.2.2 Context

A context model is needed to define and store context data in a machine processable form. Baldauf et al. [2007] summarise the most relevant context modelling approaches based on the data structures used for representing and exchanging contextual information. In their work ontologies are identified as “a very promising instrument for modelling contextual information due to their high and formal expressiveness and the possibilities for applying ontology reasoning techniques”. The term “ontology” itself is borrowed from philosophy, where it has a long history in referring to the subject of existence. In IT systems, ontologies are used to express knowledge about concepts (classes of subjects) and their attributes, as well as their interrelationships [Gruber, 1993]. Although context has never been used in product support applications, context modelling is an emergent area in services oriented and ubiquitous computing, where location and time are of main importance. For example, Kwon [2006] represent the context of services based on a classification initially proposed by Abowd et al. [1999], which involves the TILE (Time-Identity-Location-Entity) model. The time context is usable when the user enters a service zone and if the service is available at that time, then the service is working for the user. Identity context indicates the people with who the person offering the service communicates. Location identifies the place where an entity of interest can be found and entity context can be described as the data of sensed objects. Binemann-Zdanowicz et al. [2004] utilise context in the area of web information systems (WIS) design. More specifically, their work focuses on complex non-monolithic WIS, which include different media types and dynamic databases. They base their context modelling on story spaces (i.e. integration of all scenarios in a 9 PROGNOSIS: An Adaptive System for Virtual Technical Documentation 107 story) and identify context as a three place predicate C(S, H, A) where A is an actor that needs helper H to act reasonably on S. Thus they identify different contexts according to the actors interacting with the system, such as customers and developers. Preuveneers et al. [2004] advance the state of the art by integrating location and time as environmental variables and incorporating the device’s characteristics with the user, environment and services offered. The resultant structure is an ontology describing context in the area of ambient intelligence. Souchon et al. [2002] introduce the idea that in addition to location, time, platform and users, the context can also influence the task models architecture. Both of them argue that there are context-sensitive and context- insensitive tasks (called context-independent) and that task models can be configured according to the context of use.

9.3 Modelling Context for Product Support Systems

9.3.1 Deﬁnition of Context of Use

Context is a set of implicit or explicit stimuli surrounding an individual; it consists of the physical or social environment, which may affect behavior Kwon [2006]. Context-awareness provides computing environments with the ability to usefully adapt the services they provide. It is the ability to implicitly sense and automatically derive the user needs that distinguishes context-aware systems from traditional applications, and this makes them more attentive, responsive and aware of their user’s identity and environment [Prekop and Burnett, 2003]. In this work, the context of use is represented with the following models.

1. Activity Model (or Purpose Model) (AM), which is a finite set a1, a2, . . . , an where ai stands for a specific activity indicating the specific usage of the system. In this study two main abstract activities are discussed, “perform” and “learn”. 2. User Model (UM), which is a finite set u1, u2, . . . , uk where ui represents a user stereotype. 3. Physical Model (PM), which is a finite set p1, p2, . . . , pm where each pi stands for any hardware or software related property, such as an operating system or a graph-ics card memory. 4. Environment Model (EM), which is a finite set e1, e2, . . . , el where each element represents environmental conditions (e.g. location).

A context instantiation Ci, is denoted by the aggregation of the aforementioned models’ instantiations and can be represented as < ai, ui, pi, ei >. In this study UM and AM are utilised to illustrate the notion of context- aware product support systems while PM and EM are not extensively analysed since distributed, service-based and mobile PRSSs (where physical and environmental conditions frequently change) are not within the scope of this work. However, in order to create an extensible representation all models are included in the context ontology, as illustrated in the next section.

9.3.2 Ontology of Context

Device Characteristics

The Physical configuration of a device is regarded as a compositional concept, which can be largely defined according to its Software and Hardware aggregates (Fig 9.1). Each of these concepts can in turn be described using the specialisation relation, as in the case of Hardware, which can be further classified into CoreCompo- nent, StorageDevice and Peripheral concepts. Peripherals are utilised by the user in order to access the physical device. Consequently, Peripheral is linked with the User concept with the relation uses, which enables the system to adapt its display accordingly (e.g. if a small monitor is only supported as in mobiles, the resolution of words can change accordingly). 108 Nikolaos Lagos and Rossitza M. Setchi

Environment Characteristics

A physical component (e.g. the PRSS) should be able to sense the environment and capture any external stimuli directly related to its behaviour. For example, if someone uses the PRSS in a noisy environment, the audio multimedia components of the support content may be disabled and be reactivated once the user moves to quiet surroundings. There are numerous factors that can deﬁne the Environment concept, which are represented as its aggregates. These include Time (very important for expressing temporal relations), Location (one of the most signiﬁcant features in mobile and service-oriented applications), Lighting and Noise. Several more can be included (e.g. humidity) but the enumeration of more environmental conditions is not within the scope of this work.

Activities

The Activity concept is utilised to describe the user’s intention. Utilising Activity performance support and e-Learning techniques can be integrated within the context of a single product support system to achieve performance and learning goals. In this study Learn and Perform are used to signify the difference between those goals. As shown in Fig 9.1 the User is directly linked to the Activity with the relation performsActivity, which suggests that context information is only relevant if it enables adaptation according to user goals. Activity is also connected to Task through the spatial relation isRealisedWith. Task includes a number of subtasks that a user may want to be supported for (e.g. installation and design). Using the formalization of McCarthy [1993], the relation “IsRealisedWith” between the is equivalent to the following.

c0 : ist(cactivity(A), xtask(T )) (9.1) where A is an activity and T a task.

Users

The user is in the focus of product support. In case that a PRSS is not accepted by the users, regardless of the technology used, it cannot be considered successful. As a result, users’ characterisation is one of the most critical stages in the creation of a Context-Aware PRSS. Figure 9.1 shows that User always interacts with the system via a software application, as indicated by the usesApplication relation and he/she visits a Context-Aware PRSS in order to perform a specific Activity. Furthermore, the user can be described according to the Role assigned to him (e.g. trainer), his/her Profile (e.g. username) and Characteristics. Role and Profile are useful only for administrative purposes. On the other hand, Characteristics includes data used as parameters throughout the computation of the user’s information needs. For example, an “expert” characterisation will indicate the ability of the user to understand technical information included in a document. Expertise, Specialisation and Receptivity are identified as the main user features directly related to a PRSS. Those features are computed with Equation (9.2):

n i=0 wAi wVi C = n × (9.2) P j=0 wAj In Equation (9.2), C stands for the CharacteristicPbeing computed (i.e. Expertise, Specialisation, Receptiv- ity). The symbols wAi and wVi represent the weight of the attribute being measured and its value. For example, one of the attributes used to calculate user’s specialisation is his/her education. The significance of the attribute itself is signified with its weight wAi = 2 (where Ai now is “education”) whereas the value of the attribute can be “graduate” and equal to w Vi = 3. The total weight of the education attribute in this scenario will eventually be the product of 2 3 = 6. By summing the products corresponding to all attributes, a quantifiable value is given to the Specialisation× characteristic for the current user. If this value is over certain thresholds the user is identified as being specialised, semi-specialised or unspecialised. Also in Equation (9.2), n is the number of attributes included within each characteristic. The advantage of this approach lies in the fact that the weight of 9 PROGNOSIS: An Adaptive System for Virtual Technical Documentation 109

Fig. 9.1. Part of the context ontology for product support systems. 110 Nikolaos Lagos and Rossitza M. Setchi

each attribute wAi can change according to the usage of the system (since this can indicate changes in each attribute’s signiﬁcance throughout the life of the product), which is expected to improve the accuracy of the classiﬁcation algorithm.

9.4 Adaptation Approach

9.4.1 Correlation between Context, Tasks and Product Support Documentation

Pham and Setchi [2003] have identiﬁed and summarised the strong correlation that exists between users, tasks and support documentation. That is extended for context-aware product support systems, as illustrated in Fig 9.2.

Fig. 9.2. Correlation between task, context and product support documentation and comparison with adaptive systems. 9 PROGNOSIS: An Adaptive System for Virtual Technical Documentation 111

The learning and task performing activities are not integrated, since the user can achieve both (i.e. learning • and performing goals) by selecting the activity context he/she prefers. The content is then adapted accordingly by utilising different documentation elements. The adaptation process is still however also related to the user. The location and time can also be used to deﬁne to content of the product support documents. For example, • the service centres that a system should propose to a driver should be tailored to his/her current location. Location and time are mostly signiﬁcant in mobile, distributed and services related systems. The form or presentation of the document is not only adapted according to the user’s characteristics and • perceptual model but also in relation to environmental attributes such as current light and sound conditions. For example, being in a noisy environment should trigger volume increase for audio components.

9.4.2 Content Adaptation

Context-Aware Virtual Document

The basic component used to develop a context-aware virtual document is the Information Object (IO). IO is the smallest constituent of a virtual document (Figure 9.3). It is defined as “a data structure that represents an identifiable and meaningful instance of information in a specific presentation form” [Pham and Setchi, 2003]. IO can therefore be a picture that illustrates a part of a product or a textual description. In this study IOs are characterized according to their form, behavior, type, quality-measure, and topic (inherited from the DocumentationComponent concept (relation HasTopic)).

1. Form denotes whether the IO is an image, text, audio, video, animation, etc. 2. Type describes the nature or genre of the content of the IO, which can be explanation, deﬁnition, comment, etc. 3. Topic stands for the theme of the IO’s content (what part of the domain it refers to). 4. Quality measure stands for qualitative characteristics of the IO. For example, the level of detail can be such a measure. 5. Behaviour can be static or dynamic. Static behavior indicates that IO remains the same under all circumstances (e.g. the deﬁnition of a clutch). Dynamic behavior means that IO changes at run-time, according to the attribute of the real world object it describes (e.g. the radius of a clutch).

The notion of Information Object Cluster (IOC) is introduced, as a means of organizing IOs. IOC is defined as a 2-tuple IOC := (IO, SIOC ) where IO is a set of IOs sharing a common property that are arranged in a structure SIOC . A structure defines the way in which they are presented within the same page, as well as the relevant links. Theme, quality and type are used to identify all the IOs that belong to the same IOC. SIOC conforms to presentation rules (e.g. a textual description should always appear before the corresponding image). The Virtual Document (VD) is generated by the aggregation of IOCs and is defined as a 2-tuple D := (IOC, SV D) where IOC is a set of IOCs sharing a common property that logically structured (SV D) in order to compose a document (VD). One of the factors that determines the structure and content of a VD is the context. The quality of the selected IOs depends on the user category (e.g. for novice user detailed IOs are presented). According to the activity, different type categories may be selected (e.g. questions are included in learning context). Links are created between different contexts for each IOC, enabling the user to change context at run-time. The segmentation of a virtual document advances the reuse of information (i.e. reuse of IOs and IOCs) and the generation of VDs at run-time.

Relating Product Support Problems and Context-Aware Virtual Documents

The knowledge that a product support system should deliver to the user is tightly linked to the problem that has to be solved. There are two basic qualities that characterise a PSP, namely its content and context. The 112 Nikolaos Lagos and Rossitza M. Setchi

Fig. 9.3. Virtual document ontology. content has to be relevant to the product that is supported and/or the task that the user wants to perform, while the context is determined according to the user characteristics and the system’s usage. The PSP therefore should contain what is needed (elements of knowledge that are missing), why it is needed and under what circumstances (context). The given deﬁnition of a PSP contains all the identiﬁed elements and is represented as follows.

Deﬁnition 1. Product Support Problem (PSP) is a 4-tuple P SP := (MOD, HY P, CON, OBS) where:

1. MOD is a ﬁnite set that represents the product and task models in relation to the IOCs and IOs that form the documents. 2. HYP is a ﬁnite set of combinations of elements of MOD representing possible documentation hypotheses. 3. CON is the context that characterises the problem and contains the user model (UM) in combination with the usage purpose. 4. OBS represents the observations acquired by the current query and are mapped to elements of MOD and CON. 9 PROGNOSIS: An Adaptive System for Virtual Technical Documentation 113

Definition 1 identifies PSPs as a specialisation of diagnostic problems, since a product support system recognises and solves PSPs in terms of the IOs and IOCs involved. The problem solving process therefore includes identifying that there is a fault (e.g. product support virtual document asked does not exist) recognising the type of fault (e.g. difference in configuration or missing IO, IOC), and choosing a strategy to be followed (e.g. provide the missing documentation element). In order to achieve that PSPs have to be appropriately represented.

Multimodal Reasoning for Virtual Document Adaptation

The problem solving process includes identifying the relevant elements of the modelled knowledge and mapping them to the corresponding IOCs and IOs for producing the most appropriate document, in accordance with the OBS set. Context-related changes, may require one of the following processes.

Knowledge acquisition in the presence of unchanged context

The aim in this situation is to ﬁnd a way to acquire solutions without repeating the reasoning process each time a query, same to a previous one, occurs. In order to establish that one PSP is the same as another, speciﬁc knowledge about each problem has to be compared. Case-based reasoning is deemed as the most appropriate technique in this scenario. Case-based reasoning caches old situations and solutions, avoiding reasoning from scratch and reducing the time needed for delivering a solution. The representation of cases follows a problem- solution pair structure, where the features that each case includes correspond to elements of the domain and context models. The solutions are pre-calculated.

Knowledge adaptation or generation in the presence of different context

Case-based reasoning provides means for reasoning by analogy and therefore adapting solutions according to previous experiences. This feature is highly utilised here, by substituting IOs with new ones that reflect the changes. IOs correspond to specific features of each case. The general knowledge that is contained in the case base is extended and the adapted case forms a new problem-solution pair. However, the existence of a new problem with context that has little or no similarity to previous experiences cannot be easily managed by CBR alone. In this case a multi-modal reasoning strategy is adopted where general knowledge about the application domains (e.g. product domain), is utilised supporting the case-based reasoning component. Models include features of the cases while they generalise cases’ specific knowledge.

9.5 PROGNOSIS

The architecture of the adaptive product support system developed called PROGNOSIS is shown in Fig. 9.4 The system includes several independent components. The Knowledge Base Layer contains the product, documentation, task and user knowledge bases, as well as the case and ﬁle system bases. These can be accessed by the administrators or knowledge engineers/managers through specialised interfaces. Manipulation tools are provided for editing, maintaining, troubleshooting and querying all knowledge bases. The Logic Layer contains the algorithms for reusing, adapting and generating new product support virtual documents. Three main groups of internal processes are also managed by the logic layer: user registration, authentication and administration. The Navigation Control Layer encompasses browsing, search, querying and validating applications that are used by PROGNOSIS end-users to access the knowledge contained in the system, in the form of product support virtual documents. Multiple end-users are able to access the PROGNOSIS interface in parallel with standard web-browsers. The Sensors Layer contains several sensors. The word “sensor” in this study refers mainly to software used to capture internal and external stimuli. External stimuli are factors that characterise the system’s environment, 114 Nikolaos Lagos and Rossitza M. Setchi the user, the user’s environment and the interaction between the user and the system. Internal stimuli are all changes happening internally to the system, such as variations in domain information. Sensing information is captured explicitly (e.g. questionnaires) and implicitly (e.g. use of the system) while sensors operation is either automatic or semi-automatic (e.g. sensing is initiated by the user with the use of a system’s function). For example, a programme that monitors the key strokes of the user and transmits them to the system is a type of a software sensor.

Fig. 9.4. PROGNOSIS architecture.

The sensors layer is used to gather all contextual data. Some of this data is directly stored in the Context KB (e.g. administrative data such as the name of the user). However, most of the acquired contextual data is too primitive to be used directly. The Context Processing Module converts such data into meaningful contextual information using Equation 9.2. The Context KB retains all contextual data according to the ontology of context described in Section 9.3. The Adaptation Module retrieves the contextual characteristics retained in the Context KB and adapts information delivery utilising the reasoning approach portrayed in Section 9.4.

9.6 Illustrative Example

The scenario discussed is that a user requests information from the context-aware product support system about the clutch design procedure. It is assumed that the user is inexperienced and wants to design a clutch (i.e. perform a task). The system responds by generating the top left document (screenshot) in Fig. 9.5. The virtual document includes IOCs that correspond to the task “design”, its subtasks and the product clutch. IOs are selected also in terms of the context (i.e. “novice”- “performing”), which is passed as a set of parameters to the document generation process. In this case, the task process is configured as follows: Determine goal Identify constraints Make preliminary calculations of metrics Analyze layout of metric attributes Re→vise and make final →calculations of metrics. The goal, constraints →metrics, metric attributes and calculations→ are specific to the supported product, which in this case is the clutch. Example metrics are the torque and moment of inertia 9 PROGNOSIS: An Adaptive System for Virtual Technical Documentation 115 of the clutch. In order to support the realization of the current task, performance tools are provided, which in this case include a calculator for the torque and one for the moment of inertia (right frame of the top left screenshot). The presented IOs are “detailed”, animations and video are preferred over still images, and include all clarification types such as “information”, “rule-of-thumb”, and “explanation”.

Fig. 9.5. Illustrative example.

Part of the user’s training is to learn about the theoretical background of the clutch design metrics. This means that the context should change from “performing” to “learning”. “Context-change” buttons are developed within the generation procedure of the document that link IOCs and the whole document to other contexts. If the user clicks the “activity-change” button, the task procedure is reconfigured in the following one: Obtain the main metric equation Obtain other metrics equations that exist within the initial equation Present theoretical aspects of these→equations Solve related assignments and case studies (or Evaluation).→The new document is illustrated in the top right→screenshot in Fig. 9.5. The “performance tools” are replaced by “learning tools” such as summarization tables. The presented IOs are “detailed”, include animations and videos, and all clarification types such as “descriptions” and “examples” (note that the clarification types change, e.g. “rule-of-thumb” is not used in learning). The user can also change the category level at run-time through the “user-change” button. This removes all clarification objects, changes the expressiveness to “general” and includes more “technically-oriented” textual descriptions and still images, as shown in the bottom right screenshot in Fig. 9.5. The utilization of the “context- change” buttons enables the user to change context at run-time and the provision of support becomes highly adaptive and interactive. 116 Nikolaos Lagos and Rossitza M. Setchi 9.7 Conclusions

Till today, adaptation in product support systems has been solely based on user models, delivering insufficiently adapted information. This chapter introduces the concept of context-aware product support systems, which are needed to meet the requirements of the user. To facilitate the development of such systems, a context ontology is proposed that includes four basic models the activity, user, environment and physical ones. The environment and physical models are particularly important in mobile and distributed applications. Utilising the activity model, performance support and e-Learning techniques can be integrated within the context of a single product support system to achieve both performance and learning goals for the first time. Through the activity model, the task model (domain knowledge) can be reconfigured according to user needs. As a result, the user model is not considered independent of the other context models, creating a holistic view of different situations to which the system needs to react. Context-specific content adaptation is achieved by adjusting the type, form, and level-of-detail documentation element attributes in different context instantiations. Furthermore, the delivered information is modified to reflect the configuration of the task model and the information describing supported products. It is therefore shown that knowledge about the context, the universe of discourse (including products and tasks), and documentation can be integrated to transform a product support system in a responsive entity able to adapt to different situations described with context-specific attributes.

References

G. D. Abowd, A. K. Dey, P. J. Brown, N. Davies, M. Smith, and P. Steggles. Towards a better understanding of context and context-awareness. Lecture Notes in Computer Science, 1707:304–307, 1999. M. Baldauf, S. Dustdar, and F. Rosenberg. A survey on context-aware systems. International Journal of Ad Hoc and Ubiquitous Computing, 2(4):263–277, 2007. W. R. Bezanson. Performance support: online, integrated documentation and training. In 13th ACM Annual Conference on Emerging from Chaos, Savannah, Georgia (USA), 1995. A. Binemann-Zdanowicz, R. Kaschek, K.-D. Schewe, and B. Thalheim. Context-aware web information systems. In 1st Asian-Pacific conference on Conceptual Modelling, Dunedin, New Zealand, 2004. S. Bosley, J. Arnold, and L. Cohen. The anatomy of credibility: A conceptual framework of valued career helper attributes. Journal of Vocational Behavior, 70(1):116–134, 2007. C. Bradley and M. Oliver. The evolution of pedagogical models for work-based learning within a virtual university. Computers and Education, 38(1–3):37–52, 2002. C. M. Garofano and E. Salas. What influences continuous employee development decisions? Human Resources Management Review, 15(4):281–304, 2005. T. R. Gruber. Toward Principles for the Design of Ontologies Used for Knowledge Sharing. Knowledge Systems Laboratory, Stanford University, 1993. T. Jokela. Evaluating the user-centeredness of development organizations: Conclusions and implications from empirical usability capability maturity assessments. Interacting with Computers, 16(6):1095–1032, 2004. E. L. Jorgensen and J. J. Fuller. A web-based architecture for interactive electronic technical manuals (ietms). In ASNE Naval Logistics Conference, Arlington, Virginia (USA), 1998. O. Kwon. The potential roles of context-aware computing technology in optimisation-based intelligent decision-making. Expert Systems with Applications, 31(3):629–642, 2006. J. Lindsey and S. E. Flowers. Beyond maintenance training-implementing performance support software on the plant floor. In Industrial Computing Conference, New Orleans, Louisiana (USA), 1995. C. Marion. Attributes of performance-centered systems: What can we learn from five years of EPSS/PCD competition award winners? Technical Communication, 49(4):428–443, 2002. J. McCarthy. Notes on formalizing context. In 13th International Joint Conference on Artificial Intelligence, Chambéry, France, 1993. D. T. Pham and R. M. Setchi. Case-based generation of adaptive product manuals. In Institution of Mechanical Engineers, volume 217, 2003. 9 PROGNOSIS: An Adaptive System for Virtual Technical Documentation 117

D. T. Pham, S. S. Dimov, and R. M. Setchi. Intelligent product manuals. In Institution of Mechanical Engineers, volume 213, 1999. P. Prekop and M. Burnett. Activities, context and ubiquitous computing. Computer Communications, 26(11): 1168–1176, 2003. D. Preuveneers, J. Van den Bergh, D. Wagelaar, A. Georges, P. Rigole, T. Clerckx, Y. Berbers, K. Coninx, V. Jonckers, and K. de Bosschere. Towards an extensible context ontology for ambient intelligence. Lecture Notes in Computer Science, 3295:148–159, 2004. R. M. Setchi, A. M. Huneiti, and C. Pasantonopoulos. The evolution of intelligent product support. In 5th CIRP International Seminar on Intelligent Computation in Manufacturing Engineering, Ischia, Italy, 2006. N. Souchon, Q. Limbourg, and J. Vanderdonckt. Task modelling in multiple contexts of use. Lecture Notes in Computer Science, 2545:59–73, 2002.

10 Investigating the Applicability of Content-Based Filtering on Digital Television Advertisements

George Lekakos

ELTRUN - the eBusiness Center, Department of Management Science and Technology Athens University of Economics and Business 47 Evelpidon Str., 11362 Athens, Greece [email protected]

Summary. The technological advances in the television domain offer marketers an unprecedented opportunity to target their advertising messages to those consumers/viewers that are most likely to be interested in them, while at the same time decreasing the information overload caused to the viewers by messages that are obtrusive and irrelevant to their information needs. Recommender systems provide methods and techniques that can be effectively utilized for the personalization of digital television advertisements. They have been widely and successfully used in order to make personalized recommendations for information, products or services. Collaborative filtering and content-based filtering represent the major recommendation approaches. Content-based filtering presents a number of disadvantages compared to collaborative filtering including the requirement to analyze content into features, which is not easily applicable in several content types including advertisements. On the other hand, there are important reasons that make content-based filtering a complementary to collaborative filtering method. Firstly, it expresses the user’s interest on a specific area that should not be ignored, while in computational terms it may operate in cases where collaborative filtering fails to do so (e.g. for new items for which no rating exists). In this chapter, we investigate the applicability of the content-based approach in the light of the above limitations, examining the conditions that it can effectively complement collaborative filtering. Along the above objective, we research and empirically evaluate advertisement features that are suitable for the digital television advertising domain that could be utilized effectively in a content-based filtering approach. Finally, we empirically examine the predictive performance of a content-based filtering approach that incorporates the features under examination compared to collaborative filtering.

10.1 Introduction

The ability to deliver personalized advertising messages has long been a major objective for marketers since it allows them to meet heterogeneous consumer needs and target their messages more effectively [Arens and Bovee, 1994]. However, traditional one-to-many marketing approaches applied in mass media suffer from their inability to meet the objective of one-to-one communication, since the targeted unit is the segment rather than the individual consumer, and therefore individual needs cannot be satisfied [Kara and Kaynak, 1997]. In the television advertising domain market segmentation methods in combination with domain-specific features such as time zones and/or program typologies [Belch and Belch, 1995] face similar challenges: media coverage either exceeds the targeted market segment or leaves potential customers without exposure to the message, thus reducing its cost effectiveness [Hoffman and Novak, 1997, Dibb, 1998]. Personalization of advertisements provides marketers with the opportunity to increase advertising effectiveness by targeting consumers who are most likely to respond positively to the advertising message. From a consumer’s perspective, personalized advertising aids consumer decisions and reduces the information overload problem, while the selection and presentation of relevant products in advertising messages is expected to yield satisfied consumers [Ansari and Mela, 2003]. 120 George Lekakos

One-to-one marketing and personalization strategies are challenged by the traditional one-to-many (broadcasting) television communication model. In technological terms, personalization over traditional TV communication channels would require the transmission of as many streams as viewers [Pramataris et al., 2001]. However, the developments in Digital TV technologies and infrastructures enable the application of personalization methods employed over the Internet. Recommender systems methods have been successfully used to recommend items relevant to the users’ interests and provide a plausible set of techniques for the personalization of typical 30” television advertisements enhanced with interactive features. Although several methods and techniques have been used in recommender systems, the most popular ones are collaborative and content-based filtering. In several domains including television advertisements [Lekakos and Giaglis, 2006] it has been shown that collaborative filtering outperforms content-based filtering in terms of predictive performance in certain conditions. However, content-based filtering can be used either on its own or in combination with collaborative filtering to improve the prediction outcome. In this chapter we investigate the applicability of content-based filtering on television advertisement with focus on the features that can be used to describe an advertisement and are applicable in any type of advertised product or service. Furthermore, we investigate the conditions that content-based filtering can outperform collaborative filtering. The remainder of the chapter is organized as follows. In Section 10.2, a short comparison of the two major recommender systems methods is presented. In Section 10.3, possible features that can be utilized in content- based filtering are analyzed. In Section 10.4, the algorithms used in the experiment are presented, followed by the experimental results in Section 10.5. Finally, in Section 10.6, conclusions and future research emanating from the present study are discussed.

10.2 Recommender Systems

In many occasions in our everyday lives our selections, choices or purchase decisions are influenced by opinions, previous experience and recommendations from family, friends, associates or professional reviews (e.g. entertainment magazines or book reviews). Well known in marketing theory and practice, the word of mouth is one of the factors that can affect the consumer’s decision towards a product selection and the automation of this social process was a motive on the development of recommender systems [Shardanand and Maes, 1995]. Goldberg et al. [1992] used the term collaborative filtering in Tapestry, an e-mail filtering system, to denote that “people collaborate to help each other perform filtering by recording their reactions to documents they read”. Resnick and Varian [1997] coined the term recommender systems for systems where “people provide their recommendations which the system then aggregates and directs to appropriate recipients”. Until then, the term referred to systems that utilized the collaborative approach to filter the available data items. Nowadays, recommender systems have expanded their scope and the approaches utilized to produce the recommendations and refer to systems that “produce individualized recommendations as output or have the effect of guiding the user in personalized way to interesting or useful objects in a large space of possible options” [Burke, 2002]. The recommendation task refers to the prediction of a user’s interest for a specific item. The user and the item the prediction refers to are indicated as the target user and the target item respectively. The recommendation process usually takes as input user ratings on observed items and/or item features and utilizes machine learning techniques to make predictions for unobserved items. The adaptation effect is then visualized using various techniques, in accordance to the characteristics of the application domain, such as presenting a ranked list of relevant items, recommending the top-n relevant items or just presenting the system’s suggestion in an advisory manner. The two major approaches utilized in recommender systems are collaborative filtering (CF) and content- based filtering (CBF). Collaborative filtering exploits the assumption that users who exhibit similar behavior in the past (or present some form of similarity) can serve as recommenders for each other on unobserved data items. So, given the user’s evaluations (rating) on observed items, the idea is to trace relationships or similarities between the target user and the remaining users in the database. Then, the similar users who have rated the target item provide their evaluations which are being summarized and directed to the target user. On the other hand, content-based filtering makes predictions upon the assumption that a user’s previous preferences or interests are reliable indicators for his/her future behavior. CBF requires the analysis of the 10 Investigating Content-Based Filtering on Digital TV Advertisements 121 content of the items into features upon which the user expresses his/her interest or explicitly requests items that satisfy specific criteria. CBF has its roots in Information retrieval research and is typically applied upon text-based documents, or in domains with structured data [Balabanovic and Shoham, 1997, Pazzani, 1999]. For example, content-based filtering has been utilized in book recommendation tasks [Mooney and Roy, 2000], using features such as title, author or theme. In such cases, the user’s previous preferences on the respective features are used to filter the available books and recommend the most relevant of them to the user. Another typical example is a search engine, where users provide the keywords of the required document (e.g. a Web page) and the system returns the most relevant to the search criteria documents. Collaborative filtering presents a number of advantages over content-based filtering which make it a suitable candidate approach in the domain of TV advertisements:

1. Enables the filtering of any type of content. Items such as videos, music or advertisements are content types that cannot be easily analyzed in features by automated processes [Balabanovic and Shoham, 1997]. Moreover, in this chapter we are concerned with any type of advertised product. Thus, while for homogeneous product categories the feature-based analysis would be meaningful, it is rather hard to define features applicable in a wide range of products and still capable of describing them sufficiently. For example, features such as color or size would be descriptive for cars, but not for detergents. Collaborative filtering is not concerned with content analysis and therefore it provides a framework applicable in different domains from news articles and books to movies and art. 2. Provides recommendations based on the quality and taste. Even in content types where the analysis into features is feasible, content-based predictions cannot reflect the quality and taste [Herlocker et al., 1999], authoritativeness or respectfulness [Resnick et al., 1994] or aesthetic quality [Balabanovic, 1997] of the item whenever this is necessary. For example, CBF will recommend books that are relevant to the user’s interests, possibly written by the user’s favorite author but it will not tell whether it is a well-written book. Similarly, in the case of movie recommendation, CBF will suggest science-fiction movies if they are within the user’s area of interest but this does not provide any evidence about the quality of the movie. In contrast, collaborative filtering does not require a content-analysis but relies on the user’s overall taste on the items. Thus, it is capable to represent at a certain extent complex human behavior affected by emotional and cognitive factors, which are particularly important in TV advertisements. 3. Provides serendipitous recommendations. Content-based filtering restricts the spectrum of recommendations within the boundaries of the user’s current interests [Balabanovic and Shoham, 1997]. Collabora- tive filtering can provide recommendations concerning content that the user may have not considered in the past but has been found interesting by similar users. For instance, a user who is interested in books written by specific authors or with specific themes might also find interesting books beyond the specified features, positively evaluated by other readers with similar preferences. This is particularly important in the advertising domain since several of the advertised products may have not been observed by the user.

Furthermore, in terms of predictive performance, collaborative filtering has been shown to produce more accurate predictions in the movie recommendation domain [Alspector et al., 1997, Basu et al., 1998] and the digital television advertising domain [Lekakos, 2004]. However, the user’s area of interest cannot be ignored [Hanani et al., 2001] and definitely plays an important role in the prediction of interest for unobserved items. Thus, in domains with well-structured content, content-based filtering undoubtedly provides useful recommendations in particular when the user presents an idiosyncratic behavior [Smyth and Cotter, 2000] and only a limited number of similar users can be traced and utilized in a collaborative filtering approach. Content-based filtering (CBF) is in a sense complementary to collaborative filtering with respect to the rationale of the prediction process. Its main difference is that it disregards all other users in the database and predicts future ratings based solely on a user’s previous preference history (as expressed through ratings). Thus, in the content-based approach, previous preferences directly affect the prediction for new items, exploiting associations between observed and unobserved items. In collaborative filtering a user’s ratings are indirectly involved in the prediction process (at the measurement of similarities phase), ignoring the content features and possible associations between the various content types. In computational terms the most important advantage of CBF in comparison to collaborative filtering is that it can produce recommendations even if no user has rated the target item, such as in the case when a new item is introduced in the database. 122 George Lekakos 10.3 Feature Selection for Personalized Advertising

One of the main challenges of personalized advertising arises from the need to deal with an extremely wide range of different products. A collaborative filtering approach presents a computational advantage in this aspect since it considers only the user’s overall evaluation and does not take directly into account different product features. Indeed, it is rather difficult to define low-level product features (e.g. price or color), which are uni- versally applicable and meaningful to a wide range of products. The majority of current work in adaptive and recommender systems which concerns product-oriented prediction, refers mainly to specific product categories such as cars [Jameson et al., 1995], telephony devices [Ardissono and Goy, 2000], books [Mooney and Roy, 2000], restaurants [Pazzani, 1999] and movies [Basu et al., 1998] in which the definition of common low-level features is feasible. However, possibly useful high-level features that can describe the majority of products and could be utilized in a content-based filtering approach include:

1. Product category, for example cars, financial services, food and drink and so on. Although product category is a quite generic feature, we add it in the list of candidate features since it can address an inherent limitation of CBF, which occurs when none or very few similar products (i.e. belonging in the same product subcategory) have been rated by the target user. In this case the search for similar products can be extended to the product category level through existing associations between product categories and subcategories. 2. Product subcategory, for example sports cars, family cars, insurance services, beers, soft drinks and so on. The product subcategory is expected to serve as a more descriptive (than product category) item feature. 3. FCB grid features (“high”, “low”, “think”, “feel”), which are features developed for the categorization of a wide range of products.1 More specifically, advertised products are typically divided into high and low-involvement products that refer to the “degree of personal relevance and importance of the product with respect to the purchase consequences or achievement of personal goals” [Peter and Olson, 1996]. Under high product involvement conditions, consumers tend to search extensively [Engel and Blackwell, 1982, Hawkins et al., 1998] and consequently present increased intention to interact [Yoo and Stout, 2001]. This trend may affect the consumer’s attitude towards the ad, since neutral or negative evaluation of low- involvement products may be attributed to the lack of searchable content rather than to actual dislikeness. With respect to the message communicated, advertisements can be divided into two classes, depending on whether the appeal focuses on product attributes and benefits or on the creation of brand image [Laskey et al., 1989]. The terms informational/rational/cognitive and image/emotional/feeling [Aaker and Norris, 1982] respectively, thinking and feeling [Vaughn, 1980] or informational and transformational [Puto and Wells, 1984] have been used to describe the two strategies. Thus, the contribution of a message’s appeal in the positive evaluation of an advertisement should also be taken into account. Ratchford [1987] proposes a product taxonomy based on the FCB grid, along the dimensions discussed above (“think”-”feel”/ “high”-”low” involvement). According to this taxonomy a product may be classified in one of the four quadrants (“think-low”, “think-high”, “feel-low”, “feel-high”). Thus, the FCB grid provides a suitable single framework that can be utilized for the examination of possible effects of the message appeal and involvement level in the application of recommendation approaches on a wide range of product advertisements.

The features described above are associated to each other: product subcategories are used to classify a product in a product category as well as in an FCB quadrant. Thus, it is rather meaningless to describe a product with a combination of the above features and therefore they will be evaluated separately.

10.4 Comparative Assessment of Filtering Techniques in Advertising

Several learning algorithms can be used for a content-based prediction such as inductive learning algorithms [Basu et al., 1998], classiﬁcation and regression trees (CART) [Karunanithi and Alspector, 1996],

1 The FCB grid is named after Foote, Cone & Belding [Vaughn, 1980]. 10 Investigating Content-Based Filtering on Digital TV Advertisements 123

Bayesian classiﬁers [Pazzani, 1999] and so on. In our implementation of CBF we select the Naïve Bayes algorithm, in which the user ratings represent ﬁve class labels. The algorithm is trained upon the available for each user {feature, class} pairs and when a product’s feature is provided as input it predicts its class membership by computing the conditional probability P(c|feature1, feature2,. . .,featuren), where c=1,2,3,4,5. The selection of Naïve Bayes algorithm is based on the following criteria:

1. It is a computationally cheap learning algorithm with performance comparable or superior to more complex algorithms in many domains [Domingos and Pazzani, 1997]. 2. It has been successfully used in content-based recommender systems in domains such as books and movies [Mooney and Roy, 2000, Melville et al., 2002]. 3. Given the low number of features used to describe products and presuming conditional independence of the features, Naïve Bayes can theoretically provide very accurate results [Mitchell, 1997].

One of the important limitations of content-based algorithms is that they require a sufficient number of training examples in order to produce reliable predictions. If a user has never rated before similar items then the probability for a class membership is computed upon the relative occurrences of each of the class labels in the training data. In the experiments presented in this chapter we adopt one of the most popular collaborative filtering algorithms [Resnick et al., 1994], based on the Pearson correlation coefficient. It can be described as a process divided in three steps:

1. Measurement of similarities between the target and the remaining users. A typical measure of similarity is the Pearson correlation coefﬁcient [Resnick et al., 1994], which is applied on the items rated in common by two users ( 10.1).

((Ri,k Ri)(Rj,k Rj )) w(i, j) = k − − (10.1) 2 P 2 (Ri,k Ri) (Rj,k Rj ) q k − k − P P where Ri,k and Rj,k refer to the rating of the k-th item commonly rated by both users i and j, and Ri, Rj refer to the mean values of the available ratings for the users i and j. The coefﬁcient value ranges from -1 to +1, tracing both positive and negative correlations. 2. Selection of the neighbors who will serve as recommenders. In this step we used the threshold-based selection [Shardanand and Maes, 1995], according to which users whose similarity exceeds a certain threshold value are considered as neighbors of the target user (selected threshold value = 0). 3. Prediction based on the weighted average of the neighbors’ ratings, weighted by their similarity to the target user:

m j=1(w(i, j)(Rj,p Rj )) Ri,p = Ri + − (10.2) P m ( w(i, j) ) j=1 | | P where Ri,p is the rating to be predicted for user i and for item p, Ri is the mean of the ratings of user i for all items that user has provided his/her ratings, the weight w(i, j) is the similarity measure between user i and j, Rj,p is the rating of user j for item p, and Rj is the mean of ratings of user j in a neighborhood of size m (the denominator serves as the normalizing factor for the use of the weights in the numerator).

10.5 Experiment

The objectives of the experiment presented below are to examine the predictive performance of the content- based approach using the features discussed above as well as in comparison with collaborative filtering. Along the above objectives we have employed a sample of 37 individuals drawn from our academic research group. The sample includes academic (19%), research (73%) and technical staff (8%), consisting of 62.2% males and 37.8% females, aged 18-24 (10.8%), 25-34 (67.6%), 35-44 (18.9%) and 45-54 (2.7%). The users 124 George Lekakos were shown 65 television advertisements selected from seven product categories (food and drink, fast moving consumer goods, computer and technology, family and home, books and magazines, public services, finance and investment and autos) and provided their ratings for each advertisement in a one-to-five scale. Similarly to other domains (e.g. movies, books) a rating concerns the degree of likeability (i.e the overall taste) for each item. The prediction error for each user is measured using the Mean Absolute Error (MAE), which is the average difference between the predicted and the actual rating value and is commonly used for performance evaluation [Breese et al., 1998, Herlocker et al., 2002, Shardanand and Maes, 1995, Claypool et al., 1999, Melville et al., 2002]. The overall error estimator for CF and CBF is the average of the MAE’s for each user. Previous research [Lekakos and Giaglis, 2006] has shown that CBF can provide reliable results when a sufficient number of ratings are available for each user. In cases where few ratings are available CBF is clearly outperformed by collaborative filtering methods due to the low levels of coverage (the items for which a prediction can be made). In this experiment we evaluate the CBF performance by withholding the ratings on 15 items formulating thereafter the test set. The remaining 50 ratings are considered as the training set (in a 65-item data set). The test set ratings were selected through stratified random sampling using as strata the four rating intervals A, B, C and D, where A = [1-2), B=[2-3), C=[3-4), D=[4-5] [Basu et al., 1998]. In this way, the training and test sets follow the same distribution in order to avoid pessimistic performance [Mitchell, 1997]. The frequencies of the test items with respect to the product subcategories, product categories and FCB groups are depicted in Table 10.1 (following coding conventions for product categories and subcategories).

Test item Product subcategory Product category FCB group (frequency) (frequency) (frequency) 1 22 (7) 2 (14) Low-Think (17) 2 22 (7) 2 (14) Low-Think (17) 3 22 (7) 2 (14) Low-Think (17) 4 13 (1) 1 (18) Low-Think (17) 5 14 (1) 1 (18) Low-Feel (19) 6 21 (7) 2 (14) High-Feel (3) 7 18 (1) 1 (18) Low-Feel (19) 8 16 (2) 1 (18) Low-Feel (19) 9 110 (0) 1 (18) Low-Feel (19) 10 41 (1) 4 (2) High-Think (7) 11 53 (2) 5 (6) Low-Feel (19) 12 23 (0) 2 (14) Low-Think (17) 13 41 (1) 4 (2) High-Think (7) 14 22 (7) 2 (14) Low-Think (17) 15 53 (2) 5 (6) Low-Feel (19) Table 10.1. Frequencies of product category, subcategory, and FCB group for each test item.

For example, in Table 10.1 above, the first line indicates that the first item in the test set, belongs in the product subcategory “22” (Detergents) that is found seven times in the training set (i.e. seven products in the training set are classified in this subcategory). Similarly, the first item in the test set belongs in the product category “2” (Fast Moving Consumer Goods) that is found fourteen times in the training set and in the FCB group “Low-Think”, which is found seventeen times in the training set and so on. The comparisons of the performances of the content-based approach for each feature as well as the performances of the collaborative filtering approach are presented in Table 10.2. Among the three types of features, the most accurate prediction in absolute values is given by the product subcategory. In statistical terms, paired comparisons (using paired t-tests and 95% confidence level) reveal that predictions based on product subcategory and product category do not differ significantly (t=-1.405, p=0.169), while CBF based on product subcategory significantly outperforms the FCB-based prediction (t=- 10 Investigating Content-Based Filtering on Digital TV Advertisements 125

Content-based ﬁltering Collaborative ﬁltering Product subcategory Product Category FCB 0.9387 0.9838 1.021 0.7528 Table 10.2. CBF performance for the selected features compared to CF.

2.196, p=0.035). However, in any feature selection the differences with collaborative ﬁltering are still signiﬁcant in favor of the latter (t=-4.636, p=0.000). The CBF performance relies on the amount of available feature, class pairs, as for example in the case product subcategory (Table 10.3).

Test item Product subcategory Mean Absolute Error (frequencies) 1 22 (7) 0.7027 2 22 (7) 0.56757 3 22 (7) 0.32432 4 13 (1) 1.02703 5 14 (1) 0.81081 6 21 (7) 0.64865 7 18 (1) 0.35135 8 16 (2) 0.78378 9 110 (0) 1.62162 10 41 (1) 0.81081 11 53 (2) 1.35135 12 23 (0) 2.13514 13 41 (1) 0.81081 14 22 (7) 0.48649 15 53 (2) 1.64865 Table 10.3. CBF performance in relation to the subcategory occurrences in the training set.

The highest errors concern the two items with zero occurrences of items of the same product subcategory in the training set (items 9 and 12 in Table 10.3). In this case the algorithm returns as the predicted class (rating) the most frequently observed class in the training set. Removing those two items, the effect on the averaged performance is signiﬁcant but still CBF’s MAE (0.7941) is higher than CF’s (0.7369).

10.6 Conclusion

The selection of product subcategory can be a fair feature option in a content-based approach. Although it is a single feature description, the experimental results demonstrate that it performs fairly well. In addition, it is applicable in a wide range of products, exploiting the existing associations between products and product subcategories. However, within specific product categories more features which are meaningful for certain groups of products could eventually improve the accuracy of prediction. Furthermore, it is possible to increase the performance of the CBF by employing regression techniques [Duda et al., 2000], which can directly predict numerical ratings instead of representing them as classes ignoring the linear scale. Compared to collaborative filtering, CBF performs worse on average, but gives comparable performance excluding items for which the target has no previous rating history (at the product subcategory level). Indeed, the accuracy of the CBF prediction varies with the number of available items and is significantly reduced when no occurrences of the target item’s subcategory are found in the training set. On the other hand, collaborative 126 George Lekakos

filtering fails to recommend items which no other user has rated before. Furthermore, for certain users who present idiosyncratic behavior and their preferences cannot be accurately inferred by other similar users, CBF performs better than collaborative filtering. As empirical researches in the domain of digital television has shown [Lekakos and Giaglis, 2007] that the two methods can be combined and exploit each other’s advantages in order to provide more accurate prediction. In the present chapter we investigated a generic form of content-based filtering based on product types. Certain elements of the advertisement itself (e.g. the actor, music, aesthetics) do play a significant role on the user’s attitude towards the advertisement. Other techniques that analyze the multimedia content would be useful for an approach that incorporates the above aspects. The main reason for choosing the approach presented in this chapter was to avoid complex content-analysis processes and stay close to collaborative filtering whose one of the most attractive aspects (and advantages) is its independence from the content. However, it is in our future plans to investigate and compare techniques that rely on the analysis of multimedia content as well as the extension of the experiments to a larger number of users.

References

D. Aaker and D. Norris. Characteristics of TV commercials perceived as informative. Journal of Advertising Research, 22(2):61–70, 1982. J. Alspector, A. Koicz, and N. Karunanithi. Feature-based and clique-based user models for movie selection: A comparative study. Modeling and User Adapted Interaction, 7(4):279–304, September 1997. A. Ansari and C. F. Mela. E-customization. Journal of Marketing Research, 2003(2):131–156, 2003. L. Ardissono and A. Goy. Tailoring the interaction with users in web stores. User Modeling and User Adapted Interaction, 10(4):251–303, 2000. W. Arens and C. Bovee. Contemporary Advertising. Boston: Irwin, 1994. M. Balabanovic. An adaptive web page recommendation service. In ACM First International Conference on Autonomous Agents, pages 378–385, Marina del Rey, California (USA), 1997. M. Balabanovic and Y. Shoham. Fab: Content-based collaborative recommendation. Communications of the ACM, 40(3):66–72, 1997. C. Basu, H. Hirsh, and W. Cohen. Recommendation as classification: Using social and content-based information in recommendation. In 15th National Conference on Artificial Intelligence, pages 714–720, Madison, Winsconsin (USA), 1998. G. E. Belch and B. A. Belch. Introduction to advertising and promotion: An integrated marketing communications perspective. San Diego: Irwin, 1995. J. S. Breese, D. Heckerman, and C. Kadie. Empirical analysis of predictive algorithms for collaborative filtering. In 14th Annual Conference on Uncertainty in Artifical Intelligence, pages 43–52, San Francisco, California (USA), 1998. R. Burke. Hybrid recommender systems: Survey and experiments. User Modeling and User Adapted Interac- tion, 12(4):331–370, November 2002. M. Claypool, A. Gokhale, T. Miranda, P. Murnikov, D. Netes, and M. Sartin. Combining content-based and collaborative filters in an online newspaper. In ACM SIGIR Workshop on Recommender Systems, Berkeley, California (USA), 1999. URL http://www.csee.umbc.edu/ ian/sigir99-rec/. S. Dibb. Market segmentation: Strategies for success. Marketing Intelligence and Planning, 16(7):394–406, 1998. P. Domingos and M. Pazzani. On the optimality of the simple Bayesian classifier under zero-one-loss. Machine Learning, 29(2-3):103–130, November 1997. R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classification. New York: Wiley, 2000. J. Engel and R. G. Blackwell. Consumer Behavior. New York: Dryden Press, 1982. D. Goldberg, D. Nichols, B. M. Oki, and D. Terry. Using collaborative filtering to weave an information tapestry. Communications of the ACM, 35(12):61–70, 1992. U. Hanani, S. Bracha, and S. Peretz. Information filtering: Overview of issues, research and systems. User Modeling and User Adapted Interaction, 11(3):203–259, August 2001. 10 Investigating Content-Based Filtering on Digital TV Advertisements 127

I. Hawkins, R. J. Best, and K. A. Coney. Consumer Behavior: Building Marketing Strategy. New York: Irwin/McGraw-Hill, 1998. J. Herlocker, K. Konstan, A. Borchers, and J. Riedl. An algorithmic framework for performing collaborative filtering. In 22nd International Conference on Research and Development in Information Retrieval, pages 230–237, New York (USA), 1999. J. Herlocker, J. Konstan, and J. Riedl. An empirical analysis of design choices in neighborhood-base collaborative filtering algorithms. Information Retrieval, 5(4):287–310, October 2002. D. L. Hoffman and T. P. Novak. A new marketing paradigm for electronic commerce. The Information Society, Special Issue on Electronic Commerce, 13(1):43–54, 1997. A. Jameson, R. Schafer, J. Simons, and T. Weis. Adaptive provision of evaluation-oriented information: Tasks and techniques. In 14th International Joint Conference on Artificial Intelligence,, pages 1886–1993, Mon- tréal (Canada), 1995. A. Kara and E. Kaynak. Markets of a single customer: Exploiting conceptual developments in market segmentation. European Journal of Marketing, 31(11-12):873–895, 1997. N. Karunanithi and J. Alspector. Feature-based and clique-based user models for movie selection. In 5th International Conference on User Modeling, pages 29–34, Kailua-Kona, Hawaii (USA), 1996. H. A. Laskey, E. Day, and M. R. Crask. Typology of main message strategies for television commercials. Journal of Advertising, 18(1):36–41, 1989. G. Lekakos. Personalized Advertising Services through Hybrid Recommendation Methods: The case of Digital Interactive Television. PhD thesis, Athens Univesity of Economics & Business, Athens, Greece., 2004. G. Lekakos and G. Giaglis. Improving the prediction accuracy of recommendation algorithms: Approaches anchored on human factors. Interacting with Computers, 18(3):410–431, 2006. G. Lekakos and G. Giaglis. A hybrid approach for improving predictive accuracy of collaborative filtering algorithms. User Modeling and User Adapted Interaction, 17(1-2):5–40, 2007. P. Melville, R. J. Mooney, and R. Nagarajan. Content-boosted collaborative filtering for improved recommendations. In 18th National Conference on Artificial Intelligence (AAAI), pages 187–192, Edmonton (Canada), 2002. T. Mitchell. Machine Learning. McGraw-Hill, 1997. R. J. Mooney and L. Roy. Content-based book recommending using learning for text categorization. In 5th ACM Conference in Digital Libraries, pages 195–204, San Antonio, Texas (USA), 2000. M. Pazzani. A framework for collaborative, content-based and demographic filtering. Artificial Intelligence Review, 13(5-6):393–408, 1999. P. J. Peter and J. C. Olson. Consumer Behavior and Marketing Strategy. Homewood, IL: Irwin, 4th edition, 1996. P. Pramataris, D. Papakyriakopoulos, G. Lekakos, and N. Mylonopoulos. Personalized Interactive TV advertising: the iMedia business model. Electronic Markets, 11(1):1–9, 2001. C. P. Puto and W. D. Wells. Informational and transformational advertising. Advances in Consumer Research, 11:638–643, 1984. B. T. Ratchford. New insights about the FCB grid. Journal of Advertising Research, 27(4):24–38, 1987. P. Resnick and H. R. Varian. Recommender systems. Communications of the ACM, 40(3):56–58, 1997. P. Resnick, N. Iacovou, M. Suchak, P. Bergstrom, and J. Riedl. GroupLens: An open architecture for collaborative filtering of NetNews. In ACM Conference on Computer Supported Cooperative Work, pages 175–186, Chapel Hill, North Carolina (USA), 1994. U. Shardanand and P. Maes. Social information filtering: Algorithms for automating “word of mouth”. In ACM CHI Conference on Human Factors in Computing Systems, pages 210–217, Denver, Colorado (USA), 1995. B. Smyth and P. Cotter. A personalized television listings service. Communications of the ACM, 43(8):107–111, 2000. R. Vaughn. How advertising works: A planning model. Journal of Advertising Research, 20(5):27–33, 1980. C. Y. Yoo and P. A. Stout. Factors affecting users’ interactivity with the web site and the consequences of users’ interactivity. In Conference of the American Academy of Advertising, pages 53–61, Villanova, Pennsylvania (USA), 2001.

11 The Role of Communities in E-Commerce

Rodolfo Carpintier Santana

Digital Assets Deployment Doctor Castelo 10, 5oB, 28009 Madrid, Spain http://www.digitalassetsdeployment.com

Summary. Migration from the Web 1.0 to concepts of the Web 2.0 have substantially changed the scene and the players. Portals and centralized locations have been removed from the centre of the action and given place to new, giant communities have changed the way we understand contacts and proﬁles around Internet businesses. User-generated content is now a centrepiece of what the Internet is about and communities like YouTube force us again to reposition the advertising around new formats that must be accepted by readers. Giant investments in Internet properties must give way to new methods of monetizing visitors and pageviews and advance both advertising and content in new directions that will position offers at their best possible context in order to gain relevance and promote results. Youtube, MySpace and now Facebook are key ingredients of this new environment although their concepts for monetizing their users and the content they produce is still in its infancy.

11.1 Introduction

This chapter will try to explain why the concepts of Web 1.0 marketing are no longer valid and need to be reviewed in light of the different approaches to community creation and user-generated content. One to one, personalized marketing à la Broadvision is no longer accepted as the way to go. Communities reach an level of understanding of their contributors and participants unknown off until now. Marketing and sales must accept the evolution and discover new paths to success.

11.2 Some of the Players

One to one targeting has been a dream ever since the Internet revolution changed the way we measure advertising. However, the one to one revolution that was a substantial part of the Web 1.0 movement around 1998-2000 never completely satisﬁed the high expectations. Broadvision valuation at the time of its NASDAQ debut was the zenith of the one to one management backed by software. While the company is still active, its 2,3$ per share valuation has limped along for the last few years as the mode of one to one marketing deviated to Web 2.0 values. Content generation by users is at the core of the Web. 2.0 movement and allows a different type of personalization, namely selection at the user’s desk. Social networks à la MySpace and more recently, Facebook have allowed microcomunities to emerge that are generated by users with friends where the aim of personalized targeting is achieved in a much more natural manner as it was the case with Broadvision software. The key to this approach is to provide the user with the necessary tools to achieve what he needs in personalized information and to allow him or her to provoke his/her friends to join them in a social network. 130 Rodolfo Carpintier Santana 11.3 Advertising Reﬂections

Advertisers are still at odds with the possibilities of exploiting these new environments for advertising and/or e- commerce. From the beginning, these social networks have developed to provide the tools and promote volume without a clear picture of the underlying business model. When Murdock bought MySpace for 580 million dollars, everybody thought the Mogul had gone mad. Only 18 months latter he signed an agreement worth 900 million with Google. The leading search engine had understood the potential of their context sensitive advertising and micro communities within MySpace. Facebook, today’s leader in social networks, has just sold 1,6% of the company to Microsoft for 240 million dollars, reaching a theoretical valuation of over 15 billion dollars. Facebook is a very new social network in its conception that applies open opportunities for third parties and, as Amazon has already done, creates dozens of successful businesses around its over 60 million people community. I believe this valuation is only theoretical because we must discount Microsoft’s valuation of Facebook as his advertising partner and the opportunity to stop Google having a go at their customers. Microsoft has indeed highly valued the application’s ability to provide personalized pages for its uses and the capacity it has to integrate third party software that further enhances these possibilities. To deliver the right ad to the right person at the precise moment of decision making is an aim that, still distant, becomes a good promise in the realm of a social network and its, user-generated groupings and content management. eCommerce site of the Web 1.0 have also developed new personalized offerings and are now trying to cooperate with social networks to provide Web 2.0 e-commerce solutions. As the Web gets mobile the opportunities for personalized offerings related to positioning are improving. Berggi, a new mobile company based in Silicon Valley with strong presence in 12 countries offers, for instance, the opportunity to access a mobile instant messaging solution that will enable users to know what friends and members of Berggi are nearby at any moment. For instance, if stacked in an elevator, a Berggi user could find out another user nearby and ask for help at the same time that notifies his/her family of the incident. A new network of outside advertisers will be able to post their ads in municipal buses that will show their different ads, related to their specific location at the time. For instance, a local restaurant will be able to announce a special menu to all busses passing by their location. In a new professional social network in Spain, Cinemavip, thousands of audiovisual experts will generate new video clips and sell them to companies in their network. Sony has already used the service to locate the new director for one of their singers. The community effect has provided Sony with a new method to locates and create content and has proven able to distribute “buzz” about the singer that, as a result, has increased its song sales four times compared to their usual impact. Networks and their mobiles extension with capacity for geolocation will change the way local advertising and e-commerce work today. In less than 3 years we will have e-commerce platforms to provide sellers with access to passers-by via their mobile phone and the form we provide after sales service and fulfilment will never be the same. As social networks mature, many will disappear, but those that remain, will provide personalized services creating powerful networks of users generated solutions and applications to supply new formats of both advertising and commerce. Behavioral Marketing (BM) will reach a new level of achievement as it is applied to this new environment of social networks and location related offerings. As described recently by ClickZ,

“Behavioral marketing targets consumers based on their behavior on Web sites, rather than purely by the content of pages they visit. Behavioral marketers target consumers by serving ads to predeﬁned segments or categories. These are built with data compiled from clickstream data and IP information. A user visits several travel category pages on a particular site, for example. She’s then served airline ads. In most cases, the ads are served through a run-of-site (ROS) placement. The user’s beheviour is the key, not the placement.”

This type of targeting will become more accurate as we are able to precisely locate our customers at any given moment. Marketers will need to learn new skills and business models will need to develop new advertising solutions to cater for these new social relationships and location based advertising. 11 The Role of Communities in E-Commerce 131

I like to define “a powerful network” as the one that obtains a higher loyalty for itself as people in their network give to companies that pay their salaries. Good examples of such a powerful network are the “power sellers” of eBay or the SAP consultants or the CISCO certified engineers. Their loyalty is to eBay, SAP or CISCO and their networks and not to the company that, at any given moment, is paying their salary. Amazon is providing new ways of creating powerful networks by giving access to partners to their technology and customer base and we will see, more and more, that social networks, as it is already doing Facebook, will open up their applications to generate this type of networks around them. Open Social is Google answer to this upcoming trend and seeks to gain momentum by creating an open standard for interfacing applications with Google and the majority of the social networks around. Unfortunately, like it is always the case, commercial constraints will make it very difficult to agree on one single standard and large social networks like MySpace or Facebook will rather use their own standard. Google Maps is probably the leading application of choice today for any location-related, application. The combination of behavioral marketing and location-related advertising and e-commerce will create new opportunities and business models that will fuel a new revolution in application development.

11.4 First Results

When Ruppert Murdock purchased Myspace he was not buying present value but future positioning. The out- rageous 240 million paid up by Microsoft for 1,6% of Facebook does not reﬂect the value of the company but, as I have already explained, the need by Microsoft to control access to new advertising environments and a way to stop the fast advance of Google in taking on all large advertising circles that emerge worldwide. This has also been the aim of eBay when purchasing Skype and although, still today, they believe the price was too high, the growth of this part of their business is well above all the others within the eBay domain. What Skype has proven is that number of users and high growth does not equate immediately with advertising revenues. Telcos have been expected to fail rapidly in view of this type of free for all offerings but, nevertheless, the traditional players are supporting the attack with great approaches to avoid loss of revenues. What the technology makes possible does not translate immediately in advertising income or replaces traditional way of communicating. There is an increasing amount of, mainly young, people that have embraced de Internet as their communication method and do not do much without letting all their friends know via their own community pages, instant messaging or online life chats. It is still a new frontier where very few advertisers know their ways and little knowledge exists on what does or does not get acceptance.

11.5 The Viral Marketing of Communities

It is very interesting to review the expansion of communities in a given country. Let us study the growth of Tuenti in Spain, a case I know very well as a small investor in the venture. Let us have a look at their ﬁgures in Alexa: Tuenti was started by a couple of 25 year old entrepreneurs, all of them leaving university and wanting to remain in touch with their friends. The community to date is by invitation only. As it can be seen in Fig. 11.1, once the community achieved a tipping point, in this case over 100.000 members, the growth started to explode by viral marketing. Member gets member works here better than in any other environment. In a community members are more or less important depending of the amount of friends they carry with them. The community has been so far funded by the owners and investors and does not produce income. What is then its value and why? Tuenti will have over half a million members by the end of 2007, all of them within the age range of 16 to 28, approximately the same amount of females and males. A dream group target for many traditional institutions. Many banks would do almost anything to have access to this community. The problem lies with the concept of a close community where success goes directly with the treatment you provide to members. Advertising must not be too intrusive and it should be well in context and provide real 132 Rodolfo Carpintier Santana

Fig. 11.1. Alexa figures for the growth of Tuenti in Spain. opportunities to people that watch them. When promoting revenues within communities these concepts must be well understood by the promoters in order to avoid rejection and profit from the viral capabilities of the environment. It has, nevertheless, a great potential for inventing new ways of accessing new pools of customers that while very critical to any sort of intrusion in their community, are excited about new opportunities to do with a better communication, new entertainment offerings and, in many cases, opportunities for studying a master or reaching a decision on a special course abroad. For instance, a specific financial offering of a new credit card for them should be done with a link to a special proposal for instance, to provide financing for a specific type of masters and should promote viral marketing within the members and possible feedback to keep improving the offer with members comments. This type of approaches is still very much in its infancy and very few marketers understand how to promote their wares in this interactive environment. The new Facebook context-driven system for advertising is a new way to approach their members much the same manner as does Google ads. Nevertheless, a great potential for communities are the sponsorships; Facebook is aiming at 90 million in this type of income by promoting home page sponsored stories and sponsored groups. A sample of companies already sponsoring Facebook in this format are the ones shown in Fig. 11.2. Heavy online advertisers like these companies have long understood that banner ads are on the retreat and less interfering advertising must be deployed to get the attention of community members. In a different opportunity, Cinemavip in Spain is teaming up with companies like SONY or El Corte Inglés to sponsor special members productions that will find extensive coverage within the community and some outside media.

11.6 The Role of Communities in e-Commerce

There is a new trend in e-commerce and it has still to face the opportunities of communities. Amazon, as I have explained or eBay have long understood that their powerful network should be the axis of future growth and have concentrated efforts in this direction with good results. Community created, mini powerful networks exist within themselves without even trying. Friends can create mini communities within the larger one that show very speciﬁc needs and can provide a ﬁeld day for advertisers and sponsors if they learn how to approach them. One of these mini communities may have 20 11 The Role of Communities in E-Commerce 133

Fig. 11.2. A sample of companies sponsoring Facebook’s context-driven advertising system. participants while a different one reaches hundreds of them. The key of success is to use the right context and provide the necessary tools to follow on with the results while responding to speciﬁc needs immediately. There is a need to understand the power of consumer-generated content and sales in context advertising that are still very rudimentary applied by the marketers. In this search engines must also evolve and be capable of improving search in context and sales in the right mood while they explore users movements throughout a community and understand better the hobbies and opportunities they search and are more respondent to. For instance, an audiovisual community like Cinemavip will have members that create mini communities to gather actors or to write a video sequence and therefore, allow marketers to understand the type of products and services they may require in their communities. e-Commerce will need to evolve to cope with this type of environments and be very responsive to group’s requirements and individual opportunities within certain of these mini communities of interest.

11.7 The Future

There is a big ﬁght gathering momentum between the big guns of Internet Advertising. Google dominates de market with their position of number one but Microsoft, Yahoo, AOL, eBay, Amazon and a few others want their share and we will see a lot of purchases and mergers over the next ﬁve years. The positioning of MySpace, FaceBook and other communities with more close membership like Likedin, Xing, Tuenti or Cinemavip, will change with their success of failure to capture relevance in the different markets but there is a clear, local, phenomenon that will provide opportunities to the leading communities is each geographical area and they will become targets to be purchased by the leading Internet companies mentioned or, in other cases, al the stop on to the Internet to incumbents. 134 Rodolfo Carpintier Santana References

R. Carpintier Santana. Los cinco mandamientos de la empresa en Internet. Ciss, 2001. B. Kasanoff. Making it personal: How to proﬁt from personalization without invading privacy. Perseus Books Group, 2001. D. Peppers and M. Rogers. Managing Customer Relationships: A Strategic Framework. Wiley, 2004. C. K. Prahalad, P. B. Ramaswamy, J. R. Katzenbach, C. Lederer, and S. Hill. Harvard Business review on customer relationship management. Harvard Business School Press, 2002. 12 Personalized Education

Rosa M. Carro, Estefanía Martín

Escuela Politécnica Superior, Universidad Autónoma de Madrid (Spain) {Rosa.Carro,estefania.martin}@uam.es

Summary. The Internet is being widely used for supporting both individual and collaborative learning. A number of Web-based systems have been developed and students are more and more used to access to this type of systems. The rapid development of mobile handheld devices and wireless technologies has given rise to a widespread use of different devices such as PDAs, mobile phones, laptops or tablet PCs, among others. These devices, along with those technologies, constitute a new and attracting opportunity for learning through the Internet. This has motivated the creation of platforms and systems to support mobile learning. Different needs, interests, preferences, cognitive abilities or learning styles can determine the way each student accesses to available learning resources, processes information, solves practical work, relates to his/her mates and teachers, and, definitely, takes advantage of the learning process. These features are part of what is called the student model and should be considered in both individual and collaborative e-learning environments. With the aim of guiding students during the learning process, adaptation techniques have been used to personalize and adapt the activities proposed to each of them at each time, the navigational possibilities and the learning resources offered (multimedia materials, collaboration tools, etc). Moreover, one learning situation can be completely different from another, and the suitability of learning activities for students in specific contexts may vary depending not only on the student features or previous actions but also on their particular situation. This suggests the convenience of modeling the student context and using adaptation techniques so that, given a student accessing to a system in a particular context, the system is able to recommend her the most suitable activities to be accomplished in that specific situation. In this chapter I will describe the work done regarding adaptive individual and collaborative e-learning and m-learning environments. The basis of three adaptive e-learning systems will be explained in detail: TANGOW, an adaptive web-based system that guides each student during the learning process by personalizing and adapting the activities and resources to the user’s needs, characteristics and previous actions; COL-TANGOW, an extension of TANGOW to support collaboration, able to form workgroups dynamically and also to generate adapted collaborative workspaces at runtime to support the realization of collaborative activities by different groups of students; and CoMoLE, a system able to take into account the student context at a certain time to recommend her the most suitable learning activities to be accomplished at that time in that specific context.

12.1 Introduction

Since its beginning, the Internet has been widely used to look for specialized information with different purposes (entertainment, working needs, learning, curiosity, and so on). During the last years, there has been a widespread social use of the Internet, giving rise to social networks and applications to meet people, share resources, create websites collaboratively (Wiki), etc. One traditional use of the Internet is related to education. Students usually look for information on the Web to complete their educational tasks. They typically interact with on-line resources by their own and sometimes connect to Web-based courses or tutorials to learn at their own pace. However, not all students have the same goals, interests and needs. Students with less experience 136 Rosa M. Carro, Estefanía Martín on the use of the Internet can suffer from disorientation and feel overloaded because of the big amount of information offered. Aspects such as student learning style or personality can also influence the way in which they interact with information and can determine their educational needs. Consequently, it is convenient to take into consideration different user preferences and needs in order to adapt the information offered to each person according to these and other relevant aspects. This is the main goal of adaptive hypermedia. In the area of education, adaptive hypermedia has been widely used for the development of adaptive Web-based courses where students are personally guided during their learning process: contents and learning paths are adapted to specific user characteristics [Brusilovsky, 2001]. Individual learning through adaptive hypermedia can be pretty effective, but it is well known that the involvement of students in collaborative activities contributes to knowledge acquisition [Dillenbourg, 1999] and helps them to develop and improve personal and social skills [Johnson et al., 1985]. This fact has been considered in Computer Supported Collaborative Learning (CSCL) systems, whose main goal is to support collaborative learning with the help of computers. The interaction of users in CSCL environments is different from that of face-to-face learners. Distance learners must adapt their interactions to the features and capabilities of available tools and the other way around. In Web-based education, one issue to be considered is the importance of making sure that the activities to be carried out, as well as the tools provided to the users, have been designed according to user needs, so that they feel comfortable when interacting with the environment and find it useful for learning. When working collaboratively, different groups of users may also have distinct needs even when tackling the same collaboration activity. Since adaptation methods and techniques are available, it is possible and, moreover, convenient, to use them to adapt collaborative issues in e-learning too [Carro et al., 2003a]. The widespread development of mobile and wireless technologies has leaded to the creation of Web-based mobile environments to be accessed from different places at different times through diverse devices. In the context of education, mobile-learning (m-learning) systems have been developed to support learning task accomplishment. The suitability of carrying out activities in particular contexts can be influenced by the context itself (i.e. device, location, available time) and by user features and preferences. This information should be considered not only to support the realization of different activities in different contexts, but also to recommend users the most suitable activities to be tackled at each time, helping students to take decisions and organize their tasks and time. Contents and tools to support m-learning should also be adapted according to specific situations. In this chapter, different possibilities of personalization in e-learning, ranging from adaptation in web-based courses to m-learning environments, passing through CSCL systems, are presented.

12.2 Adaptive E-learning

12.2.1 Basis of Adaptation in E-learning

In the context of adaptive educational hypermedia, the fact that students may have dissimilar personal features, needs and interests has been considered. Contents and navigation paths have been adapted to guide each student during the learning process. The earliest educational applications of Adaptive Hypermedia (AH) date from the beginnings of the 90s. Brusilovsky [1996] presented the first classification of AH methods and techniques , which has been widely used and referenced. It considers mainly adaptive presentation (content-level adaptation) and adaptive navigation support (link-level adaptation). The first e-courses consisted basically of electronic books with chapters and sections to read, and links between them [Brusilovsky et al., 1996]. In modern e- learning environments, chapters and sections are replaced by activities to be performed; content adaptation considers different types of educational resources as well as tools to support activity accomplishment; and navigational guidance includes recommendation capabilities to propose activities to be tackled by students in different situations [Martín et al., 2007]. With the aim to support adaptive e-learning, managing information about students is necessary. User data, such as personal features, preferences, needs or context, can be considered to select the most suitable activities, contents and guidance to be provided to each user in different situations. These data are stored in the User Model (UM) and must be updated to be used with adaptation purposes subsequently. Different methods can be used to represent user model features, such as stereotypes [Kay, 2000], overlays [Conlan et al., 2002], 12 Personalized Education 137 machine learning [Webb et al., 2001] or Bayesian networks [Li and Ji, 2005], among others. The values of each attribute can be directly asked to the user (i.e. type of information desired, language) or got on the fly (i.e. percentage of exercises correctly solved, activities performed). Sometimes specific tests can be used to get more complex information such as learning style [Felder, 1996]), personality [Costa and McCrae, 1989] or intelligence [Thurstone, 1938]. Some well-known e-learning adaptive systems are ELM-ART [Brusilovsky et al., 1996], AHA! [de Bra et al., 2003] and TANGOW [Carro et al., 1999]. Details about ELM-ART and AHA! can be found in the literature. TANGOW is explained next section.

12.2.2 Adaptive E-learning in TANGOW

TANGOW (Task-based Adaptive learner Guidance On the Web) [Carro et al., 1999] is an adaptive educational hypermedia system that supports adaptation of proposed tasks and contents to each user at each time, according to different user features and behaviors. Contents can be different depending on, for example, user learning style or type of information desired. Tasks can be organized in different ways, giving rise to different course structures for different types of users. The flexibility of the navigational guidance offered can vary depending on the user profile. For example, students with sequential learning style can be directly guided through a set of activities, while those with global learning style can be less directly guided, so that they can navigate with more freedom. TANGOW stores and manages user data such as: personal information (i.e. name, address, e-mail or phone), features (language, background, learning style, personality, navigation experience, etc), preferences (i.e. type of information desired) or behavioral information (such as activities started or finished, results obtained, or connection frequency). Any parameter whose values can be represented by a set of either discrete values or intervals can be incorporated. This information constitutes the user model. With the aim to support adaptation, course components are specified separately in TANGOW. These components are: Parameters to be considered with adaptation purposes (student features, actions, etc). • Type of adaptation to be provided (navigation, contents, learning strategies, etc). • Activities to be performed, as well as relationships between them. • Different versions of multimedia contents to be associated to each activity and dynamically selected to • compose web pages. Adaptation capabilities and educational strategies to be used in order to recommend the most suitable • activities and contents to each user.

Course structures are defined by means of tasks and structural rules. Tasks represent activities to be performed and can be related to theoretical explanations to be read, examples to be observed, or different types of exercises to be solved, such as self-assessment tests, fill in the blank, crosswords, etc. Rules describe the organization of tasks, the time when tasks will be presented and the order in which subtasks must be performed. Each of these aspects can be specified in a different way depending on student features or needs. This can be achieved by defining different rules, each of them with activation conditions including the requirements for its triggering. The use of structural rules supports dynamic course structure generation, since learning tasks are organized on the fly depending on user features and actions, giving rise to different courses at the end. Additional support for further on-the-fly adaptation based on student learning styles has been incorporated in TANGOW: the strictness/flexibility of the navigational guidance offered, as well as the type of activities to be proposed and the sequencing between them, can be automatically modified. Regarding contents, different fragment versions can be provided, each of them intended for a different type of student (i.e. different languages, general vs. detailed explanations, or versions oriented to visual vs. textual learning style). Content fragment identifiers are associated to tasks, so that, from a set of fragments with the same id, the most suitable ones are selected at runtime to compose the web page to be presented to each student. Content fragments can contain texts, images, videos, simulations, animations, etc. It is also possible to include external resources distributed in different machines [Carro et al., 2001], such as web pages, applets or more sophisticated programs able to check student actions, compile code provided by students, or give specialized 138 Rosa M. Carro, Estefanía Martín feedback. The only requirement is for these resources/programs to be able to receive information about students and to send it back along with feedback about student actions, according to the corresponding protocol. This approach has been successfully followed in courses about language processors and financial maths [Suárez et al., 2000]. Open-ended questions can also be proposed to students through ATENEA [Alfonseca et al., 2004], a system able to check these answers by using natural language processing techniques and give feedback about answer correctness. In order to illustrate the creation of an adaptive course in TANGOW, the following sample scenario is presented. A course about Geometry for secondary school students will be created. It is related to plane figures perimeters. Students will accomplish different tasks to acquire knowledge about this topic, lumped together in a task called PlaneFig (see Table 12.1). Students without previous knowledge about the subject (rule ①) will perform three learning tasks: an Introduction about points, segments and angles; TypesFig, in which they will study different types of plane figures; and Perimeters, composed by sub-activities. These three tasks will be proposed to that type of students in the order in which they appear in the rule, as indicated (direct guidance). However, the introductory task will not be presented to students with previous knowledge about the subject (rule ②), and this type of students will be allowed to select the order in which they want to perform these activities (free guidance). Task Perimeters will be composed by other tasks related to theory, examples and exercises. All the students can perform these sub-activities in any order, as it can be seen in rule ③ (free guidance). This rule has no activation condition. Therefore, the rule will be triggered for all the students. Regarding Perimeters sub-tasks, PQuad, for example, contains four theoretical explanations about how to obtain the perimeter of squares, rectangles, rhombus and trapeziums (tasks PSquare, PRectangle, PRhombus and PTrapezium), some examples about these concepts (PExamples2) and several tests (PEx2), as it can be seen in rule ⑤. These tasks will be done by all the students (the rule has no activation condition) in strict order according to the (direct) guidance specified in the rule. Each task is defined once, although it can be part of several composed activities. Regarding contents, in this example two versions of theoretical explanations, examples and exercises related to perimeters and areas will be provided: one for students with visual learning style and other for those with verbal style.

Id Act. Condition Guidance Tasks Sub-Tasks ① knowledge=novice direct PlaneFig Introduction, TypesFig, Perimeters ② knowledge=advanced free PlaneFig TypesFig, Perimeters ③ - free Perimeters PTriCircle, PQuad, PPolygon ④ - direct PTriCircle PTriangle, PCircle, PExamples1, PEx1 ⑤ - direct PQuad PSquare, PRectangle, PRhombus, PTrapezium, PExam- ples2, PEx2 ⑥ - direct PPolygon TPolygon, PExamples3, PEx3 Table 12.1. Structural rules of the Geometry course.

Once the course components have been described, the course is dynamically generated at runtime for each student. Course generation in TANGOW is based on two steps: firstly, the most suitable activities are selected by checking rule conditions and triggering the corresponding ones; secondly, the most appropriate contents are joined to build the corresponding web page. Depending on the number of available tasks, either a menu or a web page to support task realization will be generated. In the first case, a menu is built starting from available task descriptions. In the second one, the system selects the most suitable content fragments among those associated with the task, by comparing the student features with the ones each version is intended for. Every page includes a progress bar, a button bar and an annotated table of contents, so that each student can know which tasks are (un)available at each time and which have already been performed. When a student finishes one activity, dynamic parameters related to actions (i.e. page visited, score, time) are propagated and recalculated recursively to maintain the user model updated. 12 Personalized Education 139 12.3 Computer-Supported Collaborative Learning (CSCL)

Collaborative learning [Dillenbourg, 1999] is a social activity in which knowledge is built [Koschmann, 1996] by students who are actively involved in the exploratory learning process working together. It facilitates the development and improvement of social skills such as working in groups or communicating with others. It also stimulates the development of personal skills such as making ideas explicit, arguing, or interacting with others to build common solutions, and increases student motivation, participation and auto esteem[Johnson et al., 1985]. Traditional e-learning systems can be enriched by the incorporation of Computer Supported Collaborative Learning (CSCL). From the 90s, many CSCL systems and applications have been developed. A selection of best known ones was made by Soller et al. [2005].

12.3.1 Adaptation in CSCL

As it was introduced in Section 12.1, in the context of e-learning it is very important to make sure that the activities to be carried out, as well as the tools provided to users, fit user needs, so that they feel comfortable interacting with the educational environment. Regarding collaborative systems, different groups of users may have distinct needs, even when tackling the same collaboration activity. Therefore, it is useful to adapt collaborative issues to facilitate collaborative learning. In order to perform adaptation in CSCL, it is necessary to store information about individuals and groups in the corresponding user and group models. Interesting group information is related to group members and roles (if specified), activities performed and pending, results obtained in previous activities, student opinions about previous collaborations, as well as dynamic data related to group and individuals performance. When a group of students accomplish collaborative activities, many factors can affect their performance, such as student level of knowledge, roles, motivation or personality, among others. Regarding criteria for group formation, different studies have been made in face-to-face collaborative learning [Slavin, 1980], which can serve as an orientation for Web-based collaborative learning. In [Johnson et al., 1985] it is stated that homogeneous groups usually achieve better specific aims; however, heterogeneous groups regarding abilities, experiences and interests can obtain more advantages that the former. In adaptive collaborative e-learning systems, automatic group formation is an interesting service to be provided. This functionality has been incorporated in some systems, such as [Carro et al., 2003a]. Some research works related to adaptation in CSCL are COALE [Furugori et al., 2002], WebDL [Gaudioso and Boticario, 2002] or COL-TANGOW [Carro et al., 2003a]. COALE is a collaborative learning environment able to recommend exercises, as well as the appropriate partners for each collaborative activity, to students. The main goal of WebDL is to facilitate access to services, making collaboration among members of the same group easier. It focuses on adaptive support for navigation and collaboration. Finally, COL-TANGOW is explained in detail next.

12.3.2 Adaptive Collaborative E-learning in COL-TANGOW

COL-TANGOW is an adaptive Web-based system that supports dynamic generation of adaptive collaborative Web-based courses [Carro et al., 2003a]. COL-TANGOW is an extension of TANGOW that supports the generation of courses in which collaborative activities (discussions about topics, communication among students and collaborative problem solving) are seamlessly integrated within courses and, moreover, adapted to users [Carro et al., 2003b]. Courses are generated at runtime by selecting, at every step and for each student, the most suitable individual and collaborative activities to be proposed, the time at which they will be presented, the specific problems to be solved, the most suitable partners to cooperate with, and the collaborative tools provided to each group of students to support collaboration. It also provides automatic group formation. Collaborative activities can be adapted to student and group features, preferences and actions, stored in the user and group models respectively. This type of activities is composed by a problem statement and a set of tools to support its collaborative realization. In order to support it in COL-TANGOW: (i) a new type of activity was defined, (ii) a set of collaborative tools was made available to support interaction and collaboration, and (iii) new types of adaptation rules were incorporated to allow workspace adaptation specification. 140 Rosa M. Carro, Estefanía Martín

Collaborative activities are created by giving information about the activity itself, providing the wording of the task to be tackled and specifying how collaborative tools will be combined to build collaborative workspaces for different groups of students. Several wordings can be associated to the same task, so that the most suitable one for a certain group of students is selected at runtime. Workspaces include a set of tools to support the collaboration, presented in the main interface, and additional tools available from this interface. Collaboration tools can be either selected from a pool of existing tools (based on PHProjekt1 and DistView2) or provided by course developers, and can be combined in different ways for each collaborative task and group of students. In Fig. 12.1, a sample collaborative workspace is presented. It has been generated to support the realization of task CollAct for advanced visual students, as it is explained below. In this case, a collaborative graphical editor and a chat are available in the main interface. Additional tools (a text editor, a forum and the upload utility) are accessible from it. The upload utility is always available for everybody in all workspaces.

Fig. 12.1. Example of workspace dynamically generated by COL-TANGOW.

To provide comfortable workspaces to support collaborative task accomplishment, collaborative-workspace rules and collaborative-tool rules are used. Their activation conditions indicate requirements for a workspace to be generated. They can be related to any user/group model parameter. In the course Geometry, the teacher wants to incorporate a collaborative activity related to Perimeters, with different problems depending on student level of knowledge. The collaborative-workspace rules used for its specification are shown in Table 12.2. The problem statements for novice and advanced students are identified by StCollNov and StCollAdv respectively. The tool to be used by novice students is a specific collaborative graphical editor identified by EdGr01, while the combination of collaborative tools for advance students is identified by ToolsA and is explained next.

1 http://www.phprojekt.com 2 http://www.eecs.umich.edu/distview/ 12 Personalized Education 141

Id Act. Condition Coll. Task Statement Tools ① knowledge=novice CollAct StCollNov EdGr01 ② knowledge=advanced CollAct StCollAdv ToolsA Table 12.2. Collaborative-workspace rules for Geometry course.

In this example, different tools will be offered to advanced students depending on their learning style (more speciﬁcally, regarding visual/verbal dimension). As it can be seen in Table 12.3 (rule ①), workspace ToolsA is described as a set of tools for the main interface (MainH1) and a set of additional tools (AddH1). All workspaces are described by a ﬁrst rule following this format.

Id Act. Condition Tools SubTools ① - ToolsA MainH1, AddH1 ② ls=visual MainH1 GraphicalEd, Chat ③ ls=verbal MainH1 TextEd, Chat ④ ls=visual AddH1 TextEd, Forum ⑤ ls=verbal AddH1 GraphicalEd, Forum Table 12.3. Collaborative-tool rules for the Geometry course.

The main tools to be provided to students with visual learning style will be a graphical editor and a chat (rule ②), whereas the graphical editor is replaced by a text editor for students with verbal learning style (rule ③). Regarding additional tools available, all students will be able to access to a forum and a different editor (rule ④ and ⑤). However, sometimes using available generic collaborative tools is not enough. In many occasions, teachers would like to make specific tools available for the realization of particular activities. For example, focusing on collaborative graphical editors, instead of providing users with a complex tool plenty of icons and functionality, it would be useful to configure editors in which specific icons are provided for task accomplishment. In such a way, on the one hand, suitable icons would be available for students to represent the solution to the activity proposed; on the other hand, students will not be distracted or even overloaded because of a big amount of icons, many of which are useless for specific tasks. This is the case of the collaborative graphical editor that supports the realization of task CollAct by novice students (EdGr01 in Table 12.2, rule ①). An example of the workspace generated in this case is shown in Fig. 12.2. The editor is composed by five areas. The first one contains a list of exercises to be performed within the activity (area ①). On the right-hand side (area ②), the current exercise wording is presented. A bar including icons to be used for building the solution appears in area ③. The main area consists of a shared working space (④). Finally, informative messages appear at the bottom of the interface (area ⑤)). The way of specifying this type of collaborative graphical editors is described in [Martín et al., 2008]. The last step consists in adding the collaborative task to the corresponding structural rules previously defined (in Table 12.4). In this example, novice students will perform the collaborative activity in the last place (rule ①). However, students with previous knowledge about the subject will be able to accomplish them whenever they prefer (rule ②).

Id Act. Condition Guidance Tasks Sub-Tasks ① knowledge=novice direct PlaneFig Introduction, TypesFig, Perimeters, CollAct ② knowledge=advanced free PlaneFig TypesFig, Perimeters, CollAct Table 12.4. Structural rules of Geometry course with a collaborative activity. 142 Rosa M. Carro, Estefanía Martín

Fig. 12.2. Example of conﬁgured collaborative graphical editor.

Once course components have been described, the adaptive collaborative course is dynamically generated for each student by selecting, at every step, the most appropriate activities to perform, problems to be solved, contents, collaborative tools and partners for each student depending on the information stored in the user model. Rule processing is similar to the one implemented in TANGOW, already described. When students are ready to accomplish a collaborative activity, COL-TANGOW groups them automatically, taking into account information stored in the user and group models. The default group size is three, following the recommendation of [Johnson and Johnson, 2002], although the course responsible can specify any other value. The whole process for dynamic generation of collaboration workspaces, including automatic group formation, is described in [Carro et al., 2003a]. Further research about the impact of learning styles [Alfonseca et al., 2006] as well as the inﬂuence of personality and intelligence on student grouping and their performance [Sánchez Hórreo and Carro, 2007] have been done, with the goal of obtaining student grouping criteria. In the sample course about Geometry, when a novice student with visual learning style ﬁnishes task Perime- ters, if students with similar features are ready to tackle this task, COL-TANGOW builds a group and proposes the task CollAct to the students. The corresponding collaborative workspace (see Fig. 12.1) is generated accordingly.

12.4 Mobile Learning

On one hand, the widespread development of mobile and wireless technologies is leading to the creation of new Web-based systems to support the realization of activities from diverse places at different times through various devices. Access to information is becoming increasingly ubiquitous. On the other hand, people usually spend a lot of time working and traveling and time has become a really valuable good in our society. These facts motivate the appearance of Web-based applications that facilitate ﬂexible realization of activities in different situations. In educational contexts, mobile learning applications have also been developed. Mobile learning (m-learning) has been deﬁned as e-learning through mobile and handheld devices using wireless transmission [Ktoridou and Eteokleous, 2005]. 12 Personalized Education 143

12.4.1 Recommendation in M-learning Systems

The accomplishment of individual and collaborative activities in m-learning environments can be inﬂuenced not only by user features, preferences or behaviors, but also by their context (i.e. device, location, available time, environment level of noise, temperature, weather, and so on) [Müehlenbrock, 2005]. This fact, along with the ones described above, suggest the possibility of creating systems that support the realization of activities in different contexts and, moreover, the recommendation of the most suitable activities to be accomplished in each particular context. For example, it would be inappropriate to propose activities to be supported through complex interfaces to students when they have not a suitable device to accomplish them; or it would not be good to propose them the realization of a difﬁcult and time-consuming activity when they have not much available time. Therefore, information about user context should also be stored to be utilized with recommendation purposes. Verdejo et al. [2006] presented an approach to support learning activities both outside the school and in the classroom. Students use PDAs with GPS cards to record bird observations about natural environment. When they are back to school, they are asked to analyze and process the empirical data captured using personal computers. Zimmermann et al. [2005] presented two examples of context-based recommended systems: an intelligent advertisement board and a museum guide. Both applications are able to adapt their behavior according to particular context attributes when they change. The CoMoLE system is explained next.

12.4.2 Context-Based Adaptive Learning in CoMoLE

CoMoLE is a context-based adaptive hypermedia system for m-learning Martín et al. [2007] that supports the recommendation and accomplishment of the most suitable activities to be carried out by each student at each situation according to his features, behaviors and context. The recommendations offered to each user in each context can depend on: learning styles, personality, intelligence, preferences, previous actions of users and their partners; and the specific context of users and partners at that time, including their location, available time and devices to support the interactions. The mechanism that supports the recommendations [Martín et al., 2006] is fed on the specification of parameters to be considered for adaptation, the description of the activities to be supported, and the rules describing the adaptation capabilities. CoMoLE user and group model creation is similar to that in COL-TANGOW, in the sense that any parameter whose values can be represented by a set of either discrete values or continuous intervals can be incorporated in the models. In addition, contextual features and agendas are used to adapt activities to situations and to check user availability. Priorities among parameters can also be specified. With the aim of helping teachers with the creation of this type of m-learning environments, a set of parameters, along with their potential values, has been defined, such as device (PC, PDA, laptop or mobile phone), physical location (classroom, laboratory, home, others, or any of them), personality, learning styles, level of knowledge or kind of information desired. Teachers can reuse them, add/modify possible values for these parameters, or define new ones. In addition to the activities supported by TANGOW and COL-TANGOW, CoMoLE also supports activities created by users at runtime, such as uploading/downloading files, sending messages, requesting/holding tutoring or taking tests in the classroom on the fly. Activities are grouped in sets of activities that can be related with a certain subject. Regarding collaborative activities, workspace adaptation is specified in terms of COL- TANGOW workspace and tool rules. Content fragment versions can be related to any user model parameter, including context; fragments are annotated according to the type of students and contexts they are intended for. Following with the sample course on Geometry, some features stored in the user model are: level of knowledge (novice/advanced), learning style (active/reflective and visual/verbal dimensions), physical location (classroom, laboratory, home, others), device used (PC, PDA, laptop or mobile phone) and available time (values between one minute and six hours). The activities incorporated in the environment are those described in Tables 12.1 and 12.4, along with three new activities: an exercise about perimeters to be carried out in the laboratory, a review of the course (GeometryR) and a message created on the fly by a student for his partner. Different content versions will be provided, related to visual/verbal learning style dimension and also to the device used to interact with activities. 144 Rosa M. Carro, Estefanía Martín

Once learning activities have been defined, relationships between them can be established in terms of structural rules. These rules are based on those in TANGOW, which have been extended in order to support complex conditions and recommendations. For the course Geometry, let us suppose that we have the same structural rules described in Tables 12.1 and 12.4 previously. Apart from that, it is possible to give general context-based recommendations over the whole set of activities. They can be specified in terms of context-based general rules, in which it can be stated whether certain types of activities should be proposed to all or some students depending on their situation. For example, if a user is in the classroom with his PDA switched on, it makes no sense to suggest him to engage in a collaborative task, since he should be participating and paying attention to the teacher and the students. However, if the teacher asks the students to answer a question through their PDAs during a lecture, they should send their answers immediately to the system, which will be able to check them and present the corresponding statistics at that time. Furthermore, different criteria can be used for giving recommendations to different students, even in the same context, since it could be desirable to propose different types of activities to dissimilar users according to their personal features and needs. For example, an active student could tackle an activity including exercises even if he has a short (but reasonable) amount of time, while it can be inappropriate to propose the same activity to a reflective student in the same situation. In Table 12.5, context-based general rules for the example are presented. Rule ① indicates that reviews, individual exercises or simulations can be proposed to students with active learning style when they have more than 15 minutes available and are in an unknown place (neither in the classroom, nor in the laboratory or home, maybe outside). Rule ② indicates that suitable activities for reflective students having more than 40 minutes available are related to theory, reviews or simulations.

Id User Cond. Context Cond. Type ① ls=active place=unknown AND t>15’ review OR indExercise OR simulation ② ls=reﬂective place=unknown AND t>40’ theory OR review OR simulation Table 12.5. Example of context-based general rules in CoMoLE for Geometry course.

Finally, activities may have specific requirements associated. These requirements can be defined by means of individual adaptation rules. For example, a contextual requirement for a test proposed to students in the laboratory out of the blue is specified in Table 12.6.

Id Act. Condition Activity ① place=lab PerimetersExerTest Table 12.6. Example of individual adaptation rule in CoMoLE for the Geometry course.

Once all the components of the m-learning environment have been described, students can connect to it through a Web browser to get recommendations in different contexts. The recommendation process is implemented in three steps: structure-based adaptation, context-based general adaptation and individual adaptation. Each step is implemented separately in sub-modules, each of them in charge of processing the corresponding rules in order to select, from a set of activities, the most appropriate ones for a user at a specific context. The output of the recommendation process consists of an annotated list of activities. In each step, this list is updated accordingly. Activities are annotated according to their suitability, following the traffic-light metaphor: (i) recommended (green color): all activity conditions are satisfied, if any; (ii) not recommended (yellow): any contextual condition is not satisfied; and (iii) not available (red): a non-contextual condition is not fulfilled. In the course about Geometry, let us suppose that a novice student with visual and active learning style connects to CoMoLE through his PDA. He has fifteen minutes available while traveling from home to the school. His partner sends him a message and he is ready to start the activity Perimeters. In this case, the first adaptation module annotates the three sub-activities of Perimeters as available and recommended, since: (i) 12 Personalized Education 145 the student is novice (the structural rule ① in Table 12.4 is triggered), and indicates that the subtasks must be performed in order; (ii) he has already performed the sub-activities Introduction and TypesFig, so the next activity to be performed is Perimeters; and (iii) the sub-activities of Perimeters can be carried out in any order (see Table 12.1, rule ③). Activity CollAct is annotated as not available due to the type of guidance specified in rule ①. The second module processes context-based general rules. Since the student is active and context-related conditions of rule ① (Table 12.5) are satisfied, this rule is triggered. Therefore, activity GeometryR is marked as recommended. The last adaptation module annotates PerimeterExerTest as not available, since the student is not in the laboratory. Finally, his partner message is sent. When the recommendation process finishes and the student selects an activity, the content management module selects the most appropriate versions of the contents for this activity. In the example, fragment versions for students with visual learning style using PDAs are selected. When the student finishes an activity, disconnects or changes context, information is stored/updated in the user model.

12.5 Conclusion

The use of adaptive hypermedia for education makes it possible to provide individual and group guidance to students during the learning process. There exist diverse possibilities to enrich this process. In this chapter, some of them have been described, related to individual, collaborative and mobile learning. Adaptive hypermedia makes it possible to develop advanced educational environments in which different types of resources are available and diverse educational activities are proposed to each student according to his features, needs, progress and context. However, there is still a lot of work to be done in different directions. The richer adaptive e-learning systems are, the more complex their creation is. Facilitating the creation and management of adaptive educational environments, making emphasis on educational strategies, by means of advanced but easy-to-use authoring tools, is essential for teachers to use this type of systems [Muñoz and Ortigosa, 2007]. Automating parts of the authoring process is also convenient to facilitate the process. Currently, most existing educational systems have their own representation models, adaptive strategies and mechanisms. Resource reuse has been addressed and some work has been done regarding one-to-one protocols, standards and web-services. Reusing other elements such as user/group models or adaptation strategies would give rise to new and more complete educational systems, adapted to speciﬁc needs. Some efforts are being done in this sense. Finally, evaluating adaptive educational systems and providing feedback to teachers for their improvement is also a key point. Some works have been done in this direction, such as [Bravo et al., 2007].

References

E. Alfonseca, R. M. Carro, M. Freire, A. Ortigosa, D. Pérez, and P. Rodríguez. Educational adaptive hypermedia meets computer-assisted assessment. In 2nd International Workshop on Authoring of Adaptive and Adaptable Educational Hypermedia, in conjunction with AH, Eindhoven, The Netherlands, 2004. E. Alfonseca, R. M. Carro, E. Martín, A. Ortigosa, and P. Paredes. The impact of learning styles on student grouping for collaborative learning: A case study. User Modeling and User-Adapted Interaction, 16(3-4): 377–401, 2006. J. Bravo, César Vialardi, and A. Ortigosa. A problem-oriented method for supporting AEH authors through data mining. In International Workshop on Applying Data Mining in E-learning, in conjunction with EC-TEL, Crete, Greece, 2007. P. Brusilovsky. Adaptive hypermedia. User Modeling and User Adapted Interaction, 11(1-2):87–110, 2001. P. Brusilovsky. Methods and techniques of adaptive hypermedia. User Modeling and User-Adapted Interaction, 6(2-3):87–129, 1996. 146 Rosa M. Carro, Estefanía Martín

P. Brusilovsky, E. W. Schwarz, and G. Weber. ELM-ART: An intelligent tutoring system on World Wide Web. In 3rd International Conference on Intelligent Tutoring Systems, Montréal, Canada, 1996. R. M. Carro, E. Pulido, and P. Rodríguez. Dynamic generation of adaptive Internet-based courses. Journal of Network and Computer Applications, 22(4):249–257, 1999. R. M. Carro, E. Pulido, and P. Rodríguez. A co-operative distributed environment for adaptive web-based education. In 3th Workshop on Adaptive Hypertext and Hypermedia, in conjunction with Hypertext, Aarhus, Denmark, 2001. R. M. Carro, A. Ortigosa, E. Martín, and J. Schlichter. Dynamic generation of adaptive web-based collaborative courses. Lecture Notes in Computer Science, 2806:191–198, 2003a. R. M. Carro, A. Ortigosa, and J. Schlichter. A rule-based formalism for describing collaborative adaptive courses. Lecture Notes in Artificial Intelligence, 2774:252–259, 2003b. O. Conlan, D. Dagger, and V. Wade. Towards a standards-based approach to e-learning personalization using reusable learning objects. In World Conference on E-Learning in Corporate, Government, Healthcare and Higher Education, Montréal, Canada, 2002. P. T. Costa and R. R. McCrae. The NEO-PI/NEO-FFI manual supplement. Odessa, FL Psychological Assess- ment Resources, 1989. P. de Bra, A. Aerts, B. Berden, B. de Lange, B. Rousseau, T. Santic, D. Smits, and N. Stash. AHA! the adaptive hypermedia architecture. In 14th ACM International Conference on Hypertext and Hypermedia, Nottingham, United Kingdom, 2003. P. Dillenbourg. Collaborative learning: Cognitive and computational approaches. Pergamon, 1999. R. M. Felder. Matters of style. ASEE Prism, 6(4):18–23, 1996. N. Furugori, H. Sato, H. Ogata, Y. Ochi, and Y. Yano. COALE: Collaborative and adaptive learning environment. In International Conference on Computer Support for Collaborative Learning, Boulder, Colorado (USA), 2002. E. Gaudioso and J.G. Boticario. Series on Innovative Intelligence, Vol. 1, chapter Supporting personalization in virtual communities in distance education, pages 327–362. World Scientific, 2002. D. Johnson, R. Johnson, P. Roy, and B. Zaidman. Oral interaction in cooperative learning groups: speaking, listening and the nature of statements made by high, medium and low achieving students. The Journal of psychology, 119(4):303–321, 1985. D. W. Johnson and F. P. Johnson. Joining together: group theory and group skills. Pearson Allyn & Bacon, 2002. J. Kay. Stereotypes, student models and scrutability. In 5th International Conference on Intelligent Tutoring Systems, Montréal, Canada, 2000. T. Koschmann. CSCL: Theory and practice of an emerging paradigm, chapter Paradigms shift and instructional technology, pages 1–23. Lawrence Erlbaum Associates, 1996. D. Ktoridou and N. Eteokleous. Recent Research Developments in Learning Technologies, chapter Adaptive m-learning: Technological and pedagogical aspects to be considered in Cyprus tertiary education, pages 676–683. FORMATEX, 2005. X. Li and Q. Ji. Active affective state detection and user assistance with dynamic Bayesian networks. IEEE Transactions on Systems, Man, and Cybernetics, Part A, 35(1):93–105, 2005. E. Martín, R. M. Carro, and P. Rodríguez. A mechanism to support context-based adaptation in m-learning. Lecture Notes in Computer Science, 4227:302–315, 2006. E. Martín, R. M. Carro, and P. Rodríguez. CoMoLE: A context-based adaptive hypermedia system for m- learning. In Simposio Nacional de Informática Educativa, Zaragoza, Spain, 2007. E. Martín, N. Carrasco, and R. M. Carro. Authoring and recommendation of collaborative graphical activities in context-based adaptive m-learning. International Journal of Computer Science and Applications, 5, 2008. M. Müehlenbrock. Formation of learning groups by using learner profiles and context information. In 12th International Conference on Artificial Intelligence in Education, Amsterdam, The Netherlands, 2005. F. Muñoz and A. Ortigosa. Adaptive hypermedia in secondary schools: from the teacher to the student. Inter- national Journal of Learning Technology, 3(3):309–323, 2007. R. Slavin. Cooperative learning. Review of Educational Research, 50:315–342, 1980. 12 Personalized Education 147

A. Soller, A. Martínez-Monés, P. Jermanna, and M. Muehlenbrock. From mirroring to guiding: A review of state of the art technology for supporting collaborative learning. International Journal of Artiﬁcial Intelligence in Education, 15(4):261–290, 2005. A. Suárez, E. Pulido, and R. M. Carro. An adaptive web-based course in ﬁnancial engineering. In World Conference on Educational Multimedia, Hypermedia & Telecommunications, Montréal, Canada, 2000. V. Sánchez Hórreo and R. M. Carro. Studying the impact of personality and group formation on learner performance. In 13th International Workshop on Groupware, Bariloche, Argentina, 2007. L. L. Thurstone. Primary Mental Abilities. Chicago University Press, 1938. M. F. Verdejo, C. Celorrio, E. Lorenzo, and T. Sastre-Toral. An educational networking infrastructure supporting ubiquitous learning for school students. In 6th IEEE International Conference on Advanced Learning Technologies, Kerkrade, The Netherlands, 2006. G. I. Webb, M. J. Pazzani, and D. Billsus. Machine learning for user modeling. User Modeling and User- Adapted Interaction, 11(1-2):19–29, 2001. A. Zimmermann, M. Specht, and A. Lorenz. Personalization and context management. User Modeling and User-Adapted Interaction, 15(3-4):275–302, 2005.

13 New Trends for Personalised T-Learning

Marta Rey-López, Ana Fernández-Vilas and Rebeca P. Díaz-Redondo

Department of Telematics Engineering, University of Vigo, 36310, Spain mrey,avilas,[email protected]

Summary. Although distance learning has used several media since its origins, nowadays, the majority of these learning experiences are aimed to be followed using a computer connected to the Internet. The predominant use of this medium for distance education has lead to a situation of digital divide where only those having access to this technology are able to take proﬁt of this type of education. In order to solve these problems, several initiatives have arisen with the aim of extending the scope of distance learning. In this scenario, television plays an essential role since its use is almost universal in developed and developing countries. This paper presents the state-of-the-art in t-learning (learning through Interactive Digital TV) focusing on personalisation of t-learning experiences. Personalisation is a key feature in this medium, because it makes these experiences more effective and attractive. Besides, it is even more necessary than in web-based learning, since viewers are used to TV as an entertainment medium, and will probably have a passive attitude towards education. However, current personalisation initiatives in t-learning are scarce. After studying the lacks and drawbacks of these works, we expose our system T-MAESTRO that has been designed to fulﬁl the needs of personalised t-learning, creating attractive learning experiences that combine learning content and TV programmes.

13.1 Introduction

In the last years, distance education has achieved great acceptance, particularly in courses for lifelong learning. This trend is substantially motivated by the difficulties to find enough time to attend to face-to-face courses in modern ways of living. Although distance education has used different media since its origins —such as post, TV or radio— nowadays, the preferred one is a computer connected to the Internet, generally referred to as web-based learning or simply e-learning1. However, the proportion of homes provided with Internet access is still reduced, for example, the rating of Internet usage as of March 2007 in the European Union is only around 52% and 70% in the US2. This fact leads to a situation of digital divide between those who have access to information technologies and those who have not, which has motivated the governments to look for popular new media, such as mobile phones or television, to offer these longlife learning experiences and thus facilitate social inclusion. In this scenario, the TV plays a particularly important role, since its penetration is almost universal in developed and developing countries. Although the term t-learning (learning through TV) has emerged in the last years, TV has always been used for educational purposes, in documentaries or children programmes such as the well-known Sesame Street. Nonetheless, these programmes may hardly be considered formal education and should better be described as edutainment. This term, first coined by Robert Heyman in 1973, is used to describe those programmes designed both for education and entertainment.

1 Although the term e-learning has a wider meaning that comprises any form of technology-supported distance learning, it is commonly used as a synonym for web-based learning. 2 Data retrieved from http://www.internetworldstats.com. 150 Marta Rey-López, Ana Fernández-Vilas and Rebeca P. Díaz-Redondo

Besides, TV has also been used with a more formal purpose, either to support or substitute face-to-face education (many times due to geographical constraints). For example, the programmes broadcast by the public TV channel in Spain to support the classes in the Open University UNED3; the transmission of reinforcement documentaries in the University of Laval [Boulet, 1996]; the Portuguese Telescola project, started in the 60’s to offer education to children in rural, suburban or high-density areas [Costa, 2003]; or the broadcasting of lessons in Hawaii University to the other islands of the archipelago4. Despite these forms of distance education through TV have succeeded in providing education through TV, the arrival of digital technologies permits to improve them because IDTV (Interactive Digital Television) provides a better quality of the signal, the reception of more channels than analogue TV in the same bandwidth and the possibility of delivering interactive applications along with traditional programmes. Interactivity is thus the key aspect of t-learning, which establishes the difference between this new concept and the previous experiences of education through TV. Interactivity was first offered on TV in the programme Winky Dink & you, a CBS show broadcast between 1953 and 1957 that provided interactivity by means of a magic drawing screen, a large piece of vinyl plastic which held on the television screen via static electricity, where children could draw using special crayons to help Winky Dink finish the adventures, e.g. with connect-the-dot activities [Thomasson, 2003]. If we mark out interactivity as the key aspect of t-learning, the examples mentioned above are not t-learning experiences, since they are not interactive. This term, understood as a subset of e-learning technologies, is thus defined as interactive learning through TV, in other words, interactive access to video rich learning materials primarily within the home, through a TV or a device more like a TV than a personal computer [Bates, 2003]. Some authors go beyond the exclusive use of TV and are in favour of the convergence of IDTV and e-learning technologies [DiSessa, 2000, Damásio and Quico, 2004] or even more additional media such as computers or mobile phones [Aarreniemi-Jokipelto, 2005]. Taking these previous experiences as a starting point, several initiatives are promoting social inclusion by offering learning experiences through IDTV. A good example of this efforts is Brazil [CPqD, 2004], a country where only 10% of homes possess an Internet-connected computer (this figure comes down to 0.25% in disadvantaged classes), whereas 96% of homes have access to TV (91% in disadvantaged classes) [Gomes Soares and Lemos de Souza Filho, 2007]. One of the most disadvantaged areas in Brazil is Amazonas; however, the penetration of TV and mobile phones in this region is surprisingly high. Taking this fact into account, the project Interactive Education [de Oliveira et al., 2006], whose objective is using satellite TV to bring education closer to the inhabitants of this area, is developing appropriate set-top boxes as well as educational software that mimics the behaviour of the menu of a mobile phone to facilitate the learning process. Equally important is the Beacon (Brazilian-European Consortium for DTT Services) project5 that is working in the creation of t- learning services for social inclusion in the region of São Paulo with two different objectives: training teachers that work with students with special needs, as well as providing courses to prepare the exams for university access (which are currently being offered by private institutions). Apart from the Brazilian initiatives, there are also some other original efforts for social inclusion through t-learning. This is the case of T-islessia [Bordoni, 2006] whose aim is to diagnose and rehabilitate subjects who are inclined to dyslexia, or the project deployed in some British prisons [niace, 2004] where TV has been used to allow the prisoners to learn in their own cells to improve their basic skills, such as numeracy and literacy. Last but not least, The eLearning Project (TeLP) [Russell et al., 2004] is aimed at offering e-learning courses to these users who cannot access them through an Internet-connected computer. Thus, it provides a multiplatform architecture where courses are stored in XML format and adapted to the constraints of every platform on run- time. Apart from its wide world usage that facilitates social inclusion, the most important asset of IDTV concerning education is the fact that people is used to TV and hence it is regarded as an easy-to-use medium and they trust in the contents broadcast on TV. Nevertheless, the reason that causes its main asset causes its main drawback as well. It is true that people is used to TV, but as an entertainment medium that only requires them to sit on the sofa and watch what is shown on the screen. Therefore, they tend to be passive viewers instead of active

3 http://www.uned.es/cemav/tv.htm 4 http://www.tlearning.tv/news_detail.asp?id=252 5 http://www.beacon-dtt.com/ 13 New Trends for Personalised T-Learning 151 learners. This fact, together with the wide variety of learners in the context of TV —which means that they have very different learning goals, styles and preferences—, makes it necessary to motivate them even more than in web-based learning. This motivation should be accomplished by attracting users towards education, providing personalisation to avoid them to get lost in the huge amount of contents available and to offer learning experiences that are interesting and appropriate for them. Personalisation has always been accomplished by good teachers in face-to-face education, using specific strategies to address individual needs and make education more effective, since the same content may be be assimilated differently by different students because of their backgrounds. These traditional techniques should be exported to the field of e-learning (understood in a wide sense of technology-supported learning) to take profit of the benefits of personalised learning and be able to offer education with solid pedagogical impact. In this chapter, we expose the current initiatives to provide education through IDTV (Sec. 13.2), focusing in personalisation of learning experiences (Sec. 13.2.3) to analyse their restrictions. To fulfil the needs of personalisation in t-learning, we have created a system that provides personalised learning experiences that combine entertainment and education, which we present in Sec. 13.3. Finally, we draw our conclusions and motivate our future work in Sec. 13.4.

13.2 Related work

Once we have clarified the definition of t-learning, this section exposes the current experiences and research efforts in this field. These initiatives have not completely broken with previous approaches, on the contrary, they are based on the main lines exposed above: edutainment programmes as well as support and substitution of face- to-face education. In the next subsections, these initiatives are exposed, classifying them in formal, non-formal and informal education. Formal education consists of organised learning systems hierarchically structured and chronologically graded. Non-formal education is comprised by any organised educational activity outside the established formal system, with identifiable learners and evident learning objectives. Last but not least, informal education is defined as the “truly lifelong process that allows individuals to acquire knowledge, skills and values from daily experience” [Coombs et al., 1973].

13.2.1 Formal and non-formal education

Although using TV for formal education seems to be against the entertainment-oriented character of this medium, there are multiple experiences where t-learning has been used in this direction, either replacing face- to-face courses or supporting them. The most straightforward way of replacing face-to-face courses with t-learning ones is broadcasting the image of the teacher to the students and vice-versa. As mentioned above, this is the case of the University of Hawaii, where TV is used since 70s to transmit the image of the teacher to the students of other islands of the archipelago, which began to send back the image of the students to the teacher —thus becoming interactive— in 2002. A similar approach is the one deployed in the Campus Satellitare del Salento6 where previous classes can also be accessed; or the one in [Zhao, 2002], where the assets and drawbacks of this use of TV for learning are studied. Formal education on TV can be seen as an adaptation of e-learning multimedia [Sintresis, 2005], which consists in broadcasting audio and video ﬂows where the teacher explains the lesson supported by parallel graphical elements (such as slides) transmitted synchronously. In a similar way, the work presented in [Baldi et al., 2006] permits not only to receive the teacher’s image and slides, but also to access information regarding timetables or exams dates, or even chatting with the teacher by means of the return channel. Likewise, TV can also be used for formal learning without broadcasting the image of the teacher but the contents themselves, in a similar way than e-learning. This is the case of a course belonging to the Master of Computer Science in the University of Helsinki, within the scope of the Motive project [Aarreniemi-Jokipelto, 2006, 2004], where the contents are based on audio and video and can be provided with metadata in order

6 http://campusdelsalento.it 152 Marta Rey-López, Ana Fernández-Vilas and Rebeca P. Díaz-Redondo to make easier their reusability. Besides, interactivity is provided through an application that permits to send messages to other participants from a virtual keyboard on the TV, a mobile phone, a PDA or even through a form on the Internet. Several initiatives are aimed to support and training of teachers, and social inclusion by means of education. For example, the educational channel TV Escola7 has been launched in 1996 for training teachers, becoming interactive in 2003 and augmenting its target public to assist in students’ education as well. Along the same line is SAPSA (Serviço de Apoio ao Professor em Sala de Aula) [Ferreira do Amaral et al., 2004], a project that supports teachers by providing them with a TV set and a set-top box that allow them to browse learning contents, consisting of short videos, animations, text and pictures or pages with hyperlinks that permit a web- like navigation; as well as the authoring tool to help teachers and students create learning content to be watched on TV exposed in [Trindade dos Santos et al., 2006, 2005b,a].

13.2.2 Informal education

Informal education in IDTV is the natural evolution of traditional educational programmes such as Sesame Street. There are currently two different approaches to create informal learning experiences in IDTV: interactive edutainment and non-educational programmes enhanced with additional learning material. In the first group, the most relevant efforts are being taken by the BBC8, such as CBeebies, oriented to young children and designed to develop preschool skills; GCSE Bitesize, which offers reinforcement activities to secondary students; or the series of interactive documentaries Walking. . . . Besides, several other channels have created interactive learning applications for TV: such as the Oxford dictionary; the videos to learn foreign languages of HomeChoice; the LivingHealth service that allows to ask medical questions by phone and receive the answers on TV; the interactive applications of the Finnish channel YLE Teema9 or the German programme LexiTV [Krömker and Kunert, 2004, Kunert, 2003]; the Portuguese programmes BarraPanda, Batatoon and Portugal dos Pequenitos [Damásio and Quico, 2004]; the interactive contents of the Italian channel TVL for learning foreign languages; or the applications developed in the context of the Motive project [Aarreniemi- Jokipelto, 2006] for learning sign language or communicating with other viewers through messaging. In the second group, several projects are aimed to create learning experiences from existing TV programmes. This is the case of the system The Parent Trap [Lackner, 2000] that uses children programmes to suggest talking points between children and parents. This system sends an e-mail to the parents each time their children watch an educational TV programme, notifying the amount of time spent and the topics dealt with during that period and proposing several related questions that can be asked to the kids. In the same vein, the proposal exposed in [Prata et al., 2004] allows the viewers to label the programmes according to their interests and receive a link to a specially-created web page with additional information when the programme ends. Along the same line of complementing existing TV programmes, a field that can take profit of t-learning in a great extent is language learning since it can use any programme broadcast in the language to be taught [Underwood, 2002, Pemberton, 2002, Fallahkhair et al., 2004]. The most relevant effort in these field has led to the system TAMALLE [Fallahkhair, 2007] that supports English learning by enhancing TV programmes broadcast in this language with difficult vocabulary, showing it on the TV screen or on the user’s mobile phone.

13.2.3 Personalisation

In spite of the importance of personalisation in t-learning mentioned in Sec. 13.1, there are currently very few efforts in this direction. However, its importance has already been suggested, such as in [Luckin and Du Boulay, 2001], which proposes reusing and recombining existing educational material using a broadband user model, i.e. a student model in a broad sense that takes into account different contexts and perspectives; and reusable learning objects, labelled according to their relation to other elements, target user, information related to their usage and feedback from the user [Luckin and Du Boulay, 2003].

7 http://tvescola.mec.gov.br/ 8 http://www.bbc.co.uk/ 9 http://www.yle.ﬁ/teema/ 13 New Trends for Personalised T-Learning 153

Some TV channels have also carried out some efforts towards personalisation of learning content, such as the BBC, that has produced some educational contents where the students are able to follow different routes, but it is the user, not the system, who decides which is the appropriate one [Mudge, 1999]. Along the same line, we can find the different video on demand services offered by the content provider Chaos Media Network that can be personalised according to the user’s needs10 and the Portuguese programme Barra Panda, that had associated an ITS (Intelligent Tutoring System) to adaptively model the content while the child made progresses [Damásio and Quico, 2004]. On another hand, the proposal exposed in [Ghaneh, 2004] presents a filtering system that stores in the users’ PDR (Personal Digital Recorder) a user profile consisting on a set of categories, their values and relative importance. When educational contents are received, their metadata are compared to the users’ profiles to decide whether they are appropriate for them or not. Although there have been very few initiatives to offer personalisation in t-learning, we need to have in mind the numerous approaches to personalisation in web-based education. These systems are generically known as Adaptive and Intelligent Web-based Educational Systems (AIWBES). As indicated in their name, they comprise two different subtypes of personalisation systems: Adaptive Hypermedia (AH) systems and Intelligent Tutoring Systems (ITS). The former adapt to the user’s characteristics as much in the way they show the information as in the information itself. The ITSs, in turn, try to teach specific contents as an expert would do, taking into account the learners’ evolution and the knowledge acquired in order to detect and correct their mistakes.

Fig. 13.1. Adaptive systems operation.

An AH system is the one that provides personalised access to hypermedia information and can be useful in any application area where the system is expected to be used by people with different goals and knowledge and where the hyperspace is reasonably big [Brusilovsky, 1996]. These systems collect data about the user and process it to create the user model that is used, in turn, to create the adaptation effect (Fig. 13.1). The techniques employed in adaptive hypermedia systems are usually classified attending to whether the content or the links of the hypermedia documents are adapted. The former techniques are named adaptive presentation and consist of explanation variants [Beaumont, 1994], stretchtext (the explanation of a concept replaces the content if the user does not know it) [Boyle and Encarnación, 1994], conditional text [De Bra and Calvi, 1998], etc. The latter, called adaptive navigation, consists in modifying the links according to the user profile, e.g. hiding [De Bra and Calvi, 1998], ordering or annotating [Brusilovsky et al., 1996], or even offering direct guidance to the students [Conlan, 2005]. Some of these techniques can also be applied in the field of audiovisual contents, such as TV programmes, as studied in [Masthoff and Pemberton, 2005, 2003], for example, hiding some periods of the programme, making bigger the most interesting areas of the screen, etc. Intelligent Tutoring Systems, in turn, are defined as those systems that offer personalised training in a direct way, i.e. without human intervention. The techniques used to perform this personalisation can be classified in two groups [Brusilovsky and Peylo, 2003]: curriculum sequencing and problem solving support. The former is intended to guide the student in finding the optimal path through the learning material. The latter is subdivided in three subgroups:

10 http://www.pjb.co.uk/t-learning/case25.htm 154 Marta Rey-López, Ana Fernández-Vilas and Rebeca P. Díaz-Redondo

Intelligent solution analysis consists of evaluating the student’s solutions, determining whether they are • correct and offering support in those concepts that are not known [Brusilovsky et al., 1996]. In interactive problem solving support has te same goal as above, but it does not wait until the students • have ﬁnished the activities to help them [Brusilovsky et al., 1997]. Finally, example-based problem solving support consists of suggesting related examples that have already • been studied by the learner [Brusilovsky et al., 1996]. This division is certainly not disjunct, since modern personalisation systems cannot be classiﬁed in one of the groups but in both, as demonstrate the existing systems that implement both types of strategies to offer personalised education on the web, such as MEDEA [Trella et al., 2005], ADAPT2 [Brusilovsky, 2004] or TANGOW [Carro Salas, 2001].

13.3 T-MAESTRO

In order to solve the lack of personalisation in t-learning, we have designed a system that takes as a starting point the personalisation techniques exposed in the previous section that are already being used in web-based learning. The result of this work is T-MAESTRO (T-learning Multimedia Adaptive Educational SysTem based on Reassembling TV Objects), an adaptive and intelligent system designed to offer educational experiences for t-learners that combine learning elements and TV programmes [Rey-López et al., 2006a]. In its design, we have focused on the personalisation of these experiences in two different ways: selecting the most appropriate and interesting objects for the users —both audiovisual and educational ones— and adapting the learning elements to their preferences and background. In this context, we have envisaged two types of experiences that both have an informal approach to education (Fig. 13.2, bottom). On the one hand, we call edutainment experiences those ones that are intended to entertain while educating. The core of these experiences is a learning element which is complemented with some pieces of related TV programmes to make it more entertaining. On the other hand, along the lines of creating learning experiences from TV programmes exposed in Sec. 13.2.2, we have coined the term entercation for those experiences that are aimed to educate while entertaining and whose main element is a TV programme, which is complemented with related learning elements [Rey-López et al., 2007]. Its main goal is taking profit of the entertainment character of TV programmes to engage users in education. From the point of view of the users, edutainment experiences take place when they are offered a course that is complemented with some pieces of related TV programmes (TV in Fig. 13.2) that are appropriate to support the explanation or practice of the concepts, taking into account the users’ interests. In the example shown at the lower-left corner of Fig. 13.2, the viewer is following a course of Italian and he/she likes football matches, thus, the course is complemented with a piece of a football match broadcast in Italian to help him/her with listening comprehension. If the same course is followed by another viewer that likes mystery series, he/she would be shown a piece of “Provaci ancora Prof!”, an Italian series where a high school teacher helps a police superintendent in solving homicide cases. Entercation experiences, in turn, are aimed to fulfil the users’ curiosity while they are watching TV programmes. This curiosity is identified by comparing their interests with the description of the pieces of the programme they are watching and fulfilled by offering related learning elements (LEs in Fig. 13.2). These elements explain the concepts that the users do not know but are interested in learning about. The example shown at the lower-right corner of Fig. 13.2 depicts how the film “Under the Tuscan Sun” can be complemented with related activities, such as a documentary about Campania (the region where some of the scenes of the film take place), a course of Italian (because some of the characters of the film speak Italian) or a course of Tuscan art. A viewer that likes travelling and learning foreign languages will be shown the first two activities, whereas the last one is offered to those people that want to learn art. Besides, learning elements can be adaptive. This means that they will not be equally offered for all the users. On the contrary, they can have the ability to be personalised to the user’s preferences and background. In the examples explained above, the course of art in Tuscany offered to the viewer that is watching “Under the Tuscan Sun” can be personalised to his/her background, showing a simple introductory lesson for those 13 New Trends for Personalised T-Learning 155

Fig. 13.2. Phases in the creation of t-learning experiences. that have never studied art and a course to know all the architectural details of Tuscan constructions to those that have knowledge about the subject. Concerning the course of Italian exposed in the edutainment case, if it is adaptive, those lessons that are too easy according to the student’s background can be suppressed and some lessons can have different activities for students with different interests. For example, lesson 3 proposes an exercise in a travel agency that is useful for those students that like travelling but for students that like cinema, it would be more useful a situation where they learn how to buy cinema tickets.

13.3.1 Normalisation issues

To achieve our objectives, an environment based on widely accepted standards is advisable in order for reusability of components and interoperability between systems to be possible. Hence, T-MAESTRO works within the technological context deﬁned by the MHP (Multimedia Home Platform) standard [Digital Video Broadcasting (DVB), 2006], which is consolidating worldwide as one of the technical solutions that will shape the future of 156 Marta Rey-López, Ana Fernández-Vilas and Rebeca P. Díaz-Redondo

IDTV. It defines an open interoperable solution that normalises the characteristics of the set-top boxes and the applications they can execute. Concerning the structure and annotation of audiovisual content, our system supports TV-Anytime metadata [TV-Anytime Forum, 2004]. It structures the programmes in segments, defined as temporal intervals within an audiovisual stream —i.e. continuous fragments of a programme (TV in Fig. 13.2). It also allows to define segment groups, i.e. collections of segments that are grouped together, for a particular purpose or due to a shared property. TV-Anytime permits to add metadata to describe both entire contents and its segments to provide information such as its title, synopsis, genre, as well as a set of keywords, some links to external material related or the list of credits for the segment (actors, directors, etc). Concerning learning elements, given that t-learning is still in an early state, the research efforts in this field should take into account the experience accumulated in e-learning to avoid making the same mistakes, such as its initial lack of standardisation. For this reason, we have used a modified version of the ADL SCORM (Sharable Content Object Reference Model) [Advanced Distributed Learning (ADL), 2006] standard that permits adaptation. The SCORM standard references specifications, standards and guidelines developed by other organisations that have been adapted and integrated with one another to form a more complete and easier-to-implement model. It is divided into technical books grouped under three main topics:

Fig. 13.3. SCORM Content Aggregation Model.

The SCORM Content Aggregation Model (CAM) defines the components used to build a learning experience (LEs in Fig. 13.2) from learning resources and how they are aggregated and organised into higher-level units of instruction. It defines five different components (Fig. 13.3). Assets are electronic representation of media that can be collected together to build other assets. If this collection represents a single launchable learning resource that utilises SCORM RTE to communicate with an LMS, it is referred to as an SCO (Sharable Content Object). An Activity is a meaningful unit of instruction that may provide a learning resource (SCO or Asset) or be composed of several sub-activities. To create a course, the activities compose a Content Organization, a map that represents the intended use of the content through structured units of instruction. Last but not least, a Content Aggregation is an entity used to deliver both the structure and the resources that belong to a course in a Content Package, which consists of a compressed file with the physical resources of educational content and at least one XML file —called manifest— that embodies a structured inventory of the content of the package. In order to describe these learning elements, the SCORM standard has adopted the IEEE Learning 13 New Trends for Personalised T-Learning 157

Object Metadata (LOM) [IEEE Learning Technology Standards Committee (LTSC), 2002] that promotes the reusability of these contents. The SCORM Run-Time Environment (RTE), in turn, deﬁnes how to launch content objects, how to exchange information with SCOs and which are the standard set of elements that can be exchanged (Data Model).

13.3.2 Different stages in the creation of t-learning experiences

To create the t-learning experiences mentioned above, we need to compose and personalise them. The former is performed by means of establishing relationships between TV programmes and learning contents; the latter, in turn, is double-sided, since it is aimed to filter out the contents, both educational and TV programmes, that are not appropriate for the users (selection), as well as to customise the learning elements to the users’ preferences and background (adaptation). Fig. 13.2 depicts the different stages that are needed to create these learning experiences. The first stage refers to the selection of those contents that are appropriate for the users according to their preferences stored in their profiles. In t-learning, the profile is double-sided, since it takes into account the user’s characteristics as a viewer and a student. Hence, we need two different subsystems to perform the selection of the contents. On the one hand, to select audiovisual contents we use AVATAR [Blanco-Fernández et al., 2004], developed by our research group. This system is intended to recommend TV contents to the viewers according to their profiles, using a hybrid filtering approach that combines content-based methods, collaborative ones and semantic inference (see [Blanco-Fernández et al., 2007] for the details). Its reasoning algorithms are aimed to establish relationships between TV contents and user profiles. With this aim, an OWL [World Wide Web Consortium (W3C), 2004] ontology has been created, which stores a conceptualisation of TV programmes based on the TV-Anytime metadata. To sum up, the core of this ontology is a class hierarchy whose root element is the TVContent; and each instance of the ontology (corresponding to a TV content) is related to its respective semantic characteristics through explicit properties (hasActor, hasGenre, hasPlace, etc). For the viewer profile, this system has preserved the ontological philosophy used for content modelling to facilitate the inference ability. The first approach to user modelling based on ontologies was proposed by Middleton [2003]. In our case, this ontology-profile maintains the subset of classes and individuals belonging to the TV ontology that the viewers have watched as well as their opinion about them. This opinion, named DOI (Degree of Interest) can be obtained either if the users indicate voluntarily their opinion about the programme to the system or by inferring their opinion from the proportion of the programme that they have watched. In order to export these ideas to the selection of the learning content, we have created an educational ontology based on the SCORM standard (see [Rey-López et al., 2007] for more details) inspired by the one for IEEE LOM presented by Qin and Hernández [2004] and available at [Qin, 2001]. We have extended this ontology in such a way that ours not only stores data belonging to LOM categories, but it also permits the creation of instances for every SCORM element and considers their interrelations, e.g. the SCOs and Assets that belong to an Activity, the different Activities of a Content Organization, etc. The student profile maintains the ontological nature, in the same manner as the viewer’s but it contains instances of the SCORM ontology instead of the TV ontology. In this case, not only the users’ interest in the content should be taken into account, but also their knowledge on the subject. The latter, named DOK (Degree of Knowledge) is computed following the rules established by the SCORM standard from the performance of the users in the SCOs that they have followed. The algorithms for learning content selection are thus applied in the same way as AVATAR’s ones. After the selection process, those elements that are not appropriate for the user have already been filtered out. The next phase consists in creating learning experiences from TV programmes and learning elements. First, adaptation should be performed for adaptive learning elements, customising them for the users by selecting the most appropriate way —among those available— for the user to achieve the intended objective. Since SCORM does not currently permit adaptivity, we have proposed an extension to the SCORM standard that permits us to introduce it at two levels in the hierarchy: activities and SCOs, as shown in Fig. 13.3. Before explaining these extensions, it is worth defining adaptation parameters, which consist in some characteristics of the users the adaptive learning elements can adapt to, such as their favourite sport or their knowledge on a particular subject. These parameters are independent of the particular representation of the user 158 Marta Rey-López, Ana Fernández-Vilas and Rebeca P. Díaz-Redondo profile in the system that shows the content to the student and have to be inferred from the profile information by means of ad-hoc inference rules. Adaptation at activity level [Rey-López et al., 2006b] consists in providing different internal structures for an activity —so that its objectives could be fulfilled in different ways, according to the learners’ preferences and educational background— as well as the rules that the adapter should use on reception to choose the most appropriate structure among the provided ones. This means that not all the subactivities of an activity have to be studied by all the users, only those ones that are appropriate for their preferences and background. In this manner, experienced users on the topic of the activity will only be offered revision subactivities, while novice users will be shown explanatory ones as well. To indicate which activities should be offered depending on the characteristics of the target student, adaptation rules are provided to associate adaptation parameters and subactivities. For this to be possible, we have added a new category to the SCORM manifest with the objective of indicating there adaptation rules at activity level. Adaptation at SCO level [Rey-López et al., 2006c], in turn, has been added to convert SCOs in self-adaptive SCOs, providing them with different behaviours and with an adaptation file that contains the rules that are needed for these objects to adapt to users’ characteristics on run-time. The standard explains the method of data exchange for SCOs: they can request and write the values for the elements contained in the RTE Data Model using the get and set API methods. In this manner, the our extension to the standard consists in adding the adaptation parameters to the Data Model, in order for the SCOs to be able to access this information. Once the contents that are not appropriate for the users have been discarded and the adaptive learning elements have been customised for the target users, the last stage is the composition of the learning experiences. In this phase, the contents are linked to each other to be shown to the user. For this to be possible, we need to establish semantic relationships between TV programmes and learning elements. As mentioned above, the selection of contents using AVATAR’s algorithms consists in establishing relationships between an instance of the TV or SCORM ontology and the instances of these ontologies stored in the user’s profile. In the same manner, these algorithms can be easily exported to establish relationships between TV contents themselves, between LEs themselves and even between TV contents and LEs. However, the relationships between TV contents and LEs are more difficult to establish, since the former are described using TV-Anytime metadata whereas the latter are conformant with the SCORM standard. To reason over them, we can use the TV programmes and SCORM ontologies, respectively, but we need a third one to act as a bridge between them to be able to apply the reasoning algorithms. For this purpose, we use SUMO (Suggested Upper Merged Ontology) [Niles and Pease, 2001], which is being created by the IEEE Standard Upper Ontology Working Group to develop a standard upper ontology that promotes data interoperability, information search and retrieval, automated inferencing, and natural language processing. As mentioned in the previous section, TV programmes or its segments are annotated by means of keywords that describe its content. LEs, in turn, are described by means of LOM metadata, which allows to define the concepts that are explained in learning units referencing an element on a particular hierarchy. For this work, we suppose that this hierarchy is SUMO, thus, once mapped the keywords that describe the TV programmes to an element of this ontology using a lexical language database like WordNet11, AVATAR’s algorithms are applied to compute the relationships between these two elements of the SUMO ontology.

13.4 Conclusions and further work

This paper has exposed the main initiatives in t-learning and has motivated the need for personalisation in this ﬁeld to make learning experiences more attractive and effective. Personalisation in t-learning is even more necessary than in web-based learning, due to the typical passiveness of t-learners, i.e. viewers, since they are used to regard TV as an entertainment medium. After studying the scarce current initiatives in personalising t-learning experiences, we have presented T-MAESTRO, which was designed to provide personalised learning experiences to viewers, combining TV programmes and learning elements. The creation of these experiences takes place in three steps: (1) selection

11 http://wordnet.princeton.edu/ 13 New Trends for Personalised T-Learning 159 of the most appropriate elements, (2) adaptation of learning elements and (3) composition of the experiences by relating both types of elements. For the first and last steps, we have proposed to adapt the semantic reasoning algorithms based on ontologies used by AVATAR. However, these algorithms are very heavy for current IDTV receivers and have to be run in remote servers. Moreover, they assume that content creators describe the contents by means of metadata, but this assumption is not always true. In order to solve these constraints, we are working on an algorithm that takes profit of successful new technologies in the field of Web 2.0, such as collaborative tagging and folksonomies. The former permit the users to label the contents themselves, expressing their points of view and allowing them to easily retrieve those contents later. The latter are the structures that are created as a consequence of collaborative tagging. These structures provide information about the relationships between tags, assuming that two tags are strongly related if they co-occur frequently. Using folksonomies, we will be able to establish relationships between contents, by computing the relations between the tags that describe them. Another current line of research is exporting the techniques used to compose entercation experiences from LEs and TV programmes to other fields of IDTV, such as personalised advertising. In this manner, TV programmes can be used to show related advertisements to the viewers, in order to increase their effectiveness. Besides, these techniques can also be used to publicise TV programmes themselves, offering them to the viewer when he/she has finished watching a related programme in a similar way than in online stores, where they offer the client related products to the one he/she is browsing.

References

P. Aarreniemi-Jokipelto. T-learning model for learning via digital TV. In 16th EAEEIE Annual Conference on Innovation in Education for Electrical and Information Engineering, Lappeenranta, Finland, 2005. P. Aarreniemi-Jokipelto. Experiences with an interactive learning environment in digital tv. In IEEE Interna- tional Conference on Advanced Learning Technologies, Joensuu, Finland, 2004. P. Aarreniemi-Jokipelto. Modelling and Content Production of Distance Learning Concept for Interactive Digital Television. PhD thesis, Helsinki University of Technology, 2006. Advanced Distributed Learning (ADL). Sharable Content Object Reference Model (SCORM). http://www.adlnet.org, 2006. M. Baldi, A. De Santis, D. Falcone, E. Gambi, and S. Spinsante. A t-learning platform based on digital terrestrial television. In International Conference on Software in Telecommunications and Computer Networks, Split, Croatia, 2006. P. J. Bates. A study into TV-based interactive learning to the home. http://www.pjb.co.uk/t-learning, 2003. I. H. Beaumont. User modelling in the interactive anatomy tutoring system ANATOM-TUTOR. User Modeling and User-Adapted Interaction, 4(1):21–45, 1994. Y. Blanco-Fernández, J. J. Pazos-Arias, A. Gil-Solla, M. Ramos-Cabrer, B. Barragáns-Martínez, M. López- Nores, J. García-Duque, A. Fernández-Vilas, and R. P. Díaz-Redondo. AVATAR: An advanced multi-agent recommender system of personalized TV contents by semantic reasoning. Lecture Notes in Computer Sci- ence, 3306:415–421, 2004. Y. Blanco-Fernández, J. J. Pazos-Arias, A. Gil-Solla, M. Ramos-Cabrer, M. López-Nores, J. García-Duque, A. Fernández-Vilas, R. P. Díaz-Redondo, and J. Bermejo-Muñoz. AVATAR: Enhancing the personalized television by semantic inference. International Journal of Pattern Recognition and Artiﬁcial Intelligence, 21 (2):397–422, 2007. Fondazione Ugo Bordoni. Distance rehabilitation: An experimental project for Interactive TV. http://convegni.cineca.it/T-islessia/t-islessia_en.pdf, 2006. M.-M. Boulet. A television distance education course to teach the management of software projects. In IEEE In- ternational Conference on Multimedia Engineering Education, pages 129–133, Melbourne, Australia, 1996. C. Boyle and A. O. Encarnación. Metadoc: An adaptive hypertext reading system. User Modeling and User- Adapted Interaction, 4(1):1–19, 1994. P. Brusilovsky. KnowledgeTree: A distributed architecture for adaptive e-learning. In 13th International World Wide Web Conference, New York, USA, 2004. 160 Marta Rey-López, Ana Fernández-Vilas and Rebeca P. Díaz-Redondo

P. Brusilovsky. Methods and techniques of adaptive hypermedia. User Modeling and User-Adapted Interaction, 6(2-3):87–129, 1996. P. Brusilovsky and C. Peylo. Adaptive and intelligent web-based educational systems. International Journal of Artificial Intelligence in Education, 13:153–169, 2003. P. Brusilovsky, E. Schwarz, and G. Weber. ELM-ART: An intelligent tutoring system on World Wide Web. In 3rd International Conference on Intelligent Tutoring Systems, Berlin, Germany, 1996. P. Brusilovsky, S. Ritter, and E. Schwarz. Distributed intelligent tutoring on the web. In 8th World Conference on Artificial Intelligence in Education, Amsterdam, The Netherlands, 1997. R. M. Carro Salas. Un Mecanismo Basado en Tareas y Reglas para la Creación de Sistemas Hipermedia Adaptativos: Aplicación a la Educación a través de Internet. PhD thesis, Universidad Autónoma de Madrid, 2001. O. Conlan. The Multi-Model, Metadata Driven Approach to Personalised eLearning Services. PhD thesis, University of Dublin, Trinity College, 2005. P. H. Coombs, R. C. Prosser, and M. Ahmed. New paths to learning for rural children and youth. Interchange, 5(3), 1973. L. Costa. O ensino à distância em Portugal. http://ead.no.sapo.pt/mundo/frameset2P.html, 2003. CPqD. Projeto sistema brasileiro de televisão digital. modelo de implantação. www.fndc.org.br/arquivos/MapeamentoCadeiadeValor-CPQD.pdf, 2004. M. Damásio and C. Quico. T-learning and interactive television edutainment: the Portuguese case study. In 2nd European Conference on Interactive Television, Brighton, United Kingdom, 2004. P. De Bra and L. Calvi. AHA: a generic adaptive hypermedia system. In 2nd Workshop on Adaptive Hypertext and Hypermedia, Pittsburgh, USA, 1998. J. A. de Oliveira, T. Waisman, F. Santos da Silva, B. Horbe, O. B. Maia, and F. Morenghi. T-learning in Amazon: The Interactive Education Project. In 4th European Conference on Interactive Television, pages 290–295, Athens, Greece, 2006. Digital Video Broadcasting (DVB). Multimedia Home Platform specification 1.1.1. European Standard ETSI TS 102 812 V1.2.2, 2006. A. DiSessa. Changing Minds: Computers, Learning, and Literacy. The MIT Press, 2000. S. Fallahkhair. Learner-centred development of learning objects for interactive television. In World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007, Vancouver, Canada, 2007. S. Fallahkhair, J. Masthoff, and L. Pemberton. Learning languages from interactive television: Language learners reflects on techniques and technologies. In World Conference on Educational Multimedia, Hypermedia & Telecommunications, Lugano, Switzerland, 2004. S. Ferreira do Amaral, L. O. Baratti, D. Moutinho Bataca, J. H. Augustinis Franco, J. M. Martin Rios, and A. da Costa Lamas. Serviço de apoio ao professor em sala de aula baseado na TV digital interativa. Revista Digital de Biblioteconomia e Ciência da Informação, 1(2):53–70, 2004. M. Ghaneh. System model for t-learning application based on home servers (PDR). Broadcast technology, 19, 2004. L. F. Gomes Soares and G. Lemos de Souza Filho. Interactive television in brazil: System software and the digital divide. http://www.cs.tut.fi/˜lartur/euroitv07_ajp/Keynotes0.htm, 2007. IEEE Learning Technology Standards Committee (LTSC). Learning object metadata. IEEE Standard 1484.12.1, 2002. H. Krömker and T. Kunert. Designing a learning dialogue for educational applications in digital interactive TV. In 2nd European Conference on Interactive Television, Brighton, United Kingdom, 2004. T. Kunert. Interactive educational TV applications based on the Multimedia Home Platform (MHP). In Online Educa Berlin, 9th ICWE International Conference on Technology Supported Learning & Training, Berlin, Germany, 2003. T. M. Lackner. Enhancing Children’s Educational Television with Design Rationales and Justifications. PhD thesis, Massachusetts Institute of Technology, 2000. R. Luckin and B. Du Boulay. Embedding AIED in ie-TV through broadband user modelling (BbUM). In 10th International Conference on Artificial Intelligence in Education, San Antonio, USA, 2001. 13 New Trends for Personalised T-Learning 161

R. Luckin and B. Du Boulay. Lines of desire: the challenges of interactive educational TV. In 11th International Conference on Artiﬁcial Intelligence in Education, Sydney, Australia, 2003. J. Masthoff and L. Pemberton. Adaptive Hypermedia for Personalised TV, pages 246–263. IDEA Group, 2005. J. Masthoff and L. Pemberton. Adaptive learning via interactive television. In 11th International Conference on Powerful ICT Tools for Teaching and Learning, St. Petesburg, Russia, 2003. S. E. Middleton. Capturing Knowledge of User Preferences with Recommender Systems. PhD thesis, University of Southampton, 2003. R. Mudge. Interactive Television, pages 125–144. Kluwer Academic Publishers, 1999. niace. Television for learning. http://www.offenderlearning.net/?q=node/12, 2004. I. Niles and A. Pease. Towards a standard upper ontology. In 2nd International Conference on Formal Ontology in Information Systems, Ogunquit, USA, 2001. L. Pemberton. The potential of interactive television for delivering individualised language learning. In Inter- national Workshop on Future TV: Adaptive Instruction in your Living Room, San Sebastián, Spain, 2002. A. Prata, N. Guimarães, and P. Kommers. iTV enhanced system for generating multi-device personalized online learning environments. In 4th Workshop on Personalization in Future TV, Eindhoven, The Netherlands, 2004. J. Qin. The educational resource ontology (EduOnto). http://web.syr.edu/˜jqin/eduonto/eduonto.html, 2001. J. Qin and N. Hernández. Ontological representation of learning objects: Building interoperable vocabulary and structures. In 13th international conference on World Wide Web, New York, USA, 2004. M. Rey-López, A. Fernández-Vilas, and R. P. Díaz-Redondo. A model for personalized learning through IDTV. Lecture Notes in Computer Science, 4018:457–461, 2006a. M. Rey-López, A. Fernández-Vilas, R. P. Díaz-Redondo, J. J. Pazos-Arias, and J. Bermejo-Muñoz. Extending SCORM to create adaptive courses. In 1st European Conference on Technology Enhanced Learning, Crete, Greece, 2006b. M. Rey-López, A. Fernández-Vilas, R. P. Díaz-Redondo, J. J. Pazos-Arias, and J. Bermejo-Muñoz. Adaptive learning objects for t-learning. In 5th International Conference on Web-based Learning, Penang, Malaysia, 2006c. M. Rey-López, R. P. Díaz-Redondo, A. Fernández-Vilas, and J. J. Pazos-Arias. Entercation: Engaging viewers in education through TV. ACM Computers in Entertainment, 5(2), 2007. T. Russell, T. Varga-Atkins, and S. Smith. Enticing learners: Rethinking the relationship between e-learning via DiTV and the Internet. In 2nd European Conference on Interactive Television, Brighton, United Kingdom, 2004. Sintresis. T-learning - le opportunita’ offerte da una nuova tecnologia. http://www.sintresis.it/sintresis/ /documents/TLearning.pdf, 2005. M. Thomasson. Winky Dink, the history of interactive television, and you. http://www.gooddealgames.com/articles/Winky_Dink.html, 2003. M. Trella, C. Carmona, and R. Conejo. MEDEA: an open service-based learning platform for developing intelligent educational systems for the web. In Workshop on Adaptive Systems for Web-Based Education: Tools and reusability, Amsterdam, The Netherlands, 2005. D. Trindade dos Santos, R. Cascão Araújo, R. Miranda de Barros, and L. G. Pedroso Meloni. Desenvolvimento de aplicativos para ensino a distância através da TV digital. In I Workshop TV Digital - SIGGRAPI, Natal, Brazil, 2005a. D. Trindade dos Santos, M. da Rocha C. Silva, and L. G. Pedroso Meloni. Ferramentas de apoio ao ensino a distância via TV digital interativa. In Taller Internacional de Software Educativo, pages 145–152, Santiago, Chile, 2005b. D. Trindade dos Santos, D. T. do Vale, and L. G. Pedroso Meloni. Digital TV and distance learning: Potentials and limitations. In 36th Annual Frontiers in Education Conference, San Diego, USA, 2006. TV-Anytime Forum. Broadcast and on-line services: Search, select and rightful use of content on personal storage systems. European Standard ETSI TS 102 822, 2004. J. Underwood. Language learning and Interactive TV. In International Workshop on Future TV: Adaptive Instruction in your Living Room, San Sebastián, Spain, 2002. World Wide Web Consortium (W3C). Web Ontology Language (OWL). http://www.w3.org/2004/OWL/, 2004. 162 Marta Rey-López, Ana Fernández-Vilas and Rebeca P. Díaz-Redondo

L. Zhao. Interactive television in distance education: Beneﬁts and compromises. In International Symposium on Technology and Society, Raleigh, USA, 2002. 14 M-Learning: Personalized and Pervasive Education Anytime, Anywhere and for Everybody

Mario Muñoz Organero1, Elena Yndurain2 and Carlos Delgado Kloos1

1 Departamento de Ingeniería Telemática, Universidad Carlos III de Madrid Av. Universidad 30, 28911 Leganés, Madrid (Spain) munozm,[email protected] 2 Nokia Spain Cerro de los Gamos 1, 28224 Pozuelo de Alarcón, Madrid (Spain) [email protected]

Summary. M-learning is a concept associated to the use of mobile devices to access the contents and services either from distance learning management systems using Internet connections or from local context-dependent learning-aware devices and services. Although limited in computing resources, mobile devices are normally personal devices that we use anytime-anywhere adding new personal-space-time dimensions to the learning process. Using mobile devices, learning users can be pervasive consumers of distributed learning resources, but at the same time, they can be active parts in the e-learning scenario, offering contents and services to other learning users in a context dependent architecture. Mobile devices are in many cases personal devices that contain information about their associated personal user profiles as well as some user dependent learning resources. Combining these devices either in a predefined or ad hoc manner we can create personalized and pervasive distributed learning architectures. The any-time any-where association of a mobile device to a particular learning user enhances the adaptation possibilities to the physical and perceptual user-context for a user oriented learning process. This perceptual user-context can be sensed by the user’s mobile device by using short range network technologies such as Bluetooth, RFID and NFC. The intelligent devices in the user’s environment are able therefore to interact with the user’s learning mobile device providing information and adapting the properties of the user’s environment. In this chapter we are going to describe how to create personalized m-learning scenarios. First, we are going to give a detailed overview of the state of the art in mobile device’s capabilities for m-learning. Second, we are going to describe some of the major technological alternatives used by mobile devices to sense and interact with their learning environment. Third, we are also going to concentrate on the middleware needed in mobile devices to provide and consume e-learning personal services. To finalize the chapter we are going to put everything together describing some different m-learning scenarios.

14.1 Introduction

M-learning is a concept associated to the use of mobile devices to access the contents and services either from distance learning management systems using Internet connections or from local context-dependent learning- aware devices and services. Although traditionally limited in computing resources, mobile devices are normally personal devices that we use anytime-anywhere adding new personal-space-time dimensions to the learning process. Moreover, mobile technologies are experiencing a huge development in software, hardware, user interface capabilities and network connectivity creating a suitable environment not only for distance learning service access but also for learning service execution. 164 Mario Muñoz Organero, Elena Yndurain and Carlos Delgado Kloos

Using mobile devices, learning users can be pervasive consumers of distributed learning resources, and at the same time, they can be active parts in the e-learning scenario, offering contents and services to other learning users in a context dependent architecture. Mobile devices are in many cases personal devices that contain information about their associated personal user profiles as well as some user dependent learning resources. Combining these devices either in a predefined or ad hoc manner we can create personalized and pervasive distributed learning architectures. The any-time any-where association of a mobile device to a particular learning user enhances the adaptation possibilities to the physical and perceptual user-context for a user oriented learning process. This perceptual user-context can be sensed by the user’s mobile device by using short range network technologies such as Bluetooth, RFID and NFC. The intelligent devices in the user’s environment are able therefore to interact with the user’s learning mobile device providing information and adapting the properties of the user’s environment. This chapter makes first a review of the state of the art mobile technologies to later describe the conceptual and architectural issues of a personalized and pervasive distance education or m-learning environment. We present the technological innovations and future trends in mobile technologies that enable the creation of new services such as those oriented to mobile education (connectivity, user experience, hardware and software) and apply them to m-learning architectures, pervasive, personal learning management systems and context- enhanced personalized learning environments.

14.2 Mobile Technology Overview

Technology is worldwide so embedded in our day to day existence that today a mobile phone is seen as an extension of the individual; more than 1.2 billion people are connected to the Internet [InternetStats, 2007], 2.5 billion people are mobile telephony subscribers [VisionGain, 2007] and more than 43 thousand own some type of Personal Computer [IDC, 2006]. Mobile phones have now evolved into multimedia computers with high capacities that make them perfect tools for mobile education. Devices such as smart mobile devices are already heavily used globally, more than 49 million devices were sold by June 2007 [Canalysis, 2007]. In the following sections mobile technology as an enabler for mobile education is analyzed in terms of: user experience, software, hardware, connectivity and services.

14.2.1 User experience

User experience is a holistic view of the interaction of the user with the mobile device intended to make the use emotionally desirable [Nokia, 2006b]. People want intuitive, natural and ﬁt-for-purpose interaction in the mobile context. Mobile phones have evolved into hand-held multi-use devices: ﬁfteen years ago devices were heavy (around 10 kg) and power consumption driven, now they are slim (less than 100g) and computing powered that makes them friendlier for users. Their use goes beyond voice: they are now used to access, organize and store information. Mobile devices’s technology allows users to enjoy a better experience through advanced multimedia communication which results in more immediacy and spontaneity in the way we communicate. Users now interact differently with technology, they have evolved from passive to interactive and contributors. For this reason user experience is even more crucial today to enable this new ways of communication. User experience can be seen as the combination of two major factors:

Usability ensures that the user interacts with the product properly. It refers to the product’s ability to fulfill • the user’s goals and needs efficiently in a specified context of use. It is a property of the entire system, which includes the product, the user, use target and the context of use [Nokia, 2006b]. Several aspects enrich usability, one is response time and another is ease of use, all with an appealing design. Response time is affected both by the device’s processing time when navigating or executing applications and by the connectivity speed when transferring data. Easiness when interacting with the device is highly determined by the physical interaction with the device, also known as input method. Users demand new input interfaces 14 M-learning: Personalized Education Anytime, Anywhere and For Everybody 165

and redesign of traditional key layouts that make interaction simpler, several input methods are available: one-handed, two-handed, touch screen and voice interaction. Each of these options enables different user experiences which are more convenient for different situations. Keyboards are also evolving to emulate regular Personal Computer ones such as qwerty, virtual and folding ones. User Interfaces are composed of the device screen, content organization and now also Internet browsing. • Screens are deﬁned both by their size and the number of pixels they can display. The ratio of dots per inch is the resolution of the display [Zwick et al., 2005]. Display technologies and resolution are improving to allow a better image quality, today liquid color display based screens are the ones that are more used. Screen resolution and color depth has evolved and is now up to more than 176x208 with a color depth of more than 16bit [Nokia, 2006a]. Dynamic organization of space is one of the aspects that help to render content in the mobile device and make it easier for users to browse. An example is 3D user interfaces that facilitate content rendering and usage. Sensor user interfaces that use the device’s capacities such as the camera are becoming more common in new mobile solutions. Multisensioral user interfaces [Ianucci, 2006] are becoming more popular because they bring all ﬁve senses close to mobile technology; users interact with the mobile devices by tilting, shaking, tapping and gesticulating (these are all physical interaction).

User experience can highly be customized with new materials that enable users to choose their own electronics and make their own unique products. An example of this is the interaction with the environment through sensors which can be applied to m-education. Sensors are capable of collecting, processing and communicating massive amount of data with minimal size, weight and power consumption. Moreover, on Internet services users can personalize their content due to new web based technology that is easier to use. User experience opens up new possibilities to interact with the mobile device and it can improve the learning process, i.e. text input is made easier, multimedia experience allows for more visual education applications to be developed, sensorial interaction enables an interactive experience with the user.

14.2.2 Software

As mentioned before, mobile devices can offer the same functionalities as desktop computers, in addition to telecommunication ones such as voice and data communication, their operating system support both communications and computing. We can divide the mobile device’s conceptual architecture into 3 layers: User Interface, Software and Operating System as can be seen in Fig. 14.1.

Fig. 14.1. Conceptual mobile software architecture. 166 Mario Muñoz Organero, Elena Yndurain and Carlos Delgado Kloos

Interface Framework User Interface defines mobile device’s software appearance (menu and navigation). It is composed of a set of graphic components and it takes care of several functions such as: soft key definition, navigation, application swapping, call handling and text input. This framework can be highly customized by developers and personalized by end-users. Many things can be changed according to each individual’s preferences: i.e. background, color settings, icons aspect, wallpapers, keys usage, etc. The lack of space on small screens makes the dynamic organization of space one of the most important aspects in order to allow fast scrolling and the presentation of files in an easier way to find the content. Different concepts are defined in order to be able to display the whole page on those applications where it is needed. An example is Internet applications and browsing. Examples of content rendering for mobile Internet browsing is the fishnet concept in which part of the content iscompressed and another is the minimap in which the full page is rendered and part of the content is highlighted.

Application execution environment This environment contains both sets of application programming interfaces to develop applications that will run over the operating system and the software framework which contains the mobile device’s logic. The software framework is composed by the mobile device’s own modules for: communication, telephony, messaging, and multimedia as well as by other party created by third parties. Communications module enables the mobile devices to discover and exchange data with any number of • devices either mobile or fixed (servers, PCs, etc). Web services for example needs to communicate the mobile device with the Internet. Telephony module is used to handle regular phone functionalities such as initiating calls and terminating them. Messaging module is a flexible client/server-based framework for receiving, sending, and manipulating • messages such as e-mail, short message service (SMS), and multimedia messaging service (MMS). This module is highly customizable. Multimedia module has audio and video codecs, their availability and their performance are device- • dependent. Different high-quality formats are now available on mobile devices for image, sound and video such as: WB true tones, RealAudio, mp3, AAC+, RealVideo, MPEG-4,etc Third party applications are proliferating favored by several factors, • 1. Operating systems have become more flexible and open to integration of external applications. 2. They can be programmed in well known languages such as C, C++, flash or Java which makes development easier. 3. Trends on separating interfaces between front-end and business logic that enables reusability of software components. 4. Modular designing of applications that allows quick integration of different parts to create mashups. Operating Systems are optimized for small mobile devices with limited storage capacity and power supply. The best known mobile devices Operating Systems are: Apple OS X, Linux mobile, Microsoft windows mobile, Palm OS, RIM OS and Symbian OS. A comparison between these operating systems can be seen on Fig. 14.2. Software can be used to personalize the mobile device for specific uses. This personalization can even be done after the device is in use by sending a new software version (also known as firmware) remotely over the air or by installing in the same manner third party applications. Internet technologies provide a rapid application development with an excellent user experience using XML, JavaScript or CSS that is equivalent to that of standalone applications. Mobile devices can also be personalized with existing applications from different fields: multimedia players, worksheets, internet services, word processors, worksheets which enhance the mobile device in an education environment.

14.2.3 Hardware Hardware has to meet the new needs of current mobile devices related to convergence that have been speciﬁed before (phone, multimedia and desktop functionalities). This increased mobile functionality demands more memory, more processing power and better power consumption. 14 M-learning: Personalized Education Anytime, Anywhere and For Everybody 167

Fig. 14.2. Operating system comparison.

Memory

Increased mobile functionalities such as multimedia content and applications demand more storage capacity. There are currently three main types of memory solutions for mobile handsets:

In-built handset memory takes the form of semiconductor memory (typically NOR or NAND flash), a • non-volatile memory device and the form of a electrically erasable programmable read only memory Flash Memory (a hybrid between RAM and ROM). NOR is the most used currently because it is better suited to execution of code but NAND is more effective where high memory densities are required [VisionGain, 2005]. There are two main architectures NOR flash plus RAM and NAND flash plus RAM, there is also a mix between both. Mobile architecture usually involves a RAM to flash ratio of 1:4, 4MB of RAM to 16MB of flash. Removable memory cards are similar to those found in Personal Computers although they have no moving • parts and they allow faster access, they are solid state electronic flash data storage devices [VisionGain, 2005]. They allow hot swapping which means that users can switch memory cards while the mobile device is switched on. There are three main types of memory cards today: Multi-Media Card (MMC), Secure Digital (SD) Cards and Memory Stick. – MMC were created in 1997 their size is as small as postage stamps. They typically use ROM technology for read only applications and flash memory for read/write applications. They are also durable, with a protective shell and a storage capacity up to 2GB and increasing. A smaller version the MMC is also available with the same capacity. – SD is an improvement on the MMC it is very similar in size, capacity and transfer rates. The main difference is related to security features (which MMC did not allow at first) and also on this card open standards are not allowed. – Memory Stick was released in 1998 and it is mainly used by the Sony brand. They are similar to MMCs and SDs although they have some differences on the way they manage security (they also have copyright protection). A newer version, namely Memory Stick Pro, is available with faster speeds and larger capacity. New removable flash memory cards are being developed today including those based on universal flash storage formats (UFS) with better performance levels reducing to few seconds access a 4 gigabite multimedia content rather than the 3 minutes it usually takes. Remote storage consists on storing handset data on remote servers accessible by wireless access (via • TCP/IP protocol) which is done through a remote storage platform accessible via web. It overcomes the memory card problem that one user might need several cards to store all their data.

A comparison of the three solutions can be seen on Fig. 14.3 below. External memories such that can be used in education just as CDs were used with personal computers since they now have enough capacity to store educational content in them and it is easy to distribute them physically in a class-room. 168 Mario Muñoz Organero, Elena Yndurain and Carlos Delgado Kloos

Fig. 14.3. External memory comparison.

Mobile device engines Mobile devices engines have evolved in the last years to a more modular design which can respond better to new trends and adapt to new industry open standards. Evolution has been possible especially through the miniaturisation of technology; each component of the engine (Figure 14.4) has evolved to new convergent functionalities and smaller sizes. In this ﬁgure, BB (signal on its own frequency) takes care of cellular functionality processing signal and controlling • cellular logic, its advance Personal Assistance functionalities and system support functionality to manage accessory interfaces, memories, energy management and start-up/state control. RF takes care of all the communication through the antenna switch (duplexer) the receiver, transmitter and • the synthesizers. More an more transmission modes are included on the RF side which translates into more components placed together and switching needs.

Fig. 14.4. Engine.

The main current issues related to hardware components are as follows: Performances which are still based on cellular system requirements rather than the new convergent ones, • new processors are needed to meet new requirements. Power consumption which is affected by the system activity and the clock speed, voltage, integrated circuit • process type and chip size. Cost of components in terms of amount, sizes and assembly • Electromagnetic compatibility between components • Processors Processors run at an usual speed of the range of 623MHz. In order to better optimize mobile device’s performance, some manufacturers have designed their hardware based on two processors to have a low-power system that separates general devices functionality from communications. 14 M-learning: Personalized Education Anytime, Anywhere and For Everybody 169

There are basically three processor types:

The Master Controller Unit handles functionality that can be implemented in general-purpose software • such as user operating system (Personal Digital Assistant functionality) and upper layers of cellular protocols. The Digital Signal Processor handles processing-intensive algorithms such as voice compression or cellular • error correction accelerating speciﬁc algorithms. It is controlled by the Master Controller Unit software. The Hardware takes care of the low-level cellular processing, accelerating video processing and encryption. • Chip designers are adding custom accelerators, vector processors and multiple parallel processing cores to increase processing performance. Multicore chip designs, which are common in server computing, are also emerging in the mobile domain.

Energy Management

Mobile devices have a battery limitation associated to form factor restrictions and new functionalities. The use of large colored displays, sophisticated moving images and computation needs has led to a rapid growth in the amount of energy needed by mobile devices. Finding a sensible balance between battery life and advancement of multimedia technologies plus device size is a challenge. Although remarkable improvements have happened in mobile devices around miniaturization of the device engine, the mechanics, UI, battery, and antennas are still lagging behind. Growth in stand-by battery time is what allows for mobile services to become a reality. Regarding batteries, there are four basic types of rechargeable ones used in mobile phones:

Nickel Cadmium (NiCd). The technology behind this battery type has been around for many years, since • 1948. It has two major drawbacks, one is that it is a heavy metal battery which means that the chemicals in Nickel Cadmium are not environment friendly and the other is the memory effect or voltage depression which means that if the battery is not fully discharged before recharging it, after a few cycles, the battery "learns" this low water mark, and acts as if it is discharged at this point. Nickel Metal Hybride (NiMH). They were developed at the beginning of the 1990s and they are supposed to • be better than then NiCD because the do not contain cadmium, because they are less prone to the "memory effect" problem, and because they have a higher capacity in relation to their size and weight. However they do not last as long as the NiCd. Lithium Ion (Li-Ion). This is the current technology used for mobile phone batteries. Li-Ion gives excep- • tional capacity for its size and weight, and does not suffer from the memory effect. The only real drawback of Li-Ion is that they are expensive, and so they tend to be supplied only in top-of-the-range phones. Li-Ion polymer. They are very similar to Lithium Ion, except that they can be moulded into more varied • shapes, and so be squeezed into smaller phone casings.

There are two newer types of batteries nowadays:

Zinc-air. The technology behind Zinc-air batteries is not new, it is environment friendly and inexpensive. • They have high capacity and low self-charge rate when they are sealed. They have a high energy capacity up to 350Wh/kg. Direct Methanol Fuel Cell. This technology has the greatest potential since it has more efﬁciency and • energy density. Portable Power Energy density in portable power is likely to continue doubling every six years for the foreseable future, but arrival of fuel cells would change user experience from recharging to reﬁlling.

Comparing the different rechargeable battery systems and their speciﬁc energy values (Figure 14.5) we can see that the most promising one is fuel cell technology. Fuel cells, methanol, and hydrogen are likely to become the key enablers for further evolution. Nanotechnology can be applied to batteries to provide super-fast recharging and long lifetime by using nanoparticles in the negative electrode of a battery. 170 Mario Muñoz Organero, Elena Yndurain and Carlos Delgado Kloos

Fig. 14.5. Speciﬁc energy values battery comparison.

14.2.4 Connectivity

Cellular connectivity is part of the engine module of the mobile device and it is usually integrated inside although it can also be an external component such as an antenna. New connectivity options become standards that allow integrating the device within a system and with other devices in many types of networks. Connectivity technologies accelerate the commoditization of access and support the shift to services. There is an interplay of different connectivity technologies that surround the user in its environment (see Fig. 14.6)

Fig. 14.6. Connectivity technologies.

There are different connectivity technologies that live on a same mobile device and that work at different distances: 14 M-learning: Personalized Education Anytime, Anywhere and For Everybody 171

Broadcast connectivity to receive radio and television is available through different internal modules: a • radio module in the device’s baseband allows frequency modulation (FM) from normal broadcast radio stations, and DVB-H (Digital Video Broadcasting, Handheld) is the mobile version of the DVB-T digital television specification which enables reception from a regular TV broadcast station. Wide area transmission has many types of cellular connectivity living with each other on the same device • that is used for voice or data transmission purposes. Difference between these technologies is based on speed for data transmission ranging from slowest to fastest: GSM/CDMA, EDGE, WCDMA and HSDPA. Local area and Personal zone connectivity technologies like Wireless Local Area Network (WLAN), Blue- • Tooth and InfraRed connect the mobile device with other consumer electronic devices for transmission at different ranges (being the former the closest range) in a peer to peer mode. They all operate on the ISM (Industrial, Scientific, Medical) band which is 2.4GHz unlicensed band and each transceiver module is located on the device’s baseband. – WLAN is the wireless version of Ethernet allowing the device to connect into a local area network through a WLAN access point. There are several versions of WLAN which operate at different speeds and frequency: 802.11g 54Mbps 2.4GHz ISM, 802.11a 54Mbps 5GHz and 802.11b (WiFi) 11Mbps 2.4GHz ISM. – Bluetooth operates at a short/medium range radio (10m-100m) accepting both voice and data links. One device can be connected via Bluetooth to up to seven devices (one master and seven slaves). It is based on radio-based link that does not require any line-of-sight connection in order to communicate. Unique security keys and robust encryption assure reliability and confidentiality. This technology provides almost unlimited possibilities for new kinds of applications and solutions. – Infrared Data Association port is included in the mobile device for physical transmission through infrared light (875nm) at a 1 meter range. IP convergence enriches communications by allowing new services to be created that connect the mobile device into any IT or telecommunications system. Finally, Touch zone refers to a direct connection between mobile device and other devices that might be • mobile or not. Examples of this type of connectivity are Near Frequency Communications (NFC) and wired connectivity. – NFC is a short-range wireless connectivity that works at a 13.56-MHz range, with a data exchange rate between 106 kbps and 424 kbps. This technology offers simple, intuitive communication between electronic devices in which two NFC-compatible devices interact with each other within a few cen- timetres distance. A NFC enabled mobile devie works like a standard contactless smart cards that are used worldwide in credit cards and in tickets for public transit systems, all the user needs to do is wave the phone at a reader. In an educational environment, students can pick up information from the environment using this technology since information can be stored in NFC tags that can be placed on everyday objects. – Wired connectivity through the device’s USB or pop-port connectors. It is used for data transmission between the mobile device and any USB host. Nowadays it is also used for power charging the mobile device using the Personal Computer or laptop USBs port as a charger.

14.2.5 Service Evolution

Due to Information Technology, Telecommunications and Media convergence users are now experts on using technology and now expect more services that allow them to be connected, several technology evolutions have lead to this,

Evolution of software tools for data management that help content classiﬁcation and discovery. • High speed and ubiquitous access to Internet anywhere any time. • Internet applications open nature in which the user is the producer. • HW price decrease provides master devices with more storage capacity. • 172 Mario Muñoz Organero, Elena Yndurain and Carlos Delgado Kloos

Lifestyle driven holistic solutions are less about technology and more about customizing and personal technology. There has been a mindset shift in the way technology is used enabled by Web 2.0 services, people now want to be part of a social community that share the same interests. M-learning is an example of a community based service that can be used for educational purposes to enable students to work on a collaborative way. Mobile devices can be deployed to build m-learning services such that:

work on data created by students • store data on remote servers so that it will not be lost • make them accessible ubiquitously by the students, their teachers and other people involved (i.e. parents) • teachers can distribute learning material to students • Mobile communication domain has expanded embracing the user’s daily environment, new types of contextual information are now being gathered by various forms of sensors (location information, motion, physical state) and from existing behavior patterns, this allows for services personalization.

14.3 Personalized learning using mobile devices

As it has been described in the previous section, the current hardware, software and network capabilities of mobile devices (although limited compared to desktop solutions) convert them into well suited platforms for distance learning. Compared to other devices, mobile terminals are normally personal devices that accompany mobile users anytime-anywhere. These characteristics define a new learning paradigm in which new dimensions for personalization are possible. In fact, m-learning is not just a simplified extension of e-learning but a new enhanced and personalized scenario for distance education. Mobile devices can learn from the user, what, when and under which circumstances he or she performs learning actions and facilitate the learning process. Mobile devices can interact with the physical intelligent learning objects in the user’s environment adapting them to the user’s needs. Mobile devices can discover other nearby users and exchange profiles with their mobile devices so that other interesting and interested users can be found. Finally, the mobile device can be the always accessible personal repository for learning contents and services providing a collaborative environment for learning. This section examines in detail the different uses of mobile devices for distance learning. First we present a review of the general architecture for mobile learning. Then we examine the personal, distributed learning management system concept. Finally we concentrate on how the intelligent learning objects in the user’s physical environment can adapt to the user’s needs creating a context enhance personalized learning environment.

14.3.1 The mobile learning architecture

An m-learning architecture should be general (able to provide all possible services already provided to e- learning users) and generic (able to support different mobile devices). We can go a step further and state that a general and generic m-learning architecture should also enhance some key aspects of traditional e-learning architectures among which personalization is one of the most important. We dedicate this sub-section to present some proposals in the literature that try to define a framework for a mobile learning architecture. We study these proposals, analyze their merits and drawbacks and combine them in a general and generic m-learning architecture. The work presented by Trifonova and Ronchetti [2004] proposed an architecture for m-learning that sits on top o a conventional e-learning architecture. In other words, the proposed architecture uses an extension of a central Learning Management System (LMS) to provide contents and services to m-learning users. In fact, the authors proposed the use of the term mLMS (or mobile LMS) as the extension needed in an LMS to handle the interaction with mobile devices. An mLMS, therefore, sits on top of a conventional eLMS (e-learning LMS). The functionality inside an eLMS is divided into 4 main categories: resources, e-learning specific services, common services and presentation logic. The functionality of an mLMS uses the resources and services in an eLMS and executes the adaptation logic required for the visualization of contents and services by mobile devices. The main services provided by an mLMS are the capability of discovering the context of the mobile 14 M-learning: Personalized Education Anytime, Anywhere and For Everybody 173 device and the capability of adapting the e-learning resources and services to the user interface in the mobile device. The main features of the mobile architecture of Trifonova and Ronchetti [2004] is captured in Fig. 14.7. This architecture constitutes a logical evolution of traditional e-learning LMS centered architectures introducing two new features which acquire a significant relevance in m-learning: the need to adapt the contents and services to the limited user interface and the need to discovery and use not only the user’s context but the mobile device’s context. The context discovery opens the door for new opportunities for personalized and contextualized learning experiences.

Fig. 14.7. A general architecture for m-learning.

Following the aforementioned work, Specht et al. [2006] proposed a general architecture for contextualized learning experiences. They recognized the fact that based on the background of situated cognition and situated learning, authentic learning experiences can be supported by a variety of context parameters. They illustrated a general architecture for contextualized applications identifying important issues for contextualizing learning experiences and learning content. The mobile learning architecture that they proposed divides the elements in the user’s physical environment into sensors and actuators and creates a personalized process that adapts this environment to the user’s needs. The user is located in the middle of the learning environment in a contextualized way. The sensors in the physical environment surrounding the mobile user provide contextualized learning information. A 4-layered personalized application captures and processes this information and depending on the user’s profile takes the required actions on the actuators in the environment. The authors proposed the use of semantic tags for describing the user, the elements in the environment and their interactions. Martin et al. [2006] also proposed an architecture for context based adaptation in m-learning. They proposed a system that manages data about users and activities so that the most suitable activities to be accomplished at each time are proposed to each user. This decision is not only based on the user’s personal features, preferences or previous actions but also on information about the specific user’s context, including spare time, location and available devices. This architecture personalizes the learning process taking into account the users’ personal feature, their actions and their context. However, the context of the user is limited to the user’s location and the type of device he or she is using. Another important aspect for the definition of a general m-learning architecture is captured by Sharma and Kitchens [2004]. These authors concentrate on the different services provided in e-learning in general and m-learning in particular scenarios and propose a unified protocol for accessing these services based on Web Services. The authors state that by using Web Services companies can link applications and offer services regardless of the computing platforms or programming languages involved. Web Services based architectures are flexible and extensible and are currently implemented in pervasive environments including mobile devices. Figure 14.8 captures the details of a Web Service based m-learning architecture. Figure 14.8 presents a mobile device that uses Web Services (SOAP over HTTP) to access the different services in the central learning management system. In traditional e-learning scenarios, many of these services are normally accessed using HTML over HTTP. However, HTML is not always the best option to send information to mobile devices due to their limited user interface capabilities. SOAP messages, on the other hand, offer a mechanism to transport generic XML documents which can be rendered in different ways depending on the type of mobile device. Using client based personalization mechanisms for presenting information to the mobile user offers a scalable mechanism to handle the continuously increasing diversity in mobile devices. Traditional 174 Mario Muñoz Organero, Elena Yndurain and Carlos Delgado Kloos

Fig. 14.8. A Web Service base m-learning architecture. server based personalization mechanisms associated to web pages that need to be presented on different devices have two main limitations for a generic m-learning personalized scenario: they are normally limited to web information and they are normally tackling a discrete set of different types of devices. M-learning scenarios require the use of other types of information and potentially non limited types of devices. A further evolution of the architecture presented in Fig. 14.8is the one presented by Muñoz Organero and Delgado Kloos [2007a]. These authors introduce the idea of executing personal learning services in personal mobile devices. The mobile device is no longer a simple service consumer but a complete service environment in which personalized services are integrated in a distributed manner using stateful Web Services (or Grid Services). This idea introduces the concept of a pervasive distributed learning management system in which services are executed in a contextualized and personalized environment (we dedicate the following sub-section to this pervasive learning management system). Figure 14.9 captures the details of this architecture. Some services are implemented in a distributed way and deployed in a personalized way on mobile devices.

Fig. 14.9. A distributed m-learning architecture.

The fact that a personal service can be deployed in a distributed way introduces the need of service orchestration or coordination. Muñoz Organero and Delgado Kloos [2007b] propose the use of IMS-LD (Learning Design) in order to solve this problem. A ﬁnal step to deﬁne a general m-learning architecture combines the main features of the above mentioned architectures in order to create a new one which incorporates the different aspects of m-learning environments: personalization, context awareness, distribution and coordination. This combined architecture is presented in Fig. 14.10. The architecture is centered in the learning user. A personalized learning environment is deployed in his or her mobile device. This environment executes the personalized logic for providing and consuming learning services and makes relevant use of the information, objects and devices in the user’s physical environment. The coordination and integration of learning users in a common learning experience is obtained by using IMS-LD. We dedicate the next sub-sections to provide details for the distributed environment and for the context enhanced environment. 14 M-learning: Personalized Education Anytime, Anywhere and For Everybody 175

Fig. 14.10. A combined m-learning architecture.

14.3.2 The pervasive, personal learning management system

Using mobile devices, learning users can be pervasive consumers of distributed learning resources, but at the same time, they can be active parts in the e-learning scenario, offering contents and services to other learning users in a context dependent architecture. Mobile devices are in many cases personal devices that contain information about their associated personal user profiles as well as some user dependent learning resources. Combining these devices either in a predefined or ad hoc manner we can create personalized and pervasive distributed learning architectures. This sub-section is dedicated to give some details about how these devices can be combined and coordinated to create a personal distributed learning management system. In order to present and analyze the concept of the personal distributed learning management system, first we are going to review some major initiatives that have been published in the literature and second we are going to propose a generalized architecture based on the concepts described in the previous subsections. There are some interesting related works published about the concept of the personal distributed learning management system. We are going to concentrate mainly on two of the main trends and describe two initiatives that have followed those trends. The first trend is based on the decomposition of the functionality of an LMS into several services and the definition of an interface to interact with each service. The OKI initiative [of Tech- nology, 2006] follows this trend. The second trend is based on the decomposition of the e-learning content, and the associated services to interact with it, into autonomous Grid Services which can be integrated into a single unit of learning using standards such as IMS-LD [IMS Global Learning Consortium, 2006]. The Gridcole initiative presented in [Bote Lorenzo et al., 2004] follows this trend. The main ideas behind both initiatives are combined in the pervasive m-learning architecture presented in Fig. 14.10. In order to give a more detailed presentation of the state of the art in distributed LMS architectures, this subsection also makes a brief introduction to other proposals for the implementation of distributed e-learning architectures and compares the ideas behind these proposals with those in OKI and Gridcole. The OKI initiative has developed a distributed open architecture for an LMS based on the definition of open interfaces called the Open Service Interface Definitions (OSIDs), whose design has been informed by a broad architectural view. The OSIDs define important components of a Service Oriented Architecture SOA as they provide general software contracts between service consumers and service providers. This enables applications to be constructed independently of any particular service environment, and eases integration. The OKI definitions of the OSIDs are programming language neutral (although OKI has provided bindings to Java and PHP) and can be implemented using any distributed middleware such as Web Services. Implementing the functionality of the OSIDs as distributed services provides a way to build scalable and flexible LMSs. The OKI version 2 defines 18 different OSIDs. It is not the idea of this paper to make a detailed review of all of them rather than presenting the main concepts behind their definitions. Three of the most important OSIDs defined in OKI version 2 are the Authentication OSID, the CourseM- anagement OSID and the UserMessaging OSID: 176 Mario Muñoz Organero, Elena Yndurain and Carlos Delgado Kloos

The Authentication OSID gathers required credentials from an agent (agents in OKI represent individuals • or processes that invoke specific Services), vouches for their authenticity and introduces the agent to the system. The Authentication OSID permits an application to abstract the authentication process without having to manage the details of the underlying authentication service. The CourseManagement OSID primarily supports creating and managing a CourseCatalog. The catalog is • organized into CanonicalCourses, CourseOfferings for CanonicalCourses and CourseSections for Course- Offerings. The UserMessaging OSID supports communication and notification among users. • The OKI OSIDs are internal interfaces inside a distributed LMS system and are not defined to interact directly with the e-learning user. They only provide programmatic interfaces to distributed services. Although this approach is very valuable for creating scalable LMS systems is not considered to be general enough for pervasive learning environments in which autonomous distributed e-learning services in mobile devices should also be able to directly interact with pervasive e-learning users. The Gridcole initiative presented in Bote Lorenzo et al. [2004] combines grid service technologies and the IMS-LD specification in order to define an architecture in which the e-learning user can interact with IMS-LD defined e-learning services using a distributed environment based on Grid Services (which are basically stateful Web Services). The IMS-LD specification, describes how the different actors in the e-learning scenario, playing different roles, perform a flow of learning activities which contain not only content but also services creating a Unit of Learning (UoL). Yu et al. [2006] gives a good introduction to the IMS-LD standard. Grid Services are stateful Web Services which can be created and bound when needed. The service-oriented computational grid is employed in Gridcole as an infrastructure that allows multiple organizations to share a large pool of IMS-LD defined e-learning services that may be integrated in order to support all sorts of either individual or collaborative learning scenarios. Such services are shared as presentation-oriented grid services. More specifically, the business logic of the services is offered to the e-learning user following the OGSI (Open Grid Services Infrastructure) specification Global Grid Forum (GGF) [2003]. The presentation logic, with which the e-learning user can interact with the service, is offered as a Java-based client that implements a grid service based communication protocol. There are two interesting points in the Gridcole architecture for the objectives of this sub-section. The first point is that e-learning courses can be created by integrating distributed services. The second point is the use of IMS-LD to specify how these distributed services are orchestrated (how, when, by who and for what purpose these services are used). However, the Gridcole architecture also presents some issues that should be better addressed for pervasive m-learning environments. The first one is the definition of a mechanism for service access that does not require the e-learning user to download a Java client (for instance using HTML rather than grid services). The second one is the definition of open mechanisms for the communication among services that permits the implementation of more autonomous e-learning services. Apart from OKI and Gridcole there are other proposals for the implementation of distributed e-learning architectures. Anane et al. [2005] proposes the use of Web Services as a mechanism to create distributed e- learning courses. An enhanced version of BPEL4WS that uses virtual Web Services [Khalaf et al., 2003] is used for the composition and design of learning paths. The idea is that e-learning content from different sources can be used to create a composed course in a dynamic way using agents that can search, discovery and bind services. This proposal has not been considered in the research presented in this paper since the main ideas of this proposal are also captured in Gridcole. Instead of using BPEL4WS for the definition of e-learning flows, Gridcole uses IMS-LD which has been specially created for this purpose. Instead of using a new defined concept as the virtual Web Service, the Gridcole initiative proposes the use of standard Grid Services. Another Web Service based initiative for creating e-learning distributed architectures is the one proposed by Tamura et al. [2006]. This architecture decomposes the functionality of an LMS into a Web based front end and a Web Service based back end. The Web based front is used for the interaction with the e-learning users. The Web Service based back end contains learner information, learning content and learning strategy. The learner information contains the personal learning environment of the e-learning users. The learning content contains a collection of SCORM packaged SCOs (sharable content objects). The learning strategy contains how the different SCOs should be sequenced. A UDDI server is also used for finding the Web Services available for creating the back end. Although the idea of separating the e-learning user interface from the internal services 14 M-learning: Personalized Education Anytime, Anywhere and For Everybody 177 inside an LMS is very interesting, the proposal in [Tamura et al., 2006] presents some limitations: the Web based front end continues to be a central point of access in the architecture, the Web services in the back end are very limited in scope and functionality and, last but not least, the sequencing mechanism is very simple and does not follow a standard such as the IMS learning design. Another proposal for implementing distributed e-learning architectures is the one presented by Da Bormida et al. [2004]. This proposal defines an Open Mobile Access Abstract Framework (OMAF) based on the idea of creating layers of services which together can form a distributed m-learning environment. The main ideas of this proposal are taken from OKI which is the one used here because of its greater acceptance Once we have reviewed the main contributions in the related work by other authors let’s concentrate on combining their merits in order to create a generic definition for the personal distributed learning management system. As captured in Fig. 14.11, the central point in this system is the personal mobile device which knows and learns from the user habits and preferences and provides the execution environment for the user’s learning personal services. These services are personalized and adapted to the learning user. Due to the fact that these services are executed locally in the user’s mobile personal device, they can be accessed anytime anywhere even if no network connection is available.

Fig. 14.11. A pervasive, personal distributed learning management system architecture.

As presented in Fig. 14.11, the personal learning execution environment is integrated into a distributed learning management system so that the learning activities from all the users are put together in an instructor deﬁned, supervised and coordinated course. Figure 14.11 assumes that there is a central, permanent connected, Learning Management System which coordinates the distributed pieces in the architecture. This coordination is achieved using a course orchestration language like IMS Learning Design. The distribution is achieved by using a pervasive middleware based on stateful Web Services or Grid Services. The pervasive, personal distributed learning management system has to provide contents and services to e-learning (or m-learning) users. As captured in Fig. 14.11, the universal protocol for user interface generation is based on HTML over HTTP. This provides a pervasive access environment which can be used locally from a mobile device or even remotely by other users connected to the Internet. The architecture in Fig. 14.11 uses OKI deﬁned services (in order to achieve service distribution). However, the service interfaces are expanded to incorporate IMS Learning Design orchestration methods.

14.3.3 A context-enhanced personalized learning

As presented in Fig. 14.10, m-learning scenarios can benefit from the information, devices and services in the physical context of the mobile learner. Combining this physical context with the platform maintained and updated user profile, a new personalized, context-enhanced learning scenario can be built. The learning process, for example, can take the user’s location into account when providing relevant information to this user, or, as another example, the information read by the mobile device from an RFID tag can provide hints to the learning management system in order to track the user’s interests. The any-time any-where association that 178 Mario Muñoz Organero, Elena Yndurain and Carlos Delgado Kloos exists between personal mobile devices and learning users enhances the adaptation possibilities of the learning process to the physical and perceptual user-context for a user oriented learning environment. This perceptual user-context can be sensed by the user’s mobile device by using short range network technologies (technologies for personal and touch zones) such as Bluetooth, RFID and NFC (these technologies were briefly introduced in section 1 of this chapter). The intelligent devices in the user’s environment are able therefore to interact with the user’s learning mobile device providing information and adapting the properties of the user’s environment. This sub-section describes some interesting research studies and applications that sense and interact with the user’s physical environment using personal zone wireless technologies such as Bluetooth and touch zone wireless technologies such as NFC and how these technologies can be used to enhance m-learning experiences. The use of Bluetooth for sensing and interacting with the user’s environment has generated some research projects that explore among other things the use of public displays for different types of user interaction [Ran- ganathan and Campbell, 2002] or the use of headsets for obtaining information from elements in museums. One of the earliest projects was GroupCast [McCarthy et al., 2001] which identified appropriate content using profiles or information channels [Pradhan et al., 2001, Salminen et al., 2006, Siegemund, 2002]. In BlueBoard [Russell and Gossweiler, 2001] a large public display was designed to support fast, lightweight encounters with users in order to provide access to their personal content. Lawrence et al. [2006] presents three projects that exploit incidental interactions in social networks. BluScreen was expanded by Sharifi et al. [2006] with an intelligent public display which selects and displays adverts in response to users detected in the audience by detecting the nearby Bluetooth devices. As Bluetooth is able to detect only nearby devices and services it is also a good technological alternative to establish ad hoc learning networks. Yonghong et al. [2006] proposes a Bluetooth based tool for in class student to teacher interaction. This tool offers the students with mobile devices the possibility to answer questions in class and provides instant feedback to the professor. Grew et al. [2006] proposes a distributed platform designed to support pervasive learning and interactivity on a university campus and to ease tasks related to learning and teaching making use of wireless technologies including bluetooth in order to provide service access anywhere and anytime. The work proposed by Zhang et al. [2007] also presents a mobile learning system based on Bluetooth similar to that of Yonghong et al. [2006]. Creating learning networks using Bluetooth to integrate mobile personal devices and using Bluetooth also for sensing the physical user’s environment provides a perfect environment for e-learning personalization. Mitchell et al. [2006] describes a system used in the University of Lancaster which delivers personal messages to Bluetooth personal devices connected to the e-learning platform of the university in Bluetooth connected areas. The work in [Zhou and Wang, 2005] describes a system that uses emotion detection sensors to capture physiological signals that are collected using Bluetooth technology. The authors recognize that the explicit expression of emotion often involves particular facial expression, emotive speech, expressive behavior, gesture and physiological responds in autonomic nerve system associated with emotion experience. They propose a multimodal system that provides assistance to the learner in order to enhance positive attitudes and emotions, therefore enhancing their learning. NFC (Near Field Communications) based technologies integrate RFID and mobile technologies enabling the interaction with the user’s physical environment by simply approaching a mobile device to the particular thing with which the user wants to interact. There are some interesting proposals that use the presence of RFID tags to present information to users through visual signs like in [Riekki et al., 2006]. The work of Strang et al. [2006] explores the use of physical signs as anchors for digital annotations and other information as street signs and shop signs. Semacode (http://semacode.org) supports the use of visual markers and Nokia’s Field Force solution (http://www.nokia.com/fieldforce) also uses NFC. The work of Broll et al. [2006] presents a vision and a system framework for supporting mobile interaction with the Internet of Things. A mobile device can click on physical links to services that are represented by electronic or visual tags attached to objects in the real world. The work by Rukzio et al. [2005] presents a framework for physical mobile interactions in everyday live using NFC and Bluetooth-based interactions between a mobile device and a public display. Combining personal and touch zone wireless technologies increases even more the personalization capabilities for m-learning scenarios. Every learning object in the user’s physical environment can be tagged with NFC compliant tags so that the learning user can have access to the particular information related to the particular object in which he or she is interested, and at the same time, this learning user can establish a learning net- 14 M-learning: Personalized Education Anytime, Anywhere and For Everybody 179 work with other users or central learning management systems using Bluetooth or Wireless LAN technologies. Moreover, the result of the interaction between a particular user and a particular learning physical object can be written in the tag associated to this object updating the context also for other learning users. Figure 14.12 captures this scenario.

Fig. 14.12. A context-enhanced learning scenario.

14.4 Conclusions

This chapter has presented the state of the art in mobile technologies and systems and how to use them for a personalized anytime anywhere ubiquitous learning environment. We have presented how mobile technologies are experiencing a huge development in software, hardware, user interface capabilities and network connectivity creating a suitable environment not only for distance learning service access but also for learning service execution. We have also presented a personalized mobile learning architecture in which mobile devices are active peaces of a learning management system providing personalized always on learning services to mobile users. Furthermore, we have also captured how the physical context of the mobile user can be used to enhance the learning environment for a personalized m-learning architecture. Among the different technologies that can be used to create a distributed ubiquitous personalized m- learning environment we have shown how IMS-LD and stateful Web Services fulﬁll the requirements for such an environment. We have also presented how these technologies can be used to extend the OKI architecture. Finally we have seen how to sense the physical user-context using the user’s mobile device by using short range network technologies (technologies for personal and touch zones) such as Bluetooth, RFID and NFC.

References

R. Anane, B. Bordbar, D. Fanyu, and R. J. Hendley. A web services approach to learning path composition. In 5th IEEE International Conference on Advanced Learning Technologies, Kaohsiung, Taiwan, 2005. M. L. Bote Lorenzo, L. M. Vaquero González, G. Vega Gorgojo, J. I. Asensio Pérez, E. Gómez-Sánchez, and Y. Dimitriadis. GRIDCOLE: A Grid collaborative learning environment. In International Workshop on Collaborative Learning Applications of Grid Technology, Chicago, USA, 2004. G. Broll, S. Siorpaes, E. Rukzio, M. Paolucci, J. Hamard, M. Wagner, and A. Schmidt. Supporting mobile service interaction through semantic service description annotation and automatic interface generation. In International Workshop on Semantic Desktop and Social Semantic Collaboration, Athens, USA, 2006. Canalysis. Mobility data. http://www.canalys.com, 2007. G. Da Bormida, M. Di Girolamo, I. Dahn, and E. Murelli. An open abstract framework for modeling interoperability of mobile learning services. In EDUTECH Workshop on Computer-Aided Design Meets Computer- Aided Learning, Toulouse, France, 2004. Global Grid Forum (GGF). Open grid services infrastructure (ogsi) version 1.0. http://www.ogf.org/, 2003. 180 Mario Muñoz Organero, Elena Yndurain and Carlos Delgado Kloos

P. Grew, F. Giudici, and E. Pagani. Specification of a functional architecture for e-learning supported by wireless technologies. In Workshops of the 4th Annual IEEE International Conference on Pervasive Computing and Communications, 2006. R. Ianucci. In search of the next big thing revolutions in the mobile communications domain. In 31st IEEE Conference on Local Computer Networks, Florida, USA, 2006. IDC. Mobile telecom analysis, 2006. IMS Global Learning Consortium. IMS learning design, technical specification. http://www.imsglobal.org, 2006. InternetStats. Internet statistics. http://www.internetworldstats.com, 2007. R. Khalaf, N. Mukhi, and S. Weerawarana. Service-oriented composition in BPEL4WS. In 12th International World Wide Web Conference, Budapest, Hungary, 2003. J. Lawrence, T. Payne, and R. V. Kripalani. Exploiting incidental interactions between mobile devices. In International Workshop on Mobile Interaction with the Real World, Espoo, Finland, 2006. E. Martin, N. Andueza, and R. M. Carro. Architecture of a system for context-based adaptation in m-learning. In 6th IEEE International Conference on Advanced Learning Technologies, Kerkrade, The Netherlands, 2006. J. F. McCarthy, T. J. Costa, and E. S. Liongosari. Unicast, outcast and groupcast: Three steps toward ubiquitous, peripheral displays. In 3rd International Conference on Ubiquitous Computing, Atlanta, USA, 2001. K. Mitchell, N. J. P. Race, D. McCaffery, M. Bryson, and Z. Cai. Unified and personalized messaging to support e-learning. In 4th IEEE International Workshop on Wireless, Mobile and Ubiquitous Technology in Education, Athens, Greece, 2006. M. Muñoz Organero and C. Delgado Kloos. Web-enabled middleware for mobile devices. In International Conference on Wireless Applications and Computing, Lisbon, Portugal, 2007a. M. Muñoz Organero and C. Delgado Kloos. Pervasive LMS components and services for a pervasive m-learning architecture. International Journal of Mobile Learning and Organisation, September 2007b. Nokia. Screen capability. http://wiki.forum.nokia.com/index.php/Screen-capability-summary-of-S60-3rd- Edition-phones, 2006a. Nokia. Usability. http://www.forum.nokia.com/main/technical-services/usability/index.html, 2006b. Massachusetts Institute of Technology. The Open Knowledge Initiative (OKI). http://www.okiproject.org/, 2006. S. Pradhan, C. Brignone, J.-H. Cui, A. McReynolds, and M. T. Smith. Websigns: Hyperlinking physical locations to the web. IEEE Computer, 34(8):42–48, 2001. A. Ranganathan and R. H. Campbell. Advertising in a pervasive computing environment. In 2nd International Workshop on Mobile Commerce, Atlanta, USA, 2002. J. Riekki, T. Salminen, and I. Alakarppa. Requesting pervasive services by touching RFID tags. IEEE Pervasive Computing, 5:40–46, 2006. E. Rukzio, S. Wetzstein, and A. Schmidt. A framework for mobile interactions with the physical world. In International Conference on Wireless Personal Multimedia Communication, Aalborg, Denmark, 2005. D. M. Russell and R. Gossweiler. On the design of personal and communal large information scale appliances. In 3rd International Conference on Ubiquitous Computing, Atlanta, USA, 2001. T. Salminen, S. Hosio, and J. Riekki. Enhancing Bluetooth connectivity with RFID. In 4th Annual IEEE International Conference on Pervasive Computing and Communications, Pisa, Italy, 2006. M. Sharifi, T. Payne, and E. David. Public display advertising based on Bluetooth device presence. In Interna- tional Workshop on Mobile Interaction with the Real World, Espoo, Finland, 2006. S. K. Sharma and F. L Kitchens. Web services architecture for m-learning electronic. Journal on e-Learning, 2:203–216, 2004. F. Siegemund. Spontaneous interaction in ubiquitous computing settings using mobile phones and short text messages. In International Workshop on Supporting Spontaneous Interaction in Ubiquitous Computing Settings, Göteborg, Sweden, 2002. M. Specht, A. Lorenz, and A. Zimmermann. An architecture for contextualized learning experiences. In 6th IEEE International Conference on Advanced Learning Technologies, Kerkrade, The Netherlands, 2006. 14 M-learning: Personalized Education Anytime, Anywhere and For Everybody 181

T. Strang, V. Cahill, and A. Quigley. Practical issues in physical sign recognition with mobile devices. In International Workshop on Pervasive Mobile Interaction Devices, Dublin, Ireland, 2006. Y. Tamura, T. Yamamuro, and T. Okamoto. Distributed and learner adaptive e-learning environment with use of web services. In 6th IEEE International Conference on Advanced Learning Technologies, Kerkrade, The Netherlands, 2006. A. Trifonova and M. Ronchetti. A general architecture to support mobility in learning. In 4th IEEE International Conference on Advanced Learning Technologies, Joensuu, Finland, 2004. VisionGain. Strategy analytics 1q 2007 report, 2007. VisionGain. Mobile memory: Challenges and evolution, 2005-2010, 2005. Z. Yonghong, Z. Shiying, V. Son, and M. Kamran. Mobile learning with Bluetooth-based e-learning system. In International Conference on Wireless Communications and Mobile Computing, Vancouver, Canada, 2006. D. Yu, W. Zhang, and X. Chen. New generation of e-learning technologies. In 1st International Multi- Symposium on Computer and Computational Sciences, Hangzhou, China, 2006. Y. Zhang, W. Li, and Y. Fu. A mobile learning system based on Bluetooth. In 3rd International Conference on Natural Computation, Haikou, China, 2007. J. Z. Zhou and X. H. Wang. Multimodal affective user interface using wireless devices for emotion iden- tiﬁcation. In 27th Annual International Conference of the Engineering in Medicine and Biology Society, Shanghai, China, 2005. C. Zwick, B. Schmitz, and K. Ku. Designing for Small Screens. AVA Publishing SA, 2005.

15 Self-Adaptive Portals: E-Government Case Study

Ljiljana Stojanovic and Nenad Stojanovic

Forschungszentrum Informatik IPE Haid-und-Neu-Str. 10-14, 76131 Karlsruhe, Germany {Stojanovic,nstojano}@fzi.de

Summary. Transfer to on-line services posts to an e-government system a very serious challenge: how to enable personalised service delivery that citizens are used to receive in the brick-and-mortar environment. Indeed, several recent research studies have shown that a user of e-government services is usually lost in the hyperspace, primarily due to the one-size-fits-all approach in designing web access to services. In this chapter we describe a novel approach for the personalised delivery of the e-government services, that is based on the usage of semantic technologies for the automatic adaptation of an e-government portal to the individual requirements of the users. The main advantage of the approach is the context-aware adaptation, i.e. the adaptation depends on the current user’s behaviour and not on the predefined user’ profiles. Indeed, the approach is tailored to the anonymous users who experience some usability problems in the usage of the portal. The system tries to match the user’s behaviour with existing patterns in order to recognize a problem situation as early as possible and suggest corresponding support. The role of the semantics is crucial: ontologies enable semantic interpretation of user behaviour in a portal, which makes possible meaningful, effective and context-aware adaptation. The approach uses a set of ontologies, including web-portal ontology, user ontology, behaviour ontology and adaptation ontology. It has been implemented as an ontology-based tool suite that closes the adaptation loop and encompasses, among others, a tool for semantic logging of the user’s behaviour, a tool for semantic data mining and a tool for the semantic adaptation of a web portal. The approach has been applied in three public institutions and we present the results from the preliminary evaluation studies.

15.1 Introduction

In most e-government projects to date, technology was in the center of the project and not the user, although the user (e.g. the citizen or a business person) is the one who shall in the end use all the new and exciting online e-government services. As long as all efforts are technology driven, e-government will not take off, and will not reach its full potential. To confront different citizens with a one-size-fits-all Web interface is not the optimum way to deliver public sector services because every person is an individual with different knowledge, abilities, skills and preferences. The conventional brick-and-mortar office has a more human face because the clerk can respond to different people in different manners. That is why people tend to use the conventional office rather than the e-government services. To transfer some of the humanity to e-government portals, it is necessary to build adaptive portals for public services. Such user-adaptive portals will increase the usability, and, thus, the acceptance of e-government, enabling administrations to achieve the, as yet, elusive efficiency gains and user satisfaction that are the primary goals of e-government projects. This chapter describes an approach for adaptation that addresses these issues. The approach results in a self- adaptive e-government system. By self-adaptive, we mean an interactive system that automatically acquires a model of the individual user, utilizes that model to adapt the portal to the user, observes the results of its actions and adjusts itself to improve future performance. 184 Ljiljana Stojanovic and Nenad Stojanovic

Even though there are several challenges in realizing self-adaptive portals, in this chapter we focus on fact that the self-adaptation involves some form of learning. Indeed, experience shows that people tend to be reluctant to provide the feedback about their satisfaction/expectations via filling questionnaires or forms. In order to avoid asking people explicitly, means for capturing their preferences implicitly are required. To resolve this issue, we propose a new approach that combines the power of web usage mining with Semantic Web technologies to create a semantic framework for learning preference rules. Besides of learning more precise rules, the extensive usage of semantics offers many additional advantages like support for capturing the meaning of a user’s clickstream, recognition of the user’s problem at the right moment, application of declarative, machine-understandable preference rules to adapt the portal to the current situation when it is needed, etc. Semantic knowledge can be integrated and used at every phase of the web usage mining process [Dai and Mobasher, 2005]. In the preprocessing phase, the challenges are in the mapping of users’ activities at the clickstream’s level to more abstracts entities defined in the ontology. For the data mining phase, the primary goal is to develop new approaches that take into account complex semantic relationships. Finally, in the pattern analysis phase, the challenge is in developing techniques that measure successfully and efficiently semantic similarities among complex objects (e.g. user profiles) in order to generate recommendations. In this chapter we consider only the preprocessing phase, which resolves the knowledge acquisition problem by extending users data with the contextual information and prepares data for further phases. More information about other phases can be found in [Stojanovic, 2007]. The rest of this chapter is organized as follows. Section 15.2 presents some examples of e-government services, and derives requirements on adaptive e-government portals. The conceptual architecture of the semantic- based self-adaptive portals is given in Section 15.3. Section 15.4 explains the advantages of Semantic Web technologies when it comes to meeting the identified requirements and achieving self-adaptation. Section 15.5 presents our approach for combining web usage mining and Semantic Web technologies in order to learn domain-specific preference rules. Section 15.6 compares the approach to related work. Section 15.7 indicates the direction of future work.

15.2 Motivating example

A typical service provided by an e-government portal is submission of an application form related to a building project. Such a service is actually a complex process, subject to regulations that require the submission of different forms at different times, depending on the type of building project. For an inexperienced user the challenge starts here. With lack of background knowledge of the building regulations the user is confused and does not know which form to choose for her building project. Such users, unfamiliar with the portal and the specific service, need guidance to prevent them from getting stuck in the portal shallows. On the other hand, for an architect who works daily with the virtual building application within an e-government portal, any guidance would only hinder her smooth sailing. Therefore, among other things, there is a need to cater for different skill levels like novice, average and expert. Moreover, adaptation, for example adaptation to skill level, is needed for each service, since an expert in one service may be a novice in another. For example, the architect, expert at building applications, may be a novice when it comes to submit an application for child support. In fact, many services will only rarely be used by a user, so the majority of users will probably remain novices in their use. Given this example we derive five basic requirements for adaptive e-government portals: 1. A portal must provide guidance and information that matches the users’ profiles, e.g., the different skill levels and interests of its citizens. 2. As citizens typically use e-government services rarely they should not be bothered with providing and maintaining any user profiles. 3. It is mandatory to observe the crucial usability principles such as responsiveness, predictability and com- prehensibility, controllability and unobtrusiveness. Initial emphasis, in regard to usability, is placed on providing accurate, but unobtrusive, guidance, when and where it is needed by the user. 4. The portal should be subject to continual improvement. Explicit user feedback can be enormously helpful here, but the feedback requested should be relevant to the services that the user executed. 15 Self-Adaptive Portals: E-Government Case Study 185

5. A final, important requirement is related to the nature of e-government services, and to the fact that there are multiple units of e-government at different levels (e.g. local, regional and national). Given the similarity of many of the services offered by these different units, there is an enormous potential for efficiency gains through sharing best practices. Therefore, the fifth general requirement is the ability to share successful adaptation strategies and rules.

15.3 Conceptual Achitecture

To meet the requirements identiﬁed in the previous section, we propose building portals based on the ontology- based Monitor Analyze Plan Execute (MAPE) model [Stojanovic et al., 2004]. This model uses ontologies as the backbone of the adaptation process and abstracts the management architecture into four common functions:

Monitor: mechanism that collects, organizes and ﬁlters the data about users’ interactions with the portal. • Analyze: mechanism that aggregates the collected data and correlates this data with background knowledge • (e.g. domain knowledge). Plan: mechanism to make proposals for improvement (e.g. by learning preference rules). • Execute: mechanism to update the portal according to the generated plan. • By monitoring (M) the behavior of users and analyzing (A) this data, planning (P) which actions should be taken and executing (E) them, a kind of a usage loop is created. Figure 15.1 depicts this usage loop. The standard way for users to interact with a web site is to search for an e-service, to navigate through the portal or to enter information on a form, press a submit button, and wait for the page to update (cf. 1 in Fig. 15.1). All user activities have to be tracked (cf. 2), since acquisition of a model of the user is the indispensable pre-requisite to adaptation. With Ajax, as noted in [Schmidt et al., 2007], the range of user actions that can be tracked is extended beyond just mouse clicks. For example, scrolling, mouse over and keystroke events can be tracked enabling the detailed recording of user actions on the client side.

E-Users „PageXwas „Ifauserisanovicefora Request Response accessed “ buildingpermissionservice, Front-office C u s Thenpresentmoreinformation“ ut r to o ou m ab vi p is a a 1 ag ed at eh M E e D ‘ b s w s eb er us

Portal Logging 2 Enchancement 4 Ontology In f t o u rm o s a b e w t a c it io e n h n A P g re c e d e o n le f n r SemanticWeb re t ic w p e h UsageMining o ‘ x e n s t d 3 K r se u „Thereisagroupofuserswho “Pageaboutbuilding spentalotoftimetofill-ina permissionwasaccessed” buildingpermissionform“

Fig. 15.1. Closing the semantic loop in an adaptive e-government portal.

To provide meaningful adaptation, events, content, context and elements of an portal must be well understood. We propose the use of ontologies as a model for user- and portal-related content and context data. This 186 Ljiljana Stojanovic and Nenad Stojanovic semantic information is used in the Semantic Web Usage Mining module (cf. 3) to enrich the web log data. More importantly, the role of the Semantic Web Usage Mining module is to detect anomalies in the design of a portal and its services, whose repairing will improve the usability. Thus, the ontology model is used to improve the learning and to represent the results of learning as preference rules. We classify the rules into two types based upon their roles in the adaptation process: Categorization rules assign a current user to the predeﬁned user categories. For example, a user is an • expert for a service, if she uses a bookmark to load a page representing this service; Adaptation rules automate corrective actions, i.e. adapt the content, structure or layout of a page to the • current user based on the category which she belongs to. For example, if a user is not an expert, and if she spent more than 1 second reading a tool tip of some element on a page, then context-sensitive and content-sensitive help explaining the meaning of this element should be shown to this user.

Next, the discovered preference rules have to be integrated in web pages in order to be used for portal adaptation (cf. 4). With the dawn of Ajax the capabilities of adapting the user interface in a Web browser mul- tiplied [Schmidt et al., 2007]. Based on annotated Ajax widgets and the ad-hoc built user model, the utilization of preference rules for real-time portal adaptation can be performed. Finally, all preference rules embedded in the web pages are evaluated during the usage of the portal (cf. 1). In this way the portal is dynamically tailored to the users’ needs, which implicitly arose. The repetition of this cycle leads to the continual improvement of the portal.

15.4 Why to Use Semantics for Portal Adaptation?

Since the data relevant for adaptation is rather sparse, or a great deal of interpretation must be done to turn it into actually useful information, we have developed the ontology-based model of adaptive portals. This model is used to decide if and when an adaptation should take place and how to do that. The complete version of the model can be found in [Apostolou et al., 2006]. Here we note that in order to use semantics, additional effort has to be done, namely the user-specific domain ontology has to be developed and the web pages have to be annotated using this ontology. In the rest of this section we discuss how semantic technologies and in particular ontologies can be utilized for automatic adaptation of an e-government portal to the individual requirements of the users. Even though the section is motivated by using e-government examples, the proposed approach is general enough to be applied in other domains. There are several reasons to build our approach upon the intensive use of semantic technologies. Firstly, ontologies enable semantic interpretation of user behavior in a portal, which in turn enables meaningful, effective and context-aware adaptation. The building permission example from Section 15.2 is elaborated next to show how semantics enables such context-aware adaptation. Assume that the user, who wants to apply for building permission, goes to the appropriate e-government Web site. And, on this site, the user finds a list of hyperlinks to forms related to building permits. But, she does not know which one is appropriate for her building project. Being based on Ajax, the Web site implements mouse-over help for these hyperlinks. The user knows this, and places the mouse on a hyperlink for a time to make the help appear. Then the user does this for a second hyperlink, but still does not choose a form. Assuming that the hyperlinks have been associated with concepts in the ontology, the system can now make a semantic interpretation of the user’s behavior. In this case, the conclusion would be that the user has a strong interest in the concepts associated with the two mouse-over hyperlinks, and that the user needs help choosing a form. In response to this context, the system can offer the user help. Not only that, this help can be tailored to the user by taking account of the concepts in which the user showed interest, concluded from her current navigation path and behavior. Adaptation such as this is based on using semantic annotation of a page and its structural elements (e.g. hyperlinks). A second reason to use ontologies is that ontologies used in rules can make adaptation logic more explicit. This declarative representation, expressed as rules using concepts and relations from the ontology, helps the domain experts inspect, understand and even modify the rationales behind adaptive functionality. For example, the hierarchical organization of e-government services allows the expert to model adaptation rules on a more 15 Self-Adaptive Portals: E-Government Case Study 187 abstract level, i.e. covering more than one concrete service (building permission service, independently of the type of building such as house, office, ...). This reduces significantly the number of rules and makes maintenance of the system much easier. Finally, ontologies facilitate sharing knowledge between portals, especially for those offering similar services (e.g. two municipalities in one state are similar). For example, the best practices gathered in issuing building permits in one portal (e.g. inexperienced users need an additional explanation regarding the hyperlink “required documents”) can be easily transferred to other portals that implement the same regulations for issuing building permits. This sharing is greatly facilitated by the fact that all of the terms used (e.g. additional explanation, hyperlinks, required documents, ...) are well defined. It is clear that the benefits for the users as well as for e-government are enormous, since the public administration can improve its performance at much less expense.

15.5 Semantic Preprocessing

In this section we discuss the research challenges to transform raw usage data into semantically-enhanced transactions that can be used for semantic Web usage mining. We assume that the domain ontology and the knowledge base containing annotation of pages using domain ontology entities are available [Apostolou et al., 2006]. The content of the knowledge base shown in Fig. 15.2 should be interpreted as follows. There is a page P1 which is a start page and has URI http://www.voecklabruck.at/. This page is annotated with two annotations A1 and A2. Whereas A1 is about the domain entity “ApartmentBuilding” with weight 1.2, A2 refers to the domain entity “OutBuilding” with weight 2.1. The page P1 contains a hyperlink L1 which is annotated with An with reference to the domain entity “ApartmentBuilding” and weight 2.3. The similar explanation holds for pages P2 and P3.

Knowledgebase Domain Entity

Service P1isA StartPage P1hasURI http://www.voecklabruck.at/ P1isAnnotatedWith A1 A1isA Annotation A1isAboutd:ApartmentBuilding Building eForm Application A1hasWeight1.2 P1isAnnotatedWith A2 A2isA Annotation A2isAboutd:OutBuilding Apartment Out A2hasWeight2.1 Building Building P1containsL1 L1isA Hyperlink L1isAnnotatedWith An DomainOntology AnisA Annotation AnisAboutd:eForm AnhasWeight2.3 P2isA NavigationPage P2isAnnotatedWith A3 A3isA Annotation A3isAboutd:ApartmentBuilding A3hasWeight1.4 P3isA NavigationPage P3isAnnotatedWith A1 ...

Fig. 15.2. Necessary inputs for semantic web usage mining. 188 Ljiljana Stojanovic and Nenad Stojanovic

Our goal is to assign domain entities to user navigational paths/patterns by mapping the pageview names or URLs (i.e. web log entries) to the instances in the knowledge base. To be more specific, instead of describing a user’s navigational path as: “p1, p2, ... pn” (where pi is a URL pointing to a web page), we need to represent it using the domain entities used for annotating these pages in the knowledge base, such as: “ApartmentBuilding, OutBuilding, ..., Service”. However, the mapping of users’ activities at the clickstream level stored in the web log to more abstracts concepts that is based on existence of explicit annotation of visited pages is very complex, since there is no 1:1 mapping. The complexity of the mapping is illustrated in Fig. 15.3. Indeed, one web log entry may be transformed into several entries in the semantically-enriched log (e.g. in the case that a visited page is annotated with several domain entities). An example of such mapping is shown in the figure. Since page P1 is annotated with two domain entities (cf. “ApartmentBuilding” and “OutBuilding”), the first web log entry indicating that P1 was visited is transformed into two entries in the resulting log table. Additionally, several web log entities may be grouped into an entry in the semantically-enriched log (e.g. in the case that several entries belonging to the same session represent the visited pages that are annotated with the same domain entity). This is demonstrated with the session S2; even though it contains two entries in the standard web log (i.e. pages P2 and P3 belonging to this session were visited), in the semantically-enriched web log it is represented as only one entry labelled with “ApartmentBuilding” domain entity, since both pages are annotated only with this entity. Here we note that there are many other ways to accomplish this transformation which is discussed in the rest of this section.

Semantically-enrichedWeblog – pageview S isAbout Weight ... Weblog E ApartmentBuilding 1.2 ... Session Page ... M P1 OutBuilding 2.1 ... S1 P1 ... A P2 ApartmentBuilding 1.4 ... S2 P2 ... N ... S2 P3 ... T ... I C

E N R I C H Semantically-enrichedWeblog – M sessionview E isAbout Weight ... N T S2 ApartmentBuilding (1.2+1.4)/2=1.3 ...... Knowledge base Fig. 15.3. Examples of transforming web log data into semantically-enriched log data based on the knowledge base shown in Fig. 15.2.

The integration of domain knowledge in the web usage mining introduces great ﬂexibility as well as many challenges. We have identiﬁed the following challenges:

1. Content: which information should be used for transforming user transactions into content-enhanced transactions containing the semantics of the visited pages. 2. Weights: how to calculate importance of an ontology entity for an entry in the semantically-enriched log. 3. Representation: how to represent content-enhanced entry to be directly used by data mining algorithms as a training example. 15 Self-Adaptive Portals: E-Government Case Study 189

In following, we discuss our approach for dealing with the ﬁrst two challenges. More information about the third challenge can be found in [Stojanovic, 2007].

15.5.1 Content

Preprocessing based on ontology entities used for annotation can be performed by applying reasoning. Indeed, one of main advantages of using ontologies for web usage mining is reasoning, which can be deﬁned as an act of deriving a conclusion based solely on what is already known. By applying reasoning to the domain ontology and to the knowledge base, new, implicit information can be derived and the knowledge base can be extended automatically. We distinguished two types of inference: (i) subsumption reasoning that determines when one ontology entity logically implies another; and (ii) rule-based reasoning that is the ability to reach conclusion. The impact of these types of inference on semantic enrichment is subsequently discussed.

Subsumption Reasoning

Subsumption in general is deﬁned as incorporating something under a more general category. The purpose of subsumption reasoning is to verify the subclass relationship between two classes. This can be done through different ways of inference based on the information presented in the ontology. The most direct way of subsumption reasoning is based on the transitive property of a subclass relation. For example, for the ontology shown in Fig. 15.2 we can infer that the class “ApartmentBuilding”, which is explicitly deﬁned as a subclass of “BuildingApplication” class, is also a subclass of “Service” and “DomainEntity” as well. By assuming that a page is about a parent entity of an entity it is annotated with, the semantically-enriched log shown in Fig. 15.3 will be extended with the new entries as illustrated in Fig. 15.4.

Semantically-enrichedWeblog basedonsubsumptionreasoning Semantically-enrichedWeblog withoutreasoning isAbout Weight ... ApartmentBuilding 1.2 ... isAbout Weight ... OutBuilding 2.1 ... ApartmentBuilding 1.2 ... P1 BuildingApplication ... P1 OutBuilding 2.1 ... Service DomainEntity

Fig. 15.4. Semantic enrichment without and with subsumption reasoning.

The complexity of semantic enrichment is very high due to the level of hierarchy that should be taken into account. The main reason is that there may be several paths between two classes in the concept hierarchy containing different number of nodes. On the other hand, controlling the way in which the hierarchy is considered would provide the ability to ﬁlter the input for semantic enrichment. For example, classiﬁcation of ontology entities based on a class hierarchy can be used to limit the log entries to those containing pageviews about a certain level of ontology classes.

Rule-based Reasoning

Another possibility to derive implicit knowledge is rule-based reasoning. An ontology may contain two types of rules: axioms and general rules. Axioms are a standard set of rules represented through the OWL characteristics of properties, like inversion, symmetry and transitivity. For example, if A contains B, B contains C, and contains is a transitive property, then the ontology system can infer that A contains C as well. Thus, we do not need to express this information explicitly. 190 Ljiljana Stojanovic and Nenad Stojanovic

The general rules are domain-specific rules that are needed to combine and to adapt information available in the ontology. They are used to specify the relationships between ontological entities. For example, if A contains B and B is about C, then it can be concluded that A also is about C. The general rules have the form of an implication between an antecedent (body) and consequent (head). The intended meaning can be read as: whenever the conditions specified in the antecedent hold, then the conditions specified in the consequent must also hold. With the help of a pre-acquired rules —modeled manually by domain expert or in a semi-automatic way by applying data mining methods—, we may be able to infer that the pageviews are about additional ontology entities besides those explicitly stored in the knowledge base. In our context, the rules are stored in the ontology (see Section 15.3). We note here that we use OWL-DL ontology language to represent ontologies, rules are encoded in the SWRL language and KAON21 inference engine is used to perform ontology and rule-based reasoning. Returning to the example shown in Fig. 15.3, by considering the rule saying that a portal entity containing another portal entity about some domain entity refers to that entity as well, semantically-enriched log will be extended as shown in Fig. 15.5. Since the page P1 contains the hyperlink L1 which is annotated with the domain entity “eForm”, by executing the previously introduced rule, new fact (e.g. the page P1 is also about “eForm”) will we generated.

Semantically-enrichedWeblog Semantically-enrichedWeblog basedonrule-basedreasoning withoutreasoning isAbout Weight ... isAbout Weight ... ApartmentBuilding 1.2 ... ApartmentBuilding 1.2 ... P1 OutBuilding 2.1 ... P1 OutBuilding 2.1 ... eForm ...

Fig. 15.5. Semantic enrichment without and with rule-based reasoning.

An ontology may contain many rules. A challenge lies in controlling the rules that will be evaluated, since different rules produce different facts. This additionally requires presenting the SWRL rules in natural language so that they can be understood by analyzer, who does not need to be an ontology engineer.

15.5.2 Weights

There are two possible ways to create annotation: automatic annotation as a result of ontology learning process and manual annotation as result of tagging process. During the semantic preprocessing, usually different weights are associated with annotations. For annotations extracted from the document corpus during ontology learning process, weights can be normally derived in an automatic manner, for example as a function of the term frequency and inverse document frequency which is commonly used in information retrieval. For annotations defined by domain experts, weights are usually provided as a part of the domain knowledge specified by the experts. Such weights may reflect the relative importance of certain concepts. The research challenge lies here in defining weights: (i) for implicit annotation derived using subsumption reasoning ; (ii) for implicit annotation derived using rule-based reasoning and (iii) for session-based semantically-enriched log (see Figure 15.2).

Weights for Class Hierarchy

The ontology hierarchy can be of arbitrary complexity as shown in Fig. 15.6. It may be several different paths between two dependent classes. For example, there are two paths between classes C1 and C6. Additionally,

1 http://kaon2.semanticweb.org/ 15 Self-Adaptive Portals: E-Government Case Study 191 diverse paths between the same classes may be of different length, where the length is determined by the number of the intermediate classes. For example, one path between classes C1 and C6 contains, besides these two classes, the class C5 as well. The second path is of the length of 4 since it includes the classes C3 and C4. Moreover, it is allowed to have cycles in the hierarchy, there is no need to specify subsumption relation explicitly —this can be derived e.g. based on properties deﬁned for the concepts, etc.

C4 C5

C C1 2

Fig. 15.6. A very complex subsumption graph.

To cope with cycle in the hierarchy, we consider each class once and only once. The problem of no explicit subsumption relationships is resolved by materializing them before. Thus, the weight deﬁnition problem is reduced to the path calculation in the directed graph. The following metric is proposed:

weight(C,L) C⊆P numberOfP arents C weight(P, L) = ( ) (15.1) P numOfChildren(P ) where P is a parent class, L is a web log entry (i.e. pageview, session or transaction), and C is a direct subclass of a class P. A weight of a class is broadcasted only to direct parents if and only if this class is used for the annotation of a page visited in the considered web log entry. Additionally, the impact of a child on a parent is determined by the number of other parents/children. There are many other possibilities to deﬁne a weight of a parent based on the weights of its children, such as:

weight(P, L) = weight(C, L) factordistance(C,P ) (15.2) ∗ XC where factor is a value between 0 and 1 deﬁned by the end-user, and distance is the number of nodes on the shortest path between C and P in the subsumption graph. It is clear that both of formulas 15.1 and 15.2 have advantages and disadvantages, depending on their usage scenarios:

Formula 15.1 considers only the direct children of a concept, which limits the usage of the background • knowledge, i.e. the deeper hierarchies of a concept are not taken into account. However, this constrained view on the hierarchy enables very efﬁcient calculation of the relative importance of each child. Formula 15.2 takes into account all possible successors of a concept, which gives a slight advantage to the • concepts with a deeper hierarchy, but without going into the quality of that hierarchy, e.g. by considering the relative importance of each child. Indeed, there is a distance factor which decreases the importance of the remoter children, but without taking into account the number of parents of them.

We can say that the usage of the formulas should be application/domain-driven. Depending on which type of the ontology is provided (e.g. ﬂat hierarchy vs. deeper hierarchy) and which types of the annotation is performed (e.g. lots of vs. very few metadata are assigned to a page) formula 15.1 or 15.2 can be used. Table 15.1 summarizes these suggestions. 192 Ljiljana Stojanovic and Nenad Stojanovic

Table 15.1. A comparison of the scenarios in which formulas 15.1 and 15.2 can be successfully applied. Type of calculation Formula 15.1 Formula 15.2 Ontology Number of concepts high low Hierarchy ﬂat deep Number of rules low high Provided metadata Quantity lots of metadata per page a few Quality medium to high low

Weights for Results of Rules

As noted previously, the implicit information in an ontology-based system can be obtained through an inference process. This process is recorded as a derivation tree, which can be analyzed in order to provide more information about the newly created facts. In particular, we see two benefits of applying the derivation tree analysis in the context of the semantic web usage mining: Justification refers to the generation of human understandable descriptions of the inference process (i.e. • how a result was inferred). An explanation of the reasons for finding an implicit, previously unknown annotation (including the confidence of it) can be presented to analyzers for their information. In this way, the confidence of the analyzers in the response of semantic preprocessor can be significantly improved, since they can optimize their actions. Ranking involves determination of the relevance of results when many results were inferred. The estimation • of the relevance of a result can be obtained by analyzing the complexity of the derivation tree for computing that result. More breadth indicates significant support for that particular result; more depth indicates less confidence in that result. This strategy can be applied to assess the importance of proposed annotation in semantic preprocessing. An annotation that is obtained as the result of a larger number of independent rules is more important than the annotations resulting from fewer independent rules. With respect to tree depth, an annotation that is recommended directly (i.e. through only one rule) is more important than an annotation that is suggested indirectly (i.e. through the composition of several rules) because the second case requires that more conditions are fulfilled. However, we were not able to analyze the derivation tree, since the KAON2 inference engine we used does not provide such support. It means that the only information we can obtain is whether an annotation is implicit or not, but not the way this annotation is derived. Thus, we set up the weights of the implicit annotations to some constant value defined by the end-user.

Weights for Sessions

A session is defined as a chronologically ordered set of pages visited by the same user, such that two subsequent pages are visited within a given time window. Since a session may contain several pageviews annotated with the same domain entity, there is a need to define a comprehensive approach for setting up the weight of an entity for the whole session. A weight of an entity for a session can be defined in different ways. Measure 1. Average value refers to a measure of the middle value of the data set. One of possibility is to calculate the entity weight for a session as average value of weights of this entity for the all pageviews in the session, since it represents the average amount one “expects” as the outcome when the annotations with the same entity are made. Measure 2. Extreme values (i.e. maximum and minimum) refer to the largest and the smallest element of a set. The weight of an entity for a session can be defined as the greatest/lowest value of weights the entity has for all pageviews within the session. 15 Self-Adaptive Portals: E-Government Case Study 193

Measure 3. Frequency is the measurement of the number of occurrences of a repeated event per unit of time. Thus, weights can be deﬁned by the number of times an entity was used for annotation during the transaction/session. The simpliﬁed version of this approach is a simple use of binary values stating “the entity was used for annotation at least once" and "the entity was not used/visited”. The more complex approach considers whether the children are used for annotation or not.

Each of the deﬁned measures has its own speciﬁcities that can be used for deciding in which scenario to apply which of the measures. This is summarized in Table 15.2.

Table 15.2. A comparison of scenarios where measures 1, 2 and 3 can be successfully applied. Type of calculation Measure 1 Measure 2 Measure 3 Session length any short long Provided metadata Quantity any low lots Quality any high low

Measure 1 represents the most obvious solution that can be applied in a usual situation: the values of weights • are uniformly distributed across (relevant) pageviews and any kind of average value can be considered. We use arithmetic mean as a proper method for dealing with weight values lower then 1. Measure 2 is useful if there are just a few relevant pageviews for an entity in a session. In that way, • depending on the chosen strategy, a value can be seen as a predominant one and all other can be eliminated. For example, the maximum strategy, in which only the maximum value will be taken into account, means that the concept (entity) is very relevant for that session and should be presented most prominently. On the other side, minimum strategy means that the concept (entity) is rather low relevant for the whole session and its importance should be decreased. Measure 3 is interesting if there is a long session, a session with lots of pageviews. In that case it is • (statistically) very important if an entity appears in lots of pages, regardless the particular weights assigned to the entity in a pageview. Indeed, the importance of an entity which appears in 50 percent of pageviews in a session should be considered higher than just one appearance with a high weight. However, it is valid for longer sessions.

15.6 Related Work

There are many commercial portal solution products from a variety of top rank software vendors, e.g. Microsoft SharePoint, Sun Java System Portal Server, IBM WebSphere or BEA WebLogic. Although definitely varying in specific technology particulars, the list of supported features and off-the-shelf components is overwhelming. They offer consistent solutions and share many characteristics such as security, enterprise information and services integration, documentation, steep learning curve, ease and comfort use and administration. Similarly, open-source Apache Foundation projects such as Cocoon, Struts, Tapestry or Jetspeed are examples of commonly used, technically mature, reusable portal, albeit less sophisticated, frameworks for fast web application development. Some state of the art methods in web application development, based on model drive approaches, include HERA [van der Sluijs and Houben, 2006], WebML Moreno et al. [2006], SHDM/OOHDM [Ricci and Schwabe, 2006], etc. AHA system [de Bra and Calvi, 1998] is a generic adaptive system which can be used for a wide variety of applications. Ideally, these approaches are aimed at designing web applications, which are well understood and where the respective models can be (easily) defined. However, they do not directly address the integration and common aspects of different distributed web applications and/or data sources into a single portal instance. 194 Ljiljana Stojanovic and Nenad Stojanovic

The idea of using ontologies in portal solutions for the Semantic Web has already been examined in several works. OntoPortal uses ontologies and adaptive hypermedia principles to enrich the linking between resources [Kampa et al., 2001]. The AKT project aims to develop technologies for knowledge processing, management and publishing, e.g. OntoWeaver-S [Lei et al., 2004], which is a comprehensive ontology-based infrastructure for building knowledge portals with support for web services. However, even through support for the personalization (via presentation adaptation to user context) was already addressed in some approaches, they do not offer fully automatic adaptation process. Indeed, our approach starts with the discovery of the need for the adaptation and ﬁnish with the implementation of the adaptation, like a a semantic loop in which each phase beneﬁts from the data prepared in the previous phase.

15.7 Conclusion

In this chapter we presented a novel approach for the adaptation of web portals. By monitoring the behavior of users and analyzing this data, planning which actions should be taken and executing them, a kind of a "usage loop" is created. In that way, the system aims to be a user-friendly platform that integrates the results of the usage data’s analysis with guidance for portal adaptation. Even though semantic knowledge can be integrated and used at every phase of the usage loop, in this chapter we focus on combining the power of web usage mining with Semantic Web technologies to create a semantic framework for learning preference rules. We identiﬁed research challenges for the mapping of users’ activities at the clickstream’s level to more abstracts entities deﬁned in the ontology. The proposed solution takes into account different aspects and levels of semantics. We also performed the initial evaluation. The implementation details are described in [Stojanovic, 2007]. Finally, although, in this chapter, the discussion centers around e-government portals, the approach is easily generalizable to other types of portals, since at an abstract level most portals can be said to offer some combination of services and associated information.

References

D. Apostolou, D. Feldkamp, C. Halaris, K. Hinkelmann, B. Magoutas, X. Papadomichelaki, C. Prackwieser, F. Probst, K.-U. Schmidt, B. Stoiljkovic, V. Stoiljkovic, L. Stojanovic, N. Stojanovic, S. M. Thomas, B. Thönssen, W. Utz, and R. Woitsch. Framework for self-adaptive e-government. EU/IST Project FIT Deliverable D2, FZI, June 2006. H. Dai and B. Mobasher. Web Mining: Applications and Techniques, chapter Integrating Semantic Knowledge with Web Usage Mining for Personalization. Idea Group Publishing, 2005. P. de Bra and L. Calvi. AHA: A generic adaptive hypermedia system. In 2nd International Workshop on Adaptive Hypertext and Hypermedia, Pittsburgh, Pennsylvania (USA), 1998. S. Kampa, T. Miles-Board, L. Carr, and W. Hall. Linking with meaning: Ontological hypertext for schol- ars. Technical Report ECSTR-IAM01-005, Electronics and Computer Science Department, University of Southampton, 2001. Y. Lei, E. Motta, and J. Domingue. OntoWeaver-S: supporting the design of knowledge portals. Lecture Notes in Artiﬁcial Intelligence, 3257:216–230, 2004. N. Moreno, P. Fraternalli, and A. Vallecillo. A UML 2.0 proﬁle for WebML modeling. In 2nd International Workshop on Model-Driven Web Engineering, in conjunction with the 6th International Conference on Web Engineering, Palo Alto, California (USA), 2006. L. A. Ricci and D. Schwabe. An authoring environment for model-driven web applications. In 12th Brazilian Symposium on Multimedia and the Web, pages 11–19, Natal, Rio Grande do Norte (Brazil), 2006. K.-U. Schmidt, L. Stojanovic, N. Stojanovic, and S. Thomas. On enriching Ajax with semantics: The web personalization use case. Lecture Notes in Computer Science, 4159:686–700, 2007. L. Stojanovic. Methods and tools for mining the log data by taking into account background knowledge. EU/IST Project FIT Deliverable D18, FZI, July 2007. 15 Self-Adaptive Portals: E-Government Case Study 195

L. Stojanovic, J. Schneider, A. Maedche, S. Libischer, R. Studer, T. Lumpp, A. Abecker, G. Breiter, and J. Dinger. The role of ontologies in autonomic computing systems. IBM Systems Journal, 43(3):598–616, 2004. K. van der Sluijs and G.-J. Houben. A generic component for exchanging user models between web-based systems. International Journal of Continuing Engineering Education and Life-Long Learning, 16(1-2):64– 76, 2006.

16 Use Cases and Challenges of Personalized Healthcare

Mikhail Simonov

Istituto Superiore Mario Boella, Services and Applications Laboratory Via Pier Carlo Boggio 61, 10138 Torino, Italy [email protected]

Summary. The need of personalisation in healthcare is justiﬁed by the fact that no two people are exactly alike. This chapter treats personalised services in healthcare, which are tailored proactively for each patient at several levels to empower individuals. We speak about the multimedia supporting high quality sustainable healthcare service, with personalisation focusing on prevention and detection rather than treatment. We explain some selected use cases along the healthcare value chain: prevention, alert service, care and nutrition, food, informed consent, and home living and elderly.

16.1 Introduction

Personalised Healthcare Service (PHS) is the integrated practice of well-being, healthcare and patient support, based on an individual’s unique biological, behavioural, social and cultural characteristics. Personalised service tailored proactively for each patient at several levels empowers the individual by “the right care for the right person at the right time”, leads to better outcomes, improves satisfaction and keeps cost-effectiveness. No two people are exactly alike, and medical conditions may vary by age, race, gender, and a variety of other factors. Therefore, population with vulnerable groups and those with particular pathologies needs an effective individualised health support. The high quality PHS answer to individual’s chronic (long-term) and episodic (short-term) needs, reﬂect social factors and patient’s cognitive level. Sustainable services focus the prevention, early pathology detection and homecare rather than expensive clinical settings, shift the responsibility to individuals, move beyond alert systems, but check the overall well-being anticipating needs and ensuring compliance to care plans. Several personalisation opportunities are offered by the healthcare domain [Grasso et al., 2005, 2007]: hospital service, specialised physician services, day hospital, GP service, short-term post-acute medical service, long-term skilled nursing service, rehabilitation service, hospice, wellness service, home healthcare designed to preserve quality of life, immunisation, food and environmental control, veterinary service, driving and related services, invalidity, impairments and augmentation of capacities, assistive service, etc. In this chapter, we view the healthcare picture as one of several interconnected areas, generating needs and challenges, ranging from hospital (producer) to person (consumer), from work (enterprise) to home (leisure), from the state-of-the- art to future R&D (see Fig. 16.1). The following sections describe a number of selected use cases in which personalization techniques are called to playing a relevant role in the future.

16.2 Information Service, Colorectal Cancer Prevention

Colorectal cancer (CRC) affects mostly people over the age of 50, women and men equally. CRC is one of the most common types of cancer, often silent since symptoms do not always develop [Li et al., 1999] until it is 198 Mikhail Simonov

Fig. 16.1. A global picture of healthcare. difficult to care. There are known risk factors, many of which are modifiable. A diet high in fat, protein, calories, alcohol, both red and white meat, and low in calcium is more likely to develop CRC, rather that a low-fat, high- fibre diet. A diet high in saturated fat combined with a sedentary lifestyle, or smoking cigarettes increase the risk of CRC. Colonoscopy is used in testing for early signs of cancer in the colon and rectum because lets the physician look inside entire large intestine and enables to see inflamed tissue, abnormal growths, ulcers, and bleeding. CRC associated mortality can be ameliorated, because it is mostly curable if early found through regular screening tests: a good preventive challenge exists. Effective screening tests depend on high uptake and ad- herence within a target population of older adults. We may deliver specific tailored information via web to assist individuals in making a decision to undertake screening. This decision aid aims to adopt behavioural theories to define the psychosocial variables within a user model, which could be used to personalise tailored information to influence and sustain a decision [Street et al., 1995] to undertake CRC screening then. The effectiveness of such help through improving knowledge, reducing decisional conflict and stimulating more active decision-making is supported by research. We may set up the adaptive personalised multimedia informative service, which effectiveness is enhanced by tailoring to an individual and their needs. Adapted printed communications are read, remembered and perceived as more relevant than non-adapted materials. Comparison of web- and non-web information interventions in areas of knowledge and targeted behaviour change shows enhanced outcomes among web-delivery users. The tailored information web delivery therefore poses unique development challenges; impacting on a user base and their acceptance of information systems that aim to influence screening behaviours using educational and behavioural interventions. The target is those having more than 50 years old, with a limited familiarity with computer technologies. The computerised decision support system stimulating the risk perception, self-efficacy, response efficacy and an individual’s set of behavioural determinants may convince to initiate and sustain the process of CRC screening. The technology of user modelling provides a means of delivering a tailored mix of educational content directed at an individual’s specific motivation, beliefs, knowledge and other determinants that affect behaviour. A survey might identify and measure individual determinants of CRC screening beliefs and behaviours and outcomes from this will be used to inform decision support frameworks and an individual’s user model. Stage of change (readiness to screen) will also be measured. Framework should ensure the granularity of these user models, impacted against the modes of data acquisition, to develop user models, e.g. the value of tailoring surveys versus alternate acquisition through end user information system observations. 16 Use Cases and Challenges of Personalized Healthcare 199

An implementation would reflect characteristics of interpersonal communication; verbal interactivity equiv- alence within the individualised user model to act in the capacity of a pedagogical tool. A variety of sources might be used for creating and assessing applicability of educational content and feedback messages. This information should be used to develop text that describes the nature of and risk factors for CRC, the value of screening, and descriptions of testing procedures and the implications of test outcomes. Information will be personalised on the basis of name and basic personal characteristics, and also tailored to meet the specific informational needs of each user. The personalisation is applied to the delivered content, to the personalised behaviour of the system, and the adaptation to the typical needs f the expected group of users. To measure the adaptivity, the outcomes might be compared between the paper and electronic modes. To effectively compare screening uptake, a range of additional outcomes needs to be included. These consist of intention to undertake screening and the screening test selected, appropriateness of screening test selected is determined by comparison to clinical guidelines, participant satisfaction with their decision and decision- making process, satisfaction with the information provided and the mode of its delivery, anxiety caused by the intervention, decisional conflict, and cost effectiveness of the electronic process.

16.3 Ultraviolet Ray Alert

The Sun exposure might become critical. Studies have shown that reducing exposure to ultraviolet (UV) radiation decreases the non-melanoma skin cancer incidence. UV radiation is a stream of invisible to human high-energy rays coming from the Sun or artificial sunlamps. Three UV categories exist: A, B and C, but only UVA and UVB pass the ozone layer. UVA level remains stable the whole year. UV causes skin to tan, and human eye perceives UV effect in tanning. Atmospheric, seasonal and geographic variables change the UVB quantity, requiring measuring the UV level through monitoring. At noon and in summer, more UVB outstrip the ozone layer, while at dawn and dusk there is virtually no UVB. A non-burning tanning is the human body’s natural and intended response to UV exposure serving to protect against sunburn and overexposure. Melanocytes produce melanin, a protein pigment determining the skin colour protecting from overexposure to UV, which is why people have different skin colours. The colour ultimately depends on heredity and previous exposure to UV, two factors, which predetermine the amount of melanin your skin will contain. In excessive doses, UV can cause sunburn. On the micro-level, any UV exposure causes skin damage, while on the macro-level UV exposure is natural and necessary to lead a healthy life. Research indicates that repeated over-exposure and sunburning are the primary sun-related factors responsible for an increased risk of permanent skin damage. Repeated burning is believed to be the greatest risk factor for long-term skin damage, which creates a challenge for preventive services related to sunburn. Skin type classification is the Fitzpatrick phototype accounting skin’s melanin pigment, determined by con- stitutional colour (white, brown or black) and the result of UV exposure (tanning). People with skin types I (pale white skin, blue/hazel eyes, blond/red hair; always burns, does not tan) and II (fair skin, blue eyes; burns easily, tans poorly), are at the highest risk for side effects including wrinkles and skin cancer, and needs protection. Changing patterns of outdoor activities, wearing protective clothing, and using adequate sunscreen reduces Sun exposure. There is a challenge for prevention. It is estimated thus that the maximum exposition to Sun during the "first" day will be 5 minutes for those having the skin of type 1, 10 minutes for the type 2, 20 minutes for the type 3, 30 minutes for the type 4. Knowing the protection factor of sunscreen (SPF) and the skin prototype, we may calculate the maximum exposure. For instance a person with skin type 1 using SPF 8 sunscreen might enjoy the Sun for 40 minutes (8 x 5 minutes). In literature we find objective measurements of constitutive skin colour and ultraviolet light sensitivity [Lock-Andersen et al., 1999] in relation to risk of Cutaneous Malignant Melanoma (CMM). In addition to constitutive skin colour and skin ultraviolet light sensitivity, the assessment by colorimetry and Minimal Erythema Dose (MED) are suggested. Saying this we can establish a personalised (because of the personal characteristics) multimedia (because of the imaging and colorimetry) real-time (because of the calculation of the total accumulated exposure and MED) service in preventive medicine area. We adopt an instrumental measurement of skin colour and skin ultraviolet light sensitivity in order to estimate a risk of CMM, a device to account the quantity of sun received by recipient, and an ICT enhanced system able to calculate and react raising an alarm. The target group becomes the adult 200 Mikhail Simonov population having skin type 1, 2 and maybe some 3. The belonginess to the target group we decide applying the skin check-up to identify the skin phototype, which is an easy measurement based on imaging technique using a probe to apply to the epidermis, which emits an electromagnetic radiation towards the skin, while a receiver captures electromagnetic radiation reflected by the epidermis and generates an electrical signal, which will be elaborated by a microprocessor, determining the associated phototype of the user on the basis of the intensity of the signal generated by said receiver. One instrument for the objective measurement of constitutive skin colour is the portable tristimulus re- flectance colorimeter (Figure 16.2), specifically analysing hue and chroma values. The instrumental method of evaluating cutaneous sensitivity to ultraviolet radiation is based on measurement of the MED to provoke perceptible erythema of the skin 20-24 hours after exposure. Although not a perfect measure, MED provides a direct and quantitative measure of the skin’s sensitivity to UV, and there are public statistical data, showing correlations between instrumental measurements of skin colour and skin sensitivity among controls in a case-control study of CMM.

Fig. 16.2. A tristimulus reﬂectance colorimeter.

The personalised service uses a small wearable UV-sensitive device with photo-cells to calculate the Sun intensity during the day, calculating automatically the total accumulated exposure. Parameterised by skin type, this service will be an example of the parameterisation for a group, while the detection of the exposure and the counselling will be individualised, depending on the geographical situation. Colorimeter acquires the skin type automatically, or the value is input manually once, before using the service. The Sun intensity monitoring should be continuous because of above-described physical peculiarities. The information about SPF-products applied to the skin should be done in real time, snapshooting application’s time (timestamp and SPF value used), because impacting on the accumulated exposure. Micro-controller accounts the exposure, and when it overstep the maximum allowed threshold for a given skin type, an acoustic warning is issued. An option might be the remaining allowed exposition calculation warning to re-apply the SPF sunscreen again, or the approximation of the possible outside stay. The further sophistication might be the client-server application acquiring from a centralised monitoring station the data about the Sun intensity, probably less indicated in the consumer’s vision. The above-described case might bankrupt as mass-service, but will be winning if we propose it in assistive terms: Alzheimer disease sufferers lacking the ability to remember, but having the skin phototype "1" will need and use it. In this case an alarm should be sent also to the caregiver or parent for appropriate actions.

16.4 Care Plan, a Diet and Drug Service

Homo sapiens are omnivorous, capable of consuming both plant and animal products, while both a pure animal and a pure vegetable diet can lead to deficiency diseases: the first one can provoke scurvy, while a pure plant diet provokes deficiency in nutrients and vitamins (B12) suggesting supplementing them. Intensive agriculture alters the kind of food people eat, while the types of food consumed and the way in which they are prepared, vary widely by time, location, culture, religious, ethical, ecological, or health reasons. The diet is mainly reflected by culture, which has generated a food science. Being global food distribution uneven, the obesity among 16 Use Cases and Challenges of Personalized Healthcare 201 some populations has increased to almost epidemic proportions (in USA more that 60%), leading to health complications and increased mortality. Consumption of calories in excess causes the obesity, especially in a combination of overeating and insufficient exercise. We may deliver some specific tailored information to assist individuals in making a decision to change their lifestyle, similarly to CRC use case, but it should be complemented by the individual diet development and some other components, actually offered by nutritionists. A diet plan is typically designed by them to suit patient, his needs and his lifestyle [World Health Organization, 2004]. To be accepted by consumer, it is typically personalised reflecting individual likes and dislikes, specific needs, and even relates to a favourite supermarket. The further adaptivity level would offer a very flexible plan allowing to replace any meal for something different, warning should you choose something above your daily allowances. Further flexibility level might be a diet plan considering seasonable variations, economic conditions, driving to satisfy the informed choice in supermarket while purchasing. We may build a personalised multimedia diet service because developed countries encourage people to take responsibility for their own well-being. This is scientifically justified, because allow to save expenditures on both obesity-related heart disease and diabetes. The new service relies on the intensive use of ICT, starting from mail messages to allow GPs to monitor patient’s progress in following their new regimes, as well as the introduction of a new advice service using any communication channel (phone, mail, digital TV etc.) to deploy the information. Television is fundamental for modern human, but a personalised content delivery suffers because the broadcasting to millions is not adaptive. Digital TV increases the interactivity potential, provides the opportunity to change the way we use TV, and opens up the possibility of personalised, adaptive content delivery to single users or groups of users, while a content delivery in e-Health appears very attractive in commercial terms. However several questions should be solved to become real life applications: to adapt and personalise content, to deliver one-to-one, to match specific target audience (elderly/chronic patients). People tend to watch in groups because of the social experience, and discuss with others what they have seen. From a personalisation point of view this offers a challenge: to adapt needs to take place to (small) groups rather than adapting to individuals. Instead of user modelling, we switch to group modelling, even if the members of a group can be quite diverse. Imagine a family watching TV set together, where all family members kept motivated: a daily diet and varying pills list delivered for grandfather might be that case. We decided to accompany a dismissed from hospital person by a personalised DTV multimedia service, delivering one-to-one an interactive Health TV. The template diet should be personalised and be enriched by the prescription of medicines, obtaining the personalised diet plus the individual treatment plan. An automated content composition benefits from semantics, while digital content is encrypted to ensure the privacy and dis- patched to our users using real time contextualisation and filtering. DVB and smart card technologies offer the unicast and groupcast to support this scenario. To define the service fully, we should understand the need of the audience in e-Health to achieve the efficiency, while a real life typical use case suggests the following case. Mr. Smith, man, 71 y.o., typically married and living at his home with a wife, early dismissed from the local Hospital after the health incident (heart attack) treated in urgency, has never used a PC but familiar with TV, willing to receive information about his heath status on TV, assisted by his wife (supports our initial thesis about group modelling). After the de-hospitalisation, he remains in phone contact with his hospital’s doctor, and he will be visiting his family doctor. Local hospitals have several multimedia materials for de-hospitalised people, but the content personalisation [Bradley et al., 2000] needs to be automated, since there are several rules to apply before distribution. For instance a set of personalised diets is developed, guidelines explaining how to face new living conditions after given pathologies are available, but all materials should be delivered in a secure way to (i) single user, (ii) group of users suffering the same pathology, (iii) people at risk for a given factor. The envisaged service will encode deployment rules in one repository, and describe semantically the available content, rules for the content aggregation. The broadcasting facility sends data in one-to-one mode to the subscriber, securing confidential material by conditional access keys on smart card. On server side every day the central content management system automatically executes the rule for Mr. Smith to generate a personalised diet to be delivered him within next 2 weeks. On client side, Mr. Smith will simply switch on the TV set and fruit the content. The content corpus is available as a set of templates, personalised applying several rules doing some calculations. The personalisation starts by selecting a suitable template. The personalised content composer is pushing the whole 202 Mikhail Simonov template initially, but excluding all commas in conflict with rules, processed item per item. Mr. Smith case was characterised by attributes: (i) high blood pressure, (ii) pulmonary emphysema, (iii) gluten allergy, (iv) renal in- sufficiency, and this set will be properly reflected by metadata accompanying the user profile (possible indexing by individual address also in smart card). This simplifies the exclusion of several templates. Following special rules are applied while content authoring for Mr. Smith: (i) hypo-sodium diet, to minimise salt; (ii) diuretic, to promote urine flow; (iii) reduced meat, up to one time per week; (iv) no gluten; (v) always add to commercial names in pharmacology their respective generic names or molecules (preference). The Pharmacology treatment prescribed in an example was the following one: Adalat, being the commercial name this should be extended with the “nifedipina” generic equivalent, 40 mg, at 8 AM.; Lasics, e.g. “furosemide”, 20 mg, at 8 AM, at 4 PM.; Ventolin spray, e.g. “salbutamolo”, 3 times a day, at 8 AM, at 4 PM, at 10 PM. The case was illustrated (Figure 16.3) by video during EuroITV 2007 conference [Simonov et al., 2007].

Fig. 16.3. Video demonstration at EuroITV 2007.

The architecture of the above-described prototype is representative of the whole value chain, because includes: the repository of the digital content, an authoring subsystem to be used in back ofﬁce, an automated rules execution engine, a broadcasting facility coupled with IP networking, a hybrid personalised set top box on client’s side offering dual HMI facilities, such as TV set and Java-enabled web browser.

16.5 Food, an Informed Choice

We can enhance the previous example offering some cooking instructions to prepare meals, or the guidance application driving through the supermarket’s shelf to make the informed choice [UK Department of Health, 2004]. Someday soon grocery shoppers using a wireless PDA may be able to interact with a store’s computer system to locate items and learn about special promotions. Once the shopper equipped with the personalised diet enters the store, the list of good to buy is reordered to provide the most efficient route to obtain every item on the list spending less time. Shoppers check off items as they acquire them, review and add specials to the list, and view and save recipes and watch for in-store specials. Assuming the afore-mentioned use case’s user allergic to gluten, the personalised service would advert him that the “pasta” placed in the shopping cart should 16 Use Cases and Challenges of Personalized Healthcare 203 be replaced by gluten-free one. We can alert about the insufficient, forgotten of exceeding in quantity items to achieve the weekly provision to ensure the preparation of requested meals, while the missing article might generate the substitution one on-the-fly. This hypothesis implies the cooperative system enhancing the PDA by the access to the supermarket’s database of products, and the live link to recalculate the diet, whenever need to replace some of articles. This example shows benefits added to a personalised service by interoperability and cooperative approach.

16.6 Hospital, an Informed consent

In surgery multiple interventions during one surgical episode are common. Each intervention must be explained, its intended and potential consequences articulated, and informed consent [Meredith and Wood, 1998] of the patient obtained. Although the pre-surgical meeting between the patient and the surgeon is the opportunity to accomplish this, it is essential that the patient be given educational materials to complement and augment face- to-face exchange, however this is unlikely well done with brochures, because many combinations of procedures are possible, different patients have different concerns, and may have varying levels of literacy and knowledge. In the extreme, a set of brochures selected from hundreds of variants, or the same set of brochures without regard for differing needs, to be given to every patient, will not be acceptable. Creation of a tailored information document, customised for every individual patient would potentially increase relevance and effectiveness of the educational material. Customisation might be done in accordance with patient medical conditions, demographic variables, personality profile, or other relevant factors. A possible solution requires reformulation, extension, and optimisation of the content. A digital Content Management System (CMS) adopting a tailoring engine, coupled with the creation of a database of educational modules pertaining to each subcomponent of a given surgical intervention would be the challenge. An authoring tool might assist surgeons to enter the text, which will inherit some annotations from the database in order to be assembled automatically by tailoring engine into coherent material. The potential solution might be the creation of the input material for natural language CMS, solving also selection-and-repair paradigm [O’Donnell et al., 2000]. The preparation of a knowledge base, which is a resource for natural language generation, includes the manual incorporation of a taxonomic organisation, to ensure coherent and high-quality text. This led to the database of reusable texts (master templates) as the basis for “generation by selection and repair”. There are several other approaches to natural language authoring [Hartley and Paris, 1997, Power and Scott, 1998]: pure natural language generation approach, semantic input provided interactively by a person rather than by a program accessing digital knowledge presentations, user operating directly upon a knowledge model from which the final output text will subsequently be generated, and some others. At the end, an outcome drive to patient-centric healthcare: a means of shaping complex information so that it is more relevant and personalised, a mechanism for assisting in the achievement of informed consent to procedures, a method that has been shown to improve patient engagement and compliance with medical regimens, and a technique for complementing and reinforcing the information communicated during the pre- surgical encounter. The tailoring process permits inclusion, exclusion and/or modification of educational information based on a variety of criteria, including the surgical procedure(s) being performed, impact of adjuvant therapies, medical co-morbidities, and potentially any other factor deemed significant. Although no amount of supplemental documentation can replace the surgeon-patient dialogue with which informed consent is obtained, it is well documented that only a small fraction of the information communicated in this process is actually retained by the patient. Reference material for review by patient, friends, and significant others would have great value in the preoperative, perioperative, and postoperative stages if this information could be tailored to the individual patient. This observation is supported by recent work in patient education attesting to the potential value of increasing patient involvement in the surgery decision through patient-centred methods and using quality information brochures to improve surgeon-patient communication. The envisaged personalisation system will encompass following components: a natural language CMS tailoring and generating the text, a content authoring environment helping surgeons, a database of educational modules (templates) pertaining to each subcomponent 204 Mikhail Simonov of a given surgical intervention, a knowledge base with the taxonomy, clinical pathways and guidelines, and composition rules.

16.7 Home, Living with Disease but not by Disease

Many people suffer chronic diseases. Hypertension is one affecting 40% adult Europeans whose care requires mostly an attention to healthy life habits and care of blood pressure values, weight and heart rate monitoring. Clinical findings state that long-term elevated blood pressure increase the probability of serious heart and brain damages, causing stroke and myocardial infarction. Other studies state that home monitoring increase health outcomes and grant overall cost savings. Drugs available to reduce blood pressure are numerous and effective, however there is low compliance to drug prescription by GP and assumption by patients. Doctors apply best-practice care procedures and internationally accepted guidelines adapting them to patient. This opens a challenge to formalise such knowledge and enable machine reasoning by DSS. An effective care requires doctors being timely informed about the health status of their patients. They ask the blood pressure values (does require wearable measurement device), whether side effects occurred (sensors detecting them while tele-monitoring), whether patients were compliant with the prescribed therapy (pill box interfaced with DSS) and life style (activity monitoring devices). Therapy comprises drugs and life style modifications (to deliver the drug assumption and diet plan), doing more physical exercise (remote monitoring while jogging), following a diet and ceasing to smoke (artificial nose spying). Interaction between physician and patient, and the information exchange between the two, are therefore essential for the effective care chronic diseases and of hypertension. Typically organisational and economic constraints lead to few patient encounters, once or twice every year, asking a blood pressure Holter (an automatic collection of blood pressure values within 24 hours). The adaptive service enables doctors and patients to continuous information flow. The portable Holter device will be enhanced to use wireless connectivity. It transfers data to the care centre, where doctors are able to evaluate it and decide, when necessary, suitable changes to make the therapy more effective. A data collection system organises acquired series and present them in an easily interpretable way, supporting decision processes. The service rely on the patient identification, blood pressure quantitative assessment, overall patient assessment, health status understanding, side effects investigation, life style investigation, counselling and advises, next occurrence scheduling, service conclusion. The service might be set up on the continuous basis, or on-demand one, like a normal Holter examination. The continuous life monitoring and looking on adverse events or side effects is typically set up adopting wearable multi-sensorial devices, for instance engineered as wrist watches like in Fig. 16.4.

Fig. 16.4. A wrist watch featuring healthcare sensors.

Models for behaviour change highlight the importance of being conscious of the beneﬁt that will be gained by modifying patient (own) behaviour [Deci and Ryan, 1985]. The ability to convey convincing and effective messages relies on theories of behaviour adoption, such as the Social Cognitive Theory [Bandura, 1986], the 16 Use Cases and Challenges of Personalized Healthcare 205

Health Action Process approach [Schwarzer, 2001], and some others. The behaviour is detected using tele- monitoring service and wearable multi-sensorial devices. Tele-measurement reporting patient’s heart rate up might depict an abnormality, but an increased heart activity in jogging patient is normal: we need details about activities and patient role (home-staying or working).

16.8 Conclusions

Healthcare aims on the high-quality care, ready access to the system, and affordable costs, but first two goals aggravate the last. Proactive reasoning, more semantics, context understanding, and knowledge-aware assistive services depict the future scenario. Collaborative care, continuity of care, and the switch from event-ruled case-based medicine to evolving pattern-ruled systems led to data-warehouses, predictors and automated event triggering. Relevant topics include the modelling and automation of healthcare processes, the flexible implementation of medical guidelines and pathways, the realisation of lifetime patient records, the provision of patient information and medical knowledge at the point of care, the evolution of healthcare information systems, and privacy and security issues. Healthcare inflates because of ageing population, challenging to facilitate a new paradigm of personalised one within the given context. Common medical record systems will enable people to receive medical treatment anywhere without contacting their local doctor or hospital. Citizens will be enabled and supported to live more healthy lives, minimizing time in hospital, at local doctors or in care homes. Home monitoring will become more widely available for people considered at risk. This requires better monitoring regimes for chronically ill patients, through monitoring of vital signs. As a result, the increasingly elderly population will be able to live more independently in their home environment, overcoming isolation and minimizing their reliance on carers. Personalisation in healthcare implies a series of research issues in determining how information technologies can be used. The convergence with life sciences impacts on future personalised services, because it depends on capabilities of stakeholders and on how we choose the technologies to implement services. There is a lack of appropriate high-quality outcome in semantic multimedia for integration of multimedia processing. Telemedicine personalisation is influenced by different, not only technical, factors like end-users, operational issues, clinical/technological standards and clinical interoperability. The role of genomic is clear but how to incorporate clinically useful genetic/genomic information into EHR, or analytical tools supporting clinical decision-making, protecting them, remain an open issue. To design systems enabling on-demand tele- monitoring services for medical devices, unlike current store-and-forward systems that only allow viewing the last posted monitoring data of a device, is a challenge. Most standardisation work is done on single-medium objects, due to the difficulty in doing this, while there are media images with audio soundtracks, text reports containing images or bio-signals. Most work has either been on small extensions to mono-media standards, e.g. text, bio-signals in DICOM, or in carrying separate mono-media objects for each medium. Many different technologies, the wide adoption of dedicated devices around, and the extreme variety of roles and uses qualify the real life. From the research perspective, the future challenge would be the evolution towards smarter, computationally powerful, and more intelligent devices, encompassing machine-reasoning and automated decision aid to simplify real life. Another challenge is making multimedia data more machine- processable, investigation on better context-sensitivity and integration with nano-technology.

References

A. Bandura. Social foundations of thought and action: A social cognitive theory. Prentice Hall, 1986. K. Bradley, R. Rafter, and B. Smyth. Case-based user proﬁling for content personalisation. Lecture Notes in Computer Science, 1892:62–72, 2000. E. Deci and R. Ryan. Intrinsic motivation and self-determination in human behaviour. Plenum Publishing, 1985. 206 Mikhail Simonov

F. Grasso, S. Quaglini, C. Paris, A. Cawsey, and R. Wilkinson, editors. Workshop on Personalization for e- Health, in conjunction with the 10th International Conference on User Modeling, Edinburgh, United King- dom, 2005. F. Grasso, A. Cawsey, C. Paris, S. Quaglini, and R. Wilkinson, editors. Workshop on Personalization for e- Health, in conjunction with the 11th International Conference on User Modeling, Corfu, Greece, 2007. A. Hartley and C. Paris. Multilingual document production from support for translatingto support for authoring. Machine Translation, 12(1–2):109–129, 1997. F. Li, T. Kishida, and M. Kobayashi. Serum iron and ferritin levels in patients with colorectal cancer in relation to the size, site and disease stage of cancer. Journal of Gastroenterology, 34(2):195–199, 1999. J. Lock-Andersen, F. de Fine Olivarius, M. Haedersdal, T. Poulsen, and H. C. Wulf. Minimal erythema dose in UV-shielded and UV-exposed skin predicted by skin reﬂectance measured pigmentation. Skin Research and Technology, 5(2):88–95, 1999. P. Meredith and C. Wood. Inquiry into the potential value of an information pamphlet on consent to surgery to improve surgeon-patient communication. Quality in Health Care, 7(2):65–69, 1998. M. O’Donnell, A. Knott, J. Oberlander, and C. Mellish. Optimising text quality in generation from relational databases. In 1st International Conference on Natural Language Generation, volume 14, Mitzpe Ramon, Israel, 2000. R. Power and D. Scott. Multilingual authoring using feedback texts. In 17th International Conference on Computational Linguistics, Montréal, Canada, 1998. R. Schwarzer. Social-cognitive factors in changing health-related behaviors. Current Directions in Psycholog- ical Science, 10(2):47–51, 2001. M. Simonov, P. Brizzi, M. Gavelli, and A. Sciarappa. TV for health: One-to-one personalized content delivery in living room. In Workshop on Personalization in ITV, in conjunction with the 5th European Conference on Interactive TV, Amsterdam, The Netherlands, 2007. R. L. Street, B. Voigt, C. Geyer, T. Manning, and G. P. Swanson. Increasing patient involvement in choosing treatment for early breast cancer. Cancer, 76(11):2275–2285, 1995. UK Department of Health. Better information, better choices, better health: Putting information at the centre of health, 2004. URL http://www.dh.gov.uk. World Health Organization. Guidelines on diet, physical activity and nutrition. http://www.who.int, 2004. 17 Context-Aware Personalization for Services in the Smart Home: A Semantic Approach for OSGi services

Rebeca P. Díaz-Redondo, Ana Fernández-Vilas and Manuel Ramos-Cabrer

Department of Telematics Engineering, University of Vigo, 36310, Spain rebeca,avilas,[email protected]

Summary. Ideally, smart homes should be able to help their inhabitants to have more comfortable lives by anticipating their needs and satisfying their tastes, i.e smart homes should automatically provide the most appropriate services according to the inhabitants’ feelings and behaviors and the environmental atmosphere around them. Obviously, this goal entails combining a lot of different technologies and research ﬁelds, like residential gateways, context-awareness, personalization, sensors, home networks, etc. In this chapter we introduce an approach for context-aware personalized smart homes based on two main pillars: the OSGi (Open Service Gateway Initiative) platform and the Semantic Web philosophy. The former is the most adopted solution to the technological problem of building residential gateways, and the latter supports ontology-based characterization of complex knowledge. By combining both ﬁelds, we enrich the OSGi service-oriented architecture by providing a semantical conceptualization of (i) services at home, (ii) contextual information and (iii) inhabitants’ preferences. This ontological structure supports reasoning about the captured behavior and inferring new knowledge to provide a more comfortable life at home.

17.1 Introduction

Nowadays, the OSGi (Open Service Gateway Initiative) Service Platform specification [Open Service Gateway initiative, 2005] is the most adopted solution to the technological problem of building a control system for the networked home. Its success can mainly be ascribed to the following reasons: (i) the communication among all devices in home is easy because it supports different widespread protocols; (ii) it defines a cooperative model where applications can dynamically discover and use services provided by others; and (iii) it is possible a flexible remote management of these applications and the services they provide. This solution fits in with the decentralization, diversification and ubiquity of pervasive environments in general and in smart home environments in particular. However, building a smart home involves so complexity that it is essential to properly combine different technologies. OSGi could be the core of the smart home, but context-aware computing and personalization are also key factors. Context-awareness is commonly understood by those working in ubiquitous/pervasive computing, where context refers to the physical and social situation in which computational devices are embedded. These devices should act and/or react according to their contextual information. Thus, acquire, store and manage adequate this information is essential for supporting context- awareness. On another hand, personalization consists of tailoring any system (consumer product, electronic or written medium, for instance) to a user based on his/her characteristics and preferences. In the smart home field, personalization entails devices at home works accordingly to the inhabitants’ preferences and/or habits. Therefore, any intelligent house should be able to obtain contextual information about where and when the different devices are used and personal information about who and how is using them. It should also manage the acquired knowledge and be able to learn about it. So, in an ideal scenario, the smart home will help its inhabitants to have more comfortable lives by anticipating their needs: activating/deactivating the devices according to their preferences, optimizing energy consumption, providing appropriate security systems, etc. 208 Rebeca P. Díaz-Redondo, Ana Fernández-Vilas and Manuel Ramos-Cabrer

Combining the ideas above, our proposal focuses on enriching the OSGi platform to support both context- awareness and personalization. Thus, the platform we introduce is able to learn about the preferred services by inhabitants in specific contexts and, consequently, to invoke those appropriate services when applicable. This entails the services must be automatically discovered, selected according both the contextual information and user preferences and launched using the most adequate invocation parameters for the environmental atmosphere at home. Regarding automatic service discovery and invocation OSGi assume the client to have too much information about the service to use: it is obliged to know the name of the service interface (for discovery purposes) as well as its methods’ signatures (for invocation purposes). What is more, how can OSGi applications be remotely deployed and work properly on every customer platform if devices at home can be totally different from one customer to another? Besides, if the service platform is used to run services from different service providers, it is hardly realistic to suppose that a specific provider knows, a priori, the interfaces which match the services from other providers. In this context, the simple syntactic matching by name provided by OSGi is not enough. So we propose a semantic approach inspired by the Semantic Web conception [Berners-Lee et al., 2001], which is based on the markup of OSGi services by using appropriate ontologies. Therefore, OSGi services describe its properties and capabilities so that any other software element can automatically determine its purpose. Then, any software element is also able to determine how to invoke them. The technological requirements in the smart home share much in common with those of the Semantic Web. Both need to represent object-level information concerning classes and properties as well as relations that can occur among specific instances. This philosophy can support services characterization, as it was mentioned above, and may also constitutes the basis for context and user modeling, which are the core for context- awareness and personalization, respectively. Although there are different approaches to define context and user profiles, ontologies are undoubtedly a promising emerging instrument to specify concepts and interrelations. What is more, they support reasoning and inference mechanisms to identify restrictions and/or relationships among classes, properties and instances. Finally, applying ontology-based modeling techniques for managing contextual and personal knowledge perfectly fits with a semantic representation of OSGi services introduced above. To introduce OWL-OS/OSGi Framework, the context-aware personalized extension to OSGi we propose, the chapter is organized as follows. The next section is devoted to analyze other approaches related with the smart home technologies. Section 17.3 briefly overviews the characteristics of the current OSGi framework and its drawbacks regarding to automatic services discovery and invocation. The Semantic OSGi platform is detailed in Section 17.4, while all the aspects related to adding context-awareness and personalization to the Semantic OSGi platform are described in Section 17.5. All the advantages described in the previous sections are evidenced in the use case in Section 17.7. Finally, a brief discussion including conclusions and future work is presented in Section 17.8.

17.2 Related Work

Smart homes are able to automatically perceive changes at home and response consequently with the aim of autonomously help their residents to have more comfortable lives [Chenishkian, 2002]. Building smart homes entails integrating different computing technologies, such as ubiquitous computing, context-aware computing or home automation technology. This combination supports automatic interactions among residents, computer- equipped devices and the home environment.

17.2.1 Context Modeling

According to Dey [2001], context is: “Any information that can be used to characterize the situation of entities (i.e. whether a person, place or object) that are considered relevant to the interaction between a user and an application, including the user and the application themselves”. Consequently, if the goal is providing context-aware applications, context modeling is a must. In fact, there are relevant approaches to organize context 17 Context-aware Personalization for Services in the Smart Home 209 knowledge ready to be use [Strang and Linnhoff-Popien, 2004]. These proposals can be organized according to the scheme of data structures used to support contextual information exchange: In key-value models, A list of attributes in a key-value manner is the most simple technique to describe • contexts [Samulowitz et al., 2001, Schilit et al., 1994]. However, its simplicity does not support sophisticated solutions for ubiquitous computing. The markup scheme models are based on hierarchical data structures (markup tags with attributes and • content) to define profiles extending the Composite Capabilities/Preferences Profile (CC/PP) [World Wide Web consortium, 2007] and/or User Agent Profile (UAProf) [WAP forum, 2007]. However, capturing complex contextual relationships and constraints is difficult and non-intuitive using only markup schemes [Chtcherbina and Franz, 2003, Indulska et al., 2003]. The graphical models are based on general purpose modeling methods, which are extended to cover con- • text modeling: the approach of Bauer [2003] is based on UML (Unified Modeling Language), while that of Henricksen et al. [2003] extends ORM (Object-Role Modeling). Object-oriented models encapsulate the details of context at object level and hide them to other compo- • nents. Thus, access to contextual information is only provided through specific interfaces [Cheverst et al., 1999, Schmidt et al., 1999]. In logic-based models, the context is formally defined as facts, expressions and rules. Contextual infor- • mation may be inferred from the rules of the system and it is updated in the logic system as facts. Some early works in this area [Akman and Surav, 1997, MacCarthy and Buvac, 1997] have inspired more recent approaches [Guidini and Giunchiglia, 2001, Gray and Salber, 2001]. Ontology-based models use ontologies to provide an uniform way for specifying the model’s core concepts • as well as an arbitrary amount of subconcepts and facts, altogether enabling contextual knowledge sharing and reuse in an ubiquitous computing system. The approach of Ötztürk and Aamodt [1997] is one of the first works in this area, followed by [Strang, 2003, Strang et al., 2003, Wang et al., 2004]. In spite of the previous classification, having a successful context modeling usually requires of combined strategies. The application introduced in [Hatala et al., 2005] for museum guides, for instance, is based on merging an ontological context model and a logic-based reasoning engine. While a visitor is roaming around the exhibition, the proposed system can automatically provide guide or other proper audio objects in accordance with the user-defined rules in the reasoning engine. Specially relevant for the smart home, the approach of Oh et al. [2006] consists of defining any context at home by five concepts (4W1H) : who (resident identification), what (identifies the event that should happens), where (resident’s location), when (occurring time) and how (the way the event should develop within these conditions).

17.2.2 OSGi and the Smart Home

Previous approaches have tried to promote smart spaces by using OSGi, like in [Lee et al., 2003], where the authors propose using OSGi as a suitable infrastructure to integrate various devices and sensors to provide pervasive computing environments. However, they do not resolve the problem of service search and invocation. Dobrev et al. [2002] address these two problems directly but not from a semantic perspective, the authors present how services can be imported from and exported to the Jini and UPnP technologies, showing that OSGi perfectly bridges multiple discovery protocols. On another hand, Gu et al. [2004] propose to deﬁne a semantic environment for describing contextual information to be used by context-aware applications. However, OSGi is only used as a support layer, without improving the OSGi framework at all. Besides, personal data about inhabitant’s preferences is not taken into account together with the context-aware information. After an exhaustive revision of the state-of-the-art, we have not found any proposal about integrating semantic reasoning nor personalization within the OSGi framework.

17.2.3 Ontological Descriptions for the Smart Home

Some initiatives have came up in this area which lies on research issues of ubiquitous computing and user modeling. For instance, UbisWorld [Heckmann, 2003] is used to represent parts of the real world (a house, a shop, 210 Rebeca P. Díaz-Redondo, Ana Fernández-Vilas and Manuel Ramos-Cabrer etc) and all the individuals and elements belonging to it. Its knowledge is modeled by two ontologies which are under development: GUMO (General User Model Ontology) [Heckmann et al., 2005a] and UbisOntology, the ontology for ubiquitous computing. SOUPA (Standard Ontology for Ubiquitous and Pervasive Applications) [Chen et al., 2004] is another relevant approach to model and support pervasive computing applications. It consists of two set of ontologies: SOUPA Core ontologies which define generic vocabularies that are universal for different pervasive computing applications; and SOUPA Extension ontologies which define additional vocabularies for defining specific types of applications. However, no focus has been on the semantic description of the in-home services. To this aim, we propose to take advantage of the previous efforts made in the Semantic Web field, where the appropriate semantic description of services is the main point. In this area, the most salient initiatives to describe Semantic Web Services are WSMO (Web Service Modeling Ontology) [Roman et al., 2005] and OWL-S [OWL-S coalition, 2004]. Although both approaches cover the same field with the same goals trusting ontologies to get them, OWL-S1 is considered to be clearly more mature in certain aspects and expressive enough to be applied in the smart home environment [Lara et al., 2004].

17.3 An OSGi Overview

The OSGi platform [Open Service Gateway initiative, 2005] mainly consists of a JVM (Java Virtual Machine), a set of running components called bundles, and an OSGi framework. A bundle (Java ARchive, JAR) is the minimal deliverable application in OSGi and has a restrictive life-cycle externally managed by the OSGi framework. An OSGi service is the minimal unit of functionality. It is deﬁned by a Service Interface, specifying the service’s public methods, and is implemented as a Service Object, owned by, and runs within, a bundle. Thus, a bundle is designed as a set of cooperating services, which are discovered by other bundles after being published in the OSGi Service Registry. The life-cycle of an OSGi service comprises the following operations:

Registering. When a bundle registers a service object (using the REGISTERSERVICE() method), it specifies • the name of the Service Interface and may also include a further description of the service by using a collection of key/value pairs (a Dictionary object). Obtaining. Bundles can obtain an OSGi service by an active search using the GETSERVICEREFERENCE() • method, whose input parameter is the name of the Service Interface. Alternatively, the (Service Listeners) provides an event mechanism which automatically notified when a service is registered, modified, or is in the process of unregistering. Selecting. The GETSERVICEREFERENCES() method provides the possibility of refining the services • search. Its input parameters are (i) the name of the Service Interface and (ii) filtering information to help the matching process. The filter syntax is based on the LDAP language (Lightweight Directory Access Protocol) [Howes, 1996]. Invoking. Once any bundle has obtained the Service Reference object, it must obtain the service object • itself to be able to invoke the required method.

17.3.1 OSGi Drawbacks

Undoubtedly, a mechanism for automating service discovery and invocation, like the one supported by the OSGi Service Registry, is essential in ubiquitous environments like the one envisioned by a smart home conception. However, several drawbacks, mainly caused by using syntactic matchmaking, can be ascribed to OSGi mechanisms. The discovery phase is only able to retrieve Service References which (i) exactly match the Service In- terface name in the query or (ii) are described by the same syntactic keywords in their properties (if a ﬁlter is

1 The OWL-S coalition is currently working in an upcoming release 1.2 of OWL-S, whose details are available at http://www.ai.sri.com/daml/services/owl-s/1.2. 17 Context-aware Personalization for Services in the Smart Home 211 used). Consequently, both synonyms (semantically similar services, but syntactically different) and homonyms (syntactically equivalent services, but semantically different) are not properly managed. For instance, a service registered as “PRINT” is not discovered when searching for “PRINTING”. OSGi discovery may result on a list of matched services, for instance when several printers have been registered under the “PRINTING” Service Interface name. Adding ﬁltering information at the discovery phase, for instance “(TYPE=PS)”, can help to the selection process. However, the OSGi ﬁltering is again based on syntactic comparisons, so the synonym and homonym problems appear again. What is more, without a semantic interpretation of the service’s properties, there is no possibility for a smart selection of the “best” service. Finally, for service invocation, not only has the requester bundle to know the Service Interface name (“PRINTING” for instance), but it also has to know the name of the method and the number and type of its parameters. This is clearly an obstacle in a pervasive environment since it prevents “unaware” bundles (i.e. without prior knowledge about the Service Interface) from dynamically invoking the service.

17.4 A Semantic OSGi Platform

The problem of automatic discovery and invocation of services is not new and has been repeatedly tackled in the Web Services field. As a result of these previous works, it is commonly accepted that the solution for services discovery should involve the use of ontologies, since they support a common understanding [Gruber, 1993] for both the service requester and the service provider. Similarly, a semantic conceptualization applied also to the service properties can also enhance the selection of the most appropriate service. Finally, the description of OSGi services should include not only its semantic classification (category and properties) but also enough information for a requester bundle to invocate it. Consequently, we have defined a Semantic OSGi platform [Díaz-Redondo et al., 2008] where the table- based structure in the OSGi Service Registry was replaced by an ontological structure which integrates all the ingredients above: semantic classification, semantic description of properties and information of invocation. Additionally, we have also provided a way for OSGi bundles to register semantically their services and to get the correct references to the needed ones. Within this context, any software element is able not only to determine the purpose of any OSGi service and to select the most appropriate according to its interests, but also to automatically invoke it.

17.4.1 The OWL-OS Ontology

The OWL-OSGi Services ontology (OWL-OS) we have defined is an extension of the OWL-S [OWL-S coalition, 2004] ontology to create a specialized vision for the smart home. As Fig. 17.1 shows, an OSGISERVICE is only provided by instances of the devices-in-home ontology, which embraces all the typical devices and appliances at home (heating, dishwasher, doors, blinds, etc) unifying the conceptualization of this field. Besides, we capture the peculiarities of OSGi services by defining both a specific OSGi Service Profile (“what the service does”) and a specific OSGi Service Grounding (“how to access it”). Finally, and regarding the SERVICEMODEL (“how the service works”), we do not adopt any restriction because the OWL-S one fits perfectly with the OSGi perspective, i.e. any OSGi service is described by a SERVICEMODEL.

The OSGiServiceProﬁle

The OWL-S SERVICEPROFILE draws together information about who provides the service, its functional description and additional features. The latter information is stored in two attributes: (i) SERVICECATEGORY refers to an entry in some ontology or taxonomy of services, and (ii) SERVICEPARAMETER is an expandable list of properties. Although the OSGISERVICEPROFILE inherits these attributes, we have defined a specific OSGISERVICECATEGORY and an OSGISERVICEPARAMETER (see Fig. 17.2). Regarding the OSGISERVICECATEGORY, OSGi services are classified according an ontology we have defined called operations-at-home, which represents the typical services that can be found in the smart home 212 Rebeca P. Díaz-Redondo, Ana Fernández-Vilas and Manuel Ramos-Cabrer

Fig. 17.1. The OWL-OS ontology (OWL-OSGi Services Ontology).

Fig. 17.2. Details of the OWL-OS ontology. 17 Context-aware Personalization for Services in the Smart Home 213

(see Fig. 17.1). Note that an OSGi Service Proﬁle should provide more than one categorical description taken from the operations-at-home ontology: for instance, the service “opening a window” can be both an airing service or a lighting service. On another hand, the OSGi service provider may specify additional information, to enrich the OSGi service selection process. This information is maintained in a list of properties of the type OSGISERVICEPARAMETER.

The OSGiServiceGrounding

In OWL-S the SERVICEGROUNDING deals with the concrete level of speciﬁcation. Therefore, we have deﬁned an OSGISERVICEGROUNDING (subclass of the SERVICEGROUNDING) which provides a suitable way for mapping the OSGISERVICEPROFILE to the functionality offered, through a Service Interface, by a bundle running in the OSGi framework.

17.4.2 The OWL-OS/OSGi Framework Apart from the ontological structured deﬁned by the OWL-OS ontology, OSGi bundles also need a way to register, to discover and to invoke these services.

Registering Semantic OSGi Services We propose the bundle developer specifies the semantic information about an OSGi service in the bundle’s manifest file by defining a set of new manifest headers: the OWL-OS headers. The new headers describe the model and profile correspondence of every single method in the Service Interfaces the bundle provides. This information is used by the Semantic OSGi Service Registry to create new individuals and classes when a new service is registered in the ontology. Note that, as Fig. 17.3 shows, OSGi bundles use the same registering methods.

Semantic Discovery of OSGi Services For the semantic discovery mechanism we have proposed to add the two following methods, which were inspired in the OSGi original ones: SEMGETSERVICEREFERENCE(STRING ONTOLOGYURI, STRING CATEGORY) • SEMGETSERVICEREFERENCES(STRING ONTOLOGYURI, STRING CATEGORY, STRING SEMANTIC- • FILTER) Instead of providing the name of the Service Interface, the requesting bundle uses the OWL-OS ontology to specify which kind of service is needed. It provides which ontology is being use to classify the required service (ONTOLOGYURI) and the name of the service category in that ontology (CATEGORY). For instance, Fig. 3(b) shows how a bundle can discover the COLORPRINTING service by asking for any service belonging to the PRINTCOLOR category defined in the operations-at-home ontology. The second method returns an array of Service Reference objects satisfying not only the search criterion but also the selection filter which has been specified in the parameter SEMANTICFILTER. For this parameter we have proposed to replace the syntactic filtering based on the LDAP language by a semantic one based on SWRL (Semantic Web Rule Language) [Horrocks et al., 2004]. SWRL, based OWL Web Ontology, is intended to be the rule language of the Semantic Web: allowing users to write rules to reason about OWL individuals and to infer new knowledge about those individuals.

Invoking Semantic OSGi Services

To invoke semantic services, we have added several methods to the SERVICEREFERENCE interface of the OSGi Framework Speciﬁcation. These get-set style methods (see GETMETHODNAME and GETINPUTPARAMETER in Fig. 3(b)) allows the requester bundle to discover the name of the public method and the parameters in order to construct the invoking primitive (SERVICEOBJ.COLORPRINTING(FILE) in the example). 214 Rebeca P. Díaz-Redondo, Ana Fernández-Vilas and Manuel Ramos-Cabrer

(a) OSGi services

(b) Semantic OSGi platform

Fig. 17.3. OSGi services vs. Semantic OSGi services. 17 Context-aware Personalization for Services in the Smart Home 215 17.5 Context-Aware Personalization for OWL-OS/OSGi

In order to provide a context-aware personalization solution to OSGi three main aspects have to be related in the smart home: services, users and contexts. Taking OWL-OS as our base service model, we define a model for preferences and contexts in order to provide a form of service selection and invocation which is based on the actual user and the external context: context-aware personalization. We sketch our solution in Fig. 17.4, where two context-aware personalization levels are constructed on top of the semantic framework OWL-OS/OSGi. These two levels corresponds with the two elements whose selection can be automatized according to the target user and its context: (i) the service inside a specific category (OSGISERVICEPROFILE entity in OWL-OS), for instance, the preferred service for playing music; and (ii) the form of invocation (parametrization) for a specific service (OSGISERVICEINVOCATION as introduced in the following paragraphs), for instance, the preferred volume when playing music.

17.5.1 Context Modeling

For the purposes of this chapter, we suppose that an external CXMS (ConteXt Management System) exists. We assume the vision by Zimmermann et al. [2005] of CXMS as a system which involves the construction, integration, and administration of context-aware behavior by considering the definition of relevant context parameters, the link between these parameters and information sources, their utilization for the targeted adaptive behavior, and the definition of that behavior. If the aim is defining a context-aware OWL-OS/OSGi framework, the connection between the service model OWL-OS and the external CXMS is established by means of the description in services’ profiles. As it was shown in Fig. 17.2, an OWL-S Profile describes, mainly, what organization provides the service, what functionality the service provides, and a host of features that specifies characteristics of the service. Regarding the functional description, apart from specifying the inputs and outputs, the profile describes the preconditions required by the service and the expected effects that result from its execution. We propose preconditions and effects range over some query language which is understandable by the CXMS. That is, a service profile specifies some rules on the context to be satisfied in order to the service be invokable but also some effects on the context that may result from the invocation of the service. To this aim, we use the following properties of the OWL-OS class OSGISERVICEPROFILE: HASPRECONDITION and HASEFFECT. 216 Rebeca P. Díaz-Redondo, Ana Fernández-Vilas and Manuel Ramos-Cabrer

Fig. 17.4. Using Semantic OSGi for personalizing OSGi services in a context-aware solution.

The communications between the OWL-OS Service Framework and the CXMS are naturally supported by the Framework as far as CXMS should be integrated a one more OSGi service. Apart from that, an OWL-based CXMS is preferable since a full ontological approach –for users, contexts, and services– allows more ambitious reasoning by exploiting cross-ontology relationships. This kind of solution for the CXMS is part of our future work as it is describe in Section 17.8. Integrating a CXMS, which can be queried and updated by preconditions and postconditions in the service proﬁle, turns the OWL-OS/OSGI Framework in a context-aware solution for the smart home. However a personalized context-aware solution requires establishing a preference model, also connected with the CXMS, which allows the framework to adapt the context-aware response of the home according to the characteristics of the user. This context-aware preference model is introduced in the following section.

17.5.2 Preference Modeling Given a service model, in our case OWL-OS, we define a general preference model which stores, for each relevant entity E in the service model, an index to reflect the level of preference of a user U. This index, referred to as Degree of Preference (DOP ), is defined for all the specific instances as well as for all the classes contained in the service model. This simple preference model would turn OWL-OS/OSGi in a personalized framework. However, given an entity E, the preference DOP of a user U (which can represent like and dislike, the best choice and the worst choice, etc) is possibly related to certain time, certain place, what the agent is doing– related to certain context in general. With this vision, we can personalize the system in a context-aware manner. Thus, the Degree of Preference is defined as the triple DOP (U, E, C) where C is the context in which the preference of U about an entity E is DOP . Clearly, the triple DOP can be defined without any context so that the preference is applicable in any context C or without any user U so that it is a general preference. In order to capture this conceptual preference model DOP (U, E, C), the same structure in the OWL-OS ontology is reused. That way of capturing the preference model allows us to reason about not only the user 17 Context-aware Personalization for Services in the Smart Home 217 preferences for specific entities but also their semantics, that is, inferring information from the relationships defined in the ontology. So the preferences in using services at home are captured by means of ontological preference models, that is a model which ranks or qualifies the elements of a specific ontology according to a specific user and to a specific context. The ontological preference model of a user U is a projection of the OWL-OS service model which is constructed while the user consumes services in the service model. This user perspective of the service model is augmented with two properties HASDOP and HASCONTEXT, which respectively represent an cumulative index of preference and a SWRL rule which represents the context. Both properties are defined on top on the ontology so that the pair (DOP, C) qualifies every class or instance in the ontology. The value of DOP for a specific instance on the ontological preference model is computed when the user uses the service by weighting it with the number of service instances in the same category. Although the details are out of the scope of this chapter, this computation entails taking into account overlapping and disjoint contexts related to the specific instance. From instance-scope DOP s, the ones corresponding to classes are obtained by recursively propagating DOPs up through the hierarchy defined by operations-at-home.

17.6 Some Forms of Personalization

If a preference model is introduced in the OWL-OS/OSGi framework to capture the habits of the inhabitants when using services at home, different aspects of the framework can be improved. A general preference model which qualifies every class (and instance) with a context-aware preference allows the personalization strategy to range over devices, providers, groundings, models and so on, and also services, of course. In this chapter we investigate the most common applications of this preference model —service and invocation personalization—, some other are discussed in the conclusions (Section 17.8). Firstly, service selection can be personalized by using the information in the preference model to establishes the most appealing service for the user among those which meet the discovery criteria ( OSGISERVICECATEGORY and OSGISERVICEPARAMETER). The base element for this kind of personalization is the service profile (OSGISERVICEPROFILE), which categorize the service according to a set of established services ontologies ( OSGISERVICECATEGORY) and, besides, offers a set of semantic characteristics of that service ( OSGISERVICEPARAMETER).Secondly, the parametrization in the execution of the service can be personalized as well by capturing the preferences of the user in the invocation of a specific service, typically the values for input parameters. Since there is no notion of service execution on OWL-S nor OWL-OS, we introduce one new class in OWL-OS, referred to as OSGISERVICEINVOCATION which allows recording preferences about invocation values for a specific service (see details of the definition below). In the following paragraph, we show how the ontological preference models are implemented on top of OWL-OS to resolve the personalization of two of the most relevant processes in services management at home: service selection and service invocation.

17.6.1 Context-Aware Personalization of Service Selection: OSGiServiceProfile In the context-aware personalized OWL-OS/OSGi Framework, an ontological preference model is maintained in order to improve the implementation of the main method in semantic discovery, SEMGETSERVICEREFER- ENCE(STRING ONTOLOGYURI, STRING CATEGORY). To be precise, the “best” service within the category is selected according to the actual user U and its context C. For that, the OSGISERVICEPROFILE entities which match the category2 are inspected in order to find: the one with greater HASDOP among those service profiles whose context requirements (HASCONTEXT property) hold in the actual context. In order to evaluate the context fulfillment, the CXMS is queried with the SWRL axiom stored in the property HASCONTEXT. Finally, an maintaining the idea in the original OSGi specification, an extended version of the method is also implemented, SEMGETSERVICEREFERENCES(STRING ONTOLOGYURI, STRING CATEGORY, STRING SEMANTICFILTER), which selects the “best” service (according to the user and his context) within a category which additionally meets a semantic filter.

2 Additionally, if a precondition is defined in the profile (property HASPRECONDITION), only the service profiles which math the context are inspected 218 Rebeca P. Díaz-Redondo, Ana Fernández-Vilas and Manuel Ramos-Cabrer

17.6.2 Context-Aware Personalization of Service Parametrization: OSGiServiceInvocation

One step beyond selecting the service in a personalized way is adapting also the invocation of that service according to the user habits and, therefore, according to the user and his context. As it is mentioned before in this chapter, there is no notion of service execution in OWL-S nor OWL-OS (as deﬁned in Section 17.4). The feature we are at the point of introducing captures the following behavior: when playing music (the service) John (the user) selects, during the day (context), 90% of the times (DOP ) low volumes; and, at night (context), 80% of the times (DOP ) high volumes. For this to be possible we deﬁne the class OSGISERVICEINVOCATION as it is shown in Fig. 17.4. The main properties in this class are the following:

HASINPUTVALUE and HASOUTPUTVALUE which are instances of BINDING OWL-S classes. A binding • object has two properties the name of the variable, TOVAR, and the specification of a value for which we suggest VALUEDATA. HASDOP and HASCONTEXT as every entity in OWL-OS. • In order to allow the implementation of the behavior above, we have added specific methods to the OSGi interface SERVICEREFERENCE: GETINPUTVALUE and GETOUTPUTVALUE. The former returns, for a specific parameter, the preferred value (greater HASDOP) among those OSGISERVICEINVOCATION entities in the preference model of the user U which hold their context requirements HASCONTEXT. The behavior of the latter is the same but referred to an output value. Despite the fact that the application of the preferred output value remains open, it can support, for instance, an automated alarm system which notifies in case of output values different from the preferred one.

17.7 A Sample Use Case

In this section we show the potential of the OWL-OS/OSGI Framework by introducing, in a descriptive way, a real use case related to the security systems in the smart home.

Scenario: John has signed a maintenance contract for his OSGi residential gateway. As part of this contract, a “human presence simulator” is deployed at John’s home. The functionality of this bundle is trying to dissuade potential burglars by simulating that someone is at home by turning on and off some appliances (lights, TV, blinds, etc).

Taking a realistic provider, surely, the same bundle for presence simulation would be deployed to all the subscribers’ home. This bundle would hardly be programmed according to the current OSGi specification; the interactions with lighting devices (lights, blinds, etc) or sounding devices (HI-FI system, TVs, etc) at each house can be slightly different, depending on which devices are at home and their sophistication. In any case, the human presence simulator bundle needs to interact properly with the controllers of these appliances, taking into account their peculiarities and acting consequently. Even if the bundle could be programmed, it hardly works properly at every single home. With the proposed OWL-OS/OSGi Framework, the bundle is able to obtain the service/s it needs and to know how to use them without having so much information in advance as follows. Service obtaining is based on selecting potentially adequate services according to the bundle’s requirements. Without a explicit notion of context, the bundle has to be programmed in a strict way, for instance, using the system clock to activate SOUNDING SERVICES from 8:00am to 8:00pm. With the semantic framework, the appropriate services can be discovered by SEMGETSERVICEREFERENCE using the category SOUNDING in the operations-at-home ontology. Any service in this category will be selected, to be precise, the service with low service ranking (established by the framework during the registration of services). Additionally, the version SEMGETSERVICEREFERENCES with semantic filter can be used to refine the discovery process. For instance, suppose that the provider has decided that (i) the more noise the sounding service makes and (ii) the less energy it consumes, the better qualification it receives. In this case a semantic filter can be used to establish a minimum SWRL rule for noisy level and energy consumption. Note that these parameters are supposed to be incorporated 17 Context-aware Personalization for Services in the Smart Home 219 in the ontology as a service parameter which specifies the name of the parameter SERVICEPARAMETERNAME and the external ontology which conceptualizes the parameter. For instance, the vacuum cleaner (80 dB), the Hi-fi system (70 dB), the TV (60 dB) and the washing machine (54 dB) are the top four according to the noise level; whereas the TV (0.065 KWh), the Hi-fi (0.07 KWh), the vacuum cleaner (0.67 KWh) and the washing machine (0.88 KWh) are the top four according to the energy consumption. Since the TV, the Hi-fi and the vacuum cleaner are the services on top of our ranking, they are activated. So, having applied explicit reasoning by means of behavior rules and datatype properties stored in the service ontology (energy consumption and noise level) a set of appropriate services can been selected. One selected, the form of invocation can be obtained by using the primitives for extracting the grounding information from the ontology. Introducing a CXMS which interacts with the framework improves the scenario, allowing an explicit notion of context, as follows. The service provider, role responsible of populating the service ontology, can establish a precondition for TV and Hi-fi system activation services which inhibits each other, similarly BLINDS DOWN as a precondition for LIGHTS ON. So, as the TV and the Hi-fi cannot be activated simultaneously, the bundle may decide not to use one of them, or starting them in turns, depending on how it has been programmed. Similarly, and replacing the system clock, a precondition can be introduced for the activating services of the kind ((8:00AM AND WORKING_DAY) OR (10:00AM AND NON-WORKING_DAY)). Clearly, all these context conditions has to pertain to the context model in the CXMS. Finally, the personalization possibilities of the framework allows that the bundle for presence simulation, the same for all the subscribers, produces different behaviors depending on the user and not only on the available service at home. So, the behavior would be different in two houses with both TV and Hi-fi system whenever the inhabitant’s habits in using these systems are different. In this way service selection depends on a category criterion, on the context, possibly on a semantic filter but also on the habits of the user at home. As well as an inhabitant uses the service at home, his ontological preference model is constructed by associating contexts and indexes of preference to instances and classes. For instance, activity at John’s house usually starts at 9:00am on working days (lights and radio are turn on), while on non-working days activity often starts at 11:30am (the vacuum cleaner starts working). Regarding sounding services, the activation of the radio (entity E) would have a high DOP (John, E, C) for a context C which corresponds to times over 9:00am on working days. Similarly for vacuum cleaner. When using SEMGETSERVICEREFERENCE at John’s house with the parameter SOUND- ING, the personalized OSGi framework selects, from the services in this category whose property HASCONTEX is evaluated to true by the CXMS, the one with greater HASDOP. So that, it selects the vacuum cleaner or the radio as sounding service depending on the day of the week. One step beyond, when the presence simulator activates the radio, the bundle can use the information in OSGISERVICEINVOCATION to set the volume to the John’s preferred level. To sum up, the knowledge progressively captured in the ontological preference model permits reproducing the everyday habits of the inhabitants when they are on holidays, providing an “intelligent” human presence simulator able to decide when and which services must be started and/or stopped. The logic of the bundle for simulating presence can be sophisticated more an more with a minimal extra complexity in programming it. For instance, a context intruder detection could be defined which, instead of reproducing the everyday habits, makes as much noise as possible by simultaneously activating all the SOUNDING devices (vacuum cleaner, the Hi-fi system and the washing machine) whatever context is currently at home.

17.8 Conclusions and Future Work

The primary reference architecture in the OSGi specification, although not considered normative, is based on a model where a operator manages a potentially large network of service platforms. It assumes that the service platforms are fully controlled by the operator and used to run services from many different services providers. In this scenario, we have shown that the actual service discovery mechanisms in OSGi is insufficient or, at least, only allows automation and interoperability in a restricted way. In the pursuit of a really open and interoperable residential gateway, we propose the semantic description and discovery of the services in the OSGi domain. In this regard, we have defined OWL-OS, a suitable sub-ontology inside OWL-S which allows making a simple semantic search of services based on a categorized structure. Despite we propose operations-at-home as the 220 Rebeca P. Díaz-Redondo, Ana Fernández-Vilas and Manuel Ramos-Cabrer primary structure to classify the OSGi services, OWL-OS allows an OSGi service to be semantically described according to different ontological structures. This ontological structures would be downloaded on demand from the service provider. Finally, note the Semantic OSGi Framework enhance the OSGi standard, without breaking it; i.e. any non-semantic bundle can work properly within this framework, although it is not able to take advantage of the semantic reasoning for service obtaining. Moreover a clear benefit of the new Semantic OSGi platform is the possibility of supporting the automation of OSGi services composition [Díaz-Redondo et al., 2007]. This automation opens the platform to ambitious applications which relay on the idea that different services at home usually form a pool that is committed to various home activities, such as energy control, security, health care, etc. Although a semantic classification and discovery of OSGi services is a step forward in itself, semantic filters improves the semantic searching mechanisms by supporting more sophisticated queries. The objective is to allow a bundle to express more characteristics of the desired service, i.e. not only the category but also some requirements for the service parameters (quality, response time, availability, reliability, etc). In this chapter we have proposed SWRL as a language for semantic filtering. Related to this proposal, we have identified another improvement we are working on: introducing semantic Service Listeners as an analogy to the current Service Listeners. These new elements would be notified when a new service of a specific category is registered in the framework, or even when a new service of this category is registered with a concrete value for a service parameter. Also in the field of service selection, it would be possible to supplement the Semantic Registry with different specialized software agents which takes into account other factors (ambient intelligence) to automate the service selection. In this chapter we have investigated the potential of combining a personalization agent and a context management system. On its own, the personalization agent contribute to decisions on service selection by using a preference model storing historic information about preferred devices and services. On the other hand, the context management system allows the OWL-OS/OSGi Framework to be aware of the particular context (time, location, mood, etc) and to react by discovering services which meet that context. Beyond their independent behavior, connecting the personalization and context-aware agent across an ontological base, OWL-OS in our case, allow the OWL-OS framework to support scenarios as the one included in this chapter. In general, scenarios where the selection of the service depends not only on who uses the service but when, where or even why uses the service. Since we have defined a general preference model which is applicable to any entity in the service model, or even to any other service, service selection is only the first step in a complete system. Despite the fact that this chapter only discusses two forms of context-aware personalization in the smart home (service selection and parametrization), some other forms are possible with the general preference model we propose. For instance, provider selection can be personalized by capturing the preferences of the user about the provider of a specific service. The base element for this kind of personalization is the service grounding, the one which specifies implementation details such as communication protocol, message formats, etc. The lines of future research can be attached to the two main intelligent systems in the proposal: the context and the personalization agent. In this work we assume an external CXMS which is integrated as an OSGi Service. There are no technical restrictions on this CXMS provided that the context requirements in OWL-OS are written in the CXMS query language. However, and as mentioned before, we are working on an OWL-based CXMS which incorporates an OWL ontology for modeling context [Heckmann et al., 2005b] and SWRL, the rule language of the Semantic Web, as query language. A SWRL rule axiom consists of an antecedent and a consequent, each of which consists of a possibly empty set of atoms. Informally, a rule may be read as meaning that if the antecedent holds (is true), then the consequent must also hold. An empty antecedent is treated as trivially holding (true), and an empty consequent is treated as trivially not holding (false). Also informally, the property HASPRECONDITION of OSGISERVICEPROFILE will be written as an empty consequent rule; and the property HASEFFECT as a non-empty consequent rule (empty antecedent or no). Regarding the personalization strategy, supporting multi-user preference model is a real requirement in scenarios for the smart home. In fact, the humam presence simulator in this chapter is a multi-user system whose aim is simulating the behaviors of all the people who lives in the smart home. The solution to this problem is far from being trivial. An approach which simply merge the habits of all the inhabitants obviates the relationships and even dependencies among users at home. Even if they are not relevant to select the appropriate services, in a multi-user environment the personalization system has to cope with conflict resolution and distinguishing 17 Context-aware Personalization for Services in the Smart Home 221 among the user preferences. On the other hand, the personalization of the selection of the service, if valuable, is only a first step to full ambient intelligence.

References

V. Akman and M. Surav. The use of situation theory in context modeling. Computational Intelligence, 13(3): 427–438, 1997. J. Bauer. Identification and Modeling of Contexts for Different Information Scenarios in Air Traffic. PhD thesis, TU Berlin, 2003. T. Berners-Lee, J. Hendler, and O. Lassila. The Semantic Web. Scientific American, pages 35–43, May 2001. H. Chen, F. Perich, T. Finin, and A. Joshi. SOUPA: Standard ontology for ubiquitous and pervasive applications. In International Conference on Mobile and Ubiquitous Systems: Networking and Services, Boston, Massachusetts (USA), 2004. S. Chenishkian. Building smart services for smart home. In 4th IEEE International Workshop on Network Appliances, Gaithersburg, Maryland (USA), 2002. K. Cheverst, K. Mitchell, and N. Davies. Design of an object model for a context sensitive tourist guide. Computers and Graphics, 23(6):883–891, 1999. E. Chtcherbina and M. Franz. Peer-to-peer coordination framework (p2pc): Enabler of mobile ad-hoc networking for medicine, business, and entertainment. In International Conference on Advances in Infrastructure for Electronic Business, Education, Science, Medicine, and Mobile Technologies on the Internet, 2003. A. K. Dey. Understanding and using context. Personal and Ubiquitous Computing, Special issue on Situated Interaction and Ubiquitous Computing, 5(1):4–7, 2001. P. Dobrev, D. Famolari, C. Kurzke, and B. A. Miller. Device and service discovery in home networks with OSGi. IEEE Communications Magazine, 40(8):86–92, 2002. R. P. Díaz-Redondo, A. Fernández-Vilas, M. Ramos-Cabrer, J. J. Pazos-Arias, and M. Rey-López. Enhancing residential gateways: OSGi services composition. IEEE Transactions on Consumer Electronics, 53(1):87– 95, 2007. R. P. Díaz-Redondo, A. Fernández-Vilas, M. Ramos-Cabrer, J. J. Pazos-Arias, J. García-Duque, and A. Gil- Solla. Enhancing residential gateways: a semantic OSGi platform. IEEE Intelligent Systems Journal, 23, 2008. P. Gray and D. Salber. Modelling and using sensed context information in the design of interactive applications. In 8th IFIP International Conference on Engineering for Human-Computer Interaction, Toronto, Canada, 2001. T. R. Gruber. A translation approach to portable ontology specifications. Knowledge Acquisition, 5(2):199–220, 1993. T. Gu, H. K. Pung, and D. Q. Zhang. Toward an OSGi-based infrastructure for context-aware applications. IEEE Pervasive Computing, 3(4):66–74, 2004. C. Guidini and F. Giunchiglia. Local models semantics, or contextual reasoning locality compatibility. Artificial Intelligence, 127(2):221–259, 2001. M. Hatala, R. Wakkary, and L. Kalantari. Rules and ontologies in support of real-time ubiquitous application. Journal of Web Semantics: Science, Services and Agents on the World Wide Web, 3(1):5–22, 2005. D. Heckmann. Integrating privacy aspects into ubiquitous computing: A basic user interface for personalization. In Artificial Intelligence in Mobile System Workshop, in conjunction with Ubicomp, Seattle, Washington (USA), 2003. D. Heckmann, T. Schwartz, B. Brandherm, M. Schmitz, and M. von Wilamowitz-Moellendorff. GUMO: The general user model ontology. In 10th International Conference on User Modeling, Edinburgh, United King- dom, 2005a. D. Heckmann, T. Schwartz, B. Brandherm, M. Schmitz, and M. von Wilamowitz-Moellendorff. Gumo – the general user model ontology. In 10th International Conference on User Modeling, Edinburgh, United King- dom, 2005b. 222 Rebeca P. Díaz-Redondo, Ana Fernández-Vilas and Manuel Ramos-Cabrer

K. Henricksen, J. Indulska, and A. Rakotonirainy. Generating context management infrastructure from high- level context models. In Industrial Track of the 4th International Conference on Mobile Data Management, Melbourne, Australia, 2003. I. Horrocks, P. F. Patel-Schneider, H. Boley, S. Tabet, B. Grosof, and M. Dean. SWRL: A semantic web rule language combining OWL and RuleML. W3c member submission, World Wide Web consortium, 2004. URL http://www.w3.org/Submission/SWRL. T. Howes. A string representation of LDAP search filters. Technical Report RFC 1960, University of Michigan, 1996. J. Indulska, R. Robinson, A. Rakotonirainy, and K. Henricksen. Experiences in using CC/PP in context-aware systems. In 4th International Conference on Mobile Data Management, Melbourne, Australia, 2003. R. Lara, D. Roman, A. Polleres, and D. Fensel. A conceptual comparison of WSMO and OWL-S. In European Conference on Web Services, Erfurt, Germany, 2004. C. Lee, D. Nordstedt, and S. Helal. Enabling smart spaces with OSGi. IEEE Pervasive Computing, 2(3):89–94, 2003. J. MacCarthy and Buvac. Formalizing context (expanded notes). In Working Papers of the AAAI Fall Sympo- sium on Context in Knowledge Representation and Natural Language, Cambridge, Massachusetts (USA), 1997. Y. S. Oh, H. S. Yoon, and W. T. Woo. Simulating context-aware systems based on personal devices. In International Symposium on Ubiquitous VR, Yanji, China, 2006. Open Service Gateway initiative. Osgi service platform, core specification r4. http://www.osgi.org, 2005. P. Ötztürk and A. Aamodt. Towards a model of context for case-based diagnostic problem solving. In Interdis- ciplinary Conference on Modeling and Using Context, Rio de Janeiro, Brazil, 1997. OWL-S coalition. OWL-S: semantic markup for Web Services r1.1. http://www.daml.org/services/owl-s/1.1/, 2004. D. Roman, U. Keller, H. Lausen, J. de Bruijn, R. Lara, M. Stollberg, A. Polleres, C. Feier, C. Bussler, and D. Fensel. Web service modeling ontology. Applied Ontology, 1(1):77–106, 2005. M. Samulowitz, F. Michahelles, and C. Linnhoff-Popien. CAPEUS: An architecture for context-aware selection and execution of services. In 3rd International Working Conference on Distributed Applications and Interoperable Systems, Kraków, Poland, 2001. B. N. Schilit, N. L. Adams, and R. Want. Context-aware computing applications. In 1st IEEE Workshop on Mobile Computing Systems and Applications, Santa Cruz, California (USA), 1994. A. Schmidt, M. Beigl, and H. W. Gellersen. There is more to context than location. Computers and Graphics, 23(6):893–901, 1999. T. Strang. Service Interoperability in Ubiquitous Computing Environments. PhD thesis, Ludwig-Maximilians- University, 2003. T. Strang and C. Linnhoff-Popien. A context modeling survey. In Workshop on Advanced Context Modelling, Reasoning and Management, in conjunction with UbiComp, Tokyo, Japan, 2004. T. Strang, C. Linnhoff-Popien, and K. Frank. Applications of a context ontology language. In International Conference on Software, Telecommunications and Computer Networks, Split, Croatia, 2003. X. H. Wang, D. Q. Zhang, T. Gu, and H. K. Pung. Ontology-based context modeling and reasoning using OWL. In Workshops of the 2nd IEEE Conference on Pervasive Computing and Communications, Orlando, Florida (USA), 2004. WAP forum. User agent profile UAProf. http://www.wapforum.org, 2007. World Wide Web consortium. Composite Capability/Preferences Profile (CC/PP) 2.0, 2007. URL http://www.w3.org/Mobile/CCPP. A. Zimmermann, M. Specht, and A. Lorenz. Personalization and context management. User Modeling and User-Adapted Interaction, 15(3-4):275–302, 2005. 18 New Perspectives to Lifecycle Management of Context-Aware and Personalised Services

Bharat Bhushan, Stephan Steglich, Christian Räck and Bernd Mrohs

Fraunhofer FOKUS (SE CC) Kaiserin-Augusta-Allee 31, D-10589 Berlin, Germany {bharat.bhushan, stephan.steglich, christian.raeck, bernd.mrohs}@fokus.fraunhofer.de

Summary. Context-aware and personalised services are required not only to pass the stringent tests of popular demand for their unique feature, but also easily be deployable to and removable from multiple service platforms, bundled in a large package, and also have their salient featured be personalized. Crux of matter, here, is service lifecycle management. The way service lifecycle management is done is changing slowly in comparison to development of context- aware services, which is advancing rapidly, thanks to numerous R&D efforts in industry and academia. Con- ventionally, service providers and platform operators take an existing lifecycle regime and tools with some modiﬁcation and use them on context-aware service. This leaves out a whole array of mechanisms that must be employed to cater for context-aware services of a more agile and personalised nature. Furthermore, rapid roll-out, packaging, and personalisation of services are increasingly becoming requirement, and will be acute for network operator and service provider who are on their toes due to increased competition. This chapter addresses the fresh perspective in lifecycle management of personalised and context-aware services, which has become different ball game in its own right.

18.1 Introduction

When one investigates context-aware and personalised services and compares them with conventional information services, an important aspect strikes out, which is, the context aware applications are time-critical, can be personalized to a great deal, and context can change quickly. All this essentially means that applications using the context information must be ready for changes. What all this means for the service providers (SPs) is that they must manage lifecycle of context information and services far more rapidly than they would otherwise do in case of conventional service time and must do it with a good degree of automation. This chapter presents interfaces and requirements that emerged out of a service management infrastructure developed within the auspices of an IST project called MobiLife. The interfaces were developed as a means to augment the MobiLife service infrastructure with lifecycle management features. What drove the research and development into context-aware services and their lifecycle management was straightforward: to investigate and provide the SPs or mobile network operators (MNOs) with mechanism that they can use to manage the lifecycles of mobile and context aware services through MobiLife service infrastructure, thereby quickening the pace of service roll out and withdrawal. The three main goals of this work were the following: 1. To provide for the intelligent and personalized multimodal service while developing the management interfaces. 2. To address not only the service development and deployment phases of lifecycle management but also business modelling and deﬁnition of services. 3. To make sure that MobiLife infrastructure conforms to industry guidelines. 224 Bharat Bhushan, Stephan Steglich, Christian Räck and Bernd Mrohs

These goals were met by centering the interface development and requirement speciﬁcation on the work that MobiLife project has already done so far [Galli et al., 2005, Mrohs et al., 2005, Räisänen et al., 2005] and deriving requirements. The requirements were analyzed and adjusted against lifecycle steps proposed by OMA [Open Mobile Alliance, 2005], which is the main source of information to understand the general lifecycle management steps SPs need to bear in mind. MobiLife Deliverable [Galli et al., 2005] is the source the business model and stakeholders who will be the main users of MobiLife service infrastructure and the management interfaces that come with it. Deliverable [Mrohs et al., 2005] is the speciﬁcation of MobiLife infrastructure. Guidebook eTOM [International Telecommunication Union, 2004] serves as a blueprint of business and operations processes required by SPs and MNOs. The structure of the rest of the chapter is as follows. Section 18.2 presents an overview of some of the effort in industry and academia that have born upon work presented in this chapter. Section 18.3 presents a model of stakeholders involved in providing personalized context-aware services. Section 18.4 presents the requirements for service lifecycle management arising from various stakeholders. Section 18.5 presents the interfaces and establishes a relationship with the requirements. Section 18.6 presents some conclusions.

18.2 Related Work

These are some noteworthy efforts that discuss personalised and context-aware services development as well as services lifecycle management in networked and distributed environment. They also have inspired part of the work presented in chapter. This section lists these initiatives and provides a summary of the ideas that are central to the research efforts. The specifications produced by the Open Services Gateway initiative (OSGi) define a standardised, component-oriented, computing environment for networked services. OSGi specifications also include support for installing, starting, stopping, updating and uninstalling software component in the networked environment [Open Service Gateway initiative, 2005]. The specifications have been adopted for a number of application areas, including those that are also relevant to personalised mobile services [Zhang et al., 2003, Cervantes and Hall, 2004]. The Location-Aware Lifecycle Environment (LocALE) framework described in [López de Ipina and Lo, 2001] provides a simple management interface for controlling the lifecycle of CORBA distributed objects providing a location-aware service and residing in ubiquitous computing networks. Controlling here means remote construction, movement, removal and recovery. A similarity can be seen between LocALE and our work, as both concern location-aware service provided in a distributed environment. One of the fundamental goals of work presented in [Dey, 2001] and [Dey et al., 2001] is to provide an operational understanding of context from which concepts useful for context-aware applications development can be derived. It introduces a conceptual framework to assist in the design of context-aware applications. Both aspects are highly relevant for understanding what context-aware applications and services require for their lifecycle management. The Information Technology Infrastructure Library (ITIL)1 is a set of guidelines for different aspects of best-practice data centre management. The guidelines also include best practice for service delivery and ICT infrastructure management, including network services management, operations management, and systems management. Considering the fact that context-aware services are essentially information services that will be delivered on telecom networks, ITIL guidelines are highly relevant for lifecycle management interfaces presented in this chapter.

18.3 Stakeholders’ Model: Personalising and Packaging

This section presents the key stakeholders that form a supply chain to deliver a context-aware service to the end-user and are also interested in service lifecycle management. We explain how the stakeholders will become

1 http://www.ogc.gov.uk/index.asp?id=2261 18 Lifecycle Management of Context-Aware and Personalised Services 225 involved in the lifecycle management of services. These aspects were identiﬁed by MobiLife project too and are comparable to the OMA business model and many other mobile industry business models in use today. Figure 18.1 illustrates the business model. The main artifact used to represent the stakeholders is called business worker. A business worker is a class that represents an abstraction of a human that acts within an organization unit, depicted as a package symbol.

Fig. 18.1. Model of stakeholders involved in service lifecycle management.

End-user is a part of Customer Domain and most interested in the context-aware and personalization features of the services that they use. The role of end-user goes beyond usage and consumption the Internet as an interactive platform actually or potentially provides several opportunities to influence or directly participate in the product or service design or delivery. The main role of the end-user will be to provide valuable feedback to improve the personalization features and performance of MobiLife services. Mobile network operators focus on developing and maintaining the physical network. They can also adopt the role of SPs and provide value-added service (e.g. voicemail or MMS). MNOs will find that MobiLife service infrastructure is geared towards their business objectives and supports management of innovative context-aware services. Hence they will be interested in lifecycle aspects of MobiLife service infrastructure. Service providers acquire and serve customers by packaging services as well as providing their own additional services in cooperation with MNOs. Above anything, the SPs’ main priority is to launch new service packages quickly and efficiently, fine tune them, and withdraw them whenever needed. Thus SPs will expect the MobiLife service infrastructure support the requirement of service lifecycle. The platform provider (PP) develops the commercial-of-the-shelf management components, which are bought and used by MNOs or SPs. Therefore, it will be an overriding requirement on platform provider to support the service lifecycle by implementing necessary measures or mechanisms in their platform. Component/application developer is a part of SP and PP domains and defines interfaces for functions such as user profile storage, authentication, privacy control, etc. These interfaces can be defined and invoked using internationally adopted standards. This reduces the need for service integration when a new service enabler is added to an existing service environment. Standards Developer develops standards for service management functions. These standards are used mainly by PP and MNOs roles to help ensure that their respective products are interoperable and conform to the best practices of industry. The Standards Developer will act as a mediator between the Component De- veloper and standardization bodies and will keep the Developers abreast with the service lifecycle best practice, while providing standardization bodies valuable input. 226 Bharat Bhushan, Stephan Steglich, Christian Räck and Bernd Mrohs 18.4 Key Requirements for Lifecycle Management

This section presents the general service lifecycle steps and requirements that they impose on various stakeholders. The use case model illustrated in Fig. 18.1 shows the relationship between stakeholders and lifecycle management steps and then further decomposes those general steps into concrete use cases. Step 1: Idea stimulation. This is the process of stimulating the creation of the concepts for an applica- • tion. It also involves analysis of market needs, etc, and identification of opportunities for new services. Requirements for this step are: – For creation of the concepts for an application, brainstorming of ideas SHOULD be carried out among parties interested in the concept and application area MUST be identified. – Analysis of market needs MUST be performed, main stakeholders MUST be defined, and result of market analysis MUST be reported to all stakeholders. Step 2: Service planning and definition. Having analyzed market, needs, ... opportunities for new services • are identified in this step. – A commercial feasibility study SHOULD be accomplished and study results MUST be reported. – All stakeholders involved in planning the development of service and defining it MUST give formal endorsement for creation of new services. – Should a service developer want to use the multimodal capabilities of the MobiLife service infrastructure, he/she SHOULD plan in advance which modalities the service should take into account. Step 3: Service development. This is the process of implementing andtesting the applications. Speci- • fications describing the requirements of the new service, design, implementation and tests are normally included in the development of the building components of the new service. Key requirements are listed below: – Requirements specification and design specification for the service components building the end-user services and for the infrastructure components managing the end-user services MUST be produced. – Configuration of environment running and testing services MUST be done and specified using the vocabulary that is agreed by all partners developing infrastructure components. – Versioning mechanism must be applied to all software and hardware component supporting services management. Rules for forward and back ward compatibility must be specified. Step 4: Service deployment. This refers to the process of deploying the service in the SP’s environment. It • includes every step to get the service up and running and ready to be subscribed by end-users. Requirements for this step are as follows: – The dependency of end-user services on execution environment MUST be declared and documented. – All the functions (e.g. messaging, context source, location capability), and components that will be utilized for a complete and successful provisioning of end-user services MUST be documented. – All the interfaces used in management of service operations such as context gathering, personalization, performance monitoring, trouble reporting MUST be registered with MobiLife service infrastructure. – Test routine MUST be devised and executed before making the end-user service offers public. – All the key properties of services that end-user must know for subscribing to and using the service SHOULD be made public. Step 5: Service packaging. This is the process of offering the service to the customer. Commercial pack- • ages are defined, aiming at concrete user segments, with concrete service features and billing conditions. Main requirements for service packaging are: – End-user MUST be notified (e.g. utilizing user preferences) when a service becomes available. – When accessing new services, the management infrastructure MUST automatically detect additional services and components and provide support for the ability for the automatic subscription for these services. – The management infrastructure SHOULD support SPs to form a service or content supply chain from many SPs and provide a service package to end-users. Step 6: Service monitoring and maintenance. This involves understanding how well a service is perform- • ing technically and commercially. Analysis on data gathered from the service is done to ascertain trends in service growth and service-level agreement (SLA) conformance. Some of the main requirements for this step are given below: 18 Lifecycle Management of Context-Aware and Personalised Services 227

Fig. 18.2. Lifecycle management: Use case model. 228 Bharat Bhushan, Stephan Steglich, Christian Räck and Bernd Mrohs

– Infrastructure components MUST be able to generate anonymous user statistics in a flexible way. – Service infrastructure components MUST be able to generate logs. End-user related logs MUST be protected by privacy rules agreed between the End-user and SPs. – OSS (Operations Support System) and BSS (Business Support System) SHOULD be able receive notification from the service of its status (e.g. available, withdrawn, etc) automatically and in real-time. – A service infrastructure component MUST assist SPs or MNOs in alerting/notifying of change in its status if the changed status has security implications, such as authentication or policy enforcement. – Management infrastructure components MUST monitor usage and load and help SPs and MNOs in detecting problematic behavior and predicting problems. Diagnostic mechanisms MUST be defined at component level and service/application level in order to detect out of order or malfunctioning components. Step 7: Service evolution and withdrawal. The end of a service may be its evolution or termination. If a • service offer must be stopped, subscriptions are properly dealt with. Requirements for this step are: – A process for graceful withdrawal of the service MUST be defined and the dependency between services being withdrawn SHOULD be charted. – End-users subscribed to the service MUST be informed of service withdrawal (certain number of weeks or months in advance) in accordance with SLA. – End-users’ feedback SHOULD be mapped onto service features and provided to teams defining and developing service.

Out of the seven steps mentioned above, steps 1-2 involve conceptual and business aspects. They do not directly relate to development. Therefore only requirements for them are analyzed and management interfaces are not deﬁned. Interfaces for the rest of steps are presented in the following section.

18.5 Associating Requirements with Interfaces Provided By Service Infrastructure Components

Based on Fig. 18.3, this section presents the management interfaces, which meet the requirements for steps 3-7 given above. The “uses” relationship is used between the MobiLife Service lifecycle Management package, which is a logical collection of interfaces, and the components (OMF, PF, ...) providing those interfaces: UIAF masks the peculiarities of user interfaces used by various kinds of devices used to access MobiLife • and provides a device-independent description of user interfaces that can be adapted to a device’s user interface. OMF manages the operations of MobiLife application services and resources and maintains optimum per- • formance according to the Service Level Agreement. PF supports improved service usage through adaptation of mobile services and applications according to • personal and group needs and interests. GAF provides means for the management of group lifecycle aspects, group state information provisioning, • and automatic creation and deletion of groups. PTF maintains a trust relationship between two end-users, or MobiLife infrastructure and a user. One of • the main functions of PTF is to manage access of personal data. CAF focuses on handling aspects related to raw, interpreted, and aggregated context information related to • individual users and groups of users.

Interfaces that meet the requirements for services application components development

Group Awareness Test Agent provided by GAF makes use of a user agents intended for interfaces tests, • accelerates the service development for groups, and make the deployment process easier. Multimedia Component Extension provided by UIAF allows independent service developers to integrate • into and extend the multimodal user interface behavior. 18 Lifecycle Management of Context-Aware and Personalised Services 229

Fig. 18.3. Service lifecycle management interfaces. 230 Bharat Bhushan, Stephan Steglich, Christian Räck and Bernd Mrohs

Context Use Inspection provided by CAF assists developer with inspecting context aware applications and • ensuring that correct context is used by a service Policy Setup provided by PTF uses existing Policy Setup interface and Trust Engine (both part of PTF) for • the provisioning of policies at the system startup

Interfaces that meet the requirements for application service deployment User Info Registration provided by PF makes use of create, get, and modify operations of PF and support • the SPs’ with registration of information such as user identity, subscription, accounting, etc. Multimedia Content Adapter provided by UIAF enables new service components to adapt to MobiLife • service infrastructure at runtime. Service Registration provided by CAF Assists SPs with registration and discovery, in order to offer their • services to service consumers. Security Info Alarm provided by PTF allows SPs to monitor security information and raise alarms when • the information becomes unavailable. Policy-Role Equalizer provided by PTF enables end-users to match the role they adopt (e.g. employee, • private person) with the policy relevant to the adopted role.

Interfaces that meet the requirements of application service integration and packaging Service Quality Monitor provided by OMF obtains service quality information and helps SPs in developing • service packages depending on varying service quality. Usage Pattern Developer provided by PF makes use of “recommender” function of PF to learn about end- • user’s behavior and assists the SP in packaging services and targeting them to a particular group segment. Group Offer Adapter/Provider provided by GAF monitors the utilization of group templates and the users’ • feedback about the group application usage and provides input data to SPs to build an offer for speciﬁc group services.

Interfaces that meet the requirements of service monitoring and maintenance Performance Data Collection provided by OMF allows SPs to monitor, analyze, control and report perfor- • mance. Data gathered from this interface can be used to understand how well a service is performing. Service Performance Analyser provided by PF makes use of information on users or a group, and person- • alization data, to assist the SP with service performance analysis and to see where improvements can be made. User Group Feedback Analyser provided by GAF makes use of feedbacks from users group to analyze • whether the service is satisfactory to a particular group of users. Multimedia Usage Monitoring provided by UIAF enables SPs to monitor and obtain information on usage • of device and type of multimedia content. Context Monitoring provided by CAF provides functions to gather and monitor context, which is vital • information in ﬁnding out how a service performs in given context. Security Policy Switch provided by PTF enables SPs in choosing the level of security monitoring or modi- • ﬁcation that is appropriate to a particular role.

Interfaces that meet the requirements of application service removal support Service Withdrawal provided by OMF withdraws services and resources or hibernate them when service • instance has been retired. Usage Preference Collector provided by PF allows SPs to collect end-user’s feedback from MobiLife • Service and Application. Group Pattern Developer with provided by GAF makes use of information on user groups’ satisfaction and • behavior in order and assists SPs in extending existing group templates or in building new group templates. Multimedia Usage Prediction provided by UIAF obtains information on multimedia contents usage by • end-users and analyses it to ascertain the usefulness of MobiLife components in dealing with modality of services. 18 Lifecycle Management of Context-Aware and Personalised Services 231 18.6 Conclusion

This chapter presents a set of lifecycle management requirements and interfaces for context-aware services and illustrates how the interfaces meet the requirements. The interfaces and requirements are part of MobiL- ife service infrastructure. The rationale for this work was to analyze requirements, stakeholders’ models, and guidelines proposed by the industry and to provide practicable lifecycle interfaces. The main conclusion is that the interfaces meet the requirements arising from the adding features such as context-awareness and personalization to end-user services. Equally importantly, interfaces are requirements in line with the best practices adopted by mobile network industry consortia such as OMA and TMForum.

References

H. Cervantes and R. S. Hall. Autonomous adaptation to dynamic availability using a service-oriented component model. In 26th International Conference on Software Engineering, Edinburgh, United Kingdom, 2004. A. Dey. Understanding and using context. Personal and Ubiquitous Computing Journal, 5(1):4–7, 2001. A. K. Dey, G. D. Abowd, and D. Salber. Conceptual framework and a toolkit for supporting the rapid prototyp- ing of context-aware-applications. Human-Computer Interaction, 16(2-4):97–166, 2001. L. Galli, T. Haaker, O. Immonen, B. Kijl, U. Killström, O. Pitkänen, P. J. Saarinen, and H. Virola. Initial marketplace dynamics (incl. business models) analysis. EU/IST Project MobiLife Deliverable D7, MobiLife, March 2005. International Telecommunication Union. Enhanced telecom operations map (eTOM): The business process framework. ITU-T recommendation M.3050.1, 2004. D. López de Ipina and S.-L. Lo. LocALE: a location-aware lifecycle environment for ubiquitous computing. In 15th International Conference on Information Networking, Beppu City, Japan, 2001. B. Mrohs, C. Räck, S. Steglich, S. Arbanowski, R. Vaidya, A. Domene, O. Sawade, G. D’Onofrio, T. Toni- atti, W. Kellerer, A. Mamelli, F. Bataille, M. Boussard, A. Tarlano, C. del Rosso, P. Hölttä, A. Andersin, O. Karasti ad A. Andreetto, V. Räisänen, R. Haensel, O. Coutand, J. Hjelm, S. Holtmanns, and S. Cam- padello. Initial speciﬁcation of the infrastructure for MobiLife services and applications. EU/IST Project MobiLife Deliverable D35, MobiLife, June 2005. Open Mobile Alliance. OMA Service Provider Environment Requirements, 2005. URL http://www.openmobilealliance.org/release_program/rd.html. Open Service Gateway initiative. Osgi service platform, core speciﬁcation r4. http://www.osgi.org, 2005. V. Räisänen, O. Karasti, S. Steglich, B. Mrohs, C. Räck, C. del Rosso, T. Saridakis, W. Kellerer, A. Tarlano, F. Bataille, A. Mamelli, M. Boussard, A. Andreetto, P. Hölttä, G. D’Onofrio, P. Floreen, and M. Przybilski. Basic reference model for service provisioning and general guidelines. EU/IST Project MobiLife Deliverable D34, MobiLife, January 2005. D. Zhang, H. Lee, X. Ni, and S. Zheng. Open service residential gateway for smart homes. In IEEE Interna- tional Symposium on Consumer Electronics, Sydney, Australia, 2003.

19 The Role of Service Discovery in Context-Aware, Pervasive Computing Applications

Carlos García-Rubio and Celeste Campo

Departamento de Ingeniería Telemática Universidad Carlos III de Madrid {cgr,celeste}@it.uc3m.es

Summary. Context-awareness is a term used in computer science that refers to the ability for computer applications to adjust to the environment. This concept is very important in pervasive computing, with many mobile, networked processing devices surrounding us almost everywhere. Here, applications need to continuously monitor the other devices in the environment, what kind of service they offer, and how to interact with them, not just to adjust the application to the environment, but also to personalize the environment according to the user preferences. Service discovery protocols play a central role in this continuous monitoring of what there is in the environment and how it can be used. A good service discovery protocol for pervasive computing must be fast; it must discover all the services in the environment; it must work efﬁciently in wireless environments, both in ad-hoc and in infrastructure modes; and it must preserve the energy of battery powered devices. In this chapter we detail these requirements and we review the solutions proposed so far. A number of protocols have been proposed in Internet, and others tied to a high-level application technology. More recently, other service discovery protocols speciﬁcally designed for ad hoc networks have been proposed, some tied to a particular subnetwork technology, others broach service discovery and routing together and others at an application level. We will show how well these solutions adapt to pervasive computing, and we will present some improvements to overcome the limitations found in today service discovery protocols.

19.1 Introduction

Context-awareness is a term used in computer science that refers to the ability for computer applications to adjust to the environment. This concept is very important in the personalization of pervasive computing [Satya- narayanan, 2001], where many mobile, networked processing devices surround us almost everywhere. In pervasive computing changes are frequent, as other users with their devices join or leave the environment or as we move away to other place. Applications need to continuously monitor the other devices, what kind of service they offer, and how to interact with them, not just to adjust the application to the environment, but also to personalize the environment according to the user preferences. For example, in a domotic home, heating controls could interact with personal biometric sensors in our clothes, so that temperature conditions could be personalized, adjusted to our preferences continuously and imperceptibly. Two elements play a central role in this personalization of pervasive computing environments: service discovery protocols and user profiles. Service discovery protocols allow applications to discover what devices are available in the environment, what services they offer, their characterstics, and how they can control them. User profiles store the configuration settings, preferences, and other data associated with an individual user or with a group of users. Applications will use profile information to select among services discovered, and to configure the selected services according to the user. There are some initiatives related with the definition of user profiles, both for pervasive environments and for other applications, such as 3GPP GUP (Generic User Profile), RDF (Resource Description Format), 234 Carlos García-Rubio and Celeste Campo

CC/PP (Composite Capabilities / Preferences Profile), PIDF (Presence Information Data Format), vCard, P3P (Platform for Privacy Preferences), etc. In this chapter we will concentrate not in user profiles, but in the other part of the problem: the discovery of the services available in the environment that we can configure and personalize. A good service discovery protocol for pervasive computing must be fast; it must discover all the services in the environment; it must work efficiently in wireless environments, both in ad-hoc and in infrastructure modes; and it must preserve the energy of battery powered devices. In the following sections we detail these requirements and we review the solutions proposed so far. We show how well these solutions adapt to pervasive computing, and we present some improvements to overcome the limitations found in today service discovery protocols.

19.2 State of the Art in Service Discovery

Service discovery is not a new problem. There are several solutions proposed for fixed networks, with different levels of acceptance. These solutions allow applications to be aware of what the context offers to the user (e.g. printers, e-mail servers, etc.). In this section we first review which approaches are possible to solve this problem. Then, we present in more detail the protocols defined within the Internet. Finally, we resume the solutions that have been proposed in other forums.

19.2.1 Approaches

Service discovery can be solved following a centralised or a distributed approach: In the centralised approach, a new element, the directory, is introduced. Devices register their services in • the directory and other devices send their requests to the directory. However, a directory is not always available in pervasive computing environments, because of their volatile nature. In the distributed approach no additional element is needed. The discovery of services can be done in two • ways: push mode, in which devices send unsolicited advertisements, and the other devices listen to these advertisements selecting the services they are interested in; and pull mode, in which devices request a service when they need it, and devices that offer the service answer the request. These solutions are better in changing environments but with higher cost in network transmissions and so in power consumption. The protocols deﬁned so far are based in one or more of these approaches.

19.2.2 Proposals within the Internet

The problem of service discovery in Internet was first considered in 1993, in the IETF’s SRVLOC group, which defined the Service Location Protocol, SLP [Guttman et al., 1999b]. SLP defines three “agents”: User Agents (UA), for discovering services on behalf of client software, Service Agents (SA), for advertising services, and Directory Agents (DA), for storing information about the services announced. SLP has two different modes of operation: if a DA is present, it follows the centralised approach; and if no DA is present, it follows the distributed, pull mode approach. The Simple Service Discovery Protocol, SSDP [Goland et al., 1999] was also proposed within the SRVLOC group, but it did not pass the draft state and is now obsolete. It was created as a lightweight discovery protocol for the Universal Plug-and-Play (UPnP) initiative. SSDP defines a minimal protocol for multicast-based discovery. As presented in the IETF, SSDP can work both in pull or push modes. A directory-based mode was also defined within UPnP. Recently, a new IETF group, the ZEROCONF, was created to define protocols that require no user configuration and administration. For the discovery of services, ZEROCONF works in some proposals based on DNS, all of them in draft state. DNS Service Discovery, DNS-SD [Cheshire, 2006a] is an Internet draft currently under development that defines a way to discover services using the DNS protocol. It works with DNS, and 19 The Role of Service Discovery in Context-Aware, Pervasive Computing Applications 235 so it is a directory-based, centralised service discovery protocol. However, two proposals are under development to define a distributed DNS, Multicast DNS [Cheshire, 2006b] and Link Local Multicast Name Resolution (LLMNR) [Adoba et al., 2007]. They both propose the implementation of a lightweight server in all devices, and use multicast queries (pull mode). The main difference between them is that Multicast DNS uses broadcast responses while LLMNR uses unicast responses.

19.2.3 Other Proposals

Other initiatives come from different sources. Some are tied to a particular subnetwork technology, like Blue- tooth’s Service Discovery Protocol and IrDA’s Information Access Service. Others are tied to a high-level distributed application technology, like Jini, Salutation, JXTA and DEAPspace. Most of them are based on a directory: the piconet master in Bluetooth SDP; the lookup server in Jini; and the salutation manager in Salu- tation. Other proposals take the distributed approach like the DEAPspace discovery protocol in which clients broadcast their world-view in a pure push model. Finally, some proposals like JXTA allow both behaviours, and peers can get service replies directly from another peer, or redirected from special peers called rendezvous peers.

19.3 Challenges for Service Discovery in Pervasive Computing

In pervasive computing, devices often communicate via wireless links without ﬁxed networking infrastructure. These devices, most of them mobile, form networks of dynamic topology, and they have different needs, requirements and capabilities such as limited processing and communication power. The user usually carries one or more of such devices with her (PDAs, mobile phones, music players, etc). Service discovery can then be used by these user devices to personalize the service offered to the user, based on what the environment offers. For example, when a user enters a room, the lighting and heating conditions can be automatically adjusted to her preferences, or when she receives a phone call, the HiFi equipment can be automatically muted. Many parameters of the surrounding devices can be controled and personalized according to the user preferences or her current situation, but for that to be possible, discovering what devices there are around us, and what parameters they allow to control, is ﬁrst necessary. This is what service discovery is used for. We claim that legacy solutions fail in addressing these requirements. New protocols must be designed taking them into account. Here we summarise some of the most outstanding challenges that must be addressed when designing a service discovery protocol for pervasive computing environments.

19.3.1 Interactions Inversely Proportional to Distance

While in Internet interaction between devices is independent of the distance between them, in pervasive networks nearby devices are very likely to communicate and interactions between far devices will be strange. In other words, we are interested on personalizing the environment of the user, not places far from the user. This principle was stated by Satyanarayanan [2001]. Thus service discovery protocols in pervasive environments must also discover services inversely proportional to distance.

19.3.2 Minimise Network Transmissions

Devices in these networks are mostly battery powered. Battery power is a limited resource and its consumption must be minimised as much as possible. One of the main sources of power consumption is network transmission [Jones et al., 2001] (a regular WLAN PCMCIA card requires 1.5 W in transmit mode). Discovering and personalizing services will need network transmissions, one of the most important issues will be to minimise the total number of transmissions. 236 Carlos García-Rubio and Celeste Campo

19.3.3 Do Not Rely on Fixed Infrastructure

Directory-based service discovery protocols are the ones that generate less traffic. However, they are not suitable for pervasive networks because they need a fixed device, the directory, which stores the list of services offered in the network and answers all the service requests. Ubiquitous networks are formed wherever two or more devices willing to communicate meet together: at home, in a boarding gate at the airport, in a lab at the office, in a museum, etc. In these networks, most or all devices are mobile. No fixed infrastructure, including a directory of services, should be required for the devices to communicate. So, service discovery protocols must not be based on a directory, but on distributed solutions. The topology of a pervasive network changes as new devices arrive or leave. Some devices will stay for a short time and others for a long time. In certain places, but not always, some devices will be fixed and will stay forever (such as air conditioning systems, sensors, printers, etc). A service discovery protocol for pervasive networks must adapt well to all these scenarios. It must react fast to topology changes but at the same time it must generate as few traffic as possible when the topology does not change (because devices remain there for a long time or because there is no other device in the vicinity).

19.3.4 Take Advantage of Broadcasts

Ubiquitous networks usually use wireless transmission, where the physical channel is shared by all devices. In one-hop networks, transmissions are broadcasted and received by all devices within range, regardless whether the destination is one or all of them. In multi-hop networks, new MANET routing protocols are under definition that support the efficient transmission of multicast traffic. In both cases, the cost of sending a packet to just one device (unicast) is similar as sending it to all devices within range (multicast). Recall here that devices will more likely communicate with nearby devices, that will be within their radio range.

19.3.5 Maximise Cooperation between Devices

In pervasive computing, one key concept is the cooperation between devices. In environments saturated with limited devices, cooperation allows them to carry out more complex tasks. This concept, applied to service discovery and personalization, means that devices within the same environment must cooperate to discover and personalize services, for the sake of the common good. As mentioned above, perhaps it is not feasible to have a directory, but if we have a cache of known services in each device, the join of all these caches is a kind of ‘distributed directory’. This fact should be exploited by any service discovery protocol to be used in pervasive networks. The static or dynamic nature of devices (i.e. the time the device remains in the network) varies, and so there may be inconsistencies between the caches. On the other hand, pervasive networks are open environments where any device may join the network, and not all of them are trustworthy. There may be malicious devices trying to confuse or make damage. These are problems we will have to deal with.

19.3.6 Adapt to Highly Changing Environments

Ubiquitous networks are changing in nature much more than the wired Internet. Information in the cache may become rapidly outdated. Some of the legacy service discovery protocols introduce mechanisms to guarantee the consistency of the cache. For example, SLP includes a lifetime for each service, that is the time it may be stored in a cache. Similarly, DNS includes a TTL (Time To Live) in the resource records. However, these mechanisms are not enough in pervasive networks, and new solutions have to be implemented. The service discovery and personalization should work fast in changing environments. 19 The Role of Service Discovery in Context-Aware, Pervasive Computing Applications 237

19.3.7 Take into Account Different Application Needs

Finally, we may distinguish two kind of users in search for a service. Some users look for any device offering a service in the network, not caring in the particular device. Other users browse all the devices offering that service in the network. One example of the ﬁrst class is the search for a temperature sensor in the room; we want any that gives us the temperature. One example of the second class is a word processor that searches for printers and wants to display all the available printers to the user and let him select one of them based on its characteristics (resolution, colour or b&w, etc). Traditionally, service discovery protocols have not differentiated between both kinds of searches. The difference was in the applications, that waited for all responses to the query or just read the ﬁrst one and threw the rest. This is a waste of bandwidth. In pervasive networks, where minimising transmissions is so important, it is worth distinguishing between these two kinds of service searches.

19.4 Some Ideas for a New Service Discovery Protocol for the Personalization of Pervasive Computing Environments

19.4.1 Is a New Protocol Needed?

The ﬁrst question to answer is whether a new service discovery protocol is really needed or not. In pervasive computing environments, service discovery protocols that use a centralised approach (directory) are not possible because no ﬁxed infrastructure can be assumed. Only distributed solutions are possible. Regarding the push solutions, we have to keep in mind that these environments are highly dynamic. The service discovery protocol has to adapt fast to changes. Push solutions are not good for this, as we will justify now. In a push protocol, each device announces its services every Ta seconds. It can be easily proved that if the device lifetime in the network follows an exponential distribution of mean µ, then the ratio of services discovered is:

µ − Ta SrvDiscRatio = (1 e µ ) (19.1) Ta − For µ = Ta, the SrvDiscRatio is 0,63. To approach to the 95% of services discovered, Ta has to be set about 10 times less than µ. For example, if we want to guarantee that devices that stay in the network for just 10 minutes discover the 95% of the services available, all devices must be configured to announce their services every one minute, regardless of whether they are fixed or mobile, or whether any device needs their services. This is a big penalty when extremely mobile devices are considered, resulting in big bandwidth waste and, consequently, big battery consumption. Regarding the pull solutions, they have the advantage over the push mode that no transmission is made when nobody needs a service, and that they have a service discovery ratio close to 100%. The disadvantages are that when the network is more static, its performance is worst (because many devices ask for the same service several times), and that when many devices in the network offer the same service, many answers are generated for each request. Ubiquitous environments are dynamic, but certain devices may be fixed or remain there for a long time, and it will be usual that tens or even hundreds of devices form one of these networks, so both disadvantages are important. Therefore, we should define a new distributed protocol, neither push or pull, that behaves better in these environments.

19.4.2 Is a Cache Enough?

Pull solutions seam better than push solutions in these environments, but it is necessary to reduce the number of transmissions. We may introduce a cache in the devices that locally stores the services announced in the network. When an application needs for a service, it will be first searched in the cache, and if not found it will be requested in the network. This mechanism contributes to significantly reduce the generated traffic. 238 Carlos García-Rubio and Celeste Campo

For example, simulations show that in a dynamic environment with 20 devices in mean, and with 5 different services, a pull approach with a cache of 35 services, the number of messages transmitted per request is reduced from 4.8 to 1.8. One inconvenient of introducing caches is that they may content a signiﬁcant number of stale entries. In the example above, 14% of the answers correspond to stale entries. Another inconvenient is that not all devices offering a service are discovered. If when a service is found in the cache, no request is sent to the network, then the other instances of the service will not be found. So, a cache is not enough and further improvements are needed.

19.4.3 New Mechanisms to Improve Performance

Our proposal for service discovery in pervasive environments is based in a pull mode with cache, but with some improvements:

To further reduce the number of transmissions, answers are one-hop broadcasted. This way, all devices learn • from all answers in their neighbourhood, and record them in their caches as in push solutions. Minimising the transmissions power consumption is reduced. To overcome the problem of not discovering all devices offering a given service, service requests are trans- • mitted always, but including the services found in the cache. If there are additional services in the network, they will be discovered, but if not, no answer will be transmitted. To approach the behaviour of a directory, each device answers the service requests with all services it has in • its cache, not just with the ones it offers. Besides, less limited devices (less mobile and with bigger cache) answer ﬁrst, and the rest of the devices abort their reply if they have nothing new to say. This way, the devices that are in the same environment cooperate to build an answer similar to the one that would be given by a directory, and they do it in a way that minimises the power consumption of the more limited devices. Other mechanisms may be introduced to reduce the number of stale entries in the caches. One of them is to • store services in the caches for a time that is the minimum of the service lifetime and the availability time of the device that has the cache, a parameter that will depend of the mobility characteristic of the device, that is not just the service lifetime, as is usual. The other mechanism is to introduce a message, that can be issued by the device that leaves the network or by any other that detects a service that is no longer available, to indicate the others that a service should be removed from the cache.

19.5 Deﬁnition and Implementation of a New Discovery Protocol

In this section we present the deﬁnition and implementation of a new service discovery protocol, the Pervasive Discovery Protocol (PDP), specially designed to work in pervasive environments. The protocol we propose does away with the need for the central server. One of the key objectives of the PDP is to minimise battery use in all devices. This means that the number of transmissions necessary to discover services should be reduced as much as possible. A device announces its services only when other devices request the service. Service announcements are broadcasted to all the devices within range, all of which will get to know about the new service simultaneously at that moment, without having to actively query for it.

19.5.1 Application Scenario

Let’s assume that there is a pervasive environment, composed of D devices, each device offers S services, stored in an array, Local, and expects to remain available in this network for T seconds. This time T is called the availability time and it is previously conﬁgured in the device, depending on its mobility characteristics. Each device has a PDP User Agent (PDP_UA) and a PDP Service Agent (PDP_SA). The PDP_UA is a process working on behalf of the user to search information about services offered in the network. The Service 19 The Role of Service Discovery in Context-Aware, Pervasive Computing Applications 239

Agent PDP (PDP_SA) is a process working to advertise services offered by the device. The PDP_SA always includes the availability time T of the device in its announcements. Each device has a Cache containing a list of the services that have been heard from the network. Each element of the cache has two ﬁelds: the service description and the service expiration time. The service expiration time is the estimated time for the service to remain available. This time is calculated as the minimum of two values: the availability time of the local device, and the service announced lifetime. Entries are removed from the cache when they timeout.

19.5.2 Protocol Description

PDP has two mandatory messages: PDP_Service_Request, which is used to send service requests, and PDP_Service_Reply, which is used to answer a PDP_Service_Request, announcing available services. Additionally, PDP has one optional message: PDP_Service_Dere gister, which is used to inform that a service is no longer available. Now, we will explain in detail how the PDP_UA and the PDP_SA use these primitives.

PDP User Agent

When an application or the ﬁnal user of the device needs a service of a certain type, it calls its PDP_UA. In order to support different application needs, in PDP we have deﬁned two kind of queries:

one query–one response (1/1): the application is interested in the service, not in which device offers it. • one query–multiple responses (1/n): the application wants to discover all devices in the network offering • the service. In this kind of query, we introduce a special type of service, named ALL, in order to allow an application to discover all available services of all types in the network.

Both types of query use the same message, PDP_Service_Request. A ﬂag in the header of the message indicates if it is 1/1 or 1/n. In one query–one response queries, the PDP_UA searches for a service_type in the list of local services and in its cache. If it is found, the PDP_UA gives the application the corresponding service description, without any network transmission. If it is not found, the PDP_UA broadcasts a PDP_Service_Request for that service, waiting CONFIG_WAIT_RPLY seconds for replies. If no reply arrives, the PDP_UA answers to the application that the service is not available in the network. If some reply arrives, the PDP_UA gives the application the service description received. In one query–multiple responses queries, the PDP_UA makes a list of known services of the type speci- ﬁed, that is, a list of the ones offered locally or stored in its cache (all the services if the service type is ALL). Then, it sends a PDP_Service_Request including this list. It waits CONFIG_WAIT_RPLY seconds for replies and when this timer expires, it gives the application the list of known services plus, if any replies arrived, the service descriptions received. PDP_UAs in all devices are continually listening on the network for all types of messages (requests and replies) and update their caches with the services announced in them. Moreover, the device’s cache has a limited size. When a PDP_UA hears a new announcement but the cache is full, it deletes the service entry closer to expire. Figure 19.1 shows a remote services learning example.

PDP Service Agent

The PDP_SA advertises services offered by the device. It has to process PDP_Service_Request messages and to generate the corresponding PDP_Service_Reply, if necessary. In order to minimise the number of transmissions, the PDP_SA takes into account the type of query made by the remote PDP_UA. When a PDP_SA receives a PDP_Service_Request 1/1, it checks whether the requested service is one of its local services. In that case, a PDP_Servi ce_Reply is scheduled for a random time, inversely proportional to the availability time of the device. During this time, if another reply to the same 240 Carlos García-Rubio and Celeste Campo

Fig. 19.1. Remote services learning example.

PDP request is heard, the reply is aborted as the remote PDP_UA will just pass the ﬁrst service to the application and discard any others. If the timer expires and no reply has been heard, the reply is sent. The algorithm awards the more static devices with more opportunities of answering requests. Therefore the algorithm gives higher priority to answers coming from devices with longer estimated availability. When a PDP_SA receives a PDP_Service_Request 1/n, it checks whether the requested service is one of its local services, or if it is in the cache. If so, it generates a random waiting time, inversely proportional to the availability time of the device and the number of known services. During this time, the PDP_SA listens the network for any PDP_Service_Reply of the same request. When the timer expires, if the PDP_SA knows about some additional devices offering this type of service that have not been announced yet, it sends its PDP_Service_Reply. So, the more time the device is able to offer the service and the bigger the cache, the higher the probability of answering ﬁrst. We suppose the device with the highest availability time and the bigger cache is the one with the most accurate view of the world. In certain cases, a device may detect when it is about to be switched off or to roam to other network. If so, the PDP_SA of the device has to send a PDP_Service_Deregister, listing all its local services, before switching off or roaming. When other device hears this message, it must remove the services listed from its cache. In other cases, when a device tries to access a service listed in its cache and the service is down, it may also use the PDP_Service_Deregister message to inform other devices that this service is no longer available. The device that receives the message may delete the entry from the cache.

19.5.3 Implementation

The full definition of the PDP protocol, including messages format, encapsulation over UDP and TCP, transport ports and multicast addresses used, is available in [Campo, 2004, Campo et al., 2006] For the service description we have not defined a new format but we have reused the one defined by the SRVLOC group of the IETF as part of SLP protocol [Guttman et al., 1999a]. We have built the PDP protocol and software that uses it to discover the services offered in its surroundings, both in Java 2 Standard Edition (J2SE) for desktop computers and laptops, and in Java 2 Micro Edition (J2ME) for limited devices. Focusing on the J2ME implementation, it uses the Personal Profile of the Connected Device Configuration (CDC). We chose this profile and configuration because of its support of multicast transmissions. We developed it using the IBM WebSphere Studio Device Developer. It runs in all devices supported by the WebSphere Micro Environment (WME) virtual machine, also known as J9. The size of the Jar file that includes both the PDP_UA and PDP_SA is 19 KB. We have tested our implementation in pervasive environments composed of desktop computers, laptops and Pocket PC PDAs, all with IEEE 802.11b interfaces working both in ad-hoc and infrastructure modes. Some 19 The Role of Service Discovery in Context-Aware, Pervasive Computing Applications 241 of the devices offer services to the environment (printer, fax, thermometer,. . . ) and others search for services in the network, either all services or a special type of service. In our experiments fixed and less limited devices tend to act as a directory of the services. The PDA searches and selects those services that adapt to the user profile, offering a personalized view of the environment. Mobile PDAs, with more limited resources and less availability time, quickly adapt to changes as they move from an environment to other, forgetting previous services and learning new ones. Finally, we observed that a significant percent of the queries were answered by non-limited devices, saving power consumption of the PDAs. To more precisely quantify the performance of our protocol we have conducted simulation experiments.

19.6 Evaluating our New Protocol

In this section we present a performance evaluation study of PDP in a pervasive computing environment where an application is personalized according to the services offered at that moment in the environment. We compare our protocol with the theoretical distributed approaches, push and pull, and with the service discovery protocol standard in Internet, SLP. This study was carried out through simulation.

19.6.1 Simulation Environment

Simulations were done using programs specially written for this purpose in OMNeT++. We used the process oriented, discrete event simulation paradigm. The batch-means [Law and Kelton, 2000] method was used to obtain a 90% confidence level with a 10% confidence interval in the results. Since the objective was to study the behaviour of the personalization process, only the service discovery layer was simulated in detail and lower layers were simplified. Specifically, physical and MAC layers details were omitted, and the communication was assumed to be contentionless and error free. During the simulation, devices join the pervasive environment at random times, offer random services, and leave the network after a random time. Context-aware applications in the devices request services from the environment, in order to personalize the application to the user. The number of devices in the network varies over time, but its mean remains stationary. Random times follow exponential distributions, while random services follow uniform distributions. For simplicity we assume that each device offers just one service. The parameters of the simulation are: the mean number of devices, the mean time they remain available in the network, the size of the caches, the mean time between service requests, and the total number of service types. The results of interests are: the number of messages (the number of messages transmitted in the network normalised to the number of service request), the service discovery ratio (the ratio of services discovered to the total number of services available in the network) and the error ratio (the ratio of services discovered that were not available in the network to the total number of services discovered).

19.6.2 Simulation Results

Figure 19.2 shows the number of messages transmitted, the service discovery ratio and the error ratio, in an scenario with 20 devices, an average device life time ranging from 600 to 19200 seconds, a cache size of 100 entries, 5 different types of services, and each device requesting a random service every 60 seconds. The PDP number of messages is quite under the obtained for SLP and for pull solutions, while keeping the same service discovery ratio and error rate of them. Figure 19.3 shows the ratio of replies sent by each kind of devices depending on its availability time. We have considered a scenario with 40 devices in mean, with 5 different availability times: 500, 2500, 4500, 6500 and 9500 seconds, with about 20% of devices (in mean) of each type. The rest of parameters of the simulation are the same as before, except that the cache size for devices with availability time 500 and 2500 is 10 services, while for devices with availability time 4500 and 6500 is 40 services and for devices with availability time 9500 is 100 services. 242 Carlos García-Rubio and Celeste Campo

7 PDP PULL PUSH 12 s PUSH 60 s 6 SLP

2 MESSAGES PER SERVICE REQUEST −>

0 100 1000 10000 100000 AVAILABILITY TIME −> 100 PDP PULL PUSH 12 s PUSH 60 s SLP

90 % SERVICES DISCOVERED −> 85

80 100 1000 10000 100000 AVAILABILTY TIME −> 40 PDP PULL PUSH 12 s 35 PUSH 60 s SLP

10 % FALSE SERVICES DISCOVERED −>

0 100 1000 10000 100000 AVAILABILITY TIME −> Fig. 19.2. Comparison of PDP with others protocols. 19 The Role of Service Discovery in Context-Aware, Pervasive Computing Applications 243

10 % SERVICE REPLIES PER SEARCH −>

0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 KIND OF DEVICES BY AVAILABILITY TIME Fig. 19.3. Service replies per search in different kind of devices.

In this ﬁgure we see that devices with greater availability time answer more requests, preserving power consumption of devices with smaller availability time. It worths mentioning that all percents shown in the ﬁgure sum up 70%, because in PDP some requests generate no replies (all known services were already included in the request). Considering this, devices with availability of 9500 seconds answer almost 50% of the service requests. If other service discovery protocol were used, all devices would answer with equal probability, 20%. Figure 19.3 also shows that devices with very small availability time (in our case, 500 seconds) answer more requests than devices with middle availability times. This is because these devices are highly mobile, continually change of networks, and in each new environment they arrive, they have to answer requests above their own services, to make them known to the rest of the devices.

19.7 Conclusions and Future Work

Two elements play a central role in this personalization of pervasive computing environments: service discovery protocols and user profiles. Service discovery protocols allow applications to discover what devices are available in the environment, what services they offer, and their characterstics. User profiles store the configuration settings and other data associated with an individual user or with a group of users. Applications will use profile information to select among services discovered, and to configure the selected service according to the user. In this chapter we have concentrated on service discovery. A good service discovery mechanism for providing personalization in pervasive environments must address the highly changing nature of these environments, not depending on any existing infrastructure. It must consider the power constraints of most of the devices. It must take into account that interactions with nearby devices will be likely and interactions with far devices will be strange. It must exploit cooperation between devices to achieve the best performance with the less effort. Finally, it must address the different needs of applications, with some looking for any instance of a service and other looking for all instances of a service. None of current service discovery protocols fulfil these requirements. Service discovery can be done at a subnetwork level (as Bluetooth’s SDP), at middleware level (as Jini and Salutation), but we think the best way to do it is at the TCP/IP protocol stack level (as the SLP in Internet). TCP/IP protocols are ubiquitous and the only way to guarantee that service discovery will be ubiquitous is to locate it in the TCP/IP protocol stack. As a future work, we are working in the integration of our service discovery protocol, PDP, with some user profile standard. Some of them have been evaluated in the framework of the UBISEC IST project, and an 244 Carlos García-Rubio and Celeste Campo integration with GSDL (a service profile protocol) has been implemented [Campo et al., 2005]. We must now extend this integration to other profiles, including user profiles, what would allow full user personalization of pervasive computing environments.

References

B. Adoba, D. Thaler, and L. Esibov. RFC 4795: Linklocal Multicast Name Resolution (LLMNR), January 2007. C. Campo. Pervasive Discovery Protocol. Technical report, Telematic Engineering. University Carlos III of Madrid., 2004. http://www.it.uc3m.es/celeste/tesis/draft/draft_pdp.pdf. C. Campo, M. Muñoz, J. C. Perea, A. Marin, and C. Garcia-Rubio. PDP and GSDL: A new service discovery middleware to support spontaneous interactions in pervasive systems. In 3rd IEEE International Conference on Pervasive Computing and Communications Workshops, Kauai Island, Hawaii (USA), 2005. C. Campo, C. Garcia-Rubio, A. Marín-López, and F. Almenárez-Mendoza. PDP: A lightweight discovery protocol for local-scope interactions in wireless ad hoc networks. Computer Networks, 50(17):3264–3283, 2006. S. Cheshire. DNS-Based Service Discovery. Internet-Draft (work in progress), August 2006a. draft-cheshire- dnsext-dns-sd-04. S. Cheshire. Performing DNS queries via IP Multicast. Internet-Draft (work in progress), August 2006b. draft-cheshire-dnsext-multicastdns-06. Y. Y. Goland, T. Cai, P. Leach, and Y. Gu. Simple Service Discovery Protocol/1.0. Internet-Draft (work in progress), April 1999. draft-cai-ssdp-v1-03.txt. E. Guttman, C. Perkins, and J. Kempf. RFC 2609: Service Templates and Service: Schemes, June 1999a. E. Guttman, C. Perkins, J. Veizades, and M. Day. RFC 2608: Service Location Protocol, version 2, June 1999b. C. E. Jones, K. M. Sivalingam, P. Agrawel, and J. C. Chen. A survey of energy efﬁcient network protocols for wireless networks. Wireless Networks, 7(4):343–358, 2001. A. M. Law and W. D. Kelton. Simulation modeling and analysis. McGraw-Hill, 2000. M. Satyanarayanan. Pervasive computing: Vision and challenges. IEEE Personal Communications, 8(4):10–17, August 2001. 20 Personalizing Pedestrian Location Services through Ontologies and Rules

Vassileios Tsetsos, Vassilis Papataxiarhis and Stathes Hadjiefthymiades

Pervasive Computing Research Group Dept. of Informatics and Telecommunications, University of Athens Panepistimioupolis, Ilissia, 15784, Greece {b.tsetsos,vpap,shadj}@di.uoa.gr

Summary. Pedestrian wayfinding and location-based services, in general, are core components of mobile context-aware systems, partly due to the availability of mature positioning technologies. Several approaches have been proposed for implementing such services and there are already many deployed systems. However, personalization of location services has not been investigated in an adequate degree yet, especially from a knowledge engineering perspective. In this chapter, we adopt a model-driven approach to personalized location services, which employs ontologies and declarative rules in order to deliver optimal guidance and content to users. Ontologies are used in order to model the user and the physical environment. Moreover, rules are applied to the ontological instances in order to adapt the output of the involved algorithms (e.g. navigation algorithm) according to the user profile and available content. The main objective of the chapter is to present related work in this area and provide implementation details of our approach. Furthermore, several knowledge engineering issues are discussed and open research issues are identified. The presented research findings may be a useful reference for knowledge-based personalization efforts in other application domains.

20.1 Introduction

Location-Based Services (LBS) are, perhaps, the only context-aware services that have found their way from research labs to the industry. Several mobile network operators provide location-based services worldwide and, as positioning technologies (e.g. Global Positioning System, 3G positioning) become ubiquitous, more LBS deployments will follow. Moreover, car navigators have already achieved a high market penetration in most developed countries. The situations regarding indoor and pedestrian LBS are not so good yet, mainly due to technical issues in positioning methods, but many promising approaches have been proposed and many research prototypes have been developed. These last services are the focus of this chapter and, specifically, some design issues that enable personalized, human-centered provisioning of indoor LBS. By “personalized provisioning” we mean service provisioning that is aware of the user profile (e.g. preferences, abilities) and context (e.g. structure and status of surrounding environment). Personalized service behaviour may affect several aspects of an LBS system, such as its execution algorithm, the user interfaces and the interaction between the system and the user, the delivered content, etc. In this chapter we deal with technologies and techniques that enable such personalization and we provide the reader with the architecture and implementation of such system. We focus on the navigation service, since it is the most complex LBS, but the approach adopted can be applied to other services as well, e.g. for outdoor environments or for automotive applications. At the heart of our approach are modern knowledge representation and reasoning techniques. Specifically, we demonstrate how the emerging Semantic Web (SW) technologies [Berners-Lee et al., 2007] can satisfy the requirements of personalized applications and facilitate their implementation. Recently, many advances have been made in the SW area, such as maturity of ontology description and query languages, development of rule 246 Vassileios Tsetsos, Vassilis Papataxiarhis and Stathes Hadjiefthymiades languages, stable and production versions of reasoning engines and development tools. These advances enable the use of SW technologies and methods in more and more real-world knowledge-based applications. Some advantages regarding the suitability of SW technologies for personalization and context-awareness purposes are the following: Intuitive description of adaptive behavior of the personalized system through the use of a declarative form • of knowledge (such as rules). Clear semantics of application and context models through ontologies. • Efficient and unambiguous reasoning with subsets of First Order Logic. • Serialization of SW data based on XML, which is both machine-interpretable and human-readable. • Ontology-driven domain knowledge reuse and interoperability. • Modularity of knowledge bases through well established ontological engineering techniques. • Existence of quite mature development tools (e.g. editors, Application Programming Interfaces). • In the rest of this chapter, we survey some related work on context-aware and personalized applications that employ knowledge representation and reasoning techniques (Section 20.2). In Section 20.3, the main elements of a semantic LBS framework will presented, and the role of ontologies and rules is clarified. Section 20.4 demonstrates the functionality of the system through a simple use case. Finally, several open issues are identified in Section 20.5.

20.2 Personalization In Context-Aware Mobile Services

In this section we present some context- and location-aware applications from diverse domains. Moreover, we survey some work on context and location representation. All of the following applications take advantage of location information in order to provide context-aware services. Some of them describe means of applying personalization in indoor applications while others focus on outdoors or both types of environments. Most of them rely on sensing mechanisms as well as context representation formalisms. The common denominator for the majority of these applications is that they are oriented towards a knowledge-based approach. Most of them exploit some means of formal knowledge representation techniques (e.g. ontologies and rules).

20.2.1 Applications

A common category of applications that require personalization is context-aware services, and mainly location- based services (e.g. navigation, tracking or emergency services). Conventional context-aware services rely only on contextual conditions external to the user. In more advanced scenarios, service executions are also affected by user abilities, interests and preferences. For example, consider the case of a user that uses a wheel chair to move around. In this scenario, a navigation service should capture that knowledge and present to the user paths that do not include stairways or escalators. Semantic Web technologies provide an efficient way to store and exploit that kind of information using ontologies, rule languages and reasoning modules. Some personalized location-aware applications that have been developed so far can be found in [Gartner et al., 2004, Tsetsos et al., 2006]. In [Bikakis et al., 2006], a generic framework for developing applications based on Semantic Web technologies is described, focusing on indoor context-aware services that exploit location information. This approach utilizes an RDFS ontology in order to represent contextual knowledge and is capable of supporting services like information retrieval for persons and rooms, calendar, indoor navigation, etc. Compass [van Setten et al., 2004] is a context-aware mobile application that supplies tourists with personalized recommendations. The basic feature of the application is its adaptive behavior on the user preferences as well as the context information. Furthermore, Compass provides map-based services, such as access to information of specific points of interest. Another application, which is based on SW techniques and takes advantage of GPS information, is mSpace mobile [Wilson et al., 2005]. Specifically, it allows the user to semantically annotate map elements, like buildings and roads, and publish these data. Moreover, mSpace mobile seems to be appropriate for travelers or 20 Personalizing Pedestrian Location Services through Ontologies and Rules 247 persons unfamiliar with their surroundings, delivering them information while they are moving inside a region. This application targets at facilitating the retrieval of information useful to a traveler. Authors in [Chen et al., 2004] present an ontological model designed to support pervasive applications and ubiquitous computing environments. The SOUPA ontology is expressed in OWL [McGuinness and van Harmelen, 2004] and comprises an appropriate model in order to represent concepts related to contextual information (e.g. time, space, action). Some applications and projects [Moreira et al., 2007, Chen, 2004] have adopted SOUPA to capture context-awareness and apply personalization to pervasive environments through an ontological framework.

20.2.2 Contextual and Spatial modeling

Providing location-dependent information and services to users requires modeling the spatial elements of the user environment. Such modeling may exploit geometric information, symbolic information or both (i.e. hybrid model). In [Lorenz et al., 2006] a hybrid spatial model is proposed. This model is designed for indoor environments, consisting of hierarchically-structured graphs. These spatial graphs may contain geometric information (e.g. the distance between two path elements may be a weight in the corresponding arc of the spatial graph) as well as symbolic information (e.g. topology). Also in [Grütter and Bauer-Messmer, 2007], a hybrid architecture able to capture spatial information and reason over it is presented. It supports Region Connection Calculus (RCC) [Cohn et al., 1997] spatial relations and investigates the integration of terminological concepts and spatial relationships. Finally, in [Anagnostopoulos et al., 2007] several modeling techniques, that also involve ontologies, are described for representing and reasoning about user’s context and situation for pervasive computing environments.

20.3 A Framework for Semantic LBS

In this section we present our approach to personalized LBS, through exploitation of user, contextual and spatial semantics. We describe the main idea and key features of our framework, while more details on speciﬁc parts of this framework can be found on [Kikiras et al., 2006, Tsetsos et al., 2006].

20.3.1 System Architecture

The architecture of the implemented navigation system is illustrated in Fig. 20.1 and can be decomposed to the following basic components:

Fig. 20.1. System architecture. 248 Vassileios Tsetsos, Vassilis Papataxiarhis and Stathes Hadjiefthymiades

Location-based services. This component implements the business logic of the LBS. The currently sup- • ported LBS are: “Find nearest points of interest”, “Indoor navigation”, “Pull-based information delivery about current user location” and “Push-based information delivery about current user location”. The latter corresponds to a tourist guide, mainly applicable to museum visitors. For example, in case of indoor navigation, this component accepts user requests and responds with the optimal path. Path optimality depends on several factors, such as suitability for current user context and length of the selected paths. Domain ontologies. Several domain ontologies are exploited for enabling formal and controlled ways • of representing LBS-relevant information. The most important is the Indoor Navigation Ontology (INO), which describes the basic spatial and structural concepts of indoor environments as well as their relationships. User Navigation Ontology (UNO) contains user classes and elements of the user profile (e.g. demographics, motor abilities, etc.). In addition, a Content Ontology and a Device Ontology have been designed. More details on these models are provided in Section 20.3.2. Path-selection rules. The path-selection process is assisted by a set of rules. These rules are expressed • through the domain ontologies of the system. The rules are mainly applied in order to determine the paths and their associated content that are considered appropriate and accessible for each user request. More details on the role of rules in our approach are provided in Section 20.3.3. Indoor geospatial model. The INO instances are created through a geometric representation of the indoor • topology. Such geometric data initially reside in a Geographical Information System (GIS) as building blueprints and are subsequently transformed to actual ontology instances. A brief description of the INO population technique can be found in Section 20.3.2, while a more detailed report can be found in [Kolom- vatsos and Tsetsos, 2007]. Routing algorithm. This algorithm is a central element of the framework and, in combination with the • Path-Selection Rules, is responsible for the determination of the optimal path between two given endpoints. The algorithm is a combination of a k-shortest paths searching algorithm and a simplest path algorithm. The main motivation for adopting such algorithm is that the shortest path may not always be the optimal path. Moreover, path simplicity is always welcome by users, especially those facing wayfinding problems. Indoor positioning system. The positioning infrastructure designed to support the system is composed by • three components: – Wireless LAN positioning [Sekkas et al., 2006], based on a Bayesian learning technique applied to WLAN received signal strength indications. – Passive RFID tags. – A dead reckoning algorithm based on [Fang et al., 2005]. The values retrieved from all these methods feed a data fusion algorithm that calculates the final user position inside a building (we remind that GPS is, in general, not applicable to indoor environments).

20.3.2 Domain Ontologies for LBS

As already mentioned, several (domain) ontologies are used by our approach. In the following sections we describe the basic concepts, structure and purpose of each one of them. Moreover, we provide some details on how these ontologies are populated with instances.

Indoor Navigation Ontology (INO)

INO1 is a spatial ontology for the description of basic elements of the space and, especially, navigation paths. As one can see in Fig. 20.2, INO deﬁnes concepts that correspond to every basic spatial element to be found in an indoor environment. Some key concepts (whose deﬁnitions can be found in [Tsetsos et al., 2006]) are Space, Path, Path_Element, Path_Point, Corridor, Passage, Obstacle, Positioning_Point, Point_of_Interest, Entrance, etc. Additionally, INO includes several relationships among concepts in order to describe the connections between them. 1 http://p-comp.di.uoa.gr/projects/ontonav/INOdoc/index.html 20 Personalizing Pedestrian Location Services through Ontologies and Rules 249

Furthermore, INO contains concepts that can be aligned to user characteristics (defined in UNO). Moreover, each spatial concept has an associated description type (that refers to Content Ontology), information useful during guidance or for advanced path selection techniques (see also Section 20.3.3). This way, we are able to provide personalized paths and content delivery. Reasoning involved in path searching and user guidance is based on instances extracted from files produced by a Geographic Information System (GIS), axioms holding for the INO relationships (e.g. transitive relationships) and rules (i.e. user profile).

Fig. 20.2. The main concept hierarchy of INO (arrows represent is-a relationships).

In order to semi-automatically populate INO we have devised a novel methodology. For the initial deﬁnition of the navigation elements of an indoor environment, we use a GIS system. GIS systems help in visualizing spatial elements such as corridors, entrances, navigational points (end points, turn points, junctions), etc. Each GIS layer corresponds to a basic concept or set of concepts in INO ontology. For example, there are layers that describe points of building entrances and exits, room entrances and exits, elevators, ramps, etc. Figure 20.3 depicts this layered representation.

Fig. 20.3. GIS based INO population.

The blueprints of each floor are located at the bottom layer and are used as a reference for the rest of the layers. Based on this layer, corridors are represented by lines. Upon these lines, the various path points are defined as well as their most important semantic features (identification number, label, a short functional description). For inter-floor connections (i.e. vertical passages) special metadata are used for associating their endpoints. Once the GIS data are in place, one can proceed with the actual ontology population. Each of the GIS layers is exported as a shape file, which is further imported as a table in a spatial database. Subsequently, a series of 250 Vassileios Tsetsos, Vassilis Papataxiarhis and Stathes Hadjiefthymiades algorithms are used to create the instances. Firstly, we use an algorithm in order to decide for every point in a floor, which line it belongs to. The second step is to discover the endpoints of each line in order to help us in the next step (i.e. discovery of adjacent points). The result is that for every point we store its floor, the lines in which it belongs to and its adjacent points. The final step is to create the individuals based on INO and on the information that we have extracted from the previous steps. Since the system uses also graph-based algorithms for discovering paths, we have to also create a graph that corresponds to the extracted INO model. This is a straightforward process as we already have computed adjacency relationships between the various path elements. In Table 20.1, we show some indicative numbers regarding the modeling of our building, so that reader has an indication of the resulting ontologies and graph.

Building elements # of rooms 140 # of Floors 4 Building graph elements # of vertices 285 # of arcs 624 INO elements Total # of instances 550 # of concepts 90 # of properties 40 # of junctions 41 # of end-points 25 # of turn-points 20 # of corridors 71 # of horizontal passages 305 # of vertical passages 6 # of motor passages 1 Table 20.1. Sample building overview.

User Navigation Ontology (UNO)

User Navigation Ontology (UNO) is a model that describes a user profile from the perspective of navigation and wayfinding. UNO is based on wayfinding theories for pedestrians [Downs and Stea, 1973, Siegel and White, 1975] and consists of the following main components:

General user demographics. This category captures all the basic user information such as name, age, and • gender. Mental/cognitive characteristics. This category captures all information considering user’s mental/cognitive • abilities (e.g. consciousness functions which control user’s state of awareness and alertness, orientation disability, mental disabilities such as mental impairment or Alzheimer disease). User’s sensory abilities. Sensory impairments affect the way a user exploits her sensing abilities (espe- • cially viewing and hearing) during wayfinding. User’s motor abilities. Motor abilities refer to all kinetic abilities of users and not only to those associated • to their mobility, although the latter are more important from the perspective of navigation. Users may be classified as having autonomous mobility without assistive devices, mobility with wheelchair, etc. Navigational preferences. This category captures user’s navigational preferences, such as “selection of • the shortest route first”, “selection of the fastest route first”, “avoidance of crowded areas” (suitable for visually-impaired users), etc. Interface preferences. This category captures user’s preferences considering the means and the media in • which she will receive routing instructions. 20 Personalizing Pedestrian Location Services through Ontologies and Rules 251

The first version of UNO was presented in [Kikiras et al., 2006]. The most recent version2 is also aligned to the International Classification of Functioning, Disability and Health (ICF) of the World Health Organization3. For example, some modifiers that quantify the degree of a disability were introduced and some UNO classes were explicitly mapped to ICF categories. The basic class hierarchy of UNO is presented in Fig. 20.4. Regarding the creation of the UNO instances, currently this is performed in a manual way. There are some predefined user profiles and the user can select and further adjust the one that best matches her. In real world deployments of the system, a user could carry a smart card where her detailed profile is stored (such an approach is taken by many projects dealing with accessibility such as SNAPI4). Hence, the user could automatically provide all information required for creating the UNO instances.

Fig. 20.4. Basic UNO class hierarchy.

Content Ontology

This ontology is used for annotating the content associated with each path element (space, point of interest, ect.). Figure 20.5 illustrates the basic structure of this model. The main semantics described for each content element includes its type (e.g. Text, Image, Video) and the corresponding presentation modality (visual, auditory, haptic), details about encoding or format, the language, the size/length, etc. These data are stored in a semantic content management system, which provides forms and tools for manual annotation of multimedia

2 http://p-comp.di.uoa.gr/projects/ontonav/UNOdoc/index.html 3 http://www.who.int/classiﬁcations/icf/en/ 4 http://www.tiresias.org/snapi/index.htm 252 Vassileios Tsetsos, Vassilis Papataxiarhis and Stathes Hadjiefthymiades content. The LBS algorithms or other processes of the system can retrieve such metadata through predicates in rules or declarative query languages (e.g. SPARQL [Prud’hommeaux and Seaborne, 2007]).

Fig. 20.5. Extract of the content ontology.

Device Ontology

The device ontology is used for modeling the various user devices (e.g. PDAs, smart phones) in terms of capabilities/characteristics and human-computer interfaces. Main concepts in this ontology are the various types of mobile devices and handsets, peripheral devices (e.g. Bluetooth earphones, Braille display), the supported connectivity modes (WiFi, Bluetooth), the size of the display (if any), etc. The elements of this model can augment the selection of the content to be delivered, by comparison against its characteristics, or deﬁne the modality of the man-machine interaction. The population of this ontology can be either manual, based on existing speciﬁcations such as CC/PP [Klyne et al., 2004], or based on knowledge extraction from device datasheets.

20.3.3 A Hybrid Navigation Algorithm

In our system, the calculation of paths is performed in a hybrid way. An initial set of paths is created by a graph-based path searching algorithm, called k-simplest paths. This is a combined version of a k-shortest paths searching algorithm [Yen, 1971] and the searching algorithm [Duckham and Kulik, 2003]. The simplest path algorithm calculates the path from a given origin to a given destination that is the most simple to follow and has the lowest possible complexity in terms of navigation instructions (e.g. it has the minimum number of turns/junctions). We used this new combined algorithm, because both length and complexity of a path are key parameters that should be minimized during navigation. After the graph-based step, declarative rules are applied to the set of k paths in order to select the optimal path. Optimality depends on multiple criteria and is decided based on domain knowledge regarding the user, the user device, the characteristics of path elements and the content associated with each path element. The path selection rules are constructed through concepts and properties specified in the ontologies described in the previous section. The adopted hybrid approach for the navigation algorithm enables easy introduction of new routing constraints and/or preferences. Adding such constraints in a monolithic graph-based navigation algorithm is a nontrivial task, while the resulting algorithm would have to solve a multi-objective optimization problem, which is known to be NP-hard [Serafini, 1986]. Hence, the ultimate benefit of such approach is that many different navigation strategies can be modeled and implemented in a rather intuitive, declarative way, while some key objectives, such as path length and simplicity, are satisfied. In the following paragraphs we demonstrate how several navigation policies can be modeled through our approach. The rules are expressed in Semantic Web Rule Language (SWRL) format [Horrocks et al., 2004]. 20 Personalizing Pedestrian Location Services through Ontologies and Rules 253

Exclusion of non-traversable paths

This kind of rules is able to mark the non-traversable path elements for a specific user, based on her profile. The following rule (20.1) represents the common sense knowledge that a path proposed by the system to a wheel-chaired user should not include stairways. Applying this rule before the execution of the k-simplest paths algorithm, we can filter out all non-traversable elements and, thus, form a user-compatible graph.

uno : W heelchairedUser(u) ino : Stairway(s) ino : isObstacleF or(s, u) (20.1) ∧ →

Content-based navigation

Content-based navigation refers to a special case of navigation service, where a user follows a path with associated content that matches her interests. An example application of such service is a museum tour guide service. In the following example, rule (20.2) corresponds to the following natural language expression: “if a specific content element concerns a topic that a user is interested in, then the user is interested in this element”. Rule (20.3) marks the path elements described by “interesting content elements”. After applying these rules, the system could select all matching path elements, and either find a path that covers most parts of the sub-graph they form (if rules are applied prior to k-simplest paths algorithm) or select which elements of a predefined path will be presented to the user (if rules are applied after the k-simplest paths).

uno : User(u) con : Category(t) uno : isInterestedInT opic(u, t) con : Content(c) ∧ ∧ ∧ ∧ con : hasCategory(c, t) con : isInterestingContentF or(c, u) (20.2) ∧ →

ino : P athElement(e) ino : hasContent(e, c) con : isInterestingContentF or(c, u) ∧ ∧ → ino : isInterestingElementT oP resentT o(e,u) (20.3) →

Presentation-based navigation

This kind of rules is able to achieve personalization based on the presentation modalities supported by the content and the device capabilities, as well as on the user proﬁle. The goal is to provide users with paths for which appropriate instructions can be provided, or alternatively, use only path elements with appropriate presentation characteristics for the formation of the navigation instructions. An example indicating such functionality follows:

uno : V isuallyImpairedUser(u) uno : hasAbilityT oHear(u, “good00) ∧ → uno : isCompatibleF ormatF or(“audio00, u) (20.4) →

ino : P athElement(e) uno : User(u) dev : Device(d) con : Content(c) ∧ ∧ ∧ ∧ ino : hasContent(e, c) con : hasDescriptionF ormat(c, f) ∧ ∧ ∧ uno : isCompatibleF ormatF or(f, u) uno : usesDevice(u, d) ∧ ∧ ∧ dev : supportsModality(d, f) ino : isCompatibleT oP resentT o(e,u) (20.5) ∧ → Rule (20.4) expresses that auditory information is compatible with visually impaired users with no hearing problems (common sense knowledge). Rule (20.5) declares that a path element described in a format compatible to the user, can be presented to a user, if her device supports this format too. Hence, the system ensures that the path elements and content used for guiding the user take into account her (dis)abilities. 254 Vassileios Tsetsos, Vassilis Papataxiarhis and Stathes Hadjiefthymiades 20.4 Example Scenario

In this section we present a path selection example demonstrating the functionality of the navigation algorithm. In the ﬂoor plan of Fig. 20.6 we assume that a visually impaired, wheel-chaired user is located in point A, while point H constitutes her destination. There are four possible shortest and loop-less paths that the system can propose to her: (i) ABDIH (the simplest path, due to the small number of turns), (ii) ACFEH (which is the shortest path), (iii) ABDGFEH, and (iv) ACFGDIH.

Fig. 20.6. A navigation algorithm example.

We assume that points C, F and E are described through auditory content while points B, D and G through visual content (denoted with speaker and eye icons in Fig. 20.6). A smart and adaptive navigation algorithm should consider this fact, since the speciﬁc user is unable to understand the visual content. Hence, the inclusion of points described by auditory content seems more suitable to the users needs. At the ﬁrst step of this case, the system would mark point G as obstacle for the user, since she cannot traverse the steps located at that point (this implies that candidate paths (iii) and (iv) will be eventually rejected). Next, the k-simplest paths algorithm would be executed in order to discover that ABDIH is the simplest path. Finally, the algorithm would conclude (through rules similar to those already presented) that the path ACFEH is the most suitable to provide the user, due to her disability to access and understand visual content; even it is not the simplest one. This way, a presentation-based navigation scheme is adopted by the system that allows for advanced user experience.

20.5 Future Research Directions

Although we have addressed the core issues of a knowledge-based framework for personalized services, there are still several open issues that deserve further research. Most of these issues relate to the knowledge-based infrastructure of such framework. For instance, a number of limitations arise with regard to reasoning and model 20 Personalizing Pedestrian Location Services through Ontologies and Rules 255 expressiveness. Firstly, the development of more expressive ontology languages is required in order to model the features of real environments. This way, an increasing amount of relationships between application elements could be captured, leading to more complex models. Moreover, the nature of the physical environments imposes representation requirements related to uncertainty or fuzziness of the contextual information. Considering, also, the well-known trade-off between tractability and expressiveness, we conclude that more research is necessary towards developing more efficient algorithms in order to manage and reason over such application models in real-time. Concerning more practical implementation issues, existing reasoning modules lack of a unified framework able to manage ontologies and rules as an integrated knowledge base. Hence, the reasoning tasks of ontology classification and rule execution are performed independently. This could lead to inconsistencies of the knowledge base and, hence, the developer should handle these conflicts manually. Finally, most of current Semantic Web inference engines do not support non-monotonic reasoning features (e.g. negation-as-failure), mainly due to the open world assumption, thus, narrowing the inference capabilities of the services and their ability to perform model revision. Finally, another research issue that is raised is that of (semi-)automatic profile extraction and revision. Typical methods for automatic (profile) learning that employ, for example, probabilistic inferences have not been explored extensively in the context of the Semantic Web and constitute a promising research area.

20.6 Conclusion

We have discussed the application of knowledge-based techniques and technologies in the area of personalized context-aware services. The focus of the chapter was on a novel approach to Location-Based Services and, speciﬁcally, navigation. The main contribution of this approach is the ﬂexibility and simplicity that provides towards introducing user constraints in context-aware services. The applications of this approach are manifold and not limited to the domain of pedestrian location services.

References

C. B. Anagnostopoulos, Y. Ntarladimas, and S. Hadjiefthymiades. Situational computing: An innovative architecture with imprecise reasoning. Journal of Systems and Software, 80(12):1993–2014, 2007. T. Berners-Lee, D. Connolly, S. Hawke, I. Herman, E. Prud’hommeaux, and R. Swick. W3C Semantic Web Activity, 2007. URL http://www.w3.org/2001/sw/. A. Bikakis, T. Patkos, G. Antoniou, M. Papadopouli, and D. Plexousakis. A semantic-based framework for context-aware pedestrian guiding services. In 2nd International Workshop on Semantic Web Technology For Ubiquitous and Mobile Applications, in conjunction with the 17th European Conference of Artiﬁcial Inteligence, Riva del Garda, Italy, 2006. H. Chen. An Intelligent Broker Architecture for Pervasive Context-Aware Systems. PhD thesis, University of Maryland, 2004. H. Chen, F. Perich, T. Finin, and A. Joshi. SOUPA: Standard ontology for ubiquitous and pervasive applications. In 1st International Conference on Mobile and Ubiquitous Systems, Boston, Massachusetts (USA), 2004. A. G. Cohn, B. Bennett, J. M. Gooday, and N. Gotts. Qualitative spatial representation and reasoning with the region connection calculus. GeoInformatica, 1(3):215–316, 1997. R. J. Downs and D. Stea. Image and Environment, chapter Cognitive Maps and Spatial Behavior. Aldine Publishing Company, 1973. M. Duckham and L. Kulik. Spatial Information Theory, chapter “Simplest” Paths: Automated route selection for navigation. Springer-Verlag, 2003. L. Fang, P. J. Antsaklis, L. A. Montestruque, M. B. McMickell, M. Lemmon, Y. Sun, H. Fang, I. Koutroulis, M. Haenggi, M. Xie, and X. Xie. Design of a wireless assisted pedestrian dead reckoning system. the NavMote experience. IEEE Transactions on Instrumentation and Measurement, 54(6):2342–2358, 2005. 256 Vassileios Tsetsos, Vassilis Papataxiarhis and Stathes Hadjiefthymiades

G. Gartner, A. Frank, and G. Retscher. Pedestrian navigation system in mixed indoor/outdoor environment. The NAVIO project. In CORP and Geomultimedia Symposium, Vienna, Austria, 2004. R. Grütter and B. Bauer-Messmer. The European Information Society: Leading the Way with Geo-information, chapter Towards Spatial Reasoning in the Semantic Web: A Hybrid Knowledge Representation System Ar- chitecture. Lecture Notes in Geoinformation and Cartography. Springer-Verlag, 2007. L. Horrocks, P. F. Patel-Schneider, H. Boley, S. Tabetand, B. Grosof, and M. Dean. SWRL: A Semantic Web Rule Language Combining OWL and RuleML, 2004. URL http://www.w3.org/Submission/SWRL/. P. Kikiras, V. Tsetsos, and S. Hadjiefthymiades. Ontology-based user modeling for pedestrian navigation systems. In Workshop on Ubiquitous User Modeling, in conjunction with the 17th European Conference on Artificial Intelligence, Riva del Garda, Italy, 2006. G. Klyne, F. Reynolds, C. Woodrow, H. Ohto, J. Hjelm, M. H. Butler, and L. Tran. Composite Capa- bility/Preference Profiles (CC/PP): Structure and Vocabularies 1.0, W3C Recommendation, 2004. URL http://www.w3.org/TR/CCPP-struct-vocab/. K. Kolomvatsos and V. Tsetsos. Spatial ontology population using a GIS. Technical report, Department of Informatics & Telecommunications, University of Athens, 2007. URL http://p-comp.di.uoa.gr/projects/ontonav/gis2ino.pdf. B. Lorenz, H. J. Ohlbach, and E. P. Stoffel. A hybrid spatial model for representing indoor environments. Lecture Notes in Computer Science, 4295:102–112, 2006. D. L. McGuinness and F. van Harmelen. OWL Web Ontology Language Overview, 2004. URL http://www.w3.org/TR/owl-features/. E. S. Moreira, D. N. Cottingham, J. Crowcroft, P. Hui, G. Mapp, and R. M. P. Vanni. Exploiting contextual handover information for versatile services in NGN environments. In 2nd International Conference on Digital Information Management, Lyon, France, 2007. E. Prud’hommeaux and A. Seaborne. SPARQL Query Language for RDF, 2007. URL http://www.w3.org/TR/rdf-sparql-query/. O. Sekkas, S. Hadjiefthymiades, and E. Zervas. Enhancing location estimation through data fusion. In 17th IEEE International Symposium on Personal, Indoor and Mobile Radio Communications, Helsinki, Finland, 2006. P. Serafini. Recent advances and historical development of vector optimization, chapter Some considerations about computational complexity for multi-objective combinatorial problems. Springer-Verlag, 1986. A. W. Siegel and S. H. White. Advances in Child Development and Behavior, chapter The Development of Spatial Representation of Large-Scale Environments. Academic Press, 1975. V. Tsetsos, C. Anagnostopoulos, P. Kikiras, and S. Hadjiefthymiades. Semantically enriched navigation for indoor environments. International Journal of Web and Grid Services, 2(4):473–478, 2006. M. van Setten, S. Pokraev, and J. Koolwaaij. Context-aware recommendations in the mobile tourist application COMPASS. In 3rd International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems, Eindhoven, The Netherlands, 2004. M. L. Wilson, A. Russell, D.A. Smith, A. Owens, and M. C. Schraefel. mSpace mobile: A mobile application for the semantic web. In Workshop on End User Semantic Web Interaction, in conjunction with the 4th International Semantic Web Conference, Galway, Ireland, 2005. J. Y. Yen. Finding the k shortest loop-less paths in a network. Management Science, 17(11):712–716, 1971. Part III

Open Problems and Current Research Trends

21 Conclusions and Future Trends

José Juan Pazos Arias1, Carlos Delgado Kloos2 and Martín López Nores1

1 Department of Telematics Enginering, University of Vigo Campus Universitario s/n, 36310 Vigo (Spain) jose,[email protected] 2 Departamento de Ingeniería Telemática, Universidad Carlos III de Madrid Av. Universidad 30, 28911 Leganés, Madrid (Spain) [email protected]

Summary. In this chapter, the editors present some reflections about the themes elaborated in the first two parts of the book, in order to identify open problems in the area of personalized information services and, thereby, point out the challenges that will probably be the focus of research and development efforts in the following years. These observations have been driven by comments gathered from some of the invited authors, to benefit from their expertise and their knowledge of specific domains of application.

21.1 Introduction

The success of the information technologies depends ultimately on the services provided to the users, which can be rendered useless by the growth in the amount of information available. Personalization techniques aim at solving this problem by providing the users with information services that match their interests, preferences and needs in any context within the reach of technology. This goal requires a shift from the traditional search engines of the Internet to proactive recommender systems working in behalf of each user, within the limits imposed by a legal framework intended to ensure privacy. All along Part I of this book, it has been explained that personalization is achieved by running a set of filtering and information selection algorithms, which take as input (i) metadata descriptions of the contents, products and services available, (ii) profiles capturing knowledge about the interests and needs of the users, and (iii) the identification of the user’s context, taken in a broad sense as any circumstance (environmental or even personal) that defines his/her situation at the moment of computing recommendations. From Part II, in turn, it follows that there has been significant growth during the last few years of personalized information services in such fields as education and training, administration, healthcare, advertising and commerce, etc. The range of applications envisaged is still growing to harness the possibilities raised by the new communication networks and protocols, consumer devices and consumption models. Nowadays, there is still much place for innovation in the field of personalization, e.g. to enhance any of the technical aspects involved with user modeling, context awareness or filtering to improve the personalization quality achieved. In our opinion, however, the greatest challenge is to find scalable and sustainable exploitation models for the already existing solutions. These models should be based on algorithms with a computational cost that allows personalized services to become a ubiquitous reality, and also on strategies that promote the generation of contents, metadata and interactive applications without leaving all responsibilities and tasks in the service providers’ side. Furthermore, it is time to stop thinking about the different domains of application as separate niches, to establish the foundations for a global personalization framework, more ambitious and with more impact in society. In the following sections, we will devote some words to presenting some thoughts about the aforementioned problems. 260 José Juan Pazos Arias, Carlos Delgado Kloos and Martín López Nores 21.2 Generation and Management of Metadata

The quality achieved by any personalization engine depends on having rich metadata to describe the contents, products and/or services available to recommend to the users. To date, however, most of the business models behind personalized information services implied that the service providers would generate metadata documents themselves. This assumption is impossible to realize in practice, because providers do not have sufficient workforce to characterize an ever growing amount of resources, which is worsened by the fact that the Web 2.0 phenomenon is putting users to massively generate contents themselves. Going on with the Web 2.0 spirit, however, it would be possible to solve the problem if we managed to turn the users into metadata providers as well. Collaborative tagging systems (aka folksonomies) such as Flickr and del.icio.us could be showing the way to go; however, the amount and quality of the metadata so gathered may not be sufficient for effective personalization, as it frequently happens that some users provide bogus or redundant annotations. One way to boost participation and serious involvement could be to develop incentive schemes to reward the users for any valuable information they provide (e.g. with coupons for pay-per- view services, hours of premium access to certain contents, fewer publicity breaks when he/she happens to be watching TV at home, ... or even cash). Obviously, the more valuable the information, the greater the reward. This approach would require to advance research in mechanisms with which to ensure traceability of each user’s contributions, and also in indicators for the personalization engines to reason about the trustworthiness of the metadata. In the same line of thinking, it would be necessary to reconsider the current means to collect and store metadata, which have been mostly based on formalized specifications (e.g. MPEG-7 or TV-Anytime) that classify each and every piece of information. In a context where any user can provide information, it is not realistic to think that they will take time to learn about taxonomies and similar artifacts. In turn, the challenge would arise to adapt the existing filtering algorithms to work with unstructured knowledge bases, or to develop new strategies if that were not possible. Particularly, a semantic approach would require huge loads of data mining to gain insight into the meaning of the tags provided by the users. Last but not least, it would be necessary to develop interfaces that make the tagging mechanisms accessible from the new range of information devices, whose input facilities are very different from those of personal computers. Some of the contributing authors have emphasized the potential interest of speech recognition here.

21.3 Scalability in Context-Aware Recommenders

From the preceding chapters of this book, it follows that there exist many pieces of work in literature related to all of the constituent parts of a generic or purpose-specific recommender system. This includes the elements in charge of contextualization, but the research advances in this regard have not yet been fully integrated into working systems. At the most, there are location-aware recommenders that can adapt their outputs depending on the user’s spatial location. For the following years, it is expectable that a lot of research efforts will be devoted to other dimensions such as infrastructure (e.g. input devices available or surrounding communication resources), environmental conditions (noise, light, ...), time frames or even the user’s feelings (e.g. mood or stress). These advances will boost the personalization quality of the current recommender systems, far beyond the performance achievable by matching descriptions of the available resources against profiles that provide average characterizations of the users. The advances in the identification of context and the subsequent influence on filtering operations bear potential problems of scalability. To begin with, considering many dimensions of context makes it necessary to deploy many types of sensors around the users and their environment. Those sensors may provide an overwhelming amount of information, so it will be computationally very hard to identify correlations among multiple variables and high-level events. Making this processing efficient may well require advances from such diverse areas as hardware engineering, algorithmic theory, fuzzy computing, emotion and cognition, etc. Having solved the question of identifying context, it would still remain to optimize the filtering mechanisms so as to compute personalized recommendations in real time: context is something that cannot be determined beforehand, and a centralized personalization engine —like the ones existing nowadays— may find it hard to 21 Conclusions and Future Trends 261 provide timely results as the number of users grows. One way to fight this limitation would be to harness the computational power of millions of consumer devices by moving at least part of the complexity to the user terminals. For instance, a layered recommendation process could be designed so that central servers compute rough recommendations for stereotypical groups, which the terminals later refine by incorporating the specific preferences of each individual as well as his/her context.

21.4 Scalability in the Provision of Interactive Applications

The growing number of contents, products and services is not only challenging the development of recommender systems to deliver the most suitable items to the users. On the contrary, we have seen throughout this book that the new technologies are particularly advantageous for the ability to also deliver interactive applications for learning, commerce, etc. When it comes to personalizing those applications, one soon realizes that it will be unrealistic to think of humans developing specific applications for all different users in all possible contexts and managing all the contents, products and services available. Therefore, it will be necessary to extend the scope of recommender systems to reason about software components or templates that can be instantiated on demand for combination with other resources selected for a user. This will require metadata specifications to characterize the functionalities provided by each component or template, together with suitable composition/aggregation engines, in what appears to be a promising and innovative research topic halfway between software engineering and artificial intelligence.

21.5 Towards a Global Personalization Framework

As it follows from the very structure of this book, the personalization research has commonly produced specialized solutions for different devices even within a single domain of application. Probably, the clearest example lies in the development of solution for distance learning, inasmuch as there exist separate bodies of science for learning through the Internet as accessed from personal computers (e-learning), learning through digital TV (t-learning) and learning through mobile devices (m-learning). While it is true that there exist important differences regarding the reach, contents and scopes of the three approaches, it is also true that those differences arise simply to fit (radically) different contexts, so only the convergence of the three approaches would enable the greatest opportunities for ubiquitous and lifelong learning. Accordingly, many authors have suggested to start thinking about holistic applications in distance learning, in remote health care, in remote administration and so on. To date, however, there have been very few attempts in this regard, and it usually happens, say for example in TV watching, that the knowledge gathered about a user when he/she utilizes fixed devices at home cannot be merged with what we learn about him/her when using mobile devices outdoors. Actually, one step beyond unifying the various flavours of the different application domains, it is easy to foresee the advantages of a global personalization framework, capable of gathering and accumulating information about the users from different domains of application. Such a cross-media and cross-domain approach could certainly soothe ramp-up problems with new users, accelerate the user modeling processes and, ultimately, ensure greater personalization quality in the long term due to handling richer information. Again, this idea can be empowered with the ongoing development of Web 2.0 technologies, specifically with the possibility to learn about a user from his/her interactions with other people in social networks like Facebook, Myspace or Plaxo, and in virtual worlds such as SecondLife. The repertoire of a user in this vision would include basic personal data, personal relations of different kinds (friendship, family, work, admiration, etc), depictions, own or beloved photos and videos, status information (micro-blogs), intentions and plans (as in 43things.com), calendar, travels, location through time, documents, presentations, scholar publications, CV, biography and e-portfolio (studies, qualifications, etc), health record, financial information, and so on. We believe that this development is unstoppable, so we had better take advantage of all the positive possibilities arising, and also take measures to foresee and prevent the risks and abuses it could lead to.