University of Southampton Research Repository ePrints Soton Copyright © and Moral Rights for this thesis are retained by the author and/or other copyright owners. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This thesis cannot be reproduced or quoted extensively from without first obtaining permission in writing from the copyright holder/s. The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the copyright holders. When referring to this work, full bibliographic details including the author, title, awarding institution and date of the thesis must be given e.g. AUTHOR (year of submission) "Full thesis title", University of Southampton, name of the University School or Department, PhD Thesis, pagination http://eprints.soton.ac.uk UNIVERSITY OF SOUTHAMPTON Telling Your Story: Autobiographical Metadata and the Semantic Web by Mischa Moussavian Tuffield A thesis submitted in partial fulfillment for the degree of Doctor of Philosophy in the Faculty of Engineering, Science and Mathematics School of Electronics and Computer Science Intelligence, Agents, Multimedia Group July 2010 UNIVERSITY OF SOUTHAMPTON ABSTRACT FACULTY OF ENGINEERING, SCIENCE AND MATHEMATICS SCHOOL OF ELECTRONICS AND COMPUTER SCIENCE INTELLIGENCE, AGENTS, MULTIMEDIA GROUP Doctor of Philosophy by Mischa Moussavian Tuffield Given the current explosion of user-generated content driven by the ever-decreasing price of sensing and storage hardware the dream of capturing and archiving the entirety of a human life is slowly being realised. The Semantic Web, a discipline of Computer Science, aims to support the sharing and interoperation of knowledge using the Web’s infrastructure. This thesis aims to roadmap a framework utilising the principles and technologies underpinning the Semantic Web, enabling the vision of global knowledge sharing, in an open and policy aware manner, with the end aim of supporting a net- work for the exploitation of personal information. This sharing is facilitated through the adoption of a lingua franca, shared conceptualisations for domain knowledge, and some core design principles. The main focus of Semantic Web research has been the development of a web-scale knowledge-base whereby information is stored and exposed in a machine-readable format with the ultimate aim of aggregating information from disparate sources, allowing for statements to be contextualised with respect to others culminating in a web-scale knowledge resource accessible through standard protocols. The current popularity of social computing – Web 2.0 – where users post personal information to online communities is eluding to the fact that information, linked and shared within a social-context presents added value to the end-user. Given the sensitive nature of personal information, one may not wish to expose all of the information about them self to the World Wide Web, but may wish to benefit by linking to knowledge residing on this shared resource. This ability to store personal information privately, in ones own personal web-space and not on a third party server, whilst at the same time connecting to the publicly available information is presented as key challenge facing the Computer Science community today. Specific information pertaining to one aspect of a user’s activities, such as their picture taking habits or their geographic log, may not present a detailed account of a user’s actions, but as more information is pushed into the public domain and aggregation technologies mature individuals and their day-to-day activities will be easier to track. As more and more of our personal lives are pushed into the public domain, the no- tion of an online-persona is becoming more and more applicable to the average person. ii This thesis presents an infrastructure for the capturing and archival of autobiographical metadata, whereby information from multiple sensors is aggregated and stored in a per- sonal Lifelog. The surrender of digital identity has become commonplace, for purposes ranging from commerce, marketing, social networking, government, receipt of services, travel or security, Lifelogging has the potential to reaffirm the individual’s control of his or her own digital identity. The Lifelog is a constructed identity that outweighs the others simply by weight of evidence, complexity and comprehensiveness. This thesis presents an infrastructure for the capture and exploitation of personal meta- data to drive research into context aware systems. The aim is to expose ongoing research in the areas of capture of personal experiences, context aware systems, multimedia an- notation systems, narrative generation, all set in the context of enabling and supporting the Semantic Web Vision. The thesis details the work underway towards the goal of creating a multi-domain contextual log, and is followed by a discussion of how such a log can be used to drive the development of detailed Lifelog and an investigation into the amount of personal information being pushed into the public domain. Contents Acknowledgements ix 1 Introduction 1 1.1 Overview .................................... 2 1.2 Thesis Statement and Contributions ..................... 4 1.2.1 Thesis Statement ............................ 4 1.2.2 Contributions .............................. 4 1.3 Thesis Structure ................................ 5 1.4 Declaration ................................... 7 1.5 Publications ................................... 8 2 Background & Related Work 10 2.1 The Semantic Web and Personal Metadata ................. 10 2.2 Lifelogging and Autobiographical Knowledge Management Environments 13 2.3 Multimedia Asset Management: Image Annotation ............. 14 2.4 Multimedia Ontologies ............................. 18 2.4.1 The Functional Requirements for Bibliographic Records Model .. 18 2.4.2 The ABC Ontology .......................... 19 2.4.3 The Action Ontology .......................... 20 2.4.4 The Event Ontology .......................... 20 2.5 Narratives and the Semantic Web ....................... 21 2.5.1 Narrative Theory for the Semantic Web ............... 22 2.5.2 The Analogy: SW meets Narratology ................ 23 2.5.3 Memories for Life ............................ 23 3 Motivating a Digital Autobiographical Log 24 3.1 Lifelogging and the Disappearance of the Body ............... 25 3.1.1 Lifelogging: An Introduction ..................... 26 3.1.2 Lifelogging: Discussion ......................... 28 3.1.3 Privacy, Empowerment, and Identity ................. 30 3.1.3.1 The empowerment of the individual ............ 33 3.1.3.2 Alibis and unstructured data ................ 34 3.1.3.3 Logging in public and private ................ 36 3.2 Lifelogging: The Realisation .......................... 37 3.2.1 Lifelogging and Technology ...................... 38 3.2.2 Lifelogging and Memory ........................ 40 3.2.3 Lifelogging and Social Networking .................. 42 iii CONTENTS iv 3.2.4 Lifelogging and Services ........................ 43 3.3 Information Overload: Personal Information, Infosmog & the Semantic Web ....................................... 45 3.3.1 The Web and the abundance of Personal Information ....... 46 3.3.2 The Semantic Web: An Introduction ................. 47 3.3.3 The Semantic Web and Personal Information ............ 48 3.3.4 The Linked Data Initiative ...................... 51 3.4 Memories for Life and Web Science ...................... 52 4 Narratives, Stories, & Ontologies 55 4.1 Towards Ontological Narrative ........................ 55 4.2 Computational Models of Narrative ...................... 59 4.2.1 A Simple Taxonomy .......................... 60 4.2.2 Ontological Narrative ......................... 61 4.2.3 Ontological Models of the Fabula ................... 62 4.2.4 Ontological Models of the Story .................... 63 4.2.5 Ontological Models of the Narrative ................. 63 4.2.6 Modeling the Fabula .......................... 64 4.3 OntoMedia ................................... 65 4.3.1 Application Domains .......................... 65 4.3.1.1 FicNet ............................ 65 4.3.1.2 State-Based Sequencer ................... 66 4.3.1.3 Cultural Heritage ...................... 66 4.3.2 The OntoMedia Ontology: An Overview ............... 67 4.3.3 The OntoMedia Ontology: The Details ................ 68 4.3.3.1 Event Modelling ....................... 68 4.3.3.2 Entity Modelling ....................... 69 4.3.3.3 Extensibility ......................... 70 4.3.3.4 Contexts ........................... 72 4.3.4 An example story annotation ..................... 72 4.4 OntoMedia: Summary ............................. 73 4.5 OntoMedia: Conclusions ............................ 74 5 Semantic Logger: An Autobiographical Contextual Log 76 5.1 Overview : The Semantic Logger ....................... 77 5.2 Infrastructure: Semantic Logger ........................ 79 5.3 Knowledge Acquisition ............................. 82 5.4 Semantic Logger and FOAF .......................... 86 5.5 Making use of mSpace ............................. 89 5.6 Services: Semantic Logger ........................... 90 5.7 Use Cases .................................... 91 6 Photocopain: Context Driven Multimedia Asset Management 93 6.1 Photocopain: System Architecture ...................... 93 6.2 Photocopain: Annotation Sources ....................... 94 6.3 Photocopain: Architecture ........................... 98 6.3.1 Annotation Server ..........................