Welcome to the Urban Mixed Realities Workshop

We would like to welcome you to the Urban Mixed Realities workshop which is being hosted as part of CHI 2008 in the stunning city of Florence. In common with CHI, this workshop embraces the themes of art, science and balance from the perspective of urban mixed realities and brings together researchers, practitioners and students from a range of backgrounds and countries.

Urban mixed reality environments encompass a range of user experiences from games through to systems which help people uncover the invisible elements of the city. They also range from single user industrial applications through to multi-user shared experiences which utilise anything from mobile phones through to large multi-touch displays. However one aspect remains common across all these experiences in that they are inherently linked to the underlying aspects of the city, and in doing so rely as much on the advanced technologies as they do on the diversity of city life.

In common with the field of mixed reality this workshop is very much a mesh of prior and new work, with classic HCI research through to CSCW, virtual environments, mobile spatial interaction, pervasive games and art all playing a part. However it was felt that the diverse array of user experience issues within urban mixed realities represented a real challenge and one which (as the field is growing in popularity) required a specific workshop. One which would allow us to explore the wide range of aspects such as: meshing reality with unreality; understanding constructive perception and social action; presence; group behaviours and co- location; materiality vs immateriality and meaning making.

All papers and posters selected for presentation in the workshop were reviewed by a team of experts, and in all cases had to pass a quality threshold. We received three times more submissions for presentation than we could accept so in the end only chose the nine best papers, this resulted in us asking many people to present their work as a poster. We have also initiated a special edition of the “Psychnology Journal” which will cover much of the work presented today. The precise details of how to submit to the journal will be announced after the workshop.

We would like to thank you for preparing and presenting your work.

Kind regards,

The Urban Mixed Realities Workshop Organising Committee

Workshop Participants

Daniel Belcher, HITLab, University of Washington USA Wolfgang Broll, Fraunhofer FIT, Germany Thomas Calvert, Simon Fraser University, USA Dimitris Charitos, University of Athens, Greece Giulio Jacucci, Helsinki Institute for Information Technology, Finland Tim Jay, University of Bath, UK Pamela Jennings, Carnegie Mellon University, USA Eric Kabisch, University of California, Irvine, USA Kari Kuutti, University of Oulu, Finland Silvia Lindtner, University of California, Irvine, USA Rod McCall, Fraunhofer FIT, Germany Joachim Rothauer, Sony Europe, Germany Markus Sareika, Technical University of Graz, Austria Tuomo Tuikka, VTT Technical Research Centre of Finland Sean White, Columbia University, USA Andrew Wilson, DRU/Blink, UK

Organising and Programme Committee Rod McCall, Fraunhofer FIT, Germany Ina Wagner, Vienna University of Technology, Austria Kari Kuutti, University of Oulu, Finland Giulio Jacucci, Helsinki Institute for Information Technology, Finland Wolfgang Broll, Fraunhofer FIT, Germany

Additional Programme Committee Members Luciano Gamberini, University of Padova, Italy Elisa Giaccardi, University of Colorado, USA

Workshop Programme

Organisers

Wolfgang Broll is the head of the Collaborative, Virtual and Augmented Environments Department of Fraunhofer FIT in Germany. He is also a lecturer at RWTH Aachen. He has been doing research in the area of shared virtual environments, multi-user VR and 3D interfaces since 1993 and is currently involved in a number of EU projects related to mixed and augmented realities.

Giulio Jacucci, Ph.D. leads Ubiquitous Interaction, a research group at the Helsinki Institute for Information Technology. Jacucci leads national and European funded research focusing on user experience of ubiquitous and mixed reality applications. His current research includes investigating advanced interaction techniques in particular in public spaces: ubiquitous and mixed reality applications for large-scale events, multimodal emotional interfaces for performative interaction in art and entertainment.

Kari Kuutti is a professor in Human-Computer Interaction and Computer-Supported Cooperative Work in Department of Information Processing Science at University of Oulu, Finland. He has been working in the area of design issues related to space more than ten years, first in virtual reality and design of smart products, and currently in design of intelligent environments in cooperation with architects and urban planners.

Rod McCall Ph.D. is a research scientist within the Collaborative Virtual and Augmented Environments Department at Fraunhofer FIT. Prior to this post he held an ERCIM Fellowship at Fraunhofer FIT and CRP-Gabriel Lippmann, Luxembourg. Until 2005 he was a Senior Research Fellow at Napier University, Edinburgh. For the last ten years he has been engaged in research related to sense of place and presence in mixed and virtual realities.

Ina Wagner is Professor for Multidisciplinary Systems Design and Computer-Supported Co-operative Work (CSCW) and Head of the Institute for Technology Assessment and Design at TU Wien. She has been engaged in design-oriented work for the last ten years in the context of national and European projects. Among her research interests are novel interfaces, creative design methods, and the role of materiality as a resource for persuasive, narrative and experiential interactions.

Programme Committee - The programme committee consists of the organisers plus the two following members.

Luciano Gamberini, Ph.D. is Associate Professor at the Department of General Psychology at the University of Padova and head of the Human-Technology Laboratories www.psicologia.unipd.it/htlab. He is Editor in-Chief of PsychNology Journal www.psychnology.org and member of several other journals and conferences scientific boards including the International Workshop on Presence. He coordinates local units in several EU founded projects related to Presence, e-Health, Affective VR, Human-Computer Interaction. Recent research topics include: augmented interaction and social presence over networks, persuasive and serious games, elderly cognition & technology, training & collaborative environments, simulation and safe driving, NIRS-Virtual Reality based BCI. Luciano Gamberini is co-chair of the ACM Students Research Competition at CHI2008, Florence

Elisa Giaccardi Ph.D. is a research scientist within the Center for LifeLong Learning & Design, University of Colorado, Boulder. She has been combining academic and professional activities in digital media, interaction design, and community development since 1996. Her recent work is focused on meta-design strategies for public authoring, collective storytelling, and place making, with a specific interest in issues of new heritage. Research topics include: cross-media interaction, social software and participative systems, locative media and collaborative mapping, and context and emotion aware interfaces.

Workshop Agenda and Papers

08:30-09:00 Registration & Poster Set Up 09:00-09:05 Welcome 09:05-10:30 Session 1: Applications & Simulations Urban Sketcher: Mixing Realities in the Urban Planning and Design Process Markus Sareika and Dieter Schmalstieg JITC3: Just-In-Time Augmented Reality Command & Control Center, Tom Furness, Daniel Belcher, Xianhang Zhang and Anirudhan Vijayakanthan Pedestrian Navigation in Virtual Environments, Andrew Park, Tom Calvert, Paul Brantingham and Patricia Brantingham 10:30-11:15 Posters and Coffee 11:15-12:45 Session 2: Concepts and design approaches Visual Vectors and Public Interfacing, Pamela Jennings The Where of Mixed Reality: Some Guidelines for Design, Rod McCall, Iris Herbst, Anne-Kathrin Braun and Richard Wetzel Is “Presence” Important in Mobile Map Interaction? Antti Oulasvirta, Sara Eslander and Giulio Jacucci 12:45-14:00 Lunch 14:00-15:30 Session 3: Cases and Technology Approaches Mixed Realities in China’s Internet Cafes, Silvia Lindner and Bonnie Nardi Tagged Cities, Minna Isomursu and Tuomo Tuikka Orchestration and Direction of MR Games, Jan Ohlenburg, Wolfgang Broll and Irma Lindt 15:30-16:15 Poster Session and Coffee 16:15-17:15 Wrap up 20:00 Dinner Poster Sessions Emergent Narrative in Hybrid Environments, Eric Kabisch Progress towards Site Visits by Situated Visualization, Sean White, Petia Morozov, Ohan Oda and Steven Feiner Mixed Realities in the Living Tattoos Social Platform, Diana Domingues, Eliseo Raetegui, Gelso Reinaldo and Alexandre Lorenzatti Inter-Group Communication Via locative Media use in Urban Space, Charlampos Rizopoulos, Angeliki Gazi and Dimitris Charitos Moblogging the City: Accessing Personal Perceptions of Urban Spaces, Tim Jay and Danäe Stanton Fraser 12 Mixed Reality Principles of Animation – Based on Disneys Principles of Animation, Sabiha Ghellal, Jan Ohlenburg, Joachim Rothauer, Rod McCall and Stephan Harrer From City Poems to Aliens, Monsters and Sprites that Live in Phones: A Brief History of Story Worlds, Andrew Wilson and Derek Hales

Urban Mixed Realities: Technologies, Theories and Frontiers

Kari Kuutti Rod McCall Abstract Department of Information CVAE Department This workshop will address the approaches, challenges, Processing Science Fraunhofer FIT benefits and aspects of interaction within urban mixed University of Oulu Schloss Birlinghoven reality environments. It will seek to draw upon existing FIN-90014, Oulu, Finland Sankt Augustin, 53754 research into place, presence and situated interaction [email protected] Germany while exploring areas of art, flow, ambience, urban

[email protected] design, performance and technology. In doing so, it will Giulio Jacucci bridge the divide between art and science which exists Helsinki Institute for Information Ina Wagner in the growing research area of urban mixed realities. Technology Institute for Technology The anticipated outcome is a closer examination of the Helsinki University of Technology Assessment & Design issues relevant to interacting within urban mixed and University of Helsinki Vienna University of Technology realities and how to drive the research agenda forward. P.B. Box 9800 Argentinierstrasse, 8 Helsinki, Finland A-1040, Vienna Keywords [email protected] Austria Augmented Reality, Virtual Reality, Mixed Reality, [email protected] Location Aware Computing, Context Aware Computing, Wolfgang Broll Mobile Devices, User Interaction, Presence, Place CVAE Department Fraunhofer FIT ACM Classification Keywords Schloss Birlinghoven H5.1 Multimedia Information Systems – Artificial, Sankt Augustin, 53754 Augmented and Virtual Realities; H5.2 User Interfaces; Germany H.5.m Miscellaneous [email protected]

Introduction Mixed reality environments encompass a range of Copyright is held by the author/owner(s). domains from pervasive games through to systems to CHI 2008, April 5–10, 2008, Florence, Italy. support cultural heritage, and currently represent a ACM 978-1-60558-012-8/08/04. growing area of research. However the growth in such systems has resulted in a need to further explore their

situated and social nature, and how these aspects the design, construction and assessment of MR impact upon their use of such systems and alter the environments to promote an appropriate sense of environment around them. Although there has already presence in relationship to the real world, the mediated been a substantial amount of research into tele- mixed reality experience and other users. presence and sense of place [1] much of this has focused on more traditional technologies such as purely The current presence framework, where the experience virtual environments or mobile tour guides. In contrast of presence is a complex, multidimensional perception, urban mixed reality environments require a substantial formed through an interplay of raw (multi-)sensory change of research emphasis and in doing so must take data and various cognitive processes [2], has following into account the following shifts: features:

• From virtual to mixed reality environments • It is based on dualistic assumption: there are which mesh or augment places and times separate “world” and “mind”: perception of objective presence in world creates a • From psycho-physiological and “constructive subjective sense of presence in mind perception” to understanding social action, interaction and meaning making • Reality is taken as granted – there exists a “natural” world which is same to everybody • From a focus on individual behavior to • The place (either virtual or real) providing the interaction in groups who are co-located and stimuli that is perceived by a human has distributed dominating influence in the experience of presence; human actions can only strengthen • From immaterial environments to those which or diminish this experience, the origin is always combine real and virtual elements in the place

• From a passive sense of place and presence, to • Rich enough interaction can fool senses so that one where creation of place, meaning and a subjective sense of presence is created even engaging of all senses plays a critical role in a virtual environment.

If we want to take our experiments to real life in urban Approaches and Theories space, something else is needed to replace such The growth in Mixed Reality (MR) environments that laboratory-based approach. In searching that we can need to take account of the situated and social nature start from the criticism presented by Mantovani & Riva. of the real world spaces they are placed in, raises a According to them this kind of definition of presence number of significant challenges for our understanding based on physical presence is critically unfounded and of presences that go beyond the existing explorations also prejudicial for the development of systems to of ‘telepresence’. A central question is how to approach

support co-operation and communication, and they • What are the potential personal and societal suggest instead that the concept of presence should be benefits from urban mixed realities? based on social construction of presence, where reality is continually being negotiated and filtered by artifacts, • What approaches are most suitable to further by means of which we adapt the environment to our develop urban mixed realities? needs and at the same time adapt ourselves to the environment in order to exploit the affordances it offers Topics to us [3]. To grasp the active orientation, social • Interaction issues within urban environments character, or materiality and artifact-relatedness of situations of “presence” in mixed reality environments , • The role of the urban environment in shaping the origanisers of the workshop have in their own content and technologies research drawn for example from Gibson’s ecological theory of perception [4], from CSCW [5], and from • Frameworks and theories, place, presence co- Activity Theory [6], but there certainly are many other operative systems and cognition useful theories and approaches that could be brought to bear in the development of a better conceptual toolset • Technologies: from mobile phones to head to deal with mixed realities. mounted displays

Questions • Design and evaluation methodologies The objective of the workshop is to embrace the CHI2008 themes of balancing art and science, design • The role of art and performance in urban mixed and research as well as practical motivation and realities process. Although the subject area is different the workshop will ask similar questions to those discussed • Personal, societal and economic issues reality during the CHI`07 Mobile Spatial Interaction Workshop to the use and deployment of urban mixed [7] such as: reality systems

• What are the different aspects of urban mixed • Applications of urban mixed reality realities? Which fields from science, technology technologies for example: games, cultural and art should we draw upon? heritage, emergency response training, pervasive games, social networking, etc. • What are the challenges faced by urban mixed reality systems? What are the usability, Intended Audience technical and other issues which present new The intention is to bring together students, research challenges or potential future problem practitioners and researchers from a range of related areas. areas such as: virtual and mixed realities, mobile

systems and application design, interaction design, Citations pervasive games, tourism and cultural heritage, [1] Benyon, D. Smyth, M., O’Neill, S., McCall, R. hardware development and design, augmented reality and Carrol, F. The Place Probe: Exploring a Sense of Place in Real and Virtual Environments. Journal of research, art and theatre, software and hardware Presence: Tele-operators and Virtual Environments. 15, prototyping, etc. 6, (2006), 668-687. [2] Ijsselstein, W. & Riva, G. Being there: The Structure experience of presence in mediated environments. In The workshop will last for one day and will consist of Riva, G., Davide, F. & Ijsselstein, W.A. (eds.) Being 15-25 participants. The objective is to combine a formal there: Concepts, effects and measurements of user style consisting of up to ten paper presentations with a presence in synthetic environments, (2003), 3-16, IOS similar number of posters or demonstrations also being Press, Amsterdam. presented. Participants will be invited to present [3] Mantovani, G. & Riva, G. “Real” presence: How challenging research results or ideas with the objective different ontologies generate different criteria for of stimulating discussion. Discussions during the day presence, telepresence, and virtual presence. Presence: Teleoperators and Virtual environments 8, 5, (1999), will be used to formulate the agenda of the closing 538-548. panel session. At the close of the workshop the results [4] Gibson, J.J. (1986) Ecological Approach to will be summarised and presented on a poster which Visual Perception. Lawrence Erlbaum Associates. will be displayed during the duration of CHI 2008. The make up of the panel will be partially decided by a [5] Schmidt, Kjeld. The Problem with ‚Awareness’: Introductory Remarks on `Awareness in CSCW''. secret ballot. Computer Supported Cooperative Work 11, (2002), 285-298. Post Workshop [6] Cole, M., & Engeström, Y. A cultural-historical Workshop participants will be invited to write a full approach to distributed cognition. In G., Salomon (Ed.), paper outlining their work. Subject to acceptance the Distributed Cognitions: Psychological and Educational full paper will appear in a special edition of the Considerations, (1993), 1-46, New York: Cambridge Psychnology Journal. University Press. [7] Fröhlich, P., Simon, R., Baillie, L., Roberts, J. Acknowledgements and Murray-Smith, R. Workshop on Mobile Spatial The authors acknowledge the assistance of the other Interaction, Ext. Abstracts, CHI 2007, ACM Press members of the EU Funded IPCity project (2007), 2841-2844. (www.ipcity.eu). The format of this workshop and extended abstract are based on several previous CHI workshops.

Urban Sketcher: Mixing Realities in the Urban Planning and Design Process

Markus Sareika Abstract Graz University of Technology This paper describes how mixed reality (MR) technology Institute for Computer Graphics and Vision is applied in the urban reconstruction process and can Inffeldgasse 16a be used to share the sense of place and presence. It 8010 Graz, Austria introduces Urban Sketcher, an MR prototype application [email protected] designed to support the urban renewal process near or on the urban reconstruction site. Dieter Schmalstieg Graz University of Technology Keywords Institute for Computer Graphics and Vision Mixed reality, augmented reality, urban planning, Inffeldgasse 16a architecture, natural multimodal interaction, 8010 Graz, Austria collaboration [email protected] ACM Classification Keywords H.5.1 [Information Interfaces and Presentation]: Multimedia Information Systems – Artificial, augmented and virtual realities; J.5 Arts and Humanities - Architecture

Introduction In the urban design and renewal process consent, between the participants is the overall goal. In order to Copyright is held by the author/owner(s). achieve this accordance, the integration of diverse CHI 2008, April 5 – April 10, 2008, Florence, Italy points of view is necessary. The exchange of ACM 1-xxxxxxxxxxxxxxxxxx. information is the key in this process, as it allows enriching the sense of place and presence for the 2

individual helping to mediate and lead the integration process by mixing the various realities. Urban Reconstruction Site Mixed Reality Technology can enhance and provide PTU Camera additional communication channels for collaborative activities, which are richer than usual, leading to an Projection Screen MR-Tent improved shared vision of the future urban ART Tracking environment. Urban Sketcher [7], a Mixed Reality Sketching User Computer application, is especially designed to encourage and Barcode Scanner improve the exchange of information among stakeholders on urban design. Multimodal input devices enhance collaborative interaction in real-time, while Urban Planning visual feedback is given to all participants on a Collaborators Projector projected live video augmentation. Sketching, modifying the scene on site, in the space of the video augmentation encourages the individual expression and Figure 1. Mixed Reality Tent (left). Top view of tent (right). supports the exchange of information with interactive video-see through MR painting. However none of these visual support. Urban Sketcher is instrumental for systems is suitable for outdoor use. developing and sharing visions of future urban spaces by augmenting the real environment with sketches, Sketching the Urban Environment facades, buildings, green spaces or skylines. Urban planning is a melting pot for architectural visions The MR environment is set up outdoors in a tent in progress. Stakeholders – architects, politicians, (Figure 1) on the site of the urban reconstruction, so citizens and others – bring individual viewpoints into that a workshop for stakeholder participation can be the process. The objective is to refine these viewpoints arranged, which incorporates MR as well as and ultimately achieve mutual consent of all concerned conventional planning activities. parties. In order to obtain contentment, it is essential Tables with architectural scale models are established that the different points of view is are successfully tools in architectural communication, enabling an expressed and apprehended. observer to quickly grasp an overview of the planned The main application used in this process is Urban design from an exocentric view. Interactive tabletop Sketcher, an application that allows users to directly displays with MR capabilities [1, 4, 6] and tangible user alter the perceived reality by sketching in a video see- interface approaches have been developed to facilitate through augmented representation of the urban scene architectural education and also design negotiation (Figure 2). A tent is set up on site to provide shelter for [8,3]. Neumann et al. [5] describe Augmented Virtual participants and equipment. Inside the tent a video Environments combining VR models with live video feed from a controllable camera overseeing the textures. Grasset et al. [2] present an approach for reconstruction site is projected on a large screen. 3

relying on 3D modelling tools, while retaining the qualities of an MR scene registered in 3D.

Interface Experiences The Sketcher is ongoing development and has been tested in real urban reconstruction processes at the St. Anne clinic and on the TGI reconstruction site in Paris. The interface is divided into two sections: one is the view port, where the augmented scene is rendered with screen aligned 2D buttons for frequently used controls, e.g., for selecting the active tool for sketching, which can be used with various devices. The remaining controls and settings are displayed in a separate 2D Figure 2. An architect sketching on a transparent canvas. window, which is organized with tabs in order to allocate a large number of widgets for arranging Urban Sketcher enables virtual painting and modifying content in the scene as well as for configuring the the real world on the screen. The Sketcher also serves system behaviour. as a common focus for all the participants to The devices comprise a wireless game pad, which is concentrate on the space currently being discussed. It used for changing the viewing direction and zoom level has a multimodal interface for camera navigation, in the MR scene, a foot pedal for toggling selected painting, sketching and simple 3D modelling to support objects close to the user, an optically tracked pen the natural workflow of the urban planning process. which is mapped to operate the mouse cursor offering a In the conceptual phase of design, architects often variety of tools and finally a space device allowing the make quick sketches rather than accurate scale simultaneous change of the active objects position and drawings. Sketching has interactive qualities because it orientation. can be performed during a discussion, and informative Sketching in a scene means connecting for the user sketches can also be created by inexperienced people what she imagines with what is there. The advantage of to a certain degree. Therefore, we decided that a “live” sketching is that participants witness how the sketch-based interface would be most appropriate for sketch develops and changes in the scene happen the intended use. The envisioned sketching should (Figure 3). While sketching, participants create spatial follow the familiar conventions of 2D painting collages with several layers. They discover the programs, but the digital paint should be applied possibility to systematically work with layers and directly on 3D surfaces (canvas) in the video transparencies, thereby lending depth to the scene. augmented scene. In that way, the appearance of existing architecture can easily be modified without 4

Citations [1] F. Aish, W. Broll, M. Stoerring, A. Fatah, C. Mottram, Arthur - an augmented reality collaborative design system,Visual Media Production, 2004. (CVMP). 1st European Conference on, Vol., Iss., 15-16 March 2004 Pages: 277- 281 [2] R. Grasset, J.-D. Gascuel, D. Schmalstieg, Interactive mediated reality, AUIC 05: Proceedings of the 6th Australasian User Interface Conference, 2005 [3] H. Ishii, J. Underkoffler, D. Chak, B. Piper, E. Ben- Joseph, L. Yeung, Z. Kanji, Augmented urban planning workbench: overlaying drawings, physical models and digital simulation, ISMAR 2002: Proceedings of the International Symposium on Mixed and Augmented Figure 3. A user sketching in the video augmented view. Reality 2002, Pages: 203- 211, 2002 [4] H. Kato, K. Tachibana, M. Tanabe, T. Nakajima, Y. Conclusions Fukuda, A city-planning system based on augmented reality with a tangible interface, ISMAR '03: We have developed a first prototype of Urban Sketcher, Proceedings of the The 2nd IEEE and ACM International an MR application supporting a range of devices for Symposium on Mixed and Augmented Reality, IEEE collaborative multimodal interaction. Currently Computer Society, 2003 interactive view space modification, painting, sketching [5] U. Neumann, S. You, J. Hu, B. Jiang, I. O. Sebe, and simple content creation is possible in real-time. The Visualizing reality in an augmented virtual environment, spatial integration of canvas and 3D models were Presence: Teleoper. Virtual Environ, 13, 2, 222—233, MIT Press, Cambridge, MA, USA, 2004 designed with respect to usability. [6] M. Pilgrim, D. Bouchlaghem, D. Loveday, M. The continuous process with evaluation and refinement Holmes, A mixed reality system for building form and of the application will be continued. There is a demand data representation, In Proceedings of Fifth for tools allowing the acquisition of colours or textures International Conference on Information Visualisation from the video background of the augmented scene. 25-27 July 2001 Page(s):369 – 375, London, 2001 [7] M. Sareika, D. Schmalstieg, Urban Sketcher: Mixed Acknowledgements Reality on Site for Urban Planning and Architecture, In The authors would like to thank the other members of Proceedings of 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, Pages: the IPCity project (EU Grant FP-2004-IST-4-27571). 27-30, 2007 [8] J. Underkoffler, H. Ishii, Urp: A Luminous-Tangible Workbench for Urban Planning and Design, CHI, 386- 393, 1999

JITC3: Just-In-Time Augmented Reality Command & Control Center

Tom Furness Xianhang Zhang Abstract 3 ARToolworks, Inc. & Human Interface Technology Lab The Just-In-Time Command and Control Center (JITC ) Human Interface Technology Lab University of Washington is an Augmented Reality (AR) and mobile computing University of Washington Box 352142 based system for improving Situational Awareness P.O. Box 15539 Seattle, WA 98195-2142 USA during urban emergency response operations. The 3 Seattle, WA 98115 USA [email protected] JITC system integrates both synchronous and [email protected] asynchronous location-specific information updates Anirudhan Vijayakanthan from field responders carrying GPS-enabled 3 Daniel Belcher Human Interface Technology Lab SmartPhones, allowing the JITC AR equipped ARToolworks, Inc. & University of Washington coordinator to view and manipulate relevant Human Interface Technology Lab Box 352142 information in a coherent urban and geospatial University of Washington Seattle, WA 98195-2142 USA workspace augmented by images, audio and text. P.O. Box 15539 [email protected] Seattle, WA 98115 USA Keywords [email protected] Augmented Reality, Urban Emergency Response, Situational Awareness, Common Operating Picture, Mobile Computing, Portable Devices.

ACM Classification Keywords H.5.1 [Multimedia Information Systems]: Artificial, augmented, and virtual realities; J.7.1 [Computers in Other Systems]: Command and Control; K.4.1 [Computers and Society]: Human safety; K.4.3 Copyright is held by the author/owner(s). [Computers and Society]: Computer-supported CHI 2008, April 5 – April 10, 2008, Florence, Italy collaborative work. ACM 1-xxxxxxxxxxxxxxxxxx.

2

Introduction Gathering Unit, in the form of a GPS-enabled Emergency response operations demand that SmartPhone, will allow specified emergency support distributed groups of people work together both specialists to provide near real-time data to decision synchronously and asynchronously. Communication makers during a crisis. must take place between the responders in the field and the incident command and its various branches. Information must be shared to the level necessary to support quick decision-making, and enable the rapid recognition of emerging patterns in high-level geospatial data. As emergency situations grow in size and complexity, the cognitive load on any given branch of the incident command (operations, planning, or logistics) grows simultaneously. Maintaining a coherent and accurate Common Operating Picture (COP) in an emerging disaster situation is of critical importance to effective and timely emergency response [2]. A COP is made up of relevant information shared by more than one agent in a collaborative incidence response team. Presentation, format and proper maintenance of a COP is crucial to collaborative planning and gaining a high level of situational awareness.

The Just-In-Time Command and Control Center (or 3 JITC3) is an Augmented Reality (AR) and mobile figure 1. The JITC system architecture overview. computing based system that seeks to facilitate a dynamic, interactive, scalable-resolution Common System Architecture 3 Operating Picture in order to improve Situational The system architecture of the JITC consists of three Awareness. One key requirement for advanced interrelated components: 1) the Augmented Reality interfaces for emergency response is that they must Mobile Command and Control Center, 2) the support fluid teamwork among people with varied Communications Server, and 3) the Field Responder backgrounds and levels of expertise [5]. At the incident Mobile Data Collection Unit (figure 1). command level, the JITC3 facilitates mobile response by employing powerful Augmented Reality visualization At the core of our approach to Command and Control is tools that do not burden the cognitive load of the user an Augmented Reality Visualization of a geospatial or necessitate learning a new and complex piece of workspace enhanced by images, video, audio and text software. At the Field Responder level, the Mobile Data (figure 2). The AR command center allows one or more 3

users to set up a Mobile Command and Control Center Terrain models are loaded and street-grid information on-the-go, view geospatial data and Field Responder is converted and scaled. Building footprints are positions and annotations in near real-time. extruded to their apex heights, allowing the user to navigate the city in 3D, generating a compelling Augmented Reality merges virtual objects in the geospatial context for viewing the urban environment. context of the real world. Virtual objects are attached to fiducial tracking markers allowing the user to interact Interaction with the augmented geospatial with digital information in a natural manner: by simply representations is achieved in two ways: 1) by rotating picking up a tracking marker and moving it around, the and moving the physical card upon which the tracking 3D geospatial representations come to life. marker is located, and 2) by the use of a hand-held MagicLens device mounted to the end of a presentation remote-control. This device allows the user to filter different geospatial data layers, as well as zoom, pan, and select different objects within the scene.

Communications Server Component The JITC3 adopts the Client Server model. The Communications Server forms the intermediary link between the JITC3 AR interface and the Field Responder Mobile Units. The Communications Server also acts as a Database, WebService, and portal to other higher level chain-of-command and jurisdiction – such as municipal, state, and regional command centers.

We have opted for a centralized Communications figure 2. The AR visualization, viewing Seattle’s U-District. Server rather than a direct link between the JITC3 AR Interface and Field Responders. The primary All AR visualizations are generated with justifications for this arrangement are technical and OpenSceneGraph 1.2 for the renderings [3] and pragmatic: first, the sheer amount of uploaded data ARToolKit [1] for video-based position tracking. We are from the field responders in the form of images, video, currently using the latest version of the full integrated and text requires a 3rd-party database and processing OpenSceneGraph and ARToolKit package, known as system; second, we are striving to make the JITC3 as OSGART (ver 1.0) [4]. economical and portable possible, without the burden of a massive geospatial, relational and media database. The AR Visualization system begins by loading relevant As our research progresses, we plan to explore more geospatial data and converting it to 3D geometry. direct, Peer-to-Peer options. 4

Field Responder Units Component opportunity to test the JITC3 system in an actual On the ground, at the scene on the incident, and emergency response situation or drill. However, dispersed around the city, are the Field Responder simulating mock-up disaster scenarios is not beyond Units. These first responders may be medics, fireman, the realm of feasibility. police, or field observers, validating information and sending back status reports. Each Field Responder is Development of the JITC3 system is in its early stages, equipped with a GPS-enabled SmartPhone. yet much progress has been made to lay the foundation for a scalable, dynamic, robust, and powerful tool for The user interface was designed to be simple and maintaining a Common Operating Picture and intuitive. It is assumed that the Field Responders augmenting the situational awareness of team of (users) will be under both time constraints and incident commanders and first responders. tremendous amount of stress. It is necessary that the Field Responders login to the system before the mobile Acknowledgements device will be registered with the WebService on the The National Visualization and Analytics Center and the Communications Server. Once logged in, the mobile Department of Homeland Security provided primary device will automatically communicate its GPS funding for this project. The JITC3 group would like to coordinates, a time stamp, direction traveling, and the thank the NVAC and the DHS for their support. speed of travel every 3 seconds to the Communications Server, which will enter that data into a database for References use in the geospatial workspace rendered by the AR [1] ARToolKit. http://sourceforge.net/project/artoolkit. visualization system. The Field Responder also has the ability to send updates via text, picture, video and send [2] Liz Carver and Murray Turoff. Human-Computer e-mail. Interaction: The Human and Computer As a Team in Emergency Management Information Systems. Evaluation & Future Work Communications of the ACM, Vol. 50, No. 3, pages 33- At this point in the development of the JITC3, we have 38, March, 2007. limited our efforts to informal user-testing with non- specialists. However, we have continuously consulted [3] OpenSceneGraph. http://www.openscenegraph.com with professionals in the field and documented these interviews. We believe it is important to evaluate the [4] OSGART. http://www.artoolworks.com. JITC3 “in the wild” with its intended user-group: emergency management teams and first-responders. [5] James J. Thomas and Kristin A. Cook, Editors. Due to the makeup of the intended user group and the Illuminating the Path: the Research and Development unpredictability of the activity, we have not yet had an Agenda for Visual Analytics. IEEE, 2005.

Pedestrian Navigation in Virtual Environments

Andrew Park Paul Brantingham Abstract Interactive Arts and Technology School of Criminology In order to study how built urban environments Simon Fraser University Simon Fraser University influence human behaviour we can create 3D 13450 102nd Ave. 8888 University Drive virtual/mixed reality models of the real-world Surrey, BC V3T 5X3 Canada Burnaby, BC V5A 1S6 Canada environments and run experiments with real human [email protected] [email protected] subjects in these environments. This gives the experimenter much more control over the environment Tom Calvert Patricia Brantingham and eliminates the risks that may exist in a physical Interactive Arts and Technology School of Criminology environment. On the other hand, care must be taken Simon Fraser University Simon Fraser University to ensure that the virtual/mixed reality environment is 13450 102nd Ave. 8888 University Drive sufficiently immersive to ensure “suspension of Surrey, BC V3T 5X3 Canada Burnaby, BC V5A 1S6 Canada disbelief”. In this position paper we discuss the issues [email protected] [email protected] involved and describe a 3D virtual model of an urban environment that is being used to study the role of fear in a pedestrian navigation model.

Keywords Virtual environment, pedestrian navigation, CPTED, fear of crime.

ACM Classification Keywords I.6.4 [Model Validation and Analysis] J.m [MISCELLANEOUS] Copyright is held by the author/owner(s). CHI 2008, April 5 – April 10, 2008, Florence, Italy Introduction ACM 1-xxxxxxxxxxxxxxxxxx. Three dimensional virtual or mixed reality models of real-world environments can be used to study how built 2

urban environments influence human behaviour. Criminologist C. Ray Jeffery used the phrase, “Crime Experimental studies can be conducted with human Prevention Through Environmental Design (CPTED)” [6] subjects and the experimenter has much more control for the first time. The idea of CPTED is that careful over the environment than in a physical experiment. On design of environments can reduce possible crimes and the other hand, care must be taken to ensure that the fear of crime so that it improves quality of life. Another virtual or mixed reality environment is sufficiently strategy called the “broken windows” theory was added immersive to ensure “suspension of disbelief”. This to CPTED [7]. The theory says that if people do not fix position paper discusses the issues involved and broken windows, more windows will be broken. And the describes a 3D virtual model of an urban environment situation gets worse with worse crimes occurring, that is being used to study the role of fear in a including breaking into buildings. pedestrian navigation model. The relation between crimes and the environment was Crime Prevention Through Environmental Design further elaborated by Paul and Patricia Brantingham Human-constructed city environments play an who pioneered a new field, called “Environmental important role in peoples’ lives. The influence of an Criminology” [1,2]. By examining the time and the urban environment on human behaviour and life style place where crimes occurred, the spatial pattern of the has become a popular research topic. crimes and the behavioural pattern of the offenders can be discovered. One of the pillars of research on people in urban environments, Jane Jacobs, carefully observed people There has been much research that supports the and their activities in New York City. She identified ideas of CPTED, particularly, the relationship many different urban planning problems including between environments and fear of crime. Fear of crime crimes caused by bad urban design [5]. Her work has is not necessarily fear of real dangers, but rather the greatly influenced urban planners and architects and perceived fear of being victimized by possible crimes. caused them to see cities in a new perspective. Criminologists and environmental psychologists have studied how people, particularly the most vulnerable, Another person who also observed people in New York feel fear of crime in various environmental settings. was William Whyte, who was known as “people Their findings show that narrow walkways without any watcher”. The behaviour of people was systematically escape routes, hidden spaces created by corners, tall recorded using still cameras, movie cameras, and bushes, and threatening individuals generate fear in notebooks over a long period of time. His work shows people [3,10]. Lighting is another element that can architects and urban planners what works and what influence fear of crime [4]. does not [11]. Studies Using VE’s Following these ideas, criminologists began to think of CPTED research or fear of crime research has mainly new ways of reducing crime in urban environments. been done by traditional methods such as surveys, 3

interviews, case studies and experiments with human company in the Netherlands [9]. This tool focuses on subjects. For example, some research has been done visibility problems in a model environment. by asking human subjects to visit various campus sites. They were then asked to answer questionnaires Using a VE to Study Pedestrian Navigation regarding fear of crime and sense of security [8]. Other research has been done by showing human subjects We have developed a quantitative model of how photos of various city sites and asking them how they pedestrians navigate through an urban environment feel about the sites. The former kind of research could that creates fear of crime. The model chooses a path have endangered the human subjects. Because of the through the environment that minimizes passage close real possibility of danger, the research must be limited. to features known to generate fear [10]. In order to The latter kind of research loses dynamics and a sense validate this model we needed to study pedestrian of real-life situations. navigation using human subjects. However, it was difficult to find suitable locations that could be used to Virtual or mixed reality environments where the real- test our model. Observing real pedestrians in a fear- world is simulated have provided new investigative generating area would be difficult due both to ethical tools for social science research. The advantages of issues related to the risk and danger involved in the using such environments include: Control - it is much experiment and to our inability to control experimental easier to control and modify situations in a virtual variables. These reasons led us to construct a virtual environment than in a real-world environment; Cost - urban environment. We can achieve relatively good building a virtual environment is less expensive than realism using textures from photographs of real building real-world environments; Safety - by using a buildings and objects on the streets. We can easily virtual environment we can avoid any real danger, create and modify the layouts of a fear-generating area harm, or risk but still achieve the dynamics of a real- to correspond with the goals of the experiment. Control world environment. of experimental variables is easy in the VR environments and adding animated human figures These advantages must be balanced against the enhances the realism. Human subjects can freely concern that the virtual or mixed reality environment navigate the VR environments as if they were in the will not provide sufficient immersion for suspension of real-world environments. disbelief. This immersion depends on the technical quality of the virtual environment and on how the The VE has been developed using the Darkbasic subjects perceive the fear inducing features. Professional game engine. Three dimensional buildings were modeled in 3D Studio Max and textures created Computer simulation and virtual environments have from photographs taken in a well-known fear- only begun to be used for CPTED research. One tool generating area of Vancouver were mapped onto them. that we have found is Virtual CPTED developed by a A layout of the streets was designed in which we could create fear-generating features such as narrow 4

walkways, hidden spaces, bushes (or garbage they should choose. The results from our initial containers), and so on. Some animated human figures experiments are very promising. Most importantly, perceived as threatening were added to the VE. Human human subjects felt as if they had indeed been in the subjects can navigate the environment from a first target area and behaved accordingly. In post- person’s point of view as in first person shooter games experiment interviews, we reviewed the screens using a Nintendo Wii controller for navigation. recorded during the experiment and asked questions about levels of fear. Experiments have been conducted in which human subjects navigate from a starting position to a Conclusion destination position in the VE. On the way, there are The use of a virtual environment to test a model (either several decision points such as narrow/wide social or scientific) has many advantages in terms of passageways, streets with/without hidden space, cost, time, flexibility and safety. Our studies show that streets with/without dumpsters, streets with/without experiments in a VE can be a good alternative to those threatening individuals, and streets with a single in real-world environments. This approach promises to threatening individual or multiple threatening be particularly useful for urban planners and individuals. Subjects judge for themselves which route criminology researchers. [6] C. R. Jeffery. Crime Prevention Through Bibliography Environmental Design. Beverly Hills, Calif.: Sage, 1971 [1] P. J. Brantingham and P. L. Brantingham. [7] G. L. Kelling and C. M. Coles. Fixing Broken Environmental Criminology. Beverly Hills Calif.: Sage Windows : Restoring Order and Reducing Crime in Our Publications, 1981 Communities. New York: Martin Kessler Books, 1996 [2] P. J. Brantingham and P. L. Brantingham, [8] J. L. Nasar, B. Fisher, and M. Grannis. Proximate “Understanding and Controlling Crime and Fear of Physical Cues to Fear of Crime. Landscape and Urban Crime: Conflicts and Trade-Offs in Crime Prevention Planning, 26(1–4), 161–178, 1993 Planning," at pp. 43-60 in S. P. Lab (ed.) Crime [9] J. Oxley, P. Reijnhoudt, P. van Soomeren, C. Prevention at a Crossroads. Cincinnati: Anderson Beckford, A. Jongejan, and J. Jager. Crime Opportunity Publishing Co. 1997 Profiling of Streets (COPS): A Quick Crime Analysis – [3] B. Fisher and J. L. Nasar. Fear Spots in Relation to Rapid Implementation Approach. Garston, Watford.: Microlevel Physical Cues - Exploring the Overlooked. BRE Bookshop for BRE, 2005 Journal of Research in Crime and Delinquency, 32(2), [10] A. Park and T. Calvert, Validating a Social Agent 214–239, 1995 Model for Pedestrians with a Virtual Environment, Proc. [4] Hanyu, K. Visual Properties and Affective Appraisals Computer Animation and Social Agents 2007 Conference in Residential Areas After Dark. Journal of (CASA 2007), Hasselt, Belgium, June 2007. Environmental Psychology, 17(4), 301–315, 1997 [11] W. H. Whyte. The Social Life of Small Urban [5] J. Jacobs. The Death and Life of Great American Spaces. Washington, D.C.: Conservation Foundation, Cities. New York: Random House. 1961 1980

Visual Vectors and Public Interfacing

Pamela Jennings Abstract [email protected] “Distance Movement: If I could live here, I’d be home by now” was a proposal for an electronic arts festival in 2006. The project was envisioned as a visual etude – a counterpoint between two movements. The first being movement patterns of peoples walking through the Plaza de Cesar Chavez water fountain in San Jose and the second, the CALTRANS live traffic cameras that populate the Bay Area freeway system. The goal was to open the door for a broader demographic of people to engage in the wonderment of repurposing information technologies for art, fun, and critical concerns. The project aligned with my research in human to human interaction mediated with human centered computing applications that facilitate shared experiences and learning in public spaces. The proposed project united my past life as a modern day gold digger in the Silicon Valley corporate research industry, my memories of the area after I left, and a desire to explore computer vision and advanced software algorithms. Repurposing ubiquitous and surveillance technologies for applications that promote shared engagement and learning is a powerful trend in contemporary digital media production. While the project has not been made, this paper serves as a Copyright is held by the author/owner(s). catalyst for its installation in other urban locations. CHI 2008, April 5 – April 10, 2008, Florence, Italy ACM 1-xxxxxxxxxxxxxxxxxx. 2

Keywords commuter traffic, lack of money and time to move to Public Interfacing, surveillance, movement, discourse, Oakland from Palo Alto and subsequently East Palo Dykstra’s algorithm Alto. A place that I fantasized would provide a community better attuned to my lifestyle. The work ACM Classification Keywords speaks to possibility and desire, work and play, waiting H5.m. Information interfaces and presentation, H.5.1 and living, getting “there” and being “here,” nearness Multimedia Information Systems, J.5 Arts and and farness. Humanities The interactive real- Introduction time visualization We live in an era where the nine to five work ethic is no was to be a visual longer engaged. Rather people hungry with the desire etude -- a to innovate and reap the benefits work ten, twelve, juxtaposition sixteen hours a day. (Does this sound familiar?) I between two real- know, because I did it and still do. It wasn’t until I left time studies in Silicon Valley, that I was able to stop and reflect on the human motion and absolute beauty of the Bay Area. This was a pause for commuting traffic respite that I rarely allowed myself time to do except projected on a wall when participating in a six AM Boot Camp (Victory Lap) or surface in a large exercise program with other type AAA work-a-holics on outdoor public plaza. the Stanford Campus. About once or twice a month, The primary visual we climbed the steep hill behind the Stanford campus rhythm was to to the dish, an antiquated radio-telescope that dots the emphasize the Palo Alto foothills. The reward was to be greeted by the spaces, or distances, early morning rising sun. It wasn’t until I left the area in-between people as that I appreciated the beauty of where I lived. The they encounter, Fig. 1 Sketch of the project sweet smell of citrus and eucalyptus was replaced by engage, or simply visualization. The outer ring an early morning suspicious odor, similar to a wet contains live streaming video from walk past each other sneaker and the boundless Pacific Ocean with toxic CALTRAN traffic cameras. in the Frank Taylor river sludge of Pittsburgh. “Distance Movement: If I Fig. 2. The grid of the Frank Taylor fountain at the Plaza could live here, I’d be home by now” was to be a real- water fountain. de Cesar E. Chavez time projected visualization of patterns of movement in in San Jose. (Fig. 2) commuting and leisure. The project concept was a The secondary visual reflection on the six years I worked in heart of Silicon rhythm was to be a live video feed from the CALTRANS Valley at two major research think tanks. The title freeway traffic cameras that loop around the San spoke to a desire, curtailed by the incessant Bay Area Francisco Bay Area. (Fig. 3) The CALTRANS visual feed 3

would form a ring around the plaza visualization, something he vowed never to miss, regardless of the encapsulating the human movement in the reality of tasks on his work list. Sam enters into the “Distance the commuting - car culture. (Fig. 1 & 2) Movement” computer vision field of vision and notices a group of people watching a large projection mounted on a scaffold structure. The projection has two concentric circles. The outer circle contains live video feed from the CALTRANS traffic cameras. Sam, remarks to himself that he’s glad he can conveniently take the VTA home to Mountainview and doesn’t have to wrestle bumper to bumper traffic. He looks a little closer at the inner circle and sees a couple of round icons, nodes that are moving, stopping, changing direction and disappearing off the edge of the inner Fig. 3 CALTRANS Freeway Traffic Cameras Video Stream, captured circle. The are lines drawn between the icons that 12/18/2005 on a rainy day. change their thickness and transparency as the icons

move closer and further away from each other. Wow, Scenario: he thinks, “that looks really tight.” And then he stops, Discovery of Self among Many Nodes he notices that one of the icons in the visualization that Sam, a recent graduate of University of Washington is close to his actual physical location, also stopped. He and new employee at Adobe Systems has just ended an steps forward a few steps, and the icon moves again. exciting, yet exhausting day at work. His life-long Other people on their way to the VTA station walk past dream had been to work for a major company on him. Some notice the systems and others don’t. Sam creativity support tools for artists and designers and he sees that the transparency and thickness of the lines started the job a month ago. This was a very long day connecting one icon to another change as people in the at the end of a long week. His work team is developing plaza walk pass. Sam looks at his watch and sees he a new tool for 3D modeling. And as is typical to the has fifteen minutes to spare before the next train and work ethic of Silicon Valley, he’s been putting in fifteen decides to stop and watch the children running through hour days. Sam loves to walk through the Plaza de the fountain. Cesar E. Chavez at the end of the day when groups of school children joyfully run through the twenty-two Dividing Lines in Urban Development ground mounted water fountains. Sometimes they’re The history of the development of downtown San Jose in bathing suits, and other times shorts and t-shirts. is a poignant example of the culture and politics and He wonders what the conversation must be at home for the transformation of an urban environment. Architect some of the children when they enter into the door developer Frank Taylor’s reputation as a colleague and drenched from head to toe. But today, Sam has other an architect was meager. things on his mind. He’s late for his martial art class, 4

“Taylor was a mediocre architect with bland tastes, designated as a civic open space in 1849. The plaza flawed urban planning ideas and poor project has served as site for Hispanic heritage activities, public management skills. He held his cards close to his hangings, and in the 1870’s a boundary that defined chest, running a public corporation like a private the edge of the Chinese community on the east side of business….he strategically sprinkled projects around to Market Street. As a dividing line of dissention, the city leaders threatened to tear down the park. Sadly, a fire keep land values – destroying Chinatown saved the park, for there was no and their associated longer a community of people that the majority culture property taxes – viewed as undesirable. The park was selected as the rising. As rents shot home for City Hall up until the building’s demolition in up in the gentrified 1858. It was renamed Plaza de Cesar E. Chavez in areas, small 1933 after the well known and influential community businesses and lower- organizer and founder of the United Farm Worker’s income residents Union. [2] would be replaced by Fig. 4 A rather romantic painting out-of-town corporate Visual Vectors and Public Interfacing of Major Thomas McEnery gazing inquisitively at a shoeless players and monied The “Movement Distance…” visualization evolves over Architect Frank Taylor who is downtown dwellers…. time with interactions lasting as long as the person dancing in the water of the Cesar Many of the things stays within the computer vision hot zone. Visual Chavez plaza fountain. that bring joy to city properties of the connecting vector would vary Evidentially this work of art hangs life – pocket parks, depending upon the distance of one node to another. in the McEnery convention center. murals, bike lanes, For example, a child playing in the fountain, running good public art, through the water spouts with her friends may interact eclectic stores, funky old buildings, adequate parking, with the system for a half hour or longer. A business and affordable housing – are missing. And they are woman, who cuts through the Plaza on her way to the absent because of a narrow vision that tried to build an Paseo de San Antonio VTA station, will have a quick economy around conventioneers, office workers and interaction and presence in the projected movement suburban customers, rather than emphasize a urban visualization. Her vector will be bright along her path tapestry of residents, students, young adults, artists with a quickly fading trail like a shooting star across the and entrepreneurs.” [1] night sky. The child’s path may vibrate with bright spots of intensity that line his path of movement. The Taylor had the foresight to preserve the Plaza de Cesar visual qualities of their connecting vectors change as E. Chavez. The plaza and surrounding area was the distance between them shrinks and grows larger. established in 1797 as a Spanish settlement. It was transformed into an oasis of natural spender after the U.S. takeover of San Jose and was eventually 5

Tech Realization “Distance Movement” brings to light ethical questions The project would be based on a software program I about surveillance and the act of turning the private wrote several years ago called “Constellations,” an gaze of the camera around for the public to embody, animated greedy search algorithm. The nodes manipulate and own, if even for a fleeting moment. represent people recognized by a computer vision system that is optimized for tracking people in Conclusion unstructured environments. Where as the project described in this position paper has not been fully realized at the time of the writing of this paper, it brings to the foreground several interesting technical, interactive, and theoretical concepts and questions. Technically it looks at the use of a well known AI search algorithm for dynamic analysis of movement in public spaces. The interactive component looks at methods for highlighting our awareness of others in public spaces by enabling people to use their body and movement through space as a paint brush that reveals their spatial relationship to others in that space. Theoretically, the project incorporates notions of space, intersubjectivity and play

as platforms for developing engaging human centered computing experiences. Finally, ethically, the project Fig. 5 Screen shot of the “Constellations” path converts the omnipotent gaze of the covert surveillance finder application. camera into a tool for revealing patterns of interaction Technology as a Social Enabler that may be engaging, informative, and enlightening. Social enabling technologies and applications are designed to encourage people to explore the spaces References that keep distances between. This includes the [1] Pulcrano, D. (1999). “Who Built Downtown?” Metroactive News and Issues, development of human centered technologies that http://www.metroactive.com/papers/metro/02.25.99/e examine, construct and/ or deconstruct the notion of ditorslicense-9908.html . community, affinity through discourse. Whereas, my [2] Carlson E. (2006). Soft Underbelly of San Jose, project “Constructed Narratives,” is a tangible social http://www.sanjose.com/underbelly/ . interface for facilitating verbal discourse in public spaces, “Distance Movement…” explores the facilitation of movement discourse in public spaces.

The “Where?” of Mixed Reality: Some Guidelines for Design

Rod McCall Richard Wetzel Abstract Collaborative Virtual and Collaborative Virtual and This paper presents some early design concepts for Augmented Environments Augmented Environments urban mixed reality (MR) environments. The concepts Fraunhofer FIT Fraunhofer FIT are based on preliminary analysis of an urban MR Sankt Augustin, 53757, Germany Sankt Augustin, 53757, Germany game. The objective is to explore how to create a [email protected] [email protected] unified user experiences through a combination of real and virtual elements. Iris Herbst Collaborative Virtual and Keywords Augmented Environments Mixed reality, place, presence, guidelines, games Fraunhofer FIT Sankt Augustin, 53757, Germany ACM Classification Keywords [email protected] H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. Anne-Kathrin Braun Collaborative Virtual and Introduction Augmented Environments Mixed reality (MR) environments combine real world Fraunhofer FIT and computer generated elements to create a multi- Sankt Augustin, 53757, Germany sensory user experience. Indeed as noted in a recent Anne- panel at the Presence 2007 Workshop, systems similar [email protected] to TimeWarp (discussed later) force us to question issues such as place and presence [1]. Such questions include “Where am I?” and from the system designers perspective “Where do I want the users to be?”. Therefore as noted by Ciolfi and Bannon [2] it is necessary to explore the real elements of mixed reality as much as the virtual aspects.

This paper focuses on some early results from a study acknowledged that being aware of changes may also be into a visor-based mixed reality game presented in the a desirable property. This definition has many form of concepts which are specifically designed to commonalities with Gibson's concept of affordances [4], explore the relationship between real and virtual where he sees no difference between real or virtual. elements of mixed reality games. Instead affordances arise due to the user’s perception of the features in the environment. It has been argued Presence and Place by some that through these affordances the user interacts in the environment and thus feels present. By Unified Experience of exploring unified experiences and switches it also People, Activities, Objects becomes important to consider aspects such as sense of place [5][6] which often consist of disparate related elements such as people, activities, and meanings as well as relationships between self, environment and others.

Virtual Mixed Reality Timewarp Reality Reality

Figure 1: The unified experiences of mixed reality are created by through balancing the relationship between real and virtual aspects of the space.

Figure 1 is drawn from the work by Milgram [3], which is often referred to as the presence continuum. However rather than focusing on sense of presence its objective is to highlight where people feel present, for example more in the real or virtual experience. In contract a unified sense of presence occurs when people feel constantly within a new experience for the duration of the time they are intended to be there. For example the real and virtual elements combine in such Figure 2: A Heinzelmännchen in Cologne. experiences to make people feel as if they are TimeWarp is a mixed reality game which takes place in genuinely in a new time period, or a new place. Rather the City of Cologne. The objective is for the players to than experience switches in sense of presence between travel to different time periods and visit local characters real and virtual experiences – although it is known as Heinzelmännchen (Figure 2). The players

walk around within the real environment and the Avoid encouraging the user to focus on virtual system augments the real location with virtual elements when near potentially dangerous real characters and objects as well as sound. aspects e.g. road crossings.

A number of study methods were used during the (4) Design appropriate paths through the game, these included questionnaires based on MEC [7], environment the Social presence questionnaire [8] and the Place probe [9]. In addition the participants were observed Utilise real elements such as paths to provide a rich and/or videotaped during the experience. They were narrative within the game. also interviewed. Much of the system was developed prior to the evaluators being involved, hence the design (5) Understand the Locale can be through of as being separate from the evaluation phase. Spend time understanding and planning actions etc which are suitable within the locale. Understanding the Real World Based on our early results from the study we propose (6) Interaction with Others some concepts for the development of visor based MR systems. The proposed guidelines share many Try to include non-game participants in the similarities with the work of Davidsson et, al [10], experience, as well as other players. however they focus much more on the combination of real and virtual elements. (7) Seamful Design

(1) Understand Attention Allocation Issues Aspects of the environment may reduce GPS availability, where this is the case utilize faults as Virtual objects should be carefully designed so as part of the game [11]. to ensure attention is allocated towards the most relevant part of the experience. (8) Use a combination of real and virtual objects (2) Simplify the Interaction Scheme Encourage interaction with real elements such as Keep the number of interaction types to a minimum buying a drink or food. and introduce training scenarios which also form part of the game. (9) Provide a continuous experience

(3) User Safety The emphasis should be on creating experiences which last for the duration of the game, and not

ones which constantly break due to lack of virtual to make use of real spaces, people and objects in order elements or technical problems. to create a unified experience.

Conclusions Acknowledgements The guidelines presented here do not claim to be an The authors acknowledge the assistance of other exhaustive list but rather a starting point from which to members of the IPCity project and those who took part consider the design of MR experiences. They are in the study discussed here. IPCity is partially funded intended to highlight the importance of considering by the European Commission under grant number: FP- reality when building MR experiences, in particular how 2004-IST-4-27571. [7] Vorderer, P., Wirth, W., Gouveia, F. R., Biocca, F., Saari, References T., Jäncke, F., Böcking, S.,Schramm, H., Gysbers, A., Hartmann, [1] McCall, R., Wagner, I., Kuuti, K. and Jacucci, G. Urban T., Klimmt, C., Laarni, J., Ravaja, N., Sacau, A., Baumgartner, T., Mixed Realities: Challenges to the traditional view of presence. & Jäncke, P. MEC Spatial Presence Questionnaire (MEC-SPQ): 10th International Workshop on Presence. Barcelona, Spain. Short Documentation and Instructions for Application. Report to (Panel paper). p159-163 Eds. Moreno, L. International Society for the European Community, Project Presence: MEC (IST-2001- Presence Research. ISBN 0-9792217-1-4 37661). (2004). Online. Available from http://www.ijk.hmt- [2] Ciolfi, L., and Bannon, L.J. Space, Place and the Design of hannover.de/presence Tehcnoligically- Enhanced Physical Environments. In: Space, [8] Bailenson, J.N., Blascovich, J., Beall, A.C., & Loomis, J.M. Spatiality and Technology. Eds: Turner, P & Davenport, E. p217- Equilibrium revisited: Mutual gaze and personal space in virtual 232. (2005) Springer Verlag. environments. Presence: Teleoperators and Virtual Environments, 10, (2001). p583-598. [3] Milgram, P., Takemura, H., Utsumi, A. and F [9] Benyon, D. Smyth, M., O’Neill, S., McCall, R. and Kishino. "Augmented Reality: A class of displays on the Carrol, F. The Place Probe: Exploring a Sense of Place reality-virtuality continuum". SPIE Vol. 235 1(1994) in Real and Virtual Environments. Journal of Presence: pp282-292. Tele-operators and Virtual Environments. 15, 6, (2006) [4] Gibson, J.J. The Ecological Approach to Visual Perception. p668-687. (1979). Erlbaum, Hilldale, NJ. [10] Davidsson, O., Peitz, J., Björk, S., (2004) Game Design [5] Relph, E.(1976) Place and Placelessness. Pion Books, Patterns for Mobile Games. Project report to Nokia Research London. Center, Finland. [6] Gustafson, P. Meanings of Place: Everyday experience and [11] Chalmers, M. Seamful Design and Ubicomp Infrastructure. theoretical conceptualizations. Journal of Environmental Proceedings of Ubicomp 2003 Workshop at the Crossroads: The Psychology 21. (2001). p5-16 Interaction of HCI and Systems Issues in Ubicomp

Is “Presence” Important in Mobile Map Interaction?

Antti Oulasvirta Abstract School of Information Presence, the sense of “being there instead of here”, University of California, Berkeley has become the driving design goal and topic in virtual reality research. The key idea of mixed reality Helsinki Institute for Information interaction (MRI), by definition, is interaction “in” the Technology HIIT blending of virtual environment (VE) and “real” or physical environment (PE). Mobile devices complicate Sara Estlander MRI, because digital information cannot be directly projected onto the physical world. We ask if the notion Figure 1. Real world scene and a Helsinki Institute for Information view on a mobile device to a 3D city Technology HIIT of presence is important in such circumstances. The use model (top). Below, the phone (Nokia of 3D mobile maps for identifying correspondences N93) used in this study. Giulio Jacucci between buildings in VE and PE was examined. No Helsinki Institute for Information signs were found of users exhibiting presence in VE. Technology HIIT Instead, 1) they constantly switch between the two viewports, 2) they are aware of PE contents and 3) they interact in a way that tries to align the two spaces. Mobile MRI is better understood in terms of a dyadic process constituted by the user’s agency and cognitive capacities being “split” between VE and PE. The question is how actions in the two environments can cohere in the pursuit of a single task goal.

Keywords Presence, mixed reality, 3D interaction, mobile maps Copyright is held by the author/owner(s). CHI 2008, April 5 – April 10, 2008, Florence, Italy ACM Classification Keywords ACM 1-xxxxxxxxxxxxxxxxxx. H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. 2

Introduction Figure 2 describes this view in a case where the two Presence has been defined as the sense of being planes can be conceived as Euclidian spaces, as in the physically transported to a remote workspace and as a case of mobile maps. “perceptual illusion of non-mediation” [1]. When “present”, “perceived self-location and realization of There are two major a priori categories of how users action possibilities are connected to a mediated spatial can be “present” in a mixed reality task: Figure 2. The MR mapping environment; mental capacities ... bound by the problem: how to understand the mediated environment instead of reality” [5]. Presence 1. “Unal presence”, presence in one world at a time: referential relationship between two has become the prime approach of psychology to the a. Physical environment (PE) spaces. In mobile MR, the user has design of virtual environments and basically to all to mentally construct the referential b. Virtual environment (VE). media where the goal is to surpass the limits of human relationship between the virtual and 2. “Dual presence”, presence in two environments: the physical, because viewports to sensory capacities. the two spaces cannot be projected a. Simultaneously in PE and VE onto each other. One may ask what the role of this notion is in newer b. Sequentially in either PE or VE at a time. forms of technology that have been deliberately All four options are possible a priori, although they do designed to leverage the physical presence of the user, presume different cognitive mechanisms underlying the or, conversely, to blur the boundary of the real and the feat of achieving presence. virtual. Milgram and Kishino [2] defined mixed reality as “the merging of real and virtual worlds somewhere Regardless of which type of presence there is, the along the 'virtuality continuum' which connects effects H1-H5 mentioned in the sidebar should be completely real environments to completely virtual observed if presence is important in this task at all. As ones.” Mixed reality systems either augment the virtual a source for these inferences, we refer to Lombard and world with physical features or augment the physical Ditton’s [1] well-known review. with virtual ones. Mixed reality interaction (MRI) is a broader concept that involves tasks in which actions Case: Localizing Objects with a Mobile Map and processing of information takes place in both the As a case we look at data from a field experiment [4] PE and the VE. Thus, by definition, there is always an on how users localize PE buildings with 3D and 2D aspect of objects being somehow “divided” “mapped” mobile maps. This is a MR task that explicitly requires or “shared” between two planes. mapping of two planes. Technical details of the study are given in the original article. Of the system and Mobile devices complicate MRI in the sense that they Method, it is worthwhile to know that: technically cannot be based on the projection of digital Figure 3. A roof-top view (top) and information onto physical world. Moreover, it is a track-based maneuvering scheme ƒ The design of the 3D map instantiates several physiologically impossible to (foveally) process a small (street following, bottom). The known principles of good virtual reality design, default view was the street-level mobile device display and surrounding scene including photorealism and maneuvering assistance. view of Figure 1. simultaneously. We call this the “mapping problem”. 3

IMPLICATIONS FROM PRESENCE ƒ The default mode of movement and camera in 3D Assessment of Evidence RESEARCH is based on a street-level view. Other views and The evidence speaks against the idea that presence is maneuvering schemes are optional (e.g., see Figure 3). important in mobile MRI. The data is reported in its (H1) Realism in VE is beneficial: entirety in the original article; here, we concentrate on Realism of the virtual model should ƒ There is a 2D map as a comparison condition that observations related to the hypotheses. increase the sense of presence. does not have any “presence-enhancing” features. The map is a simple cartographic street map. (H2) Consistency of switching ƒ The task is a pointing task: a target is indicated in ƒ First, users are perpetually conscious of aspects of behavior: As a behavioral indicator PE while interacting with VE, and vice versa. The of presence, users should not the VE (map, 2D or 3D), and the user is asked to point exhibit abrupt switches between VE in the direction of the target in the PE. The real target predominant strategy for solving the task, in both 2D and PE, but either switch in PE may or may not be visible. The user can move her and 3D maps, was finding a point that maps a location consistently between the two or body freely during task. in the VE to one in the PE (hereafter: a reference stay concentrated on one point). One can infer the target’s direction by environment for a longer time while ƒ 16 subjects performed 24 tasks, spending about estimating the angle difference Δ between the target disregarding stimuli in the other. 1.5 h on the field with an experimenter. and the reference point, and the target can then be ƒ Verbal protocols, multi-video data, interaction logs, (H3) Mimicking: Users achieving pointed at Δ degrees to left or right from the reference presence should generally prefer workload measures, performance measures, and point, assuming that one knows one’s own position in maneuvering in 3D in ways that various background information are collected. See relation to the reference point. One can achieve this mimic real world movement. Figure 4 for an illustration of the multi-camera setup. also without locating oneself on the map, by

(H4) Presence is useful: An “sandwiching” the target with two external reference increased sense of presence should Cue type 3D 2D points. (This did not always work: In verbal protocols generally improve performance. Known landmarks Very often Often we found frustration for searched-for PE visual features Building shapes Often - not found in VE and vice versa.) Now, importantly, this (H5) Suspension of disbelief: Façade details Often - tactic, by definition, entails keeping in mind a Presence should be associated with negligence of the physical world. Façades (whole) Often - description of the target which represents the other Relative directions Often Sometimes environment. Thus, we believe that “suspension of Street names Often Very often disbelief” cannot be in play here. In contrast, users are Street crossings Sometimes Often constantly evaluating similarities between the worlds. Blocks, or part of blocks Rarely Sometimes ƒ Second, photorealism of the model does not help, it Parallelism of streets Rarely Rarely distracts. Table 1 below shows some of the cues users Cardinal directions Very rarely Sometimes relied on for finding a reference point. An important Store/office names - Rarely observation we made is that in 3D use, the search Street number - Rarely Figure 4. The recording equipment. proceeded mostly by scanning buildings that surround The user carries one camera on his Table 1. A catalogue of cues used and their frequencies. the target in the VE, for example to spot a yellow chest (1) and two attached to the Based on analysis of think aloud data of 75 tasks. building in the midst of gray ones. We observed that mobile device (2 and 3). The users cannot operate with a photorealistic model moderator follows one step behind, shooting with a wide angle lens (4). 4

efficiently because their attention is constantly guided for visuo-spatial working memory span). The high span to cues that may be salient in the VE but are ineffective group was 16% quicker and used Switch perspective and undiagnostic in the PE. In this respect, 2D worked 26% more in 3D. They gazed at the environment much more efficiently and led to faster task completion 11.8% less per task, although the two groups’ times. Thus, contrary to findings of presence research, frequency of gazing at the environment was at the realism is not needed. 2D maps as representations rely same level. Similar differences were not found in 2D on centuries of experience on effective symbols and use. These results suggest that 3D performance, but layouts of a map. not 2D performance, may be dependent on visuo- ƒ Third, movement in 3D is not “realistic”. In spatial working memory span. Due to more fragmented principle, if the users so wanted, they could have viewing of VE, and less help from bodily strategies, 3D immersed themselves in the VE to a view that users have to keep in memory more locations of corresponds to their real street-level view and offers interest in order to solve the problem. rich visual detail. This was the default view. However, ƒ Sixth, bodily strategies in PE help performance. We to mitigate the “photorealism trap”, many 3D users were surprised by the general tone of the results Figure 5 Time spent looking at would turn on the Tracks feature see the street names concerning bodily conduct: 2D rather than 3D tasks the PE was higher in 2D than in to use those as cues. Some also learned to ascend to involved more efficient use of body and gaze. 2D users 3D. Conversely, 3D users were the rooftop view and rotated there in search of statues, turned their upper bodies more in search of cues like more immersed in VE than 2D parks, recognizable buildings and rooftop logos of street names and crossings. They deployed gaze users. Taken together with the companies. If this did not work, they “flew” around in significantly more effectively to find cues like parallel fact that 3D users were slower, we conclude that preferring the area above the target. Thus, realistic movement did streets. They tilted their heads and rotated their interaction with the VE hampered not seem important either. devices in their hands more than 3D users. Their performance. Good strategies ƒ Fourth, more “immersion” in VE means worse walking was significantly more efficient. 2D users did allow users to concentrate on the ego-centric alignment more, even when the target was boundaries and cross-overs. performance. Time spent looking at the PE was higher in 2D than in 3D use (Figure 5). Conversely, 3D users remote and not visible in PE, by relying on the zoom were more immersed in VE than 2D users. During that out function to see the current position and the target time, they traveled the same distance in VE as 2D POI’s position in the same view. The crux of these users. Taken together with the fact that 3D users were strategies may be that they allow 2D users to avoid slower, we conclude that preferring interaction with the mental manipulation and rely more on perception in VE actually hampered performance. Good strategies in solving the task. We see a parallel to experienced Tetris MR tasks allow users to concentrate on the boundaries players’ tactic of rotating a piece in order to see, rather and cross-overs of VE and PE. than mentally simulate, its fit. This means that it is more important to align bodily engagement in two ƒ Fifth, navigating in a 3D VE increases working environments, PE and VE, rather than be immersed in memory load. We divided subjects into two groups, acting in only one environment, the VE. according to a median split for their Corsi score (a test 5

SUMMARY OF EVIDENCE performance. More attention to boundaries and cross- AGAINST PRESENCE Conclusions overs of the two environments should be associated (1) Users have to be conscious of Work on scientific concepts proceeds in a cycle of two with improved performance. However, our analytical aspects of PE while interacting with endeavours: charting the boundaries of existing observations on the task and cognitive requirements in VE. Being aware of the other environment's contents runs concepts and proposing new concepts. This paper has MR tasks hint that a sense of presence is unlikely; the counter to the very idea of presence proposed that the notion of presence may not be useful distracting elements are too numerous. as “sense of non-mediation”. in all areas of mixed reality. Our examples indicate that (2) Photorealism of the model does when operating at the augmented reality end of the We leave it as a challenge for future research to come not help but distracts and misleads. spectrum, presence might not be a key issue, neither up with a more descriptive concept. Importantly, it Here, realism does not increase for design nor as a phenomenon describing interaction. should capture the synergistic relationship between performance. This runs counter to actions in and perceptions of the two environments. one finding of VR research. The results point out that comments like the following (3) Movement in 3D is not “realistic”. Users prefer 2D-like on 3D mobile maps must be overly optimistic and Acknowledgements th strategies that are not realistic but detached from the requirements of action arising This work has been supported by EU 6 Framework that allow faster and more efficient interacting with mobile maps: “the most positive projects IPCity and PASION and by the Academy of finding of cues. feature was found to be the possibility to recognize the Finland project ContextCues. The m-LOMA project was (4) More “immersion” in VE means features in the surrounding environment, which funded by EU InterregIIIA. worse performance. Contrary to provides a link between the real and virtual worlds. This main claim of presence research, removes the need to map-read, which is required when References even if users were present, it would [1] Lombard, M. and Ditton, T. At the heart of it all: not help their performance but attempting to link your position in the real world with a The concept of presence. Journal of Computer-Mediated make it worse. Efficient strategies 2D map, hence the VR interface offers an effective way Communication 3, 2 (1997). rely on awareness of both VE and to gauge your initial position and orientation” [3]. Our PE and thus are associated with findings lend support to a less Cartesian and more [2] Milgram, P. and Kishino, F. A taxonomy of mixed switching behavior and focus on PE reality visual displays. IEICE Transactions on constructivist conception of human perception, where which is more information rich. Information Systems 77, (1994), 1321-1329. the person is seen as an active, intentional actor in an (5) Navigating in the VE in a 3D environment that offers different resources for actions. [3] Mountain, D., Liarokapis, F. Mixed reality (MR) street-level view increases working interfaces for mobile information systems. Aslib memory load in this MR task as it Proceedings, Special Issue: UK Library & Information increases the discrepancy between We acknowledge the tentativeness of these Schools, Emerald Press 59, 4&5 (2007), 422-436. the VE and PE. Users rather go for conclusions. One can state that presence could not [4] Oulasvirta, A., Estlander, S. and Nurminen, A. direct perceptual matching or ego- even emerge in mobile VE interaction, because of the centric alignment to avoid this. Embodied Interaction with a 3D versus 2D Mobile Map. small display, limited and slow interaction, poor visual Accepted for publication in a forthcoming Special Issue (6) Bodily strategies in PE help quality, and other factors. This may be the case. The on Mobile Spatial Interaction for the journal Personal performance. It is more important and Ubiquitous Computing. to align bodily engagement in two desiredatum for testing our claim would pair standard environments, PE and VE, rather measures of presence to those of performance. Our [5] Wirth et al. A process model of the formation of than be immersed in acting only in hypothesis is directly operationalizable and testable: spatial presence experiences. Media Psychology 9, 3 only one environment, the VE. higher presence should be associated with poorer (2007), 493-525.

Mixed Realities in China’s Internet Cafes

Silvia Lindtner Introduction Department of Informatics In our recent work we have investigated playful activities in University of California, Irvine urban environments. In particular, we have examined how Irvine, CA 92697-3425, USA people transition between play and other activities such as [email protected] work, tourism, and sports [6][8]. In these playful settings we found that technology gains relevance not only as an Bonnie Nardi infrastructural tool to support a particular activity, but as an Department of Informatics instance of a socio-technical system dynamically shaped by University of California, Irvine the interplay of the virtual and the physical. Whereas Irvine, CA 92697-3425, USA approaches within Ubiquitous Computing and Game Culture [email protected] Studies have challenged the divide between virtual and physical game environments, e.g. [1][2][11], studies of online games within HCI tend to focus on interaction and collaboration within the virtual realm [4][5]. Little attention has been paid to the intersection of one into the other within a particular local context.

We believe that much can be learned from a perspective that looks at online game worlds as negotiated and culturally and socially embedded entities within activity outside of the game. For example, millions of people play the online role playing game World of Warcraft (WoW). The single largest group of WoW players is in the People’s Republic of China1. Copyright is held by the author/owner(s). The game is based on the same logic, strategies, and CHI 2008, April 5 – April 10, 2008, Florence, Italy interaction mechanisms for American and Chinese game ACM 1-xxxxxxxxxxxxxxxxxx.

1 http://www.blizzard.com/press/070111.shtml 2

servers. However, the cultural contexts that shape has been studying playful interactions in work environments understandings of the game are quite different. We propose [6] and public urban environments [8]. We will continue to to look at online activities in games like WoW as mixed explore dynamics in playful settings and between the virtual reality phenomena that are tied to their spatial, cultural and and physical realm, with the aim to inform system design. economic contexts. By mixed reality we refer to the blending between virtual and physical, but also to the mix of social Findings realities when people transition between different spaces In what follows, we present a subset of our findings relevant like work and play. To understand how technology is to the theme of this workshop. Compared to players in the incorporated into daily routines and habits we have to make United States and Europe, who tend to play games like sense of the mixed realities within existing technology sites. WoW at home, Chinese players play mostly at Internet cafes (Figure 1 and 2). [12] point out that players go to In this paper, we reflect on the ethnographic research on Internet cafes to access not only the Internet but also their World of Warcraft we conducted in Beijing and Shanghai in close friends and peers. While we found similar evidence, July and August, 2007. We describe the types of activities we also observed that tight living spaces and family that took place in the game and their impacts on the culture dynamics have impact on the decision of where people play specific renderings of the game. We also examine economic the game. Most student dormitories in Beijing and Shanghai activities, governmental regulations, and cultural values that provide little or no private space for students. Dormitory influenced social dynamics within the game. rooms are shared by 4-8 people and fit not more than a bed for each that is simultaneously used as workspace (Figure Methodology 3). Many young professionals in Beijing and Shanghai live at Over a period of 6 weeks we observed and interviewed home with their parents until they are married. The small WoW players in Internet cafes, university dormitories, and living spaces as well as parental disapproval of game play apartments in Beijing and Shanghai. We interviewed 30 render the local public play space in Internet cafes an people between 19 and 37 years old, 25 male and 5 female, attractive choice. observed their play behavior in the game and interactions at the physical game location. We conducted the interviews “The Burning Crusade” (TBC), an expansion to the game with translation and cultural interpretation provided by three that introduced among other features the increase in level native speakers of Chinese. Two were graduate students at cap to 70 (60 before), new professions and new playable Beijing University and one Chinese-American collaborator. races, was released on January 16, 2007 in Europe, United Our findings are limited by a small sample size, but we States and Australia, on April 20, 2007 in Taiwan, as well as believe they provide important preliminary data on online the regions of Hong Kong and Macau. In China, however, game play in China. the TBC was not released until September 2007. Our informants reasoned that The92, distributor of WoW and Figure 1 Internet Café at a train station. This ethnographic inquiry is one phase in an ongoing other games in China, should be made responsible for the project. Nardi has been conducting immersive ethnographic research on WoW since December 2005 [3][9][10]. Lindtner 2 http://www.the9.com/en/ 3

delay, because of their general bad reputation in terms of did not require the player to pay, others were implemented server maintenance and player support. The delay of the with a quite different economical system compared to the release was hence attributed to low quality of service original game version. For example, players could literally provided by the distributor. buy their way into the server through membership payments that provided high-level virtual characters based on the However, players did not only speculate about the reason amount a player was willing to pay: behind the delay of the release. Many of them also took action to get their grip on the TBC, which should change You pay a 30RMB, 50RMB, 100RMB, or 1000 RMB several dynamics in and outside of the game. For example, membership. I went in there and my character was already a strategy players pursued was to create an account on a at level 60. It’s a rip off from the other server. server outside of China, e.g., a Taiwanese or American server. While others quit playing the game, some players In North America, Europe and Taiwan players generally pay made use of the local media-pirate industry, installed an a monthly fee for a game account. In China players unlicensed version of the TBC on a privately maintained purchase hourly-based point cards that are not linked to one server, which other players could access. Even though specific account to play the game. A point card costs about these “private servers” were very often unstable and did not 30 Chinese Yuan Renminbi (or ~4 US$) and allows a player provide the full set of features as the commercial TBC, to be logged into the game for 66 hours. This payment players logged onto them to experience and practice the system affords quite different game strategies: since the Figure 2 Inside of an Internet Café in new game features. Some of the players we talked to had at point cards are not linked to a specific player account, Shanghai. least one additional account on a private server, which they players trade point card IDs for in-game currency. A point played regularly especially during downtimes of the regular card could be acquired for about 400-550 in-game gold Chinese game servers. coins at the time of our study. The value fluctuated between servers and was based on the current activities in the game. Players quitting the game and moving to private or foreign For example, during a time of high guild activity, a point card servers also changed certain dynamics in the game. Guilds, could be worth up to 600-700 gold coins. Players who sell an important social structure in the game that allow people point cards usually have less time to invest playing the to group together in order to achieve high-level in-game game. For them, selling point cards was a mechanism to goals collaboratively, broke apart because of the decreased earn gold coins in the game without spending too much time number of online members. Consequently, many players in the game. One of our informants described how this could engaged in activities that could be successfully influence the value of the in-game currency: accomplished as a single player, like simple quests or training their characters in their specific professions such as Some players don’t play the game often, and bring a lot of Figure 3 Work, living and sleep space in a fishing or collecting herbs. Others mentioned that they point cards to sell. And the economy in game will decline student Dormitory began using the game to chat and meet friends online, like during inflation times. When the game came out, one because they couldn’t find enough people to collaborate on player could buy a point card for only 70 or 80 gold. a bigger in-game task. Whereas some of the private servers 4

Others saved gold they acquired through the sale of point The Development of Tactics and Strategies in a Mobile cards as preparation for the release of the TBC. Whereas Game, Proc. Ubicomp 2005, 358-374. some players used the point card system to buy their way [2] Bell, M., Chalmers, M., Barkhuus, L., Hall, M., into the higher levels of the game, others, who could not Sherwood, S., Tennent, P, Brown, B., Rowland, D., Benford, afford to do so or did not want to spend too much money on S., Hampshire, A., Capra, M. Interweaving Mobile Games with Everyday Life. Proc. ACM Conf. Human Factors in the game, looked at it as a cheap way of playing. Investing Computing Systems CHI 2006, Montreal, Canada. time in the game now affords playing the game in the future: [3] DiGiuseppe, N. and Nardi, B. Real Gender Choose Fantasy Characters: Class Choice in World of Warcraft, Silvia: Do you spend a lot of money to play the game? First Monday, April, 2007. Informant: No, not a lot. I didn’t buy so much equipment, but [4] Duchenaut, N., Yee, N., Nickell, E., Moore, R.J. Alone rather point cards. For example, 400 gold coins for a point Together? Exploring the Social Dynamics of Massively card. I looked at it as a free game, because I exchanged it Multiplayer Online Games, Proc ACM Conf. Human Factors for point cards. in Computing Systems CHI 2006. [5] Kellogg, W.A., Ellis, J., Thomas, J.C. Towards Supple Conclusion Enterprises: Learning from N64’s Super Mario 64, Wii In this paper, we have introduced a subset of our findings of Bowling, and a Corporate Second Life, Position Paper for a 6 week long study of players of the online game World of the CHI 2007 workshop Supple Interfaces, San Jose, CA. Warcraft and their social environment in Beijing and [6] Lin, J.J., Mamykina, L., Lindtner, S., Delajoux, G., Shanghai to illustrate how local structures and cultural Strub, H. Fish’n’Steps: Encouraging Physical Activity with an values influence the process of adopting the technology. Interactive Computer Game, Proc. Ubicomp 2006. Local governmental regulations, family dynamics, and living [7] Lindtner, S. and Chen, J. mopix: Playful Encounters standards, as well as economic infrastructures in these with Surveillance in Everyday Urban Settings, in Extended cities shaped dynamics in and around the game, creating a Abstracts of Ubicomp 2007, Innsbruck, Austria. reciprocal back and forth between the game and its local [8] Lindtner, S. and Nardi, B. Venice, California and World context. Many in-game strategies were developed to of Warcraft: Persistence and Ephemerality in Playful accommodate regulations devised outside of the game or to Spaces, to appear in Proceedings of HICSS 2008. adapt to the specific infrastructural settings such as slow [9] Nardi, B. and Harris, J. Strangers and Friends: Internet connections. We believe that a perspective of virtual Collaborative Play in World of Warcraft. Proc. CSCW 2006. worlds as not distinct but in dialectal relationship to the local [10] Nardi, B., Ly, S., and Harris, J. Learning Conversations culture’s context is valuable for future technology design for in World of Warcraft. Proc. HICSS 2007. ubiquitous computing and mixed reality systems. [11] Taylor, T.L. Play between Worlds. Exploring Online Game Culture, MIT Press 2006. References [12] Qiu, J. L. and Liuning, Z. Through the Prism of the [1] Barkhuus, L., Chalmers, M., Tennent, P., Hall, M., Bell, Internet Café: Managing Access in Ecology of Games, M., Sherwood, S., Brown, B. Picking Pockets on the Lawn: SAGE Publications 2005, 261-297.

Tagged Cities

Abstract This paper discusses the issues related to managing tags that have been embedded into the urban Minna Isomursu environment and public spaces. Tags provide a bridge VTT between the physical and digital worlds, supporting an Kaitoväylä 1 intuitive touch-based interaction and access into the FIN-90571 OULU, FINLAND mobile Internet. We have implemented and piloted [email protected] several NFC (Near Field Communication) based applications, and summarize here the findings related Tuomo Tuikka to managing tags in a city environment. VTT Kaitoväylä 1 Keywords FIN-90571 OULU, FINLAND NFC, RFID, tags, tag management [email protected] ACM Classification Keywords H5.2. User interfaces

Introduction The NFC and RFID technologies are finding their ways to the everyday life of urban dwellers, for example, in ticketing applications [3] and payment solutions [6]. The research discussed here has been done within the context of the SmartTouch project [8]. The project aims at exploring the possibilities of touch-based user interaction enabled by NFC technology. Field studies were used for examining touch-based interaction in several application domains. Here, we discuss the findings related to managing the NFC tags embedded in the city environment and public spaces. 2

Tags bridging the physical with the digital Physical browsing [1] allows connecting the physical world with the digital world via tags. The tags can provide access points for the user to access and interact with the digital world. As tags can be embedded into our physical environment, they can form a net that provide the user with an access to the digital realm of the Mobile Internet. Tags provide a direct access to the services and information without the need for browsing or searching. The physical world contains the context information that helps the user to understand where the tag takes her.

Research pilots Figure 1. Meal-ordering application in use. In the SmartTouch project, we have implemented and piloted applications utilizing a touch-based user Tags can be also embedded in the public spaces. The interface within several application domains. smart poster concept embeds tags into print media and allows the user to access digital information related to For example, one of the field studies evaluated a meal- the printed media. For example, a Madonna concert ordering application that provided the elderly home- poster displayed in a shop wall embeds NFC tags which care clients with the opportunity to choose which meal can be used for accessing mobile Internet to view they would receive the following day (more details in information about the upcoming Madonna concert in [4]). Without the mobile Internet application, the the city, and purchasing the ticket for the concert via elderly homecare clients did not have a possibility to mobile banking service. Tags in public spaces can also express their preferences and the same food was provide access to public services. For example, a delivered to all clients. The user interface consisted of a parking pilot ongoing in the city of Oulu enables parking meal menu that embedded NFC-tags, and a mobile ticketing through tags that are attached into lamp phone that was used for touching the tags to make posts. The user is then able to pay the parking fee meal selections (see Figure 1.). The meal selection was simply by touching a tag nearby when arriving, and then delivered to the meal service providers. This when leaving the parking lot. application is an example where the tags are embedded inside the home of the user, therefore being part of a Tag management private environment. Using tags as access points to various services and applications usually mean distributing tags into environment that usually cannot be strictly controlled. The amount of tags can be high, they must resist 3

weather, wear-and-tear and vandalism. As the tags are income and monetary efficiency of tag placement. The components of both the physical and digital realm, they space owner may receive a certain amount of money need to evolve as the physical and digital environment from each interaction initiated by touching the tag. This evolves. Here, we examine the various requirements would result in higher income in places where tags can and design considerations that need to be taken into be easily used and accessed by their potential users. account in managing tags in such an environment Tag as an application function Visual design In the pilot case a real physical table was connected to Tags can be visible or they can be hidden. For example, the application logic. The selection of meal provides touching a picture frame holding a picture of a input for the application to continue. There may be grandson with a mobile phone may initiate a call to the cases where tags are touched in a sequence, such as grandson. The picture frame would look just as any selection, and OK, or cancel. In the latter case the picture frame as the tag can be placed behind or inside application functionality is brought into the the frame. If the tag is invisible, the user may face environment. In city life, restaurants could have tagged problems in finding it. The NFC tags need to be menus that correlate to the choices in application logic touched, so the user needs to know where the tag is. Thus, a more common scenario is that the tags are Error detection visible. The visual design of a tag needs to give the The NFC tags are able to communicate with the digital user information about the purpose and target of the world only when they are activated by an NFC reader. tag. The visual design is one tool for creating the sense This means, that the maintenance personnel of NFC-tag of trust and security in the user. In domain, such as based systems cannot see in real-time, if a tag fails to transportation the visual design should be identifiable. operate correctly. Testing and detecting malfunctioning tags manually is time consuming. The users could be Obviously, if tags are numerous they all compete with recruited to help in locating broken tags. The tag could each other about the visual attention of the user. Public include information on how to report break-up, and the spaces are often very loaded with visual information, so user could be rewarded from reporting. designing tags that can be found and recognized with ease is challenging. Standardization efforts might help Malicious tags in launching standardized icons and symbols that could Using tags and physical browsing as a shortcut for be used for indicating tag position. accessing mobile Internet and related services has obvious security challenges. Tags embedded into our Tag placement environment may lure unsuspecting browsers to access In public spaces, the space/surface owner can decide places or services they do not wish to enter, causing where tags can be placed, and on what conditions. As financial or emotional worries. Especially, if the tag can the tag is connected with the digital world, usage-based initiate payment or billing actions, there always is a risk payment schemes can be introduced for optimizing the that the users are lured into paying on the basis of 4

false information. One solution is to only allow tags Acknowledgements provisioned by a trusted service provider to initiate This work was done in SmartTouch action. However, this would set up “walled gardens” [5] (www.smarttouch.org) project (ITEA 05024) within and limit the freedom of the user. However, it must be ITEA 2 (Information Technology for European noted that humans tend to prefer free, unlimited Advancement), a EUREKA strategic cluster. Project has choice, even if they would be happier with limited or been partly funded by Tekes, the Finnish Funding fixed choice [2]. Agency for Technology and Innovation.

Retirement Citations Eventually, all applications become obsolete. Also the [1] Ailisto, H., Pohjanheimo, L., Välkkynen, P., tags need to be removed at some point. Obsolete, non- Strömmer, E., Tuomisto, T., Korhonen, I. Bridging the physical and virtual worlds by local connectivity-based operating tags cannot be just left to litter public spaces, physical selection. Personal and Ubiquitous Computing, as that would result in bad user experience, when the 10, (2006), 333-344. user tries to use tags that are not anymore operational, [2] Brown, N., Read, D., and Summers, B. The Lure of or contain outdated information. This will corrupt trust Choice. Journal of Behavioral Decision Making. Wiley towards the tags in general. InterScience. 16, (2003) [3] Card Technology Today. NFC technology begins There are basically two options for tag retirement. new trial. Volume 17, Issue 1, January 2005, 4-5. Firstly, the tags can be manually removed. If large [4] Häikiö, J., Isomursu, M., Matinmikko, T., Wallin, A., amounts of tags are distributed over geographically Ailisto, H., Huomo, T. Touch-based user interface for large areas, this may require lots of manual work. The elderly users. In Proc. MobileHCI. (2007) second option would be to create biologically [5] Isomursu, P., Hinman, R., Isomursu, M. degradable tags which would just gracefully disappear Spacojevik, M. Metaphors for Mobile Internet. To when they are not used and needed anymore. appear in Journal of Knowledge, Technology, Policy, December (2007) Discussion [6] Ondrus, J., Pigneur, Y. An Assessment of NFC for This paper discusses some of the problems related to Future Mobile Payment Systems. In Proc. International embedding simple technology, i.e. NFC tags, into the Conference on the Management of Mobile Business. environment. Embedded tags enable integrating (2007) information technology with everyday life and activities [7] Rukzio, E., Leichtenstern, K. Callaghan, V., of the users, and bridge our physical world with the Schimdt, A., Holleis, P. & Chin J. An experimental digital. However, management of ubiquitous tag Comparison of Physical Mobile Interaction Techniques: Touching, Pointing and Scanning. In Proc. The Eight network in unpredictable and uncontrollable public International Conference on Ubiquitous Computing. space introduces design challenges that need novel (2006) innovative solutions. Tag management may create new [8] SmartTouch project Web pages. business models and chains of interoperating actors. http://www.smarttouch.org 5

Orchestration and Direction of Mobile MR Games

Jan Ohlenburg Abstract Fraunhofer FIT Mobile Mixed Reality Games are embedded in the Schloss Birlinghoven physical environment of a user and pose therefore new 53754 Sankt Augustin, Germany requirements on design, authoring, orchestration and [email protected] direction of the game. New supporting tools are required to meet the challenges of such games and to Wolfgang Broll create an exciting and coherent experience for the Fraunhofer FIT users. Schloss Birlinghoven 53754 Sankt Augustin, Germany In this paper we focus on the aspects of orchestration [email protected] and direction of Mobile MR Games and how tools can support the developers and the people setting up such Irma Lindt games. We highlight also the fact that supporting tools Fraunhofer FIT might become an important game element. Schloss Birlinghoven 53754 Sankt Augustin, Germany Keywords [email protected] Mixed Reality Games, Orchestration, Direction, Tools

ACM Classification Keywords H.5.1 Multimedia Information Systems – Artificial, augmented, and virtual realities, Evaluation/ methodology. K8.0 General-Games. K8.3 Management/Maintenance. Copyright is held by the author/owner(s). CHI 2008, April 5 – April 10, 2008, Florence, Italy Introduction ACM 1-xxxxxxxxxxxxxxxxxx. In Mixed Reality (MR) environments the real world is superimposed by virtual objects, enhancing the user’s 2

natural environment and allowing him to interact with illustrate this upon two different games. Usage of these them. Bringing MR into mobile pervasive games adds tools has not been limited to the game developers, but fascinating possibilities to the aspect of traditional also became a part of the game itself and hence computer games. support the whole process chain of creating, authoring, orchestrating and directing for Mobile MR Games (see Since traditional computer games do not relate in any figure 1). Design way on the environment in which they are played, no work has to be done to setup the game or adopt it to Related Work the local environment. Additionally, gamers are Although MR Games have been developed for a couple Authoring typically familiar with the standard gaming interaction of years, there are a lot of examples where the game devices. This is not the case in Mobile MR Games, thus does not relate to the environment or it has been set the developers have to cope with new challenges. In up for a specific environment. An example for particular, although the game design might be environment unrelated games is ARQuake [5] and Orchestration independent from the actual game environment – the ARPacman [3] for a fixed location game. real game area – game actions, items, events and virtual characters have to be closely integrated and Direction require a strong relation to the current real environment. Additionally, the technology used for playing Mobile MR Games is in general new and Evaluation mysterious for the players, therefore the game play typically has to be adapted to meet the requirements of the different user groups. Such pre-game configuration and setup is known as game orchestration and is an essential part of a Mobile MR Game. In case the Mobile MR Game has an event like character – where the figure 1. The process chain of creating Mobile MR Games. It is an interactive players are equipped with special technology and the process after the design and the environment needs to be setup as well – it is also very authoring phase, the game play is tested crucial that such adaptations to the game play can be and evaluated, afterwards, it is adapted performed during a gaming session in order to again. (The focus of this paper is on influence the game play. In this paper we use the term orchestration and on direction.) figure 2. The orchestration board of the game Epidemic direction for in game monitoring and adapting the Menace. game play. The mobile SMS game Day of the Figurines the staff managing and leading the game, used orchestration in In this paper we describe a set of tools helping the order to manage and track the production of game play developer to meet the challenges of orchestration and narratives during the game session. They state that direction for Mobile MR Games. Additionally we will 3

orchestration is inseparably intertwined with the game game session while it is played. In general all aspects play [2]. of the game that are configurable can be adapted during the orchestration phase and those which can Epidemic Menace [4], a cross media pervasive AR change the state during a session can be adapted game, used orchestration tools for pre-game setup and during the direction phase. This close relation between in-game adaptations. In this game two teams have to orchestration and direction suggests a tool that covers fight a spreading viral menace. While the outdoor both aspects and is aware of the current phase. In the players have to fight the virus, the indoor players have next section we will present the tool AuthOr, which is the overview of the game are and communicate that to designed to support these phases of the process chain. the outdoor players. Supporting Tools A good overview about the technology challenges of Within our recent MR games, namely Epidemic Menace pervasive augmented reality games is given by Broll et [4] and Timewarp, we used a set of supporting tools for al. [1], where the different aspects are closely reviewed orchestration and direction. Within Epidemic Menace, a and solutions are presented. pervasive AR game, we developed an orchestration board. It uses a stylized map of the game area and provides functionality for configuring the teams, placing viruses, assigning devices to individual players (see figure 2). The initial configuration of a session can be saved and used in the game engine. It was also used during the actual game play for receiving information about the current location of the players and the viruses and displaying this information on the map. The orchestration functionality was also enabled during the sessions allowing the game supervisors to add/remove players, place viruses and assign devices to the figure 3. The Mobile Information Terminal of Timewarp. The players, making the game flow adjustable. Due to the map on the right shows the current location of the player and monitoring functionality of the board, it was additionally the screen on the left important game items. used within the game as an overview map for the Game Orchestration and Direction teams, where they could observe the outdoor players. Game orchestration and direction are major issues for Epidemic Menace has another direction tool: web cams Mobile MR Games, since they allow for easy adaption of in the game environment provide augmented video the game before and during the game session. Game streams to the players as well as to game observers, orchestration deals with all aspects of configuring and allowing for audience participation and also supporting setting up the game for a certain game session, while the evaluation of the game play. For the Mobile MR game direction deals with monitoring and adapting the Game Timewarp, we redesigned and reimplemented 4

the orchestration board in order to meet the In our future work, we will continue to focus our work requirements stated above. While in Epidemic Menace a on AuthOr in order to support even more steps of the relocation of the game to a different location would Mobile MR Game process chain, e.g. using it for have meant a great effort, this new tool – called AuthOr supporting evaluation by playing recorded sessions. – is much more general. One of the key features of Such functionality can include showing paths of the AuthOr is hence augmentation of arbitrary maps with players, events that happens and so on. overlays (images, lines, text, …), e.g. for player positions, paths, game items, game events and so on. Acknowledgements An interface to map services such as Google Maps, Parts of the work described in this report were Google Earth and Virtual Earth allows playing the game performed within the projects IPerG (FP6-2003-IST-3- 004457) and IPCity (FP6-2004-IST-4-27571) partially everywhere in the world. Therefore, it is possible to funded by the European Commission as part of the 6th register and display virtual objects and game actions in Framework. the real environment. Further, the effort for adapting a game session to a new location becomes reasonable. If References used as an orchestration tool, the game area can be [1] Broll, W., Ohlenburg, J, Lindt, I., Herbst, I, and defined, and arbitrary game items can be placed. Braun, A.-K. Meeting Technology Challenges of During a game session it is used by the staff to Pervasive Augmented Reality Games. In Proc. of ACM SIGCOMM NetGames 2006, 2006. supervise the game play and relocate, add or delete game items, additionally it is provided to the players as [2] Cabtree, A., Benford, S., Capra, M., Flintham, M., a map tool, where they can see their current location Drodz, A., Tandavanitj, N., Adams, M., and Farr, J.R. The Cooperative Work of Gaming: Orchestrating a on the map and interesting spots (see figure 3). As the Mobile SMS Game. In Computer Supported Cooperative playing area becomes larger, zooming functionalities of Work, 16(1-2), 2007. the map are essential in order to get an overview of the [3] Cheok, A., Goh, K. H., Liu, W., Farbiz, F., Fong, S. game, or a close up look for more detailed information. W., Teo, S. L., Li, Y., and Yang, X. Human pacman: a For efficient zooming support the maps can be specified mobile wide-area entertainment system based on for different levels. physical, social, and ubiquitous computing. Personal Ubiquitous Computing, 8(2), 2004. Conclusion and Future Work [4] Lindt, I., Ohlenburg, J., Pankoke-Babatz, U., and As we showed, supporting tools for the process chain of Ghellal, S. A Report on the Crossmedia Game Epidemic creating and managing Mobile MR Games are a crucial Menace. ACM Computers in Entertainment (CIE), 5(1), part for balancing and managing the game and can 2006. even be part of the game play itself. If the location, [5] Piekarski, W. and Thomas, B. ARQuake: the where the game is played, is not fixed, such tools allow outdoor augmented reality gaming system. Communications of ACM, 45(1), 2002. for easy adaptation.

Emergent Narrative in Hybrid Environments

Abstract Eric Kabisch This work focuses on worlds of digital information that Department of Informatics describe – and regulate – the embodied physical spaces University of California, Irvine people inhabit. I have developed a mobile interactive Irvine, CA 92697-3440, USA art installation as a way to bring geographically [email protected] referenced information out of databases and into everyday experience of traveling through the world. Datascape enables a hybrid environment whereby participants author dynamic geographic narratives that compose a digital world. A vehicle-mounted digital periscope engenders action between passengers and a visual and sonic landscape that unfolds and emerges based on conversations between people, georeferenced data, and dynamic representational entities that compose the landscape.

Keywords Geographic visualization, mixed reality, hybrid, interactive music, mobile, urban computing

Introduction Scores of databases contain information about our physical world and its inhabitants. Every time a credit card is swiped, a cell phone beacon is received, a field Copyright is held by the author/owner(s). survey is recorded or a crime is reported it adds to our CHI 2008, April 5 – April 10, 2008, Florence, Italy digital landscape. Each of these actions in the world ACM 1-xxxxxxxxxxxxxxxxxx. generates a digital trace – whether intentional (as in a field survey) or incidental (a transaction with a buyer 2

rewards card) – which is stored as an entry in a Datascape is an ongoing art project enabling an open database. environment for the creation of hybrid worlds and narratives that correlate to the planet Earth. Immersive As we move through our daily lives we encounter sound and a computer-generated graphical spaces, networks, people and systems that are both environment are shared with vehicle passengers as described and regulated by databases. In many senses, they travel through the city. A digital periscope allows bits of data can be considered situated at the locations its operator to view and interact with information about they describe and regulate, however, they reside in the surrounding environment. As the riders travel isolated servers and are only brought out into a spatial through geographic space, they simultaneously explore context via a specifically produced map, visualization or a 3D virtual topography built from invisible datasets. tabular/textual representation. In effect, the data is Music and sounds are embedded in the landscape, with hidden from view while interpreted stories are the only locally aggregated data driving musical styles and other record – sometimes due to data sources being privately musical events attached to places, objects, and actions held, but more importantly because there is a lack of within the hybrid world. A typical commute or drive tools to view them in context in the lived world. How around town is turned into a sonic and visual can we enable awareness of, and interaction with, the exploration of hidden narratives that surround us and information that describes and shapes our cities? envelop the city.

Representation and narrative Much work in hybrid environments is focused on an object-based or coordinate-specific representation of information1. This focus has two implications for interaction with which I am primarily interested – that current interactive technologies encourage a direct fusing of the synthetic and the physical; and that such a fusing breaks interaction down into a chain of discrete encounters as opposed to a continual unfolding of meaning and experience. In order to reconsider this object-specific mode of engagement with interactive systems, I am working to enable the creation of participatory emergent narratives. I introduce a

representational ontology containing entities that are figure 1. (a) Datascape v1.0 screenshot, (b) prototype periscope installation, and (c) participant at an exhibition. 1 For example: augmented reality, barcodes and RFID, Bruce Sterling’s spimes, the internet of things, mixed reality games, spatial annotation projects. 3

dynamic, programmable, interactive and data driven. symbol how fast to rotate. Behaviors can also interact This ontology enables the world come to life through with the actions of a passenger - through dynamics and the participation of the user, narrator, data, and hybrid location of the vehicle or through user interaction. For environment through which the narratives emerge. The instance, a behavior could tell the symbol only to rotate act of narrative creation consists of choosing existing when a user comes close or touches it. Behaviors also datasets – or adding or creating new ones – and then allow symbols to interact with each other, and to have selecting from a range of embodiments and actions to interdependent relationships where one behavior might represent the data in the digital world. As a starting be waiting on another to complete or issue a command. point I have created an ontology of representations, or In addition to symbols, behaviors can be associated entities, consisting of symbols, behaviors, forces, with zones and forces. zones, paths, links, portals and sounds. These entities make up a dynamic ecology that unfolds in guided but Zones and forces- Zones are geographic regions emergent and unplanned ways. defined by a polygon boundary. Examples are city boundaries, census divisions, or user-generated regions Symbols are entities that have a “physical” presence in such as gang territories or neighborhoods. Zones are the digital world. A symbol, for instance, can be not necessarily visually represented in the environment selected as the representational entity for a series of (though they can be through symbols) but they are locations in the geodatabase, where the symbol would programmed to have impacts on the behaviors or appear at the corresponding locations in the digital appearance of other entities in the environment. A zone world. The symbol representation can be selected from can be programmed to effect symbols that are within many classes including 3D models, procedurally its boundaries, or it can be programmed to illicit certain generated geometries, particle systems, and behaviors once the vehicle has entered its boundary, agents/avatars. Symbols in the environment typically such as presenting the user with a sound or textual respond to touch. The default response when touching information upon entering the zone. The combination of a symbol is to receive a view of the symbol’s metadata zones and symbols can be very powerful – a zone – creator, data type, date, etc. – and the values for might, for instance, make symbols behave differently database attributes of that entry. Symbols can also be based upon the region they occupy. Forces are similar programmed to respond to touch in other ways such as in scope to zones, except they are tied to points instead attaching symbol behaviors. of regions and their effects taper off at a given distance from the center of the force. Forces can also migrate in Behaviors enable the data to come to life through relation to other forces or behaviors, allowing for action within the digital environment. A behavior can be dynamic systems and providing a helpful balance to the tied to a symbol, such as animating it to rotate. When a defined and bounded nature of zones. behavior is attached to a symbol, some parameter of the behavior is typically controlled by an attribute value Paths are temporal and directional in nature. They are of the underlying data source – such as telling the defined either by mathematical functions or by a series 4

of points. Paths can be used to guide the movement of music is based on a continuously evolving groove symbols or forces, and can also be derived from such generated in real time through stylistic parameters movements. Examples of paths might include the based on the aggregation of surrounding data. These movement of vehicles through the city, flows of migrant parameters can be controlled by datasets, or by other workers from season to season, or movement of capital behaviors, zones or forces. In addition to the groove, from overseas investors. Additionally, a mathematical other types of sounds can be enacted based on function or a variable from a dataset can be mapped behaviors and other events. Interaction with the into a geographic context to create a path. The coupling environment generates sounds and music by sonifying of paths and forces allows for behaviors to unfold the selected datasets. These patterns can conform across a spatio-temporal domain. within the rhythm and harmony of the main groove. Additionally, sampled or generated sounds can be Links and portals - Links are used to establish linked to a specific symbol, zone, force or behavior. relationships between various entities within the digital These triggered sounds are played when the vehicle landscape. Links are automatically created between passes a given threshold or comes within a distance of symbols that belong to the same dataset – such as a symbol. Continuous located sounds can also be linked distributed sensors in an air quality data layer – and to specific locations or symbols. These sounds are can also be established between datasets or between placed into the user’s 3D world and proximity specific symbols. Additionally, passengers can create determines volume and spatialization. links between symbols as they explore the environment and uncover relationships to share. Links can be added Background to symbols that are in close proximity to one another or A Datascape prototype was built and exhibited in 2006. to symbols that are geographically distant but related Here I outlined the current design phase of the ongoing in some other manner. iterative development cycle. Due to space constraints this essay does not contain full details or Portals – which are enabled by links – allow for the contextualization. Further reference for Datascape can decoupling of the digital topography from geographic be found in [1], [2] and at datascape.info. space. When a symbol portal is acted upon, the digital environment is reconfigured to spatially present the References linked symbols according to relationships between the [1] Kabisch, E. A periscope for Urban Discovery and symbols instead of their physical location in the world. Narrative. CHI 2007, Imaging the City workshop. Portals thus enable a way to view spatiality in relational [2] Kabisch, E. Datascape: a synthesis of digital and and hodological terms, not just cartographic. embodied worlds. Space and Culture Journal, 2008 (in press). Sounds As described above, a musical soundtrack accompanies travel and interactions in the hybrid environment. The

Progress towards Site Visits by Situated Visualization

Sean White Abstract Department of Computer Science We describe our experience and lessons learned from Columbia University conducting a CS course final project to explore urban New York, NY 10027 USA design site visits using augmented reality. [email protected] Keywords Petia Morozov Augmented reality, site visit, situated visualization Graduate School of Architecture, Planning and Preservation ACM Classification Keywords Columbia University H.5.1. [Information interfaces and presentation] Multi- New York, NY 10027 USA media Information Interfaces—Artificial, augmented, [email protected] and virtual realities; H.5.2 [Information Interfaces and Presentation] User Interfaces—Theory and methods; Ohan Oda J.5 [Arts and humanities] Architecture. Department of Computer Science Columbia University Introduction New York, NY 10027 USA At Columbia University, we are investigating how mo- [email protected] bile computing can support the site visits that architects and urban designers make as they begin a design activ- Steven Feiner ity. We base our primary tools and methodologies on Department of Computer Science more than a decade of mobile augmented reality re- Columbia University search by Columbia University’s Computer Graphics and New York, NY 10027 USA User Interfaces Lab [1]. These practices and technolo- [email protected] gies were further developed in Spring 2007 through a Computer Science course on 3D user interface design [5], taught in collaboration with faculty and students Copyright is held by the author/owner(s). from the Graduate School of Architecture, Planning, and CHI 2008, April 5 – April 10, 2008, Florence, Italy Preservation. ACM 1-xxxxxxxxxxxxxxxxxx. 2

roundings. During the Spring 2007 semester, students addressed the site of Columbia’s proposed Manhattan- ville expansion using curated data collected specifically for the project, as well as GIS data.

Site Visit With more than half of the world’s populations living in urban environments, the expectations for future devel- opment in our cities require greater degrees of redun- dancy across infrastructural, cultural and environmental systems. Hybrid programming of densely urban sites, such as mixed-use transit hubs, renewable energy pro- ducing parks, and live-work waterfronts, has surged in the last decade. Design fields are expanding disciplinary interaction in order to coordinate this complex enter- prise of scales, stakeholders and resources. Vital to this effort is the ability to maintain evenly balanced expo- sure to each system’s design variables, regardless of its immediate visibility or presence.

In architecture and urban design, the concept of site is

fluid. The most straightforward definition considers a

figure 1. (a) Tablet PC with video camera attached to back site to be an entity defined by “boundaries that delimit overlaying a virtual model on the site map. (b) Example of it from the surroundings,” where boundaries are typi- selecting building in virtual model associated with site plan (Courtesy of Mike Sorvillo, Levi Lister, Taran Singh, and Sasha cally considered to be spatial. In a more expansive Stoeva). view, the site can be understood as the superposition of The prototypes we have been developing augment 2D all aspects of the entity, beginning with the specific paper site plans (figure 1), and actual 3D physical sites place but including spatial and temporal surroundings, (figure 2). Our primary platform is a laptop computer multiple histories, and physical, social, political, envi- with an attached camera that records the physical ronmental and cultural characteristics—that is, all scene. Computer vision algorithms recognize and track things that influence or are influenced by the site. printed paper markers in the scene in real time, making it possible to overlay relevant information interactively Our research endorses the designer’s ability to dynami- on top of what the camera sees. This lets us create live cally visualize a site—to study, map, document, tag and “situated visualizations”, described later, that depict interact with a site in multiple scales, contexts, time- otherwise invisible characteristics of a site and its sur- lines and agendas. 3

The Site 3D User Interface Class Project Our research activities focus on a 17-acre industrial As part of our investigation, the team final project in a district of New York City known as Manhattanville, class on 3D user interface design asked students to where rezoning efforts are raising disputes over land explore different scales of user interface and interaction use, economic development, socio-cultural diversity in the urban environment. We asked the students to and environmental stewardship. This area has been design and develop a prototype visualization tool that proposed as the site for Columbia University’s campus could help architects and urban designers better under- expansion, for which preliminary design propositions stand a site prior to or during a design activity by visu- have been publicized using various visualization strate- alizing unseen or invisible aspects of the site. The pro- gies. ject had two stages: augmenting a flat 2D site plan (figure 1) and augmenting a portion of the 3D physical site (figure 2). Students curated their own data and developed situated visualizations of selected invisible aspects of the site, along with techniques and new af- fordances [3] for interacting with those aspects.

Situated Visualizations. Typically, visualizations are shown on a stand-alone display, whether desktop, hand-held, or head-worn. In the figure-ground rela- tionship, the physical environment that serves as the ground in which the visualization is presented has no meaningful relationship to it. In contrast, we use the term situated visualization to describe a visualization that is related to its environment; for example, by be- ing specific to the surrounding spatial context. Situated visualizations gain meaning through the combination of the visualization and the relationship between the visu- alization and environment. In the context of site visits, the visualizations become part of the site.

Platform. We provided students a platform [4] built on figure 2. (a) Positioning a virtual fiducial in the site plan top of an existing 3D engine [6] extended with aug- virtual model to correspond with the location of a physical fiducial in the real world (Courtesy of Varun Maithel, Ar- mented reality capabilities including live video capture, mando Ramirez, Michael Wasserman, and William Yin). (b) 6DOF fiducial marker tracking [2], and the ability to Related virtual information overlaid on real buildings, regis- combine live video with 3D graphics. This made it pos- tered with full-size fiducial (Courtesy of Mike Sorvillo, Levi Lister, Taran Singh, and Sasha Stoeva). sible for students to, for example, display virtual ob-

4

jects registered with optical markers overlaid on the guage of presentation and interaction? How are the real environment, captured with an inexpensive web flows of people, culture, and concerns best represented camera. and reflected? Relative to design activities, do visually situated site conditions accelerate or optimize the en- Data Curation. We provided students with a 2D image tire design process? Does the site have authority in its of the site plan with fiducial markers and a 3D digital transformation? model of the site generated from GIS data. Students were asked to gather their own additional data about Future Directions the site. Two problems arose in our data curation. Mod- To address these questions, we are investigating how els from GIS were inaccurate and improperly repre- to localize and register visualizations without global sented slope and elevation differences. In addition, models. We will pair this with participatory localized students had to generate their own spatial data be- authoring and shared curation of spatial data, along cause the granularity of much of the data found was with common presentation and interaction techniques only at the block or building level, which does not take to develop a framework for situated visualization. advantage of the richness found in environmental aug- mented reality. Acknowledgements We thank the students of COMS4172 for their contribu- Presentation and Interaction. Projects explored a vari- tions and Elizabeth Barry and Shriram Surendhranath ety of presentation and interaction techniques. Of par- for discussions on site and data curation. This work was ticularly interest was veridicality of visualizations, con- funded in part by a gift from Microsoft Research. necting techniques at multiple scales and between ego- centric and exocentric interaction, and display-fixed References versus world-fixed representations. For instance, figure [1] Feiner, S., MacIntyre, B., Höllerer, T., Webster, A. 2a shows a technique for associating a marker in the A Touring Machine: Prototyping 3D Mobile Augmented Reality Systems for Exploring the Urban Environment. physical world with a location in the 3D model. Figure Proc. ISWC 97, 74–81. 2b shows a display-fixed legend and world-fixed high- [2] Fiala, M. ARTag, a fiducial marker system using lighting of buildings based on age. The same highlight- digital techniques. Proc. CVPR 2005, 590–596. ing model is used in the site plan in figure 1b. [3] Gibson, J.J. The Ecological Approach to Visual Per- ception. Lawrence Erlbaum Associates, 1986. Situated visualizations provide an enhanced way of in- teracting with urban environments, but many questions [4] Goblin XNA. http://www.cs.columbia.edu/graphics/projects/goblin/ arise here. Are the visualizations now part of the urban environment? How do they change perception of the [5] COMS 4172. space? What authorship and provenance representa- http://www.cs.columbia.edu/graphics/courses/csw4172 tions are required? What conceptual model and meta- [6] Truevision 3D. http://www.truevision3d.com/ phor should be presented? Is there a canonical lan-

Mixed Realities in the Living Tattoos Social Platform

Diana Domingues between virtual and real world by providing users with Universidade de Caxias do Sul different interactive possibilities. Rua Francisco Getúlio Vargas 1130 95070-560 – Caxias do Sul Brazil Keywords [email protected] Social networks, Interactive styles, aLife, Flash mobs

Eliseo Reategui ACM Classification Keywords Universidade de Caxias do Sul H5.m. Information interfaces and presentation (e.g., Rua Francisco Getúlio Vargas 1130 HCI): Miscellaneous. 95070-560 – Caxias do Sul Brazil [email protected] Introduction Little by little, the internet has become a very Gelson Cardoso Reinaldo important place for social exchanges through various Universidade de Caxias do Sul synchronous and asynchronous communication Rua Francisco Getúlio Vargas 1130 services, such as email, bulletin boards, instant 95070-560 – Caxias do Sul Brazil messaging, and other. With the rising popularity of [email protected] virtual worlds and chat applications, many people have feared that face-to-face communication would Alexandre Lorenzatti disappear and that social spaces would transfer to the Universidade de Caxias do Sul internet. Despite the growing tendency for people to Rua Francisco Getúlio Vargas 1130 interact remotely, it is noticeable that remote and 95070-560 – Caxias do Sul Brazil presential communication intermingle in a mixed [email protected] paradigm for social interaction.

Nevertheless, the blending of physical and digital spaces only start to happen effectively when computer Abstract users can leave their desks and cabled desktops to This paper describes the Living Tattoos project, a connect to the web with wireless notebooks, PDAs and platform for social networks that blurs the boundaries smart phones, giving them physical mobility to 2

navigate in digital spaces. These mobile devices have contents by customizing their own pages with texts, enabled the inclusion of the internet in day to day images and videos. Members of the community can activities that take place in urban areas. A new concept also exchange ideas, post testimonials and plan social of hybrid space has emerged, in which physical and gatherings through the communication services digital worlds have been blended. The project Living provided by the platform. Figure 1 shows a sample Tattoos makes use of several technologies in order to website page of the project, in which a gallery of exploit the possibilities of co-existence in digital and tattoos posted by the users is presented. These are physical worlds. These include a platform for social images of the users’ own tattoos which have been networking, an artificial life (aLife) environment, a photographed either by a digital camera or by a cell mobile messaging service for posting images, a flash phone, and sent to the system through the internet or mob organization scheme for setting up urban through MMS (figure 2). interventions. The next section describes each of these elements, showing how they contribute to the project.

The Living Tattoos Platform Throughout human history, people have added to their bodies iconic shapes in a way to look for identity transcendence: from flowers, dragons, snakes and butterflies, to angels and other religious figures.

figure 2. Picture of user being taken and sent by MMS.

After posting a picture in the system, the image may be manipulated by an online graphical editor. The idea is that the users can adjust small details of the 2D images figure 1. Living Tattoos website. and prepare them for a 3D transformation. The Living Tattoos project is centered in a social network platform that enables people to produce 3

From 2D to 3D The Artificial Life Environment In a subsequent step, the photographs sent by the user Subsequent to the transformation of the tattoos into 3D are processed automatically by the system in order to images, the participants may bring their tattoos to life to emphasize contrast and minimize effects of low by placing them in an aLife environment where the quality pictures. Specific filters are employed in this tattoos may interact, feed, breed, and die according to process, such as gaussian blur and sharpenning filters. a particular architecture [4]. Each tattoo also has its Afterward, the images are converted to a grey scale own sensors. These sensors are responsible for representation and saved in a heightmap [1], storing endowing them with perceptive capabilities, helping the values of how “raised” the surface of each area of the tattoos to find food, to detect danger, to search for image is in a 3D graphical system. mates, etc. [5]. Sensor controllers make up a reasoning mechanism that enables the living tattoos to prioritize certain actions, such as stop eating when a dangerous creature approaches. The breeding of two tattoos generate a new creature with a particular 3D representation which is the result of a morphing process combining the graphical representations of the two parent tattoos. This process has been implemented by the Roaming Library, a proprietary graphic library that has been used in other works featuring in [6]. The new offspring also bears a combination of their parents’ characteristics. Going back along the breeding line, we may find features of the graphical images stamped on the actual tattoo owners, leaving an imprint of figure 3. Tattoo image transformed into a 3D model. materialized images in a growing population of virtual creatures. Then, a displacement mapping procedure is triggered [2], using the heightmap to affect the real geometrical Creating Urban Interventions position of the points in the object mesh, which are In the Living Tattoos social platform, a special section is displaced along a normal vector according to the value dedicated to the organization of flash mobs. In this web of the texture function computed in each surface point. space, users may talk and discuss about the places This procedure produces interesting perceptions of where they are going to meet. A GPS tracking depth and detail, such self-oclusion, self-shadowing and mechanism helps users to get together by showing silhouettes [3]. Figure 3 shows the image of a tattoo where they are in a Google Map. transformed into a 3D model by this process. 4

has been to exploit different devices and technologies to provide several possibilities of interaction such as human to human, human to computer, human to physical world and internal system interactions produced by the aLife environment [7]. The project has exploited the possibility of being continuously connected through new wireless devices, transforming the solitary experience of being online and alone in front of a computer screen into a dynamic experience in which real and virtual worlds blend and modify our perception of urban spaces.

References [1] Max, N. L. and Becker, B. G. 1994. Bump Shading for Volume Textures. IEEE Computer Graphics and figure 4. Simulation of tattoo image projected on building. Applications n14, v4. Jul. 1994. [2] Gumhold, S. and Hüttner, T. Multiresolution Furthermore, every once in a while urban interventions rendering with displacement mapping. In Proceedings are set up in which users are invited to participate. In of the ACM SIGGRAPH/EUROGRAPHICS Workshop on these interventions, tattoo images sent by community Graphics Hardware. Los Angeles, CA, August 8-9, 1999. members are displayed on large building walls (figure [3] Pedersen, H. K. Displacement mapping using flow 4). The landscape of the city becomes an alternative fields. In Proceedings of the 21st Annual Conference on canvas for body tattoos. Computer Graphics and interactive Techniques SIGGRAPH '94. ACM Press, New York, 279-286. 1994. Final Considerations [4] Porcino, N. An Architecture for A-Life. In: Rabin, The project Living Tattoos provides an expressive and Steve(Ed.), AI Game Programming Wisdom 2. Charles exploratory environment to support social interaction River Media, 2003. and enables users to act as content producers by [5] Brooks, R. A. How to build complete creatures sharing experiences and information. The rather than isolated cognitive simulators. In Kurt communication possibilities provided by the platform VanLehn (Ed.), Architectures for Intelligence, 225-240. Lawrence Erlbaum Associates, Hillsdale, NJ, 1991. enable users to set up urban interventions and shift the focus from remote and distributed social networks to a [6] Jones, S. Towards a philosophy of virtual reality: Issues implicit in consciousness reframed. Leonardo 33, situation in which people once again communicate by 2, 2000. pp.125–132. sharing the same physical space. A second level of interactivity [7] is experimented by users when they [7] Couchot, E. Pour une pensée de la transversalité. In Soulages, F. (Ed.), Dialogues sur l’Art et la create 3D models of their tattoos and place them in an Technologie, L’Hartmattan, Paris, France, 2001. aLife environment. The main contribution of this project

Inter-group communication via locative media use in urban space

Abstract This paper discusses the social and communicational Charalampos Rizopoulos aspects of using locative media within the urban con- [email protected] text. It briefly presents a part of the research con- ducted in a project titled LOCUNET. The paper mainly Angeliki Gazi focuses on inter-group communication in urban space [email protected] via using locative media. Additionally, locative media use is examined from the point of view of media studies Dimitris Charitos and a relevant communication model is presented. The [email protected] paper concludes with some research questions, which guide our investigation during this research project. Department of Communication and Media Studies Keywords National and Kapodistrian Univer- Location awareness, computer-mediated communica- sity of Athens tion, groups, urban space 5 Stadiou str. Athens, 105 62 Greece ACM Classification Keywords J.4 Computer Applications: Social and Behavioural Sci- ences. H5.3. Information interfaces and presentation (e.g., HCI): Group & Organisational Interfaces.

Introduction As wireless communication technologies, geographical

positioning systems, and mobile computing converge, they substantially transform – and are at the same time . transformed by – urban space. As a result, novel forms of Human-Computer Interaction (HCI) and Computer- Mediated Communication (CMC) emerge which warrant 2

systematic investigation. LOCUNET1 (LOCation-based are essential for the transformation of spaces to places. Urban NETwork) is a research project that aims to in- In addition to the social significance attributed to vestigate the social and communicational aspects of spaces in this fashion, locative media superimpose a using mobile multi-user location-aware systems in an layer of digital information over the urban landscape. everyday urban context. The expected outcome of this Thus, physical space is enhanced, and at the same time project is a theoretical model that will accurately and the layer of digital information is mapped onto it. The comprehensively describe locative media use at various result is an effective combination of the digital layer’s levels (HCI, interpersonal communication, inter-group fluidity and the durability and permanence of real space communication). This model will ultimately inform the [7]. design and development of locative media. For the pur- poses of evaluating this model, a location-aware appli- In summary, the nature of urban space as a stage for cation will be designed in order to support a multi-user computer-mediated activity and communication is pervasive game activity which will take place in central threefold: geographical (arrangement of physical Athens, Greece. space); social (the concept of place); and digital (the layer of digital information). Pubic space that becomes Locative media and the hybridisation of ‘mediatised’ (i.e. that successfully integrates digital urban space media with the physical environment) and imbued with To city dwellers, the urban context is not merely a geo- social significance may give rise to new forms of social graphical term. Depending on a dweller’s preferences, activity, as in the case of pervasive games, social net- experience, and daily routines, parts of urban space will working, etc. be perceived as socially significant, as people attach meaning to them and appropriate them. Attribution of Groups and locative media meaning, appropriation, and regular occupation of The concept of “group” refers to a “collection consisting spaces may lead to their transformation to places [3]. of a number of interacting partners as interdependent Space and place are not identical. Space refers to the individuals, conditioned from a formal or informal struc- spatial arrangement of elements which establishes an ture, individuals that should complete a task” [5, environment, whereas place has more social connota- p.116]. A group consequently becomes perceptible as a tions and is not exclusively confined to the material type of “social field”, while what actually takes place in world. Regular occupation and appropriation of a space this field is the combined result of the interdependen- cies among individuals.

1 The research project titled “LOCUNET” is supported by the Greek General Secretariat of Research and Technology under The existence of a common, public space is important the framework of the Operational Program PEP Attikis, Measure for both ‘traditional’ (i.e. non-mediated, physically col- 1.2. The Program is co-financed (70%) by the European Fund located) and mediated groups. In the case of mediated of Regional Development (EFRD), which aims to facilitate the reduction of the inequalities within the European Union re- groups, examples of virtual spaces fulfilling this func- gions. tion are abundant on the Internet. Locative media con- 3

A communication model of locative media Activities / Actions use In this section we propose a model that has been in- Objective / Objective / spired by media studies and Activity Theory [5]. This is goal goal a first step towards the comprehensive theoretical model that is the project’s desired outcome. The final Environment model is expected to contribute towards a more thor- ough understanding of locative media use for the pur- Physical poses of supporting the design of such systems. Virtual The communication model is a modified version of the co-orientation model [8], which, in turn, is based on User A User B Newcombe’s ABX model. In short, two locative media COMMON 3 GROUND users (User A and User B) , who may or may not be- long to the same group, communicate with each other Locative Media through wireless location-aware devices. Additionally, they may engage in interpersonal communication with Figure 1: The modified co-orientation model describing com- each other if both are located in the same place simul- munication through locative media taneously. At all times, users influence and are influ- stitute a hybrid type of common space, wherein group enced by the environment(s) they inhabit. The users, members simultaneously operate on both real (the the environment, and the medium define a common physical environment) and virtual space (the digital ground, that is, a shared context of locative media layer of information exchanged among networked use4. Users of the same locative medium share the 2 handheld devices) . same virtual space. Additionally, they populate their respective physical environments, which may coincide Groups may also appropriate spaces, since they are depending on their proximity to one another. composed of individuals. In this case, appropriation provides groups with common grounds wherein group Activity Theory suggests that a subject performs an cohesion and identity are reinforced. We suggest that activity in the pursuit of an objective. The objective is locative media users constitute an active and dynamic formed so that a need may be satisfied. Activities and social group. It is possible that the defining characteris- tics of such groups closely resemble the characteristics 3 of unmediated groups. Although the model can accommodate numerous users, only two are depicted here in the interest of simplicity and adher- ence to space limitations.

2 For a more in-depth discussion of the concept of hybridity in 4 This common ground is graphically represented as the base of the context of locative media see [1]. the pyramid in figure 1 4

objectives are typically subdivided to actions and goals . What is the role of locative media users as mem- respectively. Actions are tasks the subject performs in bers of mediated groups? Are users indeed “agents” order to fulfil the goals, which are “steps” towards the and is, accordingly, locative media use a form of social accomplishment of the objective and will thus bring the action? activity closer to completion. Actions can be further . How do factors such as real or virtual environ- divided to operations, which are tasks that are per- mental elements influence the generation of social formed automatically and do not require significant meaning on the part of the communicating users? amount of cognitive resources. . How do users cope with the hybrid nature of such spaces? To what extent does real space become a vital The proposed model incorporates the distinction be- part of a location-aware device’s interface? tween activity and action and, consequently, the one between objective and goal. Either combination (activ- . How does the configuration of such hybrid spaces ity – objective or action – goal) may apply depending affect the course of user activity? on whether locative media use is an objective in itself. Each user’s objectives or goals define his or her activity Our main research objective is to investigate whether or action respectively. Activity or action is placed at the locative media support the relationship between group top of the pyramid in figure 1, as it affects or is af- members, and how effectively they do so. To that end, fected by all other elements in the model. More specifi- LOCUNET will adopt a methodological approach that cally, the environment is enhanced by the addition of includes both quantitative and in-depth qualitative re- digital information in it, and this digital layer may be search, for the purpose of evaluating the system and accessed by the users through the medium. In other users’ attitudes towards locative media. Quantitative words, the medium gives users the ability to sense and methods include questionnaires and system logs, while act upon digital content bound to a real-world, physical qualitative approaches include observation, interviews location. In Activity Theory terminology, the medium and focus groups [2,4]. enhances the user’s capabilities, thus forming a func- tional organ through which the user may take advan- Furthermore, we attempt to determine whether and in tage of a novel form of environmental features. which way mobile locative media contribute to the con- stitution of what we call "social world", a hybrid and Research issues to be addressed synthetic formation that is constituted by non- LOCUNET aims to investigate certain issues related to homogeneous correlations of human and non-human inter-group relations and group formation, and the rela- entities. The answers to these questions are expected tionship between individuals and the environment in to provide a valuable theoretical grounding for the the context of locative media. More precisely, our re- processes of developing locative media and investigat- search will attempt to provide answers to the following ing their use as an act of communication. questions: 5

References [4] Hare, P. & Bales, R. F., Seating Position and Small [1] Charitos, D. Precedents for the design of Locative Group Interaction. Sociometry. 26, 4 (1963) 480-486. Media as hybrid spatial communication interfaces for [5] Kaptelinin, V. and Nardi, B. Acting With Technol- social interaction within the urban context. In: Isomäki, ogy: Activity Theory and Interaction Design. Cam- Η. (ed.). Future Interaction Design II, Springer Verlag bridge, MA: The MIT Press (2006). (in press). [6] Maisonneuve, J. Introduction à la Psychosociologie. [2] Diamantaki, K., Charitos, D., Tsianos, N., Lekkas, Presses Universitaires de France (1977). Z. Towards Investigating the Social Dimensions of Us- ing Locative Media Within the Urban Context. Proc. In- [7] McCullough, M. Digital Ground: Architecture, Per- telligent Environments 07 (2007), 53-60. vasive Computing, and Environmental Knowing. Cam- bridge, MA: The MIT Press (2004). [3] Dourish, P. Where the Action Is: The Foundations of Embodied Interaction. Cambridge, MA: The MIT [8] McQuail, D. and Windahl, S. Communication Models Press (2001). For The Study of Mass Communication. London and New York: Pearson Education Ltd. (1993).

Moblogging the City: Accessing Personal Perceptions of Urban Spaces

Abstract We describe an example of the use of moblogging as a research method to investigate perceptions of the city. Here, mobile phones are used to explore personal Tim Jay neighbourhood schemas. University of Bath, UK [email protected] Keywords Moblog, Neighbourhood Danaë Stanton Fraser University of Bath ACM Classification Keywords [email protected] H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.

Introduction Cityware is a three year interdisciplinary project with the aim of developing theory, principles, tools and techniques for the design, implementation and evaluation of city-scale pervasive systems as integral facets of the urban landscape. A key aim of Cityware involves increasing our knowledge and understanding of people’s relationships with urban spaces. This includes factors such as pedestrian flows, methods of wayfinding, use of landmarks and patterns of social encounters. We are interested in defining space in terms of physical, social and digital senses. 2

This city-schema can be difficult to access. Within nature, the means of moblogging are always available Cityware, a range of methods have been employed in to a smartphone user. As soon as a thought occurs, it order to generate a picture of the city from the point of can be recorded. view of its users. Space syntax methods have been used in order to predict pedestrian movements around Our Study the city [1]. Bluetooth gate counts have been used in An innovative part of the Cityware project is our order to analyse actual movements of Bluetooth recruitment of a cohort of 30 participants for the 3-year devices around the city [2]. Work is also in progress duration of the project [5]. These participants all that is concerned with the analysis of social networks in regularly visit the city of Bath, and 21 live within the the city. The research presented in this paper reviews city boundary. Each member of the cohort has been one study aimed at addressing the issue of city issued with a Nokia N70 mobile phone, a Qstarz perceptions and use in detail. Bluetooth GPS unit and a Vodafone SIM card. Here we review one study carried out with the cohort which Moblogging as a Method involves the use of moblogs to learn about our cohort’s Smartphones can generate content in a number of perceptions of urban space. formats, providing the means to depict segments of a life in some detail. Photos, videos, audio recordings, The first stage of this investigation involved a study of text notes and GPS trails can be produced and collated neighbourhoods. We asked participants to sketch a map relatively easily. Collections of material such as this, of their neighbourhood. We intentionally left the collected over time, are defined as mobile logs, or definition of ‘neighbourhood’ open so as not to ‘moblogs’. Moblogs are increasingly being used as a influence the activity in any way. Figure 1 shows one tool for researching perceptions and behaviour. Okade participant’s sketch-map. et al (2005) used moblogging in order to investigate participants’ use of mobile media devices [3]. Hulkko et Next, participants were asked to take a tour of their al (2004) have used moblogs as ‘cultural probes’, neighbourhood, as if showing a friend around. Mobile generating a detailed picture of participants’ lives [4]. phones were used to record GPS trails (via the Bluetooth GPS units) for the journey. Participants were The use of moblogs in research is essentially an asked to use photos, video, audio and text in order to extension of the diary study. However, moblogs have document their tours. Data from the tours were collated significant advantages over a pencil-and-paper diary in and used to create a mashup visualisation. The GPS research contexts potentially increasing the depth of trail was overlaid on a Google Earth image of the area, possible content. An annotated photograph, or a video, and icons for photos, video, audio and text were can provide more detail about a person’s thoughts in attached to the trail. Figure 2 is an example of one of context (of course, traditional written diary entries can these mashups – from the same participant as also form part of a moblog). In conjunction with this produced the sketch-map in Figure 1. level of detail is a very high level of immediacy. By its 3

minutes walk from home, and a larger area encompassing regularly used shops and services (in figure 3, the participant has drawn a loop to the north to encompass a regularly used gym). Rural residents tended to represent their neighbourhood using a single scale.

Rural residents’ landmarks were traditional, including churches, pubs, schools and the like. Urban landmarks were much more personal to the individual. Examples include particular shops, friends’ houses, dentist and allotments. One particularly personal example of a landmark was an empty building that had once been a pub. This was a landmark for one participant because, when open, this pub had been the source of a lot of noise and the participant was glad that it had closed! figure 1. Neighbourhood sketch-map

Following the generation of neighbourhood tours, participants completed some further activities, including drawing a second sketch-map, completing a questionnaire focused on landmarks in the neighbourhood and marking the boundary of their neighbourhood on a printed map, centred on their home. The boundary drawn by the same participant as in the examples above is shown in figure 3.

Key Findings Initial analysis has focused on the scope and scale of neighbourhoods and on the use of landmarks. For example, to explore perceptions of the scale of neighbourhood, we compared participants’ figure 2. Neighbourhood tour mashup representations of their neighbourhood across each format (sketch-map, tour, boundary-drawing). In Conclusion summary, we find urban residents tended to show two The moblog has proved to be a valuable tool in categories of neighbourhood – a ‘local area’ roughly 4-5 accessing personal perceptions of space. In 4

combination with the other methods used, we have We will also start to explore the ways in which the been able to start to explore personal perceptions of digital landscape (Bluetooth/Wifi activity) plays a part urban space. One of the advantages of having a cohort in participants’ behavior. of people working alongside the project means that we Acknowledgements will be able to explore these perceptions over time. Cityware is funded under the EPSRC WINES programme. We thank our Cityware colleagues and all of the participants in the project.

References [1] A. Fatah gen. Shieck, A. Penn, V. Kostakos, E. O'Neill, T. Kindberg, D. Stanton Fraser, and T. Jones, "Design tools for pervasive computing in urban environments," in Innovations in design & decision support systems for architecture and urban planning: Springer Netherlands, 2006. [2] E. O'Neill, V. Kostakos, T. Kindberg, A. Fatah gen. Shieck, A. Penn, D. Stanton Fraser, and T. Jones, "Instrumenting the city: Developing methods for observing and understanding the digital cityscape," Proceedings of the 8th International Conference on Ubiquitous Computing, 2006. [3] D. Okabe, K. Anderson, S. D. Mainwaring, and M. Ito, "Location-based moblogging as method: New views into the use and practice of personal, social and mobile technologies," presented at Seeing, Understanding, Learning in the Mobile Age, Budapest, Hungary, 2005.

[4] S. Hulkko, T. Mattelmaki, K. Virtanen, and T. Keinonen, "Mobile probes," presented at 3rd Nordic figure 3. Neighbourhood boundary drawing conference on human-computer interaction, Tampere, Finland, 2004. The depth of detail and the immediacy of moblogging to document experience has provided a [5] T. Jay and D. Stanton Fraser. The Role of a level of insight that would be impossible to obtain using Cohort in the Design and Evaluation of Pervasive any other (unattended) method. The data produced will Systems. Proceedings of Designing Interactive Systems feed into the design and development of urban (DIS 2008). Cape Town, South Africa (In press). pervasive systems. In the next month we will be extending the use of these methods to the city centre.

12 Mixed Reality Principles of Animation – Based on Disney’s Principles of Animation

Sabiha Ghellal Rod McCall Abstract Sony Europe GmbH Fraunhofer FIT This paper presents some of our research into shaping Kemperplatz 1 Schloss Birlinghoven content and technologies in urban environments. Using D-10785 Berlin D-53754 Sankt Augustin Disney’s principles of animation we created 12 [email protected] [email protected] principles for Mixed Reality (MR) 3D content creation

and animation. Mainly focusing on interaction issues Jan Ohlenburg Steffen Harrer within urban environments the application scenario Fraunhofer FIT Freelance 3D Artist “Leo’s Adventures” features a mixed realty user Schloss Birlinghoven Ratiborstr. 8 generated content tool which features a web 2.0 based D-53754 Sankt Augustin D-10999 Berlin application. Web users can upload videos and animate [email protected] [email protected] 3D animations into their video via a simple to use post

productions tool. The conclusion describes the work so Joachim Rothauer far and the questions within the paper. Sony Europe GmbH Kemperplatz 1 Keywords D-10785 Berlin Disney’s Rules of Animation, Presence, 3D, Urban, [email protected] Mixed Reality, Augmented Reality, Web 2.0, Video, Post

production, Six Degrees of Freedom,

ACM Classification Keywords H5.2. User Interfaces: Graphical user interfaces (GUI), input devices and strategies. H5.1. Multimedia Information Systems: Artificial, augmented, and virtual realities; video. 2

Introduction Since Disney’s [1] principles where created, animation Disney’s Principles of Animation techniques and styles, and the scope of productions, The twelve principles of animation created in the early have dramatically changed. We have moved from 1930s by animators at the Walt Disney Studios were hand-drawn pose-to-pose cartoon narrative animation used to guide production and to train animators. By to non-linear interactive videogames, non-linear applying these principles many of the early feature editing, compositing, motion capture etc. Modern 3D films such as Snow White [3] (1937) or Bambi [4] artists have rewritten the principles to fit the new (1942) have proved to be a great success. technology tools and narrative trends [2]. In order to cater for the special circumstances of merging the real Even though animation techniques and styles, and the with 3D animations we rewrote Disney’s principles e.g. scope of productions, have changed since the 1930s the Squash and Stretch rule illustrated in figure one, the principles still apply as they still enable a designers where the character Leo squashes on the ground and and animators to create believable characters. stretches in the air when he jumps up. The main goal of applying Disney’s rules of animation was to create a 12 Mixed Reality Principles of Animation step by step instruction of how to produce appealing 3D The 12 MR principles of animation focus on stipulating content for MR applications. rules to ensure that design methodologies a suitable for interaction issues within urban environments.

SQUASH AND STRETCH: Traditionally applied to give the illusion of weight, volume and gravity this principle also applies to MR content creation with an emphasis on ensuring that the created character matches the gravity details of real environment in which he/she or it will be placed.

ANTICIPATION: If narration is used in an MR animation then care has to be taken to ensure appliance in various real world environments. Tests should be Figure 1: Leo's Squash & Stretch carried out in order to test the validity of the story across various locations always considering the unknown factors stipulated by the real environment. 3

STAGING: Since the MR animation will be staged in the slowing down the beginning and the end of an real world it is important to visit and examine the animation one can not only add realism but also ensure intended environment so that the animation can be that MR users get a chance to process any additional correctly placed. It is essential that six degree of information. freedom orchestration tools are used so that virtual (MR) characters and objects can be correctly placed ARCS: No creature moves in straight lines, making the e.g. a construction worker should be placed next to the creature move in an arc in the real world adds the hole they are digging and not floating above it. complexity of fitting the arc into the real environment.

ANIMATION METHODS (Originally named STRAIGHT SECONDARY ACTION: Again, the secondary action may AHEAD & POSE TO POSE): MR applications often be provided by the real world in the MR environment however there is also the possibility to add secondary require interactive animation. This means that a action to the main MR animation to improve narration. content creator requires a tool for placing animated 3D characters into an interactive graphical application. In TIMING: Timing is also a core element of a MR addition the heavy rendering capacity requested to run animation however here are more factors that have to a MR application will make it necessary to control the be taken into consideration. These include time that is level of detail of the 3D character by reducing or stipulated by the real environment and the time a user needs to move around in the real environment. increasing the polygons that make up the character at runtime. This requires a continuously flexible character EXAGGERATION: In a MR application it is sometimes animation tool. necessary to exaggerate in order to fit an animation into the real world e.g. the colors and contrast have to FOLLOW THROUGH AND OVERLAPPING ACTION: For be exaggerated in order to make an animation visible in MR Animations it is necessary to define the borders in a bright environment. the real environment in which the character will move. E.g. a tree or a house should be treated as obstacles in SOLID DRAWING: Even though rendering capacities for order to prevent the character to walk through them. MR applications are not ready to deal with highly One possible approach could be to create transparent pixilated animation details such as hair, it is important obstacles within the animation or one could use to apply solid drawing. Only if a character or a 3D computer vision tracking for realistic augmentation of a image is believable will it be able make real people feel character in the real world. present with it. The trade off between light animations and realism of the character or image will have to be SLOW-OUT & SLOW-IN: To create believable characters carefully considered. and animations in the real environment one has to blend characters into the selected environment. By 4

APPEAL: Most MR animations have been produced by scientific researchers with an emphasis on the technical solution; which means that the appeal of the animations was secondary. However successful commercialization of MR 3D animations will rely upon broadening the appeal. Figure 3: Leo in Various Environments

Application Scenario For Leo’s adventures we developed the following objectives based on the principles: Create a believable and appealing character, see figure 2. Allow the users to select 3 locomotion animations: jump, hover and turn (see figure 4). Here it is up to the user whether he/she will place the character on the ground or let’s Leo hover in from the sky.

Figure 2: Leo in a Real Environment

Leo, the main 3D character (see figure 2) has been created and animated especially for mixed reality environments. Leo is based on projections and his semi transparent features make it easy for Leo to blend into several real environments see figure 3.

Figure 4: Locomotion Animations

Create a set characteristics that can merge with diverse narration styles see figure 5. 5

The prototype has been designed to allow easy placing, resizing, web based locomotion interface and mood selected of the character.

We tested our initial designs & animations in peer studies where we asked four selected designers to comment on the moods and movements provided. The result was that the animations appear suitable but the peer testers where skeptical if web 2.0 users will like them and if they will be able to successfully use the web 2.0 content creation and navigation interfaces.

Conclusions and outlook Figure 5: Mood Animations We managed to create an appealing (and MR suitable) set of animations but share the concern of the peer For the initial prototype we selected happy and sad, evaluators. We are in the process of creating the MR winking and kissing as the main mood animations. user generated web interface using the animations described earlier and are planning to conduct an Create a web interface low fidelity prototype where the explorative evaluation. The evaluation will explore how animations can be navigated easily by average Web 2.0 users find the process of creating MR content via a web users. See figure 6. interface.

Acknowledgments The presented work is partially funded by the European Commission (IST program, project no. 27571). Special thanks to Nina Dautzenberg the 2D Leo designer.

2.) Web based References locomotion tools [1] Disney’s Rules of Animation 1.) Placing & resizing [2] From Isaac Kerlow's “ The Art of 3D Computer Animation and Effects” 2003. 3.) Mood selection [3] Snow White (1937), Figure 6: User Generated MR Mock-Up [4] Bambi (1942) [5] Snow White (1937), Bambi (1942)

From City Poems to Aliens, Monsters and Sprites that Live in Phones: A Brief History of Story Worlds

Abstract Beginning with City Poems in 2003, this paper presents a brief history of the evolution of ideas and techniques for creating “story worlds”, that is, stories that employ Andrew Wilson and Derek the contexts and practices of mobile phone use as tools Hales to generate and experience a narrative. Blink Ltd and Digital Research Unit University of Huddersfield Keywords Bates Mill Artspace, Narrative, Story, Paracosm, Fiction, SMS, Text Fairfield Mills, Messages, Mobile, City, Sharing, Context, Vernacular, Milford St, Creativity, Writing, Paper, Interactive Huddersfield HD1 3DX UK ACM Classification Keywords [email protected] H5.m. Information interfaces and presentation (e.g., [email protected] HCI): Miscellaneous.

The World of a Story I had in my hands a substantial fragment of the complete history of an unknown planet…with its architecture and its playing cards, its mythological terrors and the sound of its dialects, its emporers and its oceans, its minerals, its birds and its fishes, its algebra and its fire, its theological and metaphysical arguments…[4]

Jorge Louis Borges: Tlon, Uqbar, Orbis Tertius 2

City Poems Launched on February 14th 2003, City Poems[5] was Borges portrays in Orbis Tertius, an effort to imagine Blink’s first attempt to create a story world (Blink[3] (and thereby create) a world: Tlon. This paper are an arts organisation with a long standing interest in discusses locative [12] storytelling as a subset of literature and portable technology, and part of the Mixed Reality and relates this to simple acts of creating University of Huddersfield’s Digital Research Unit[7]). It shared, imaginary or make-believe worlds. The make- was a non-fiction story, being a biography of the city of believe worlds that we are interested in the context of Leeds, written by the people who live and work in the locative storytelling relate to those described as city, and delivered direct to mobile phones from a ‘paracosms’. [6][13]Paracosms are persistent and network of Poem Points at key locations throughout the consistent evocations of imagined places, sometimes city. inhabited by imaginary friends and creatures, shared paracosms are the setting of a pretend game and Poem Points were identified by signs. include the general tenets of the setting and provide its narrative components. Collaborative play in paracosms parallels that in socially generative pen and paper role- playing, LARPs and contemporary Alternate Reality Games[8]. Each of these requires that situations of clashes in the construction of the imaginary world or that require modifications of that world, to add elements or describe story components, are negotiated by the players. [2]

A Story World The development of portable communications devices that let us access databases from anywhere means that Figure 1. Poem Point signs at a hospital and a bar. the world of a story can be turned inside out. A story The signs featured written instructions explaining to world, a paracosm, is now the everyday cityscape that users how to read a poem about the place they were in, we move around in. Stories are to be found by going for example a hospital or a bar, by sending a key out into it. The success of a story in this context numeral (the numerals 6 or 3 on the signs in Figure 1) depends in part on how uniquely and richly it responds as a text message to the City Poems mobile phone to the world in which it is found, rather than how number, and a poem would be sent back to their convincingly a fictional world is created within the mobile by SMS. This could be repeated up to 10 times pages of a book. at each Poem Point. 3

If users wanted to add their own poem, they sent it as Anywhereblogs is being used for FREE MANCHESTER’S a text message to the City Poems phone number. MONSTERS![10], a campaign of political action by the monsters who have lived in the area covered by the Around 3000 poems were downloaded over 2 years, city of Manchester since long before the industrial and around 400 uploaded. While these figures certainly revolution, but who are now “trapped under the tall make for a successful work, they could have been much buildings made of brick and stone and concrete”[10]. higher, but for the one problem with using printed Citizens of Manchester are invited to take part in the signs: host venues took them down within a few campaign by choosing a place in Manchester, imagining months, if not weeks, assuming the project was over. what kind of monster would once have lived there, and planting the monster in that place using the Sadly, City Poems had no resources to conduct Anywhereblogs system. Users are told that “monsters evaluation with users, but it seems likely, given the are set free as soon as you conjure them up”[10]: once number of poems downloaded, that the citizens of a monster has been imagined, written about and can be Leeds were genuinely interested to discover the traces found by other people, it exists and is as real as any of other people who they shared the city with, and other part of Manchester. So far around 40 monsters understood that a poem could be an intimate, have been set free, and they are represented on a meaningful response to a place. google map[11].

FREE MANCHESTER’S MONSTERS! One of the attractive things about story worlds is that City Poems was a story world with an editor: Blink there is no room (or too much room) for a Hollywood or decided where to put the 20 Poem Points, choosing Harper Collins (the publishing conglomerate owned by places that best described the city of Leeds. Users Rupert Murdoch) to develop. Story worlds have to be could submit poems to be added to the system, but intimately linked to a particular place and the user’s there was no mechanism for them to choose their own understanding of it, and it would simply be too big a places for Poem Points, no matter how much they task for a centralised agency to broadcast a rich, might have wanted to mark a place that was meaningful experience that was different on every meaningful to them. street corner of every cityscape in the UK, let alone worldwide. Story worlds are a folk literature, that must It seemed important that users should be able to create be created by users. FREE MANCHESTER’S MONSTERS! their own places – to write their own story worlds – and makes a connection between location and literature by Blink’s Anywhereblogs[1] platform allows them to do asking users to create genius loci, spirit of the place, of so. Again using text messages, Anywhereblogs lets the kind found in Shinto mythology. users enter any location that is important to them into the system, and add and receive messages, for Users do seem to have imagined that the monsters example poems, about that place. embody the spirit of the place. For example, this disturbing monster was created about Strangeways 4

prison, already the title of an album by Manchester “YES! Thank you. I am attaching my stretch powerpad group The Smiths (“Strangeways Here We Come”): now. I will be able to get fired into orbit. Just two more powerpads to go.” “STRANGEWAYS CONVICT HE LIVES IN THE WALLS OF THE PRISON WAITING 2 POUNCE ON FRESH NEW Echo asks children to send him suggestions for a new PRISONERS AND UNLEASH HIS WICKED WAYS!” “power pad” to add to his space ship, and the replies sent to Echo by text message suggest a high level of Most monsters have been more humorous: engagement between players and the Echo character:

“The monster who lives here is mini as its a small “OXYGEN PAD SO YOU CAN BREATH IN SPACE WHEN street.he helps clean up when 100s of people come out YOU TRAVEL AROUND PLANETS MARC RAVENHALL drinking in the sun-a kind little kretin” AGED 8”

Echo the Alien “the new powerpad would be able to translate any alien It may be that City Poems and FREE MANCHESTER’S languages in the universe so you can make new MONSTERS! are third person narratives (though this friends. From Affelia aged 17.” distinction is open to question). Users read about a city and about the monsters that live there but there is no These messages address Echo directly as “you”, clear sense of a voice telling us these things. suggesting that the users have suspended disbelief and are happy to talk to Echo as though he were real for Echo[9] is a first person narrative. It is made up of a the duration of the story. When we use a mobile phone, short printed booklet in which Echo, an alien schoolboy, we expect to send messages and receive replies from addresses the reader directly: other people and so making conversations by SMS a part of a story world is a powerful tool for creating an “Greetings...my name is Echo and I am just like you. illusion. It may be that this experience is richer than Except I come from a tiny purple planet called Fizzix, using more complicated devices that have multi media which is 100 light years past the moon.” capabilities but are not so commonplace in the everyday world. Echo asks children to help him repair his spaceship by answering questions about physics, and they do this by Next Steps and Concluding Remarks sending text messages, to which Echo replies by text, as though in a conversation. An example of Echo’s reply by SMS is: The above projects illustrate practice-based research in the field of locative media, and the intersection of social interaction, place and the imagination, through mobile, 5

text-based and narrative approaches to urban mixed [3] Blink reality. http://www.blinkmedia.org/mainsite/index.asp accessed 13.02.2008

[4] Borges, J.L, :Tlon, Uqbar, Orbis Tertius Porcupine's Quill (1982)

The next step in our development of story worlds will [5] City Poems. http://www.centrifugalforces.co.uk/citypoems/pages/0 be to use phones with Near Field Communication (NFC) 1_05.html accessed 13.02.2008 capacity to give users a stronger illusion of the presence of genius loci by allowing them to pick up [6] Cohen D, Mackeith S.A, The Development of Imagination: The private world of Childhood. London, sprites on their phones and pass them from hand to Routledge, 1991 hand, again talking directly to the sprites by text [7] Digital Research Unit message. http://www.digitalresearchunit.org/ accessed 13.02.2008 The research shows that co-created urban mixed [8] LifeLike, Knudepuknt 2007 Conference proceedings realities do not necessarily need to be shared through Eds. Donnis,J. Gade,M., Thomp,L www.knudepunkt.org representation or presentation using video or graphics [9] Echo technique. The locative story worlds illustrated in this http://fisharepeopletoo.blogs.com/1/2007/10/i-come- paper show how text-based, locative, participatory and from-a-t.html accessed 13.02.2008 shared electronic spaces can successfully augment [10] FREE MANCHESTER’S MONSTERS! urban experiences through shared and imaginary http://fisharepeopletoo.blogs.com/1/2007/05/free_man worlds. The co-creation of user generated narrative plot chester.html accessed 13.02.2008 and playable content produced in this process, forms [11] Google map of FREE MANCHESTER’S MONSTERS! the basis for future interdisciplinary research, http://tinyurl.com/2jwle2 accessed 13.02.2008 particularly in critiquing co-created story worlds [12] Hemment, D., ed. Leonardo Electronic Almanac through the empirical analysis of these as mixed reality special edition on locative media. ‘paracosms’[6][13] imaginary places and imaginary http://leoalmanac.org/journal/Vol_14/lea_v14_n03- characters. 04/guested.asp accessed 13.02.2008 [13] Hoff, E. Imaginary Companions, creativity and self image in middle childhood. Creativity Research Journal 2005 vol 17 no.2 and 3 pp167-180 [1] Anywhereblogs http://www.ablogs.org/ablogs.htm accessed 13.02.2008 [2] Berger, P., LucKmann T., The social construction of reality (1966) Penguin London 1991