Superfight I:The Battle to Understand a Space Through

SuperFight I: The battle to understand a space through the behaviour of its occupants

Karen Martin

This thesis is submitted in partial fulfilment of the requirements for the degree of Master of Science in Virtual Environments from the University of London

Bartlett School of Graduate Studies University College London September 2003

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

Abstract This dissertation describes an interaction design project which investigates whether the symbiosis of physical and digital environments might be used to create a stronger sense of ‘place’ for the occupants of the physical space. Sensing technology, implicit interaction, ambient interfaces, game strategy and a network connection were combined in an attempt to increase participants awareness of their physical actions and location. The principles and theory underpinning this project are discussed, after which a list of criteria for an interactive system designed for public spaces is drawn up. The design of SuperFight I is described and evaluated in relation to this theoretical background. Finally suggestions are made for future areas of research that might be undertaken in order to develop the system further.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

Contents

1. Introduction 1.1. Hypothesis 1.2. Overview of this dissertation 1.3. Contribution of this dissertation

2. Design process and methodology

3. Principles and theoretical background 3.1 Displaying interaction: Output devices 3.1.1 Individual display devices 3.1.2 Large-screen display devices 3.2 Action and interaction: Input devices 3.2.1 Environment as input device 3.2.2 Spatial metaphors 3.2.3 Incidental interaction 3.3 A sense of place in digital environments 3.4 Play and Games 3.5 Criteria for an interactive system for public space

4. SuperFight I Project 4.1 Design considerations 4.1.1 Action as information 4.1.2 Encouraging participation 4.1.3 Games, play and competition 4.1.4 Design of the visual content 4.1.5 Arrangement in physical space 4.2 Construction / Build 4.3 Description of experiment 4.3.1 Pre-installation communication 4.3.2 System Evaluation 4.4 Research outcome and application 4.5 Analysis and interpretation of data 4.5.1 Evaluation of pre-installation communication 4.5.2 Results of SuperFight I experiment

5. Future work

6. Conclusion

Appendix I. Initial communication with participants Appendix II. Potential action / response pairings Appendix III. Observational study of a space Appendix IV. Data from experiment log files Appendix V. Questions for participants

Bibliography

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

List of figures

1. CAVE environment at UCL 2. Example of head-mounted display 3. Example of view through headset of augmented reality system 4. Combat situation in Pirates! game 5. Photograph of ambient displays at a train station 6. Photograph of Groupcast system. 7. Photograph of the set up of the Opinionizer project 8. Players participating in Brainball game 9 & 10. Photographs of players of Metropolis Challenge 11. Diagram showing relationship between intended and incidental interactions 12. Photograph of Active Badge 13. Media Cup tracking object 14. Series of images showing the relationship between remote and local participants in Mixed Reality Architecture 15. Screenshots of Pirates! game 16. Screenshot of Pirates! hi-score board 17. A losing player suffers in a PainStation game 18 & 19. Photograph and screenshot showing the runner and online game environment of Can You See Me Now? 20. Screenshot of partially built tower, created by movement in physical space 21. Screenshot of MArch tower wobbling due to noise levels in the Virtual Environments studio. 22. Two screenshots showing the result of the ‘special move’ 23. Series of examples of experimental graphics for SuperFightI 24. Screen shot of the end of a SuperFight game 25. SuperFight I hi-score screen 26. Plan showing how SuperFight I system was arranged in Virtual Environments studio 27. Plan showing how SuperFight I system was set up in MArch studio 28. Photograph of SuperFight I projection in MArch studio 29. Photograph of SuperFight I projection in Virtual Environments studio 30. Diagram showing technical setup of SuperFight I system 31. Screenshot of opening game position 32. The posters that were put into the studios prior to the installation of SuperFight I. 33. Photograph of objects intended to be used as ‘cultural probe’. 34. Photograph of the alterations to the poster in the MArch studio. 35. Sample readings from log file. 36. Graph showing activation of break-beam sensor on first day of installation 37. Sample visualisation of log file information showing patterns of interaction

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

With very many thanks to:

Alan Penn All the MArch students All the Virtual Environments students Cristiano Bianchi Dong Seop Kim Jody Hudson-Powell Lesley Gavin Nancy Diniz Olujide Esan Stefan Kueppers Stephen Gage Unit 14 Wonjoo Park & Jon

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

5 1. Introduction 1.1 Hypothesis New communications technologies disregard for geographical constraints is generally perceived as a good thing. While the ability for individuals to communicate from any location is undoubtedly useful, enjoyable, life-enhancing and, in certain situations, life-saving, I am interested in the idea that the invasion of public spaces by communications technologies often reduces the level of engagement between individuals who share that physical environment.

Watson, please come here. Alexander Graham Bell. FIRST WORDS SPOKEN ON THE TELEPHONE.

Inhabitants of the same geographical space are sharing a spatial and temporal experience, perceiving identical sensory inputs. The implicit understanding of what it means to be 'right here, right now' creates links between people, giving a set of disparate individuals a sense of community. But communications technologies remove the user's focus from their physical location and re-focuses it in some 'third' space'[MULLER. 2001] or 'digital' space [MUIR. O'NEILL. 1995]. What is the consequence of this on the shared experience of the co- inhabitants of a place?

Can technology help the physical space to fight back and increase interactions between inhabitants?

This project aims to discover whether the symbiosis of digital and physical environments can refocus the attention of inhabitants on their actual location by increasing their awareness of the space and its relationship in space and time to the wider environment, and by making them aware of their actions within the space.

1.2 Overview of this dissertation This dissertation is divided into six sections. The first section introduces the subject of this dissertation as well as outlining what it contributes to the field of research. In Chapter 2 I describe the design process I followed to develop this project. This chapter outlines the unique challenges faced by interaction designers when using design as research. Chapter 3 takes a look at related work. This chapter is divided into four sub-sections, the first two sub-sections of this chapter look at the various input and output devices used by interactive systems in public spaces. The next sub-section give an overview of the role of ‘place’ in digital environments and the final sub-section examines how play and computer games engage participants, and how new forms of computer-mediated play are emerging out of mixed reality and ubiquitous computing. To conclude this chapter I suggest a framework for the design of a digital environment capable of raising participants’ awareness of

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

their physical environment. Chapter 4 describes the design and evaluation of SuperFight I, a system which attempts to exemplify these criteria. This sub-section examines the design considerations faced during the development of this project. The second sub-section describes an experiment in which SuperFight I was installed into a public space for three days. The results from this experiment are evaluated in the final sub-section of Chapter 4 and the design of the system is considered in light of these results. Chapter 5 suggests areas for further research. Finally, in the conclusion I summarise the SuperFight I project and consider the results from the testing of the system in relation to the initial hypotheses.

1.3 Contribution of this dissertation The aim of this dissertation is to investigate how interactive digital environments might increase participants’ awareness of their physical environment. The contributions of the dissertation include:

1) A design process for using interaction design as research.

2) An overview of existing interactive systems which integrate the physical and digital environments with the intention of increasing awareness and social interaction between participants.

3) A framework for the design of a digital environment which aims to increase participants awareness of the physical space.

4) A description of the design of the SuperFight I project, illustrating these criteria.

5) An evaluation of the design and testing of SuperFight I, including the strengths and weaknesses of the visual, technological and methodological aspects of the project.

6) Suggestions for further research into the design of interactive systems which increase participants awareness of the physical environment based on the lessons learnt by SuperFight I.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

2. Design process and methodology Design as research is used as a way of provoking further questions to be investigated [DUNNE. 1999]. It is difficult to reach a definitive conclusion about the success or failure of a hypotheses tested by design because users’ acceptance of design is so closely tied to cultural, social and environmental factors. consider the very different experiences of two seemingly similar uses of video to link public spaces, one at Xerox (reported by Olson and Bly [1991]), and one at Bellcore (reported by Fish et al. [1990]). Both experiments linked public spaces in R&D office environments with audio and video, to foster informal communication. However, the groups had very different experiences of the successes and failures of their connections.

Steve Harrison and Paul Dourish. RE-PLACE-ING SPACE: THE ROLES OF PLACE AND SPACE IN COLLABORATIVE SYSTEMS. 1996

The difficulty in obtaining objective, conclusive results makes it essential that the methodology of the project - the question to be investigated, all possible outcomes of the experiment, and the evaluation methods - is established in the early stages of the design process. This defines the focus of the research. Careful preparation is important for interaction design projects because of the open-ended nature of the design solutions. The goals of interaction design are quite distinct from the goals of other design fields. Yvonne Rogers of the Interact Lab at the University of Sussex describes the difference as being:

Rather than make people’s lives easier through using technology to support our activities or mediate our communication, etc, we are interested in how combinations, mergings and mixings of the two can enhance, extend, and enrich people’s lives. The shift of emphasis towards ‘adding something’ that was not there before is quite different from the traditional goals of HCI, which are to improve upon the way people do things (e.g. make it easier, quicker, less errors).

Yvonne Rogers. UNDERSTANDING INTERACTION: GOING BEYOND THE SEAMLESS INTEGRATION OF THE DIGITAL AND THE PHYSICAL. 2002.

Very often a project’s aim is to enable participants to alter or appropriate the design in some way. This makes it almost impossible - and undesirable - to control the outcome of the experiment. Consequently many traditional evaluation techniques are redundant in the case of interaction design and new methods for measuring and evaluating interactions are being experimented with.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

Traditional evaluation techniques like questionnaires, experiments, surveys, etc seem totally inappropriate for understanding the interactions we are trying to design for. Likewise, usability criteria, like efficiency, ease of use, learnability, etc., are inappropriate criteria by which to assess the kinds of interactions we are designing for. Other, more nebulous concepts are needed (which of course will be much harder to cash in everyday concrete terms or indeed ‘numbers’). These currently include aesthetics, presence and suspension of disbelief, but we need to consider others, too. Collection of video, audio and observational notes can provide us with the data but what we do with it is critical. Ethno-based analyses can provide us with some in- roads, but we may need to develop new analytic frameworks by which to ‘measure’ our designs. Existing frameworks, like discourse analysis, conversational analysis, distributed cognition are from a different era, developed to address the specific needs of the interactions that were the focal points at the time. Hence, we need to consider what kinds of alternative ‘interactional’ analyses we want to develop, that enable a better understanding of interaction.

Yvonne Rogers. UNDERSTANDING INTERACTION: GOING BEYOND THE SEAMLESS INTEGRATION OF THE DIGITAL AND THE PHYSICAL. 2002.

To measure the extent to which a design project can change people’s behaviour requires an understanding of the usual behaviour of the participants. Once we know this we can make comparisons between behaviour before and after the introduction of the design. One way of establishing common behaviour is by observational study. The results from these studies can be surprising, even places with similar functions can reveal very different social rules and behaviour [HARRISON, DOURISH. 1996].

The methodology I used in this project was firstly to make an observational study of one of the spaces I planned to use for the installation. The results of this study informed the design and identity of the project. The information collected in the study also determined the nature of the pre-installation communication between myself and the occupants of the spaces. The project was installed in two spaces for three days, no instructions were given to inhabitants on how the system worked. During this time I watched how the system was being used and the action and interaction taking place around it. The system itself was used to record the interactions. Finally I emailed the occupants of both spaces asking them to answer a number of questions about their perception of the SuperFight I project.

The limitation of the methodology is that the results of the experiment are closely tied to the social and environmental situation in which the experiment took place. The choice of physical location, the form and characteristics of the visual content, how the design is arranged within the physical space, and the appropriateness of the system for the physical spaces might all influence the results.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

3. Principles and theoretical background

Interaction is nothing new but communications technology brings a further dimension of interaction into our everyday lives. As the Interactive Institute at Ivrea explains: computers and telecommunications allow people to interact indirectly. Interactive technologies have become a medium through which we interact with each other and with our environment; and they are transforming every aspect of our lives. http://www.interaction-ivrea.it/en/vision/interactiondesign/index.asp

Indirect interaction is often enabled by devices which remove the focus of our attention from our physical location and re-focus it in a ‘third’ [MULLER. 2001] or ‘digital’ space [MUIR. O’NEILL. 1995]. This has the effect of reducing awareness of our physical location and the other occupants of that environment. This chapter looks at assorted interactive systems which enable direct and indirect interaction between occupants of public spaces, and considers the effects of these systems on the participants’ perception of their physical environment.

Digital environments can be divided into four categories, immersive environments, augmented reality, mixed reality and ubiquitous computing. Each of these categories takes a distinctive approach to the integration of physical and digital spaces. Immersive virtual environments aspire to replace all sensory information from the physical environment with computer-generated information. In (1) Presence is defined as ‘the these systems total sensory immersion is considered an important psychological sensation of ‘being factor in increasing the sense of presence(1) felt by participants. In there’, having a sense of being in the field of immersive virtual environments higher levels of the place specified by the virtual presence are believed to achieve a stronger bond between the environment rather than just participant and the digital environment [SLATER, LINAKIS, USOH, seeing images depicting that KOOPER]. Augmented reality systems combine information from the place.’ digital and physical environments by overlaying the digital environment onto a mediated view of the physical space. The Juan S. Casanueva and Edwin H. participant must wear a significant amount of paraphernalia, and a Blake PRESENCE AND CO- large amount of technology is needed to keep the digital and PRESENCE IN COLLABORATIVE physical environments aligned. Both of these systems are user- VIRTUAL ENVIRONMENTS. 2000 centred, requiring every participant to wear specialist equipment before they can enter the digital environment.

The phrase ‘ubiquitous computing’ was first used by Mark Weiser(2). In 1988 he defined it’s aim as ‘enhancing computer use by (2) Chief technologist at Xerox making many computers available throughout the physical environment, PARC 1996 – 1999. but making them effectively invisible to the user’. Ubiquitous computing creates a responsive environment, aware of the actions of its occupants and able to respond accordingly. In mixed reality systems

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

participants co-located in a physical space and remote participants are combined in a shared, digital environment. The system is comprised of a digital environment creating the illusion that all participants are co-located in this third space.

Mixed reality and ubiquitous computing systems equip the environment for interaction instead of the user. Projects from these fields will provide the majority of examples in this chapter.

Figure 1. CAVE environment at UCL. Figure 2. Example of head- Figure 3. Example of view From mounted display. From through headset of augmented www.cs.ucl.ac.uk/research/ www2.dcs.hull.ac.uk/simmod/ reality system . From vr/projects.htm Technology/HMD.htm www.nottingham.ac.uk/aims/ ar-seminar/imain.htm

3.1 Displaying interaction: Output devices 3.1.1 Individual display devices All the various types of collaborative and shared environments can (3) Where transportation is be placed onto the continuums of transportation(3) and measured as ‘the extent to which artificiality(4) [BENFORD, GREENHALGH, REYNARD, BROWN, KOLEVA. the display of the remote 1998]. environment excludes the local While head-mounted displays are considered more transporting environment and the amount of information that is projected from than other display devices [BENFORD, GREENHALGH, REYNARD, the local into the remote.’ BROWN, KOLEVA. 1998] this high level of transportation is often accompanied by a corresponding reduction in awareness of co- inhabitants. Usually projects that use head-mounted displays are (4) The definition of artificiality the extent to which a space not overly concerned with supporting social interaction. MIT’s being ‘ is either synthetic or is based on Museum Wearable project makes use of individual eye-piece the physical world.’ displays but the aim of the project is to ‘deliver a personalized audiovisual narration to the visitor’ [SPARACINO. 2002] and not to Benford, Greenhalgh, Reynard, encourage social interaction. Brown, Koleva. UNDERSTANDING AND CONSTRUCTING SHARED While this [use of head-mounted displays] allows for many interesting SPACES WITH MIXED REALITY possibilities, a headworn display might be in the way when two or more BOUNDARIES. 1998 players are interacting socially. Partly for this reason, we chose to use a handheld rather than a head-worn device.

Staffan Björk, Jennica Falk, Rebecca Hansson, Peter Ljungstrand. USING THE PHYSICAL WORLD AS A GAME BOARD. 2001

As the designers of Pirates!, a multi-player game combining ubiquitous computing and wireless networks, explain in the quote

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

above, they chose to use handheld devices over head-mounted displays as the display device for the digital element of the game. However, the intimate relationship between the user and handheld device can have the effect of isolating the participant from their environment by focusing attention onto the display. Aside from the issue of whether or not head-mounted displays and handheld devices are effective at encouraging awareness and social interaction between participants, there is the question of the cost and convenience of the equipment. Using handheld or head-mounted displays means that every participant must be equipped with the technology. Unless the device is common enough that each participant might be expected to provide their own display device Figure 4. Combat situation in Pirates! (eg. mobile phones) then the cost could be prohibitive if researchers game. From Using The Physical World want to make the system available to many users simultaneously. As A Game Board

3.1.2 Large-screen display devices Large screen public display systems surround us in the shape of advertising billboards, timetable and information boards, video screens, message boards and posters. These displays act as ambient communicators, continuously broadcasting information whether anyone is paying attention or not. Consequently we focus on the information at appropriate times, returning the display to the periphery of our attention when the information is no longer required.

A number of projects examining how to increase awareness by the Figure 5. Photograph of ambient use of digital environments have re-interpreted community displays at a train station message boards as interactive displays. Three examples of this type of project are Greenberg’s Notification Collage (2001), XRCE’s Community Wall (2002) and FXPAL’s Plasma Poster (2002). Greenberg’s Notification Collage is a system designed for a work environment to allow co-workers to attach text messages, images and video to a shared display system that can be viewed either on a large public screen or a personal computer monitor. XRCE’s Community Wall was designed for an exterior public space and offers locals and visitors information on activities happening in the vicinity. FXPAL’s Plasma Poster project is a series of interactive display screens installed in a work environment, anyone in the environment can interact with the screens to read the content while Figure 6. Photograph of Groupcast authorised members of the community can also add content to the system. From Using Public Displays to screen. All of these projects aim to give people awareness of events Create Conversation Opportunities and activities taking place in the local area through the use of an interactive, public display screen.

Our level of awareness of ambient displays is not only affected by our need for the information provided but is also related to the position of the display in the space.

People’s initial understanding of a public display system is based on peripheral awareness, i.e. fleeting glances from a distance. This means that

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

they will judge the system rapidly on the ‘broad-grained’ details available to them at a distance. These should be designed to show the display and the activities around it in an attractive and easy to ‘pick-up’ way. It is also important to plan for bodily occlusion, and so placing the display at least partially above head height will allow people to see it from a distance.

Harry Brignull & Yvonne Rogers ENTICING PEOPLE TO INTERACT WITH LARGE PUBLIC DISPLAYS IN PUBLIC SPACES. 2002.

3.2 Action and interaction: Input devices 3.2.1 Environment as input device When the input device for an interactive system using a large-screen display is primarily single-user, people in the space other than the person controlling the system are reduced to the role of observers. This is demonstrated very well by the Opinionizer project. In this system comments inputted via a keyboard were projected in real- time onto a large display screen. The researchers found that this combination of input and output devices actually inhibited interaction.

Participants who were interviewed who chose not to provide their own opinions all indicated embarrassment as the core reason, and that they expected not to be relaxed if they were to have had a go. Furthermore, over half of those interviewed who did have a go experienced embarrassment and did not feel relaxed. As one participant noted: “I was definitely aware of other people watching, which made it kind of awkward.”

Harry Brignull & Yvonne Rogers. ENTICING PEOPLE TO INTERACT WITH LARGE PUBLIC DISPLAYS IN PUBLIC SPACES. 2002. Figure 7. Photograph of the set up of the Opinionizer project. From Enticing People To Interact With The designers suggest that greater anonymity for participants might Large Public Displays In Public Spaces. 2002 be one solution to this problem, but warn that this may cause other problems in facilitating interaction

However, while potentially reducing social awkwardness there is a downside- it removes the honey-pot effect. The opportunities of socializing are greatly reduced as people focus on their handsets. Moreover, they become inward-looking and in many ways enter an anti-social space, one which they are only party to.

Harry Brignull & Yvonne Rogers. ENTICING PEOPLE TO INTERACT WITH LARGE PUBLIC DISPLAYS IN PUBLIC SPACES. 2002.

Alex Pentland, head of the Perceptual Computing Group at MIT Media Lab is one of many researchers who believes that the traditional form of human-computer interaction is a constraint on the development of multi-user systems and who are developing new types of input devices.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

The problem, in my opinion, is that our current computers are both deaf and blind: they experience the world only by way of a keyboard and a mouse….. I believe computers must be able to see and hear what we do before they can prove truly helpful…. To that end, my group at the Media Laboratory at the Massachusetts Institute of Technology has recently developed a family of computer systems for recognizing faces, expressions and gestures. The technology has enabled us to build smart rooms… furnished with cameras and microphones that relay their recordings to a nearby network of computers. The computers assess what people in the smart rooms are saying and doing. Thanks to this connection, visitors can use their actions, voices and expressions – instead of keyboards, sensors or goggles – to control computer programs, browse multimedia information or venture into realms of virtual reality.

Alex Pentland. SCIENTIFIC AMERICAN pg 68 and 71. April 1996

Microcomputers embedded into the physical environment make it possible for the physical space to act as an input device. This improves the potential for integration of digital spaces into physical space and leads to more natural and spontaneous interactions [McCARTHY 2002]. Typically these responsive environments use participants movements and the flow of people around the space to direct the system’s responses. The methods used by these systems for detecting action include tracking participant’s movements as they walk across a pressure sensitive surface, [METASPACE 2002], tracking potential participants by wearable technology or handheld devices [GROUPCAST 2002][PIRATES! 2002], vision-based tracking Figure 8. Players participating in systems [COMPUTER VISION 2002] and biosensors [BRAINBALL 1999]. The Brainball game. From http://smart.tii.se/smart_old/smart_en use of natural behaviour and actions creates a closer relationship g/brainball_eng/new_brainball.htm between the participants and their physical environment, however the input systems described here remain inherently single-user and in the case of wearable technology, require the participants to be identified in advance so they can be fitted with the necessary equipment.

3.2.2 Spatial metaphors In their arrangement of elements on-screen, many virtual and mixed reality environments use spatial metaphors. This helps participants to understand the relationship between objects.

The spatial organisation of the world is the same for all of us. ``Down'' is towards the center of the earth, and ``up'' is towards the sky; we recognise ``front'' and ``back'', and understand what that implies for our field-of- view. Our common orientation to the physical world is an invaluable resource in presenting and interpreting activity and behaviour. Since we know that the world is physically structured for others in just the same way as it is for ourselves, we can use this understanding to orient our own behaviour for other people's use.’

Steve Harrison, Paul Dourish. RE-PLACE-ING SPACE: THE ROLES OF PLACE AND SPACE IN COLLABORATIVE SYSTEMS. 1996

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

Spatial metaphors make use of the ‘affordances’(5) of our physical environment to provide a common frame of reference for all (5) Affordances: defined by participants and so cultivate awareness and social interaction Don Norman as being ‘the perceived and actual properties of [SCHNADELBACK, PENN, BENFORD, KOLEVA.. 2003]. the thing, primarily those The affordances of our physical environment are used in a different fundamental properties that way by ‘Metropolis Challenge’, an attraction at Futuroscope, a determine just how the thing could French theme park. Instead of organising the information on the possibly be used.’ screen in a spatial manner, the participants common understanding Don Norman. THE PSYCHOLOGY of the physical environment is exploited to stimulate collaborative OF EVERYDAY THINGS. 1988 action. The audience sits in a cinema and is divided into two teams on the right and left of the space. The two teams use their common understanding of ‘up’, ‘down’, ‘red’ and ‘green’ to control the game elements on the screen. Each participant has a red and a green phosphorescent wand. Cameras mounted above the screen register the colours of the wands and calculates the greater colour of both teams. The game updates accordingly.

Figures 9 & 10. Photographs of players of Metropolis Challenge.

From www.futuroscope.com

3.2.3 Incidental interaction Other interactive systems which exploit natural actions are those which use incidental interactions. Incidental interaction is defined as being ‘when a user action intended for one purpose is interpreted to achieve some other goal.' Alan Dix. BEYOND INTENTION. PUSHING BOUNDARIES WITH INCIDENTAL INTERACTION. 2002 .

Although the projects described in the following paragraphs are not Figure 11. Diagram showing relationship between intended and specifically designed for multi-user interaction in public spaces, the incidental interactions. From From principles could be applied to this context. Beyond Intention. Pushing Boundaries With Incidental Interaction. 2002

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

At XEROX Parc the Pepys project used an ‘active badge’ system to track the current location of workers. At the end of each day the system used the logged information to create a record for each worker of their movements that day. The system was capable of making assumptions about the context of the actions, as Alan Dix describes:

Because Pepys knew about the layout of the offices, and who was where when, it was able to detect when two or more people were in the same location and create a diary entry "had meeting with Allan and Victoria".

Note again Allan and Victoria's purpose is to visit Paul's office and Figure 12. Photograph of Active incidentally a diary entry is created for each of them. Badge. From Beyond Intention. Pushing Boundaries With Incidental Interaction Alan Dix. BEYOND INTENTION. PUSHING BOUNDARIES WITH INCIDENTAL INTERACTION. 2002.

The concern with incidental interaction is that if the user is initially unaware that their actions are having a secondary effect then when they become aware of this they may feel deceived and suspect an invasion of their privacy. Similarly the information collected by the MediaCup [1999] may not always be information that participants would choose to share with their fellow workers. In this project the sensor detecting the action is attached to an object rather than an individual, but the use of the system to monitor the behaviour of individuals within it’s scope is identical to use of the ActiveBadge system.

The designers of the Opinionizer project, described in the previous section, concluded that

Participants need to be able to learn how to interact with the system vicariously, rather than be told or have to follow a set of instructions. They need to be able to simply walk up and use it, having watched others do the Figure 13. Media Cup tracking object. From same. http://mediacup.teco.edu/

Harry Brignull & Yvonne Rogers. ENTICING PEOPLE TO INTERACT WITH LARGE PUBLIC DISPLAYS IN PUBLIC SPACES 2002.

Incidental interactions allow exactly this type of learning by observation to take place.

Encouraging people to make the transition from being an observer to being an active participant is recognized as one of the most difficult elements to design for. [BRIGNULL, ROGERS 2002]. In their conclusions on the Opinionizer project Harry Brignull and Yvonne Rogers drew up the following list of facts that observers like to possess before engaging with an interactive system in a public space.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

These are:

- Beliefs as to whether the public display is interesting, enjoyable or worthy of attention. - Perception of what it is, how to use it, and how long it takes to use. - Understanding of its social standing and required etiquette (what other people think of it, the type of people who use it and how they behave towards it). -Knowledge of the social system of practices around it, for example, the nature of the queue or the socializing activities going on in the immediate locality.

Harry Brignull & Yvonne Rogers. ENTICING PEOPLE TO INTERACT WITH LARGE PUBLIC DISPLAYS IN PUBLIC SPACES 2002.

With incidental interactions, because the response is triggered unintentionally by the participant, the threshold from observer to participant is crossed almost by accident. Once the transition has been made, then a large number of the participant’s concerns about interacting with the system have already been answered.

3.3 A sense of place in digital environments The Netville study[2001] looked at a residential development in North America designed to provide inhabitants with the best available computer-mediated communications technologies. Netville residents enjoyed high-speed internet access, videophone, online health services, local discussion forums and various online educational and entertainment applications. The researchers were interested in observing the effect of these indirect means of communication on the residents’ relationships with the local community. The study concluded that:

In a situation where there was near ubiquitous access to CMC, Internet use encouraged visiting, surveillance, neighbor recognition, collective action and the maintenance of local social ties (Hampton 2001). There was no indication that Internet use inhibited or substituted for other forms of social contact. Contact lead to contact, CMC encouraged additional social contact through multiple means of communication: online, in-person and over the telephone.

Keith Hampton. PLACE-BASED AND IT MEDIATED COMMUNITY. 2002

The creation of community within the local environment was measured by looking at residents awareness of, and interaction with, their neighbours. Other studies of the effect of internet use on social interaction have concluded that home-based internet use reduces time spent interacting with people in the local community [HAMPTON 2002]. The difference between Netville and these other studies is that

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

all residents of Netville had equal access to the network technologies while the other studies looked at social interaction within across communities with unequal access to computer communications technologies.

The idea that equal access to information can result in greater awareness of co-participants is also present in the system of Mixed Reality Architecture developed by Schnadelback, Penn, Benford and Koleva.

Mixed Reality Architecture combines participants who share a physical space, and remotely located participants, in a digital environment. The participants are shown in the space by video (if they are present in the physical location) or as an computer-generated avatar (if they are a remote participant) and the system creates the illusion that all participants are co-located in a shared digital space.

Figure 14. Series of images showing the relationship between remote and local participants in Mixed Reality Architecture. From Mixed Reality Architecture. Concept, Construction, Use. 2003.

The system explores how spatial metaphors applied to digital environments might use the affordances of the physical environment to encourage awareness of co-participants. The designers of Mixed Reality Architecture suggest that using a spatial metaphor for the organisation of the digital environment provides opportunities for social interaction and awareness between participants that non- spatial communications technologies lack. However other research asserts that digital environments which do not use spatial metaphors are also capable of supporting a sense of place. One example of communities developing a sense of ‘place’ without having a ‘space’ are internet newsgroups and mailing lists.

The technology of each USENET group is exactly the same, and yet the resultant groups exhibit very different notions of place. It's not simply that they separate discussion into topics, making certain postings appropriate to one group or another; but that they also make distinctions between styles of posting. Neophyte queries may be more or less appropriate, depending on the culture of the group; so are flames. These styles are relatively independent of topic. Complaints about spelling or grammatical errors are acceptable (or

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

even encouraged) in alt.peeves, but they would be inappropriate in comp.protocols.tcp-ip. The different groups serve different purposes to overlapping constituencies and communities; and they exhibit different social norms. They're different places.

Steve Harrison and Paul Dourish. RE-PLACE-ING SPACE: THE ROLES OF PLACE AND SPACE IN COLLABORATIVE SYSTEMS. 1996

Unfortunately for designers, the transformation of a space into a place can only be initiated by the occupants of that space. In part, the ability for participants to set the rules defining appropriate behaviour for their own environment is a reflection of the extent to which the designer allows the environment to depart from the initial design [CRAWFORD. 1982]. This principle of adaptation, appropriation and unpredictability is present in many types of digital environments designed to encourage awareness and social interaction, and is recognised as a key ingredient in engaging participants.

One critical element in the emergence of a sense of place and appropriate behaviour is support for adaptation and appropriation of the technology by user communities. This applies to physical places as well as technological ones. We make a house into a home by arranging it to suit our lives, and putting things there which reflect ourselves. People make places in media spaces with just the same ideas of adaptation and appropriation. Like tacking pictures to the walls, rearranging the furniture or placing personal artifacts around a room, these are the ways that people can turn a space into a place.

Steve Harrison and Paul Dourish. RE-PLACE-ING SPACE: THE ROLES OF PLACE AND SPACE IN COLLABORATIVE SYSTEMS 1996

3.4 Play and games In his exploration of the field of computer games written in 1982, Chris Crawford defines a game as being ‘a closed formal system that subjectively represents a subset of reality… By formal I mean only that the game has explicit rules.’ Chris Crawford. THE ART OF COMPUTER GAMES. 1982.

Researchers at the Play studio in Sweden recognise a close relationship between game rules, or ‘game mechanics’, and a games’ potential interactions.

A game mechanic is simply any part of the rule system of a game that covers one, and only one, possible kind of interaction that takes place during the game, be it general or specific. A game may consist of several mechanics, and a mechanic may be a part of many games.

Sus Lundgren & Staffan Björk. GAME MECHANICS: DESCRIBING COMPUTER-AUGMENTED GAMES IN TERMS OF INTERACTION. 2003

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

PLAY studio developed the game Pirates! which uses ubiquitous computing technology and game mechanics to create a game which takes place in both the physical and digital environments. In Pirates! each participant is the captain of a ship who completes trading missions by sailing between islands, avoiding cannibals and battles with other pirates.

The game is a multi-player game, implemented on handheld computers Figure 15. Screenshots of connected in a wireless local area network (WLAN), allowing the players to Pirates! game. From Using The Physical World As A roam a physical environment, the game arena. An important reason to make Game Board Pirates! a mobile game, is to make real world properties, such as locations, objects, and states of co-location between multiple players, intrinsic elements of the game. To determine the physical locations of the players, we connected proximity sensors to the handheld computers, as well as placed similar sensors at different locations in the arena. The players’ movement between these locations triggers different game events.

Staffan Björk, Jennica Falk, Rebecca Hansson, Peter Ljungstrand. USING THE PHYSICAL WORLD AS A GAME BOARD. 2001

The researchers identify two forms of social interaction supported by games – spontaneous social interaction where the game simply provides the opportunity for interaction, and stimulated social interaction where the game mechanics necessitate interaction with other participants [BJÖRK, FALK, HANSSON, LJUNGSTRAND]. Pirates! supports the first type of interaction as participants are not required to interact with each other to complete missions, although interaction with computer-controlled characters is essential. In tests, the researchers found that the lack of opportunities for collaboration or non-combative interaction with other participants was commented on by some players.

Another shortcoming in the current implementation is that there are not many possibilities for player-to-player interaction besides the combat functionality. A few users expressed that they would have wanted to interact more with others, e.g. by swapping goods, or team up against dangerous monsters, etc.

Staffan Björk, Jennica Falk, Rebecca Hansson, Peter Ljungstrand. USING THE PHYSICAL WORLD AS A GAME BOARD. 2001

Although conflict is an vital part of a game [CRAWFORD 1982], in Pirates! combat between participants was not a successful technique for encouraging interaction because the penalty for losing was too high for most players to be willing to take the risk.

In the current implementation of Pirates!, only one captain can survive a battle while it is Game Over for the defeated captain. This was described as discouraging player-to-player combat because players generally did not want to risk Game Over.

Staffan Björk, Jennica Falk, Rebecca Hansson, Peter Ljungstrand. USING THE PHYSICAL WORLD AS A GAME BOARD. 2001

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

Generally the penalty for losing computer games is loss of dignity and social standing. To some extent the motivation to prove skill and prowess is present in all games, single or multi-player [CRAWFORD 1982]. Like many computer games, Pirates! uses a hi- score chart to celebrate the abilities of the winners. An interesting development in ubiquitous and tangible gaming has been the development of games where not only does the gameplay use natural movement but the penalty for losing is also represented physically. PainStation [2001] inflicts lashes, heat or electric shocks on the unfortunate loser of a game of Pong. Figure 16. Screenshot of Pirates! hi- score board. From Using The Physical Can You See Me Now? [Blast Theory 2003] is a mixed reality game World As A Game Board played by ‘runners’ who move around a physical location, and online players who interact with the game over a computer network. The runners chase the remote players, if they get within five metres of the their location then the player is caught. The runners use handheld devices to orientate themselves in the game environment and to transmit their location to the online players. The use of a physical environment as a game area adds to the tension of the game. The runners feel the physical sensation of tiredness and even remote participants can be affected by the game environment.

A player from Seattle wrote: "I had a definite heart stopping moment when my concerns suddenly switched from desperately trying to escape, to Figure 17. A losing player suffers desperately hoping that the runner chasing me had not been run over by a in a PainStation game. From reversing truck (that's what it sounded like had happened).” www.painstation.de http://www.canyouseemenow.v2.nl

Essentially Can You See Me Now? offered two separate gaming experiences, that of the runner and that of the online player.

Figures 18 & 19. Photograph and screenshot showing the runner and online game environment of Can You See Me Now? From www.canyouseemenow.co.uk

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

3.5 Criteria for an interactive system for public space Following the investigation of interactive systems described in the previous section, I have outlined the criteria I believe are necessary for an interactive system in a public space to successfully increase participants awareness of the physical environment.

- The system should be accessible to many people simultaneously. This applies equally to the input device(s) and display system(s). - Participants should be able to learn by observation alone, what their role in the system is. - The system should make use of a common frame of reference for all participants. - Participants should be able to appropriate and adapt the system. - The system should require little or no alteration of common behaviour to trigger the interaction.

These points are explained in more detail below:

Accessibility It is important to avoid the performer / audience relationship between the participant and the observers. This can be done by using input device(s) which allows for several people to participating with the system at the same time. Similarly, all observers and participants should have equal access to the information on the visual display.

Observation and Comprehension To encourage people to interact with the system it is essential that certain questions are answered concerning their commitment and reward for participation before they commit themselves to interacting with the system. Unless the system has a ‘monitor’ introducing the system to each participant, these questions must be answered by the system itself.

Common frame of reference This puts all participants on an equal footing. The common frame of reference can be exploited to encourage interaction between participants.

Appropriation and adaptation To feel ‘ownership’ of a system, participants need to be able to personalise it by adapting the system to their own situation and personalities. Within a public space this ‘ownership’ will tend to be collective and would encourage awareness to increase between participants who are located in the same physical environment. The capability of the system to support appropriation and adaptation also implies that the system can act in unpredictable ways, which supports the long-term use of the system.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

‘Connectedness’ and natural action What constitutes natural behaviour varies enormously between different places. The important thing is that the system integrates as seamlessly as possible with the identity and function of the physical environment. This gives the system the greatest chance of being accepted by the participants and encourages the audience to participate.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

4. SuperFight I Project

This section describes the system I designed and built to investigate the principles outlined in the previous section. The objective of this system was to explore the idea that occupants of a physical space might develop greater awareness of that location by the integration of a digital environment into the physical space. The digital environment would make a connection between two physical environments, and the identity of both spaces would be represented in the digital environment. By being able to compare and contrast the nature of their own space to another remote space inhabitants might gain a greater understanding of the identity of their physical location. The character of the physical environment would be expressed by a representation of the actions and interactions that took place in the physical environment. To test this hypotheses I designed an experiment which aimed to answer two questions. Firstly, would the inhabitants understand that the display of the digital environment was related to their action, and secondly, would the inhabitants begin to alter their behaviour in the physical space in an attempt to influence the digital environment.

4.1 Design considerations My primary concerns when designing this system were that the relationship between the participants’ actions and the projected visual display should be explicit and that the response of the system to the participants’ actions should happen with as little delay as possible. I was also concerned that the projected visuals had what is described by Steve Harrison and Paul Dourish as ‘connectedness’ with the physical space.

Connectedness is the degree to which a place fits with its surroundings, maintaining a pattern in the surrounding environment (such as color, material or form)--or responding to those patterns, even if it does not maintain the patterns explicitly. It is when these relationships are broken down that we say that something is ``out of place''.

Steve Harrison and Paul Dourish. RE-PLACE-ING SPACE: THE ROLES OF PLACE AND SPACE IN COLLABORATIVE SYSTEMS. 1996

My final concern was the use of the system over time. While I might be able to rely on novelty to attract participants in the first instance, they would soon be put off if the interaction was too simple or too complicated [BRIGNULL, ROGERS 2002].

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

4.1.1 Action as information For SuperFightI I chose to use a large-screen projection to display the digital environment because this made the game visible to many people simultaneously. The visibility of the projected display extended beyond active participants to include other occupants of the space. This meant people could keep an eye on the progress of the game even when they were involved in other activities.

While many interactive, large-screen display systems are concerned with representing the actions of individuals [OPINIONIZER 2002][ GREENBERG’S NOTIFICATION COLLAGE 2001][GROUPCAST 2002], I wanted the display to represent the actions happening in the physical environment, without disclosing whether these actions had been carried out by an individual or a group. Partly I hoped this would reduce the potential for participants to feel social embarrassment, as this can inhibit the transformation of observers into participants [BRIGNULL, ROGERS 2002]. More importantly, I hoped to use the digital environment to increase the level of engagement between the participants and their physical environment and I aimed to do this by emphasizing the identity of the physical space, and the collective role played by all of the inhabitants in defining that identity. While knowing who is in a space will tell you a certain amount about a place, as much or more, can be learnt by knowing what those people are doing there.

Our level of engagement with a particular physical space is created as much by the actions and behaviour we associate with that space as by the 3-dimensional material structure [HARRISON, DOURISH]. This makes it possible for a particular physical environment to be used for a wide (6) Events at Aldbrough St John variety of functions without being physically re-configured.(6) As the nature of the action in the space changes, so we update our behaviour Village Hall, Northumbria. accordingly. 2003

A community hall might, on different evenings in a week, be used as a rock Mother and toddler group venue, a sports arena, and a place of worship. On these different occasions, Art class it's not the structure of the space which frames people's behaviour, but the Aerobics place where they find themselves. May Ball Christmas Lunch Cookery demonstration Steve Harrison and Paul Dourish. RE-PLACE-ING SPACE: THE ROLES OF PLACE AND SPACE IN COLLABORATIVE SYSTEMS. 1996 Evening classes Quiz night Hair-dressing demonstration If the function of a space at a particular time affects the behaviour of its occupants then, looked at from the opposite point of view, this suggests that the identity of a space might be understood by an analysis of the actions and behaviour which occur in that place.

Using sensors to detect behaviour within the physical environments From allowed me to gather information about the physical spaces without http://www.communigate.co.uk/ne/asj identifying individuals. Movement allows for a more intuitive vhf/index.phtml

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

interaction as it makes use of our innate understanding of the relationship between ourselves, the environment around us and objects placed in that environment. The use of natural movements to interact with digital environments alters our relationship with the system because it reduces the amount of conscious thought needed to perform the interaction.

We have evolved as creatures to cope with physical things and other creatures, not technological devices. Although we have higher level reasoning that enables us to cope – the same reasoning that enables us to create technology, this is only significant when we 'think about' things, our more innate cognitive abilities are shaped by the natural. Computer systems (and other complex technology such as electrical and pneumatic) break these intrinsics of physicality. Computation creates complexity of effect, networks introduce non-locality in space, memory non-locality in time and a computer has a vast number of invisible variables in its hidden internal state. We cope (just) with this because either we rationalise and use higher-level thinking to make sense and to make models of these complex non-physical interactions, or we treat the computer as animate. In addition, one of the reasons for the development of the GUI interface style is that it makes the electronic world more like real (inanimate) things.

Alan Dix. BEYOND INTENTION. PUSHING BOUNDARIES WITH INCIDENTAL INTERACTION. 2002

The choice of the sensors was important because the information gathered had to be capable of communicating something about the identity of the space to a remote location. I chose not to use video to detect the actions because I wanted the emphasis to be on the actions and interactions happening within the space and not on the appearance of the space and its inhabitants. Using purely graphic visuals allowed me to abstract the information collected by the sensors to a greater extent than I could have done with video information. This gave the system an identity of it’s own, beyond the identity of the physical spaces involved. I chose an infra-red break- beam sensor which would detect movement past a certain point in the space, and a microphone to detect the noise level. Using more than one sensor created a fuller picture of the current atmosphere of the space than would have been possible with just one sensor. This was because the information received from the sensors can be checked for redundant or complementary actions. [SPARACINO 2002] For example, if the break-beam sensor has triggered only twice, but the sound levels are high, then we could speculate that the few people in the space are having a lively conversation (either with each other or on the phone) , or are listening to music.

The sensors were selected after a brief observational study of the Virtual Environments studio (SEE APPENDIX 3). This study gave me an understanding of the day-to-day use of the space meaning I accurately predict which actions would happen on a regular basis.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

This was important because the occupants could only learn about the system by observing how it responded to their actions. If the actions needed to trigger the sensors were not natural to the function of the space, then the occupants would not have the opportunity to observe the interaction. Following the observational study I drew up a list of potential sensors, based on the possible actions that might occur in one or both spaces (SEE APPENDIX II). Other sensors on the list were rejected for various reasons including cost, technical complexity, and appropriateness for the purpose.

4.1.2 Encouraging participation For people to participate with an interactive system it is essential that the relationship between action and response is instantly obvious. As Yvonne Rogers writes

The timing between a person doing or acting and the responses from the designed space is also critical. A lag in feedback between the two can break the suspension of belief and effectively ‘kill’ the moment of experience – arguably more so than with more traditional interfaces (e.g. desktop).

Yvonne Rogers. UNDERSTANDING INTERACTION: GOING BEYOND THE SEAMLESS INTEGRATION OF THE DIGITAL AND THE PHYSICAL. 2002.

This means that the actions and response have to be (at least on one level) uncomplicated and simple, with no room for ambiguity or visual embellishes. In SuperFight I each sensor maps to a single response on the screen, repetition of the action will result in repetition of the response (though the elements involved in the response might change.)

Basic action / response pairings.

Sensor Response Transition Infra-red break beam Object added to None. Binary action (movement past a elements on ‘your’ point) half of the screen. Object removed from the ‘other’ half of the screen Microphone (sound Objects on screen Louder volume = level) shake and wobble. greater movement. Sound level in ‘your’ space affects the objects in the ‘other’ space on the screen

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

It was important that each action mapped to a logical response in relation to the participants’ perceived affordances of the action. For example, the action of passing a certain point in space is unequivocal – you have either passed the point or you haven’t. The response on the screen reflects the nature of this action. On the other hand, sound levels in a space are constantly changing and groups and individuals are able to control these levels. In SuperFight I the visual response to the sound level also happens constantly and can be affected directly by the participants. Sound in the physical spaces causes the bricks of the tower to wobble. The choice of movement to represent sound reflects the ability of sound waves to cause vibrations in objects.

Figure 20. Screenshot of partially built Figure 21. Screenshot of MArch tower tower, created by movement in physical wobbling due to noise levels in the space. Virtual Environments studio.

I hoped that the logical mapping of the action to the response would make it easier for observers to understand immediately how their actions affect the visual display. But it is also recognised that if an interaction always produces the same response then participants can become bored with the system. [SPARACINO 2002]

4.1.3 Games, play and competition To try to fight familiarity I borrowed a technique from computer (7) Example of a special move games designers. Generally, computer games are accessible to from the PlayStation2 game, everyone at a certain level of play because of the simple mapping of Metal Gear Solid 2 actions to the controller. Usually, there is also a more advanced level of control learnt by experience or observation(7). That is to say, Codec Craziness (ps2): your level of skill develops in relation to the amount of effort and While watching any Codec attention you give to the game. This keeps players interested for conversation, move the left & longer as the game provides a challenge for both novice and expert right analog sticks and the games players. character's faces will move. Press on one of the sticks, and the faces Games without smooth learning curves frustrate players by failing to will zoom in close. Press the R1 provide them with reasonable opportunities for bettering their scores. or R2 button while Snake or Players feel that the game is either too hard, too easy, or simply arbitrary. Raiden is listening to someone, Games with smooth learning curves challenge their players at all levels and and you can hear their thoughts. encourage continued play by offering the prospect of new discoveries. A smooth learning curve is worked into a game by providing a smooth progression from the beginner’s level to an expert level. This requires that the game designer create not one game but a series of related games. Each

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

game must be intrinsically interesting and challenging to the level of player for which it is targeted. Ideally, the progression is automatic; the player starts at the beginner’s level and the advanced features are brought in as the computer recognizes proficient play.

Chris Crawford. THE ART OF COMPUTER GAME DESIGN. 1982

So in SuperFight I, alongside the basic action / response mappings described earlier, I added a ‘special move’ which should not be immediately apparent to participants.

Special move action / response pairing:

Sensor Trigger Response Microphone (sound Sound level passes The tower built by level) a certain volume actions in the ‘other’ physical space falls down and remains still for 5 seconds

Figure 22. Two screenshots showing the result of the ‘special move’.

The technique of abstracting reality and presenting a selective representation of the actual situation is commonly used in game design. It is partly what sets games apart from simulations.

A game is not merely a small simulation lacking the degree of detail that a simulation possesses; a game deliberately suppresses detail to accentuate the broader message that the designer wishes to present. Where a simulation is detailed a game is stylized.

Chris Crawford. THE ART OF COMPUTER GAME DESIGN. 1982

Game designers use abstraction and limitations on players’ actions as a way of focusing attention on the central characteristics of a game. For example, in a racing game like Gran Turismo players cannot get

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

out of the car. Although constraining players’ actions in this way reduces the realism of the game, it gives a structure and direction to participants’ interaction that can be lacking in a digital environment where the goal is simply to explore the space. With SuperFight I, I wanted to structure the players interaction with the game in such a way that their awareness of the physical environment was increased through interacting with a physical environment.

4.1.4 Design of the visual content Deciding how the information collected by the sensors should be interpreted and displayed took careful consideration. The type of display, the form of the display and the style of the content would all affect the participants perception of the system[HARRISON, DOURISH 1996] which in turn would affect their willingness to interact.

During the construction of the project I developed various graphic forms for representing the information collected by the sensors. While these graphics were often visually appealing, I felt they were better suited to a reactive digital space than an interactive environment. With the abstract forms it was difficult to find ways for the forms to have an effect on each other. Using abstraction made it difficult to communicate to participants how long the game might last and what their reward for participation might be.

Figure 23. Series of examples of experimental graphics for SuperFight I.

In the end I gave the visuals the look and feel of an early computer game. This aesthetic is instantly recognizable to the majority of students. The benefit of appropriating an existing aesthetic is that people have already learnt to associate it with certain characteristics.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

Using this aesthetic, I hoped people would understand that the only reward for their participation was fun and ‘oneupmanship’ [CHRIS CRAWFORD 1982] and that a second player is able to see and respond to their actions and. The aim of the game is to complete ‘your’ tower. The participants in each physical space identified ‘their’ tower by its colour which corresponded to the poster put up in their studio a couple of weeks earlier to advertise the project, and by the name of each space written at the top of the screen.

When the break-beam sensor is triggered a block was added to ‘your’ tower. Simultaneously a block is removed from the other tower. It takes 22 blocks to complete a tower and there are always 22 blocks on the screen. At the start of each game these 22 blocks are divided equally between both towers. I chose to use a tower because this was Figure 24. Screen shot of the end of a SuperFight game. a recognisable structure. By looking at the graphics it is easy to tell which tower is nearer completion, and how many more blocks are needed to complete the game. The combination of the gameplay and graphics is the means by which SuperFight I answers Brignull and Rogers’ points about participants’ concerns relating to the goal, duration and intention of an interactive system. To further emphasise how close the game was to completion the current scores, showing how many blocks each tower currently had, were written at the top of the screen.

A tally was kept of how many games each space had won, and the totals were visible at the bottom of the screen. At the end of each Figure 25. SuperFight I hi-score screen game SuperFight I displayed a hi-score screen showing which space had won each of the past eight games. I hoped that this reminder of past successes or failures might motivate participants to play with the system for a longer period of time.

The projected display was identical in both spaces. This was important as it set the relationship between the players in both spaces on an equal footing where neither space had more or less information than the other. This visual equality reflected the common frame of reference furnished by the players’ spatial understanding of the physical environment.

4.1.5 Arrangement in physical space The overall aim for the installation of this system was to integrate the digital display in as natural a way as possible with the physical space. My intention was to exploit the natural actions of people in the physical spaces so that the integration of the digital space would not require any change in behaviour in order to learn how to play with the system.

The equipment to be installed in each physical space was a break beam sensor, microphone, microcontroller, computer and projector. The primary goal when choosing how to arrange these elements within the space was to bring the largest number of possible people into contact with the system. The range of the sensors was relatively

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

small – about 1.5m for both the break beam sensor and the microphone, so it was important to get the maximum flow of people through the sensing area. For this reason I chose to set up the system near the door of each space as this would provide the most reliable flow of potential participants. Both studios are fairly large, open plan spaces and it is difficult to predict where a flow of people will occur except for around the doorways.

In both spaces the display was projected onto a wall rather than a fabric or paper screen. This improved the brightness of the display, increasing it’s visibility within the space. The projected display was approximately the size of a large TV. The brightness of the projections meant there was no need to darken the physical spaces at all, so causing the minimum amount of disruption and alteration to the physical spaces.

Figure 26. Plan showing how SuperFight I system

was arranged in Virtual Environments studio.

Figure 27. Plan showing how SuperFight I system was set up in MArch studio.

The position of the display in relation to the position of the participant at the moment they trigger the break beam sensor was also planned. To give maximum emphasis to the connection between the action and the visual response it was important the participant should be able to see the display at the same time as they trigger the sensor. In the Virtual Environments studio the display faced the participant while in the MArch studio the display was at right angles to the participant. In both cases the top of the display is positioned at around 1.5 – 2m from the ground because usually the sensors would be triggered by people as they walk and this height corresponds to the natural eyeline. Raising the display slightly also makes it visible from further away, allowing people to observe the progress of the game even when they are not actively participating.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

Figure 29. Photograph of SuperFight I projection in Virtual Environments studio. Figure 28. Photograph of SuperFight I projection in MArch studio.

When the SuperFight I visuals were first tested as a projection I found that the visual response to the break-beam sensor was not eye- catching enough. I felt I could not be sure that people’s gaze would be drawn to it and the connection between action and response might not be established. The addition of a block to the tower which seemed obvious on a computer monitor, was too subtle to guarantee the game would win people’s attention for the moment when they triggered the sensor. To strengthen the visual response to the action, a flash of colour was added which would fill the screen for a split second when the break-beam was triggered. The sudden and dramatic change of the visual display was now enough to ensure that the game caught people’s attention when a block was added. Not only did this guarantee people entering the studios were aware of the connection between their action and the visual response, but that people already in the room were aware of the progress of the game. The colour of the flash corresponded to the colour of the space where the sensor had just been triggered and so this worked as a peripheral update for the game scores.

4.2 Construction / Build Two separate physical spaces are connected by a two-computer network. Sensors detect actions and interactions occurring in both spaces. In each space, the sensor information is received by a microcontroller and processed by a perl socket server running on a local computer. Each computer also runs a Macromedia Flash MX file

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

which is connected to the perl socket via Flash’s XMLSocket. The Flash file receives the sensor data and adds a local microphone volume reading to the sensor information. The string containing all these readings is then sent to the perl socket server running on the computer in the remote space. When the socket servers have two Flash clients each (one in their local space and one remote) the combined sensor readings are sent to the local Flash client. The Flash file interprets the readings into visuals and the output is projected into the physical space.

Figure 30. Diagram showing technical setup of SuperFight I system.

The display presents the information in the form of a battle between the two spaces. The display is identical in both spaces. Participants actions within each physical space will determine which side appears to be winning in the digital space. Initially the projected visuals show two half-built towers, one red, the other blue. Numbers at the top of the screen reveal that both spaces have a score of 11/22. The total number of games won by each space is shown at the bottom of the display.

Figure 31. Screenshot of opening game position

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

4.3 Description of experiment The experiment took place in an uncontrolled environment in the sense that the subjects of the experiment had no knowledge of, and no vested interest in, the results of the experiment. They did not know what they were expected to do, or what the desired outcome might be. Hopefully this encouraged them to behave naturally in their interactions with the system, meaning that the information received is a genuine indication of their engagement with the system. However it also greatly increased the chance that the system might fail to return any meaningful information. The design of the system and the pre-installation communication with the occupants was intended to minimize the chance of this.

4.3.1 Pre-installation communication The public spaces where SuperFight I was installed were two studios in the Bartlett School of Architecture, UCL. The MArch and Virtual Environments students are the only students regularly in college during the summer holidays yet traditionally there is little contact between students of the two courses.

The first contact with the participants was via an email I sent to every occupant of both spaces. The email explained that I had been given permission to use these spaces for my final project, that the project would not interfere with the normal work of the space and that interaction with the system was not compulsory. (SEE APPENDIX 1) I didn’t explain the project in any detail as part of the aim of the pre- installation communication was to build an air of anticipation and expectation about the project and to try to give it an identity of fun, playfulness, competition and mystery.

I also wanted to give the occupants plenty of time to raise any concerns they had about the use of their space for the project. The success of the project depended on the acceptance of the system by the occupants of the space and it seemed important that they considered themselves to have been included in the planning stages of the project, even if, in this case, they weren’t involved in the design stage.

The second form of communication was a poster I designed to go into each space to advertise the project.

The major goals of the poster were to build an identity for the project, to increase anticipation and to provide a low level of information concerning when and where the project would take place. The two posters were almost identical but used different colours and named different locations for where the project would take place. The use of colour was intended to start an association between that colour and the physical environments. The posters were pinned up in each space a couple of weeks before the installation was due to begin.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

Figure 32. The posters that were put into the studios prior

to the installation of SuperFight I.

The process was intended to start with an assessment of people's attitudes and relationships towards their physical space, co- inhabitants and technology. This would be done using 'cultural probes' [DUNNE, GAVER. 1999] which were initially developed to provide inspiration for user-centred design, but have since proved useful in gathering information about participants [HEMMINGS, CRABTREE, RODDEN, CLARKE, ROUNCEFIELD. 2002]. The cultural probes consisted of a selection of photographs and some A4 pieces of paper. Each photo had a question printed on the back with space for an answer to be written below, instructions on the pieces of A4 paper ask inhabitants to draw maps of various routes. They were to be left in each space allowing people to complete them whenever they chose. The completed probes would be put into a collection box in each space. Having constructed two of the cultural probe objects I placed one of them in the Virtual Environments studio. This was put directly below the poster advertising the project as I wanted to make the connection between the objects and poster as clear as possible. The objects were left in place for a week, but during this time only one map was completed and posted in the collection box. I decided not to install the objects in the MArch studio.

There was also a short observational study of the use of one of the spaces to give an idea of how inhabitant’s behaviour changed when Figure 33. Photograph of objects the project was installed. This gave useful information about actions intended to be used as ‘cultural which happen regularly in a college studio. (SEE APPENDIX 3). probe’.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

4.3.2 System evaluation The sensors used to detect the participants actions are also the evaluation method used to observe how the system is being used. The perl socket server generates a text file recording which sensors have been triggered. This built up into a picture of the actions occurring in the space. Within this data I looked for patterns of behaviour and how these changed over the period of time the system is installed in the space. The inhabitants of each space were also given questionnaires to answer to assess if their perception of the effect of the digital environment is similar to the actual effect (as revealed by the computer log).

4.4 Research outcome and application The possible outcomes of this project are: o People don't understand how the digital environment relates to physical reality o People don't like the integration of digital space into physical space o People understand the relationship between digital and physical space but don't co-opt digital environment o People understand relationship between digital and physical space and co-opt, or play with, the digital environment o People start to 'read' the ambient information to extend their knowledge about the physical space outside of their immediate environment. For example, is anyone else in the building? How long have they been there? What is the noise level of the space? o People's behaviour is unconsciously affected by their reading of this information. For example they start leaving if there is no-one else in the physical OR digital spaces, they make more noise if the other space has a high volume etc... o People use the digital space to communicate and play with the people in the other physical space

All of these outcomes would contribute to research into the comprehension and co-option of action as a means of understanding space as well as establishing a range of questions around the effect of communications technologies on our relationship to physical space.

This research might be applied all forms of public space where people’s primary goal in visiting the space is not the creation of community but where the creation of community is helpful in achieving the primary goal. Examples of this type of environment would be shops, schools and community centres.

4.5 Analysis and interpretation of data SuperFight I was a system designed to investigate how actions might be used to carry the identity of a physical environment across an indirect communication network, and how this might affect the behaviour of occupants of the local and remote spaces. SuperFight I

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

was installed in the Virtual Environments studio and MArch studio at the Bartlett School of Architecture, UCL for three days to assess how it might used in a real-life setting. The results of this experiment suggested that SuperFight I had succeeded in it’s aims. Participants understood how their actions related to the visual display and then chose to interact with the system. The system was also proved to be capable of supporting associations between physical and digital space, and of communicating the character of physical space across a computer network.

4.5.1 Evaluation of pre-installation communication The posters were very helpful in initiating communication about the project. Putting the posters up meant I had to visit each studio before the installation date, so I had the opportunity to speak to occupants and explain what I was doing. I returned to check on the posters a couple of days later and a number of people in both spaces asked questions about the project. These visits were also useful as I gained information about the use of the spaces and the actions of the occupants. For example I learnt that the MArch students had their final assessment very soon, and two weeks later they had to hand in their portfolios. This helped in the planning of the project as I was able to estimate when the space might be most used. The posters also provided a clue as to the different nature and identity present in both spaces. While the poster in the VE studio was left untouched, the MArch students added comments and stickers to the poster in their studio. Generally, the posters were a very successful way to Figure 34. Photograph of the introduce the project to the participants. alterations to the poster in the MArch studio.

The cultural probes were not successful. Partly, I think, this was due to the presentation and nature of the objects. I underestimated how critical the design of the cultural probes objects was in communicating their purpose and what people were supposed to do with the objects. I still believe cultural probes are a very powerful means of learning about the use of a particular space but the design of the cultural probe objects needs to explain their function very clearly.

4.6.2 Results of SuperFight I experiment The conclusions and analysis of the SuperFight I experiment are drawn from the information logged by the Perl script and a series of questions which were emailed to the occupants of both spaces. Being able to make comparisons between these two forms of data was invaluable as they provided very different types of information. While the log files gave an analytical and objective view of the actions in the spaces, the participants responses to the questions offered subjective opinions of the installation.

I expected that the novelty of the SuperFight I system would generate interest among the occupants of the studios when it was first installed. This was proved to be the case and the use of SuperFight I

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

by participants was the greatest on the first day of the installation. (SEE APPENDIX IV). But as the installation was set up for three days I had a chance to observe the use of the system over time, once the first novelty had worn off.

Earlier I listed all of the outcomes I imagined possible from the installation of this system. I will look at each of these in turn:

People don't understand how the digital environment relates to physical reality None of the evidence suggests that participants did not understand the relationship between their actions and the response on the projected display. There are examples of intentional interaction with the system (SEE APPENDIX IV) and the participants responses show a full understanding of the relationship between their actions and the game display.

Question to participants: - How do you play the game? MARCH: just walked infront of the sensor VE: by passing through the infrared sensors and by making sound (blowing)at the sound detector device. It builds rectangles in "our" tower. VE: By activating the sensors in the entrance and blowing through the microphone. VE: By entering the VE studio, intentionally crossing the infra-red beam and whistelisng at the microphone.

Indeed, one of the limitations of SuperFightI was that the participants learnt how to interact with the game, including how to control the ‘special move’, far quicker than I had anticipated. I think that if SuperFight I contained more levels of interaction then the system would keep participant’s interest for longer.

“Is that it?” MArch student when we were discussing the various types of interaction.

Despite this, I don’t believe the base level of interactivity was too low. SuperFight I achieved it’s aim of enabling participants to learn how to use the system by observation alone, and the importance of this should not be under-estimated. The addition of more layers of interaction could build on this achievement to prolong the lifespan of the game but the raising the lowest level of accessibility might reduce the intuitive understanding of the action / response relationship.

The use of incidental interactions worked well. SuperFight I did not disturb the normal activities of the space’s occupants yet the rules for interaction were clearly communicated.

Question to participants: - Did you play the game? VE: Probably.. I was there several days

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

The break-beam sensor achieved it’s goal of detecting the presence and movement of people in the space. The positioning of this sensor resulted in it being triggered on a regular basis by natural movement around the space. As a result the link between action and visual response was continually reinforced.

The type of microphone used meant that the normal sound levels of the spaces were difficult to detect. Although the microphone was omni-directional it was not designed to pick up ambient sound so the microphone only really registered noises when they happened very close to it. This meant that the variety in the sound levels of the spaces was not reflected accurately on the projected display.

People understand the relationship between digital and physical space and co-opt, or play with, the digital environment

Question to participants: Did you play the game? VE: I wouldn't say that I identify myself as a 'player' of the game but rather as a participant. I would enjoy the impact of my actions when entering/leaving the room but I wasn't motivated to go and activate the sensors for the sake of it. I thought it would be cheating!.. I 've seen xxxx repeatedly triggering the sensors to increase our score... At the beginning I thought that he was misusing the game, but probably I was missing the point of the game. It is probably because I 've never noticed or never happened while I was in the studio a similar behaviour from the other side, that would make me think that hey, somebody is there and is challenging me to respond!

Whenever the break-beam sensor was triggered then the Perl socket server would write the data to a text file. The information recorded was: the state of the break-beam sensor in the local space, the state of the break-beam sensor in the remote space, the microphone reading from the remote space and the time that this data was recorded.

Thu Aug 28 11:20:05 2003 : 7.275390625 7.275390625 1.3232421875 14 Thu Aug 28 11:20:15 2003 : 7.2900390625 7.2900390625 1.3232421875 16 Thu Aug 28 11:20:16 2003 : 7.275390625 7.275390625 1.3232421875 13 Thu Aug 28 11:22:04 2003 : 7.294921875 7.294921875 1.3232421875 17

Figure 35. Sample readings from log file. The readings show that the local sensor had been triggered but the sensor in the remote space had not. The sound level was average at 13 – 17.

As the perl script reads each sensor numerous times every second the log files are far too long to be included in this dissertation. However

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

the information recorded in these files over the three days of the installation can be examined for patterns of action and interaction.

The graph below shows the number of times the break-beam sensor was triggered in the first five minutes of every hour on the first day of the installation. It clearly shows that the greatest amount of activity happened in the first two hours which supports the idea that the novelty of the installation would initially encourage participation.

Figure 36. Graph showing activation of break-beam sensor on first day of installation

When this information is compared to similar data for the other two days of the installation (SEE APPENDIX IV) then we can begin to see patterns of interaction occurring in how the system was used. The information shown in the graphs reveals that the occupants of each space intentionally interacted with the system. This can be seen in the extraordinary peaks of activity that occur at certain times.

We can look for evidence of synchronous and asynchronous interaction between participants in both studios by searching the log files for times when the break-beam sensors in both spaces were triggered, and the microphone in either space registered 100 (the maximum reading, and trigger for the ‘special move’).

Figure 36. Graph showing activation of break- Figure 38. Sample visualisation of beam sensor on first day of installation log file information showing patterns of interaction

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

The probability that these readings indicate intentional interaction is high as we can see from the visualizations of this information (SEE APPENDIX IV) that interaction with the system tended to happen in both spaces at roughly the same times. This suggests that interaction initiated by participants in one space would cause a reaction by participants in the other space, indicating the power of games and competition to motivate participation.

Interestingly, the visualisations reveal that participants would rarely interact deliberately with SuperFight I until the system registered the presence of people in the remote space.

The times when the break-beam sensor in both studios were triggered and the sound level reached 100, are shown below.

12.00 13.00 14.00 15.00 16.00 17.00 TUESDAY MARCH 12.34 16.21, 17.19 16.29 17.54 VE 16.14 17.16 17.42 WEDS MARCH 17.31 VE 17.55 17.56 17.57 THURSDAY MARCH 12.01 14.49 14.50 14.51 14.52 14.53 VE 13.48 14.27 13.51 14.31 13.52 14.42 14.44 14.45 14.46 14.47

Looking at this information we can find two occasions when it is reasonable to assume that synchronous interaction happened. These times are:

TUESDAY 17.16 - 17.19 THURSDAY 14.42 - 14.53

From observation of the use of the system I know that in fact, real- time and asynchronous interaction between the two spaces took place

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

even more frequently than it is reasonable to assume from analysing the log files. Observation revealed that asynchronous interaction usually involved one team playing until their game total was higher than that of the opposing space, while in real-time interaction participants would play as strategic a game as possible given the nature of the sensors.

People start to ‘read’ the ambient information to extend their knowledge about the physical space outside of their immediate environment.

Participants responses indicate that they were able to ‘read’ the information presented in the visual display and that despite the abstraction of the game context, they did gain an increased understanding of the characteristics of the other space.

Question to participants: What did you learn about the MArch / VE studio and students from playing the game? VE: That is is much busier than our studio! VE: I would take a look at the game every now and then - my place was well positioned in respect to the projection. Many times I would start thinking or imagining what might be happening in the distant studio and how people would be using it. I had the tendency to believe that people would come in and out from the studio, being lively & noisy with each other: I think if the towers had more steps in their trembling behaviour (different noise levels), it would provide more degrees of freedom for my imagination. VE: I did realise that they are working during summer semester, something I wasn't aware of before the game. VE: That there are more of them, and the are probably a little bit more inquisitive and interested in play than us. VE: I could feel the existance and movement of the others.

The understanding of the remote physical environment shown by respondents is directly linked to the information that the sensors transmit. The results suggest that movement and action are capable of communicating the character of a space and its inhabitants and that this information can be understood through abstract representation. So long as the sensors reflect natural actions for the particular space, the use of more sensors should increase this level of understanding.

People’s behaviour is unconsciously affected by their reading of this information.

There is no evidence of this happening in this experiment, but it is possible that it might occur if the system was installed for a longer period of time. One indication that behaviour might be affected by an

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

ambient display is that interaction with the system tended to happen in both spaces at roughly the same time. This might suggest that at some level, participants were aware of the information on the display even when they were not actively engaged with the system.

People don't like the integration of digital space into physical space

This is the most subjective aspect of the design and installation. While I received various comments and suggestions about the visuals and installation, there was nothing that implies the actual integration of digital and physical space is unwelcome.

Question to participants: - Did you find the game fun, annoying, intrusive, boring, entertaining, interesting? Or choose your own word to describe the game… MARCH: interesting, it was weird not having it around at the end of the week. VE: interesting but not entertaining. maybe because I could'nt see the poeple in MArch So It seemed there was no relationship with MArch.. It might have been helpful if I'd been able to see them. VE: I find it entertaining and it keeps you company, feeling others people presence is important, knowing that there are people working as hard as you, you're not alone! specially if you are working a lot of hours per day. Even though I appreciate your "retro" 70's stylism in the visuals, and not being flashy to not distract the user, I think the visual graphics could be more expressive. VE: Overall, I found it interesting, by no means intrusive and very well installed. I think it worked well on the column, because it was in an ideal size and the first thing you saw while entering- the effect of the screen become red was very effective. VE: Entertaining, and engaging insightful as to the other people within our own space and there relationship to the game VE: I like the idea that the game is played by two group each of which is in a sparated area. But I wanted to be more active, and the image of the game should have more exciting dynamics, such as repulsion or attraction.

The response to the installation revealed the extent to which the identity of the physical spaces differed, even though both spaces exist in the same context. While the MArch students appropriated the system set up in their space, the Virtual Environments students played with the system within the parameters I had set. One of the limitations of SuperFight is that there is relatively little scope for appropriation so it is a credit to the imagination of the MArch students that they found a way to make the system their own by adding to the visual display through drawing on the wall where the display was projected.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

The installation of the various elements of the system in the physical space did not seem to inhibit interaction. In fact, projecting the visuals onto existing surfaces in the physical spaces rather than a onto a screen appeared to help the integration of the digital space into the physical environment. It was this that gave the MArch students the opportunity to alter the display as they did.

For security reasons the installation was taken down every evening, so the game was generally in play from between 12pm and 6pm. The number of people around between these times was different for the two spaces. It would have been interesting to run the system continuously to see how this affected the interaction. Possibly games would have developed an extended playing time with one space playing by day and the other space responding in the evening.

I feel the co-option of the system by participants, and the display of differences in character between the two physical spaces supports the need to understand the identity of the physical spaces before the system is designed and installed. In this case the observational study allowed the system to successfully develop an identity with which participants could relate.

‘Place-ness’ and the relationship between the physical and digital environments

The identification of the physical space with the tower in the projected display worked well, with people making comments identifying themselves with the game such as ‘we’re winning’, ‘we beat you yesterday’, ‘they made our tower fall down’.

Question to participants: - How do you play the game? VE: With interest and load noises to destroy there castle

The competition between the physical spaces which the game inspired was an important factor in encouraging interaction. The inhabitants of the physical space with the higher score in SuperFight, would tend not to initiate deliberate interaction with the system. When the space with the lower score interacted with the system they would play until their score became higher than the score of the other space and then stop. This meant that the initiation of play was passed back and forth between the two spaces.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

5. Future work

The relationship of actions to visuals is a fascinating area and relatively unexplored. There are many questions that remain to be answered: How can the affordances of actions be represented through abstract representation? Could a language be developed for the visual interpretation of movement that would allow movements to be ‘read’ by a series of graphic forms? Or should the connection be purely arbitrary and reflect the personal choice of the system designer? Investigating whether the visual interpretation of a movement has an effect on the comprehension and co-option of participants would be an interesting study.

Similarly, it would be interesting to examine how the process of collecting information on the natural behaviour of participants in the physical environment might be developed. Could a greater understanding of the physical ‘place’ help the designer create a system which integrates more closely with the physical environment? And would this increase the likelihood of participant’s deliberately interacting with the system?

Another area that would be intriguing to explore is how the system might be given ‘intelligence’ allowing it to adapt to the changing use of the physical environment. This might be done by using the sensors to examine what behaviour is currently occurring in the space as well as to trigger the visual response. For example, a space might be equipped with all possible sensors, detecting all types of action including those which may well never happen in that space. The system would be capable of evaluating what is currently happening in the space, and comparing this information to what has previously taken place there. As spaces change function over time this would allow the system to adapt to these changes and remain relevant to the users of the space for longer.

The installation of the equipment within the physical environment might also be investigated to see whether this has any effect on the engagement of the participants. SuperFightI used a large screen display but there are many other types of output that might be used with this system. These include all types and sizes of display screen, but also lights, sound, motors and other mechanical and electrical devices which can communicate a change in state. Similarly the design of the sensors might be investigated to see whether a greater amount of interaction is generated by hiding the sensors from view, or alternatively, by integrating the sensors the physical environment as well-designed objects in themselves.

The design process succeeded in producing a generic system for using digital environments to interact indirectly between physical spaces. The underlying technology of SuperFight I can be used as the

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

basis for new projects which aim to expand on the themes and questions raised in this dissertation. The technology and concept can be extended to connect more than two physical environments. This means that future work can focus on improving understanding of how actions can be interpreted visually, how the identity of a space can be understood by the actions occurring in it and how this identity can be communicated over a computer network.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

6. Conclusion

The history of architecture offers innumerable examples of places which embed and narrate a story through their spatial layout and décor. By looking at the sequence of floor plans of historical buildings through the centuries, from the Greek temple, to the Roman church, the medieval dome through today, we understand how a rectangle, a circle, a cross, or other more complex figures, transmit a message through the centuries

Flavia Sparacino. NARRATIVE SPACES: BRIDGING ARCHITECTURE AND ENTERTAINMENT VIA INTERACTIVE TECHNOLOGY. 2002

Architecture embodies the narrative and identity of space and the relationship between a space and its inhabitants. Telecommunications technology mediates interaction, allowing us to interact with people outside of our immediate environment. New technological developments now allow us to use more natural actions to interact across communications networks. This dissertation investigated how actions might be used to carry the identity of a physical environment across these indirect communication networks, and how this might affect the behaviour of occupants of the local and remote spaces. It presented an overview of existing interactive works developed for public spaces, and examined their strengths and weaknesses in encouraging participation with the system and awareness of the other inhabitants of the space. This investigation encouraged me to devise a framework for the construction of an interactive system to be installed in a public space, which would increase participants awareness of their physical location.

I hope that this dissertation provides evidence that the identity of a physical space can be communicated and understood through the integration of a digital environment. I hope that the questions raised by the development of the SuperFight I system, and the conclusions drawn from the experiment, will help extend how physical space can be understood by the actions of the people who inhabit them.

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

Appendix I

Initial contact with the occupants of the spaces concerning the project was by email. The subject line was MSc Final Project: Super Fight I and the text read: hello,

I’m writing to introduce Super Fight I, my final project for the MSc Virtual Environments, which Peter Cook and Lesley Gavin have agreed can be installed in the MArch and Virtual Environments studios from the start of august.

The idea of the project is to try to establish a relationship between two physical spaces via a shared digital space. There’s no obligation to take part and the installation won’t get in the way of anyone’s work.

Before the project is set up there will be some pre-installation stuff appearing in both spaces to introduce the project to you, and to help me understand how the spaces are used.

I hope this is ok with everyone, but if you have any questions or comments please email me. thanks, karen

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

Appendix II

Potential action / response pairings for studios

ACTION SENSOR TRIGGER STATE CHANGE RANGE No. of people Motion - IR 1 person When anyone enters 0 – max no. of in room Vision – webcam or exits people mobile Radio wave Making or When phone call is 0 to max no. of phone call detector receiving made or received phones in the mobile phone room call. Sending receiving text message? music Audio detector/ Sound in Music 0 – max volume microphone room starting/stopping Volume of music increasing/decreasing talking Audio detector/ Sound in Talking 0 – max volume microphone room starting/stopping Volume of talking increasing/decreasing typing Touch Touch on Individual keys being 1 person to max Computer activity? keyboard touched no. of people 0 activity to 1 key at a time using Touch Touch on Individual keys being 1 person to max computers Network activity keyboard, touched no. of computers. Electromagnetic Info being Increase/decrease of 0 activity – 1 key radiation? sent/received network traffic at time over network Number of monitors 0 network Monitor is on are on/off traffic(?) to max 0 monitors on to max no. of monitors on Eating / Noise? Sound / smell Increase in number 0 to ? drinking Smell? of eating of people eating. Temperature? Room Change in type of Liquid detection? temperature food being eaten. heats up Something wet gets spilt on sensor moving Vision tracker - 1 person or Co-ordinates of 0 to lots of around webcam object moving movement Of people moving around Motion tracker – ir, around room or objects 0 to max no. of theremin? No. of people moving people moving IR motion detector passing People moving past distance from specific point certain place sensor to in room As people move in person/object/wall distance to front of fixed objects object making stuff Sound Sound Volume increase / 0 to max volume Touch Stuff moving decrease 0 to max no. of Pressure around on Stuff being put down touch sensors table or picked up over shortish time reading Sound? / quiet Someone in When one person 0 to 1 person Pressure room but no around but no noise computer or network activity activity or noise

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

Someone sitting on seat used for reading emailing Network traffic, on Sending or Email being sent or 0 to max no. of email server? receiving received computers pop3 email looking at Network traffic on Viewing web Web page being 0 to ?? web web server pages viewed time of day Computer system Change in Clock changes 0 to 23.59 clock clock time Light Change in Light 0 to max. light increases/decreases conditions date Computer system Change in Date changes 01.08.03 to date system date 21.08.03 temperature Room temperature Change in Temperature rises or ? room lowers temperature Looking at Light / dark Shadow Shadows move nearer 0 (no shadows) to output falling on or further away fully covered by display sensor on shadow screen Stress / Air pressure sighing When someone sighs 0 to ? sighing near sensor Output sudden changes in projector When display starts 0ff to on display is light switches on, active or if transformatio n turns screen from dark to light orientation of Orientation Where the None static room, north, room is south, east, west

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

Appendix III

Observational study of Virtual Environments studio

DAY 1 DAY 2 DAY 3 09.30 09.45 Person 1 opens up the studio 10.00 Person 1 in the studio looking at website. 10.15 Person 1 working on laptop. Person 2 arrives, sets up laptop in usual place and writes emails 10.30 Person 3 calls in. Chats to Person 1 for few minutes then leaves again 10.45 Person 2 goes to shop, asks if anyone wants anything. Quick chat between Person 2 and Person 1 about c++ projects and future while drinking coffee. 11.00 Person 1 working on C++ project at laptop Person 4, setting up flock of birds equipment 11.15 11.30 Person 4 leaves Person 2 goes to library. Quick chat with Person 1 about what he's looking for. 11.45 Person 2 arrives and sets up his laptop in the usual place 12.00 Person 5 arrives and has quick chat with Person 1. Person 5 sits on sofa to read book Person 2 is back from library. 12.15 12.30 Person 6 arrives and checks email at usual computer 12.45 Friend of Person 7's calls in and chats to Person 5. Person 2 borrows £1 and goes to the gym 13.00 Person 7's friend leaves Person 8 arrives and chooses a computer for email. Person 8 takes mobile call 13.15 Person 9 arrives, has question about scanning (for some web work), Person 9 checks email and looks at web 13.30 Person 8 gets text message Person 1 goes to get lunch

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

and to a book shop 13.45 Person 6 checks email at usual computer 14.00 Person 2 arrives. with lunch Person 3, Person 10, Person 11 and sets up laptop in usual and Person 12 arrive for a place meeting about a presentation they have to give a week on Monday. Sit in a corner of the room in front of a computer. Person 10’s friend is here too and takes a computer on the other side of the room to browse the web. 14.15 Person 9 and Person 6 leave together (but will be back because stuff is still here) 14.30 Person 9 and Person 6 are back, work together to try to fix one of the computers Everyone working separately at their own computers. No noise 14.45 15.00 Person 1 returns with Person 3, Person 11 and Person book. 12 leave. Shows book to Person 6, Person 10 searches the web for a Person 2 and Person 5. while Person 1 and Person 5 look through chapter of the book. Person 1 eats lunch 15.15 Person 2 makes phone call Person 10 and his friend leave. on landline phone and leaves but will be back as computer is still here Person 9 and Person 6 both take phone calls, Person 9 has had 2 text messages 15.30 Person 5 leaves Person 2 leaves Person 3 pops in looking for Person 7 15.45 16.00 Person2 returns Person 13 arrives and Person2 returns Person 1 leaves for a while chooses a computer 16.15 Person 14 arrives and chooses a computer 16.30 Person 1 back Person 14 leaves. 16.45 Person 9 leaves Person 13 leaves Person 7arrives and sits at usual computer 17.00 Person 2 leaves for a while Person 7 arrives 17.15 Person 9 leaves 17.30 Person 1 goes to make a phone call. Person 2 leaves but will be back 17.45 Person 7checks email then leaves for the library 18.00 Person 1 leaves Person 6 and Person 2 work together to try to sort out Person 2’s C++ problems 18.15 Person 1 gets phone call, leaves studio for few

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

minutes 18.30 Everyone working quietly. Person 2 returns Person 6 has music on Person 1 leaves very quietly 18.45 19.00 Person 7 leaves Person 1 leaves

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

Appendix IV

A) Graphs showing sensor activity during first five minutes of every hour the system was running.

DAY 1

DAY 2

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

DAY 3

B) Diagrams showing times when break beam sensors were triggered in both spaces simultaneously, and microphone reading was 100.

DAY 1

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

DAY 2

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

DAY 3

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

Appendix V

Questions emailed to all occupants of MArch studios asking about their perception of the SuperFight project. A similar email was sent to all users of the Virtual Environments studio.

Hello,

Thank you very much for letting me install the SuperFight project in your studio, and thanks for playing with the game. You were the winners…

If you have time, would you please answer the 5 questions below and email them to me? The answers don’t have to be long but it would be very helpful to my dissertation to get your replies.

Thanks again, and good luck with all your work. Karen

Questions about the SuperFight project. 1. Did you play the game? 2. How do you play the game? 3. Do you know anyone on the Virtual Environments course? 4. Did you find the game fun, annoying, intrusive, boring, entertaining, interesting? Or choose your own word to describe the game… 5. What did you learn about the virtual environments studio and students from playing the game?

SuperFight I: A battle to understand a space through the behaviour of it’s occupants

Bibliography

Benford, S., Greenhalgh, C. Reynard, G., Brown, C. & Koleva, B. UNDERSTANDING AND CONSTRUCTING SHARED SPACES WITH MIXED REALITY BOUNDARIES. 1998

Brignull, H. & Rogers, Y. ENTICING PEOPLE TO INTERACT WITH LARGE PUBLIC DISPLAYS IN PUBLIC SPACES. 2002

Dix, A. BEYOND INTENTION. PUSHING BOUNDARIES WITH INCIDENTAL INTERACTION. 2002.

Dunne, A. HERTZIAN TALES. 1999

Gaver, B., Dunne, T. & Pacenti, E. CULTURAL PROBES. 1999

Hampton, K. PLACE-BASED AND IT MEDIATED 'COMMUNITY'. 2002

Hampton, K. THE NOT SO GLOBAL VILLAGE OF NETVILLE. 2001

Harrison, S. & Dourish, P. RE-PLACE-ING SPACE: THE ROLES OF PLACE AND SPACE IN COLLABORATIVE SYSTEMS. 1996

Hemmings, T., Crabtree , A., Rodden, T. & Rouncefield, M. PROBING THE PROBES. DOMESTIC PROBES AND THE DESIGN PROCESS. 2002

Mccarthy, J. F. USING PUBLIC DISPLAYS TO CREATE CONVERSATION OPPORTUNITIES. 2002

McCloud, S. UNDERSTANDING COMICS. 1993

Muir, E. & O'Neill, R. ARCHITECTURE OF DIGITAL SPACE. 1995

Muller, M. Participatory Design: THE THIRD SPACE IN HCI. 2001.

Norman, D. THE PSYCHOLOGY OF EVERYDAY THINGS. 1988

Rogers, Y., UNDERSTANDING INTERACTION: GOING BEYOND THE SEAMLESS INTEGRATION OF THE DIGITAL AND THE PHYSICAL. 2002

Schnadelbach, H., Penn, A., Benford, S. & Koleva, B. MIXED REALITY ARCHITECTURE: CONCEPT, CONSTRUCTION, USE. 2003

Sparacino, F. NARRATIVE SPACES: BRIDGING ARCHITECTURE AND ENTERTAINMENT VIA INTERACTIVE TECHNOLOGY. From Generative Art Conference 2002

Waterworth, J. A. TECHNOLOGY IN SUPPORT OF RETURNING: FROM CONSCIOUS DOING TO CONSCIOUSLY BEING. 2000

SuperFight I: A battle to understand a space through the behaviour of it’s occupants