Equator technical innovation in physical and digital life

A PROPOSAL FOR THE FORMATION OF AN INTERDISCIPLINARY RESEARCH COLLABORATION

Case for Support

Professor David May, University of Bristol Dr Matthew Chalmers, University of Glasgow Professor Tom Rodden, Lancaster University Professor Steve Benford, The University of Nottingham Professor Gillian Crampton-Smith, Royal College of Art Professor Wendy Hall, University of Southampton Dr Yvonne Rogers, University of Sussex Professor Mel Slater, University College London Summary The central goal of the Equator IRC is to promote the integration of the physical with the digital. In particular, we are concerned with uncovering and supporting the variety of possible relationships between physical and digital worlds. Our objective in doing this is to improve the quality of everyday life by building and adapting technologies for a range of user groups and application domains. Examples include: − combining physical and digital cities to promote people’s understanding of the world within which they live, and to enhance wayfinding and access to physical and digital artefacts, information and people. − creating new forms of play, performance and entertainment that combine the physical and digital so as to promote learning, participation and creativity. − exploring how new technologies that merge the physical and the digital can support activities outside of the workplace, including maintaining family and social relationships in the home, and supporting work in the open air. Meeting this objective will require us to address fundamental and long-term research challenges. We will conduct research into new classes of device that link the physical and the digital, including embedded devices that are integrated into physical environments, information appliances that combine computing functionality with purpose designed physical objects, and wearable devices that are carried on the person. In turn, these activities will be supported by fundamental research into adaptive software architectures that can knit together heterogeneous collections of such devices, as well as new design and evaluation methods that draw together approaches from social science, cognitive science and art and design. To achieve these goals we have brought together a group of the UK’s leading, internationally known, academic researchers in the design, development and study of interactive technologies for everyday settings. All have a track record of working in interdisciplinary teams, and many have worked together and continue to do so on current EPSRC, ESRC and EU funded projects. The expertise of the group is rich and diverse: it includes hardware engineering (Bristol), computer graphics (UCL), mobile multimedia systems (Lancaster, UCL), art and design (RCA), software development and system architecture (Lancaster, Nottingham, Southampton, UCL), information sciences (Glasgow) and social and cognitive sciences (Sussex, Lancaster, Nottingham).

The Equator vision As digital technologies have matured, they have begun to move beyond the workplace to other domains in our everyday lives: our homes, neighbourhoods, what we wear and carry with us. At the same time, the phenomenal spread of the Internet has enabled the public to participate in a variety of new online experiences, such as email, distributed hypermedia and virtual reality. The current convergence of interactive digital systems, networks and mobile devices is further transforming the ways that we carry out our everyday life, e.g. how we entertain ourselves, work, shop and converse. We increasingly undertake everyday activities and share our lives with others in both physical and digital environments, continually stepping over the border between the two. Many actions in our physical environment have analogues and effects in the digital, and vice versa. It becomes possible to link geographically distant people, to access information from remote locations, and to draw from the recorded past to support ongoing activity and plans for the future. For example, a car driver checks her location, plan, and route recommendations on a dashboard display, circumventing a traffic jam while hardly aware of the system of computers and satellites in the background. An Internet shopper chooses an item from a web site, triggering a chain of events in the physical and digital worlds that end with a physical book being delivered to their home. Despite these ongoing developments, there are still many everyday activities where the boundary between physical environments and digital space is often over-complex and poorly designed. For example, many people are frustrated with the digital information found in public kiosks when trying to find their way in a strange city. The physical technology can be cumbersome and awkward to use, while the online information is often difficult to navigate and understand. In contrast, our vision is to allow people to pass between the physical and the digital so readily that the boundary becomes just a line on a map, rather than an obstacle to their activities, goals and desires. The physical and the digital need to be seen as integrated and interdependent aspects of our everyday world, rather than disjoint and independent spheres of activity. Creating such a seamless integration, however, is difficult. It requires long term research into new models of interaction, new interface and distribution technologies, new applications and new methods.

Equator objectives and results To achieve our vision of the universal integration of the physical and digital worlds we have set ourselves the following objectives: 1. To develop new theories and concepts to understand the interplay between the physical and the digital. 2. To create devices to establish new relationships between the physical and the digital. 3. To develop new forms of adaptive infrastructure to support heterogeneous collections of these devices.

1 4. To inform this research with direct experience of how these technologies can be used to support interaction, exploration, communication, play and learning by real users in a variety of everyday settings. 5. To generate design and evaluation methods appropriate to these technologies based on a combination of approaches from cognitive science, social science and art and design. 6. To disseminate the results of this research to the international research community, the IT industry, user groups, and the general public.

The main outcomes of Equator will be technological advances, new applications, methodological advances, and advances in our understanding of the relationships between the physical and the digital.

Technological advances – Equator will produce new techniques for linking the physical and the digital that move beyond the design assumptions inherent in the current personal desktop computer, including: − Virtual environments that link to physical environments in new ways, for example through the integration of video and audio and other sensory data. − Embedded devices that support new forms of reactive and traversable public display that embed digital spaces into everyday physical spaces. − Information appliances that link digital information to purpose designed physical objects. − Wearable computers that provide continuous and parallel access to physical environments and digital spaces as people move about. − Adaptive middleware that supports large collections of heterogeneous devices that link the physical to the digital.

New applications – Equator will create new applications that focus on different aspects of people’s everyday lives: − Community – applications to enhance existing physical communities and to create new forms of digital community. − Creativity – applications to support self-expression. − Education – applications to provide learning experiences for all ages. − Leisure – applications to increase public participation in new forms of art, performance and entertainment. − Home – applications to enhance domestic life, especially support for family and social relationships. − Work – applications that focus on work outside of the conventional workplace, for example in the open air.

Methodological advances – Equator will generate a suite of interdisciplinary design and evaluation methods, including: − User studies– interventionist-based methods from art and design will be used together with ethnographic techniques from social science to understand everyday practice and to inform the design process. − Envisionment techniques–drawing on ideas from the visual and conceptual arts, new techniques will be evolved that engage users’ imaginations with impressionistic design proposals in order to support the early stages of design. − Evaluation methods – experimental techniques from cognitive science, naturalistic observation techniques from social science, and dramatic interventions from the arts will be integrated to help understand users’ experiences.

Advances in understanding - Equator will produce new understandings of the relationships between the physical and the digital. These will be based on the need to anticipate scenarios of use in which the technologies will be embedded. Our collective experience has been that current methods of ‘forecasting’ the value of new technologies have been of very limited success. We shall overcome this problem by synthesising the conceptual and methodological strengths of the partners in cognitive and social science, design practice and system engineering and implementation to consider innovation at several levels of analysis.

The Equator Approach Pursuing the Equator vision requires a shift in our thinking about research, towards using and developing theories, practices and techniques that are interdisciplinary and also following a process that directly engages users, while still supporting fundamental long term research. The distinctiveness of Equator arises from the disciplines involved, its research process and its management. The disciplines in Equator are computer science, electronics, cognitive science, social science and art and design. The development of embedded devices, information appliances and wearable devices that link the physical and digital requires technical expertise in both software and hardware. Experts in virtual reality, hypermedia, distributed systems, interface design and information systems must work together with experts in electronics and mobile devices just to realise the underlying technologies. Developing these in a manner that makes them appropriate for the general citizen requires further expertise in design and evaluation that draws on the social, cognitive and artistic.

2 The future information citizen is central to our vision of a seamless integrated environment where the physical and digital worlds form part of an everyday experience. The successful realisation of this vision demands that we develop and apply techniques for grounding this new technology in the real, lived experience of everyday life. Equator’s model is to use expertise from cognitive and social science to specify how information technology can and does play a role in current social practice, and to combine this with imaginative design methods to suggest how innovation might reshape these activities. Cognitive science is required to understand how people interact with the technologies. Social science is required to understand how the technologies become situated within everyday settings. Art and design is required to understand people’s aspirations and needs and to help design technologies that are not only useful, but are also engaging, stimulating and creative. Equator’s research process combines practical and focused experiences with long term research into new technologies and methods. Equator carries out experience projects in order to apply a combination of technologies and methods to a particular domain. They engage real users, often the public, in the research process. They may involve ethnographic studies of the chosen domain or interaction with users through ‘cultural probes’ and other methods from art and design in order to inform the design of new applications. They will involve taking new technology out of the laboratory and placing it in public domains such as city centres, theatres, schools and the home, and then evaluation using both cognitive and social science methods so as to feed back into long term research. At the same time, Equator carries out long-term research challenges to explore fundamental issues. They generalise the lessons from experience projects to provide new conceptual frameworks and understanding, and also create new technologies and methods to support them. The key to combining practical experience projects with fundamental research challenges is iteration over an extended time period so that lessons can be learned, fed back into research, solutions developed and then fed back into further experiences. The iterative combination of experience projects with fundamental research can not be achieved through a collection of separate projects or even a managed programme where a collection of projects address a set of themes. This approach is only made possible through the mechanism of an IRC where it is possible to strongly manage the process to ensure cooperation and multidisciplinary working. The management of Equator is the final key feature of our approach. Equator will be undertaking research in a rapidly evolving area that is subject to continual fluctuation and change. The adventurous nature of the research is reflected in the management process used to direct it. Continual risk assessment and re-planning is the key tenet of the management philosophy. In addition to regular six monthly management meetings, Equator will instigate an annual formal review of progress against its workplan and of the workplan and the mission of the IRC itself. This will be supported by an invited panel of international experts drawn from the pool of associates listed in Appendix I. This review will take place following an annual conference at which results are presented to the panel and to the broader research community, users and the public. The rigorous nature of the review is underpinned by the ability of a management committee to annually re-profile a significant proportion of Equator’s resources to target particular activities and needs. This approach is only possible within the framework of an IRC. Professor Tom Rodden, who has extensive experience of managing large interdisciplinary projects in this manner, will manage Equator. These previous projects included the COMIC project, an international, twelve site multi-disciplinary basic research project involving over sixty researchers that is still publicised as one of the EU’s 101 Key Success Stories.

Why the Equator IRC? Our research is based on a vision of extending and enhancing everyday activities in innovative ways. The scale of this vision is such that it is outside the scope of any single research group. The discussion of our approach has already stressed that our research is interdisciplinary, it requires iteration over an extended time period to establish feedback between experiences and fundamental research, and it requires a strong management process to ensure responsiveness and integration. These three key characteristics are inherent to the research problem and suggest an IRC as the ideal approach to tackling this challenge. However, other strong reasons also exist for the formation on an IRC. Sharing facilities – our research requires access to a wide range of specialised facilities including large-scale embedded displays such as CAVEs and Reality Centres, wearable computers, facilities to fabricate information appliances, powerful media servers, as well as public test facilities to support experience projects. All of these facilities are available across the Equator partners, many of them funded through the JREI initiative. UCL provides a five sided CAVE (JREI funded). Nottingham provides the Mixed Reality Laboratory (JREI). Lancaster provides the ‘Wireless Lancaster’ testbed for mobile communications within the city of Lancaster, and is developing an interaction lab to support the work of the IRC. Southampton provides a multimedia database server (JREI). Bristol provides wearable computers called Cyberjackets. Glasgow provides whole body scanners through the 3Dmatic Faraday centre. These facilities will support specific projects within Equator. They will also be used to host ‘technology camps’, extended workshops where researchers from different partners gather at a facility for an intensive exploration of its potential. Establishing a community – the topic of integrating physical and digital interaction is a new one, but one that is currently gaining momentum on the international scene as is witnessed by panels and demonstrations of mixed reality and related topics at major conferences such as SIGGRAPH and CHI. The strength of interest is also reflected in the number of leading research centres wishing to be associated with this IRC. The convergent nature of research in this

3 area requires considerable synergy between disciplines and a critical mass of researchers. However, UK interests in this area are currently disparate. The formation of an IRC would be a major step in creating a new research community in the UK in an emerging area. It is already worth noting how significant an impact the suggestion of forming this IRC has had in raising the collective research profile of the UK in this area and how this is reflected in the letters of support from world leading research groups in Appendix I. Strengthening the UK’s scientific position – the kind of interdisciplinary research proposed for Equator has also been adopted in several commercial R&D organisations. For example, Apple, Xerox, Hewlett Packard, Interval, Sony and Philips have all at various times created ‘future technology’ labs, bringing together multidisciplinary research groups. In building such teams, companies have relied heavily on the input of academic researchers (e.g., the successful Interval Corporation programmes with Stanford and the RCA). However, commercial demands have often prevented these labs from carrying out long-term interdisciplinary research into technology innovation. A converse model, where R&D is based in a university environment, and commercial bodies are asked to act as collaborating partners offers more scope for such a venture. However, there is currently only one correspondingly impressive example of this approach. That is the MIT Media Lab. Our goal is not to emulate the Media Lab, but instead to pursue our own research agenda and approach, bringing together a unique interdisciplinary group of internationally acclaimed researchers from the UK who, crucially, share an underlying vision, who are situated in Europe, and who seek to engage users. Equator is distinct from these other commercial and academic ventures in two ways. First, it draws on a very broad base of technical expertise. In particular, it brings together international experts in both digital world technologies such as hypermedia and virtual reality, with experts in physical world technologies such as wearables and information appliances. It also draws upon traditional UK strengths in the social sciences and art and design and it is committed to designing and evaluating new technologies in collaboration with end users. It is determined to pass beyond building impressive demonstrators in the laboratory and to engage end users directly in the scientific process. This distinctiveness is reflected in the support we have gained from the industrial associates in Appendix I.

Relevance to beneficiaries There will be four main classes of beneficiary from Equator: research communities, industry, user groups, and the public. Research communities that will benefit span the key disciplines involved in Equator: computer science, electronics, sociology, psychology and art and design. Equator will provide these with new models, methods and techniques for designing and evaluating technologies that merge the physical and digital in a variety of settings. Our results will be particularly relevant to the HCI and mutlimedia networking communities. Delivery of these benefits will be through normal dissemination routes and cooperation with the world leading research centres. The attached letters of support show the involvement in Equator of international research laboratories including MIT, GMD, Georgia Tech, Xerox PARC and the Australian Distributed Systems Technology Centre as International Research Associates. The IT industry will benefit from the Equator research in terms of new understandings, devices and software architectures. The migration of digital technology into our everyday life represents a considerable potential for the establishment of new market areas for devices, services and applications and is attracting considerable worldwide attention as a future source of revenue. The attached letters of support reflect the commercial interest in this area. They demonstrate the involvement of large companies and SMEs across a variety of sectors including telecommunications and networking (BT, AT&T, CISCO), IT hardware and software (IBM, Xerox, Virtual Presence, Lucent, Intel, HP), electronics (Philips, The Appliance Studio, ST) and others. These benefits will be delivered through engagement with these Industrial Associates many of whom have stated a willingness to develop future research project that link with Equator. User groups that will benefit also span a range of sectors. They will benefit from understanding the potential ways in which emerging technologies that link the physical and the digital may impact on their customers’ and clients’ lives in the future. The attached letters of support demonstrate the direct involvement of the media (Illuminations Television, the Independent Television Commission), museums and galleries (The Hunterian Museum, Glasgow Science Centre, The Lighthouse Trust), tourism (Lancaster Tourism), the arts (Nottingham City Council NOW festival), banking and finance (Natwest, NCR), and the emergency services (Cumbraa Emergency Services). The general public will benefit from Equator’s research programme which is aimed at enhancing and extending everyday life in a number of different domains, such as travel, play and entertainment, the home and work. Involvement of museums and galleries (The Hunterian Museum, Glasgow Science Centre, The Lighthouse Trust), a television production company (Illuminations) and the expertise of partners in producing public demonstrations and exhibits provides a means of delivering benefits. In addition, the involvement of Independent Television Commission as the UK regulator for media will allow a direct engagement with public policy. These different beneficiaries are represented in the Equator Associate Programme (see Appendix I) which is designed to ensure that the critical mass of researchers resulting from the formation on an IRC can deliver maximum benefits.

4 The Equator research programme An IRC provides an opportunity to undertake research in a way that is not possible in a normal three year EPRSC project or even within a managed research programme where a number of projects loosely address a set of research themes. In equator we exploit this opportunity to develop a focused and directed programme of research that promotes the convergence of disciplines and technologies to realise a future vision of the nature of computing. At the heart of the Equator research programme are a set of research challenges and a set of experience projects. Equator research challenges address fundamental research issues arising from the mixing of the physical and digital. Research challenges define both the technical and theoretical foundations for Equator. They run throughout its lifetime, being focused at particular points through integration activities. By supporting the cumulative development of knowledge and the iterative integration and generalision of results, research challenges are the core long-term initiatives that ensure that as an IRC, it is more than a thematic collection of projects. Equator experience projects provide practical tests of new technologies, practice new design and evaluation methods, engage users in the research process and publicly demonstrate results. Experience projects are application-oriented activities. They focus on supporting real users and exploit the methods, models and technologies emerging from the research challenges. The collective experience of the Equator partners is that interdisciplinary work succeeds by focusing on the construction of new technologies to inform fundamental research. Experience projects reflect this approach to research through continual demonstration and evaluation of emerging techniques. These projects are clustered into three research themes that address key aspects of everyday life, exploring and understanding, playing and learning, and living and working. Research challenges and experience projects are closely coupled and we anticipate considerable overlap of staff across them. The basic research undertaken within the research challenges is directly informed from the experiences of design, development and evaluation in the experience projects. Similarly, experience projects exploit and exercise the emerging result of the research challenges. This synergistic relationship allows us to simultaneously combine longer-term research activities with focused projects that ground the research in real world experiences, as is shown by the figure one.

Playing Living and and Experience Learning Working Projects Exploring and Understanding

Provide generic Generalise and supporting reflect on mechanisms, common services and experiences concepts

Device Challenges

Wearable Information Embedded Devices Appliances Devices

Research Challenges

Adaptive Understanding Computing Interaction

Core Challenges

Figure 1: The relationship between research challenges and experience projects In addition to the research challenges and experience projects, a variety of other mechanisms are defined to promote integration within Equator, to involve industrial and international collaborators and to disseminate and promote research results to the wider research community, potential users and the public. Equator integration mechanisms promote cross-fertilisation between the research challenges and experience projects and also between the partners and the disciplines. Equator will begin with a series of IRC challenge workshops that educate project staff and that define the research challenges in detail. Toolkit and framework activities integrate the results of the different research challenges to produce common frameworks and infrastructure. Themed integrated demonstrators integrate multiple experience projects into focused public demonstration on an annual basis. Technology

5 camps bring researchers together at one site to work with a particular facility or technology. Regular exchanges of students and staff promote cross partner working and understanding. Equator involvement mechanisms enable collaboration with external organisations. The Equator Associate Programme enables international research centres, industrial organisations and user groups to collaborate with Equator. Associates may collaborate on an experience project, host Equator students as interns, host Equator academics on sabbatical, place their own staff with the Equator partners, take part in the annual review process, and loan or exchange facilities (see Appendix I). Equator dissemination mechanisms target research results at other potential beneficiaries. An annual Equator conference presents results in a public forum and is part of the annual review process. Summer schools provide training and education for both Equator and external research students and staff. A book series will publish Equator results in a systematic and integrated way. Equator will participate in exhibitions at academic conferences such as SIGGRAPH and CHI and industry fairs such as CEBIT. Through MIT (letter 21) we will be able publish special issues of the academic journal Presence, through Illuminations we will be able to produce broadcast material (letter 15) and through the ITC (letter 17) we will have direct channel to a major UK media regulator. The remainder of this section focuses on the research challenges and experience projects. Integration, involvement and dissemination mechanisms are discussed later within the management section.

The Equator Research challenges Equator’s research is driven by five long-term and fundamental research challenges. Two core challenges provide the technological and methodological foundations for our work. The understanding interaction challenge develops the human centred design, theories and understanding needed to underpin the future development of these applications. The adaptive computing challenge focuses on the core computing questions arising from the need to support a highly distributed context-aware environment that spans physical and digital worlds.

The understanding interaction challenge What new design and evaluation methods are required to support the merging of physical and digital worlds?

While there exists considerable human factors guidance for designing various kinds of interface (e.g., graphical displays, direct manipulation interfaces, multimedia systems and Web-sites), this has largely been aimed at displaying information on desktop computers. We need to ask whether current approaches are appropriate to the design of interfaces where the relationship between the virtual and the physical extends beyond placing a monitor, keyboard and mouse on a desk. In particular, are they appropriate when computing functionality becomes embedded into the surrounding environment, into specific physical objects, or into objects that are worn? The preliminary research of the Equator partners suggests that this is not the case, and that new conceptual models, theories and guidelines are needed for explaining and informing the design of these emerging technologies. The understanding interaction challenge will establish new models of human-computer interaction that are appropriate to the mixing of physical and digital environments. It will develop guidelines to inform the design of new applications, which involve different configurations of virtual representations and physical environments. It will consider when is it best to synthesise, integrate, juxtapose, link or keep distinct different virtual/physical representations? When is it desirable not to do so? What are the implications for designing applications and novel user experiences? What kinds of virtual representations are best to use and how do we decide what is the optimal way of interacting with them in different contexts? These questions will be answered through two strands of research: − Design methods – this strand will extend current techniques, adapted from sociology and psychology, to include new techniques from art and design. Techniques from sociology include the use of ethnography to inform co- operative systems design. This involves observing different social settings to identify subtle and tacit working practices, and is a theme of research at both Lancaster and Sussex. Techniques adapted from psychology have resulted in interface design guidelines to promote system usability, a theme of research at Nottingham and Sussex. Techniques from art and design include the use of ‘cultural probes’ to discover people’s desires and aspirations and to evoke the imagination of designers, a theme of research at the RCA. The long term goal of Equator will be to blend these into a common armoury of design methods to support the design of new technologies that are usable, stimulating, engaging and account for actual social practice. − Evaluation methods – this strand will focus on how to evaluate applications that span the physical and digital in order to generate new insights for the technical research challenges. Experimental techniques adapted from psychology will be used to test specific design features and hypotheses in controlled laboratory settings. These will be complemented by ethnographically informed techniques adapted from sociology that observe the use of new technologies in everyday environments. Finally, techniques from art and design, such as public interventions, improvisational role playing and critical review, will be used to gauge people’s reactions to and feelings about new technologies. Integration between these strands will consider the relationships between these different techniques and the extent to which they fit into an overall design and evaluation methodology. This will include an ongoing reflection on the

6 relationship between the disciplines of computer science, electronics, social science, cognitive science and art and design, including documentation of success stories and difficulties from the experience projects.

The adaptive computing challenge What common software infrastructure is required to support the merging of physical and digital worlds?

This challenge focuses on an adaptive software infrastructure to support ‘smart spaces’ that span both the digital and the physical. Such spaces will naturally be composed of a heterogeneous mix of communications technologies, information services, and devices, including collaborative virtual environments, streamed real-time media, distributed object services, mobile and wearable devices, and tracking and sensing technologies. Applications that combine these technologies will require support from a highly flexible middleware that provides a uniform model of location and context, realised through a set of common services. A key feature of this middleware will be the need to adapt to dynamic changes in physical and virtual context. The Equator research will focus on three classes of common service. − Linking services – the transition from hypertext to hypermedia involved extending the notion of links between fragments of text to links between different media types. A similar shift is required to support applications that link digital objects to physical objects. Equator will explore the notion of these kinds of hyper-physical links. In so doing, it will address a number of issues. How are such links created? Are they permanent or temporary? Can they be fully symmetrical (for example how can interaction with virtual objects affect physical objects and vice versa)? In addressing these issues, Equator will build upon previous work at Southampton on link services for hypermedia and at Glasgow in distributed information models. − Communication services – new session management mechanisms will be required to dynamically connect users and objects as they move between different physical and virtual contexts. Quality of service management will be highly dynamic, especially as users swap devices (e.g., a user may remain within a digital environment as they move from a CAVE connected to a high-speed network, to a wearable connected to a mobile network). This research will extend previous work at Nottingham and UCL on communication services for collaborative virtual environments (e.g., the MASSIVE and COVEN projects), at Lancaster and UCL on communication services for mobile devices (e.g., the GUIDE project), and at Bristol on communication services for wearables. − Interface services – user interfaces will need to recognise and adapt to their context of use, for example recognising requirements for privacy and differing styles of interaction as users move between home, work and leisure situations. User interface services will also need to adapt themselves to run across a heterogeneous collection of devices. In addressing these issues, Equator will extend previous work at Lancaster on adaptive interface frameworks, at Bristol on interfaces for wearables and at UCL on guaranteed rendering for virtual environments. These two core challenges are complemented by closely related device challenges that provide a structured means of funnelling the results of the core challenges into the creation of new technologies and associated models and understandings (figure 1). We initially divide these technologies into three different classes of device, embedded devices, information appliances and wearable devices, each of which combines the physical and the digital in a different way to establish a different kind of boundary between physical and digital worlds. However, these are closely related and we would see these distinct challenges merging over the six years of Equator.

The embedded devices challenge What is the best way to embed display and interaction technologies in our surroundings?

Embedded devices are those that are integrated into our surroundings. One approach to combining the physical and the digital is to integrate devices into the physical fabric of everyday environments in the form of public displays, specialised sensors, video walls, ambient displays and active furniture. In the most extreme case of a fully immersive CAVE, the embedded device becomes our entire surroundings. The goal of this challenge is to understand how these kinds of embedded devices may be realised and used. While we have some understanding about the physical construction of these devices, little is known about how these will be used and the appropriate software and hardware arrangements needed to support interaction with these devices. We illustrate this research challenge with two examples of specific issues to be addressed: − Group interaction with embedded devices – nearly all physically environments may be shared and so embedded devices will be shared too. How can a group of users collectively interact with an embedded device? Equator will develop new interaction techniques to capture and analyse group gestures by extending video tracking techniques. For example, a group of users standing in front of a projected interface might co-ordinate their movements in order to steer a shared viewpoint. In turn, group interaction with embedded devices suggests the need for group embodiments within digital worlds. Equator will explore how group activity can be represented so that other participants can make sense of it. Extending the previous example, the group of participants might be represented by a group avatar in the virtual world.

7 − Embedded devices as boundaries – it is now a common idea that people can use a projected display to create a window into a virtual world. But what do the people in the virtual world see? Equator will consider how embedded devices can act as symmetrical boundaries between the physical and digital so that the occupants of each can see into the other. In the extreme, these boundaries will be made traversable so that people and objects can appear to physically pass from physical to digital and vice versa. For example, a non-solid projection surface could be used to create the illusion that a performer enters a virtual world by physically stepping into its projected image (extending the current idea of immersion in virtual reality to also include leaving one's current physical world behind). Integration across these ideas will involve developing common services and infrastructure for embedded displays that may be used across a range of advanced projection and immersive facilities within the project, including UCL’s CAVE.

The information appliances challenge What forms of information appliance will emerge and how do we support their development?

Computers have traditionally been general purpose devices, capable of running a wide range of applications on a single hardware platform. This has led to a general division between the development of hardware and software that has only recently been challenged by ideas such as information appliances. Information appliances are dedicated computational artefacts that fulfil specific purposes, rather than being general-purpose devices. They combine purpose-designed physical objects, hardware, software and communications functionality into a single artefact. Information appliances typically aim to support aspects of everyday life seldom addressed by conventional computers, investigating new aesthetics and new functionality in support of a wider range of personal and social values and require a synthesis of art and design, technology and an understanding of potential use. Whereas some embedded devices such as projected displays are now commercially available, information appliances generally are not. A key aspect of this research challenge will therefore be to fabricate a number of information appliances for the experience projects and to develop an infrastructure to make their fabrication easier as Equator progresses. In other words, this research challenge will provide core infrastructure for the other challenges and experience projects. In addition, the information appliances research challenge will define a long-term research agenda for information appliances. The following are examples of the kinds of issues to be addressed. − New roles for digital technologies - As digital technology moves out of centralised computers, it can play many new roles in people's lives. So far, these are dominated to a great degree by applications for work or entertainment. We will use design investigations and user studies to explore new roles that technology might play in people's lives through the design of new forms of information appliance. − Common toolkits for dedicated appliances – although each information appliance is designed to be dedicated to a specific purpose, it is necessary to consider to what extent they can share a common infrastructure. How can we establish a common base of hardware and software to support the rapid design and prototyping of information appliances while still allowing each appliance to be purpose designed and built. − Context adaptive appliances – information appliances may be moved between different contexts or the details of their current context may change. We will explore techniques that allow potentially highly specialised information appliances to adapt themselves to changes in context.

The wearable devices challenge What form will wearable computers take and how will users interact with them ?

Wearable devices are display and interaction technologies that are worn by users and that are therefore carried about with them as they move from place to place. They provide users with continual and open access to digital worlds wherever they are and may also support activities outside of traditional office environments, such as in the open-air or the home. Like information appliances, wearable technologies are currently at a very early stage of development, with one of the main centres for this research in the UK being Bristol. Consequently, much of this research challenge will be concerned with the core integration of wearable hardware, software and mobile networking so as to provide a resource of wearable computers for experience projects. A core activity will be the continual redesign of wearable prototypes so that they are convenient to wear and robust enough to use for prolonged periods of time. In addition to providing a common resource for other projects, the wearable devices challenge will address the following issues: − Tailoring wearables for different uses – we will consider the design and tailoring of wearables for disparate communities including children, emergency service workers and the elderly as well as the development of applications to promote different forms of interaction at different distances and across different communities. − Communication among wearables – we will explore how people using mobile devices and wearable computers can communicate with one another in social settings. Current wearable computers hardly communicate (with one exception being multi-user games on Nokia phones), but in a situation where the wearable computer is a

8 commodity object, wearables will interact all the time. We will therefore establish concepts and models to support a community of wearables. − Interacting with wearables – there are as yet, no satisfactory models and techniques for interacting with wearables. We will explore usability issues such as comfort and awkwardness. We will also consider the importance of the visibility of interactions with wearables to others. To what extent can other people make sense of a user’s interaction with their wearable and how can we design this interaction so that it is socially appropriate and does not make the wearer appear foolish?

The Equator experience projects Equator’s research is informed, demonstrated and evaluated through experience projects that are clustered into three themes. Each theme focuses on a different aspect of everyday life and allows us to directly engage different groups of users. This section provides an overview of these themes and identifies some initial experience projects for each. The workplan description later on describes these projects in more detail.

The exploring and understanding theme How can we combine physical and digital worlds to enhance the comprehension of both? Making sense of the world, finding our way through large and complex environments, and locating other people, artefacts and information are familiar problems, whether we are in a physical city or a digital environment such as the World Wide Web or a virtual environment. These tasks may become unmanageable as physical and digital worlds increasingly overlap. People may become lost in a mix of information, may be unable to make sense of the information that is available to them, and may be unable to understand the complex relationships between the physical and digital. On the other hand, we propose that, with careful design, the physical and digital might actually enhance one another, making it easier to understand and to navigate both. New kinds of public and wearable display might introduce digital information into a physical city to support its occupants in wayfinding and exploring. Virtual environments might enable people to plan and rehearse travel in advance of visiting a physical location. Digital and physical information might be overlaid in libraries, museums and galleries to help visitors access their contents. On the other hand, new physical interfaces might improve interaction and navigation within these environments. Three of our initial experience projects are clustered within this theme. − Adaptive information in the physical city – this project will develop and evaluate applications to enhance wayfinding within a physical city, exploiting a combination of wearable displays and adaptive public signposts (interactive shared public displays). These will be deployed within the city of Lancaster over the existing ‘Wireless Lancaster’ mobile networking infrastructure and will be evaluated through use by the general public. − Extending the digital city – this project will enhance ‘Virtual London’, a large 3D model of London implemented in a collaborative virtual environment that is available at UCL, with live information from the physical world. Evaluations of this enhanced model will focus on whether the general public can effectively use the virtual model to rehearse physical travel and to find out about conditions in the real London. − Linking artefacts to electronic collections – this project will develop a combined physical and digital environment to access collections of documents and other artefacts. Evaluation of this environment will focus on how effectively physical objects can be linked to their virtual equivalents, and how digital information can be displayed as part of a physical museum and physical artefacts can be made available within a virtual museum. This project will build on on-going work at Southampton and Glasgow with local museums and collections.

The playing and learning theme How can we combine physical and digital worlds to enhance creativity, participation and learning? Creativity and playfulness are vital human qualities and are closely coupled with learning. Information technology is playing an increasing role in creativity, play and learning in areas as diverse as the arts, entertainment, toys and education. Equator will explore how these may be enhanced by new technologies that link physical and digital worlds. It will seek to enhance creativity, by stimulating the imagination and providing new forms of self-expression. It will seek to increase participation and interactivity in shared cultural events, for example, by allowing people to play roles within online performances and television shows. It will also explore how new creative and playful technologies can enhance learning. Equator will target this research at both adults and children and will involve end users in the research process through the staging of public performances and events and through working is schools. Two of our initial experience projects are clustered within this theme. − Digital toys and collaborative playgrounds – this project will focus on how to enhance play and learning through new forms of digital toy that link physical toys to virtual environments. This will involve embedding computing functionality within physical toys in order to increase their interactivity and expressive potential (examples of information appliances), and then situating these within environments where groups of children could use them to interact with virtual worlds as part of play and learning. Evaluation will involve the technologies being used by

9 children in schools in Sussex and Nottingham, extending on-going collaborations in the KidStory and Puppet projects within the European i3 programme. − Performing and participating in physical and digital stories – this project will explore how performers and the public can engage one another in new kinds of performance that span physical and digital worlds. In the first phase, this will involve creating digital theatres where the public can participate in new kinds of online performance, building on previous research into inhabited television at Nottingham. In the second phase, we will use mobile devices to enable the public to participate in extended performances that are interleaved with their everyday activities in the physical world. For example, people will be able to become involved in stories and events in a digital world that appear to unfold around them as the move through a physical city. Both phases will involve evaluation through the staging of actual public performances.

The working and living theme How can we combine physical and digital worlds to enhance work and home life and to help people manage the relationship between them? The digital world is increasingly present in our everyday lives, both at work and at home. Furthermore, the spread of ubiquitous and mobile computing and communication devices means that home, work and leisure activities are interwoven in ever more complex ways. People can engage in all of these activities through digital means, no matter whether they are at home, at work, at play, or on the move. Equator will explore how the merging of physical and digital worlds can enhance work and home life, can allow them to be layered in new ways, and can help manage the transitions between them. It will focus on the maintenance of social relationships among family, friends and colleagues and across generations. It will also consider how the physical and digital can act as focus and periphery for one another so that people can focus on one facet of their lives while maintaining a background awareness of others. For example, people might maintain a peripheral awareness of loved ones while working, or keep track of unfolding events from work while at home. Equally, we must recognise that people require peace, quiet and solitude at times. This theme will therefore also consider how technologies that merge the physical and digital can be sensitive to people’s need for privacy and isolation when they so choose. A further focus for this theme will be situations in which new technologies have to continuously adapt to new situations and contexts as people move about, with a particular focus on working environments other than the traditional office. Two of our initial experience projects are clustered within this theme. − Domestic information appliances – this project will explore new forms of information appliances to enhance life in domestic settings (building on on-going work at the RCA ad Lancaster). It will combine approaches from art and design with ethnographic studies and technical development to construct and evaluate technologies that offer a sense of comfort and well being within the home. − Working in the open – this project will consider situations where a number of users need to co-operate and co- ordinate activities in potentially hazardous environments (building on on-going collaborations with the emergency services and mountain rescue crews at Lancaster). The project will explore how wearable displays and mobile devices might make digital information available to those working in such environments and how information visualisations might assist controllers in monitoring the state of complex situations. We will construct focused integrated demonstrators within each theme. These will draw clusters of individual projects together to create a major public demonstration. They will be staggered so that there is approximately one such demonstration each year (see the workplan). The three initial integrated demonstrators are: An integrated city that combines the physical city, digital city and artefact collections projects into a single public demonstration that allows members of the public to access physical and virtual cityscapes and collections in order to explore the relationships between them. A kids theatre that combines the digital toys and collaborative playgrounds project with the performing and participating in physical and digital stories project to create a public demonstration where children can interact with digital toys, and digital and physical performers in a virtual puppet theatre. A day in the life that combines the domestic information appliance and working in the open projects. One possible theme would be to follow a day in the life of a volunteer rescue worker as she uses different digital and physical environments to support the various interleaved aspects of her life.

The Equator workplan The Equator workplan describes how the framework of research challenges, experience projects and integration mechanisms will be carried out in practice over the six years of Equator. Please refer to the workplan diagram in figure 2 when reading this section. Core to the workplan is an annual review and re-planning workshop that allows the workplan to be refined in response to external comments and changing demands. The formal annual review takes the form of an Equator conference, followed by a meeting with the review team and one month later by a re-planning meeting.

10 The research challenges run throughout the six years of Equator. All five research challenges will be active in the first three years. We anticipate that the three device research challenges might be re-organised after that, as we come to understand the possible relationships between embedded devices, information appliances and wearable devices in depth, and in the light of results from the experience projects. The research challenges, indeed the whole of Equator, begin with a series of initial IRC challenge workshops. IRC challenge workshops focus on establishing the common mission for the IRC, defining the research challenges in detail and promoting links between different institutions, disciplines and cultures of research. The importance of this linkage and integration is reflected in the decision to devote the first four months of the work of the IRC exclusively to these workshops. Workshops will be organised and run by appropriate senior researchers associated with the IRC. Presentations will focus on the elaboration of the state of the art at different sites, the transfer of technologies, exposure to common resources and fleshing out the core research challenges. Given the rapidly changing nature of technology in this area, we have only defined experience projects for the first four years of Equator. Plans for the last two years will be in place by the mid term scientific review at the end of year three. The experience projects are clustered according to our three themes. Each cluster is launched with a theme workshop. These are staggered so that the whole project can focus on launching one theme at a time and so that the project management team can be fully engaged in the detailed planning of each theme as it comes on-stream. The integrated demonstrator projects within each theme are also staggered so that there is a major integrated demonstration in each of years two, three and four. The Kids Theatre is the first of these and occurs at the end of year two (each of the projects in this theme is split into two phases so that work can continue after this integrated demonstrator). The remainder of this section presents more detailed plans for each of the research challenges and experience projects, identifying timescales and the roles of the different partners.

Research Challenges: activities and roles of each partner IRC challenge workshops (see page 10) Start Month 0 Duration 4 months Site Responsibility Glasgow Hosting the adaptive computing challenge workshop Sussex Hosting the understanding interaction challenge workshop UCL Hosting the embedded devices challenge workshop RCA Hosting the information appliances challenge workshop Bristol Hosting the wearable devices challenge workshop

The understanding interaction challenge(see page 6) Start Month 4 Duration 68 months Site Responsibility Sussex (Coordinator) Extend and integrate cognitive science design and evaluation methods Glasgow Extend and integrate human factors design and evaluation methods Lancaster Extend and integrate ethnographic design and evaluation methods Nottingham Extend and integrate cognitive science design and evaluation methods RCA Extend and integrate design and evaluation methods from art and design UCL Extend and integrate user assessment techniques for interaction in virtual environments

The adaptive computing challenge (see page 6) Start Month 4 Duration 68 months Site Responsibility Lancaster (Coordinator) Interface and communication services – build on previous co-operative interface architectures and work on mobile communications and adaptive architectures Southampton Linking services – build on previous work on hypermedia links to explore hyper-physical links Glasgow Linking services – build on previous work on distributed information models Nottingham Communication services– develop previous work on spatial model of interaction Bristol Interface services – develop interface architecture for wearables UCL Interface services – build on previous work on guaranteed real-time rendering

The embedded devices challenge (see page 7) Start Month 4 Duration 26 months Site Responsibility Nottingham (Coordinator) Video tracking techniques for group interaction and techniques for physically traversing embedded displays UCL Group interaction with immersive projected interfaces (e.g., CAVEs) and symmetrical visibility between physical and virtual worlds Glasgow Techniques for shared interaction with display technologies in public space Sussex Assessment and evaluation techniques for shared public displays

11 The information appliances challenge (see page 8) Start Month 4 Duration 26 months Site Responsibility RCA (Coordinator) Infrastructure for fabricating domestic information appliances Southampton Devices to link physical documents to their digital counterparts Nottingham Infrastructure for new forms of ambient display Lancaster Infrastructure for connecting devices

The wearable devices challenge (see page 8) Start Month 4 Duration 26 months Site Responsibility Bristol (Coordinator) Fabrication of and infrastructure for wearables, building on current cyberjackets Southampton Integrate wearables with multimedia information sources RCA The aestethics and form of wearable devices Sussex Interaction models and human factors for wearable devices.

Framework and toolkit Start Month 30 Duration 6 months All partners to integrate work from the five research challenges to produce a common conceptual framework as well as project wide toolkits to support future experience projects.

Experience Projects: descriptions, milestones and roles of each partner This section provides details of the different experience projects to be undertaken during the first three years of the Equator IRC. These projects provide the principle mechanism of engaging with users and grounding research challenges. To promote and encourage integration across these projects and with the Equator research challenges we have explicitly identified milestones for each of these projects. In the following tables we provide a brief summary of the projects, the role of each partner and an indication of the milestones associated with each.

Theme One: Exploring and Understanding Project Title Adaptive Information in the Physical City Theme Exploring and Understanding Project Start Month 4 Project Duration 32 months This project focuses on ‘embedding’ information in public spaces in the city, and making it accessible to people walking through those spaces. Remotely located devices as well as mobile devices will both display information and be used for visitors’ expression of interests and activity. These devices will communicate with an information service that will store both descriptive content and the information on visitor activity, and will deliver tailored information to display devices. Our intention is to offer access to historical and cultural information about the city, but also information about other people’s paths and explorations. An individual would have information displayed to them not only on the basis of their current location but also the route they took to get there, the information they have shown interest in along the way, and how this combined activity relates to that of other earlier visitors. Curators and designers would initially provide information about the buildings, locations and artefacts in these public spaces, but gradually visitors’ implicitly and explicitly expressed interests would become apparent, enriching the resources for exploration and understanding of the city. We wish to explore placing both factual ‘guided tour’ information in the system as well as more personal and even fictional narratives, so as to offer curators, designers and visitors significant breadth of expression of the character and history of the city. This project will collaborate with Lancaster Tourism and the Lighthouse Trust as user groups (see letters 18, 29) . The project will make use of the Wireless Lancaster facility and the Multimedia Server at Southampton. Site Responsibility Glasgow (Coordinator) Development and assessment of adaptive information systems Bristol Provision of wearable guides and devices Southampton Integration of hypermedia information service Lancaster Network services and communications infrastructure Milestones Month 12 Initial prototype for use in a public setting Month 24 Refined version for use across museums, and initial adaptive information access system offered Equator-wide Month 36 Completed city-wide public trials, and developed information access system

12 Project Title Building the digital city Theme Exploring and Understanding Project Start Month 4 Project Duration 32 months This project is a natural complement to the work on the physical city in that it will consider the setting where people in a virtual representation of a large scale urban environment are looking out to the physical world: planning their interactions with the physical world, enjoying the virtual representation of the physical world for its own sake, or inhabiting the virtual because it is impractical to visit the real. The vision here is of a living virtual urban environment with dynamic display and interaction dependent on live feeds from the corresponding real environment. It will take as its starting point previous work at UCL on the construction of a model of much of London. The model is, of course, that of a real city. However, the model is currently static, representing the buildings and roads only. It does not represent the life of the city, the transport situation, the weather, and facilities at any moment in time. The goal is to make the model persistent, to take real-time feeds and update the model to reflect the changing state of the environment. A further goal is to allow people immersed in the model to carry out functions that are relevant to their activities in every day life: planning a journey, visiting a hotel and inspecting its facilities, meeting other people in virtual situ prior to really meeting them there. The project will collaborate with Virtual Presence and NCR as user groups (see letters 31,23) The project will make use of UCL’s CAVE and Nottingham’s Mixed Reality Laboratory. Site Responsibility UCL (Coordinator) Development of and presentation of the digital city Southampton Integration with multimedia information model Nottingham Use of mixed reality boundaries to link to a real city Milestones Month 12 Initial demonstration of the digital city demonstrating some live media links Month 24 Population of the digital city with a large number of users and agents Month 36 Assessment of the digital city in use and experience through different forms of display

Project Title Linking artefacts to electronic collections Theme Exploring and Understanding Project Start Month 4 Project Duration 32 Months This project focuses directly on how information is linked between the real and the digital and seeks to extend existing hyperlink techniques to consider the development of ‘hyper-physical links’ that allow richer forms of link to be supported. These links may include activity and content based links, links that emerge through a sense of agency, links that decay over a period of time and most critically links that span the digital to the physical. While some of these fundamental concepts and theories will be developed within the core research challenges this project will seek to explore these in use through the development of a set of supporting facilities for an artefact based collection. The work will explore the ways in which handheld devices, inexpensive sensors (for example, passive RF transducers) and mobile communication services may be used to manage an artefact collection so that it is equally accessible from both the physical and the digital. The lessons of linkage developed during this project will inform both the further refinement of the core information model and the development of a linking demonstrator. The results of this work will feed into the devices challenges. The project will collaborate with the Turing collection, the Glasgow Science Centre and the Hunterian Museums as user groups. (see letters 8,12 ) The project will make use of the multimedia server at Southampton. Site Responsibility Southampton (Coordinator) Development of core information model and presentation techniques Lancaster Assessment and exploration of interaction techniques and approaches. RCA Information appliances to access artefact collections Milestones Month 12 Initial prototype for use in to browse the artefact collection from both the real and the physical Month 24 Refined version allowing different forms of linkage between the physical to digital. Month 36 Completed trials, and developed of refined information models

13 Theme Two: Playing and Learning Project Title Digital toys and collaborative playgrounds Theme Playing and learning Project Start Month 5 Project Duration 4 Years (Two phases) In this project we shall explore and extend current forms of playful interaction by developing a combination of digital toys and collaborative playgrounds. Currently, there is a big difference between how children play in their everyday life (using physical toys and props) and what they are able to do with the kinds of virtual playing currently supported by computer-based technology. In the former they are typically very creative using much imagination. In the latter (e.g. video games) they tend to be highly focused, solving complex problems but using very little imagination. Our overarching goal is to combine the computational power of digital technology with the pleasure and imagination engendered by interacting with physical toys. An example is the design of physical musical instruments that when interacted with create synasthesic digital representations (e.g. animations) in a virtual space. In so doing, we aim to enable children (and adults) to create new forms of expression. To achieve this we will draw on experience at Sussex in developing technology for children and combine this with the wearables infrastructure development at Bristol and the mixed reality research and facilities at Nottingham. A long term goal is to develop the digital toys so that they can be played with collaboratively in shared physical/virtual playgrounds by groups of children. The project will collaborate with the Glasgow Science Centre and will extend an on-going collaboration with the Albany Infant School in Nottingham as part of the KidStory project (EU i3 programme). (Letters 8) The project will make use of the wearables facility at Bristol and the Mixed Reality Laboratory at Nottingham. Site Responsibility Sussex (Coordinator) Develop models of physical/virtual play. Design and evaluate digital toy prototypes Bristol Development of cyberjackets for children and virtual playground infrastructure Nottingham Development of virtual environments software to be incorporated into the collaborative playgrounds. User evaluation studies Milestones Month 12 User studies of existing digital toys and how to extend current play practice. Month 18 Development of digital toy prototypes. Conceptual design of collaborative digital toys. Month 36 Implementation of wearables and other sensor-base infrastructures to support collaborative digital toys in virtual and physical playgrounds. Month 48 Evaluation of digital toys and collaborative playground in the community (e.g. schools, playgroups, museums)

Project Title Performance and Tales of the City Theme Playing and learning Project Start Month 5 Project Duration 4 Years (Two phases) Virtual reality has been widely used in art, performance and entertainment. Examples range from games, to interactive art installations; and from the use of virtual studios to create conventional television programmes, to the use of distributed virtual reality to support public participation in online TV shows. This project will explore how mixed reality technologies raise further possibilities for performance. It will focus on whether mixed reality technologies can be used to engage performers and audiences who are online in a digital theatre with those who are located in a more traditional physical theatre. A key focus of the project will be the use of a traversable interface to allow performers and audience members to seemingly step through a projected display that joins the physical and digital theatres. The second phase of this project will follow on from performing in physical and digital theatres by considering how mixed reality technology can be used to engage members of the public and performers in new kinds of narratives. However, as opposed to using a projected display to link physical and digital theatres, the approach will be to use personal technologies such as wearable computers to introduce fictional characters and events into the experience of a real city. Participants may become involved in an on-going story as they travel about a physical city. Digital characters and events that only they and their fellow participants can perceive may be overlaid on the backdrop of everyday events. Distant performers will monitor the experience and respond to the their actions with yet more events, slowly involving them within an emerging plot. The aim of this project will be to explore the role of persistent interfaces and mobile technologies in creating new forms of narrative. The project will collaborate with Nottingham City Council (the NOW arts festival), Illuminations TV and BT as user groups. (see letters 4,2) The project will make use of the Mixed Reality Laboratory at Nottingham and the wearables facility at Bristol. Site Responsibility Nottingham (Coordinator) Design and implementation using the MASSIVE platform as well as new techniques for traversable interfaces UCL Development of virtual performers and linking these with real performers RCA Collaboration with content developers and design of potential scenarios Bristol Implementation of wearable interfaces for performance and city visitors Milestones Month 9 Initial content and technical design of performance Month 12 Start of public performances Month 27 Initial content and technical design for Tales of the City Month 36 Start of public usage

14 Theme Three: Living and Working Project Title Domestic information appliances Theme Living and Working Project Start Month 6 Project Duration 30 months The goal of this project is to develop new forms of domestic information appliance that support the social and domestic aspects of everyday life. This is seldom addressed by existing digital technologies that currently merely migrate the concepts from the office to the home. One reason for this is that the appropriate concepts have yet to be uncovered for the home. This project will build upon a set of existing ethnographic studies of home life undertaken by Lancaster University and initial design work from the RCA to explore new aesthetics and new functionality in support of a wider range of personal and social values. The project will use a combination of ethnography and design as a primary approach to investigation. Design techniques centred on evoking the imagination of designers as well as partners and users throughout the development process will be complemented by the results of ongoing studies of domestic environments and the assessment of devices in these environments. As a starting point, we plan to explore values such as diversions from task-oriented activities, emotional communications, insight and awe, and community engagement. However, a major component of the project will be to engage in an iterative dialogue with perspective users as we develop product prototypes. Initially, we will build up a portrait of several target populations via their responses to ‘cultural probe’ materials, coupled with ethnographic observations of the process. In a second phase, we will expand the design space via a large number of concept proposals, sketches, and scenarios, and use these materials to further probe the values of our target audience. In the third phase, we focus on creating a number of working product prototypes which themselves will serves as probes as well as objects for evaluation. Many of these will be assessed in a purpose built house of the future that one of our associates (Georgia Tech) is providing access to. The project will collaborate with Phillips Design, Phillips Research and the Georgia Institute of Technology as user groups. (see letters 24,25,10) Site Responsibility RCA (Coordinator) Development of research probes, concept generation, design of prototypes Lancaster Ethnographic input into design-led user studies and Access to network services Glasgow Have developed shared and adaptive displays for the home and city Milestones Month 12 Detailed results of design-led investigation into diverse user groups. Concept sketches and scenarios. Month 24 Concept proposal trials, generating focused feedback and more general target-group information. Refined concept proposals, scenarios, and early prototypes. Month 36 Product prototypes and user feedback.

Project Title Working in the Open Theme Living and Working Project Start Month 6 Project Duration 30 months This project will explore the use of wearables to support work in open spaces. One of the goals of this research is to understand how universal access to information and the mixing of the digital and physical can be exploited to support forms of work that have traditionally not been amenable to the desktop centric view of computing. Previous work on wearables has tended to focus on their use by operators who need hands free access to information. This project will advance the state of the art by considering a situation where a number of users need to cooperate and co-ordinate activities in potentially hazardous environments. The aim of this work is to uncover techniques to allow wearables to be used in an unconscious manner so that the wearer is not distracted from the particulars of the task at hand. As a driving application we will focus on providing support for the emergency services and mountain rescue in particular. The work will be driven by a series of ethnographic studies of the Cumbria mountain rescue team who have agreed to provide access to their work and to test equipment in trials and rehearsals. The assessment and results of this work will feed directly back into the wearables research challenge. This project will collaborate with Langdale and Ambleside Mountain Rescue Team as a user group.(see letter 19) The project will make use of the wearables facility at Bristol. Site Responsibility Bristol (Coordinator) application programs for wearables, and construction of wearable Sussex Evaluation and assessment Glasgow Contextual access to online information through devices. Lancaster Ethnographic study to understanding the interaction between the groups and assessment Milestones Month 12 Initial prototype applications for use Month 24 Refined applications, results of evaluation, generic mechanism input to research actions Month 36 Completed public trials supported by developed platform mechanisms

Management, integration, involvement and dissemination mechanisms In order to develop and manage Equator as a focused IRC, a set of coherent and co-ordinated activities will be managed by an overall technical director. The Equator director will oversee the work of the IRC and be responsible for technical and administrative co-ordination; control of planning, reviews and dissemination activities; and the identification of risks and the planning of correcting actions in contact with an overall management strategy (see below). The Equator director will be Professor Tom Rodden, an experienced manager of large interdisciplinary projects. Professor Rodden directed the COMIC project between 1992 and 1995, an ESPRIT basic research project involving computer scientists and social scientists from twelve partners across eight countries. He is currently directing the ESPRIT i3 eSCAPE project, a collaborative project involving computer scientists, artists and social scientists due to

15 finish in 2000. Lancaster will host the office of the director and provide co-ordination for the IRC. This office will provide the administrative support to implement the mechanisms central to the IRC. These mechanisms provide support for undertaking research, developing and training a research community, and disseminating the results of research. Each partner (including the co-ordinating partner) nominates a project leader who is in charge of the management of the activities associated with their organisation, and acts as the partner’s focal point of contact. The Equator Management Committee will be composed of the project leaders from each site and the Director. It will meet at least twice a year. Its main activities are overall management; progress reviews and early detection of problems during the project; proposing and approving major changes to project plans, in response to unexpected problems or conflicting situations (for all these topics, a voting procedure will be settled during the first meeting); technical co-ordination and exchange between projects. Project leaders will also be responsible for technical progress in those research challenges and experience projects where their site is named as the coordinator. Equator will undertake research in a rapidly evolving area that is subject to continual fluctuation and change. The adventurous nature of the research is reflected in the management process used to direct it. Management will be focused around annual Equator plenary conferences where the aims and objectives of Equator will be subject to significant internal and external review and risk assessment. The general structure of the review process is: • Sending deliverables reflecting the work of the project to reviewers prior to the annual conference. • The presentation of the work of Equator in an open forum across a number of days. • A focused Equator strategy meeting with the Equator review panel where the work of the project is discussed and a plan for the coming years activities presented. The review panel will consist of selected external international experts drawn from our associate programme and will be fully open to the EPSRC. The Equator budget includes funds to support the involvement of these people and the attached letters of support indicate the willingness of major international and industrial research organisations to take part in this process. • An Equator planning meeting one month after the annual review where reports from the Equator review panel are discussed and the activities for the coming year confirmed and the aims and objectives of the IRC re-aligned. The aim is to make this process open and the EPSRC are welcome to be involved in any aspect of the process. This process also seeks to ensure that Equator is flexible to change and that its core goals and mission is kept relevant throughout its lifetime. The importance of the need for responsive and flexible management is also reflected in the project’s approach to resource management. In addition to defining a set of communally managed resources we have divided the overall budget for staff into two distinct portions. • Core IRC staff are funded from a budget allocated to each site for the duration of the IRC. • Responsive IRC staff are funded from a centrally managed budget that is allocated annually at the Equator strategy meeting. This ensures responsiveness to changing needs and circumstances and also lends greater significance to the annual review process.

Integration mechanisms Integration mechanisms support research integration across Equator. Initial IRC challenge workshops during the first four months provide a focused mechanism for educating project staff, exhanging perspectives and defining research challenges and experience projects in full detail. Theme start-up workshops launch several experience projects within a single theme. Tool kit and framework activities integrate the results of the research challenges into common conceptual models and infrastructure at key points in Equator. Integrated demonstrators stage integrated public demonstrations of several experience projects within a theme. Technology camps gather researchers together at a specific technical facility to undertake an intensive exploration of its potential over several days. Equator students will undertake research associated with the work of the IRC. PhD students will be expected to engage with other sites in the IRC, and will normally spend a significant period of time at another research site; Equator sabbaticals will allow members of the different partners to spend time at other partners. These will be open to students, research associates and senior academics, and we would envisage significant use of the summer vacation period for this form of sabbatical.

Involvement and engagement mechanisms A number of mechanisms will support the IRC’s involvement and engagement with other organisations including user groups, industry and other research organisations. These are wrapped up into the industrial associate and international associate schemes. A student intern programme will allow Equator research students to be hosted by associates as will a sabbatical programme for academics and staff. An annual IRC conference involving the user panel and other associate members will be held that is heavily based on presentation of demonstrators and prototypes. The conferences will also be open to the wider research community and will provide a significant milestone for measuring the work of the IRC;

16 An IRC industrial secondment programme will enable staff from associates to spend time at Equator sites. Summer schools will be held to allow research students to meet with more senior academics in the research area, and to provide training in the core research issues.

Dissemination and exploitation mechanisms Additional mechanisms will disseminate Equator’s results to the wider research community and to the public. An IRC book series has been agreed in principle with John Wiley. This book series will seek to raise the international profile of the IRC and will provide a significant and distinctive dissemination mechanism for Equator. The Journal Presence will disseminate Equator results through focused special issues. Broadcast quality TV will be developed to convey results to the general citizen in partnership with Illuminations. The Equator Annual Conference will be an open event and will be widely publicised to encourage open attendance. We shall also continue to use dissemination routes that we have found to be successful in our previous and current research projects. These include journal and conference articles, online demonstrations of interactive software, CD- ROMs, websites, videos, public performances, shows and installations. Shareware licensing of tools and patenting will also be promoted, in accordance with the individual partner’s existing policies. Further investment (e.g. venture capital, corporate sponsorship, university innovation centres, setting up spin-off companies) will be arranged, especially where it is viewed to be mutually beneficial to both the academic and commercial communities. Guidance and IRC policy on these matters will be established at the beginning of the IRC

Partners, expertise, roles and facilities This section summarises the skills of the Equator partners, the personnel involved and their roles. Full details, including selected publications, are provided in Appendix II. Bristol – the Computer Architecture, Machine Learning and the Multimedia groups in the Computer Science Department at Bristol will participate in Equator lead by Prof. David May FRS. Other staff involved are Prof Barry Thomas (Multimedia), Dr Nathan Sidwell (VSE), and Dr Henk Muller (Wearable & Mobile computing). Research programmes of relevance are wearable computing, shared virtual environments, mobile computing, architectures for media appliances, and virtual reality for the nearly blind. Bristol’s main role in Equator will be to carry out fundamental research into wearable devices, including the adaptive computing architecture required to support them. This will involve co-ordinating the Wearable Devices research challenge and making a major contribution to the Adaptive Computing challenge. Bristol will contribute wearable technologies to experience projects in each of the three research themes. Bristol brings a range of facilities to Equator including two Cyberjackets (wearable computers), a multimedia supercomputer, an image and video server, and a VR centre. Glasgow – GIST is an inter-disciplinary research group at the University of Glasgow investigating the design and implementation of interactive systems. Dr. Matthew Chalmers and Prof. Chris Johnson will lead the Equator effort, working along with Dr. Stephen Brewster and Mr. Philip Gray in Computing Science, and Dr. Stephen Draper in Psychology. GIST’s research covers a wide range of topics, including mobile computing applications, non-speech audio and force feedback techniques, information visualisation and retrieval, highly reconfigurable user interface architectures, evaluation methods and tools. GIST is strongly linked with both the SHEFC-funded Revelation Project and the 3DMatic Faraday Centre. Glasgow will be heavily involved in the Adaptive Computing challenge with a particular focus on extending information models. They will also work on the Embedded Devices challenge where they will focus on new forms of display technology for public spaces. The Glasgow team will lead the Adaptive Information in the Physical City experience project. They bring a number of user groups to Equator including the Hunterian Museum, the Glasgow Science Centre and the Lighthouse. Glasgow brings a range of facilities to Equator including a whole body scanner. Lancaster – the work on CSCW and mobile systems at Lancaster University involves researchers from Computer Science, Sociology and Communications Engineering. This includes expertise in virtual environments, the construction of co-operative systems informed from studies of work and mobile computing. Local site management will be undertaken by Prof. John Hughes (Sociology) and Dr Nigel Davies (Mobile Systems) supported by Prof. Tom Rodden (CSCW) who will also direct the IRC as a whole. Dr Gareth Smith (Virtual Environments), Dr Adrian Friday (Mobile Infrastructures), Dr John Mariani (Cooperative Information) and Dr Keith Cheverst (Interfaces for Nomadic Devices) will direct specific IRC activities. Lancaster will provide an extensive wireless testbed for mobile communication systems in the city of Lancaster – ‘Wireless Lancaster’. In co-ordinating Equator, Lancaster will draw upon its record in leading cross–European, inter–institutional inter–disciplinary research projects. Lancaster will work on the Adaptive Computing challenge in the area of adaptive interface architectures and adaptive middleware; on the Understanding Interaction challenge, where they will seek to integrate ethnographic techniques with other methods; and on the Information Appliances challenge, where they will explore the further development of mobile device used in previous projects such as GUIDE. They will contribute to experience projects in the working

17 and living and exploring and understanding themes. Lancaster brings the Wireless Lancaster mobile network infrastructure to Equator. Nottingham – the Mixed Reality Laboratory (MRL) at the University of Nottingham involves researchers from Computer Science, Psychology and Engineering. It is led by Prof. Steve Benford and also includes Dr Chris Greenhalgh, Dr Claire O’Malley, Dr Tony Pridmore and Professor John Wilson. The expertise of the MRL is in middleware for distributed virtual environments, mixed reality boundaries, entertainment and education applications and the evaluation of collaborative technologies. Nottingham is currently researching new technologies for performance, art and entertainment, exploring collaborative storytelling technologies for young children and new techniques to support large-scale online TV shows. The projects include a strong commitment to staging public events and performances, working closely with arts promoters and television companies. The MRL has been awarded a JREI grant of over £1 million for mixed reality equipment, including projection, tracking, wireless and mobile technologies and also has access to the University’s Reality Centre. Nottingham will lead the research challenge on Embedded Devices where they will extend their previous work on mixed reality boundaries to address the issue of traversable interface between real and virtual worlds. They will also lead the Performance and Tales of the City experience project. They will contribute to the Adaptive Computing challenge by extending their previous work on session management and quality of service management for collaborative virtual environments to also consider communication between physical objects. Finally, they will contribute a cognitive science perspective to the Understanding Interaction challenge. Nottingham brings the Mixed Reality Laboratory and its Reality Centre facilities to Equator. Royal College of Art – The RCA CRD Research Studio draws on a wide range of art and design disciplines as well as psychology and engineering in investigating the design of new IT devices. The studio is also currently involved in two ESPRIT projects: PRESENCE, which has two themes: Connected Community -in this case the elderly - and ‘interfaces in the territory’ – public places and spaces. The second project, where the main partners are Philips Research Laboratory, Helsinki Telephone Company and Infogrammes (a large French games company) looks at new services for mobile phones. Prof. Gillian Crampton Smith heads the Department; groups are led by Dr Anthony Dunne (Product Design) Dr Bill Gaver (Psychology), Fiona Raby (Architecture) and Richard Brown (Fine Art).The involvement of RCA will be managed by Dr Bill Gaver who has extensive experience in developing systems that space the physical and digital. The RCA will contribute to the Understanding Interaction method with design and evaluation approaches such as the use of cultural probes to explore people’s aspirations and to evoke designers’ imaginations. They will lead the Information Appliances challenge with new techniques for creating purpose built physical artefacts with embedded computing functionality. They will lead the Domestic Information Appliances experience project and will also contribute to the Performance and Tales of the City and Linking Artefacts to Electronic Collections projects. The RCA will contribute facilities for manufacturing information appliances to Equator. Southampton – the Multimedia Research Group is a major research group of over 50 people in the Department of Electronics and Computer Science with an international reputation in the area of open hypermedia and its application to distributed multimedia systems. The group is led by Prof. Wendy Hall, an EPSRC Senior Fellow, and includes Dr David De Roure, Dr Paul Lewis, Dr Kirk Martinez, and Dr Les Carr as well as Professor Stevan Harnad, a cognitive scientist who moved recently from Psychology. Previous research has resulted in the open hypermedia system Microcosm. This system has been exploited commercially and is the subject of patents. The group is also involved in a number of digital library projects including the JISC funded MALIBU project, and the JISC/NSF funded E-print Archive project. Southampton will contribute to the Adaptive Computing challenge in the area of new link services that support hyper- physical links between physical and digital objects. They will also explore how such links can be embodied in physical artefacts in the Information Appliances challenge. Southampton will lead the Linking Artefacts to Electronic Collections experience project. Southampton will contribute their JREI funded multimedia database server to support various experience projects. Sussex – the School of Cognitive and Computing Sciences (COGS), at Sussex University, has an international reputation as a centre for multi-disciplinary research. The Interactive Media and Virtual Environment Lab (IMVEL) will be involved in Equator. The Sussex involvement will be led by Dr Yvonne Rogers (HCI) and will involve Dr Mike Scaife (Psychology), Dr Ian Wakeman (Computer Science), Dr Phil Husbands (Neuroscience and Alife) and Dr Rose Luckin (AI and Education). The group are working on a number of interdisciplinary projects in the area of digital technology, including the use and design of innovative web solutions for e-commerce, information visualisation, interactive multimedia applications for education, collaborative work and virtual environments. Sussex will lead the Understanding Interaction research challenge and will play a key role in integrating design and evaluation methods from different disciplines. They will also contribute their experience with digital toys to the Information Appliances research challenge and will lead the Digital Toys and Collaborative Playgrounds experience project. UCL –is world-renowned for its work on virtual environments, real-time graphics and networking and multimedia systems. The Virtual Environments and Computer Graphics (VECG) group is led by Prof. Mel Slater, a senior EPSRC

18 fellow. It also includes Dr Yiorgos Chrysanthou and Dr anthony Steed. Its expertise is in shared virtual environments, real-time graphics for large scale VEs, tele-presence studies, including applications to rehearsals in the context of stage acting and psychotherapy. The VECG group has been awarded a JREI grant amounting to £849,330 for an immersive CAVE system. The Advanced Internet Research group is led by Prof. Jon Crowcroft. Its expertise is in the protocols and structures needed for the efficient use of many networks including Arpanet, packet satellite networks, X.25, the Cambridge Ring, ATM links, and SuperJANET. UCL will contribute to the Adaptive Computing Challenge in the areas of interface services, extending Prof. Slater’s work on guaranteed real-time rendering for virtual environments, and communication services, and extending Prof Crowcroft’s work on multimedia networking. UCL will also work with its five-sided CAVE system as part of the Embedded Devices challenge as well as in the Building the Digital City project that it will lead. UCL will bring its CAVE system, two immersive VR systems, and a whole body scanner to Equator.

Equator Resources A full per partner breakdown of the resources requested for the IRC are provided in Appendix III. The resources requested reflect the size of the Equator IRC (eight partners), the ambition of its goals, the nature of its approach to research, and the need for flexible and focused managed.

Coordination and Management Budget The work of the IRC is managed by a Director responsible for the overall technical direction of the IRC, developing and fostering links between projects and sites and ensuring scientific progress of the IRC. Prof. Tom Rodden will be appointed as Director of the IRC and funds are sought for 50% of his time. Lancaster University is demonstrating its commitment to the IRC by freeing Prof. Rodden from other duties to ensure he can be exclusively devoted to the IRC for its duration. Funding is also sought for a Coordinator who will support the Director and be responsible for the organisational arrangement of the IRC, liaison with press and industry and the arrangement of IRC events and workshops. Support is also sought for a central server facility to act as a repository of research results.

Shared Resources Budget Equator seeks to ensure a critical mass of researchers through a closely managed research programme. This is reflected in the communal management of a set of shared resources. This approach allows the project to be responsive to changing demands during the six years of the project. This includes a responsive pool of research staff (funding for 8RAs) to be allocated to meet specific demands (e.g Demos and Integrators) or to be reallocated to reprofile activities subject to reviews. This flexibility also allows us to reallocate effort over the duration of the IRC. For example, our current allocation of effort for the first year of the IRC anticipates that increased effort will be needed for the integration activities in the second year and beyond. Our shared budget also includes resources for novel devices and the construction of new devices by the IRC reflecting the rapid changes developments in technology in this area. We also seek support for the expenses associated with the annual review central to the management of the IRC and for Equator dissemination activities.

Site Budgets Each site has a budget for three dedicated research staff. The importance of the IRC to each site member is reflected in the allocation of named senior staff to Equator to ensure that the IRC can start as quickly and as smoothly as possible and that high quality research can be undertaken from the outset of the project. Travel and subsistence support is requested for both internal IRC travel and external dissemination activities. We also request an equipment budget for each site to allow the purchase of machines to support researchers and emerging interactive devices.

19 Initial Allocation of Staff The funding model adopted by Equator makes a separation between dedicated site staff and shared responsive staff. This separation allows us to target staff as we see fit through the lifetime of the IRC. The currently proposed allocation of staff across the research themes and challenges for the first year of the project is shown in the table below. Notice that those co-ordinating a project or challenge have a dedicated senior staff member assigned to it while other staff are involved in more than one project or challenge. This allocation of staff will be open for review throughout the lifetime of the project based on the annual review process and the strategic re-planning of the project each year.

Experience Projects Lanc Nott UCL RCA South Sussex Glasgow Bristol TOTAL Theme 1 : Exploring and Understanding Adaptive Information in the Physical City 0.5 0.5 1 0.5 2.5 Building the Digital City 0.5 1 0.5 2 Linking artefacts to electronic collections 0.5 0.5 12

Theme 2 : Playing and Learning Digital toys & collaborative playgrounds 0.5 1 0.5 2 Performances & Tales of the City 1 0.5 0.5 0.5 2.5

Theme 3: Working and Living Domestic Information Appliances 0.5 1 0.5 2 Working in the Open 0.5 0.5 0.5 12.5

Research Challenges

Device and Appliances Wearable Devices 0.5 0.5 0.5 12.5 Information Appliances 0.5 0.5 1 0.5 2.5 Embedded Devices 0.5 1 0.5 0.5 2.5

Core Challenges Adaptive Computing 1 0.5 0.5 0.5 0.5 0.5 3.5 Understanding Interaction 0.5 0.5 0.5 0.5 1 0.5 3.5

Site Total 4 4 3.5 4 3.5 3.5 3.5 4 30

Cordinator Involved Table 1: The first year allocation of staff across challenges and projects

20 The Equator Workplan

Year One Year Two Year Three Year Four Year Five Year Six

Experience Projects Adaptive Information in the Physical City

Exploring and Understanding Building the Digital City Integrated City Linking artefacts to electronic collections Theme Workshop

Digital Toys Digital Playground Kids Playing and Learning Theatre Performance Tales of the City Theme Workshop

Domestic Information Appliances Home Living and Working Demo Working in the Open

Theme Workshop

Research Challenges Wearable Devices

Information Appliances New arrangements of Devices IRC Framework Framework Embedded Devices Challenge & & Workshops Toolkit Toolkit Adaptive computing for smart environments Adaptive computing for smart environments

Understanding Interaction Understanding Interactivity

Annual Review Annual Review Annual Review Annual Review Annual Review Annual Review

Annual equator Review Experience Project Equator Research Challenge Public/Demo Project

Figure 2: The Equator IRC activities