Flintham, Martin (2009) Supporting mobile mixed-reality experiences. PhD thesis, University of Nottingham. Access from the University of Nottingham repository: http://eprints.nottingham.ac.uk/10632/1/thesis_251108_corrections_submitted_printed.pdf Copyright and reuse: The Nottingham ePrints service makes this work by researchers of the University of Nottingham available open access under the following conditions. · Copyright and all moral rights to the version of the paper presented here belong to the individual author(s) and/or other copyright owners. · To the extent reasonable and practicable the material made available in Nottingham ePrints has been checked for eligibility before being made available. · Copies of full items can be used for personal research or study, educational, or not- for-profit purposes without prior permission or charge provided that the authors, title and full bibliographic details are credited, a hyperlink and/or URL is given for the original metadata page and the content is not changed in any way. · Quotations or similar reproductions must be sufficiently acknowledged. Please see our full end user licence at: http://eprints.nottingham.ac.uk/end_user_agreement.pdf A note on versions: The version presented here may differ from the published version or from the version of record. If you wish to cite this item you are advised to consult the publisher’s version. Please see the repository url above for details on accessing the published version and note that access may require a subscription. For more information, please contact [email protected] SUPPORTING MOBILE MIXED-REALITY EXPERIENCES by Martin Flintham, BSc, MSc Thesis submitted to the University of Nottingham for the degree of Doctor of Philosophy December 2007 Abstract Mobile mixed-reality experiences mix physical and digital spaces, enabling participants to simultaneously inhabit a shared environment online and on the streets. These experiences take the form of games, educational applications and new forms of performance and art, and engender new opportunities for interaction, collaboration and play. As mobile mixed-reality experiences move out of the laboratory and into more public settings they raise new challenges concerning how to support these experiences in the wild. This thesis argues that mobile mixed-reality experiences in which artists retain creative control over the content and operation of each experience, particularly those that are deployed as theatrical performances, require dedicated support for content authoring and reactive orchestration tools and paradigms in order to be successfully and robustly operated in public settings. These requirements are examined in detail, drawing on the experience of supporting four publicly toured mobile mixed-reality experiences; Can You See Me Now?, Uncle Roy All Around You, I Like Frank in Adelaide and Savannah, which have provided a platform to practically develop, refine and evaluate new solutions to answer these challenges in the face of presenting the experiences to many thousands of participants over a four year period. This thesis presents two significant supporting frameworks. The ColourMaps system enables designers to author location-based content by directly colouring over maps; providing a simple, familiar and yet highly flexible approach to matching location-triggers to complex physical spaces. It provides support for multiple and specialised content layers, and the ability to configure and manage other aspects of an experience, including filtering inaccurate position data and underpinning orchestration tools. Second, the Orchestration framework supports the day-to-day operation of public experiences; providing dedicated control-room tools for monitoring that reveal the content landscape and historical events, intervention and improvisation techniques for steering and shaping each participant‟s experience as it unfolds both physically and virtually, and processes to manage a constant flow of participants. i Selected Published Works At the time of writing, sections of this thesis have appeared in several peer- reviewed publications as long papers, a selection of which are referenced here: Flintham, M., R. Anastasi, S. Benford, A. Drozd, J. Mathrick, D. Rowland, N. Tandavanitj, M. Adams, J. Row-Farr and A. Oldroyd (2003) Uncle Roy All Around You: Mixing Games and Theatre on the City Streets. In Proceedings of Level Up: The First International Conference of the Digital Games Research Association (DIGRA), Utrecht, Netherlands, November 2003. Flintham, M., S. Benford, R. Anastasi, T. Hemmings, A. Crabtree, C. Greenhalgh, N. Tandavanitj, M. Adams and J. Row-Farr (2003): Where on-line meets on the streets: experiences with mobile mixed reality games. In Proceedings of the SIGCHI conference on Human factors in computing systems, Fort Lauderdale, Florida, April 2003. Flintham, M. (2005) Painting the town red: configuring location-based games by colouring maps. In Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology, Valencia, Spain, June 2005. Benford, S., R. Anastasi, M. Flintham, C. Greenhalgh, N. Tandavanitj, M. Adams and J. Row- Farr (2003) Coping with uncertainty in a location-based game. In Pervasive Computing, IEEE 2(3): pp 34-41. Benford, S., A. Crabtree, M. Flintham, D. Rowland, B. Gaver, M. Adams, J. Row-Farr, N. Tandavanitj, A. Oldroyd and J. Sutton (2004): Provoking Reflection Through Artistic Games. In Proceedings of the 2004 CHI Conference on Human Factors in Computing Systems (Workshop 19. Social Learning through Gaming), Vienna, Austria, April 2004. Benford, S., D. Rowland, M. Flintham, R. Hull, J. Reid, J. Morrison, K. Facer and B. Clayton (2004) Savannah: Designing a location-based game simulating lion behaviour. In Proceedings of the 2004 ACM SIGCHI International Conference on Advances in computer entertainment technology, Singapore, June 2004. Benford, S., A. Crabtree, M. Flintham, A. Drozd, R. Anastasi, M. Paxton, N. Tandavanitj, M. Adams and J. Row-Farr (2006) Can you see me now? In ACM Transactions on Computer- Human Interaction (TOCHI), 13(1): pp 100-133. ii Acknowledgements I gratefully acknowledge the support of the Engineering and Physical Sciences Research Council (EPSRC) through their support of the Equator project. I would like to particularly thank my supervisor, Professor Steve Benford, and my collaborators and friends at Blast Theory; Matt Adams, Ju Row Farr and the inimitable Nick Tandavanitj. Without their vision and passion, and most importantly the faith they put in me to „get the job done‟, none of this work would have been possible. I would also like to thank the supporting cast I have had the opportunity to work with. Particularly, the performers; Paul Dungworth, Dicky Eton, Sheila Ghelani, Jamie Iddon, Caitlin Newton-Broad and Hannah Talbot, and the developers; Deyan Raykov and Jon Sutton. I could not have asked for a more dedicated and professional group of people to work with. Thanks to my long-suffering colleagues and friends at the Mixed-Reality Lab past and present for putting up with me for so long, and their unwavering support both academically and socially; Alan Chamberlain, Adam Drozd, Chris Greenhalgh, Jan Humble, Pilar Herrero, Alex Irune, Dave Lloyd, Jim Purbrick, Stuart Reeves, Duncan Rowland, Holger Schnädelbach, Ian Taylor and far too many others to mention. Finally I would like to thank my parents, who convinced me that I should do this in the first place, my brother, and my friends – especially Michelle for keeping me sane in spite of this occasionally being “a little bit rubbish”. iii Table of Contents ABSTRACT ............................................................................................................................................... I SELECTED PUBLISHED WORKS ................................................................................................. II ACKNOWLEDGEMENTS ............................................................................................................... III TABLE OF CONTENTS .....................................................................................................................IV LIST OF FIGURES .......................................................................................................................... VIII LIST OF TABLES .................................................................................................................................XI LIST OF TABLES .................................................................................................................................XI 1 INTRODUCTION ........................................................................................................................ 1 1.1 INTRODUCTION ...................................................................................................................... 1 1.2 MOBILE MIXED-REALITY EXPERIENCES ........................................................................... 2 1.3 METHODOLOGY ..................................................................................................................... 4 1.4 MOTIVATION AND FOCUS .................................................................................................... 6 1.5 THESIS STRUCTURE............................................................................................................... 9 2 A REVIEW OF MOBILE MIXED-REALITY EXPERIENCES .................................. 12 2.1 INTRODUCTION ...................................................................................................................