UNIVERSITY OF OSLO Department of informatics A Conceptual Framework for Development Iterations of Digital, Interactive Art; System Response Patterns and Multi-Narrative Temporal Data based Presentation Master thesis 60 credits Martin Havnør 31. January 2006 - ii - - iii - Copyright @ 2006 by Martin Havnør - iv - - v - Abstract Digital, interactive art (DIA) installations often consist of both live and pre-recorded audiovisual material in their presentation of non-linear narrative experiences. Designing and implementing the software supporting the necessary manipulation of such media is complex. Installations of a broad experimental scope require qualifications in a number of fields besides computer science. This further complicates the process by adding multi- discipline communication and negotiation. This thesis addresses two problems related to this process; choice, classification and identification of interactivity components, and choice of software platform. A review of existing literature on interactivity theory is used as a base for defining a theoretical framework supporting part of the development process. The framework consists of a software design pattern based on the Model View Controller, and a set of empirically based “System Response Patterns,” a concept contributed by this thesis. Specific to DIA installations using temporal presentational data, the framework should help in choosing between different means of communication between audience and installation, and be useful in the development of related software. Additionally the use of Java in this context is thoroughly explored through the development of a case installation with the Java Media Framework. A methodology based on Agile Software Development is employed. - vi - - vii - Acknowledgements I would like to take this opportunity to express my gratitude to the following that have contributed or helped in realizing this thesis: Ole Smørdal, my supervisor whose academic and technical insight has been of tremendous assistance every step of the way. Idunn Sem for valuable input and co-ordination of many Tapet testing processes. Andrew Morrison for valuable input on all art and media related issues. Jo Herstad, my internal supervisor. Jan Arild Dolonen for valuable technical assistance. Even Westvang for shepherding me into the Tapet project. My co-workers from the Tapet project (In alphabetical order): Siri Gjønnum Per Christian Larsen Andrew Morrison Inger Reidun Olsen Knut Qvale Idunn Sem Ole Smørdal And finally Intermedia at the University of Oslo for providing me with the opportunity to write this thesis and work on the Tapet project. - viii - - ix - Table of contents ABSTRACT ..................................................................................................................................................V ACKNOWLEDGEMENTS ......................................................................................................................VII TABLE OF CONTENTS ........................................................................................................................... IX LIST OF FIGURES.................................................................................................................................XIII PREFACE ..................................................................................................................................................XV 1 INTRODUCTION ......................................................................................................................................1 DIGITAL, INTERACTIVE ART.........................................................................................................................1 INFORMATION SYSTEMS DESIGN (ISD) .......................................................................................................1 THE JAVA DEVELOPMENT PLATFORM ..........................................................................................................1 OBJECTIVES.................................................................................................................................................2 RESEARCH METHOD.....................................................................................................................................2 RESEARCH FINDINGS ...................................................................................................................................3 THESIS OUTLINE ..........................................................................................................................................4 2 DEFINITIONS AND BACKGROUND INFORMATION .....................................................................5 2.1 INTERACTIVITY......................................................................................................................................5 Existing definitions and dimensions of interactivity...............................................................................6 Sociology .......................................................................................................................................................... 6 Media and communication................................................................................................................................ 6 Business school................................................................................................................................................. 8 Computer Science ............................................................................................................................................. 9 Human Computer Interaction............................................................................................................................ 9 Observing interaction ..........................................................................................................................11 2.2 DIGITAL ART........................................................................................................................................14 Product.................................................................................................................................................14 Process.................................................................................................................................................15 Subject..................................................................................................................................................15 2.3 INTERACTIVE DIGITAL ART .................................................................................................................16 2.4 EMBODIED INTERACTION .....................................................................................................................17 2.5 NARRATIVITY AND NARRATIVE STRUCTURES ......................................................................................19 History of the phenomenon: narrative and narrativity ........................................................................19 Narrative in psychology.......................................................................................................................19 Narrative in computer science .............................................................................................................20 2.6 MODEL VIEW CONTROLLER ................................................................................................................21 2.7 TEMPORAL DATA .................................................................................................................................23 2.8 STATE MACHINES.................................................................................................................................23 TECHNOLOGIES OF DIGITAL, INTERACTIVE ART ......................................................................25 3.1 HARDWARE .........................................................................................................................................25 Responsive Environments ....................................................................................................................25 Real-time manipulation of video ..........................................................................................................25 Head-mounted display (Virtual reality) ...............................................................................................26 Augmented reality ................................................................................................................................27 Sensory devices ....................................................................................................................................28 3.2 JAVA SOFTWARE SOLUTIONS FOR HANDLING MEDIA............................................................................29 Java Media Framework (JMF) ............................................................................................................31 History ............................................................................................................................................................ 31 Architecture..................................................................................................................................................... 31 Main components/classes................................................................................................................................ 32 - x