Creating Illusion in Computer Aided Performance
Total Page：16
File Type：pdf, Size：1020Kb
Creating Illusion in Computer Aided Performance Joe Marshall, BA Thesis submitted to the University of Nottingham for the degree of Doctor of Philosophy March 2009 Abstract This thesis studies the creation of illusion in computer aided performance. Illusion is created here by using deceptions, and a design framework is presented which suggests several different deception strategies which may be useful. The framework has been developed in an iterative process in tandem with the development of 3 real world performances which were used to explore deception strategies. The first case study presents a system for augmenting juggling performance. The techniques that were developed to control this system demonstrate how de- ception may become useful even when the core of the performance is not deceptive in any way. This is followed by a magic performance called the Cup Game, which was designed to explicitly test the strategies of deception described in the frame- work. The final case study is an interactive art installation which presents the illusion of a pet rock that lives in a cage. This demonstrates the usefulness of suspension of disbelief in the creation of illusions. It also demonstrates interesting social effects that are used to strengthen this suspension of disbelief. The idea of creating the impression of a false situation is inspired particularly by previous HCI work on public interaction. This work demonstrated the useful- ness of hiding interface use or computer outputs from some people in a situation. The creation of deliberately ambiguous computer interfaces, which allow for a wider variety of interpretations to be made by the user has also been described. The work here goes beyond these techniques to use technology to actively create false impressions. The techniques used in this process are guided by the work of magic performers, and by psychological studies of how magic performance works. As well as artistic performance, it is envisaged that this work may prove ap- plicable to more traditional situations. In addition to the framework itself, the development of the case studies has created several useful algorithms which have wider applications. The case studies are also useful guides for those creating per- formance systems, or other systems where deceptive techniques may be useful. A web page with videos of the three systems is available at http://www.mrl.nott.ac.uk/~jqm/thesis Acknowledgements I’d like to thank my academic supervisors, Steve Benford and Tony Pridmore (and also Steve Mills who moved to New Zealand), for their support during this PhD. This work wouldn’t have been possible without their input, inspiration, opportunities, and encouragement. Being part of the University of Nottingham’s Mixed Reality Lab was also a great help and inspiration during this work. Thanks go to everyone in the lab. Secondly, it wouldn’t be possible to do this work without anyone to watch it, so thanks to all the lovely people who have given me opportunities to perform, or have taken part in or watched my performances. Finally, I owe a lot to my wife Penny Munro, for encouraging me when I decided to give up a well paid job to do a PhD in juggling. Little did she know what she was letting herself in for. CONTENTS 3 Contents 1 Introduction 8 1.1 Overview................................. 8 1.1.1 AFocusonComputerVision . 9 1.1.2 Deception&Illusion . 10 1.1.3 Contribution .......................... 10 1.2 Structure ................................ 11 1.3 DevelopmentandMethodology . 12 1.3.1 PracticalWorkandLivePerformance . 12 1.3.2 Relationship between Practical Work and Framework . .. 13 1.4 ComputerVisionPerformance . 14 1.4.1 ThesisWorkinContext . 17 1.5 Impact.................................. 18 1.6 Conclusion................................ 18 2 Awareness, Unawareness and Performance 19 2.1 Introduction............................... 20 2.2 ThreeTypesofAwareness . 21 2.2.1 Human Awareness of a Computer . 21 2.2.2 Computer Awareness (of Humans and Context) . 22 2.2.3 HumanAwarenessofHumans . 24 2.2.4 AwarenessesinPerformance . 25 2.3 ANotationforDescribingAwareness . 26 2.3.1 PerceptionsandAwareness. 29 2.4 Public Interaction and the Value of Unawareness . .... 30 2.4.1 Perceptions and Unawareness . 31 2.4.2 Actively Creating Unawareness . 33 CONTENTS 4 2.5 Conclusions ............................... 34 3 Altering Awareness Using Illusion 36 3.1 IllusioninHCI ............................. 36 3.2 ThePsychologyofMagic. 40 3.3 KeyPointsfromthePsychologyofMagic. 45 3.4 Examples of Magical HCI Experiences . 46 3.5 A Deception Framework for Describing Illusion . .... 48 3.5.1 FactorstoConsider. 48 3.5.2 TypesofDeception . 50 3.5.3 TwoPersonDeceptions. 50 3.5.4 DeceivingaComputer . 52 3.5.5 DeceivedbyaComputer . 53 3.5.6 Computer Mediated Deceptions . 54 3.6 An Example of a Computer Mediated Deception . 60 3.6.1 The Audience Member Picks the Card . 60 3.6.2 ThePerformerDrawstheRectangle. 61 3.6.3 The Computer Guesses the Card . 62 3.7 Conclusion-WhatisItFor?. 62 4 Eye-Balls: A Juggling Tracker 63 4.1 Why? .................................. 64 4.2 RelatedPerformanceWork. 65 4.3 Technology ............................... 67 4.3.1 TheTrackingAlgorithm . 67 4.3.2 ScriptSystem .......................... 70 4.4 The Juggling Tracker Design Process . 72 4.4.1 DesigningwithJugglers . 72 4.4.2 DesignConstraints . 73 4.5 AnExamplePerformance . 74 4.6 Visibility in the Juggling Tracker . 76 4.6.1 Whocanseewhat? ....................... 77 4.6.2 Howdotheyseethings? . 78 4.6.3 Foreground and Background Interaction . 82 4.6.4 Control, Visibility and Attention . 83 CONTENTS 5 4.7 Conclusions ............................... 89 5 The Cup Game 90 5.1 Why? .................................. 91 5.2 RelatedPerformanceWork. 91 5.3 Technology ............................... 94 5.3.1 TheCupTable ......................... 95 5.3.2 Software............................. 98 5.4 ThePerformance ............................ 99 5.4.1 Stage1: Introduction. .102 5.4.2 Stage2: TheCupsbySound. .103 5.4.3 Stage3: InFrontofYourEyes . .104 5.4.4 Stage4: GetYourOwnBack . .106 5.4.5 Stage5:TheBigReveal . .107 5.4.6 Stage6: BackgroundInformation . .108 5.5 DevelopmentofthePerformance. .109 5.5.1 TrackingtheTracker . .109 5.5.2 PeopleTrustComputers . .109 5.5.3 AudienceFailures . .109 5.5.4 FollowingtheRules. .110 5.5.5 SoftwareErrors . .110 5.5.6 PerformerErrors . .110 5.5.7 MisdirectionFailures . .110 5.6 PerformanceTrial. .111 5.6.1 Demographics. .111 5.6.2 DiditWork? ..........................111 5.6.3 Howdiditwork? . .. .. .112 5.7 Conclusions ...............................118 6 Rock 119 6.1 Why? ..................................119 6.2 RelatedWork ..............................120 6.3 Technology ...............................122 6.3.1 Hand and Pressure Sensing Algorithms . 123 6.3.2 Sound System and Personality . 126 CONTENTS 6 6.4 Performance&InstallationDesign . 127 6.5 PublicInteractionwiththeRock . .130 6.6 Conclusions ...............................135 7 Discussion: Revisiting Deception 137 7.1 RevisitingtheFramework . .137 7.1.1 FramingofDeception. .138 7.1.2 Trajectories, Transitions and Reveals . 140 7.1.3 The7TypesofDeception . .142 7.1.4 Trajectories and Framing: Multi-Layered Deceptions . .149 7.1.5 The Deception Framework Evolved . 156 7.2 Extending the Definition of Performance . 157 7.2.1 PerformanceswithManyRoles . .157 7.2.2 Remote and Asynchronous Deception . 158 7.2.3 SocialDeception . .159 7.2.4 Non-Entertainment Performance . 161 7.2.5 Public and Ubiquitous Interaction . 162 7.2.6 ComputerSecurity . .163 7.3 EthicsofDeception. .165 7.3.1 PotentialHarmofDeception. .165 7.3.2 Do People Know they Are Being Deceived? . 166 7.3.3 Approaches to the Ethics of Deception . 166 7.3.4 WhatDoesthisMeanforPerformance?. .168 7.4 Conclusion................................170 8 Conclusions 171 8.1 BacktotheBeginning . .171 8.1.1 QuestionsaboutPerformance . .171 8.1.2 EvaluatingtheContribution . .173 8.2 Deception: Future Implications . 175 8.2.1 AugmentedPerformance . .175 8.2.2 ComputerVision . .176 8.2.3 GeneralHCI...........................177 8.2.4 Wider Computer Science . 178 8.3 Conclusion................................179 CONTENTS 7 A Using Object Interactions to Improve Particle Filter Performance194 A.1 Introduction...............................194 A.1.1 ExistingWork. .195 A.1.2 The Example Tracking System . 196 A.2 TheProblem ..............................196 A.2.1 TheProposedSolution . .197 A.2.2 Calculating the Approximate Distribution . 199 A.2.3 Extensions............................201 A.3 Evaluation ...............................201 A.3.1 ImplementationDetails. .201 A.3.2 TestingResults . .203 A.4 Conclusions ...............................205 B Cup Game Script 207 C Pressing the Flesh: Sensing Multiple Touch and Finger Pressure on Arbitrary Surfaces 213 C.1 Introduction...............................214 C.2 PriorWork ...............................215 C.3 FingernailSensing . .216 C.3.1 InitialEvaluation . .218 C.4 SkinonGlass ..............................221 C.5 CaseStudy:Rock............................224 C.5.1 Technology implementation . 225 C.5.2 TestingtheRock . .225 C.6 Using a Bayesian Classifier to Detect Push Events . .227 C.7 Conclusions ...............................229 C.7.1 PotentialApplications . .230 C.7.2 FutureWork...........................230 CHAPTER 1. INTRODUCTION 8 Chapter 1 Introduction 1.1 Overview In her survey of recent human computer interaction (HCI) theory, Bødker [15] identifies 3 successive waves of HCI. The