Virtual Rear Projection: Improving the User Experience with Multiple Redundant Projectors
Total Page:16
File Type:pdf, Size:1020Kb
VIRTUAL REAR PROJECTION: IMPROVING THE USER EXPERIENCE WITH MULTIPLE REDUNDANT PROJECTORS A Thesis Presented to The Academic Faculty by Jay W. Summet In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the College of Computing Georgia Institute of Technology December 2007 VIRTUAL REAR PROJECTION: IMPROVING THE USER EXPERIENCE WITH MULTIPLE REDUNDANT PROJECTORS Approved by: Professor Gregory D. Abowd Professor Gregory M. Corso College of Computing School of Psychology Georgia Institute of Technology Georgia Institute of Technology Professor James M. Rehg Dr. Jeffrey S. Pierce College of Computing Almaden Research Center Georgia Institute of Technology IBM Professor Elizabeth Mynatt Dr. Claudio Pinhanez College of Computing T.J. Watson Research Center Georgia Institute of Technology IBM Date Approved: 31 July 2007 To my parents, who made sure I had everything I needed to succeed, and to my sister, le Petit Chaperon rouge. iii ACKNOWLEDGEMENTS Many people have helped me along the way, but my advisers, Jim and Gregory, have always been in the forefront. I am thankful to Jim for introducing me to an exciting research topic and guiding the technical development and Gregory for his advice on evaluation and the PhD program in general. I am especially grateful for the time and effort my external committee members, Claudio Pinhanez and Jeff Pierce spent working with me on my research and the document. Other professors at Georgia Tech have helped me both with my thesis and with other interests. Greg Corso encouraged and improved my user evaluations even before he was on my committee. Beth Mynatt provided guidance on balancing the technology and human side of the research, as well as encouragement throughout. John Stasko provided valuable advice and feedback on publications. Jim Foley, Mark Guzdial and Thad Starner provided advice, support and encouragement in non-thesis areas of my academic career. I wish to express my gratitude to my wife Valerie for her continuous moral support. My peers at the College of Computing provided advice and guidance, and were great people to spend a large part of my life with. To those who went before me, thanks for the encouragement and guidance. To those who traveled with me, thanks for all the fun. And for those who are following, good luck! iv TABLE OF CONTENTS DEDICATION ......................................... iii ACKNOWLEDGEMENTS................................... iv LISTOFTABLES ....................................... ix LISTOFFIGURES ...................................... x SUMMARY........................................... xiv I DISPLAYTECHNOLOGIESFORINTERACTIVESURFACES . 1 1.1 Overview of Display Technologies . ...... 3 1.1.1 Direct Image Display Technologies . .... 3 1.1.2 Projection Technologies . 7 1.1.3 Eye-Coupled Display Technologies . .... 9 1.2 Why Virtual Rear Projection? . 10 II RELATEDWORK.................................... 13 2.1 Large Displays & Applications . 13 2.2 Projected Display Technology . ..... 14 2.3 Shadow Elimination and Blinding Light Suppression . ........... 17 III INITIAL DEVELOPMENTOFFRONTPROJECTIONFORINTERACTIVE SURFACES 19 3.1 Warped Front Projection . 19 3.2 Passive Virtual Rear Projection . ...... 20 3.3 Computer Vision and Homographies for Calibration . .......... 21 IV PVRPEVALUATION .................................. 23 4.1 Projection Technologies Studied . ....... 23 4.2 StudySetup..................................... 25 4.2.1 EquipmentSetup.............................. 25 4.2.2 StudyParticipants ...... ..... ...... ...... ..... 26 4.2.3 StudyTasks ................................ 26 4.3 Results ....................................... 27 4.3.1 SubjectiveResults ...... ..... ...... ...... ..... 28 v 4.3.2 Quantitative Measures: Speed & Accuracy . ..... 30 4.3.3 CopingStrategies ............................. 32 4.3.4 Participant Awareness of Shadow Coping Strategies . ......... 34 4.4 Followup Blinding Light Comfort Level Study . ........ 34 4.5 Discussion ..................................... 36 V ACTIVEVIRTUALREARPROJECTION . 39 5.1 ShadowElimination ............................... 39 5.1.1 Occlusion detection . 40 5.1.2 EliminatingShadows ........................... 41 5.2 Shadow Elimination + Blinding Light Suppression . .......... 42 5.2.1 OcclusionDetection . 44 5.2.2 Iterative Photometric Compensation . ...... 44 5.2.3 ShadowElimination ............................ 46 5.2.4 Blinding Light Suppression . 46 5.2.5 Integrated System for Shadow Elimination and Blinding Light Suppression 47 5.3 Switching...................................... 49 5.3.1 OcclusionDetection . 50 5.3.2 Photometric Uniformity . 51 5.3.3 EdgeBlending............................... 53 5.3.4 Improving Performance using the GPU . 55 5.4 Quantitative Evaluation of Virtual Rear Projection Methods ........... 56 5.4.1 ExperimentalSetup ............................ 58 VI PROCAMSTOOLKIT.................................. 61 6.1 PROCAMSAbstractions. 62 6.2 PROCAMSApplications. 63 6.2.1 Redundant Illumination - WinPVRP . 64 6.2.2 Warped Front Projection - Banner Display . ..... 66 6.3 PROCAMSArchitecture. 67 6.4 PROCAMScodesamples ............................. 69 6.4.1 Allocating and Positioning a Display . ..... 69 6.4.2 Calibrating Redundant Projectors using Computer Vision ........ 70 vi 6.4.3 NativeImageFormat ........................... 72 VII EVALUATIONSTUDIES ............................... 74 7.1 ResearchQuestions............................... 74 7.2 StudyFormat.................................... 76 7.2.1 Tasks.................................... 77 7.2.2 Rationale for Task Selection . 78 7.3 Participants.................................... 79 7.3.1 Aerospace Engineering Students (Aerospace Task) . ......... 80 7.3.2 College Students (Hangman task) . 80 7.4 Experimental Procedure . 80 7.4.1 ResearchProcedure ............................ 81 7.4.2 Researcher Focus Group Questions . 81 7.5 Analysis & Results: Aerospace Task . ...... 82 7.5.1 Research Metrics & Analysis . 83 7.5.2 UserPreference .............................. 85 7.5.3 Annoyance of Blinding Light . 88 7.5.4 ImageQuality ............................... 90 7.5.5 MeanGroupActivity ........................... 91 7.5.6 Interaction Patterns with the Board . ..... 94 7.5.7 PerceivedValueofAVRP . 97 7.6 Analysis & Results: Hangman Task . 98 7.6.1 UserPreference .............................. 99 7.6.2 Annoyance of Blinding Light . 102 7.6.3 ImageQuality ............................... 106 7.6.4 MeanUserActivity ............................ 107 7.6.5 PerceivedValueofAVRP . 110 7.7 Study Similarities and Contrasts . ....... 112 7.8 Reflections on Research Methodology . 115 7.8.1 Equipment & Researcher Reliability . 116 7.8.2 Reflections on Task Selection . 117 7.9 Conclusions..................................... 118 vii 7.9.1 UserPreference .............................. 119 7.9.2 Benefits of Redundant Illumination & Blinding Light Suppression . 120 7.9.3 Claims ................................... 122 VIII FUTUREDIRECTIONS&CONCLUSIONS. 124 8.1 FutureDirections................................ 124 8.2 Conclusions..................................... 125 Bibliography ....................................... 133 viii LIST OF TABLES 1 Mean (Standard Deviation) subjective measures on a 7 point scale, on image quality and annoyance of projected light on a front projection screen. Bold data indicates statistical significance. 35 2 Algorithm Performance Measures . 58 ix LIST OF FIGURES 1 left to right: Front Projection, Virtual Rear Projection, Rear Projection....... 2 2 WarpedFrontProjection ............................. 19 3 Passive Virtual Rear Projection . ...... 20 4 Taxonomy of Projection Technologies in our study. .......... 24 5 Center target and the eight possible box starting positions. ............. 27 6 (Top) Subjective scores from participant questionnaires. (Bottom) Pairwise com- parisons of Image Quality, Preference, and Acceptance scores based upon treatment condition. Positive numbers indicate the condition scored higher than the “com- pared with” condition. Statistically significant differences (p<0.05) are presented in bold........................................... 28 7 (Top) Acquire times in the Box task with number of occluded boxes in each condi- tion. (Bottom) Pairwise comparisons of Box Acquire Time (in milliseconds) based upon treatment condition. Positive numbers indicate how much slower the “condi- tion” is than the “compared with” condition. All statistically significant differences (p<0.05) are presented in bold............................. 31 8 Acquire time for occluded and unoccluded boxes. ......... 32 9 Participant exhibiting the edge-of-screen coping strategy while working the Box Task in the Front Projection condition. ....... 33 10 Projector locations and beam-paths for a 17.5ft (5.3m) wide electronic whiteboard using passive virtual rear projection. Users find it extremely difficult to avoid stand- ing within projection beams. 35 11 Additions to projection technologies taxonomy. ............ 38 12 Left:Shadow Elimination. Right: Penumbral shadows are eliminated but the blind- inglightremains. ................................... 39 13 This diagram summarizes the occlusion detection and shadow elimination algo- rithms. The images in the left column were taken by the system camera during operation. The two penumbral occlusions caused by the person blocking both pro- jectors are identified and corrected to create a shadow-free display (bottom left). Seetextfordetails. ................................