A FRAMEWORK FOR CREATIVE VISUALIZATION-OPPORTUNITIES WORKSHOPS by Ethan Kerzner A dissertation submitted to the faculty of The University of Utah in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science School of Computing The University of Utah May 2019 Copyright c Ethan Kerzner 2019 All Rights Reserved The University of Utah Graduate School STATEMENT OF DISSERTATION APPROVAL The dissertation of Ethan Kerzner has been approved by the following supervisory committee members: Miriah Meyer , Chair(s) August 9, 2018 Date Approved Erik Brunvand , Member August 9, 2018 Date Approved Jason Dykes , Member Date Approved Charles Hansen , Member September 13, 2018 Date Approved Alexander Lex , Member September 14, 2018 Date Approved by Ross Whitaker , Chair/Dean of the Department/College/School of School of Computing and by David B. Kieda , Dean of The Graduate School. ABSTRACT Applied visualization researchers often work closely with domain collaborators to ex- plore new, useful, and interesting visualization applications. The early stages of collabo- rations are typically time consuming for all stakeholders as researchers piece together an understanding of domain challenges from disparate discussions and meetings. A number of recent projects, however, report on the use workshops to accelerate the early stages of applied work, eliciting a wealth of requirements in a few days of focused work. Yet, no guidance exists for how to use such workshops effectively. This dissertation’s primary contribution is a framework — created through the meta-analysis of 17 workshops in 10 visualization contexts — that describes how and why to use workshops in the early, formative stages of applied work. The framework: 1) describes characteristics of effective workshops; 2) identifies a process model for using workshops; 3) describes a structure of what happens within effective workshops; 4) recommends 25 actionable guidelines for future workshops; and 5) proposes three example workshops as a starting point for researchers who are interested in using workshops in future projects. The creation of this framework exemplifies the use of critical reflection to learn about visualization in practice from diverse studies and experience. The framework is grounded in two formative design studies that provide this disser- tation’s secondary contributions. In the first formative design study, we worked with defense analysts focused on improving the safety of military vehicles. From this design study, we contribute task analysis, data abstraction, and a validated visualization tool for the visual analysis of spatial and nonspatial ballistic vulnerability data. In the second formative design study, we worked with neuroscientists focused on retinal connectomics. From this design study, we contribute two new visualization techniques and a prototype system for visualizing connectivity in large graphs. Our experiences in these two design studies motivated and informed this dissertation’s primary contribution. To D, L, J, M, and A. CONTENTS ABSTRACT ............................................................. iii LIST OF FIGURES ....................................................... viii ACKNOWLEDGMENTS .................................................. ix CHAPTERS 1. INTRODUCTION .................................................... 1 1.1 Motivation and Overview . .1 1.2 Contributions . .4 1.3 Organization . .5 2. BACKGROUND AND RELATED WORK ................................ 6 2.1 Terminology . .6 2.2 Workshops in Applied Visualization . .8 2.3 Visualization Design Methods . 10 2.4 Participatory (and Similar) Design Methods . 12 2.5 Creativity Workshops and Support Tools . 13 2.6 Conclusion . 14 3. FORMATIVE DESIGN STUDY — SHOTVIEWER: VISUAL ANALYSIS OF BALLISTIC VULNERABILITY DATA ................................... 15 3.1 Formative Aspects . 15 3.2 Motivation and Overview . 16 3.3 Methods . 17 3.4 Problem, Data, and Tasks . 19 3.4.1 Problem Characterization . 19 3.4.2 Data Abstraction . 20 3.4.3 Task Analysis . 22 3.5 Related Work — Spatial and Nonspatial Data . 22 3.6 Shotviewer . 23 3.6.1 Shotline View . 24 3.6.2 Geometry View . 27 3.6.3 System View . 28 3.6.4 Implementation . 30 3.7 Validation . 30 3.8 Reflections . 31 3.8.1 View-Design Parallelism . 32 3.8.2 Recommendations . 34 3.9 Conclusion . 35 4. FORMATIVE DESIGN STUDY — GRAFFINITY: VISUAL ANALYSIS OF CONNECTIVITY IN LARGE GRAPHS .................................. 36 4.1 Formative Aspects . 36 4.2 Motivation and Overview . 37 4.3 Requirements . 39 4.4 Related Work — Graph Connectivity . 40 4.5 Connectivity Overviews . 43 4.5.1 Connectivity Matrix . 43 4.5.2 Intermediate Node Table . 45 4.6 Graffinity . 47 4.6.1 Queries . 47 4.6.2 Connectivity Overview . 49 4.6.3 Supplemental Views . 51 4.6.4 Implementation . 53 4.7 Validation . 53 4.8 Discussion . 55 4.9 Conclusion . 57 5. A FRAMEWORK FOR CREATIVE VISUALIZATION-OPPORTUNITIES WORKSHOPS ....................................................... 59 5.1 Motivation and Overview . 60 5.2 Workshop Experience and Terminology . 61 5.3 Research Process . 65 5.4 Fundamentals of the Framework . 68 5.4.1 Tactics for Effective Workshops . 68 5.4.2 Process Model and Structure . 69 5.5 Before the Workshop: Define & Design . 72 5.6 Workshop Structure and Methods . 74 5.6.1 Workshop Opening . 74 5.6.2 Workshop Core . 75 5.6.3 Workshop Closing . 78 5.6.4 Examples Workshops and Methods . 79 5.7 During the Workshop: Execute & Adapt . 86 5.8 After the Workshop: Analyze & Act . 88 5.9 Conclusion . 90 6. DISCUSSION ....................................................... 92 6.1 CVO Workshops in Applied Research . 92 6.2 CVO Workshop Framework . 95 6.3 Critically Reflective Practice . 97 7. CONCLUSION ...................................................... 99 7.1 Summary . 99 7.2 Future Work . 100 vi APPENDIX: LIST OF WORKSHOP GUIDELINES ............................ 102 REFERENCES ........................................................... 104 vii LIST OF FIGURES 2.1 CVO workshops in the visualization design process. 10 3.1 Timeline of the Shotviewer design study. ..