Visualizing Uncertainty in Drug Checking Test Result Reports During the Opioid Crisis: A Design Study by Jorin Diening Weatherston B.Seng, University of Victoria, 2017 A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Computer Science in the Department of Computer Science c Jorin Diening Weatherston, 2020 University of Victoria All rights reserved. This thesis may not be reproduced in whole or in part, by photocopying or other means, without the permission of the author. ii Visualizing Uncertainty in Drug Checking Test Result Reports During the Opioid Crisis: A Design Study by Jorin Diening Weatherston B.Seng, University of Victoria, 2017 Supervisory Committee Dr. Margaret-Anne Storey, Supervisor (Department of Computer Science) Dr. Charles Perin, Committee Member (Department of Computer Science) Dr. Dennis Hore, Committee Member (Department of Computer Science) iii ABSTRACT Potent opioids (fentanyl) are entering recreational drug manufacturing processes, sometimes without the knowledge of people who use drugs. This is contributing to tens of thousands of accidental overdose deaths each year. Recreational drug checking services during the opioid crisis face unique challenges in delivering test results to peo- ple who use drugs. These challenges are caused by uncertainties in drug composition, the chemical analysis processes used, and the complex contextual considerations of drug checking services themselves. In this thesis I describe a design study in collab- oration with a local drug checking service to explore visualizing uncertainty in drug checking test result reports. From this research we generate a number of research con- tributions. I have identified the new and impactful application domain of visualizing uncertain drug checking test results. Within this application domain I conducted a design study to generate a test result report that suits the problem context and ac- complishes the design goals described by the drug checking service stakeholders. This design study generates reflective considerations on conducting design studies in this context, intermediate design artifacts, and finally a test result report software applica- tion. The design study also led to the identification of a new uncertainty visualization design space for proportional charts. I apply that design space in the generation of some intermediate design artifacts. I position these research contributions within the drug checking and uncertainty visualization research fields, and describe our planned future work in the hopes that future research will positively impact the application domain. iv Contents Supervisory Committee ii Abstract iii Table of Contents iv List of Tables viii List of Figures x Acknowledgements xiv Dedication xv 1 Introduction 1 1.1 Motivation . 1 1.2 Contributions . 3 1.3 Thesis Layout . 4 1.3.1 Part One . 4 1.3.2 Part Two . 5 1.3.3 Part Three . 5 2 Methodology 6 2.1 The Design Science Paradigm . 6 2.2 Information Location and Task Clarity in Design Studies . 7 2.3 The Relevance, Rigour, and Visual Cycles . 9 2.4 Presenting Contributions using Technological Rules, Novelty and a Vi- sual Abstract . 9 2.5 Design Study Structure . 10 2.6 Methodologies used Within Design Iterations . 12 v 2.6.1 Five Design Sheet Methodology . 12 2.6.2 Design Space Exploration Process . 14 3 Drug Checking Service Context 16 3.1 Stakeholder Types . 16 3.1.1 Clients . 18 3.1.2 Harm Reduction Workers . 19 3.1.3 Chemical Analysts . 21 3.2 Drug Testing Systems and Test Result Formats . 22 3.2.1 Component and Percent Composition Test Results . 23 3.3 Uncertainty in the Drug Checking Service . 24 3.4 Existing Test Result Delivery Methods . 27 3.4.1 Literature Concerning the Communication of Drug Checking Test Results . 29 4 Requirements Analysis 32 4.1 Requirements and Acceptance Criteria Gathering Processes . 32 4.1.1 Semi-Structured Interviews . 33 4.1.2 Interview Protocol . 34 4.1.3 Design Feedback Meetings . 34 4.1.4 Design Feedback Survey . 35 4.2 Requirements and Acceptance Criteria . 37 5 Design Goal One: Visualizing Percent Composition and Compo- nent Composition 40 5.1 Improving Drug Checking Test Results Delivery Using Charts . 40 5.2 Selecting Appropriate Charts . 41 5.2.1 Percent Composition Chart . 43 5.2.2 Component Composition Chart . 44 6 Design Goal Two: Visualizing Uncertainty in Percent Composition 46 6.1 Empowering Clients with Uncertainty and Confidence Data in Test Results . 46 6.2 Uncertainty in Percent Composition . 48 6.2.1 Characterizing Uncertainty in Percent Composition Test Results 50 6.2.2 Uncertainty Visualizations for the Public . 52 vi 6.2.3 Design Guidance for Visualizing Uncertainty . 52 6.3 Unquantified Uncertainty Design Space . 55 6.3.1 Preliminaries . 56 6.3.2 Step 1 - Breakdown . 56 6.3.3 Step 2 - Dimensions . 57 6.3.4 Step 3 - Systematic Exploration . 58 6.3.5 Step 4 - Application . 59 7 Design Goal Three: Visualizing Confidence in Component Com- position 62 7.1 Confidence in Component Composition . 62 7.1.1 Characterizing Confidence in Component Composition Test Re- sults . 64 7.1.2 Design Guidance for Visualizing Confidence in Component Com- position . 65 7.1.3 Generating Confidence Indicator Design Alternatives . 66 8 Additional Report Design Goals 69 8.1 Design Goal Four: Digital and Handout Reports Must be the Same . 69 8.2 Design Goal Five: The Visual Report Must Present Basic Drug Check- ing Service Information . 71 8.3 Design Goal Six: The Visual Report Must Present Descriptors of the Drug Sample . 72 8.4 Design Goal Seven: The Visual Report Must Highlight Fentanyl In the Test Results . 74 8.5 Design Goal Eight: Chemical Analysts Must be Able to Interpret the Test Results . 76 8.6 Design Goal Nine: The Visual Report Must Explicitly Disclaim Itself 77 8.7 Final Report Design . 78 9 Implementation 82 9.1 Artifact Deployment Iteration: Make Stage . 82 9.2 Artifact Deployment Iteration: Deploy Stage . 84 9.3 Using the Application . 85 10 Discussion 91 vii 10.1 Contribution 1: Design Study . 91 10.2 Contribution 2: Design Space . 97 10.2.1 Evaluating the Unquantified Uncertainty Design Space . 98 10.2.2 Design Space Consistency . 99 10.2.3 Design Space Completeness . 100 10.2.4 Using the Unquantified Uncertainty Design Space as a Visual- ization Researcher . 102 10.2.5 Understanding Designs Produced by the Unquantified Uncer- tainty Design Space as an End User . 103 10.3 Contribution 3: Visual Report . 103 10.3.1 Dominant Design Trade-offs . 103 10.3.2 Transferability . 104 10.3.3 Limitations . 105 11 Future Work & Conclusions 107 11.1 Future Work . 107 11.1.1 Evaluation of the Drug Checking Test Result Digital Report and Handout Report . 107 11.1.2 Unquantified Uncertainty in Proportional Charts Design Space 108 11.2 Conclusions . 109 Bibliography 111 A Drug Checking Service Flow 117 B Design Feedback Survey 119 viii List of Tables Table 2.1 Design iterations and deployment iteration with primary activities. 13 Table 3.1 Chemical analysis methods and type of drug checking test results produced. 23 Table 3.2 A depiction of components and their representative hit-scores. Hit-scores go from low to high confidence. Ratios of 1 represent a complete identification. The SERS scale is qualitative, and based on an unexposed internal set of thresholds; chemical analysts only get to see the colours. 24 Table 3.3 This is an example of an FTIR test output. It shows a list of components and the percent of the sample's composition they each comprise. 24 Table 3.4 In-person communication of results directly to clients across 31 services. Note that some services deliver results in multiple ways. [4] . 27 Table 4.1 This tables presents the connections between my stakeholders, information gathering processes, design constraints, and design goals. The Requirements and Acceptance Criteria column includes short descriptions of design guidance gathered at a high, medium and low level of abstraction. The Design Goals col- umn indicates which design goals satisfy the requirements and acceptance criteria. The Source column indicates where require- ments and acceptance criteria were collected from. The Process column indicates which technique was used to collect the require- ments and acceptance criteria. I describe the codes below the data. 38 ix Table 5.1 A prioritization of common visualization tasks to select charts for each data type [45]. The original set of tasks is in the left column, rank numbers for my orderings are in the middle-left column, task ordering for percent composition is in the middle-right column, and task ordering for component composition is in the right column. 42 x List of Figures Figure 2.1 Sedlmair et al.'s[47] diagram depicting information location and task clarity. Design projects can be placed within these axes to understand whether or not a project could be conducted using a design study. 8 Figure 2.2 The complete research timeline composed of design and deploy- ment iterations, and five design activity stage types. 12 Figure 3.1 A diagram of service flow within Substance with stakeholders and stages. 17 Figure 3.2 A row from the EcstasyData.org database. Accessed: 01/06/2019 28 Figure 3.3 A sample test result report from the EcstasyData.org database. Accessed: 01/06/2019 . 29 Figure 4.1 The complete research timeline with stakeholder feedback and requirements analysis processes highlighted. 33 Figure 5.1 The percent composition pie vs cake charts (left), and the com- ponent composition table chart (right). 43 Figure 6.1 The percent composition pie and cake charts with 100% axis. 47 Figure 6.2 Decomposition of the pie and cake chart into visual marks. 57 Figure 6.3 Examples of low, medium and high manipulations to individual visual variables of individual visual marks.