Surfacing Visualization Mirages Andrew McNutt Gordon Kindlmann Michael Correll University of Chicago University of Chicago Tableau Research Chicago, IL Chicago, IL Seattle, WA
[email protected] [email protected] [email protected] ABSTRACT In this paper, we present a conceptual model of these visual- Dirty data and deceptive design practices can undermine, in- ization mirages and show how users’ choices can cause errors vert, or invalidate the purported messages of charts and graphs. in all stages of the visual analytics (VA) process that can lead These failures can arise silently: a conclusion derived from to untrue or unwarranted conclusions from data. Using our a particular visualization may look plausible unless the an- model we observe a gap in automatic techniques for validating alyst looks closer and discovers an issue with the backing visualizations, specifically in the relationship between data data, visual specification, or their own assumptions. We term and chart specification. We address this gap by developing a such silent but significant failures visualization mirages. We theory of metamorphic testing for visualization which synthe- describe a conceptual model of mirages and show how they sizes prior work on metamorphic testing [92] and algebraic can be generated at every stage of the visual analytics process. visualization errors [54]. Through this combination we seek to We adapt a methodology from software testing, metamorphic alert viewers to situations where minor changes to the visual- testing, as a way of automatically surfacing potential mirages ization design or backing data have large (but illusory) effects at the visual encoding stage of analysis through modifications on the resulting visualization, or where potentially important to the underlying data and chart specification.