Data Illustrator: Augmenting Vector Design Tools with Lazy Data Binding for Expressive Visualization Authoring Zhicheng Liu1 John Thompson2 Alan Wilson3 Mira Dontcheva1 James Delorey3 Sam Grigg3 Bernard Kerr3 John Stasko2 1Adobe Research 2Georgia Institute of Technology 3Adobe Systems Inc. Seattle, WA Atlanta, GA Lehi, UT & San Francisco, CA {leoli,mirad}@adobe.com {jrthompson,stasko}@gatech.edu {alawilso,delorey,grigg,bkerr}@adobe.com ABSTRACT appearance of a visualization in terms of layout and space Building graphical user interfaces for visualization author- configuration, and focus on encoding real data into the visuals ing is challenging as one must reconcile the tension between later [3, 56]. The discipline of graphic design has also estab- flexible graphics manipulation and procedural visualization lished a rich set of concepts and tools that are widely used generation based on a graphical grammar or declarative lan- in the community. For example, professional vector editors guages. To better support designers’ workflows and practices, enable designers to work with shape geometries at the level we propose Data Illustrator, a novel visualization framework. of anchor points and curve segments. The grid system and In our approach, all visualizations are initially vector graphics; smart guides serve as two powerful tools to precisely structure data binding is applied when necessary and only constrains visual elements and configure display space [32, 40, 56]. interactive manipulation to that data bound property. The Despite the plethora of existing visualization creation tools, framework augments graphic design tools with new concepts few tried to incorporate designers’ workflow and practices into and operators, and describes the structure and generation of system and interface design. Vuillemot and Boy [56] argue that a variety of visualizations. Based on the framework, we de- most visualization tools follow a bottom-up, data-to-graphics sign and implement a visualization authoring system. The process as described in the information visualization (InfoVis) system extends interaction techniques in modern vector design reference model [8]: starting with data, one performs data tools for direct manipulation of visualization configurations transformation, visual mapping, and view transformation to and parameters. We demonstrate the expressive power of generate visualizations. This model informed the development our approach through a variety of examples. A qualitative of powerful visualization algebra and declarative languages study shows that designers can use our framework to compose [29, 49, 52, 58]. However, these tools often require coding visualizations. expertise, or are not flexible enough for design practices. ACM Classification Keywords Systems like Lyra [48] and iVisDesigner [44] offer graphical H.5.m. Information Interfaces and Presentation (e.g. HCI): UI user interfaces (GUI) for visualization authoring without pro- gramming, thus are more flexible. These efforts start with tem- Author Keywords plate or grammar-based visualization generation engines, and Data visualization; graphic design; interaction techniques; design interfaces for changing generative parameters. Such ap- framework; authoring; systems. proaches still need to reconcile the potential tension between flexible change of graphical configurations and the formal- INTRODUCTION ism imposed by generation engines [4]. To bridge the gap Graphic designers have been producing infographics and between generation engines and drawing tools, Hanpuku [4] charts well before the recent proliferation of computer gen- implements a streamlined model for visualization authoring erated visualizations [20, 37]. As visualization becomes an across multiple tools. increasingly popular medium for storytelling and communi- Recent work also began to explore visualization authoring cation, there is a renewed and growing interest to understand without programming from a purely graphic design perspec- visualization creation from the perspective of graphic design tive. With Data-Driven Guides [32], designers can create [3,4, 32, 56, 57]. Prior studies show that graphic designers freeform guides and sketch graphics with the guides. d3- approach visualization authoring differently from computer gridding [56] enables the creation of quick mock-ups with scientists: they often start by thinking about the high-level minimal or no data. These systems adopt a “lazy data binding” approach: visualizations are first and foremost vector graphics Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed with no underlying templates or declarative languages. De- for profit or commercial advantage and that copies bear this notice and the full citation signers use familiar tools to draw, select and manipulate vector on the first page. Copyrights for components of this work owned by others than ACM graphics, and apply data encoding only when it is necessary. must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a Compared to template or grammar based systems, this ap- fee. Request permissions from [email protected]. proach is more compatible with designers’ workflows and CHI 2018, April 21–26, 2018, Montreal, QC, Canada practices. Users do not have to align their mental models with © 2018 ACM. ISBN 978-1-4503-5620-6/18/04. 15:00 DOI: https://doi:org/10:1145/3173574:3173697 1 the grammar or model assumed by the system. Furthermore, vector design tools are highly flexible and expressive: with enough time and patience, one can create virtually any graph- ics. Augmenting these tools with data encoding support can reduce manual effort without disrupting designers’ workflows. The lazy data binding approach is promising, but needs to be developed further to support a wide variety of visualiza- tions. Data-Driven Guides [32] only focus on infographics with simple layouts. d3-gridding [56] primarily supports de- sign mockups, and still requires programming. It remains a challenge for designers to create high-fidelity data visualiza- tions with complex visual mappings and layouts. Consider the visualizations in Figure1: Figure1(a) is a slope graph used on the cover of Alberto Cairo’s book The Func- tional Art [7], showing U.S. states’ obesity and education percentages (hereafter referred as the “Obesity vs. Education” visualization); Figure1(b) visualizes the NBA draft over the past 20 years (x axis) and the order of players in terms of draft pick (y axis) [15] (hereafter referred as the “NBA Redraft” visualization); Figure1(c) is a multi-series line graph visu- alizing four companies’ monthly stock prices [36] (hereafter referred as the “Stock Prices” visualization); Figure1(d) is “A Field Guide to Red and Blue America” by Wall Street Journal, showing the PVI (Partisan Voter Index) for each state over the past 9 elections [61] (hereafter referred as the “Red and Blue America” visualization). Each small bar chart represents a state, and is positioned according to US geography. Figure 1: Visualizations with varying levels of complexity Designers might be able to use existing drawing tools or Data- Driven Guides to create these examples, but the process will be designers, focusing on whether they can understand and use painful. Generating shapes or points on lines (Figure1(c)) can the framework for visualization composition. be tedious and slow; organizing the shapes into meaningful layouts (Figure1(b) and (d)) and map data to positions and color (Figure1(c)) are daunting manual tasks. To enable FORMATIVE STUDY AND DESIGN ITERATIONS designers to keep using the powerful drawing tools and to To understand how different visualizations could be described automate the repetitive work, we need a systematic framework and created from a graphic design perspective, we held one- with sufficient descriptive and generative power. hour weekly meetings with three designers over a period of In this paper we propose a novel framework for visualization two years. All three designers have more than ten years of authoring based on the lazy data encoding approach. This experience in graphic design, digital illustration, web design framework describes components in a visualization using and print design. Two of the designers have also created graphic design concepts such as shape, anchor point, seg- infographics and data visualizations on a regular basis as part ment, and group. Two operators, repeat and partition, gener- of their work. The designers frequently used applications such ate shapes and anchor points, and attach data to them. The as Adobe Photoshop [25], Illustrator [23], InDesign [24] and resultant visual components each has a data scope, and are Experience Design (XD) [22], Sketch [10], and Figma [12]. considered peers of each other inside a collection. Collections These applications represent the industry standard for design use layouts to arrange shapes, and can be nested to create more professionals. They share a similar set of features and tools, complex organizations. Data serves as constraints when bound varying in terms of interaction and interface design. to visual properties, and unbound properties can be freely ma- In the initial meetings, we collected visualizations by sam- nipulated. These components and operators can describe the pling chart types from systems like Tableau [51]