Sketch-N-Sketch: Output-Directed Programming for SVG
Total Page:16
File Type:pdf, Size:1020Kb
Sketch-n-Sketch: Output-Directed Programming for SVG Brian Hempel Justin Lubin Ravi Chugh Department of Computer Science, University of Chicago, USA {brianhempel,justinlubin,rchugh} @ uchicago.edu 1/8/2019 Sketch-n-Sketch { a e Sketch-n-Sketch File Code Tools View Options Output Tools Context: Program Undo Redo Clean Up Run b 1 equiTriPt [x3, y3] [x2, y2] = 2 [ (x2 + x3 + sqrt 3! * (y2 - y3))/ 2!, (y2 + y3 - sqrt 3! * (x2 - x3)) / 2!] c 3 4 oneThirdPt [x3, y3] [x, y] = d 5 [ x / 1.5!+ x3 / 3!, y / 1.5! + y3 / 3!] 6 7 point = [39, 314] 8 9 point2 = [490, 301] 10 11 makeKochPts depth point point2 = 12 let oneThirdPt2 = oneThirdPt point point2 in f 13 let oneThirdPt3 = oneThirdPt point2 point in 14 let equiTriPt2 = equiTriPt oneThirdPt3 oneThirdPt2 in 15 if depth < 2 then 16 [point, oneThirdPt3, equiTriPt2, oneThirdPt2] 17 else 18 let makeKochPts2 = makeKochPts (depth - 1) point oneThirdPt3 in 19 let makeKochPts3 = makeKochPts (depth - 1) oneThirdPt3 equiTriPt2 in 20 let makeKochPts4 = makeKochPts (depth - 1) equiTriPt2 oneThirdPt2 in 21 let makeKochPts5 = makeKochPts (depth - 1) oneThirdPt2 point2 in ⦀ 22 concat [makeKochPts2, makeKochPts3, makeKochPts4, makeKochPts5] 23 24 depth = 3{1-5} 25 26 topPts = makeKochPts depth point point2 27 28 botCorner = equiTriPt point2 point 29 30 rightPts = makeKochPts depth point2 botCorner 31 32 leftPts = makeKochPts depth botCorner point 33 34 snowflakePts = concat [topPts, rightPts, leftPts] 35 36 polygon1 = 37 let pts = snowflakePts in 38 let [color, strokeColor, strokeWidth] = [124, 360, 2] in 39 polygon color strokeColor strokeWidth pts 40 41 svg (concat [ 42 [polygon1] 43 ]) Figure 1. A recursive program built through output-directed program transformations, without any text-based programming. ABSTRACT tem for creating programs that generate vector graphics. For creative tasks, programmers face a choice: Use a GUI and To enable output-directed interaction at more stages of pro- sacrifice flexibility, or write code and sacrifice ergonomics? gram construction, we expose intermediate execution products for manipulation and we present a mechanism for contex- To obtain both flexibility and ease of use, a number of sys- tual drawing. Looking forward to output-directed program- tems have explored a workflow that we call output-directed ming beyond vector graphics, we also offer generic refactor- programming. In this paradigm, direct manipulation of the ings through the GUI, and our techniques employ a domain- program’s graphical output corresponds to writing code in a agnostic provenance tracing scheme. general-purpose programming language, and edits not possible with the mouse can still be enacted through ordinary text edits To demonstrate the improved expressiveness, we implement arXiv:1907.10699v4 [cs.HC] 10 Aug 2019 to the program. Such capabilities provide hope for integrating a dozen new parametric designs in SKETCH-N-SKETCH with- graphical user interfaces into what are currently text-centric out text-based edits. Among these is the first demonstration of programming environments. building a recursive function in an output-directed program- ming setting. To further advance this vision, we present a variety of new output-directed techniques that extend the expressive power of Author Keywords SKETCH-N-SKETCH, an output-directed programming sys- Output-Directed Programming; SVG; Sketch-n-Sketch CCS Concepts • Software and its engineering ! Application specific devel- This is the authors’ preprint version. It is posted here for your personal use. opment environments; Language features; • Human-centered The definitive version was published in the following venue. computing ! Graphical user interfaces; UIST’19, October 20–23, 2019, New Orleans, LA, USA © 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM. ISBN 978-1-4503-6816-2/19/10. $15.00 DOI: https://doi.org/10.1145/3332165.3347925 INTRODUCTION A number of systems have made initial steps towards realizing Direct manipulation [40] graphical user interface (GUI) appli- this paradigm. After first drafting a program using ordinary cations are ubiquitous. Every day, direct manipulation GUIs text edits, several ODP systems allow direct manipulation of are used by millions for office tasks such as presentation and the output to change constant literals in the program [46,9, document preparation as well as for specialized creative tasks 24, 39, 29]. To also relieve the initial text editing burden, for such as engineering and graphic design. Direct manipulation’s programs with graphical output a few systems provide ODP intuitive point-click-modify experience enables a large number mechanisms for program construction [31, 38, 19], akin to of people to leverage computers to create novel artifacts. drawing tools in graphics editors. Even so, experts may sometimes forgo a GUI application and All prior ODP systems presented their workflow as a mix of create their artifact using a programming language. In contrast text edits and direct manipulation. Although the selling point to the intuitive experience offered by direct manipulation GUIs, of ODP is that text edits can compensate for any missing direct text-based coding does not offer immediate and direct artifact manipulation features, the standing question is how thoroughly construction, but programmers use text-based languages to direct manipulation can subsume text-based editing. Therefore, gain flexibility not afforded by any GUI application. in this work we wonder: What kinds of programs can be constructed entirely through output manipulations? Both paradigms—direct manipulation and text-based programming—have proven useful. Naturally, there have New ODP Techniques in SKETCH-N-SKETCH been efforts to combine their distinct strengths. We extend our prior work on SKETCH-N-SKETCH [9, 19], For example, some professional creative applications do of- a programming system for creating vector graphics, with new fer a scripting API (e.g. Maya [2]), but, beyond perhaps an ODP techniques that enable the system to construct 16 ex- without initial macro recording, script construction is a text editing ample programs text edits on the code—even though process with no direct manipulation support. Furthermore, in ordinary text editing remains possible at any time during the the programming world, non-text paradigms such as blocks- construction of each example. Specifically, we: [4] or wires-based visual programming [43] allow direct ma- 1. Expose intermediate execution values for manipulation, nipulation of program elements. Even so, most professional instead of just the final output. programming remains a strictly plain text activity. 2. Offer expression focusing to enable contextual drawing. Instead, could direct manipulation augment text-based pro- 3. Expose generic code refactoring tools through output- gramming, providing user interfaces more like standard cre- directed interactions. ative application GUIs, such as graphics editors? Rather than directly manipulating program elements construed as blocks 4. Use generic run-time tracing to track value provenance, and wires, could one construct a textual program by directly to associate output values with source code locations. manipulating a program’s output? Although the SKETCH-N-SKETCH system is specialized for Output-Directed Programming (ODP) programs that output vector graphics, the four principles above We refer to this paradigm—in which a programmer constructs are relevant for future output-directed programming systems a plain text program by mouse-based operations on the pro- of all kinds. gram’s output—as output-directed programming (ODP). Each Paper Outline and Supplementary Materials operation triggers a transformation on the text-based program, To introduce the output-directed programming workflow in akin to automatic refactoring tools. Successive operations SKETCH-N-SKETCH, in the next section we walk through the build the program step by step. Because the program is plain creation of a simple example program. After this overview, we text throughout the process, desired changes that are not pos- present the core ODP mechanisms and our four innovations in sible through the provided direct manipulation interactions more detail, followed by an examination of the construction of can be enacted by ordinary text editing. The system treats the the 16 example programs. We conclude by discussing current plain text as the artifact’s primary representation so that text limitations of SKETCH-N-SKETCH, related work, and future editing does not disable future direct manipulation actions. directions for ODP. Our implementation, examples, videos, This ODP paradigm for altering a program by manipulating and appendices are available as Supplementary Materials and its output offers two tantalizing possibilities: on the Web (http://ravichugh.github.io/sketch-n-sketch/). • Creative applications could represent the user’s artifact OVERVIEW not as an opaque internal data structure but as a visible, SKETCH-N-SKETCH is a bimodal programming environment, editable, textual program. Direct manipulation of the arti- as depicted in Figure 1: the left pane is an ordinary source fact can be freely mixed with computational generation. code text editor; the right pane renders the scalable vector • For general-purpose programming, ODP could facilitate graphics (SVG) [47] design generated by the code and also general program construction via output-directed ma- offers a graphical interface for performing transformations on nipulations. For experts, these ODP interactions