Designing Multiple Relation Visualizations: Case Studies from Text Analytics Christopher Collins University of Ontario Institute of Technology Duke University 5 April, 2013 Outline 2 Introduction Research Method Multiple Relation Visualization & Spatial Rights Selected Projects Looking to the Future Linguistic Visualization Divide Card et al., 1999 Linguistic Visualization Divide “the gulf separating sophisticated natural language processing algorithms and data structures from state-of-the-art interactive visualization design” “the gulf separating sophisticated natural language processing algorithms and data structures from state-of-the-art interactive visualization design” Bridging the Divide Can linguistic data and algorithms, designed for NLP and CL purposes be re-appropriated to drive interactive visualizations of language? Successful InfoVis 7 Grounded in Reality Generalizable Research Method 8 Meet with data Evaluate experts discoveries •Basic grasp of data Design initial •Meet with data and questions visualization(s) experts for review Acquire the data Revise •Analysis infrastructure visualization(s) •Meet with data experts for review Grounded evaluation is a process that attempts to ensure that the evaluations of information visualizations are situated within the context of intended use. Grounded Evaluation for Information Visualizations Isenberg, P.; Zuk, T.; Collins, C. ; Carpendale, S. Proc. ACM BELIV 2008 9 design evaluation implementation design evaluation implementation Types of Evaluation design evaluation implementation Pre-design evaluation of context informs Grounded evaluation of design and implementation Outline 13 Introduction Research Method Multiple Relation Visualization & Spatial Rights Selected Projects Looking to the Future Multi-Dimensional Data 14 Heer & boyd, 2005 Elmqvist et al., 2008 Multi-Dimensional Data 15 Heer & boyd, 2005 Elmqvist et al., 2008 Multi-Dimensional Data 16 Heer & boyd, 2005 Elmqvist et al., 2008 17 Spatial Rights “The assignment of position as a visual encoding for a data dimension or relation.” Collins and Carpendale, 2007 Spatial Encoding 18 Selective, Spatial Encoding 19 Selective, Associative, Spatial Encoding 20 Selective, 7 Associative, 6 5 Ordered and Quantitative, 4 3 2 1 1 2 3 4 5 6 7 Spatial Encoding 21 Selective, Associative, Ordered and Quantitative, Long in length , Spatial Encoding 22 Selective, Associative, Ordered and Quantitative, Long in length , “Preattentive" Outline 23 Introduction Research Method Multiple Relation Visualization & Spatial Rights Selected Projects Looking to the Future Bubble Sets: Revealing Set Relations with Isocontours over Existing Visualizations 24 Christopher Collins, Gerald Penn, and Sheelagh Carpendale, InfoVis 2009 Simultaneous Spatial Rights Primary Set Spatial Rights for Non-set Relation: Colouring Primary Set Hybrid Spatial Rights: Rearrange Primary Set Spatial Rights for Set Relation Primary Set Spatial Rights for Non-set Relation: Convex Hull Primary Set Spatial Rights for Non-set Relation: Bubble Set Primary Set 34 Motivating Problem Machine Translation Parse Trees Ethnographic Study 35 Contextual interviews with research team Participatory observation Artifact analysis Goal is to describe meaning not make statistical inference Thanks to our collaborators: Kevin Knight’s Statistical MT Research group @ ISI, University of Southern California Qualitative Analysis 36 Open Coding 6 interviews X 60-90 minutes transcribe code for interesting statements actively work against bias (don’t only seek answers to pre-determined questions) refine code set and review (Strauss & Corbin, 1998) Complexity Brings Externalization 37 Very complex or very large: Cannot Complex: practically sketch sketch to aid manually understanding Simple: analyze internally Visualization `in the Wild’ 38 Some visualizations were in common use Large binders of parse trees and data tables Mostly in printed form Individual ad hoc design and usage Periodic analysis tasks mixed with programming Non-interactive Vocabulary differs from InfoVis community 39 Cooperative Design Sketches 40 Knight, 2007 Collins, 2008 Bubble Sets 41 Bubble Sets for Machine Translation Convex vs. Concave 42 Surface Routing 43 Order Effects 44 2 4 3 4 3 1 1 5 5 2 Order Effects 45 2 4 3 4 3 1 1 5 5 2 Order Effects 46 2 4 3 4 3 1 1 5 5 2 Calculating the Contour 48 ? Calculating Contours Calculating Contours Find Set Items Calculating Contours Derive Virtual Edges Calculating Contours Derive Active Region Calculating Contours 53 Calculate energy in active region Calculating Contours 54 Marching squares to find the contour Labelling Clusters 55 Labels along longest unobstructed virtual edge (before calculating energy) Heuristic Optimizations 56 Iterate until connected: 1-N: Reduce contour threshold: O(H+W) N-M: Increase positive weights and recalculate field: O(HWK) For region of area H by W pixel groups containing K items. 58 Additional Applications 59 Hans Rosling, GapminderTrendalyzer 60 61 62 Hans Rosling, GapminderTrendalyzer 63 64 InfoVis Papers Over Time ([L]-Tat, 2007, [R]-Bubble Sets) 65 Challenges 66 Stability + speed → - stability (‘jiggle’ caused by super-pixel sizes) ‘hysteresis’ on virtual edges may reduce changes Incremental updates recalculate surface only over subsets treat proximal items as a super-node Obstacle Avoidance Failure Cannot guarantee non-set members are avoided When to give up trying? “by using the bubble sets representation [...] the researchers noticed the missing context information” Smith, A.; Xu, W.; Sun, Y.; Faeder, J.R.; Marai, G.E. RuleBender: Integrated Visualization for Biochemical Rule-Based Modeling. IEEE Biovis Symposium 2011 [Best Paper Winner] -> University of Pittsburg Strobelt, H.; Braun, J.; Deussen , O.; Groth, U.; Mayer, T.; Merhof, D. HiTSEE: A Visualization Tool for Hit Selection and Analysis in High-Throughput Screening Experiments. IEEE Biovis Symposium 2011 [Best Paper Runner Up] -> University of Konstanz BioVis Symposium 2011 VisLink: Revealing Relationships Amongst Visualizations Christopher Collins and Sheelagh Carpendale, InfoVis 2007 Chang, M.-W.; Collins, C. 68 Exploring Entities in Text using Descriptive Non-Photorealistic Rendering. Pacific Vis, 2013. Brighton and Kirby, 2006 69 Linguistic Visualization Divide (Collins, 2007) (Heer, 2006 [prefuse]) & (Fry, 2004) Understanding Multiple Relations What is the relationship… across different views of the same data? across different relations in the same dataset? across multiple relations and datasets? VisLink Overview Any number of 2D visualizations, each on its own plane in 3D space Adjacent planes connected by bundled edges Shortcuts and constrained widgets aid usability Enables powerful inter-visualization queries Formalizing Multiple Relations Visualizations Dataset Relation Visualization Conference Attendee Data Professor / Student Node-link social network graph Formalism for Multiple Relationship Visualization Comparison Formalizing Multiple Relations Visualizations Dataset Relation Visualization DA Formalism for Multiple Relationship Visualization Comparison Formalizing Multiple Relations Visualizations Dataset Relation Visualization R D DA A ( A ) Formalism for Multiple Relationship Visualization Comparison Formalizing Multiple Relations Visualizations Dataset Relation Visualization R D Vis → R D DA A ( A ) A A ( A ) Formalism for Multiple Relationship Visualization Comparison Formalizing Multiple Relations Visualizations Relation RA (DA ) Dataset Visualization Vis → R D DA A A ( A ) Formalism for Multiple Relationship Visualization Comparison Formalizing Multiple Relations Visualizations Relation RA (DA ) Dataset Visualization Relation D R D Vis → R D A B ( A ) A A ( A ) Formalism for Multiple Relationship Visualization Comparison Formalizing Multiple Relations Visualizations Relation Visualization VisA → RA (DA) RA (DA ) Dataset Visualization Relation VisB → RA(DA ) D R D A B ( A ) Formalism for Multiple Relationship Visualization Comparison Formalizing Multiple Relations Visualizations Relation Visualization VisA → RA (DA) RA (DA ) Dataset Visualization Relation VisB → RA(DA ) D R D A B ( A ) Visualization Formalism for Multiple Relationship VisualizationVisC → RB (DA C)omparison Multiple Relation Visualizations Individual Visualizations Coordinated Views Compound Graphs Semantic Substrates VisLink Formalism for Multiple Relationship Visualization Comparison Individual Visualizations Any datasets, relations, and visualizations Manually compare e.g. different charts in Excel Formalism for Multiple Relationship Visualization Comparison Coordinated Views Formalism for Multiple Relationship Visualization Comparison Coordinated Views VisA → RA (DA ) Formalism for Multiple Relationship Visualization Comparison Coordinated Views VisA → RA (DA ) VisA → RB (DA ) Formalism for Multiple Relationship Visualization Comparison Coordinated Views VisA → RA (DA ) VisA → RB (DA ) Any datasets, relations, and visualizations Interactive highlighting e.g., Snap-Together Visualization (North & Shneiderman, 2000) Formalism for Multiple Relationship Visualization Comparison Compound Graphs Formalism for Multiple Relationship Visualization Comparison Compound Graphs VisA → RA (DA ) Formalism for Multiple Relationship Visualization Comparison Compound Graphs VisA → RA (DA ) + RB (DA ) Formalism for Multiple Relationship Visualization Comparison Compound Graphs VisA → RA , RB (DA ) Secondary relation has no spatial