Visualisation and Its Importance in the Analysis of Big Data Stefano De Francisci
Total Page:16
File Type:pdf, Size:1020Kb
Visualisation and its importance in the analysis of Big Data Stefano De Francisci THE CONTRACTOR IS ACTING UNDER A FRAMEWORK CONTRACT CONCLUDED WITH THE COMMISSION Eurostat Outline Introduction Visual cognition process Graphic information processing Big Data visualization Storytelling Eurostat Outline Introduction Visual cognition process Graphic information processing Big Data visualization Storytelling Eurostat To be learned #1: Data-ink ratio “Above all else show the data” (Edward Tufte) http://www.infovis-wiki.net/index.php/Data-Ink_Ratio Eurostat To be learned #2: Visual information-seeking mantra “Overview first, zoom and filter, then details-on-demand” (Ben Shneiderman) 1. Overview 2. Zoom 3. Filter 4. Details-on-demand 5. Relate 6. History 7. Extracts Eurostat To be learned #3: Storytelling, or the weaving of a narrative “Don't just get to the point – start with it” (David Marder) Gapminder «We all have important stories to (Hans Rosling) tell. Some stories involve numbers» «Data are not intrinsically boring. Neither are intrinsically interesting.» (S. Few) NoiItalia (Istat) «Behind the scene stands the Young people and storyteller, but behind the storyteller the jobs crisis in stands a community of memory» numbers (H. Arendt) (Oecd) Eurostat Why we need to visualize information? «By visualizing information, we turn it Not to get lost in into a landscape that you can explore with the data your eyes, a sort of information map. And when you’re lost in information, an information map is kind of useful.» (D. McCandless) http://www.brainyquote.com/quotes/quotes/d/davidmccan630550.html Visualization of [Big] Data gives you a way So… we can quote «to find things that you had no theory Baudelaire: about and no statistical models to identify, «Le beau est toujours but with visualization it jumps right out at bizarre» you and says, ‘This is bizarre’ » (B. Stensrud) http://bollier.org/sites/default/files/aspen_reports/InfoTech09.pdf Eurostat Reality and representation In Plato’s allegory of …in the same the cave the shadows way, the data is of real objects is all all the analyst the spectator sees. sees, and the world Real visualization of that data is all the viewer sees. But shadows are not Data reality But which kind of visualization could better represent the data? Visual representation Eurostat http://www.worldmapper.org/animations/wm01to02.html Eurostat Importance of visualization: Anscombe's quartet Eurostat http://en.wikipedia.org/wiki/Anscombe's_quartet Eurostat http://en.wikipedia.org/wiki/Anscombe's_quartet Eurostat Some definitions Visualization Information visualization The act or process of interpreting in The use of computer-supported, visual terms or of putting into visible interactive, visual representations of form Merriam-Webster Dictionary abstract data to amplify cognition Visualization is both a process of S.K. Card, J.D. Mackinlay,B. Shneiderman presentation and discovery Readings in Information Visualization: Using Vision to Think, Morgan Kaufmann, 1999 J. C. Roberts, Display Models - ways to classify visual representations, International Journal of Computer Integrated Design and Construction, 2000 Visual analytics Visual Analytics Systems The science of analytical reasoning Systems for visual analysis of large, facilitated by interactive visual complex data integrate computational interfaces knowledge discovery with interactive J.J. Thomas, K.A. Cook, eds. Illuminating the path: The visualization research and development agenda for visual analytics. T. Schreck, D. Keim, Visual Analysis of Social IEEE Computer Society Press, 2005 Media Data,Computer, vol. 46, no. 5, 2013 Eurostat Outline Introduction Visual cognition process Graphic information processing Big Data visualization Storytelling Eurostat Visual cognition process. A general overview Visual cognition process The visual cognition process can help people to understand the sense of data, solve problems and make decisions Phases Aims • Representation: a mapping • Communicate and explain from raw data to a visible (known ideas) representation • Analyse and discover • Interaction: changing what is (unknown reality) immediately viewable • Evaluate and decide (on • Presentation: organizing this the basis of graphical visible representation into the evidences) space available Eurostat Visual cognition building blocks Raw Data Visual Patterns Visual representation Visual interaction Visual Visual Visual Visual synthesis presentation exploration analysis Infographic GAV Storytelling Dashboard Sense-making Understanding Problem-solving Making decisions Knowledge cristallization Knowledge discovery Eurostat Visual cognition pipeline Visual interaction Exploring systems Problem- Raw solving Data Visual interaction Dashboard Sense- Visual Making analysis Visual exploration Visual Visual Under- Visual presentation Visual synthesis representation standing Patterns Visual interaction Making Infographic decisions Visualization practitioner Storytelling User Analyst Eurostat Visual cognition process. Conceptual models KNOWLEDGE IMAGE etal. ABSTRACTION Mi. Mi. Chen M.S.T. Carpendale M.S.T. DATA DATA IMAGE etal. DATA KNOWLEDGE Keim C.Ware D. D. ABSTRACTION Eurostat Outline Introduction Visual cognition process Graphic information processing Big Data visualization Storytelling Eurostat The essence of visual representation A few words are enough to explain the act of visualization “Visualization is about seeing the unseen and gaining an understanding of the underlying information” (J.C.Roberts) https://kar.kent.ac.uk/21931/1/display_models-ways_roberts.pdf Eurostat The generic dataflow model of visual representation Raw data are preprocessed… …filtered… to select the required information …mapped… into an Abstract Visualization Object …rendered to generate the image Eurostat The essence of visual analytics The (visual) analytics process can be seen as the combination of two human activities that enable to discovery knowledge into the world complex information space: to see and to think “Discovery consists of seeing what everybody has seen and thinking what nobody has thought.” (A. von Szent-Gyorgyi ) Static, Dynamic, rational, instinctive, passive interactive Eurostat Goals underlying visual analytics The goal of visual analytics is the creation of tools and techniques to enable people to: • Synthesize information and derive insight from massive, dynamic, ambiguous and often conflicting data. • Detect the expected and discover the unexpected. • Provide assessments that can be timely, defensible, and understandable. • Communicate assessment effectively for action J.J. Thomas, K.A. Cook, eds. Illuminating the path: The research and development agenda for visual analytics. IEEE Computer Society Press, 2005 Eurostat The essence of visual communication Our sense of pattern is an extremely important part of all intellectual activity, and the externalization and representation of pattern in visual communication is... at the core of many advances. (D. Sless ) “Graphic communication involves transcribing and telling others what you have discovered. Its aim: rapid perception and, potentially, memorization of the overall information. Its imperative: simplicity” (J.Bertin) Eurostat Information visualization uses Scenarios Core Roots Maxim Communication Presentation Organizing as well as possible Computer “Above all else show the visible representation into Graphics + the space available Information the data” E. Tufte Design ( ) Analysis Interaction “Overview first, zoom Dialog between the user and and filter, then details- the system as the user HCI+KDD explores the data set to on-demand” B. Shneiderman uncover insights ( ) Decision Support Synthesis/Evaluation Dashboard + “Eloquence through KPI + Measuring and monitoring the Balance simplicity” S. Few achievement of fixed objectives Scorecard ( ) Eurostat Information visualization solutions Infographic, information design, The art and science of preparing and presenting the information so that they can be used by humans in an efficient and effective way Geo-analytics visualization systems Graphic techniques to analyze and make sense of the data Dashboard Graphic techniques to measure and monitor relevant data of an organization, in order to achieve their fixed objectives Eurostat Infographic Information graphics or infographics are graphic visual representations of information, data or knowledge intended to present complex information quickly and clearly. They can improve cognition by utilizing graphics to enhance the human visual system’s ability to see patterns and trends. http://en.wikipedia.org/wiki/Infographic Infographics examples • Tables • Flow diagrams • Conceptual maps • Hystograms • Graphs • Maps • Topographic maps • Shemas • Signage Systems Eurostat Infographic. Two styles All in one Scrolling screenshot down Infographics as Infographics as snapshot storytelling http://www.coolinfographics.com/blog/ http://www.dailyinfographic.com/100- 2015/4/17/your-life-in-weeks.html years-of-change Eurostat Visual analytics Human Visual analytics integrates scientific disciplines to improve the Machine division of labor between human and machine. Eurostat Visual analytics Techniques that take advantage Techniques that enable of the human eye’s broad users to obtain deep bandwidth pathway insights that into the mind to directly support allow users to see, assessment, explore, and planning, understand large and decision amounts of making Visual information at once Analytics Techniques to present the results of an analysis That