Data Visualization Short Course 3 April 2017 Jim Wisnowski [email protected] (210) 218-1384 1 2 MCOTEA Example 3 Air Force Example ▪ Air Force Magazine Feb 2017 trends for women as a percent of the force 4 http://www.airforcemag.com/MagazineArchive/Magazine%20Documents/2017/February%202017/0217infographic.pdf Callan Chart of Sector Performance (Quilt Chart) 5 https://www.callan.com/wp-content/uploads/2017/01/Callan-PeriodicTbl_KeyInd_2017.pdf One Last Warm-Up ▪ Stephen Few is a guru in the data visualization world ▪ Let’s take his quiz on best practices at www.perceptualedge.com ▪ Goal is to get every one wrong—0/10 is success! 6 Objectives ▪ Appreciate the historical perspective of data visualization ▪ Know the value of data visualization offers to analytics and Big Data ▪ Understand what makes a good graphical display and some of the common mistakes to avoid in graphical design ▪ Be familiar with some methodologies for the data visualization process ▪ Appreciate how to do data viz with a few common software packages 7 Data Visualization is Not New Scottish political economist William Playfair in 1786 recognized superiority of graphs over tabular presentations— published 43 time series plots and one bar chart Developed the first pie chart in 1801 to show distribution of Turkish Empire over Europe, Africa, and Asia Stephen Few states we really didn’t progress much from these original ideas until late 1970s with Princeton’s John Tukey and his Exploratory Data Analysis (EDA) He argues most are unaware of modern methods 8 Data Visualization is Not New ▪ Area chart using color was masterful ▪ Playfair credited with the introduction of bar charts 9 Data Visualization is Not New 10 Exploratory Data Analysis John Tukey, Princeton, 1977 Too much emphasis on hypothesis tests as confirmatory analysis—focus should also be on discovery Objectives – Suggest hypotheses of observed data – Assess statistical test assumptions – Support selection of appropriate methods and tools – Serve as basis for further data collections and experiments If we need a short suggestion of EDA, I would suggest that – It is an attitude; a flexibility; and requires graph paper and transparencies The greatest value of a picture is when it forces us to notice what we never 11 expected to see…John Tukey Data Visualization Fuel Most important aspect of data visualization is the data itself Value goes beyond the enterprise/transactional data itself – Unstructured data, social networks, Internet of Things Data quality is key and dataviz can help improve that! Phil Simon rates organizations on visualization framework – Data (big or small) – Visualization (static or interactive) Start small and scale If we have data, let’s look at the data. If all we have are opinions, let’s go with mine. 12 Jim Barksdale, Netscape Data is Growing • Big Data is overused term, but we know there is GOLD in those data mountains • 15 Tb of Twitter daily is a lot of data generated; how much gold do we have? We are exposed to more information in a day than someone from the 15th century was over a lifetime. 90% of today’s data was created in last 2 years (IBM); 2.5 quintillion bytes per day In 2015 the number of networked devices doubles the entire global population Of interest: Tera, Peta, Exa, Zetta, Yotta, Brontabytes Graphic from IBM Research India, presented at Text Mining Workshop Jan 2014 13 Data Visualization Needs Credible Data! Do not trust any statistics you did not fake yourself…Churchill Figures don't lie, but liars do figure…Twain 14 Traits of Meaningful Data High Volume Historical Consistent Multivariate Atomic Clean Clear Dimensionally Structured Richly Segmented Of Known Pedigree Data Map and Contour Plots are “best practices” 15 Reference: Now You See It by Stephen Few Data Visualization Definition Data is the new business capital. Data visualization: discovery of solutions that offer highly interactive and graphical user interfaces, are built on in-memory architectures, and are geared toward addressing business users’ unmet ease-of- use and rapid deployment needs. These solutions typically enable users to explore data without much training, making them accessible by a wider range of employees than traditional business analysis tools. SAS Key to making “analytics” approachable is visualization – Visual thinking is essential skill for all – Both an art and science => craft (Berinato, Harvard Business Review) Data is a great but messy story; visual analytics is the master filmmaker to bring the story to life (SAS) Not a great term…was Shakespeare a word sequencer? A picture is worth a thousand data points 16 Data Visualization Characteristics (Card et al, Information Visualization) – Computer supported – Interactive – Visual representations of location, length, size, color, shape to allow us to see trends – Abstract data with no physical form (e.g. human body) Amplify cognition by assisting memory by representing data in ways our brain can easily comprehend 3 facts: Pervasiveness has raised quality expectations, Big Data is here, and the Democratization of Data 90% of data analyses required by most organizations is possible with simple data visualization methods – Excel is getting better – Boss wants to know why graphs in meetings are not nearly as pretty as she sees on fitness tracker (Berinito) Everyone in our business knows they need to visualize data, but it’s easy to do poorly. We invest in it. We want to use it right while they use it wrong. Daryl Morey 17 Interactive Data Visualization with Excel Consider recent data on automobile fuel economy from the EPA for 2017 year vehicles Attributes such as make, model, mpg, class, cylinders, transmission, valve timing etc Downloaded from http://www.fueleconomy.gov/feg/download.shtml Quick exploration with Excel Pivot Tables, Tableau, and JMP 18 Data Visualization Allows viewing of vast quantities of data quickly and efficiently Provides better insight into the business problem through discovery Generates a call to action Performs better if interactive and not static for quick stratification, drill down, and filtering Relies less on the IT department and empowers workers once they have access to the data with intuitive tools 19 www.introtopolicyinformatics.wikispaces.asu.edu Democratization of Data Viz Data visualization methods should allow employees who are not data analysts or scientists the ability to quickly and easily explore data Domain and business expertise critical to data understanding More rapidly find trends, generate hypotheses, identify inconsistencies, and determine additional data support requirements Reduce IT and analyst staff burden—everyone should be numerate Tension growing in non-data driven organizations Need to shorten the “kill-chain” of time data is collected until presented as actionable solution to decision makers – Find, Fix, Track, Target, Engage, Assess (F2T2EA) Goal: Self- Service Approachable Analytics 20 Interactive Data Visualization For All Flight misery map 21 Source: Sviokla, Harvard Business Review Police Department: Interactive Criminal Activity 22 http://www.raidsonline.com/?address= San%20Antonio%20TX San Francisco Police Department with JMP Data is sample file in jmp Use Graph Builder to plot each crime by color Add street map Add filter on station Create html with data file 23 San Francisco PD with JMP A bit more interactive is the Distribution platform Where is there a disproportionate amount of drug activity What days of the week correlate with runaways? What are some safe precincts? 24 Democratization of Data Analytics Data visualization is no longer just static charts created by IT professionals for meetings Even this graphic is outdated. Many are creating graphs continuously Source: TDWI Research, 2013 25 The Human Side of Data Visualization Huge advances in past 25 years in data collection, storage and access; have ignored the primary tool to make information meaningful—the human brain We acquire more information from vision than from all other senses combined 20 Billion neurons in brain used to form patterns from visual information The eye and visual cortex of brain form a massively parallel processor that provides highest bandwidth channel into human cognitive centers—Colin Ware, UNH We seek patterns Strive for Interocular Traumatic Impact 26 The Human Side of Data Visualization We have selective visual attention; we are drawn to familiar patterns, and our working memory is limited Jacque Bertin’s Semiologie Graphique in 1967 describes basic vocabulary of vision of abstract data – Pre-attentive attributes form the core of good data visualization methods – Pre-attentive means without prior conscious awareness—the things that “pop out” most We can only “remember” at most chunks of 3 visualizations and even then for only a short period – So don’t make comparisons difficult-like on next chart or scroll down further. Side-by-side is best. 27 Pre-Attentive Attributes Shape Length Hue/Contrast Size Position Color Enclosure Symmetry 28 Grouping Xan’s Pre-attentive Processing Quiz 29 Pre-attentive Processing 30 Graphic Attributes: Quantitative Scales Position Length Slope Area Color Hue Better Position (unaligned) Angle Color DensityWorse Based on “Graphical Perception: Theory, Experimentation, and Application …” by William Cleveland and Robert McGill, JASA, Sept. 1984 31 The Human Side of Data Visualization Color is a key pre-attentive attribute 5% Females and 9% Males are color blind – Red-Green is most