Nodexl, Gephi, Tom Sawyer •

Ensuring Human Control & Responsibility While Increasing Automation: Design Principles for Supporting Insight & Reducing Errors

Ben Shneiderman [email protected] @benbendc

Founding Director (1983-2000), Human-Computer Interaction Lab Professor, Department of Computer Science Member, Institute for Advanced Computer Studies

University of Maryland College Park, MD 20742

Enabling Teams of Humans to Harness Networks of Machines

Ben Shneiderman [email protected] @benbendc

Founding Director (1983-2000), Human-Computer Interaction Lab Professor, Department of Computer Science Member, Institute for Advanced Computer Studies

University of Maryland College Park, MD 20742

Interdisciplinary research community - Computer Science & Info Studies - Psych, Socio, Poli Sci & MITH

(www.cs.umd.edu/hcil)

Design Issues

• Input devices & strategies • Keyboards, pointing devices, voice • Direct manipulation • Menus, forms, commands • Output devices & formats • Screens, windows, color, sound • Text, tables, graphics • Instructions, messages, help • Collaboration & Social Media www.awl.com/DTUI • Help, tutorials, training Fifth Edition: 2010 • Search • Visualization HCI Pride: Serving 5B Users

Mobile, desktop, web, cloud

 Diverse users: novice/expert, young/old, literate/illiterate, abled/disabled, cultural, ethnic & linguistic diversity, gender, personality, skills, motivation, ...

 Diverse applications: E-commerce, law, health/wellness, education, creative arts, community relationships, politics, IT4ID, policy negotiation, mediation, peace studies, ...

 Diverse interfaces: Ubiquitous, pervasive, embedded, tangible, invisible, multimodal, immersive/augmented/virtual, ambient, social, affective, empathic, persuasive, ...

Integrating Humans, Machines & Networks

1) key research goals as they relate to the abilities of humans, machines and networks to share the cognitive load to make decisions

2) relevant milestones that have been reached in subfields

3) relevant impediments to achieving technological breakthroughs

4) systems-integration challenges to improving data-to-decision capabilities

5) the scope & character of international approaches

6) policy implications of international research for the U.S.

Enabling Teams of Humans to Harness Networks of Machines

My position: Teams & Humans have initiatives, goals, and responsibility

Machines & Networks are remarkable tools, operated, maintained & designed by teams & humans

Enabling Teams of Humans to Harness Networks of Machines

My position: Teams & Humans have initiatives, goals, and responsibility

Machines & Networks are remarkable tools, operated, maintained & designed by teams & humans

More effective systems recognize the differences between humans & machines Clarifying responsibility supports continuous improvement Visual presentations provide high bandwidth, while simple clicks/gestures enable rapid operation

Enabling Teams of Humans to Harness Networks of Machines

Research agenda: - better design for individuals - easier collaboration for groups - effective social mechanisms for teams/organizations/communities Visual design promotes: - sense-making - situation awareness - accurate decision-making in stressful distracting situations Metrics expand: - peta-flops, tera-bytes & giga-hertz - peta-contribs, tera-thank-yous & giga-collabs

Information Visualization

• Visual bandwidth is enormous • Human perceptual skills are remarkable • Trend, cluster, gap, outlier... • Color, size, shape, proximity... • Goals • Detect errors & understand source data • Develop & test hypotheses • Make insights & support decisions • Collaborate & persuade • Challenges • Meaningful visual displays of massive data • Interaction: widgets & window coordination • Process models for discovery

Information Visualization & Visual Analytics

• Visual bands • Human percle • Trend, clus.. • Color, size,..

• Three challe • Meaningful vi • Interaction: w • Process mo

1999 Information Visualization & Visual Analytics

• Visual bandwidth is enormous • Human perceptual skills are remarkable • Trend, cluster, gap, outlier... • Color, size, shape, proximity...

• Three challenges • Meaningful visual displays of massive da • Interaction: widgets & window coordinati • Process models for discovery

1999 2004 Information Visualization & Visual Analytics

• Visual bandwidth is enormous • Human perceptual skills are remarkable • Trend, cluster, gap, outlier... • Color, size, shape, proximity...

• Three challenges • Meaningful visual displays of massive data • Interaction: widgets & window coordination • Process models for discovery

1999 2004 2010 Business takes action

• General Dynamics buys MayaViz • Agilent buys GeneSpring • Google buys Gapminder • Oracle buys Hyperion • Microsoft buys Proclarity • InfoBuilders buys Advizor Solutions • SAP buys (Business Objects buys Xcelsius & Inxight & Crystal Reports ) • IBM buys (Cognos buys Celequest) & ILOG • TIBCO buys Spotfire

Spotfire: Retinol’s role in embryos & vision Spotfire: DC natality data http://registration.spotfire.com/eval/default_edu.asp 10M - 100M pixels: Large displays 100M-pixels & more 1M-pixels & less Small mobile devices Information Visualization: Mantra

• Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand • Overview, zoom & filter, details-on-demand

Information Visualization: Data Types

. • 1-D Linear Document Lens, SeeSoft, Info Mural

• 2-D Map GIS, ArcView, PageMaker, Medical imagery SciViz SciViz

• 3-D World CAD, Medical, Molecules, Architecture

• Multi-Var Spotfire, Tableau, Qliktech, Visual Insight • Temporal LifeLines, TimeSearcher, Palantir, DataMontage

• Tree Cone/Cam/Hyperbolic, SpaceTree, Treemap InfoViz InfoViz Network Pajek, UCINet, NodeXL, Gephi, Tom Sawyer •

infosthetics.com visualcomplexity.com eagereyes.org flowingdata.com perceptualedge.com datakind.org visual.ly Visualizing.org infovis.org Obama Unveils “Big Data” Initiative (3/2012)

Big Data challenges: • Developing scalable algorithms for processing imperfect data in distributed data stores

• Creating effective human- computer interaction tools for facilitating rapidly customizable visual reasoning for diverse missions.

http://www.whitehouse.gov/sites/default/files/microsites/ostp/big_data_press_release_final_2.pdf ` Temporal Data: TimeSearcher 1.3

• Time series • Stocks • Weather • Genes • User-specified patterns • Rapid search

Temporal Data: TimeSearcher 2.0

• Long Time series (>10,000 time points) • Multiple variables • Controlled precision in match (Linear, offset, noise, amplitude)

LifeLines: Patient Histories

www.cs.umd.edu/hcil/lifelines LifeLines2: Contrast+Creatine LifeLines2: Align-Rank-Filter & Summarize LifeFlow: Aggregation Strategy

Temporal Categorical Data (4 records)

LifeLines2 format

Tree of Event Sequences

LifeFlow Aggregation

www.cs.umd.edu/hcil/lifeflow LifeFlow: Interface with User Controls

Treemap: Gene Ontology

+ Space filling + Space limited + Color coding + Size coding - Requires learning

(Shneiderman, ACM Trans. on Graphics, 1992 & 2003) www.cs.umd.edu/hcil/treemap/ Treemap: Smartmoney MarketMap

www.smartmoney.com/marketmap Market falls steeply Feb 27, 2007, with one exception Market falls steeply Sept 22, 2011, some exceptions Market mixed, February 8, 2008 Energy & Technology up, Financial & Health Care down Market rises, September 1, 2010, Gold contrarians Market rises, March 21, 2011, Sprint declines Treemap: Newsmap (Marcos Weskamp)

newsmap.jp Treemap: WHC Emergency Room (6304 patients in Jan2006)

Group by Admissions/MF, size by service time, color by age Treemap: WHC Emergency Room (6304 patients in Jan2006) (only those service time >12 hours)

Group by Admissions/MF, size by service time, color by age Treemap: Supply Chain

www.hivegroup.com Treemap: Nutritional Analysis

www.hivegroup.com Treemap: Spotfire Bond Portfolio Analysis

www.spotfire.com Treemap: NY Times – Car&Truck Sales

www.cs.umd.edu/hcil/treemap/ Treemap (Voronoi): NY Times - Inflation

www.nytimes.com/interactive/2008/05/03/business/20080403_SPENDING_GRAPHIC.html

VisualComplexity.com : Manuel Lima Discovery Process: Systematic Yet Flexible

Preparation • Own the problem & define the schedule • Data cleaning & conditioning • Handle missing & uncertain data • Extract subsets & link to related information

SocialAction

• Integrates statistics & visualization

• 4 case studies, 4-8 weeks (journalist, bibliometrician, terrorist analyst, organizational analyst) • Identified desired features, gave strong positive feedback about benefits of integration

www.cs.umd.edu/hcil/socialaction Perer & Shneiderman, CHI2008, IEEE CG&A 2009 Network from Database Tables

www.centrifugesystems.com NodeXL: Network Overview for Discovery & Exploration in Excel

www.codeplex.com/nodexl NodeXL: Network Overview for Discovery & Exploration in Excel

www.codeplex.com/nodexl NodeXL: Import Dialogs

www.codeplex.com/nodexl Tweets at #WIN09 Conference: 2 groups ‘GOP’ tweets, clustered (red-Republicans) Twitter networks: #SOTU WWW2010 Twitter Community Twitter Network for “TTW” Twitter Network for #CI2012 No Location Philadelphia Innovation Clusters: People, Locations, Companies

11,000 nodes 26,000 links

Pharmaceutical/Medical

Pittsburgh Metro

Westinghouse Electric No Location Philadelphia Innovation Clusters: People, Locations, Companies

Pharmaceutical/Medical

Pittsburgh Metro

Westinghouse Electric No Location Philadelphia Innovation Clusters: People, Locations, Companies

Patent Tech Navy SBIR (federal) PA DCED (state) Related patent 2: Federal agency Pharmaceutical/Medical 3: Enterprise

Pittsburgh Metro 5: Inventors 9: Universities 10: PA DCED 11/12: Phil/Pitt metro cnty

13-15: Semi-rural/rural cnty 17: Foreign countries Westinghouse Electric 19: Other states

CHI2010 Twitter Community

www.codeplex.com/nodexl/ Flickr networks Analyzing Social Media Networks with NodeXL

I. Getting Started with Analyzing Social Media Networks 1. Introduction to Social Media and Social Networks 2. Social media: New Technologies of Collaboration 3. Social Network Analysis

II. NodeXL Tutorial: Learning by Doing 4. Layout, Visual Design & Labeling 5. Calculating & Visualizing Network Metrics 6. Preparing Data & Filtering 7. Clustering &Grouping

III Social Media Network Analysis Case Studies 8. Email 9. Threaded Networks 10. Twitter 11. Facebook 12. WWW 13. Flickr 14. YouTube 15. Wiki Networks

www.elsevier.com/wps/find/bookdescription.cws_home/723354/description Social Media Research Foundation

Researchers who want to - create open tools - generate & host open data - support open scholarship

Map, measure & understand social media

Support tool projects to collection, analyze & visualize social media data.

smrfoundation.org Sense-Making Loop

Thomas & Cook: Illuminating the Path (2004) Sense-Making Loop: Expanded

Thomas & Cook: Illuminating the Path (2004) Discovery Process: Systematic Yet Flexible

Preparation • Own the problem & define the schedule • Data cleaning & conditioning • Handle missing & uncertain data • Extract subsets & link to related information

Discovery Process: Systematic Yet Flexible

Preparation • Own the problem & define the schedule • Data cleaning & conditioning • Handle missing & uncertain data • Extract subsets & link to related information

Purposeful exploration – Hypothesis testing • Range & distribution • Relationships & correlations • Clusters & gaps • Outliers & anomalies • Aggregation & summary • Split & trellis • Temporal comparisons & multiple views • Statistics & forecasts

Discovery Process: Systematic Yet Flexible

Preparation • Own the problem & define the schedule • Data cleaning & conditioning • Handle missing & uncertain data • Extract subsets & link to related information

Situated decision making - Social context • Annotation & marking • Collaboration & coordination • Decisions & presentations CHMNI Agenda

• Brain-computer interface • Machine learning • Natural language dialogue • Sensing & perception • Software agents • Cognitive & social science CHMNI Agenda: Extended

• Brain-computer interface • Visualization • Machine learning • Collaboration • Natural language dialogue • Social media networks • Sensing & perception • Persuasion & Motivation • Software agents • Trust & Responsibility • Cognitive & social science • History-keeping & Logging • Continuous improvement

Research Agenda in Visualization

• Presenting complex information to diverse users on small mobile devices • Offering busy users timely information in the right format to support rapid and accurate decision making • Providing decision-makers powerful temporal and geo-spatial tools to detect patterns or subtle changes over years. • Thorough data cleaning to cope with missing values, duplicated records, incorrect data entry, patient name entity resolution, and proper date-time stamps. • Efficient and effective anonymization and de-identification algorithms so data sets can be made more widely available, while protecting patient privacy. • Scaling visualization techniques to billions of records by filtering and dynamically forming aggregated values. This supports retrospective analyses by researchers who may seek to compare across patients, physicians, hospitals, time periods, and geographic regions. • Systematic yet flexible visual analytics processes that promote complete coverage of data and analyses questions, while preserving the option of exploration in depth when novel insights are found. • Logging of complex sequences of visual analytics operations so users know what was done to produce results, can save these workflows to reapply to new datasets, and can share the workflows with colleagues. • Collaborative insight gathering methods so that multiple analysts can work independently and then integrate their findings. Team-oriented medical decision-making is especially challenging since legal liability is involved so all team members must signal their concurrence. • Better guidelines for presenting insights to diverse audiences. Interactive visualization results, even color-coded sortable tables, when well-designed can be enormously helpful to many users. UN Millennium Development Goals

To be achieved by 2015 • Eradicate extreme poverty and hunger • Achieve universal primary education • Promote gender equality and empower women • Reduce child mortality • Improve maternal health • Combat HIV/AIDS, malaria and other diseases • Ensure environmental sustainability • Develop a global partnership for development

30th Anniversary Symposium May 22-23, 2013

www.cs.umd.edu/hcil

For More Information

• Visit the HCIL website for 650 papers & info on videos www.cs.umd.edu/hcil • Conferences & resources: www.infovis.org • See Chapter 14 on Info Visualization Shneiderman, B. and Plaisant, C., Designing the User Interface: Strategies for Effective Human-Computer Interaction: Fifth Edition (2010) www.awl.com/DTUI • Edited Collections: Card, S., Mackinlay, J., and Shneiderman, B. (1999) Readings in Information Visualization: Using Vision to Think Bederson, B. and Shneiderman, B. (2003) The Craft of Information Visualization: Readings and Reflections For More Information

• Treemaps • HiveGroup: www.hivegroup.com • Smartmoney: www.smartmoney.com/marketmap • HCIL Treemap 4.0: www.cs.umd.edu/hcil/treemap

• Spotfire: www.spotfire.com • TimeSearcher: www.cs.umd.edu/hcil/timesearcher • NodeXL: nodexl.codeplex.com • Hierarchical Clustering Explorer: www.cs.umd.edu/hcil/hce

• LifeLines2: www.cs.umd.edu/hcil/lifelines2 • Similan: www.cs.umd.edu/hcil/similan