Stories in GeoTime
Ryan Eccles, Thomas Kapler, Robert Harper, William Wright Information Visualization ( 2008 )
© Stefan John 2008
Summer term 2008 1
Introduction
• Story a powerful abstraction − Used by intelligence analysts to conceptualize threats and understand patterns − Part of the analytical process • Oculus Info’s GeoTime™ − Geo-temporal event visualization tool − Augmented with story system − Narratives, hypertext-linked visualizations, visual annotations, and pattern detection − Environment for analytic exploration and communication − Detects geo-temporal patterns − Integrates story narration to increase analytic sense-making cohesion
Summer term 2008 2
1 Introduction
• Assisting the analyst in: − Identifying, − Extracting, − Arranging, and − Presenting stories within the data • Story system − Lets analysts operate at story level − Higher level abstractions of data (behaviors and events) − Staying connected to the evidence − Developed in collaboration with analysts • Formal evaluation showed high utility and usability
Summer term 2008 3
Overview
• Storytelling
• Related Work
• Geo-Temporal Visualization in GeoTime
• Stories in GeoTime
• Evaluation
Summer term 2008 4
2 Storytelling
• First described in Aristotle’s Poetics − Objects of a tragedy (story): - Plot -> arrangements of incidents -Character - Thought -> processes of reasoning leading characters to their respective behavior • Narrative theory suggests: − People are essentially storytellers − Implicit ability to evaluate a story for: - Consistency -Detail - Structure
Summer term 2008 5
Storytelling
• Story offers context − Understand activities and plots played by characters or actors − Better comprehend the point being communicated by author − Helps audience to apply their tacit knowledge • Knowledge distinction − Tacit knowledge = knowledge carried by people in their minds - Difficult to access and to communicate - Not aware of − Explicit knowledge = knowledge that is easy to communicate − Process of transforming -> Articulation
Summer term 2008 6
3 Storytelling
• From analyst’s perspective: − Story offers a common form of communication − Investigating the feasibility of a connected collection of characters and their motives • Potent way of capturing the analysts’ insights − Promote sharing of observations − Facilitate understanding of complex phenomenon • Story as a mental framework -> Allows to organize daily observations into meaningful knowledge
Summer term 2008 7
Related Work
• Famous aphorism (M. Polanyi):
“We know more than we can tell.”
• Various approaches to connect information visualization with communicative power of storytelling
• Investigated in context of usage by intelligence analysts
• Common technique: Hypertext links to multimedia or additional textual content
Summer term 2008 8
4 LifeLines Project
• Allowed for incorporation of multimedia and interactions • Creation and playback of connected temporal data • Interactively linked 2D displays
LifeLines – Visualizing Personal Histories Plaisent C., Milash B., Rose A. Widoff S, Shneiderman B. Proceedings of CHI ( 1996 )
Summer term 2008 9
MyLifeBits System
• Incorporates lifetime’s media to tell a story
MyLifeBits – Fulfilling the Memex Vision Gemmell J., Bell G., Lueder R., Drucker S., Wong C. ACM Multimedia ( 2002 )
Summer term 2008 10
5 Textable Movie System
• Video content associated with text • Assemble pre-recorded video clips together to form a movie • Add narrative structure to existing content • Intended to be viewed, not manipulated or investigated further
A System to Compose Movies for Cross-Cultural Story-Telling – Textable Movie Vaucelle C., Davenport G. Summer term 2008 Proceedings of TIDSE ( 2004 ) 11
Sense.us
• Exploration of social aspects of collaborative analysis
• Combined with annotation on visualization
• Uses blog-style discussion workflow
• Basis for communicating analysis
• Annotations implemented as vector graphics overlaid above 2D visualization
Voyagers and Voyeurs – Supporting Asynchronous Collaborative Information Visualization Heer J., Viegas F.B., Wattenberg M. CHI Proceedings ( 2007 )
Summer term 2008 12
6 Sense.us
Summer term 2008 13
GeoTime
• Time-space visualization framework • Unified temporal and geospatial analysis • Designed to improve perception and understanding of: − Entity movements, − Events, − Relationships, and − Interactions over time within a geospatial context
GeoTime Information Visualization Kapler T., Wright W. IEEE InfoViz ( 2004 )
Summer term 2008 14
7 Geo-Temporal Representation
• Events represented within an X, Y, T coordinate space − X and Y plane represents geographic space − Z-axis represents temporal space
Summer term 2008 15
GeoTime
• Enables analysis of information connectedness over time and geography within a single, highly interactive 3D view
Summer term 2008 16
8 Velocity Annotation
• Movement speed function − Annotates geo-temporal path with visual indicator of velocity − Red is fast / white is slow
Summer term 2008 17
Movement Visualization
• Also annotation with icons possible • Expressing speed by indicating mode of transportation • Moving by: − Foot − Car − Aircraft
Summer term 2008 18
9 Annotation System
• Annotations also manually applied to events by analyst − Highlight events − Direct attention of viewer • Choose from palette of annotations: − Curves, arrows, callouts, outlines, enlargement • Annotations implemented as data objects referenced to actual event data − Rather than separate graphical overlay − Semantic connection • Supports interactions such as mouseovers and menus on annotation graphics
Summer term 2008 19
Annotation System
• User annotation menu with five annotation styles
Summer term 2008 20
10 Temporal Navigation
• Events animated in time through 3D space as slider bar is moved • Selection of time range displayed − Years, weeks, days, hours, minutes
• Allows perception of “who” and “what” in the “where” and “when”
Summer term 2008 21
Stories in GeoTime
• Stories system built on GeoTime time-space visualization framework • Introduces new sets of features: − Space-time pattern finding system − Search for common behaviors and relationships among events and entities − Capture management system − Stores and invokes snapshots of GeoTime space-time views • Provides specialized text editing panel (story window) − Allows analysts to author narratives and explanations − Links to bookmarked views − Essentially HTML text editor
Summer term 2008 22
11 Story Window
• Color-defined category
• Categorize content to support collaboration or multiple story threads
• Filter interface to hide or show text sections
• Text sections titled ‘Pattern:…” generated by pattern functions
• Click on link fetches and displays stored snapshot
Summer term 2008 23
Stories in GeoTime
• Supports representations for base elements of a story: − Events − People − Objects − Places − Relationships • Display elements in time and space
-> Makes it suitable platform on which to build representations and interactions with stories
Summer term 2008 24
12 Applications
• Intelligence analysis
− Criminal investigations
− Analysis tradecraft training
• Military
• Law enforcement
• Business
• Historical visualizations
Summer term 2008 25
Intelligence Data
• GPS vehicle tracks
• Personal transaction billing data
• Phone calls
• Credit card transactions
• Proprietary operational data sets
Summer term 2008 26
13 Story Visualization
• Concept sketch of annotated GeoTime scene representing events of a fictional story
Summer term 2008 27
The Little Red Riding Hood
Summer term 2008 Picture: © Hoodwinked! 2005 28
14 The Little Red Riding Hood
• Text of story "Little Red Riding Hood"
• Meeting of Wolf and Red Riding Hood in forest
• Visualize space-time paths (assuming GPS tracking data were available)
• Point out:
− Crossing paths by accident or
− Wolf was waiting deliberately
• Distinction immediately apparent in story visualization
Summer term 2008 29
Behavior Patterns
• Large volumes of low-level data − Extracting behaviors and patterns in order to understand situation − Common problem for analysts • Detection of patterns in movement activity − Discovery of possible meetings, speed of movement, percentage of time spent at unique locations, identifying gaps in observations
Summer term 2008 30
15 Sense-Making
• Evaluating alternative explanations accounting for some given evidence (important analysis technique) -> Competitive hypotheses
Summer term 2008 31
Sense-Making Loop through Story Visualization
Summer term 2008 32
• Integration of scientific disciplines to improve division of labor between human and machine
Summer term 2008 33
Story Spectrum
• Relationship between story concepts and analytical workflow
Summer term 2008 34
17 Evaluation
• Assess new capabilities and usability • Formal evaluation performed by NIST (National Institute of Standards and Technology) showed: − High utility − High usability • Experienced intelligence analysts selected • Tested authoring and asynchronous communication of intelligence reports using GeoTime Stories − Integration of narrative text, snapshots, and pattern results into text report − Understandability by others (unfamiliar with situation and data)
Summer term 2008 35
Evaluation Procedure
• Participants presented with fictional scenario • Mimic scenarios encountered in a criminal investigation • Evaluation process divided into two stages: − Stage 1: Write report analyzing information in scenario − Stage 2: New scenario + review report authored by another analyst
Summer term 2008 36
18 Example Report
• Stories-based example report generated by one of the subjects
Summer term 2008 37
Evaluation Results
• Most GeoTime Stories capabilities rated by analysts between high and highest for functionality, utility, and usability
Summer term 2008 38
19 Critique
• Feedback overall positive • Pros: − Reduce time spend by analysts on low-level data analysis and report generation − No imposed workflow − High demand for story visualization tools − “Link analytical thought and hypotheses to moments in information time and space” • Cons: − Stated concern that consumers of report require GeoTime software and training in order to view report
Summer term 2008 39
VAST 2007
• Visual Analytics Science and Technology Contest 2007 • Winning corporate entry: Oculus Info’s GeoTime • High ratings for overall utility and visualization quality
Evaluating Visual Analytics at the 2007 VAST Symposium Contest Potel M., et al. IEEE Computer Graphics and Applications ( 2008 )
Summer term 2008 40
20 Conclusion
• Storytelling effective means of communication and collaboration • Used by analysts for reporting complex events to other analysts and decision makers • Relation between story and analysis process investigated • Presented capabilities help analysts communicate insights and explanations in an expressive analytical product • Ability to perform analyses at higher level of abstraction made possible (story level)
Summer term 2008 41
Additional References
• www.oculusinfo.com
Summer term 2008 42
21 Intelligence Analysis Books
Summer term 2008 43
System Architecture
• Pattern matching system includes API • Allows third-party developers to write new plugins
Summer term 2008 44
22