The Problem Step 1: Event Generaon Experimental Evaluaon

• Google is great but not perfect: Learning about • Use knowledge base to generate events • User studies on AMT (>1200 raters)

some new topic’s history remains challenging 1-hop • Pairwise comparisons à relave judgments April 4, 1965 event • Search engines do not provide representave view DoB Baseline vs Base ● or support exploraon of topic and its relaonships Robert Downey Jr. May 4, 2012 starIn No Date vs Full−RelE2D ● Boom line: All 2-hop The Soluon: Timeline Visualizaon relDate event No Enty vs Full−RelE2E ● criteria important! The Avengers No Temp. vs FullDiv−TD ● • Summarizes most relevant events / relaonships Subject No Cont. vs FullDiv−CD ● starIn Related Entity • Interacve exploraon, e.g. zoom and topic-switch related through 0.5 0.6 0.7 0.8 0.9 1.0 Samuel L Jackson 2-hop event Timestamp • Adapts to available screen estate Fraction preferring Full (RPref) • Filtering step to exclude non-informave events • Our task: Given enty, generate meline • Web-based co-occurrence signals strongly improve over baseline (global importance) 1-hop April 4, 1965 event Robert Downey Jr. (1965—) Timeline for Robert Downey Jr. DoB Step 2: Event Selecon • Temporal and content diversity are crucial May 4, 2012 Robert Downey Jr. starIn 2-hop relDate event ingredients for good melines The Avengers Opmizaon Problem Subject

starIn Related Entity related through Samuel L Jackson 2-hop event Timestamp Deborah The Party's Ben Stiller Fiona Apple Susan Downey Iron Man 2 Iron Man 3 Falconer Over Another Example: John F Kennedy

Robert Paramount 1. Correctness (Event Generaon) Chaplin Ally McBeal Gothika Iron Man The Avengers Downey, Sr. Pictures

• Guaranteed by knowledge base & construcon 1985 1990 1995 2000 2005 2010 2015 2. Relevance Signals (Objecve)

• Global: # of search queries for enty Quality Criteria • Enty-Enty & Enty-Date co-occurrence from 1. Correctness: Only show actual events/relaonships 10B document web corpus (NER + CoRef + NPMI) 2. Relevance: Only display most “relevant” events 3. Content Diversity (Objecve) 3. Content Diversity: Display diverse set of event types • Encode diminishing returns in objecve 4. Temporal Diversity: Produce balanced layout Conclusions 4. Temporal Diversity (Constraint) • Goal: Scalable meline generaon • Enforce balanced layout during Our Approach • Challenge: Jointly opmize for relevance, opmizaon as constraint content diversity, and temporal diversity 1-hop Robert Downey Jr. (1965—) April 4, 1965 event Provable approximaon guarantee Deborah The Party's Ben Stiller Fiona Apple Susan Downey Iron Man 2 Iron Man 3 • Falconer Over Efficient algorithms with theorecal guarantees

Robert Paramount • Chaplin Ally McBeal Gothika Iron Man The Avengers Submodular opmizaon + p-system constraint DoB Downey, Sr. Pictures • User studies show that all criteria are important 1985 1990 1995 2000 2005 2010 2015 May 4, 2012 • Lazy-greedy algorithm: ≥33% of opmal soluon RobertEnty Downey Jr. List of events Timeline • Demo! cs.stanford.edu/~althoff/memachine starIn 2-hop Thanks Dafna! relDate event The Avengers Subject starIn Related Entity related through Samuel L Jackson 2-hop event Timestamp