The Problem Step 1: Event Genera on Experimental Evalua on
• Google is great but not perfect: Learning about • Use knowledge base to generate events • User studies on AMT (>1200 raters)
some new topic’s history remains challenging 1-hop • Pairwise comparisons à rela ve judgments April 4, 1965 event • Search engines do not provide representa ve view DoB Baseline vs Base ● or support explora on of topic and its rela onships Robert Downey Jr. May 4, 2012 starIn No Date vs Full−RelE2D ● Bo om line: All 2-hop The Solu on: Timeline Visualiza on relDate event No En ty vs Full−RelE2E ● criteria important! The Avengers No Temp. vs FullDiv−TD ● • Summarizes most relevant events / rela onships Subject No Cont. vs FullDiv−CD ● starIn Related Entity • Interac ve explora on, e.g. zoom and topic-switch related through 0.5 0.6 0.7 0.8 0.9 1.0 Samuel L Jackson 2-hop event Timestamp • Adapts to available screen estate Fraction preferring Full (RPref) • Filtering step to exclude non-informa ve events • Our task: Given en ty, generate meline • Web-based co-occurrence signals strongly improve over baseline (global importance) 1-hop April 4, 1965 event Robert Downey Jr. (1965—) Timeline for Robert Downey Jr. DoB Step 2: Event Selec on • Temporal and content diversity are crucial May 4, 2012 Robert Downey Jr. starIn 2-hop relDate event ingredients for good melines The Avengers Op miza on Problem Subject
starIn Related Entity related through Samuel L Jackson 2-hop event Timestamp Deborah The Party's Ben Stiller Fiona Apple Susan Downey Iron Man 2 Iron Man 3 Falconer Over Another Example: John F Kennedy
Robert Paramount 1. Correctness (Event Genera on) Chaplin Ally McBeal Gothika Iron Man The Avengers Downey, Sr. Pictures
• Guaranteed by knowledge base & construc on 1985 1990 1995 2000 2005 2010 2015 2. Relevance Signals (Objec ve)
• Global: # of search queries for en ty Quality Criteria • En ty-En ty & En ty-Date co-occurrence from 1. Correctness: Only show actual events/rela onships 10B document web corpus (NER + CoRef + NPMI) 2. Relevance: Only display most “relevant” events 3. Content Diversity (Objec ve) 3. Content Diversity: Display diverse set of event types • Encode diminishing returns in objec ve 4. Temporal Diversity: Produce balanced layout Conclusions 4. Temporal Diversity (Constraint) • Goal: Scalable meline genera on • Enforce balanced layout during Our Approach • Challenge: Jointly op mize for relevance, op miza on as constraint content diversity, and temporal diversity 1-hop Robert Downey Jr. (1965—) April 4, 1965 event Provable approxima on guarantee Deborah The Party's Ben Stiller Fiona Apple Susan Downey Iron Man 2 Iron Man 3 • Falconer Over Efficient algorithms with theore cal guarantees
Robert Paramount • Chaplin Ally McBeal Gothika Iron Man The Avengers Submodular op miza on + p-system constraint DoB Downey, Sr. Pictures • User studies show that all criteria are important 1985 1990 1995 2000 2005 2010 2015 May 4, 2012 • Lazy-greedy algorithm: ≥33% of op mal solu on RobertEn ty Downey Jr. List of events Timeline • Demo! cs.stanford.edu/~althoff/ memachine starIn 2-hop Thanks Dafna! relDate event The Avengers Subject starIn Related Entity related through Samuel L Jackson 2-hop event Timestamp