Linking Wikipedia Events to Past News
SIGIR 2014 Workshop on Temporal, Social and Spatially-aware Information Access
Arunav Mishra, Dragan Milchevski, and Klaus Berberich Today’s WIKIPEDIA & Yesterday’s NEWS Linking Year Page Events to News
1987 Archives
This article is about the year 1987. For the Books in Brief: Nonfiction By Tammy Sill Nesmith number, see 1987 (number). For other uses, Published: October 31, 1999 see 1987 (disambiguation). Aretha From These Roots. Events[edit] By Aretha Franklin and David Ritz. Villard, $25.
''I could hear the words my father had spoken to me when I was a child: 'One day January[edit] you will sing for kings and queens,' ''Aretha Franklin recalls about an early 1980's London gala attended by the Prince of Wales and the Queen Mother. She has also •January 1 – Frobisher Bay, Northwest Territories, sung in the White House for Presidents Jimmy Carter and Bill Clinton, although she turned down a ball in Monte Carlo for Princes Albert and Rainier and a performance changes its name to Iqaluit. for Queen Beatrix of the Netherlands. The reason? Her fear of flying. Franklin -- •January 2 – Chadian–Libyan conflict – Battle of Fada: who is 57 years old, has won 15 Grammy Awards and in 1987 became the first woman inducted into the Rock and Roll Hall of Fame (though her fear The Chadian army destroys a Libyan armored brigade. of flying also kept her from that ceremony) -- has written a self-congratulatory yet entertaining autobiography. The fourth of five children born to a famous Baptist •January 3 – Aretha Franklin becomes the first woman preacher and a nurse's aide, she began singing gospel in her father's Detroit church. At the age of 14, she gave birth to the first of her four children. At 16, after the birth inducted into the Rock and Roll Hall of Fame. of her second child, she dropped out of high school. She writes openly of her two divorces and such failed romances as her on-and-off relationship with the •January 4 – 1987 Maryland train collision: Temptations' Dennis Edwards and with a man she refers to as Mr. Mystique: An Amtrak train en route from Washington, ''Because he is a public figure, I prefer to protect his privacy.'' However, she is open about her feuds with other singers, like Gladys Knight and Cissy Houston. Despite D.C. to Boston, Massachusetts collides the emotional upheavals and financial problems, Franklin sang the aria ''Nessun dorma'' at the 1998 Grammy telecast when Luciano Pavarotti took ill, and has with Conrail engines at Chase, Maryland, killing 16. recorded ''A Rose Is Still a Rose,'' written by Lauryn Hill, which seems the perfect ending for this bittersweet yet sassy chronicle. Tammy Sill Nesmith
Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 3 Retrieval Task
Goal: Given a query event , retrieve a ranked list of news articles providing details
Given, Query Event Ranked list of Documents Books in Brief: Nonfiction January 3, 1987 Published: October 31, 1999 Aretha- From These Roots. retrieve Aretha Franklin …The reason? Her fear of flying. becomes the first Franklin -- who is 57 years old, has won 15 Grammy Awards and in 1987 woman inducted became the first woman inducted into the Rock and into the Rock and Roll Hall of Fame Roll Hall of Fame. (though her fear of flying also kept her from that ceremony) …
Temporal expression : • Discrete notion of time where smallest unit equals 1 day • is an interval in time domain • Example, May 2014 = [2014.05.01, 2014.05.31]
Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 4 Overview
✓ 1 Approaches
2 Experiments
3 Summary
Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 5 Text-Only (LM)
Books in Brief: Nonfiction Query Event Document Published: October 31, 1999 Aretha- From These Roots. January 3, 1987 …The reason? Her fear of flying. Aretha Franklin Franklin -- who is 57 years old, has becomes the first won 15 Grammy Awards and in 1987 became the first woman inducted woman inducted into the Rock and Roll Hall of Fame into the Rock and (though her fear of flying also kept Roll Hall of Fame. her from that ceremony) …
Language models rank documents according to :
• Considers only and
• Query-likelihood approach with Dirichlet smoothing Collection as background model
Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 6 Publication Dates (LM+P)
• Independently generate the from the publication dates
• First factor is estimated with query-likelihood approach • The second factor is estimated with sigmoid function as
r=0.015 • Favors documents that are published close to
Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 7 Temporal expressions (LM+T)
• Independently generates from the
• First factor is estimated with query-likelihood approach • Second factor is estimated as
Indicator function
Document intervals Length of interval
• Favors documents with many temporal intervals at finer temporal granularity containing the
Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 8 Publication Dates + Temporal Expressions (LM +PT)
• Combine publication dates and text only approaches as
• Retrieves documents published close to the and also mentions it in content
Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 9 Two-stage Cascade Model (CM)
Top-K ranked Stage 1 documents (eg. top-30)
D1 D1 Text D2 Query, q Retrieval Dk DK Top-k documents (eg. top-10)
D Top-10 5 100 Temporal D7 50 Temporal Expression feedback
Re-ranker 0 extraction
10…
230… 340… 450… 560… 670… 780… 890… D1 120… Query temporal model Re-ranked K from top-k (k Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 10 Query Temporal Model: • Assumption: feedback temporal expressions represent salient time points • Estimate query model from , a set of top-k documents retrieved according to • The generative probability is estimated as Any time point Query temporal model Normalized score • The first factor is the normalized query likelihood of the document • The second factor estimated as Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 11 Document Temporal Model: • We estimate the document temporal model as • We use Jelinek-Mercer smoothing • The smoothing has two effects: • Addresses the Zero-probability issue • IDF like effect for frequent temporal expressions Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 12 Document Scoring: • Re-rank top-K documents retrieved according to • We re-rank documents according to KL-divergence Captures salient time Captures points Preserves textual recency relevance • The individual components are computed as 1. Query and document temporal models 2. Event happening date and publication date • As final output we present top-10 documents of re-ranked list Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 13 TAIA 2014: Linking Wikipedia Events to Past News 14 Overview 1 Approaches ✓ 2 Experiments 3 Summary Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 15 Datasets, Benchmark, and Evaluation Dataset: • The New York Times Annotated Corpus • 2 million articles published between years 1987 and 2007 Benchmark: • Random sample of 50 Wikipedia events as queries • Events that happened between 1987 and 2007 Evaluation: • We pooled top-10 from all the methods – 1297 unique query-document pairs • Crowdflower assessors judge an article as (2) Highly Relevant: Main topic is the query event (1) Somewhat Relevant: Mentions the query event (0) Irrelevant: Completely unrelated to the query event Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 16 Results LM LM+P LM+T LM+TP CM 0.66 0.63 0.63 0.62 0.62 0.61 0.61 0.6 0.6 0.6 0.59 0.58 0.58 0.57 0.55 0.55 0.54 0.54 0.53 0.48 0.45 0.43 0.42 0.4 0.35 MAP P@5 P@10 NDCG@5 NDCG@10 LM LM+P LM+T LM+TP CM MAP 0.35 0.43 0.40 0.42 0.45 P@5 0.55 0.63 0.61 0.61 0.66 P@10 0.48 0.57 0.54 0.55 0.58 NDCG@5 0.53 0.59 0.58 0.60 0.60 NDCG@10 0.54 0.62 0.60 0.62 0.63 Insight: CM consistently outperforms the baselines Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 17 March 19 2002: US war in Afghanistan: Operation Anaconda ends after killing 500 Taliban and Al-Qaeda fighters, with 11 allied troop fatalities. Cascade CM LM+P LM+T LM+TP LM approach wins P@10 0.8 0.6 0.5 0.6 0.5 Aretha Franklin becomes the first woman inducted into the Rock January 3, 1987: and Roll Hall of Fame. Temporal CM LM+P LM+T LM+TP LM expressions win P@10 0.7 0.2 0.7 0.1 0.4 February 27 1991: President Bush declares war over Iraq and orders cease-fire. Publication CM LM+P LM+T LM+TP LM dates win P@10 0.7 0.7 0.6 1 0.2 Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 18 Summary and Future Directions Summary: • Considered a novel linking task • Designed simple approaches to address the linking task • Found that our two-stage cascade approach outperforms simpler baselines Future directions: • Query contains at least one named entity and many mention a geographic location • Consider events not covered by the collection • Consider events that span over a large time period • Extension to queries that • do not contain temporal expressions • contain multiple temporal expressions Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 19 Thank You We make our experimental data available at : http://www.mpi-inf.mpg.de/~amishra/TAIA2014/ Monday, November 09, 2015 TAIA 2014: Linking Wikipedia Events to Past News 20