Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence
Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence Tal Schuster Adam Fisch Regina Barzilay Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology {tals,fisch,regina}@csail.mit.edu Abstract Beaverton, Oregon From Wikipedia, the free encyclopedia Revision ID: 336934876 Typical fact verification models use retrieved its population is estimated to be 86,205, almost 14% more written evidence to verify claims. Evidence than the 2000 census figure of 76,129. Revision as of 04:10, 10 January 2010 sources, however, often change over time as its population is estimated to be 91,757, almost 14% more more information is gathered and revised. In Refutes than the 2000 census figure of 76,129. order to adapt, models must be sensitive to Claim: More than 90K people live in Beaverton Supports subtle differences in supporting evidence. We present VITAMINC, a benchmark infused with Figure 1: In VITAMINC, we focus on Wikipedia revi- challenging cases that require fact verification sions in which the factual content changes. This exam- models to discern and adjust to slight factual ple revision now supports an initially refuted claim. changes. We collect over 100,000 Wikipedia revisions that modify an underlying fact, and measured by how well they adjust to new evidence. leverage these revisions, together with addi- In this way, we seek to advance fact verification by tional synthetically constructed ones, to create requiring that models remain reliable and robust to a total of over 400,000 claim-evidence pairs. the change present in practical settings. Unlike previous resources, the examples in To this end, we focus on fact verification with VITAMINC are contrastive, i.e., they contain contrastive evidence.
[Show full text]