International Journal of Communication 8 (2014), 1745–1764 1932–8036/20140005 Working Within a Black Box: Transparency in the Collection and Production of Big Twitter Data KEVIN DRISCOLL1 University of Southern California, USA SHAWN WALKER University of Washington, USA Twitter seems to provide a ready source of data for researchers interested in public opinion and popular communication. Indeed, tweets are routinely integrated into the visual presentation of news and scholarly publishing in the form of summary statistics, tables, and charts provided by commercial analytics software. Without a clear description of how the underlying data were collected, stored, cleaned, and analyzed, however, readers cannot assess their validity. To illustrate the critical importance of evaluating the production of Twitter data, we offer a systematic comparison of two common sources of tweets: the publicly accessible Streaming API and the “fire hose” provided by Gnip PowerTrack. This study represents an important step toward higher standards for the reporting of social media research. Keywords: methodology, data, big data, social media Introduction The instruments register only those things they were designed to register. Space still contains infinite unknowns. —Spock (Black, Roddenberry, & Daniels, 1966) Twitter is increasingly integrated into the visual presentation of news and scholarly publishing in the form of hashtags, user comments, and dynamic charts displayed on-screen during conference presentations, in television programming, alongside political coverage in newspapers, and on websites. In 1 This research is based in part upon work supported by the National Science Foundation under the INSPIRE Grant Number 1243170. Kevin Driscoll:
[email protected] Shawn Walker:
[email protected] Date submitted: 2013–04–10 Copyright © 2014 (Kevin Driscoll & Shawn Walker).