Delft University of Technology The effectiveness of surveillance technology What intelligence officials are saying Cayford, Michelle; Pieters, Wolter DOI 10.1080/01972243.2017.1414721 Publication date 2018 Document Version Final published version Published in Information Society Citation (APA) Cayford, M., & Pieters, W. (2018). The effectiveness of surveillance technology: What intelligence officials are saying. Information Society, 34(2), 88-103. https://doi.org/10.1080/01972243.2017.1414721 Important note To cite this publication, please use the final published version (if applicable). Please check the document version above. Copyright Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons. Takedown policy Please contact us and provide details if you believe this document breaches copyrights. We will remove access to the work immediately and investigate your claim. This work is downloaded from Delft University of Technology. For technical reasons the number of authors shown on this cover page is limited to a maximum of 10. THE INFORMATION SOCIETY 2018, VOL. 34, NO. 2, 88–103 https://doi.org/10.1080/01972243.2017.1414721 The effectiveness of surveillance technology: What intelligence officials are saying Michelle Cayford and Wolter Pieters Faculty of Technology, Policy and Management, Delft University of Technology, Delft, The Netherlands ABSTRACT ARTICLE HISTORY In recent years, Western governments have come under sharp criticism for their use of surveillance Received 5 October 2016 technology. They have been accused of sweeping up massive amounts of information without Accepted 27 November 2017 evidence of the technologies being effective in improving security. The view of critics is clear, but what do intelligence officials themselves say? This paper analyzes statements of intelligence officials in the U.S. and U.K. from 2006 to 2016, examines what criteria officials use in their discourse on effectiveness, and investigates how considerations of cost and proportionality factor into the equation. It identifies seven measures of effectiveness in the statements of intelligence officials, and finds that cost, though rarely discussed, is the driver behind formalized evaluations of surveillance programs. Introduction approach is not to question the truth of what officials say, nor to judge actual effectiveness, but to delve into Surveillance technology is pervasive in our society today, the meaning and significance of intelligence officials’ leading to fierce debate between proponents and oppo- statements, to identify values they place on effectiveness nents. Government surveillance, in particular, has been and the measures they use to assess it, and their reason- brought increasingly under public scrutiny, with propo- ing. This study is not hostile to security forces, but an nents arguing that it increases security, and opponents attempt to honestly understand what considerations decrying its invasion of privacy. Since the Snowden leaks, intelligence officials take into account when they speak critics have loudly accused governments of employing about the effectiveness of surveillance technology. surveillance technologies that sweep up massive amounts Because so much surrounding surveillance technology is of information, intruding on the privacy of millions, but controversial, how it is discussed matters. with little to no evidence of success. And yet, evaluating The paper proceeds as follows: after briefly addressing whether surveillance technology increases security is a related work, terminology is defined and the research difficult task. How does one measure the value of one bit methods of this study are described. Thereafter what of intelligence that contributes to the greater whole? intelligence officials are saying about effectiveness of sur- How do we measure the role of intelligence in informing veillance technology is examined. Next, statements about decision-makers? cost are analyzed to understand how officials factor it This paper focuses on what intelligence officials in the into their assessments is followed by an analysis of what U.S. and U.K. themselves say about the effectiveness of they are saying with regard to proportionality. Lastly, surveillance technology. In their own words, what are officials’ statements regarding effectiveness, cost, and the criteria for evaluating whether a particular piece of proportionality are considered together and critiqued, surveillance technology meets the goal that motivated its and recommendations offered for a more holistic deployment? Even in the absence of explicit evaluations, approach. intelligence bodies must constantly make judgments about effectiveness to determine if they will continue to use and redeploy a particular surveillance technology. Related Work This evaluation of effectiveness may be implicit, but it is there. This section covers the parts within the broad literature This study does not examine the veracity of officials’ on security and surveillance literature that are most rele- statements, nor does it determine whether or not a par- vant to this paper. One of these is the privacy and secu- ticular surveillance technology is actually effective. Our rity debate, which has been at the forefront in recent CONTACT Michelle Cayford [email protected] Safety and Security Science, TU Delft, P.O. Box 5015, Delft, NL 2600GA. Published with license by Taylor & Francis © Michelle Cayford and Wolter Pieters THE INFORMATION SOCIETY 89 years. Here, the issue of effectiveness is central, as the study by van Dongen (2015) constructs a new framework basis for arguing for the use of any given surveillance for evaluating counterterrorism policies and examines program must be that it is effective in increasing security. whether there is a relation between the type of terrorist Moreover, any proportionality judgment must consider organization and the effectiveness of the counterterror- the privacy intrusion of the program against its effective- ism approaches applied to combating it. ness. This literature can be classified into three catego- There are some government-related reports that ries: 1) actual effectiveness 2) belief of effectiveness and address the question of the actual effectiveness of secu- 3) statements of effectiveness. rity measures. They include two in-depth reports by the Many authors have written on the privacy concerns U.S. Privacy and Civil Liberties Oversight Board raised by the intersection of government surveillance (PCLOB), which discuss measures of effectiveness used and modern technology (e.g. Greenwald 2014, Berghel by intelligence officials and present the Board’s own con- 2013, Morgan 2014, Monahan 2016, Bigo et al. 2013). clusions about the effectiveness of NSA surveillance pro- Some of these authors accuse Western democracies of grams (PCLOB, Jan. 2014 and July 2014). Another exaggerating the threat of terrorism to justify mass sur- report by the Congressional Research Service makes veillance (e.g. Greenwald 2014) and also of exaggerating some rather obvious points, such as increased expendi- its role in preventing terrorist activity (e.g. Bergen et al., tures in counterterrorism does not necessarily result in 2014). progress (Perl, 2007). A study of the U.S. Department of Homeland Secur- One way to evaluate the actual effectiveness of surveil- ity’s fusion centers, which are meant to facilitate infor- lance technology is a cost-benefit analysis. Mueller and mation sharing among relevant government agencies, Stewart (2011) use risk assessment to determine the risk finds that there is confusion regarding legal and account- of a terrorist attack and then gauge whether the costs of ability frameworks that should be followed, resulting in security measures are outweighed by the benefitofa mission creep and privacy violations (Regan & likely-prevented attack. They argue that the enormous Monahan, 2013). Another study on these fusion centers amounts of spending in the U.S. on anti-terrorist meas- finds not only that mission creep occurs and thereby ures far outweigh the benefits gained. In the law enforce- potential privacy violations, but also that the centers are ment realm Edwards et al. (2014) point to a lack of ineffective in their primary tasks of information aware- evaluation of online data mining technology. Hewitt ness and sharing (Monahan & Palmer 2009). Ineffective- (2014) looks at the actual effectiveness of law enforce- ness at achieving initial goals leads to subsequent efforts ment tactics for dealing with the various types of terror- to make the centers useful and effective in other ways, ism in America and concludes that whether or not and thereby mission creep occurs with changes in cen- different types of tactics are effective depends on the ters’ tasks. type of terrorism. Additionally, Ekblom (2010), develops The largest number of studies on effectiveness exam- a framework for crime prevention and security in the ine the actual effectiveness of surveillance technology – community and Sproles (1999), a method for establish- assessments and measurements of whether or not a given ing measures of effectiveness that can be applied to any security program accomplishes its security goal. There is field. a significant body of work on evaluation of the effective- A small body of work deals with the actual effective- ness of counterterrorism measures. Lum et