Article Science & Technology Studies XX(X)

Rethinking the ‘Great Divide’: Approaching Interdisciplinary Collaborations Around Digital Data with Humour and Irony

David Moats Tema-T, Linköping University, Sweden/[email protected]

Abstract It is often claimed that the rise of so called ‘big data’ and computationally advanced methods may exacerbate tensions between disciplines like data science and anthropology. This paper is an attempt to reflect on these possible tensions and their resolution, empirically. It contributes to a growing body of literature which observes interdisciplinary collabrations around new methods and digital infrastructures in practice but argues that many existing arrangements for interdisciplinary collaboration enforce a separation between disciplines in which identities are not really put at risk. In order to disrupt these standard roles and routines we put on a series of workshops in which mainly self-identified qualitative or non-technical researchers were encouraged to use digital tools (scrapers, automated text analysis and data visualisations). The paper focuses on three empirical examples from the workshops in which tensions, both between disciplines and between methods, flared up and how they were ultimately managed or settled. In order to characterise both these tensions and negotiating strategies I draw on Woolgar and Stengers’ use of the concepts humour and irony to describe how disciplines relate to each others’ truth claims. I conclude that while there is great potential in more open-ended collaborative settings, qualitative social scientists may need to confront some of their own disciplinary baggage in order for better dialogue and more radical mixings between disciplines to occur.

Keywords: digital data, interdisciplinarity, mixed methods, quant/qual, data visualizations

“Why don’t we just focus on things we can quan- the tools, or their colleagues, or perhaps me, the tify?” asks a computer scientist. It’s day two of a organiser and facilitator. I ask if anyone has any three-day ‘data sprint’ workshop and we’re in the advice to give to any of the other groups. A heavy middle of a feedback session. The three teams, silence hangs in the air, mercifully ended by the each of which are composed of 4-6 research- computer scientist’s provocation. ers from medicine, anthropology, computer sci- This rhetorical question seems to imply that ence and science and technology studies (STS), we have been spending too much time on things have just been reporting back to the larger group which we cannot quantify. In this case, she is on the progress of their mini-projects. It is not probably referring to the long, messy and poorly going very well. The teams seem frustrated with formatted textual accounts we have been mired

This work is licensed under a Creative Commons Attribution 4.0 1 International License Science & Technology Studies XX(X)

in over the previous days. She suggests that we Encounters like the one detailed above, could overcome the current impasse by focusing however, suggest that things are more compli- on the data which is more amenable to repre- cated. It suggests that frictions between disci- sentation as numbers (time stamps, rankings plines and between methods are still very much and categorical data). On some level, I know that present: that computer scientists might misunder- she is right. If the goal of a workshop like this is stand the value of qualitative analysis or that self- to mock up some data analysis tool or data visu- identified qualitative researchers might have their alisation in a very short amount of time or to gain own resistances to these tools. It also suggests some cursory insight into a difficult data set, then that these divisions are not fundamental but play it would make sense to focus on what is ready-to- out in situated, practical negotiations over, for hand and feasible. example, what sort of data to use and in what However, in these workshops, I have been ways. How might we characterize these tensions actively trying to resist this sort of understanding and how could they be navigated? of the objectives, defined instrumentally in terms This paper contributes to a body of litera- of tools or results. I was interested in how one ture which analyses interdisciplinary encoun- could conduct research with digital data from ters around new forms of digital research and online platforms without falling into standard data infrastructures empirically (Blok et al., 2017; routines and divisions of labour between, say, Kaltenbrunner, 2014; Neff et al., 2017). This work quantitative and qualitative or technical and non- is increasingly vital as governments and funding technical researchers. In this particular workshop, bodies frequently demand interdisciplinarity I had been encouraging the researchers, including but often only understand the term through the more technical ones, to close-read the data. abstract pronouncements. These empirical This had yielded all sort of interesting insights studies contribute to our understanding of inter- about the substantive topic, but it seemed to disciplinarity as a practical and situated activity by produce (for many of the participants) a skepti- observing novel approaches to data analysis and cism towards the automated tools, resulting in a detailing messy interactions between different palpable slump in the room and a lack of direction sorts of researchers and disciplines. However, I within the teams. argue that existing roles and routines in these settings may be strong enough to paper-over Introduction many potential sources of tension and even prevent more radical mixings, and that these disci- It is often claimed that the increasing availability plinary tensions (and mixings) may require more of digital data (from online platforms, tracking active cultivation or interventions in order to be devices, and open government portals) and the drawn out. prominence of semi-automated forms of data This paper also contributes to work within STS analysis (like data visualisations, machine learn- from researchers who have adopted quantitative ing and neural networks) may exacerbate already tools but employed them largely in the service existing tensions between ‘quantitative’ and ‘qual- of qualitative research (Callon et al., 1986; Latour itative’ research or between different disciplines, et al., 1992; Rogers, 2013; Rogers and Marres, like computer science and anthropology (Bur- 2000). These experiments, arguably, go further rows and Savage, 2014; Marres, 2012; Wouters et than many established forms of interdisciplinary al., 2013). At the same time, others proclaim that collaboration or mixed methods approaches certain methods (particularly network graphs) because they have incorporated interpretivist used in combination with these new data sources critiques of data and computational methods into finally allow the reconciliation of macro and micro the practical application of using such tools. These approaches (Venturini and Latour, 2010) and ena- researchers have also been highly reflexive about ble new types of collaborations and contributions their own practices – how these tools incline (Blok et al., 2017; Neff et al., 2017; Vinkhuyzen and them in certain directions as opposed to others. Cefkin, 2016). However, I ague that more could be done to

2 Moats observe how these idiosyncratic approaches fare newness of these data sources and the power of in the wider landscape of established disciplines these computational methods, these performa- and frameworks. tive claims are nonetheless reshaping industries In this paper, I will examine disciplinary and and academic disciplines. New approaches like methodological tensions as they played out in data science (Schutt and O’Neil, 2013), compu- three workshops which were set up to expose tational social science (Lazer et al., 2009) and mainly self-identified ‘qualitative’, ‘non-tech- digital humanities (Berry, 2012) are moving into nical’ researchers to simple digital tools (such as traditional social science and humanities terri- scrapers, automated textual analysis, data visu- tory, given that much of this newly amassed data alisations). I will discuss three examples in which is nominally ‘social’ in character. These develop- tensions flared up and how they were ultimately ments might necessitate closer collaboration managed or settled. I will propose that these between social scientists and computer scien- tensions and the various responses to them can tists but they also might involve computationally be understood in terms of ‘irony’ and ‘humour’ advanced methods supplanting what one might as understood by Woolgar (1983) and Stengers call, for lack of a better word, ‘qualitative’ or ‘inter- (2000) respectively. Woolgar (1983) characterised pretivist’ forms of knowledge (Marres, 2012).1 constructivist sociologists of science as ‘ironists’ In this section I will discuss different reactions because they ‘reveal’ natural science accounts to the state of affairs engendered by the rise of of reality to be constructed without subjecting digital social data from ‘qualitative’ social scien- their own science (ethnography) to the same tists. These reactions range from outright critique criteria. It was in reference to Woolgar’s paper to calls for convivial but, as I will suggest, rather that Isabelle Stengers advocated that analysts of safe collaborations. What I want to argue is that science approach their subjects and interlocu- much of both the critical and convivial relation- tors, with ‘humour’, that is, with the understanding ships represent a settlement in which there is not that their fates are intertwined with those they much at stake and there is little chance of either observe (2000: 65). At the end of the paper, I will party being changed in the process. At worst, draw on Katie Vann’s (2010) more recent analysis this takes the form of an ‘ironic’ distance, as I will of these concepts in order to question how we explain, and at best this results in siloed modes might interpret these orientations in terms of of working. To unthink this settlement I argue interdisciplinary encounters. My objective is not that we need more studies of interdisciplinarity to offer some definitive account of interdiscipli- in practice, which see both tensions and nego- nary interactions, but to expand the lexicon for tiations as not given but as accomplishments of talking about these tensions as well as the arsenal situated practice. However, we also need studies of tactics for moving past them. that do not presume from the onset that we know what disciplines are composed of. The current settlement One of the dominant responses to the prolif- eration of data and computational methods has As we are repeatedly told: the last several years been largely critical. Anthropologists and quali- have seen governments and private companies tative social scientists have long raised concerns amass unprecedented amounts of data, housed about data-driven techniques on epistemological, in ‘data warehouses’, dumped in ‘data lakes’. ethical and political grounds (Iliadis and Russo, These masses of found or ‘transactional data’ from 2016; Manovich, 2012), arguing that they fail to online platforms and open government reposito- capture the nuance of situated practice (boyd ries, often positioned in contrast to survey data and Crawford, 2012) or lead us toward simplistic (Burrows and Savage, 2014), are seen by many as research questions (Uprichard, 2013), that they naturally amenable to much-hyped techniques exacerbate existing asymmetries of access and like machine learning, Artificial Intelligence (AI) visibility (Benjamin, 2019) and that they remain and data visualisations. While it is important to largely unaccountable (Pasquale, 2015) to the be sceptical towards these narratives about the people whose lives they affect. STS scholars in

3 Science & Technology Studies XX(X)

particular have examined how algorithms and While I do not wish to return to long-dormant data analytics achieve their performed neutrality, debates about constructivism, and this argument commensurability of different types of data (Slota mainly relates to the written accounts of ethnog- et al., 2020) and gloss over gaps and silences raphers and scientists: I think this is a useful way (Coopmans, 2014; Leonelli et al., 2017; Lippert and of thinking more generally about how disciplines Verran, 2018; Neyland, 2016). relate to one another and think about the status While these critiques draw much-needed of each other’s truth claims. Do they dismiss each attention to the politics of automated, data- other’s methods and facts out of hand or see driven approaches, particularly to the effects of knowledge production as a shared and ongoing these systems, it is not self-evident that the more problem? Stengers starts The Invention of Modern methodological or epistemological critiques of Science (2000) by asking why scientists have not these systems have resonated with the data scien- responded well to social science analyses of their tists who design them (see Moats and Seaver work. She argues that social scientists should 2019). One reason for this might be because approach the sciences not with irony but with these critiques often seem to judge data science ‘humour’, by which she means “…the capacity or data visualisations implicitly vis a vis ethnog- to recognize oneself as a product of the history raphy or other qualitative methods – that they whose construction one is trying to follow” are reductive or simplistic when compared to (Stengers, 2000: 65), to put their own identities qualitative methods or ‘small’ curated datasets at risk. So, while these critiques of the new data (Abreu and Acker, 2013; boyd and Crawford, science are important ones, I wonder if the sepa- 2012). And while asserting the value of qualitative ration effected between them and their object of methods in relation to computational methods study makes it unlikely that computer scientists is an important task, such criticisms risk unfairly will adopt these critiques from outside or that framing data science as a failure to capture qualitative social scientists will propose viable nuance and complexity when the potential value alternatives. of computational methods may lie in simplicity The other dominant reaction to this situation is and abstraction. to call for more and better collaborations between These claims are also potentially in danger of interpretive social scientists and computational falling into the ironic fallacy described by Woolgar researchers. There is a long tradition in STS but also (1983): they purport to show the limits, social anthropology, sociology and human computer determinants and constructedness of data and interaction (HCI) of productive collaborations with data science, while the methods used to demon- computational disciplines in the academy and strate this fact (often ethnography) are seen to in industry. Vertesi and others’ (2016) contribu- represent reality faithfully. Of course, ethnogra- tion to the STS Handbook describes four modes phers are first to admit the constructedness and of engagement with computational researchers partiality of their own accounts, but Woolgar’s ranging from ‘corporate’ and ‘critical’ to ‘inventive’ point is they often slip into an implicit corre- and, most radically, ‘inquiry’.2 spondence – or in his words ‘reflective’ (1983: However, for every apparently ‘successful’ 243) – theory of truth in order for their account collaboration (as the authors note, one of STS’s of ‘social factors’ or ‘politics’ to be believed by the main contributions to these fields is to ask ‘success reader. This reliance on a conventional report of for whom?’ (Vertesi et al., 2016: 176), there are what-was-witnessed is in some sense unavoidable many other more fraught encounters, where (Woolgar, 1983: 244), Woolgar notes, but when ethnographers complain about being misunder- social scientists temporarily exempt themselves stood (Dourish, 2006) or shut out of the process, from this fundamental problem, they sidestep or where computer scientists relate to social scien- important questions about what makes an tists in what Barry, Born and Weszkalnys (2008) account of some reality adequate for this or that might call a ‘subordination-service’ mode. In any audience, which are arguably central to interdisci- case, most ethnographers or micro-sociologists plinary relations. in these projects would probably admit that their

4 Moats influence on the proceedings is often limited keep researchers at a distance, separating them and circumscribed: they are often relegated to into different phases of the project or in different attending to so-called ‘social factors’, ethics and parallel tracks with intermittent contact, they do effects of technical systems, rather than their tech- not allow for the possibility that these roles might nological design and implementation. be transformed in the interaction (Stengers, 2000), But why is this so often the case? One possible that anthropologists might take up quantitative reason has to do with roles which ethnographers tools in a different way or that computational disci- and social scientists take on, or which are assigned plines might integrate social science criticisms of to them. These include: detached observers their approaches (as mentioned above) into their watching from the side-lines; token ethicists; tools. In these frameworks, (potential) tensions experts in science communications; reluctant might be hidden from view and alternative config- spokespeople for end users (Woolgar, 1990) or for urations of researchers, disciplines and methods publics. Researchers have occasionally been able might never emerge. So while critiques and to assert different priorities within these programs collaborations seem like contradictory responses, (Neyland, 2016) or argue for one set of technique they both result in what I will call a ‘settlement’ in as opposed to another (Adams, 2016; Vinkhuyzen which disciplines are kept separate and there is and Cefkin, 2016), but in general, many of these little chance of radical mixing happening. roles assume that qualitative social scientists will Now some might argue that such a settle- not dirty their hands with statistics and algorithms ment is inevitable: that most anthropologists and or visual representations of data. qualitative sociologists do not have the technical Of course, there have been many attempts to literacy to take up these tools in different ways, address this longstanding ‘siloing’ of disciplines. though as I will discuss later there are plenty of Discussions around mixed methods (Denzin, researchers working between different tradi- 2010) have long provided models for practically tions (e.g. Murthy, 2008). Others might claim that combing different methods and philosophical these relations are underwritten by historical paradigms (Tashakkori and Teddlie, 2010) in distinctions between quantitative and qualita- the same study, in more productive ways than tive methods, scientific and humanistic disciplines the above roles might allow.3 Grounded theory (Gould, 2011; Snow, 1998), objective and subjec- (Glaser and Strauss, 1967), in its many forms, tive epistemology (Daston and Galison, 2007), proposes that qualitative insights can be built variable or process orientations (Maxwell, 2010) up inductively into theories (through achieving or research which is communicated in terms of ‘saturation’) which can then be tested or modelled “stories” or “numbers” (Smith-Morris, 2016). quantitatively. While these frameworks are widely Much important work has been done to accepted, even beyond the academy, central question these divides (Hammersley, 1992), debates about validity (Clavarino et al., 1995), reli- to trace alternative genealogies in which, for ability and triangulation (Denzin, 2012; Silverman, example, anthropologists have engaged with 1985) suggest that these disciplinary or meth- techniques of counting, calculating and mapping odological tensions are by no means settled, (Munk and Jensen, 2015; Seaver, 2015). Quanti- only sublimated.4 More recently, Blok, Pedersen tative sociologists have also made overtures to and collaborators (Blok et al., 2017; Blok and qualitative researchers by taking into account Pedersen, 2014) have proposed a ‘complementa- traditionally interpretivist concepts like meaning- rity’ between ethnography and data science: that making (Mohr, 1998), narratives and emergent both sets of methods are mutually exclusive yet phenomena (Abbott, 2016). But even if such mutually necessary. divisions are not inevitable or hard-wired, they But while mixed methods, grounded theory cannot so easily be wished away. We know that and complementarity may be very effective digital technologies are not parachuted in out strategies for managing collaboration, even if (or of nowhere, they must take root in the existing, precisely because) they do not resolve philosoph- evolving infrastructures (Edwards, 2010; Wouters ical tensions, because these frameworks tend to

5 Science & Technology Studies XX(X)

et al., 2013) which often are maintained by and do not take for granted, from the onset, that we within disciplines (Kaltenbrunner, 2015). know what, say, ethnographers and data scien- While these alternative histories offer important tists do, when as Kaltenbrunner’s account shows, inspiration, the point is that neither the tensions, what they do must be examined and rethought. nor the successful negotiations are natural or As suggested above, when observing mixed- given, but are rather accomplishments of situated methods style projects, it becomes very difficult practices. These divisions and relations are to see past these inherited divisions of labour. For enacted in everyday interactions and entrenched this reason, I think the most interesting studies routines and even instances of boundary work seem to focus, not on successes, but on tensions, (Gieryn, 1983) – invocations of charged pejora- problems and failures and attempts to surmount tives like ‘positivist’ and ‘relativist’. And likewise them. alternative configurations of researchers are Another place we might look such alternative fragile, modest and extremely hard won. So one disciplinary configurations is in a longstanding important place to look for alternative possibilities movement within STS and related disciplines is in detailed empirical studies of collaborations in which largely qualitative researchers have between different sorts of researchers – because been adopting and adapting quantitative tools they give us hints as to exactly what tensions and to their own ends (Callon et al., 1986; Latour et negotiations are made of. al., 1992; Rogers and Marres, 2000). In doing so, There is a long tradition of such empirical they incorporate STS understandings of methods studies (Wouters et al., 2013). For example in a as performative (Law, 2004) and social science companion piece to their paper about comple- critiques of quantitative research into their own mentarity, Blok, Carlsen and colleagues (2017) practices (Marres, 2017). These researchers are discuss an interdisciplinary project in Copenhagen also highly reflexive about their struggles and pairing data obtained from Facebook with ethno- negotiations with these tools (Birkbak, 2016; graphic observations. They give rich, situated Jensen, Forthcoming; Munk et al., 2019; Pantzar accounts of how the ethnographic fieldnotes et al., 2017), though some of the most inter- were used to raise questions about data science esting moves remain tacit, not always explicated findings and vice versa. Other studies, however, outside the community. For example, they tend suggest more messy encounters. Kaltenbrunner to use graphs not as demonstrations of findings (2014), in his study of collaboration between but rather as exploratory maps to locate cases computer scientists and humanities scholars, to investigate qualitatively (Rogers and Marres, describes how different researchers working with 2000). They deploy these techniques in order to a common dataset fail to agree on the project document the partiality and constructedness of goals because their approaches have different the tools (Venturini et al., 2014), or of the under- ‘hinterlands’ (Law, 2004) and disciplinary ways lying data and devices behind them (Gerlitz and of phrasing research questions. Collaboration Helmond, 2013; Rogers, 2013) and the normative cannot proceed, he argues, until they ‘decompose’ commitments they smuggle in (Madsen and the process, placing these different ways of doing Munk, 2019). They also prefer to only use catego- research on the table. Neff and colleagues (2017) ries or dataset demarcations (Marres and Moats, examine several instances of anthropologists and 2015) which arise empirically, rather than impose data scientists experiencing problems with data, their own assumptions onto the proceedings finding that their data science colleagues exhibit (Uprichard, 2011). the sort of reflexivity and critical attention to data These are interesting tactics which fold some provenance normally attributed to qualitative of the criticisms of interpretivist social science researchers. researchers about computational data analysis These studies offer invaluable glimpses of into the practice of data analysis itself, in a interdisciplinarity in practice: both how tensions way which starts to repair the ‘ironic’ distance might flare up and how they can be resolved. mentioned above – raising, rather than settling, However, these studies are at their best when they questions about the status of knowledge claims.

6 Moats

However, these observations are largely circulated where participants collaborate on small projects within homogeneous teams of STS researchers over two to three days. Normally the participants and have rarely been tested in the wider academic are split into sub-groups based around shared community where expectations of what consti- interests, data-sets, methods or problems. In these tutes ‘quantitative’ and ‘qualitative’ methods interactions, the horizon of possibilities is often abound and roles are more entrenched. set by the more technically-capable participants In this section, I have argued that both ironic (Ruppert et al., 2015), while qualitative researchers critique and convivial collaborations amount and anthropologists become ‘topic experts’ who to a settlement which I think may prevent both relinquish responsibility for the analysis or using productive dialogue and alternative configura- the tools. It seemed clear that these encounters tions of disciplines from emerging. I suggested would need to be modified in order to avoid that in order to move past this impasse, we need participants falling back into established roles and to study interdisciplinary interactions in practice, routines. particularly ones in which tensions manifest A group of us at Linköping University decided themselves. The aim of this paper is to add to to put on a series of workshops, each one focusing these empirical studies of tensions and negotia- on a particular area of social life which was being tions between different approaches around digital transformed by the rise of digital data. These were data. But how can we observe situations in which based on hackathons and data sprints but tweaked disciplinary identities are put at risk, which allow in various ways to unsettle these knee-jerk roles for both disciplinary tensions and more radical and ways of working. Firstly, we involved mostly mixing to unfold? participants who self-identified as ‘non-technical’ including researchers from a variety of disciplines Three workshops including STS, medical sociology, medicine, media studies and anthropology. The idea was that this In thinking about this problem of how to shake would encourage these participants to get their up disciplinary routines, I have been inspired by hands dirty with the tools, rather than have a recent calls for ‘situated interventions’, in which technical expert do it for them. I was also curious researchers take concrete actions in the social set- what these ostensibly sympathetic disciplines tings they are embedded in, both with the aim would make of recent STS experiments with data of making a difference and learning about how and digital tools. The workshops also included actors respond when pressed in various ways (Zui- more technically capable researchers from infor- 5 derent-Jerak, 2015). For example, Zuiderent-Jerak, mation systems, computer science and library as a participant observer embedded in a hospital, sciences; however, we tried to shake them out tested some of his ideas by translating them into of established routines by using different sorts forms more amenable to his informants like flow of data than they were used to. Secondly, we charts and economic models. Analysing reactions encouraged the participants to spend more time to these interventions allowed Zuiderent-Jerak on ‘problem definitions’ – we discussed particular to reflect on the different normativities at play in social and intellectual problems related to the particular settings but also make visible and chal- topic before we made any mention of possible lenge some his own (Stengers, 2000). For example, digital tools and data sets. This was because much Zuiderent-Jerak was able to, among other things, research about computational techniques shows rethink his hard-wired disciplinary resistance to how readily available tools and data may incline practices of standardisation. us to focus on what is easy to analyse (Uprichard, So, what sort of intervention would put both 2011) rather than what is important to analyse, as anthropological and computer science identities the opening vignette of this paper also eludes to. 6 at risk? There are several established settings in Thirdly, we focused on producing simple data which qualitative researchers and programmers visualisations, mostly network graphs. Visu- already collaborate. Hackathons (Irani, 2015) alisations are interesting because, while they and Data Sprints (Munk et al., 2016) are events necessarily involve algorithms and metrics, they

7 Science & Technology Studies XX(X)

foreground the role of (equipped) human inter- and finally a less common interaction which was pretation in the process (Card et al., 1999). They both more fraught and, arguably, more radical also, it is claimed, can open the research process in character. I hope, given the discussion thus to a wider array of less technically-minded partici- far, that it goes without saying that my accounts pants and, as has already been noted, anthropolo- of these workshops are partial and interested, as gists in particular have a long history of employing are my strategic choice of vignettes. My purpose mapping approaches (Munk and Jensen, 2015).7 here is not to convince you, the reader, that the We provided slides of several unconventional workshops played out in exactly this way, or visualisations which we felt were more compat- that they are perfectly typical of interdiscipli- ible with anthropological or micro-sociological nary relations. However, through the positioning approaches because they addressed some of the of these vignettes I hope that qualitative social criticisms from these fields: for example, they were scientists might reconsider the ways in which they seen to lend themselves to exploratory analysis conceptualise their ways of knowing in relation to and to avoid aggregation and researcher-defined those of their disciplinary ‘others’. categories where possible (see discussion in previous section). Finally, as workshop organiser, Encounter 1 I actively intervened in various groups’ projects. One of the workshops focused on the use of digi- Sometimes I helped with suggesting data sources tal data and digital tools in academia. While the and tools of analysis while at other times delib- sciences have long produced data about them- erately detached myself to allow a group to find selves (Wyatt et al., 2017), there are increasing their own way. Sometimes I took on the role of drives to measure and make academic research technical expert, offering computational solutions more accountable, resulting in new approaches or demonstrating tools, while other times I like alt-metrics (Costas et al., 2015) and countless became more like a curmudgeonly anthropolo- rankings of academic output. This workshop was gist, slowing things down and raising annoying attended by a variety of researchers from STS, questions about computational practices. As anthropology, scientometics and information someone who is part of the STS community sciences (12 in total). All of them were sceptical experimenting with computational tools, this about current, rather simplistic ways of measuring was not a huge leap as, I often find myself caught academic output, yet their very attendance at the between these roles anyway. But as the opening workshop suggested that they were not against vignette suggests, I was not always in control of measurement per se. Indeed, many of the par- the proceedings, or my place in them. ticipants were interested in using computational, It should also be said that these workshops automated techniques to demonstrate the exist- were primarily set up to cultivate networks ence of phenomena which current metrics and of researchers and foster new approaches to measurement make invisible. Despite this inven- important empirical topics, but they also offered tive set of goals, because the participants came occasions to reflect on interdisciplinary relations from relatively mixed departments (scientomet- (something which I made clear to all the partici- rics and information science departments have pants). In what follows, which is based on my included quantitative and qualitative research- fieldnotes made at the time, I will discuss three ers for some time) it was perhaps easier for them moments in which disciplinary or methodological to slip into existing divisions of labour, as I will tensions manifested themselves and how they explain. were navigated. I will discuss one vignette from One team of four was interested in whether each of the workshops because each of them or not computational tools could be used to involved different configurations of researchers, detect some of the performative effects (Callon, which may have impacted how these interac- 1998; MacKenzie et al., 2007) of measurement tions played out. I will first discuss a more conven- systems: the ways in which different institutions tional disciplinary situation, followed by one reacted to or oriented themselves towards being which exemplifies the more reflexive STS work measured. One group member was experienced

8 Moats in both quantitative scientometrics and qualita- stronger the connection. These relationships are tive STS literature, while the other three had an then represented as a network of words, so that STS background but varying degrees of experi- words with more connections are brought closer ence with digital tools. The group quickly decided together into clusters (see also Callon et al., 1986; that they wanted to experiment with a tool called Danowski, 2009). VOSviewer, developed by the University of Leiden Only a couple of the participants had used (van Eck and Waltman, 2009). VOSviewer works by the tool before and the others were curious to scraping the Web of Science database to obtain see what it could do. As I had feared, this quickly lists of scientific articles and abstracts as well as became a show-and-tell scenario with the scien- metadata like publication date and disciplinary tometric researcher demonstrating the tool to tags. The tool then identifies terms (noun phrases, the others on the projector. But the scientometric to be precise) that appear together in the articles: researcher also slipped into another familiar role the more abstracts they appear together in, the of merely implementing the other’s ideas (Kalten-

A

B

Figure 1. Co-word of article results in economics (above) and sociology (below)

9 Science & Technology Studies XX(X)

brunner, 2015), acting as a kind of tech support. passages which spoke to the author(s) orienta- The other participants asked him to search for the tion to ranking and measurement. The group following terms in journals tagged as ‘Economics’ found, perhaps unsurprisingly, that economics and ‘Sociology’ in order to obtain a list of articles framed academic evaluation as a technical explicitly dealing with forms of academic assess- problem – the measurements are wrong – while ment. most sociologists treated it more like a threat to academic practice. Both used lots of jargon, but TS=‘academic evaluation*’ OR the economic jargon was more technical while TS=‘research excellence framework’ OR the sociological jargon was theoretical. It was TS=‘Norwegian system’ OR only after this exercise that the more interpre- TS=‘Performance based funding’ tive researchers saw traces of their findings in the TS=‘research assessment’ original maps. At the end of the workshop, the interpretivist The resulting articles were then visualised as two researchers had ended up confirming some of co-word networks, one for the Sociology-tagged their suspicions about economics and sociology, articles and one for the Economics articles. while the other pair of researchers had ended These networks, which showed different up with an impressive visualisation, in fact an configurations of key words used by the different animation, showing the relationship between disciplines, seemed to raise more questions economics and sociology journals on the topic than answers. In general, the participants were of research assessment over the past 20 years. confused as to what the maps were “saying”. Interestingly the animation did not show the They also could not seem to use the maps in an disciplines separating into distinct clusters as exploratory sense to find interesting papers to the teams had suspected, but instead clustered read because this way of using co-word did not around empirical topics (particular evaluation make visible the articles which contained the techniques). The presumed distinction between key terms. I asked them if this demarcation of the fields was not evident, at least to this partic- economics from sociology made sense because it ular usage of VOSviewer. meant accepting the definitions of economics and This brief account speaks to one fairly common sociology provided by Web of Science. It was then manifestation of disciplinary tensions in the proposed that the journal articles from the two workshops and also one way in which it was disciplines could be pooled and allowed to cluster managed. The tensions here appear as disappoint- so that journals which use similar keywords could ment, the disappointment that graphs do not be brought closer together – the distinction, or show what they are supposed to or that they did lack thereof, between economics and sociology not guide the research process. One of the partici- could be interrogated empirically with the graph. pants after the workshop pointed out in an email I regretted raising this issue because what that “…the more qualitatively oriented partici- happened next was that the scientometric pants were more optimistic regarding the quan- researcher and one of the others continued to titative methods compared to those having more work on this alternative graph, hunched over a experience in that sort of work.” The graphs have laptop, while in parallel the traditionally quali- farther to fall if one does not know how messy and tative researchers switched to what they knew confusing they can be to work with. best: close reading the texts.8 Their hypothesis Perhaps for this reason, the groups ended up (or hunch) was that economists, who are closer slipping into a standard mixed-methods division in certain ways to the methods of measuring of labour: to work separately but equally and then academia, might articulate the problem in compare results at the end. They were happy to more standardised ways (there would be more find some felicitous correspondence between alignment in responses from economics and more the two processes but the insights came mostly diversity in sociology). They then read a handful from the qualitative analysis and they were, as the of these articles, trying to pick out particular participants admitted, not particularly ground-

10 Moats breaking. It was unfortunate that they ended had been experimenting with web scrapers, up reading economics and sociology articles as text analysis and network graphs (Jensen, 2013; separate batches, which confirmed some of their Rogers, 2013). While these groups were very suspicions about the differences between them adept at using digital tools, and had written because, as suggested by the animated visualisa- extensively about them, most of these methods tion shown at the end, not presuming the distinc- have been leveraged to analyse social media and tion could have allowed them to find more hybrids other online platforms, which are mostly publicly between the two. available and comparatively well-formatted. This The same could be said about the research topic however entailed that they analyse other process itself: the two approaches were kept sorts of documents and online data, which shifted largely separate, which inevitably confirmed the research from more anthropological ‘how?’ expectations of what these approaches were questions to simple ‘who?’ or ‘what?’ questions: capable of. Because of this distance, the qualita- who were these political consultants and what tively-inclined researchers only projected instru- sorts of data and technologies were they using? mental uses onto the graph but did not imagine I had suggested that the group could use the a way in which their close-reading work could electoral registers for the United States and the be used instrumentally to help refine the graph. UK. These are public databases which list expendi- Relations were highly respectful and there was tures by political campaigns and their proxies in a no ‘ironic’ sense of either scientometrics or quali- given election. The larger collective quickly agreed tative analysis being raised above the other but that, if these two lists were combined, they could this was also not ‘humorous’ because no identi- be represented as a bi-partite network diagram ties had been put at risk. The participants noted (a network with two types of nodes) connecting after the fact that their interdisciplinary ambitions payers (political campaigns and proxies) and were quickly “funnelled” by technical possibilities their payees (various suppliers, consultants and and time constraints which meant that they were, services, including data analysis and targeted sadly, kept “in their silos” as they put it. advertising). Hopefully this would allow them to identify which types of campaigns made use of Encounter 2 social media data for micro-targeting. Another workshop focused on the use of data One team of two participants (an anthropolo- analytics in recent political campaigns, particu- gist studying data privacy and an STS scholar larly the use of machine learning, big data and experienced with digital methods) decided to psychological profiling to target political adver- analyse this dataset. Since the databases placed tisements to increasingly specific types of vot- limits on how many records could be down- ers (Anstead, 2017; Barocas, 2012; Loukissas and loaded at one time, they ultimately had to limit Pollock, 2017). The group, composed of 12 par- the search to individual expenditures over $1000 ticipants, was interested in how data-driven and disbursements over $10,000 for the US, and political consultancies like Cambridge Analytica a similar level for the UK. They also limited the positioned what they were doing, how they were records to the years 2013-2016 so that they could involved in redrawing the boundaries between focus on the 2016 election and EU referendum. science and politics through their hyperbolic The anthropologist started to ask questions like public pronouncements. However, the industry, “how long does a campaign work in advance of understandably given recent scandals, proved to an election?” or “what size expenditures are most be relatively opaque: there were no obvious data- interesting?”. The more technical researcher joined sets or materials through which their activities in on these speculations. This became another could be observed. moment of tension, but this time not between the This workshop mostly included participants two researchers but between the researchers and from the Digital Methods Initiative (Amsterdam) the structure of the database they were dealing and Techno-Anthropology Lab (Copenhagen), two with. key centres in which STS-influenced researchers

11 Science & Technology Studies XX(X)

Combining the UK and US datasets was rela- diagram inclined them to resolve these alternative tively easy, but participants described the data spellings into singular entities. The anthropologist as “messy”, being collated from a host of different asked if it could be assumed that these alternate organisations and sometimes converted from spellings were the same company: “What about paper submissions. Through keyword searchers, their subsidiaries and proxies?”, “Are they also ‘the the group realised that the company “Facebook” same’?” Her more technically adept partner was was spelled 23 different ways (Facebook, Face also concerned by these questions, but knew that Book, Facebook Inc. etc.). Yet making a network

friends of kerith a fair & just virginia pac

parrish for congress

indigo strategies llc

lubin strategies llc

clean slate baltimore pac

wwltv

last word strategies

defend louisiana pac cafe media

the equality equation llc practice makes progress media strategy group llc

minnesota democratic-farmer-labor party women online collective conscience, llc g7 group mary ellen balchunis for congress grow the economy pac, llc margaret for alaska alaska democratic party muzzatti, anthony diana degette for congress franklin, eroyn townsend for congress committee to elect jim reed for congress tony cardenas for congress connolly for congress ridder-braden clarify agency adrian garcia for congress yarmuth for congress afl-cio cope political contributions committee dspolitical llc l pac the mobilization project christina maria hernandez, llc cummings for congress campaign committee 21st century leaders eggman for congress kumar for congress phillipson for congress brpr llc bill otto for congress kessler content

citizens for strength and security pac

kos media, llc lake research partners american association for justice political action committee (aaj pac) lois frankel for congress lindbeck for alaska environment america progressive women silicon valley federal pac dr. matt heinz for arizona switchboard communications vermont democratic party international union of painters and allied trades political action together political committee friends of john plumb scott fuhrman for congress laborers' international union transport workers union political contributions committee mark pocan for congress brian sims for congress engineers political education committee (epec)/international union of operating engineers 1199 service employees int'l union federal political action fund united association political education committee (united association of journeymen and apprentices of the plumbing & pipefitting industry of the united states and canada) pennsylvania democratic party united we can cory booker for senate food policy action barve for congress working americans for affordable healthcare intl union of bricklayers & allied craftworkers pac castor for congress hawks for u.s. house louise slaughter re-election committee randy perkins for congress the voices of the american federation of government employees buzzfeed, inc. the independent project pac (tip pac) national association of letter carriers of u.s.a. political fund (letter carrier political fund) utah state democratic committee teal media working for working americans - federal feminist majority working for us political action committee inc alma adams for congress national air traffic controllers association pac liuna building america melissa gilbert for congress santarsiero for congress golderer for congress actblue dr. raul ruiz for congress uaw education fund blumenauer for congress hamburger gibson creative kaplan for congress gps impact susannah randolph for congress deborah ross for senate carroll for colorado perlmutter for congress international alliance of theatrical stage employees federal speech pac friends of christina m. hartman new blue interactive, llc chane for congress winning mark r4c16 legislative majority pac val demings for congress al media, llc kim myers for congress priorities usa action piotrowski for idaho amalgamated transit union - cope americans for accountability in leadership lift leading illinois for tomorrow friends of jared polis committee d.r.i.v.e. - democrat, republican, independent voter education (the pac of the international brotherhood of teamsters) mundy katowitz media friends of raja for congress american federation of teachers, afl-cio committee on political education gateway media llc the blue deal kurt schrader for congress 18th street projects, llc talbot digital suzanna shkreli for congress equality pac boulder strategies llc mavi social butterfly 15104; the schatz for senate not who we are pac friends for gregory meeks adelstein and associates ami bera for congress jimmy panetta for congress van hollen for senate friends of jane dittmar mike honda for congress jim barksdale for senate kemper for congress schneider for congress colorado democratic party beaconfire consulting inc arch global solutions company minerva insights llc michigan democratic state central commitee friends of anna throne-holst courtney for congress sheet metal workers' international association political action league friends of shaughnessy naughton denny heck for congress coleman for congress murphy vogel askew reilly llc illinois voices matter new jersey democratic state committee pete aguilar for congress jamie raskin for congress orentlicher for congress nevada state democratic party titus for congress friends of patrick murphy lon johnson for congress chc bold pac quigley for congress women vote! earn international union of operating engineers a new voice for maryland friends for joe mcdermott paul tonko for congress gregory for congress snap inc. rosen for nevada georgia federal elections committee ruben kihuen for congress luann bennett for congress nationaljobs and democratic opportunity training committee pac hubspot inc shelli yoder for indiana will jawando for congress mowrer for iowa priorities usa action suozzi for congress mothership strategies cain for congress democratic action women vote! jerry sturgill for idaho pivotal targeting llc wisconsin elections commission north carolina democratic party - federal digital turf, llc mceachin for congress america votes action fund nadler for congress clarity campaign labs, llc analyst institute, llc screen straregied media united steelworkers political action fund bill foster for congress hlinko, john fuse marketing limited people for ben the atlas project government accountability bd. bryan caforio for congress center forward committee el super pac voto latino kyrsten sinema for congress delbene for congress american academy of family physicians political action committee united steelworkers united food and commercial workers international union working families advocacy project dixon/davis media group llc precision network, llc ohio environmental council action fund inc. debbie wasserman schultz for congress kirkpatricknelson forfor wisconsinsenate colleen deacon for congress cory simpson for congress latino victory projectcalifornians for innovation united food and commercial workers international union active ballot club american federation of state county & municipal employees p e o p l e julia brownley for congressbrad ashford for congress usw works mosaic strategies group joseline pena-melnyk for congress texans for pete new american jobs fund stand up for ohio pac american osteopathic information association - osteopathic political action committee deseret digital media kully hall llc dba kully hall struble sierra club arizona accountability project global strategy group, llc league of conservation voters inc coalition of black trade unionists patty judge ellisonfor iowa for congress nebraska democratic party accountable leadership seiu cope (service employees international union committee on political education) murphy victory fund al franken for senate cartwright for congress friends of sherrod brown kansas democratic party yume inc. calone for congress ohio voter fund political action committee donald norcross for congress well & lighthouse,;;c ralston lapp media american college of surgeons professional association pac people for derek kilmer kount inc. rms interactive udall for us all missourians for kander mcnerney for congress progressive turnout project american federation of state county and municipal employees afl-cio citizens to elect rick larsen lisa blunt rochester for congress tammy baldwin for senate patriot majoritydwight evans for congress house majority pac iamerica action votevets bennet colorado victory dccc jva campaigns the campaign group, inc. standing up for california's middle class the new media firm, inc snapstream media shorr, shorr johnson maggie for nh lcv victory fund charlene for congress neikrug, matthew adam smith for congress committee carol shea-porter for congress friends of cheri bustos majority forward educators for ohio people for patty murray bob casey for senate inc armour media friends of brady walkinshaw catherine cortez masto for senate gillibrand for senate blue labs analytics, inc. nancy pelosi for congress iowa democratic party democratic executive committee of florida flores for congress friends of don beyer workers' voice friends ofend schumer citizens united scott peters for congress waterfront strategies democratic party wildfire contact llc maloney for congress the integration works, llc senate majority pac emilio j. huerta for congress 2016 nextgen california action committee democratic national conventionamericans forcommittee responsible 2016 solutions-pac friends of elizabeth esty dscc immigrant voters win pac jeff merkley for oregon 76 words corp. applegate for congress american academy of ophthalmology inc political committee (ophthpac) pacer services trilogy interactive llc akpd message and media, llc latino victory fund joe kennedy for congress national emergency medicine political action committee - american college of emergency physicians sherzan for iowa person for congress angus king for us senate campaign himes for congress mandate media, inc. salud carbajal for congress america working together donna edwards for senate strickland for senate chambers lopez strategies the american congress of ob-gyns pac (ob-gyn pac) america's renewable future inc kuster for congress, inc silver state pac, inc. anne lewis strategies llc planned parenthood californians for opportunity floridians for a strong middle class strategies 360 nolan for congress volunteer committee tim walz for us congress planned parenthood votes nea advocacy fund nativo minnesota senate majority caucus data farm consulting, inc tammy for illinois denise juneau for congress protecting choice in california, a project of planned parenthood affiliates of california bluprint strategies leahy for u.s. senator committee monica vernon for congress blueprint interactive espaillat for congress 2016 freedom force, llc cooperative of american physicians ie committee sean patrick maloney for congress mary lawrence for congress votevets.org action fund wyden for senate blumenthal for connecticut susie lee for nevada stephanie murphy for congress national treasury employees union advocacy committee splunk taddeo for congress steve stern for congress aft solidarity forward for a better future hard hats for america bennet for colorado civis analytics, inc. dollars for democrats j street pramila for congress mosaic emily's list truth pac aapi victory fund gmmb inc civic innovation usa inc nextgen persuasion llc kathleen matthews for congress catalist llc green room creative voterlabs, inc. jim gray for kentucky vicente gonzalez for congress new partners serrano for congress precision strategies zephyr teachout for congress accurate append nextgen climate action committee people for the american way david trone for congress josh gottheimer for congress kathleen rice for congress working america coalition gene green congressionalwashington campaign state democratic central committee charlie crist for congress grace for new york shawn for new hampshire constitutionalafl-cio committee responsibility on political project education treasury fund solidarity strategies democracy for america committee to re-elect nydia m. velazquez to congress independence party of minnesota latin insights, llc yougov america, inc. russ for wisconsin angle mastagni mathews political strategies red horse strategies communications workers of america-cope political contributions committee sierra club independent action thermopylae science & technology bully pulpit ineractive friends of cj for congress patriot contact ready pac veda data solutions friends of joe sestak swindell, russ hillary for america tim canova for congress kantar media | cmag democratic state committee (delaware) indiana democratic congressional victory committee chl/donahoe + cole/duffey elizabeth for ma inc oneamerica votes crowdstrike, inc. sway bullseye marketing fetterman for senate sittenfeld for senate ken mcgurn for congress rev.com judy chu for congress human rights campaign skdknickerocker creative majority political action committee melissa data corporation katie mcginty for senate glenn ivey for congress tulsi for hawai'i colorofchange pac blue state digital uk ltd 270 strategies inc schiff for congress hillary victory fund ultraviolet action el toro.com llc barragan for congress napolitano for congress foster campbell for the us senate new york state democratic committee forward together pac revolution messaging, llc fund for economic growth evan bayh committee for our future a whole lot of people for grijalva congressional committee friends of john delaney organizer national intervention our revolution julie stauch & associates friends of chris murphy rising tide, llc becker digital strategies the campaign workshop ngp van, inc. people's action direct opportunities group, llc kaine for virginia friends of michelle baron hill for indiana environmental defense action fund national nurses united for patient protection bluelabs, llc bernie 2016 vote.org conner for arkansas committee to elect alan grayson national association of home builders casa in action inc. revolution advertising we've got your backpac chsim enterprises llc chism strategies common sense leadership for america bob poe for congress independence usa pac art not war llc heckenberger, annie berlin rosen, ltd convergence targeted communications allpoint voter services, inc philadelphia 2016 host committee united for progress pac humanity for progress communications workers of america working voices dnc services corp./dem. nat'l committee texas democratic party california democratic party safety & justice hustle im4 communications llc wayne r. blanchard equinix services, inc. #774252 fifty second street fund return path inc ir media llc nclr action pac consumers for responsible government progress ohio voice of teachers for educ/cmte on pol educ of ny state untd teachers (vote/cope) of nysut infogroup media solutions jill stein for president merkle inc./cognitive data civix strategy group frelinghuysen for congress jelly digital, llc ohio democratic party revolution media enterprises inc rocky 2016 llc naral pro-choice america association of state democratic chairs kamala harris for senate america progress moveon.org political action american chemistry council inc infocus campaigns llc targetsmart chong & koster pallone for congress concur technologies, inc. european movement of the uk ltd sixteen thirty fund/make it work action long islanders for truth inc

new hampshire democratic party wright 2016 isa marketing services working families farrell, george us chamber of commerce

big bowl of ideas mercury llc friends of clyde williams joe garcia for congress liberty t black pac yoshimura, calvin friends of dena

community action now world journal san diego county democratic party (fed. acct.) american bridge 21st century golden state leadership fund pac audience partners

sierra club political committee lexis nexis mountaintop media middle class values pac patriot data services courageous conservatives pac derrick for congress american hospital association pac building bridges, not walls kullberg, kelly californians for fiscal responsibility

america speaks pac political data, inc. prosperity for pennsylvania inc gary johnson 2016 rallypac international brotherhood of electrical workers local 98 committee on political education mccubbin, john austin ipromoteu run ray run rally forge llc kellipac committee to restore america's greatness pac ward for senate pape for congress my ride to vote clear channel outdoor wendy long 2016 inc hans 2016 llc freethought equality super pac ro for congress inc. hines digital united food and commercial workers international union l2 political adroll, inc. sean barney for congress georgia republican party, inc. conservative america now pac 314 pac unlocking potential pac o'malley for president joe trippi and associates, inc. everytown for gun safety action fund ivc media llc tea party patriots citizens fund conservative liberty coalition digital triumph bob corker for senate 2018 inc salesforce.com spurlino for congress restore american leadership jennifer bukowsky save the american way mcconnell senate committee restoring american leadership pac hall for congress amagi strategies midwest growth pac luke messer for congress wethepeople foundation northstar campaign systems warrior pac wade, john evan ii patriots for economic freedom right to rise pac, inc. automattic inc. american target advertising restore the constitution coalition the conservative strikeforce convergence consulting group inc. cambridge analytica ratcliffe for congress imaginexit limited american college of radiology association pac friends of john thune make america number 1 restore american freedom and liberty fedup political action committee aka fedup pac five points consulting women speak out pac mary hoeft for congress friends of the earth define idea cbeyondata gary johnson victory fund active engagement the lukens company the in campaign ltd conservative freedom fighters brian fitzpatrick for congress opportunity and responsibility restored in our nation pac blue america pac independent expenditure committee majoritas usa, llc scott rigell for congress smartech corporation santorum for president 2016 missouri jobs with justice voter action postup digital llc info group inc the 2016 committee trades union congress (tuc) devin nunes campaign committee tom atkinson for congress fleming for louisiana washington media group scottish national party (snp) frontline political strategy amish pac savanna communications llc working for maryland conservative outsider pac pat roberts for us senate inc securing america's future pac smucker for congress blackjobs conservatives opportunity andfund freedom political action committee - jofpac hispanic victory pac mayday pac lt. colonel gregory ellison for congress pace for congress the national republican trust pac omega list company royce campaign committee y2 analytics volunteers for shimkus twitter political action committee of the american association of orthopaedic surgeons--pac of aaos crowdskout friends of john boehner mike bost for congress committee capitol media partners billy long for congress grapeseed media mr edward izzard wake up and vote creative response concepts honold communications friends of julia fletcher committee for accountable government in missouri republican national committee brabenercox,llc dsb and p llc sheriff scott jones for congress madison project inc. king for congress veralith, inc. amazon google trusted leadership pac friends of susan brooks greg evers for congress avaaz foundation prevail strategies american unity pac inc red metrics llc defending main street superpac inc. qev analytics scott taylor for congress american academy of dermatology association political action committee (skinpac)texans for senator john cornyn inc ted budd for congress defend rural arizona pac doug lamalfa committee srcp media, inc. mr darren grimes friends of scott desjarlais bucshon for congress make louisiana great again campaign for american principles lou barletta for congress indiana republican state committee, inc. scientists for eu limited rollins communications watchdog pac youtube rodney for congress awareness analytics partners llc new power pac ddb uk limited erin houchin for congress susan b anthony list inc indiana jobs now fp1 strategies fp1 digital llc right way superpac chinese americans for keeping america great enterprise freedom action committee families for james lankford walters for congress bilirakis for congress revily, llc zeldinmatthews for congress for nevada friends of todd young, inc. wagtv limited stripes agency llc new jersey family first inc darlene miller for congress make dc listen ryan costello for congress sawyer studios tea party forward martins for congress imge llc cfb strategies intermarkets inc essential liberty pac may for congress stand fortruth, inc. bluegrass committee committee for election of brian malnes to the us house of representatives; the avaaz campaigns uk martha roby for congress jpm strategies the catholic association aipr llc walorski for congress inc redprint strategy kristi for congress valor america bend the arc jewish action inc plaid cymru - the party of wales issa for congress humane society legislative fund restoration pac spotxchange texas patriots pac michelle ovens ltd the associated press granite state solutions boustany for senate inc cavalry strategies llc evolving strategies gilliard blanning & associates vern buchanan for congress bolding, darren k bobby mahendra for senate another europe is possible keep the promise pac victory phones nrcc coffman for congress 2016 friends of mia love kentucky tomorrow, inc. trump make america great again committee kyle mccarter for congress committee ted cruz for senate byrne for congress esafund civil liberties assault fund timd2016 david j. putmancatholicvote.org for congress mckinleyfreedomworks for congress for america acquire digital llc jeb 2016, inc. daniel webster for congress elbert guillory for america becker, beth connectivist media mds communications corporation friends of kelly ayotte inc mountaineers are always free pac mustard seed media llcalliance strategies group, llc campaign grid the eu referendum campaign limitedvote leave limited trade brigade of citizens trade thecampaign freedom association limited del ray media llc friends of kip tom cowan for congress, inc. louisiana prosperity fund fighting for louisiana friends of neal dunncruz for president culac the pac of credit union national association granite voices pac dga action progressive change campaign committee jim costa for congress digital freedom, llc local voices referendum facts ltd denham for congressmchenry for congress win right data company llc vets for economic freedom trust dba concerned veteransdon bacon for america for congress labour leave limited retail association of maine inc. jason lewis for congress lynn jenkins for congress friends of john mccain inc scott walker inc newsmax media inc. huckabee for president, inc. liberal democrats keep the promise pete king for congress committee ann wagner for congress mooney for congressnational 2016 right to life the richard burr committee let's do better america action fund climate hawks vote political action labour party jeff stone for congress lobiondo for congress bill cassidy for us senate cresent hardy for congress randy hultgren for congress citizens for bongino right way initiative inc. mcsally for congress friends ofnew frank hampshire guinta republican state committee green party universities uk nrsc american wind action scott garrett for congress dan barrett for congress unison - the public service union chuck fleischmann for congress committee, inc. rooney for congress jan pac friends of stewart mills jason smith for congress lisa murkowski for usnevada senate republican central committee rebuilding america now carpenter for senate direct selling empowers americans deep roots analytics llc mentzer media services, inc. labour movement for europe friends of roy blunt biggs for congress new river research institute llc faith family freedom fund conservatives in ltd west virginia chamber of commerce blum for congress future in america inc. john kennedy for us derickson for congress uk independence party (ukip) committee to re-elect trent franks to congress wendyrogers.org elevated campaigns kyzle.com, llc tenney for congress walker 4 nc cogzero duffy for wisconsin texans for lamar smith kansans for marshall maxwell for congress free and strong america pac, inc. jim banks for congress, inc. targeted victory ending spending inc. cotton for senate fastly inc. yoder for congress, inc lasee for congress, inc. stutzman for senate romney victory, inc. missouri republican state committee-federal red curve solutions romney for president, inc. mullin for congress trump victory gravis marketing lynch for congress 16 digiscore capital, llc strategic media placement lieutenant colonel oliver jenkins for congress wilson, wilson perkins better louisiana pac john bolton pac senate conservatives republican majority fund louie gohmert for congress committee americarisingpac.org team ryan stacy ries snyder for congress kirk for senate independent voice for illinois pac del beccaro for senate tom reed for congress heartland values pac liveintent inc. trott for congress, inc. target point consultinginc campaign solutions fund this advert axiom strategies deloitte political action committee bill mcclure for congress grow wv inc progressive wisconsin - let's rebuild america marsha blackburn for congress, inc. american crossroads gowan for arizona, inc majority committee pac--mc pac cd inc. citizen super pac believe againrick w. allen for congress character counts political action committee grassroots targeting llc portman for senate committee ddc advocacy llc compass bancshares, inc. pac black america's political action committee andy barr for congress, inc. phillips for congress wicker for senate justin fareed for congress jobs opportunity and new ideas pac marco rubio for senate 2016 right to rise usa rothfus for congress monstah pac boozman for arkansas roberson for congress georgians for isakson california republican party federal acct senate leadershipbill shuster fund for congress service honor country action fund our principles pac enterprise holdings, inc. political action committee freedom for all pac drew ferguson for congress inc. friends of joe heck collins for congress elise for congress denise gitsham kansas republican party mike johnson for louisiana stop hillary pac the herald group, llc americas pac bookbag video trey gowdy for congress carly for america comstock for congress national rifle association citigroup inc. political action committee - federal (citigroup pac-federal) iheart radio committee to elect darryl glenn washington state republican party direct response strategies great america pac andrea zopp for us senate grant starrett for congress stivers for congress mike gallagher for wisconsin strategic perception, inc.amie hoeber for congress rga right direction pac message made easy usa business freedom pac future45 45committee inc. constitutional conservatives political list brokers llc kefalas for maryland inc carly for president friends of dennis ross friends of dave joyce ken bennett for congress zinke for congress national cable and telecommunications association political action committee (ncta pac) cole for congress emergency committee for israel giles-parscale napoleon for illinois maria for america 2016 gun owners of america, inc. tusk digital grassley committee inc buck for colorado 2016 hudsonpatterson for congress for congress opinion strategies llc mullins, brian tiberi for congress illinois republican party fls connent conservative action fund congressional black caucus pac jack flanagandr trey for congressbaucum campaign congressional leadership fund connell donatelli inc friends of paul mitchell 1360 llc resurgent media revive america pac phyllis schlafly's eagle pac friends for chris stewart, inc. Facebook public strategy associates llc faso for congress heaney for congress lcm strategies media placementfriends ofservices hans tanzler glenn grothman for congress harris media llc ello creative llc friends of dave brat inc. new day for america push digital llc rick kozell for congress raul labrador for idahoaustin scott for congress inc kathy for maryland shelby for u s senate donald j. trump for president, inc. quantcast corp. vertical strategies oliva for congress gravesrevolvis for congress consulting inc. ron johnson for senate inc american action news mckinnon publishing co. never means never pac arizona republican party crane for congress inc advoc8 ron desantis for florida zealous llc the edge ideas right strategiesluttrell llc for congress dold for congress pete sessions forvaladao congress for congress colorado republican committee rick reed media, inc. brilliant communications carlos curbelo congress ny republican federal campaign committee patel for louisiana marco rubio for president republican jewish coalition swing state strategies conservative, authentic, responsive leadership for you and for america kansans for huelskamp your voter guide culberson for congress national federation of independent business federal political action committee special operations for america texans for jodey arrington carlos beruff for senate inc paul gosar for congress john bolton super pac robert aderholt for congress america ascendant pac hurd for congress house liberty project rebecca negron for congress thomas massie for congress the stoneridge group llc secureamericanow.orgoptimus consultingfriends of david llc jolly cleveland 2016 host committeechatham inc light media llc cris dosev for congress taboola relentless positive action pac (rpa pac) olson for congress committee walberg for congress ohioans for a better future independent leadership fund robert blaha for colorado abis & company mcclung communications and public relations lloyd for congress arena online, llc pat meehan for congress volunteers for nehlen medium buying, llc pellerin for us senate llc red maverick mediafriends llc of pat toomey swiftkurrent, inc. committeebatts for oncongress, arrangements inc. for the 2016 republican national convention socially liberal fiscally conservative political action committee incorporated digital core campaign llc freedom partners mclaughlin & associates mulvaney for congress jones for congress america's liberty pac republican federal committee of pennsylvaniafriends of mike lee inc sundheim for senate emmer for congress friends of jim bridenstine inc. media dc huck pac rand paul for us senate let america work carolinians for freedom jamestown assoc. the washington times acxiom liveramp inc. ben west for oregon america leads liz cheney for wyoming tenacious truth pac mario diaz-balart for congress grow nc strong inc ad astra insights onmessage missouri rigt to life victory fund americans united for values mike rogers for congress jon keyser for senate trey for congress our rebellion media, llc kustoff for congress uaw - v - cap (uaw voluntary community action program) red digital resonate diane black fortaylor congress griffin for congress bill flores for congress i360, llc connect strategic communications, llc we work sales teads inc. libre initiative red sea, llc ros-lehtinen for congress renee ellmers for congress committee young for iowa, inc. advictory llc noelle nikpour for congress inc christopher peters for congress perdue for senate mica for congress cathy mcmorris rodgers for congress friends of scott angelle, llc mary thomas for congress carl domino for congresss house freedom fund targoz strategic marketing jim renacci for congressvirginia foxx for congress the prosper group corporation conserv america building a brighter future together committee for american sovereignty liquidus marketing citizens for cochran new jersey republican state committee gary palmer for congress sanford for congress maryland republican state central committee greer for congress gibbs for congress ohio republicanfreedom party state partners central &chamber executive committeeof commerce inc. growth & opportunity pac, inc. elect blake farenthold committee american future fund michigan republican party paul babeu for congress upton for all of us data targeting, inc. rob wittman for congress mike crapo for us senate tarkanianbob goodlatte for congress for congress committee forlini for us lance for congress comer for congress george holding for congress inc. professional data services forbes for congress moranarizona for kansas grassroots action pac wells for securitycharlie dent for congress barringer for congress friends of matt gaetz mobile walla moolenaar for congress state conservative reform action pac (scrap) innocenzi, sandler americanscampaign for committee prosperity for mark freeman katko for congress conservative solutions pacdan newhouse for congress kentuckians for strong leadership treehorn, llc prodege alamo pac oklahoma leadership council friends of sam johnson florida first project grit creative llc kevin mccarthy for congress citizens for john rutherford pittenger for congress llc pinpoint media llc concerned taxpayers of america voter gravity inc. tim scott for senate american opportunity pac friends of colonel rob maness davidson for congress kasich for america inc target enterprises, llc meridian pacific, inc. liberty and leadership fund eleventy marketing group llc hoosiers for rokita, inc. principled leadership majority strategies effective leadership for america katcho for congress carson america innovative advertising brady for congress bask digital media, llc friends of bill posey ryan-mccarthy victory french hill for arkansas citizens for turner steve knight for congress inboxfirst.com american action network inc optimizely republican party concerned american voters alternativepac north florida neighbors wirecom media hometown freedom action network gopac election fund rand paul for president, inc. rand paul new hampshire priorities huizenga for congress gus rantz for louisiana go big media, inc. garret graves for congress tmone, llc dba mass markets external affairs, inc. cando politics american medical association political action committee roskam for congress committee tom rooney for congress texanschris for christieintegrity for president inc reignite gurfein for america harmon brothers llc right digital llc fighting for ohio fund one nation ryan for congress, inc. chris vance for us senate maryland usa new york wins pac republican central committee of san luis obispo county-federal massachusetts republican party north dakota republican party ssc strategies, llc david rouzer for congress pompeo for kansas, inc. y2k service inc values are vital righters group llc victory enterprises heartland resurgence bergmanforcongress love for louisiana political action committee citizens for josh mandel inc causeway solutions llc strategic digital services, inc. justin grabelle for congress causeway consultants conston communications concentric direct republican hindu coalition rebekah for congress on 3 public relations, llc valor fund conquest communications group carlos lopez-cantera for senate house conservatives fund friends of becky gerritson plaster for congress arena communications scalise for congress club for growth primary data solutions coolhead inc. faye stewart for oregon rampart pac americanskiehne for congressfor concrete solutions, inc. cure 4 louisiana's future pac zoeller for congress henning for congress calabur productions llc conservative congress now! 406 enterprises llc war room strategies phone2action llc republican campaign committee of new mexico spoutable llc friends of roger reynolds montana republican state central committee e3 post connors for senate people for pinellas mike bishop for congress eagle forum pac north carolina republican party burns, clare synovation solutions mark harris for congress roger williams for u s congress committee patch media eighth congressional district republican party of minnesota pioneer outdoor utah republican party friends of erik paulsen sermo digital llc vici media group salmon for congress samuel martin rob bishop for congress strong nh action protect louisiana, inc. meath media services move maryland forward terra eclipse sensenbrenner committee ray strauss 4 congress protect the harvest political action committee strategic partners & media idaho republican party upstream communications jim for nc alaska republican party spotx stephen mackenzie for congress 2016 national association of realtors congressional fund madison pac for consitutional limited government america is worth it arizona rock product association right voter, llc the guardian fund price for congress 720 strategies llc vermont republican federal elections committee proofpoint noiseworks media leadership matters for america pac, inc. shark tank media llc friends of martin babinec raidious llc extensive enterprises atlantic list company brian mast for congress federation of american hospitals pound, feinstein & associates deb fischer for us senate inc fenn communications group the casale group security is strength pac strong leadership for america mccaul for congress, inc john carter for congress kunnect llc colm for congress tom garrett for congress tma direct house freedom action fund panhandle victory fund

1and1.com morgan griffith for congress seabaugh, catherine media bridge llc

renew america's greatness pac american legacy political action committee

donor bureau

world net daily

casperson for congress freedom's defense fund hsp digital ainsley shea

vigop (virgin islands republican party) wwp strategies veterans for a strong america action group campaigngrid direct phil pavlov for congress

campaign for jobs and opportunity

Figure 2: Payers (in red) and Payees (in green) in the UK and US election cycle 2014-6. Payees are sized by number of connections in this version of the graph.

12 Moats

they had to move on in order to have something assumptions about the data, something which to present at the end of the workshop. they did only reluctantly. The solution was a rather amusing assembly- Unlike in the previous encounter, there was line in which the more technical researcher would less of a clear demarcation of different types of bring up a list of similarly spelled companies research, despite the technical gulf between in data processing tool Open Refine, while the the two researchers, possibly because both anthropologist would Google the names to researchers had STS training. Instead of a clear determine if they were the same and could thus division of labour, they both actively engaged be merged. This took them the better part of in counting, cleaning, interpreting and making

friends of kerith a fair & just virginia pac parrish for congress the second day (at least 4 or 5 hours), with the decisions. So, one might say that the respon- indigo strategies llc lubin strategies llc clean slate baltimore pac repeated chime of “what about these two?” sibility for producing the graph was shared

wwltv last word strategies followed by lots of “umming” and “awings” over between them. But how did they reconcile their defend louisiana pac cafe media the din of the other groups quietly typing at their doubts about their assumptions, and their STS- laptops. Once they had satisfactorily “cleaned” the infused scepticism towards graphs ‘revealing’

the equality equation llc practice makes progress media strategy group llc database and created the graph, still more work hidden insights, with the seemingly clear findings minnesota democratic-farmer-labor party women online collective conscience, llc g7 group mary ellen balchunis for congress grow the economy pac, llc margaret for alaska alaska democratic party muzzatti, anthony diana degette for congress franklin, eroyn townsend for congress committee to elect jim reed for congress tony cardenas for congress connolly for congress ridder-braden clarify agency adrian garcia for congress yarmuth for congress was required. The graph contained relatively clear presented by the graph? afl-cio cope political contributions committee dspolitical llc l pac the mobilization project christina maria hernandez, llc cummings for congress campaign committee 21st century leaders eggman for congress kumar for congress phillipson for congress brpr llc bill otto for congress kessler content citizens for strength and security pac kos media, llc clusters but in order to understand what each one One way in which this tension was resolved lake research partners american association for justice political action committee (aaj pac) lois frankel for congress lindbeck for alaska environment america progressive women silicon valley federal pac dr. matt heinz for arizona switchboard communications vermont democratic party international union of painters and allied trades political action together political committee friends of john plumb scott fuhrman for congress laborers' international union transport workers union political contributions committee mark pocan for congress brian sims for congress engineers political education committee (epec)/international union of operating engineers 1199 service employees int'l union federal political action fund united association political education committee (united association of journeymen and apprentices of the plumbing & pipefitting industry of the united states and canada) pennsylvania democratic party united we can cory booker for senate food policy action barve for congress working americans for affordable healthcare intl union of bricklayers & allied craftworkers pac castor for congress hawks for u.s. house louise slaughter re-election committee randy perkins for congress the voices of the american federation of government employees buzzfeed, inc. the independent project pac (tip pac) national association of letter carriers of u.s.a. political fund (letter carrier political fund) represented they had to scan a handful of nodes is that the researchers performed themselves utah state democratic committee teal media working for working americans - federal feminist majority working for us political action committee inc alma adams for congress national air traffic controllers association pac liuna building america melissa gilbert for congress santarsiero for congress golderer for congress actblue dr. raul ruiz for congress uaw education fund blumenauer for congress hamburger gibson creative kaplan for congress gps impact susannah randolph for congress deborah ross for senate carroll for colorado perlmutter for congress international alliance of theatrical stage employees federal speech pac friends of christina m. hartman new blue interactive, llc chane for congress winning mark r4c16 legislative majority pac val demings for congress al media, llc kim myers for congress priorities usa action piotrowski for idaho amalgamated transit union - cope americans for accountability in leadership lift leading illinois for tomorrow friends of jared polis committee d.r.i.v.e. - democrat, republican, independent voter education (the pac of the international brotherhood of teamsters) (payers and payees) residing in each, track down as “sober and modest”, to use Shapin’s (1984: mundy katowitz media friends of raja for congress american federation of teachers, afl-cio committee on political education gateway media llc the blue deal kurt schrader for congress 18th street projects, llc talbot digital suzanna shkreli for congress equality pac boulder strategies llc mavi social butterfly 15104; the schatz for senate not who we are pac friends for gregory meeks adelstein and associates ami bera for congress jimmy panetta for congress van hollen for senate friends of jane dittmar mike honda for congress jim barksdale for senate kemper for congress schneider for congress colorado democratic party beaconfire consulting inc arch global solutions company minerva insights llc michigan democratic state central commitee friends of anna throne-holst courtney for congress sheet metal workers' international association political action league friends of shaughnessy naughton denny heck for congress coleman for congress murphy vogel askew reilly llc illinois voices matter new jersey democratic state committee pete aguilar for congress jamie raskin for congress orentlicher for congress nevada state democratic party titus for congress friends of patrick murphy lon johnson for congress chc bold pac quigley for congress women vote! earn international union of operating engineers a new voice for maryland friends for joe mcdermott paul tonko for congress gregory for congress snap inc. rosen for nevada georgia federal elections committee ruben kihuen for congress luann bennett for congress nationaljobs and democratic opportunity training committee pac hubspot inc their web pages and quickly get an impression 495) phrase, by describing their difficulties and shelli yoder for indiana will jawando for congress mowrer for iowa priorities usa action suozzi for congress mothership strategies cain for congress democratic action women vote! jerry sturgill for idaho pivotal targeting llc wisconsin elections commission north carolina democratic party - federal digital turf, llc mceachin for congress america votes action fund nadler for congress clarity campaign labs, llc analyst institute, llc screen straregied media united steelworkers political action fund bill foster for congress hlinko, john fuse marketing limited people for ben the atlas project government accountability bd. bryan caforio for congress center forward committee el super pac voto latino kyrsten sinema for congress delbene for congress american academy of family physicians political action committee united steelworkers united food and commercial workers international union working families advocacy project dixon/davis media group llc precision network, llc ohio environmental council action fund inc. debbie wasserman schultz for congress kirkpatricknelson forfor wisconsinsenate colleen deacon for congress cory simpson for congress latino victory projectcalifornians for innovation united food and commercial workers international union active ballot club american federation of state county & municipal employees p e o p l e julia brownley for congressbrad ashford for congress usw works mosaic strategies group joseline pena-melnyk for congress texans for pete new american jobs fund stand up for ohio pac of their political leanings, country of origin, or the uncertainty around the ‘findings’. They then american osteopathic information association - osteopathic political action committee deseret digital media kully hall llc dba kully hall struble sierra club arizona accountability project global strategy group, llc league of conservation voters inc coalition of black trade unionists patty judge ellisonfor iowa for congress nebraska democratic party accountable leadership seiu cope (service employees international union committee on political education) murphy victory fund al franken for senate cartwright for congress friends of sherrod brown kansas democratic party yume inc. calone for congress ohio voter fund political action committee donald norcross for congress well & lighthouse,;;c ralston lapp media american college of surgeons professional association pac people for derek kilmer kount inc. rms interactive udall for us all missourians for kander mcnerney for congress progressive turnout project american federation of state county and municipal employees afl-cio citizens to elect rick larsen lisa blunt rochester for congress tammy baldwin for senate patriot majoritydwight evans for congress house majority pac iamerica action votevets bennet colorado victory dccc jva campaigns the campaign group, inc. standing up for california's middle class the new media firm, inc possible uses of voter data. demonstrated the constructedness and possible snapstream media shorr, shorr johnson maggie for nh lcv victory fund charlene for congress neikrug, matthew adam smith for congress committee carol shea-porter for congress friends of cheri bustos majority forward educators for ohio people for patty murray bob casey for senate inc armour media friends of brady walkinshaw catherine cortez masto for senate gillibrand for senate blue labs analytics, inc. nancy pelosi for congress iowa democratic party democratic executive committee of florida flores for congress friends of don beyer workers' voice friends ofend schumer citizens united scott peters for congress waterfront strategies democratic party wildfire contact llc maloney for congress the integration works, llc senate majority pac emilio j. huerta for congress 2016 nextgen california action committee democratic national conventionamericans forcommittee responsible 2016 solutions-pac friends of elizabeth esty dscc immigrant voters win pac jeff merkley for oregon 76 words corp. applegate for congress american academy of ophthalmology inc political committee (ophthpac) pacer services trilogy interactive llc akpd message and media, llc latino victory fund joe kennedy for congress national emergency medicine political action committee - american college of emergency physicians sherzan for iowa person for congress angus king for us senate campaign himes for congress mandate media, inc. salud carbajal for congress america working together donna edwards for senate strickland for senate chambers lopez strategies the american congress of ob-gyns pac (ob-gyn pac) When presenting their results, the group arbitrariness of the relatively clear clusters in america's renewable future inc kuster for congress, inc silver state pac, inc. anne lewis strategies llc planned parenthood californians for opportunity floridians for a strong middle class strategies 360 nolan for congress volunteer committee tim walz for us congress planned parenthood votes nea advocacy fund nativo minnesota senate majority caucus data farm consulting, inc tammy for illinois denise juneau for congress protecting choice in california, a project of planned parenthood affiliates of california bluprint strategies leahy for u.s. senator committee monica vernon for congress blueprint interactive espaillat for congress 2016 freedom force, llc cooperative of american physicians ie committee sean patrick maloney for congress mary lawrence for congress votevets.org action fund wyden for senate blumenthal for connecticut susie lee for nevada stephanie murphy for congress national treasury employees union advocacy committee splunk taddeo for congress steve stern for congress aft solidarity forward for a better future hard hats for america bennet for colorado civis analytics, inc. dollars for democrats j street pramila for congress mosaic emily's list truth pac aapi victory fund gmmb inc civic innovation usa inc nextgen persuasion llc kathleen matthews for congress catalist llc green room creative voterlabs, inc. jim gray for kentucky vicente gonzalez for congress new partners serrano for congress precision strategies zephyr teachout for congress accurate append first explained their trials and tribulations with the graph above by clicking through different nextgen climate action committee people for the american way david trone for congress josh gottheimer for congress kathleen rice for congress working america coalition gene green congressionalwashington campaign state democratic central committee charlie crist for congress grace for new york shawn for new hampshire constitutionalafl-cio committee responsibility on political project education treasury fund solidarity strategies democracy for america committee to re-elect nydia m. velazquez to congress independence party of minnesota latin insights, llc yougov america, inc. russ for wisconsin angle mastagni mathews political strategies red horse strategies communications workers of america-cope political contributions committee sierra club independent action thermopylae science & technology bully pulpit ineractive friends of cj for congress patriot contact ready pac veda data solutions friends of joe sestak swindell, russ hillary for america tim canova for congress kantar media | cmag democratic state committee (delaware) indiana democratic congressional victory committee chl/donahoe + cole/duffey elizabeth for ma inc oneamerica votes crowdstrike, inc. sway bullseye marketing fetterman for senate sittenfeld for senate ken mcgurn for congress rev.com judy chu for congress human rights campaign skdknickerocker creative majority political action committee cleaning the data. They then showed the above settings to show different possible realities they melissa data corporation katie mcginty for senate glenn ivey for congress tulsi for hawai'i colorofchange pac blue state digital uk ltd 270 strategies inc schiff for congress hillary victory fund ultraviolet action el toro.com llc barragan for congress napolitano for congress foster campbell for the us senate new york state democratic committee forward together pac revolution messaging, llc fund for economic growth evan bayh committee for our future a whole lot of people for grijalva congressional committee friends of john delaney organizer national intervention our revolution julie stauch & associates friends of chris murphy rising tide, llc becker digital strategies the campaign workshop people's action graph (figure 2), explaining that that the two suggested. So, just as many ethnographic STS ngp van, inc. direct opportunities group, llc kaine for virginia friends of michelle baron hill for indiana environmental defense action fund national nurses united for patient protection bluelabs, llc bernie 2016 vote.org conner for arkansas committee to elect alan grayson national association of home builders casa in action inc. revolution advertising we've got your backpac chsim enterprises llc chism strategies common sense leadership for america bob poe for congress independence usa pac art not war llc heckenberger, annie berlin rosen, ltd convergence targeted communications allpoint voter services, inc philadelphia 2016 host committee united for progress pac humanity for progress communications workers of america working voices dnc services corp./dem. nat'l committee texas democratic party california democratic party safety & justice hustle im4 communications llc wayne r. blanchard major clusters did not correspond to US and accounts (perhaps in response to Woolgar’s (1983) equinix services, inc. #774252 fifty second street fund return path inc ir media llc nclr action pac consumers for responsible government progress ohio voice of teachers for educ/cmte on pol educ of ny state untd teachers (vote/cope) of nysut infogroup media solutions jill stein for president merkle inc./cognitive data civix strategy group frelinghuysen for congress jelly digital, llc ohio democratic party revolution media enterprises inc rocky 2016 llc UK, as one might expect. Rather, the top cluster essay) reflexively draw attention to doubts and naral pro-choice america association of state democratic chairs kamala harris for senate america progress moveon.org political action american chemistry council inc infocus campaigns llc targetsmart chong & koster pallone for congress concur technologies, inc. european movement of the uk ltd sixteen thirty fund/make it work action long islanders for truth inc new hampshire democratic party wright 2016 isa marketing services working families farrell, george us chamber of commerce seemed to consist of mostly US Democratic party ambiguities and poke holes in their own authori- big bowl of ideas mercury llc friends of clyde williams joe garcia for congress liberty t black pac yoshimura, calvin friends of dena community action now world journal san diego county democratic party (fed. acct.) american bridge 21st century golden state leadership fund pac candidates and organisations and their payees, tative statements, these researchers did the equiv- audience partners

sierra club political committee lexis nexis mountaintop media middle class values pac patriot data services courageous conservatives pac derrick for congress american hospital association pac building bridges, not walls kullberg, kelly californians for fiscal responsibility america speaks pac political data, inc. prosperity for pennsylvania inc gary johnson 2016 while the bottom cluster seemed to contain the alent for their graph. rallypac international brotherhood of electrical workers local 98 committee on political education mccubbin, john austin ipromoteu run ray run rally forge llc kellipac committee to restore america's greatness pac ward for senate pape for congress my ride to vote clear channel outdoor wendy long 2016 inc hans 2016 llc freethought equality super pac ro for congress inc. hines digital united food and commercial workers international union l2 political adroll, inc. sean barney for congress georgia republican party, inc. conservative america now pac 314 pac unlocking potential pac o'malley for president joe trippi and associates, inc. everytown for gun safety action fund tea party patriots citizens fund US Republican party and several of the tech giants However, I think there is something else worth ivc media llc conservative liberty coalition digital triumph bob corker for senate 2018 inc salesforce.com spurlino for congress restore american leadership jennifer bukowsky save the american way mcconnell senate committee restoring american leadership pac hall for congress amagi strategies midwest growth pac luke messer for congress wethepeople foundation northstar campaign systems warrior pac wade, john evan ii patriots for economic freedom right to rise pac, inc. automattic inc. american target advertising restore the constitution coalition the conservative strikeforce convergence consulting group inc. cambridge analytica ratcliffe for congress imaginexit limited american college of radiology association pac friends of john thune make america number 1 restore american freedom and liberty fedup political action committee aka fedup pac five points consulting women speak out pac mary hoeft for congress friends of the earth define idea cbeyondata gary johnson victory fund active engagement the lukens company the in campaign ltd conservative freedom fighters brian fitzpatrick for congress opportunity and responsibility restored in our nation pac (Facebook, Google etc.), as well as most major UK noting about this encounter. Munk, Madsen blue america pac independent expenditure committee majoritas usa, llc scott rigell for congress smartech corporation santorum for president 2016 missouri jobs with justice voter action postup digital llc info group inc the 2016 committee trades union congress (tuc) devin nunes campaign committee tom atkinson for congress fleming for louisiana washington media group scottish national party (snp) frontline political strategy amish pac savanna communications llc working for maryland conservative outsider pac pat roberts for us senate inc securing america's future pac smucker for congress blackjobs conservatives opportunity andfund freedom political action committee - jofpac hispanic victory pac mayday pac lt. colonel gregory ellison for congress pace for congress the national republican trust pac omega list company royce campaign committee y2 analytics volunteers for shimkus twitter political action committee of the american association of orthopaedic surgeons--pac of aaos crowdskout friends of john boehner mike bost for congress committee capitol media partners billy long for congress grapeseed media mr edward izzard wake up and vote creative response concepts honold communications friends of julia fletcher committee for accountable government in missouri republican national committee brabenercox,llc dsb and p llc sheriff scott jones for congress madison project inc. king for congress veralith, inc. amazon google trusted leadership pac friends of susan brooks greg evers for congress avaaz foundation prevail strategies american unity pac inc red metrics llc defending main street superpac inc. qev analytics scott taylor for congress american academy of dermatology association political action committee (skinpac)texans for senator john cornyn inc ted budd for congress defend rural arizona pac doug lamalfa committee srcp media, inc. payers. However, as the more technical participant and Jacomy (2019) argue that visualisations mr darren grimes friends of scott desjarlais bucshon for congress make louisiana great again campaign for american principles lou barletta for congress indiana republican state committee, inc. scientists for eu limited rollins communications watchdog pac youtube rodney for congress awareness analytics partners llc new power pac ddb uk limited erin houchin for congress susan b anthony list inc indiana jobs now fp1 strategies fp1 digital llc right way superpac chinese americans for keeping america great enterprise freedom action committee families for james lankford walters for congress bilirakis for congress revily, llc zeldinmatthews for congress for nevada friends of todd young, inc. wagtv limited stripes agency llc new jersey family first inc darlene miller for congress make dc listen ryan costello for congress sawyer studios tea party forward martins for congress imge llc cfb strategies intermarkets inc essential liberty pac may for congress stand fortruth, inc. bluegrass committee committee for election of brian malnes to the us house of representatives; the avaaz campaigns uk martha roby for congress jpm strategies the catholic association aipr llc walorski for congress inc redprint strategy kristi for congress valor america bend the arc jewish action inc plaid cymru - the party of wales issa for congress humane society legislative fund restoration pac spotxchange texas patriots pac michelle ovens ltd the associated press granite state solutions boustany for senate inc cavalry strategies llc evolving strategies gilliard blanning & associates vern buchanan for congress bolding, darren k bobby mahendra for senate another europe is possible keep the promise pac victory phones nrcc coffman for congress 2016 friends of mia love kentucky tomorrow, inc. trump make america great again committee kyle mccarter for congress committee ted cruz for senate byrne for congress esafund civil liberties assault fund timd2016 david j. putmancatholicvote.org for congress mckinleyfreedomworks for congress for america acquire digital llc jeb 2016, inc. daniel webster for congress elbert guillory for america becker, beth connectivist media mds communications corporation friends of kelly ayotte inc mountaineers are always free pac mustard seed media llcalliance strategies group, llc campaign grid the eu referendum campaign limitedvote leave limited trade brigade of citizens trade thecampaign freedom association limited del ray media llc friends of kip tom noted: “these are the clusters according to this invite people to read into them their own pre- cowan for congress, inc. louisiana prosperity fund fighting for louisiana friends of neal dunncruz for president culac the pac of credit union national association granite voices pac dga action progressive change campaign committee jim costa for congress digital freedom, llc local voices referendum facts ltd denham for congressmchenry for congress win right data company llc vets for economic freedom trust dba concerned veteransdon bacon for america for congress labour leave limited retail association of maine inc. jason lewis for congress lynn jenkins for congress friends of john mccain inc scott walker inc newsmax media inc. huckabee for president, inc. liberal democrats keep the promise pete king for congress committee ann wagner for congress mooney for congressnational 2016 right to life the richard burr committee let's do better america action fund climate hawks vote political action labour party jeff stone for congress lobiondo for congress bill cassidy for us senate cresent hardy for congress randy hultgren for congress citizens for bongino right way initiative inc. mcsally for congress friends ofnew frank hampshire guinta republican state committee green party universities uk nrsc american wind action scott garrett for congress dan barrett for congress unison - the public service union chuck fleischmann for congress committee, inc. rooney for congress jan pac friends of stewart mills jason smith for congress lisa murkowski for usnevada senate republican central committee rebuilding america now carpenter for senate direct selling empowers americans deep roots analytics llc mentzer media services, inc. labour movement for europe friends of roy blunt biggs for congress new river research institute llc faith family freedom fund conservatives in ltd west virginia chamber of commerce blum for congress future in america inc. john kennedy for us derickson for congress uk independence party (ukip) committee to re-elect trent franks to congress wendyrogers.org elevated campaigns kyzle.com, llc tenney for congress walker 4 nc cogzero duffy for wisconsin texans for lamar smith kansans for marshall maxwell for congress free and strong america pac, inc. jim banks for congress, inc. targeted victory ending spending inc. algorithm…”, at which point, he clicked through conceptions of what is in the data. But what cotton for senate fastly inc. yoder for congress, inc lasee for congress, inc. stutzman for senate romney victory, inc. missouri republican state committee-federal red curve solutions romney for president, inc. mullin for congress trump victory gravis marketing lynch for congress 16 digiscore capital, llc strategic media placement lieutenant colonel oliver jenkins for congress wilson, wilson perkins better louisiana pac john bolton pac senate conservatives republican majority fund louie gohmert for congress committee americarisingpac.org team ryan stacy ries snyder for congress kirk for senate independent voice for illinois pac del beccaro for senate tom reed for congress heartland values pac liveintent inc. trott for congress, inc. target point consultinginc campaign solutions fund this advert axiom strategies deloitte political action committee bill mcclure for congress grow wv inc progressive wisconsin - let's rebuild america marsha blackburn for congress, inc. american crossroads gowan for arizona, inc majority committee pac--mc pac cd inc. citizen super pac believe againrick w. allen for congress character counts political action committee ddc advocacy llc several settings and windows, displaying different is interesting is that often these assumptions grassroots targeting llc portman for senate committee compass bancshares, inc. pac black america's political action committee andy barr for congress, inc. phillips for congress wicker for senate justin fareed for congress jobs opportunity and new ideas pac marco rubio for senate 2016 right to rise usa rothfus for congress monstah pac boozman for arkansas roberson for congress georgians for isakson california republican party federal acct senate leadershipbill shuster fund for congress service honor country action fund our principles pac enterprise holdings, inc. political action committee freedom for all pac drew ferguson for congress inc. friends of joe heck collins for congress elise for congress denise gitsham kansas republican party mike johnson for louisiana stop hillary pac the herald group, llc americas pac bookbag video trey gowdy for congress carly for america comstock for congress national rifle association citigroup inc. political action committee - federal (citigroup pac-federal) iheart radio committee to elect darryl glenn washington state republican party direct response strategies great america pac andrea zopp for us senate grant starrett for congress stivers for congress mike gallagher for wisconsin strategic perception, inc.amie hoeber for congress rga right direction pac message made easy usa business freedom pac future45 45committee inc. constitutional conservatives political list brokers llc kefalas for maryland inc carly for president friends of dennis ross friends of dave joyce ken bennett for congress zinke for congress national cable and telecommunications association political action committee (ncta pac) cole for congress emergency committee for israel giles-parscale napoleon for illinois maria for america 2016 gun owners of america, inc. tusk digital grassley committee inc buck for colorado 2016 hudsonpatterson for congress for congress opinion strategies llc mullins, brian tiberi for congress illinois republican party fls connent conservative action fund congressional black caucus pac jack flanagandr trey for congressbaucum campaign congressional leadership fund connell donatelli inc friends of paul mitchell 1360 llc resurgent media revive america pac phyllis schlafly's eagle pac friends for chris stewart, inc. Facebook public strategy associates llc faso for congress heaney for congress lcm strategies media placementfriends ofservices hans tanzler glenn grothman for congress harris media llc ello creative llc friends of dave brat inc. configurations of the network, complicating the would remain unarticulated without the graph. new day for america push digital llc rick kozell for congress raul labrador for idahoaustin scott for congress inc kathy for maryland shelby for u s senate donald j. trump for president, inc. quantcast corp. vertical strategies oliva for congress gravesrevolvis for congress consulting inc. ron johnson for senate inc american action news mckinnon publishing co. never means never pac arizona republican party crane for congress inc advoc8 ron desantis for florida zealous llc the edge ideas right strategiesluttrell llc for congress dold for congress pete sessions forvaladao congress for congress colorado republican committee rick reed media, inc. brilliant communications carlos curbelo congress ny republican federal campaign committee patel for louisiana marco rubio for president republican jewish coalition swing state strategies conservative, authentic, responsive leadership for you and for america kansans for huelskamp your voter guide culberson for congress national federation of independent business federal political action committee special operations for america texans for jodey arrington carlos beruff for senate inc paul gosar for congress john bolton super pac robert aderholt for congress america ascendant pac hurd for congress house liberty project rebecca negron for congress thomas massie for congress the stoneridge group llc secureamericanow.orgoptimus consultingfriends of david llc jolly cleveland 2016 host committeechatham inc light media llc cris dosev for congress taboola relentless positive action pac (rpa pac) olson for congress committee walberg for congress ohioans for a better future independent leadership fund robert blaha for colorado abis & company mcclung communications and public relations lloyd for congress arena online, llc pat meehan for congress volunteers for nehlen medium buying, llc pellerin for us senate llc red maverick mediafriends llc of pat toomey swiftkurrent, inc. committeebatts for oncongress, arrangements inc. for the 2016 republican national convention socially liberal fiscally conservative political action committee incorporated digital core campaign llc freedom partners mclaughlin & associates mulvaney for congress jones for congress america's liberty pac republican federal committee of pennsylvaniafriends of mike lee inc sundheim for senate emmer for congress friends of jim bridenstine inc. media dc huck pac rand paul for us senate let america work carolinians for freedom seemingly clear ‘finding’. In this particular case, the graph materialised an jamestown assoc. the washington times acxiom liveramp inc. ben west for oregon america leads liz cheney for wyoming tenacious truth pac mario diaz-balart for congress grow nc strong inc ad astra insights onmessage missouri rigt to life victory fund americans united for values mike rogers for congress jon keyser for senate trey for congress our rebellion media, llc kustoff for congress uaw - v - cap (uaw voluntary community action program) red digital resonate diane black fortaylor congress griffin for congress bill flores for congress i360, llc connect strategic communications, llc we work sales teads inc. libre initiative red sea, llc ros-lehtinen for congress renee ellmers for congress committee young for iowa, inc. advictory llc noelle nikpour for congress inc christopher peters for congress perdue for senate mica for congress cathy mcmorris rodgers for congress friends of scott angelle, llc mary thomas for congress carl domino for congresss house freedom fund targoz strategic marketing jim renacci for congressvirginia foxx for congress the prosper group corporation conserv america building a brighter future together committee for american sovereignty liquidus marketing citizens for cochran new jersey republican state committee gary palmer for congress sanford for congress maryland republican state central committee greer for congress gibbs for congress ohio republicanfreedom party state partners central &chamber executive committeeof commerce inc. growth & opportunity pac, inc. elect blake farenthold committee american future fund michigan republican party paul babeu for congress upton for all of us data targeting, inc. rob wittman for congress mike crapo for us senate tarkanianbob goodlatte for congress for congress committee forlini for us lance for congress comer for congress george holding for congress inc. professional data services forbes for congress moranarizona for kansas grassroots action pac wells for securitycharlie dent for congress barringer for congress friends of matt gaetz mobile walla moolenaar for congress state conservative reform action pac (scrap) innocenzi, sandler americanscampaign for committee prosperity for mark freeman katko for congress conservative solutions pacdan newhouse for congress kentuckians for strong leadership treehorn, llc prodege In this second encounter, the tension took the unspoken assumption, apparently held by many alamo pac oklahoma leadership council friends of sam johnson florida first project grit creative llc kevin mccarthy for congress citizens for john rutherford pittenger for congress llc pinpoint media llc concerned taxpayers of america voter gravity inc. tim scott for senate american opportunity pac friends of colonel rob maness davidson for congress kasich for america inc target enterprises, llc meridian pacific, inc. liberty and leadership fund eleventy marketing group llc hoosiers for rokita, inc. principled leadership majority strategies effective leadership for america katcho for congress carson america innovative advertising brady for congress bask digital media, llc friends of bill posey ryan-mccarthy victory french hill for arkansas citizens for turner steve knight for congress inboxfirst.com american action network inc optimizely republican party concerned american voters alternativepac north florida neighbors wirecom media hometown freedom action network gopac election fund rand paul for president, inc. rand paul new hampshire priorities huizenga for congress gus rantz for louisiana go big media, inc. garret graves for congress tmone, llc dba mass markets external affairs, inc. cando politics american medical association political action committee roskam for congress committee tom rooney for congress texanschris for christieintegrity for president inc reignite gurfein for america harmon brothers llc right digital llc fighting for ohio fund one nation ryan for congress, inc. chris vance for us senate maryland usa new york wins pac republican central committee of san luis obispo county-federal form of a wariness on the part of both researchers of the participants, that the UK was politically massachusetts republican party north dakota republican party ssc strategies, llc david rouzer for congress pompeo for kansas, inc. y2k service inc values are vital righters group llc victory enterprises heartland resurgence bergmanforcongress love for louisiana political action committee citizens for josh mandel inc causeway solutions llc strategic digital services, inc. justin grabelle for congress causeway consultants conston communications concentric direct republican hindu coalition rebekah for congress on 3 public relations, llc valor fund conquest communications group carlos lopez-cantera for senate house conservatives fund friends of becky gerritson plaster for congress arena communications scalise for congress club for growth primary data solutions coolhead inc. faye stewart for oregon rampart pac americanskiehne for congressfor concrete solutions, inc. cure 4 louisiana's future pac zoeller for congress henning for congress calabur productions llc conservative congress now! 406 enterprises llc war room strategies phone2action llc republican campaign committee of new mexico spoutable llc friends of roger reynolds montana republican state central committee e3 post connors for senate people for pinellas mike bishop for congress eagle forum pac north carolina republican party burns, clare synovation solutions mark harris for congress roger williams for u s congress committee patch media eighth congressional district republican party of minnesota in relation to their efforts at what they called “data closer to the left of US politics and that a use of pioneer outdoor utah republican party friends of erik paulsen sermo digital llc vici media group salmon for congress samuel martin rob bishop for congress strong nh action protect louisiana, inc. meath media services move maryland forward terra eclipse sensenbrenner committee ray strauss 4 congress protect the harvest political action committee strategic partners & media idaho republican party upstream communications jim for nc alaska republican party spotx stephen mackenzie for congress 2016 national association of realtors congressional fund madison pac for consitutional limited government america is worth it arizona rock product association right voter, llc the guardian fund price for congress 720 strategies llc vermont republican federal elections committee proofpoint noiseworks media leadership matters for america pac, inc. shark tank media llc friends of martin babinec raidious llc extensive enterprises atlantic list company cleaning”, interpretations of the graph and the political technologies would fall along political brian mast for congress federation of american hospitals pound, feinstein & associates deb fischer for us senate inc fenn communications group the casale group security is strength pac strong leadership for america mccaul for congress, inc john carter for congress kunnect llc colm for congress tom garrett for congress tma direct house freedom action fund panhandle victory fund “findings” they presented to their peers. Other lines. So while the researchers were highly sensi- 1and1.com morgan griffith for congress seabaugh, catherine media bridge llc renew america's greatness pac american legacy political action committee donor bureau datasets might have allowed them to defer their tised to their assumptions about the data being world net daily casperson for congress freedom's defense fund hsp digital ainsley shea vigop (virgin islands republican party) wwp strategies veterans for a strong america action group campaigngrid direct cuts and categorical decisions to actors in the field, baked into the “data cleaning” and ultimately phil pavlov for congress campaign for jobs and opportunity but in this case their impulse to be more empir- informing the graph, they were less attuned to ically-grounded clashed with the requirements their own assumptions about what they might of the chosen approach. They were required to find. This suggests that instead of graphs being make somewhat arbitrary choices based on their used to definitively demonstrate the existence of

13 Science & Technology Studies XX(X)

some phenomena, they could be used to provoke automated.9 I suggested to them, for example, reactions, to materialise unspoken expectations that one could automatically extract all the nouns, and assumptions. ‘the cast of characters’, in Smith’s language. They then picked a handful of stories and started high- Encounter 3 lighting the nouns. The last workshop I want to discuss concerned The participants quickly encountered online patient feedback and involved 13 par- problems. One pointed out that it was unclear if ticipants from a variety of backgrounds: health the repeated nouns are the same entities each researchers, former nurses and doctors, medi- time: “the nurse” and “the paediatrician” could cal sociologists and experts on health insurance represent multiple people. They also noticed some and digital health. The UK’s National Health Ser- interesting features of the texts, such as the way vice (NHS) among others have been attempting the opening sentences provided ‘instructions’ for to process mountains of digital patient feedback how to read what follows: for example, to suggest using machine learning and a technique called who the story is addressed to, or who is poten- sentiment analysis (which I will describe later). But tially responsible or culpable for what happened. these attempts to automatically extract the topic They also noticed that certain stories convey a and (positive or negative) sentiment of the feed- moral economy through what Smith (1978) calls back, belied fundamental sociological questions ‘contrast structures’ or the juxtaposition of two about what patient feedback is for and whether statements, implicitly rendering one as ‘good’ and or not patients, doctors and hospital manag- one as ‘bad’. ers understand it in the same way. For example, I had to admit to them that such implicit feedback might be used instrumentally to change devices, which have to do with sequences and policy, as an idle threat or as a cathartic unloading omissions in the text, are hard to capture auto- without expectation of a response. matically with current forms of automated textual The website Care Opinions (careopinions. analysis. We discussed possible automated org), which collects public and anonymised approaches, such as creating different corpuses patient feedback, agreed to give us access to with key-word queries and then using co-word their platform, from which we downloaded one maps or using word trees (Wattenberg and Viégas, month’s worth of anonymised feedback narratives, 2008) to pull out repeated phrasings. But each of referred to as “stories”, and their metadata. While these seemed to be dismissed as they discovered the researchers in this workshop were closer to interesting patterns which the tools would neces- the topic compared to the researchers in the other sarily miss. This is understandable: as researchers two workshops, they were also, on the whole, less who study technology, they were accustomed to experienced with these sorts of tools. Some of the looking for what different technologies, like these participants had worked on quantitative surveys tools of textual analysis, render invisible and leave and qualitative in-vivo coding, but never with out. This brings us to the slump I mentioned in web-based scrapers and network graphs. the introduction. Mid-way slumps are common One team, composed of three STS inclined in workshops of this sort, and they happened to ethnographers, was interested to see if there were some extent in the other two workshops, but this any automated means of applying Dorothy Smiths’ one seemed more oppressive, possibly because brand of textual analysis to these texts (Smith, these researchers were less technically adept 1978). As I explained in the opening vignette, after overall or because I had forced them to confront the experience of other workshops where groups the full texts first. too quickly started experimenting with tools like After some much needed coffee and pacing co-word maps, I purposely slowed things down around outside the stuffy room, I thought of a new and forced the group to analyse some of the provocation for the group. Instead of trying to feedback stories manually. However, I asked them approximate close readings in an automated way to read while “thinking like a computer” – keeping or scaling up Dorothy Smith’s approach, I asked in mind what aspects of their analysis could be them to start with a computational approach and

14 Moats 0 4 5 1 2 7 6 2 4 0 0 9 9 3 3 3 5 1 0 8 9 3 1 4 6 6 6 5 5 0 9 3 2 5 7 6 2 8 7 0 9 9 5 3 7 2 0 9 6 4 5 3 6 6 0 0 6 9 4 8 5 9 9 4 1 0 3 4 5 2 2 6 9 -1 -2 -1 -3 -2 -1 -3 -1 -2 -1 -1 28 12 14 34 15 14 14 11 13 24 17 21 10 21 14 12 47 47 26 26 26 35 22 14 14 37 11 10 30 61 25 30 13 18 43 20 15 24 14 64 18 26 38 12 15 19 16 26 18 10 34 12 39 12 17 14 24 10 31 34 12 24 14 11 20 28 14 27 Responsibility Score Responsibility +/- Score 1 0 2 1 0 1 0 0 0 2 3 0 0 6 0 3 3 1 24 24 0 4 2 0 6 6 6 1 1 2 5 5 1 Individuals 3 0 2 0 0 1 3 0 1 3 3 3 0 1 2 3 3 0 2 0 0 0 0 0 0 0 0 0 0 0 0 3 9 0 0 0 2 0 0 0 0 2 2 1 3 9 1 1 5 2 2 0 0 1 4 2 1 1 0 1 1 4 0 1 3 0 0 1 0 2 2 0 6 1 2 1 0 0 0 1 1 2 4 0 0 1 0 3 0 0 0 0 3 3 1 0 1 0 1 3 2 2 0 1 4 0 2 0 4 0 1 3 6 0 0 0 1 0 1 0 0 0 5 5 2 28 2 7 14 17 6 2 40 15 2 0 5 38 38 0 28 0 9 9 9 9 18 22 16 52 52 0 Collectives 14 0 23 10 12 2 16 9 6 29 29 29 0 36 10 12 6 22 3 14 14 4 6 6 37 11 10 6 5 5 0 33 18 3 61 2 27 5 7 6 30 4 15 19 11 6 0 44 25 17 0 7 0 10 28 16 10 6 64 4 0 22 26 8 5 38 0 10 12 8 17 4 11 20 5 7 6 0 0 17 27 8 13 18 4 11 34 11 5 12 39 12 12 12 18 4 2 0 15 6 26 12 31 5 38 12 7 24 18 2 3 14 26 6 28 9 15 27 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 worker 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 1 0 0 0 0 someone 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 anaesthetist 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 councellor 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 5 0 0 0 0 0 0 surgeon 0 0 1 0 0 0 0 0 0 1 1 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 4 0 midwife 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 staff nurse receptionist doctor gp secretary consultant matron midwife surgeon councellor anaesthetist someone worker 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 matron 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 2 0 0 0 0 0 0 0 10 10 0 0 0 0 0 0 0 1 0 0 1 1 0 consultant 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 3 1 0 1 1 0 0 0 0 1 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 2 0 0 1 0 1 0 0 0 0 0 1 0 0 1 0 0 0 2 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 3 3 0 0 0 0 0 0 0 0 0 2 0 0 0 secretary 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 staff nurse receptionist doctor gp secretary consultant matron midwife surgeon councellor anaesthetist someone worker 2 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 5 5 0 0 2 0 0 0 0 0 0 0 0 0 1 gp 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 2 0 0 0 5 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 doctor 0 0 0 0 0 1 2 0 0 0 0 0 0 0 0 2 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 2 1 0 1 0 0 0 0 0 0 0 0 0 0 0 2 0 0 1 0 1 1 0 0 0 1 1 1 1 0 0 0 0 2 0 0 0 0 1 2 1 0 0 0 0 1 0 0 0 1 1 0 2 0 0 0 0 1 2 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 receptionist 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 staff nurse receptionist doctor gp secretary consultant matron midwife surgeon councellor anaesthetist someone worker staff nurse receptionist doctor gp secretary consultant matron midwife surgeon councellor anaesthetist someone worker Specific 1 0 0 2 0 0 0 0 0 0 1 0 0 0 3 0 0 0 0 5 5 0 4 0 0 1 1 1 0 0 0 0 0 0 nurse 0 0 0 0 0 0 0 0 1 2 2 2 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 4 0 0 0 1 0 0 0 0 0 2 0 0 0 0 0 2 0 0 0 0 0 2 1 0 0 0 0 0 0 0 1 1 0 0 0 0 0 1 0 5 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 1 0 1 0 0 1 0 0 0 2 0 1 2 3 0 0 0 0 0 0 2 0 0 0 0 0 1 0 1 0 1 1 0 1 3 1 1 0 0 0 0 0 3 0 0 0 0 0 0 0 0 4 4 0 staff 1 0 2 0 1 1 0 0 0 3 3 3 0 0 0 4 0 3 0 0 0 2 1 1 0 1 0 0 1 1 0 3 1 0 2 1 1 0 1 0 0 0 0 3 0 0 0 2 2 1 0 1 0 2 1 0 1 0 4 0 0 2 3 1 0 2 0 2 1 2 2 2 0 1 0 1 0 0 0 0 0 1 0 1 0 2 1 1 1 1 0 1 1 3 0 2 1 0 2 0 0 0 1 1 2 1 0 1 2 1 0 1 1 0 0 0 1 1 0 3 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 2 0 0 0 1 0 0 0 0 0 team 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 1 0 0 0 3 1 1 0 0 0 0 1 0 0 0 3 0 0 2 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 1 1 0 0 3 0 0 0 1 0 0 0 3 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 1 0 1 0 1 0 0 0 0 0 0 1 1 0 ward 0 0 1 0 0 0 0 1 0 4 4 4 0 0 0 0 0 2 0 1 1 0 0 0 1 1 0 1 1 1 0 2 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 2 0 0 0 0 1 2 0 0 1 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 1 2 0 0 0 0 0 0 0 0 0 1 0 0 0 1 1 0 1 1 0 0 0 0 2 1 0 10 10 10 7 6 5 5 5 4 3 3 3 2 10 10 10 7 6 5 5 5 4 3 3 3 2 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 suite 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 1 0 0 2 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 2 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 2 0 0 0 0 0 0 0 0 0 0 department 0 0 0 0 0 0 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 place 0 0 0 0 0 0 NHS everyone no one A&E hospital centre clinic place department suite ward team staff NHS everyone no one A&E hospital centre clinic place department suite ward team staff 5 0 0 0 0 1 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 1 0 1 1 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 clinic 0 0 0 0 0 0 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 1 0 0 0 1 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 centre 0 0 0 0 0 0 6 1 0 1 1 1 1 0 3 0 0 0 0 0 1 1 0 0 0 4 1 0 0 0 0 0 1 0 0 4 0 0 0 0 1 0 0 1 0 1 1 0 1 2 1 0 0 0 1 0 0 0 1 4 0 0 2 2 1 0 0 0 1 0 0 0 0 1 3 0 0 1 0 0 2 2 1 0 1 0 0 4 0 0 0 1 0 0 0 1 0 0 0 1 1 1 1 2 0 0 1 0 1 0 0 0 0 0 1 3 1 0 2 0 0 0 0 0 0 0 3 0 0 1 0 1 0 4 1 0 0 0 0 0 0 1 0 0 0 0 0 0 2 1 6 6 0 hospital 2 0 1 0 0 0 5 4 3 3 3 2 10 10 10 7 6 5 5 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 2 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 A&E 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 2 2 0 1 0 0 0 0 0 0 0 1 0 0 0 no one 0 0 0 1 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 everyone 0 0 0 0 0 0 NHS everyone no one A&E hospital centre clinic place department suite ward team staff General General 10 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 1 0 0 1 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 2 0 1 0 3 0 0 0 0 0 0 0 0 0 0 0 0 1 0 NHS 0 0 1 0 1 0 kind and kind , So many people I wanted to mention mention to wanted I people many So . days 3 and everyone I had contact with was lovely was with contact had I everyone and 8 I was extremely ill and the doctors and nurses were amazing getting me getting amazing were nurses and doctors the and ill extremely was I . better ready to go home within within home go to ready better caring Body I seeked help from my doctor for my mental health issues with anxiety and anxiety with issues health mental my for doctor my from help seeked I and helpful Really amazing. were they and IAPT Ealing to referred was I depression. even realised never I that speech children's about things some learnt idea Fantastic each enjoyed and sessions the from benefited definitely has son My   existed. was nurse The professionally. very conducted was nurse diabetic a with review My queries My involved. feel to made and to listened was I caring. and polite courteous, blustery and wet very very a 999 on to call a in resulted injury football nasty A to Trying Glasgow. in someone by promptly answered was call The morning. Sunday discharged. been have I Mary with appointments three After after pregnant weeks 20 at girl little another lost sadly I and here daughter's 3 had I maternity the found have I great how say to want just I and stopped heart her praise not can I removal. polyp and colonoscopy a for hospital Pontefract Visited my during levels all at professionalism and care of level excellent the enough care The life. her of end atthe unit this in week a over just spent law in mother My was she last at and us to talk to busy too never were they amazing, just was staff of recount I and father her with appointment her attended daughter old year seven My follows: as experience the of opinion her for went I myself by bed of out get even couldn't I abscess spinal a had I there Hi in RVI the to rushed immediately was then hospital tyneside South at scan mri clinician assigned being to due now appointments of number a attended have We has each and once than more clinician same the see to yet have We care. led test. blood my for went I years 92 am I fall. a following received I care excellent the for staff the to Thanks me make to could they all did they but incident the by shocked quite was and old staff as the do with to nothing is today review my say too like would I all of Forts a taken had child old year 1 seen. My got eventually we when brilliant were they the to credit a were operation hernia my of day the on duty on team staff The the in time my throughout respect and courtesy with treated was hospital.I and professional supportive ,kind, helpful be up practice GP this found always have I and listener good very a is particular in GP One . GPS the to staff reception the from his for late be would he know them let to Oct 3rd the on rang husband My change to have would and late be not could he told was he bus the to due screening my with me help to week a once out come (Emily) worker start sure a had I the around safe children my keeping and routines time bed behavior, children's 6- around were there pit arm right my under however years for hydradenitis had Ive consultant with appointment an had i refilled and burst continually that abcesses 8 6- around were there pit arm right my under however years for hydradenitis had Ive consultant with appointment an had i refilled and burst continually that abcesses 8 wishaw at triage call to decided i so morning sunday on movements reduced had I hospital. general them. endanger and patients ill of family and friends the to listen they feel don't I acted. they've way the to listened not have they illness, relative's my During it but jab flu the from effect side acceptable an was throat sore mild a thought I a have still I later week a and years, in had I've cold worst the become to on went (the recorder loop internal my of removal for Suite Catheter Cardio the visited I myself both after once cancelled been already had appointment version)..My older 2017. august in Edinburgh in ENT to GP her from referral a received daughter My have to likely is she long how out work and progress check to today called I When I area, "their" of out moved now having despite and baby 4th my with pregnant I'm of continuity some for midwives community of team same the with stay to wanted few a wrist my broken having September 2017 in Royal Bolton at E & A attended I 4 of date operation standby a g professional. Had a series of CBT over a couple of months and feel in a much more much a in feel and months of couple a over CBT of series a Had professional. you. thank big a and lady Fantastic   session. mutual a were conclusions and making Decision appropriately. to responded were in was location the if asked was I indeed tricky, was location exact the across get better a have now I resolved. been has years many of problem my her to Thanks my at looked lovingly midwife my way the From wight. of isle the on here unit/staff and Pinderfields at out carried procedures and ops many had have I procedure. respect. with treated be and comfortable made be could she where somewhere was Friday previous the Physio by Disease Severs of diagnosis the that Confusion - I'm even though brilliant were cleaner's to nurses from staff 16 ward the Newcastle advice This baby. the deliver to wife my for way best the on advice differing offered afternoon. Monday early chose and anytime go could I comfortable. after face his in it popped and machine washing my of out tablet wash bio non fairy the of each stage at happening was what of informed kept and department . me to important is this approach, their in flexible is not so appointment, important very very a cancelled already had I appointment, his helped she much how and was she amazing how enough express not can I home, heal. wont and tracked have they as surgery need would i said who heal. wont and tracked have they as surgery need would i said who to Matron Community the asked doctor my hospital, in stay short a had recently I visit. home a arrange and me ring 'mild' as this, about statistics the to added be to story my like would I cough. painful my for arrived I day the On work. with arrangements making husband my and assured but weeks wasn't it checked double I months. 18-20 is it told was I wait to who people those of one not am I exemplary. been has care their say must I care. I area, "their" of out moved now having despite and baby 4th my with pregnant I'm of continuity some for midwives community of team same the with stay to wanted who people those of one not am I exemplary. been has care their say must I care. 8 for standby new a given was I and cold a to due cancelled was This later. days 8 for standby new a given was I and cold a to due cancelled was This later. days 8 for standby new a given was I and cold a to due cancelled was This later. days attended I tests. my for especially appointment the instating re trouble of lot a had I my mid-labelling like hospital this about dreadful astonishingly things certain found I vice and folder my in scan else's somebody had I meant which knee my of scan MRI they what explain to able being without samples blood take to arriving Nurse versa. I me. to given where numbers other two so , call my transfer not can .they board i mail voicemail to goes just it and day single every ringing been I've the if voicemail and organised very process whole the found initially and useful as provided and organised very process whole the found initially and useful as provided required many the asked immediately, ease my at me put and home my to came in seen be would she thought we which to 101 ward to admitted was daughter My in actually was case the not was this Unfortunately discharged. then clinic the and dignity with treated was daughter my 101 ward In weeks. three for hospital in seen be would she thought we which to 101 ward to admitted was daughter My in actually was case the not was this Unfortunately discharged. then clinic the and dignity with treated was daughter my 101 ward In weeks. three for hospital in seen be would she thought we which to 101 ward to admitted was daughter My in actually was case the not was this Unfortunately discharged. then clinic the and dignity with treated was daughter my 101 ward In weeks. three for hospital called be not will You to. need you when appointments get to able be not will You to sent been have prescriptions told be not will You abnormal. are results test when need. you that suppor Title Depleted mental health mental Depleted Talk Home nurse with review Diabetic disappointed and Angry clinic bowel and bladder Hill Biggin children my Having categories. all in Excellent Exemplary... wing care Palliative injury heel Child's u thank consultants. and 16 Ward clinic and scan Maternity Test Blood Patients Out Efficient received. I care excellent The seriously baby my with wait hours 3 staffed under Centre Treatment the in staff The 2017 October 2nd Practice GP Good Very screening Diabetic worker Start Sure op post - Surgery op post - Surgery weeks 33 at movements Reduced pregnant ward Finch Care Integrated South Ashfield Team Flu jab Best the Not child for list waiting month 18-20 ENT exemplary been has care Their exemplary been has care Their Bolton at treatment wrist Broken hospital. Royal Bolton at treatment wrist Broken hospital. Royal Bolton at treatment wrist Broken hospital. Royal Thanks Dreadful shoulder total - unit surgical Day replacement Waiting dept physiotherapy the to... appointment.Tryin answer No treatment prostate aparently apointments cancelled normal Confusion Appointment Maternity Confusion Appointment Maternity served better been have not Could and stay hospital daughters My surgery unexpected and stay hospital daughters My surgery unexpected and stay hospital daughters My surgery unexpected dangerously care of Level insufficient shambles of cataract A system appointment Inefficient Department Outpatient Mixed Experience 2017 May augmentation Breast 5 ward in stay day 6 Positive E and A in grandad My OT the how of experience positive A recovery my facilitated service OT the how of experience positive A recovery my facilitated service experience endoscopy Great staff all for Appreciation staff all for Appreciation hospitals in gel hand Purell staff all from attention of Best advice Good help their for team the all to Thanks here be to pleasure a was It the all makes face cheery a Seeing difference one day from amazing been has She care and treatment Excellent relaxing and Modern cemo for treatment on follow Bad was... moved when again! Never service excellent on mother old year 84 with visit 01.10.17 seating moulded for Waiting Pathway PD Hospital Rampton service staff Professional clinic the to getting with Issues the from communication No department service efficient Quick support of level Highest hysterectomy Gynecology admin Poor - service ,efficient Excellent staff centre birth with care Excellent and... care excellent 14: Ward emergency c6 Distectomy life my changed doctor This clinic Great people Amazing Surgery Knee Arthroscopic everyone to you Thank Ward Kilverstone in stay Short Dept Ent in treatment Great Shocking more for ask couldn’t Fabulous Dermatology samples Fertility 2nd Monday on Surgery Day 2017 October service better for ask not Could anywhere! service Faultless much! so Thanks experience!! pleasurable A back life my got have I feel I with contact phone make to trying an... cardiology dept x-ray in walk the Attending Admission Emergancy service Wonderful clinic Eye unit surgery Day scan MRI consultants the with pleased Very Clinic Care Urgent clinic menopause in service Excellent appointments Podiatrist grateful very are son my and Me Overcrowded A+E in blood my Taking treatment Excellent list waiting good, nurse and doctors not nurses Lovely Endoscopy the to you thank big A Dept. appointment Outpatient care eye Diabetic colonoscopy CT Continuation + Dr by care Excellent Yoga... of team friendly wonderful A Appointment Dental care NHS Excellent Centre Care Urgent in staff Lovely ward nelson fault not could repair hernia a for on operated Was efficient Very therapy Physio experience caring Excellent expertise Superb Replacement Hip Total times few a Urology the visited I it... recently, NHS for service different A patients?? wonderful are team Endoscopy Care Excellent Fantastic worse. be couldn't Attitude service excellent doctors Arrogant Surgery Bladder Gall ankle on surgery Had outstanding is Service fault Couldn't Team Endoscopy Brilliant experience Terrible go. tests blood as far as Great Can't treatment. mid left been Have to... seem Unit Endoscopy Care Wonderful service wonderful A receptionist the that Horrified a... be to seems Link https://www.careopinion.org.uk/opin ions/399572 https://www.careopinion.org.uk/opin ions/399575 https://www.careopinion.org.uk/opin ions/399583 https://www.careopinion.org.uk/opin ions/399587 https://www.careopinion.org.uk/opin ions/399589 https://www.careopinion.org.uk/opin ions/399590 https://www.careopinion.org.uk/opin ions/399599 https://www.careopinion.org.uk/opin ions/399600 https://www.careopinion.org.uk/opin ions/399601 https://www.careopinion.org.uk/opin ions/399602 https://www.careopinion.org.uk/opin ions/399603 https://www.careopinion.org.uk/opin ions/399604 https://www.careopinion.org.uk/opin ions/399605 https://www.careopinion.org.uk/opin ions/399609 https://www.careopinion.org.uk/opin ions/399610 https://www.careopinion.org.uk/opin ions/399611 https://www.careopinion.org.uk/opin ions/399612 https://www.careopinion.org.uk/opin ions/399617 https://www.careopinion.org.uk/opin ions/399622 https://www.careopinion.org.uk/opin ions/399622 https://www.careopinion.org.uk/opin ions/399624 https://www.careopinion.org.uk/opin ions/399626 https://www.careopinion.org.uk/opin ions/399634 https://www.careopinion.org.uk/opin ions/399636 https://www.careopinion.org.uk/opin ions/399643 https://www.careopinion.org.uk/opin ions/399650 https://www.careopinion.org.uk/opin ions/399653 https://www.careopinion.org.uk/opin ions/399653 https://www.careopinion.org.uk/opin ions/399664 https://www.careopinion.org.uk/opin ions/399664 https://www.careopinion.org.uk/opin ions/399664 https://www.careopinion.org.uk/opin ions/399665 https://www.careopinion.org.uk/opin ions/399667 https://www.careopinion.org.uk/opin ions/399668 https://www.careopinion.org.uk/opin ions/399669 https://www.careopinion.org.uk/opin ions/399670 https://www.careopinion.org.uk/opin ions/399671 https://www.careopinion.org.uk/opin ions/399672 https://www.careopinion.org.uk/opin ions/399673 https://www.careopinion.org.uk/opin ions/399684 https://www.careopinion.org.uk/opin ions/399684 https://www.careopinion.org.uk/opin ions/399687 https://www.careopinion.org.uk/opin ions/399690 https://www.careopinion.org.uk/opin ions/399690 https://www.careopinion.org.uk/opin ions/399690 https://www.careopinion.org.uk/opin ions/399693 https://www.careopinion.org.uk/opin ions/399696 https://www.careopinion.org.uk/opin ions/399697 https://www.careopinion.org.uk/opin ions/399700 https://www.careopinion.org.uk/opin ions/399704 https://www.careopinion.org.uk/opin ions/399706 https://www.careopinion.org.uk/opin ions/399708 https://www.careopinion.org.uk/opin ions/399710 https://www.careopinion.org.uk/opin ions/399710 https://www.careopinion.org.uk/opin ions/399724 https://www.careopinion.org.uk/opin ions/399725 https://www.careopinion.org.uk/opin ions/399725 https://www.careopinion.org.uk/opin ions/399728 https://www.careopinion.org.uk/opin ions/399729 https://www.careopinion.org.uk/opin ions/399732 https://www.careopinion.org.uk/opin ions/399733 https://www.careopinion.org.uk/opin ions/399734 https://www.careopinion.org.uk/opin ions/399735 https://www.careopinion.org.uk/opin ions/399736 https://www.careopinion.org.uk/opin ions/399738 https://www.careopinion.org.uk/opin ions/399739 https://www.careopinion.org.uk/opin ions/399740 https://www.careopinion.org.uk/opin ions/399742 https://www.careopinion.org.uk/opin ions/399743 https://www.careopinion.org.uk/opin ions/399745 https://www.careopinion.org.uk/opin ions/399747 https://www.careopinion.org.uk/opin ions/399748 https://www.careopinion.org.uk/opin ions/399749 https://www.careopinion.org.uk/opin ions/399750 https://www.careopinion.org.uk/opin ions/399752 https://www.careopinion.org.uk/opin ions/399777 https://www.careopinion.org.uk/opin ions/399779 https://www.careopinion.org.uk/opin ions/399783 https://www.careopinion.org.uk/opin ions/399786 https://www.careopinion.org.uk/opin ions/399791 https://www.careopinion.org.uk/opin ions/399792 https://www.careopinion.org.uk/opin ions/399793 https://www.careopinion.org.uk/opin ions/399794 https://www.careopinion.org.uk/opin ions/399800 https://www.careopinion.org.uk/opin ions/399809 https://www.careopinion.org.uk/opin ions/399810 https://www.careopinion.org.uk/opin ions/399811 https://www.careopinion.org.uk/opin ions/399813 https://www.careopinion.org.uk/opin ions/399814 https://www.careopinion.org.uk/opin ions/399816 https://www.careopinion.org.uk/opin ions/399817 https://www.careopinion.org.uk/opin ions/399818 https://www.careopinion.org.uk/opin ions/399819 https://www.careopinion.org.uk/opin ions/399820 https://www.careopinion.org.uk/opin ions/399821 https://www.careopinion.org.uk/opin ions/399822 https://www.careopinion.org.uk/opin ions/399823 https://www.careopinion.org.uk/opin ions/399832 https://www.careopinion.org.uk/opin ions/399837 https://www.careopinion.org.uk/opin ions/399839 https://www.careopinion.org.uk/opin ions/399840 Stories with cumulative generalised responsibility scores responsibility generalised with cumulative 3. Stories Figure

15 Science & Technology Studies XX(X)

modify it. I gave them the example of sentiment the sorting and analysis of stories with a more analysis, mentioned earlier, which Care Opinions open mind. What practitioners would make of already used in the backend of their website. this metric, however can only be tested through Sentiment analysis, in its simplest form, works dialogue with them, though both the medical using a library of words that are deemed to be professionals and the computer scientists in the inherently positive or negative (ranging from -5 room seemed to be intrigued by the approach. to +5). The words in a sentence are added up to The tension in this encounter, once again, produce a sentence score (taking account of basic manifested itself differently than in the other modifiers like “not ____” and other rules).10 The workshops. The group’s scepticism towards the objective, as I put it to them, was then to come up tools seemed to be based on a lack of fit between with a system which mirrored sentiment analysis what the graphs could see and what they could but improved on it by making it either more see as textual analysts. Something was deemed sensitive, more nuanced or more targeted to the to be ‘lost’ in the translation of full texts to texts- specific problem of analysing patient feedback. as-data; between close and ‘distant’ reading. What the group arrived at, after some delibera- While this is understandable given the immediate tion, was that entities named in the story could juxtaposition of the two, it puts STS scholars and be conceived of on a spectrum of more specific anthropologists in the odd position of being to more general. “A nurse” or “the nurse” was more “realist” about texts as one participant put it after specific than “the staff”, “the hospital”, “the NHS” or the workshop – believing that textual extracts, the practice of medicine in general – and this has or rather manual readings of them, are more real very different implications for how responsibility than computational representations of them. Yet, was being distributed in the text. They made a in other situations, the same researchers might brief library of common nouns and pronouns and protest that these texts are also a very partial, then assigned them rankings from 0 – 10. There performed account of another reality: the rich was much joking about the absurdity of assigning social world of interactions in a hospital.11 number scores to these words, but the group It was also interesting that the participants seemed happier to commit to the process, given seemed stuck when starting from qualitative close that it was undertaken with a sense of play. The reading and approaches like ethnomethodology, words appearing in the text could be added up to but by starting with a tool and asking how to determine what we called “generalised responsi- modify it, they actually managed to engage in the bility scores” for the story as a whole, and stories design of a computational approach. This seemed could then be colour coded (as in the above to satisfy their scepticism but also allowed them image), ranked or graphed in various ways. to create something legible to computational Now, obviously ranking the generality of words researchers in the room at the same time. also relies on faulty assumptions and requires abstracting words from their wider context, but Discussion such a provisional metric is still a more compel- ling or social scientific way of sorting texts than I think these brief vignettes demonstrate that, whether they are positive or negative. In addition, while there is certainly a genuine will by many such an approach could be used to launch a researchers to have closer collaborations between critique or light parody of sentiment analysis disciplines and make use of new digital tools, and related techniques, without dismissing such there are still tensions, disciplinary baggage and automated techniques all together. For this resistances which need to be dealt with. What I reason, it could also make an interesting inter- want to talk about in this section is how to charac- vention in the field because, while doctors and terise these tensions as well as ways of overcom- hospital bureaucrats probably already have their ing them. I described the first tension in terms of preconceptions about ‘positive’ and ‘negative’ disappointment that is a disappointment at the feedback, they likely do not have preconceptions tools not performing as they are supposed to, of ‘generalised responsibility’ and may approach instrumentally speaking. The second I described

16 Moats as a wariness toward the assumptions imbued in As Katie Vann (2010) notes, when Stengers these tools and the third I described as a scep- invokes a distinction between ‘humour’ and ‘irony’ ticism toward the reduction of one reality to (2000), she is possibly drawing on a distinction another. between humour and irony made by Deleuze and I think that each of these tensions, interestingly, this use of irony is subtly different from the one entail some version of the ironic fallacy as under- Woolgar deploys. She refers to an obscure discus- stood by Woolgar (1983): applying certain criteria sion of Sadism and Masochism in which Deleuze to one’s disciplinary other, without applying the distinguishes the two not as positions in a fetish same criteria to one’s self. In the first case, the relationship but in terms of the ‘scenography’ disappointment arose because interpretivist employed by the two authors de Sade and Masoch researchers seemed to have instrumental expecta- in response to modernity – the situation in which tions of the graphs but did not imagine an instru- ‘the law’ is no longer grounded on founding prin- mental use of their own methods in the service of ciples (Vann, 2010). Sade is ‘ironic’ because he making better graphs. In the second, the group critiques the law by ‘ascent to higher principals’, were wary of the assumptions required to make in his case, committing to Evil. This exposes the the graphs work but then were surprised when law as comparatively without principles. Masoch confronted by other assumptions they had held is ‘humorous’ because he subverts the law by which were materialised by the graph. In the third, ‘descent to consequences’, adopting the law and the researchers critiqued computational repre- the punishment in the absence of any crime. This sentations for failure to live up to another set of also exposes the absurdity of the law, by pushing qualitative representations which were, at least in it to its logical conclusion. the heat of the moment, treated as somehow less This schema does not seem very helpful for constructed. the first example because if we consider ‘the law’ This is not to criticise these researchers or to be something like the rules and routines of propose that they did something wrong. Rather, I a discipline or method then not much subver- think that slipping into these positions is an occu- sion was happening, because researchers and pational hazard of doing interdisciplinary work, methods were kept separate. If we wanted to use which others working in these ways will hopefully the casual metaphor of sexual relationships we recognise. These insights come from discussions might characterise this as a loveless marriage, or with the participants afterwards who, with the at least a marriage without anything ‘kinky’ going benefit of hindsight, regretted various ways things on. Now on first glance we might say that the played out. In fact, one of the main effects of these second example is humorous because it is maso- workshops has been to make the author more chistic, in the sense that it employs a strategy of aware of his own tendency to fall into this ironic self-effacement and self-critique. We might then stance. say that the third example is ironic because it As for the responses, which purport to address takes on the tools of data analysis ‘ironically’, in the or contain these frictions or tensions, the first colloquial sense of, ‘not how they were intended’, example I characterised as ‘diplomatic’,12 keeping or with a distancing wink. But I think Vann’s (2010) methods separate but relations respectful, the analysis suggests otherwise. Perhaps the second second was a ‘modest’ and ‘reflexive’ response example is actually ironic because it exposes the and the third was a playful appropriation of a laws of computational methods to be absurd method by a team who might otherwise have by recourse to a higher principle: in this case rejected it. The first response should be familiar reflexivity or a belief in the constructedness of to many researchers and the second is very all knowledge (a principle which both ethnog- common in Digital STS circles, but the third I think raphy and data analysis are both subject to). It is more surprising. How might we make sense also could be the case that the third example is of these negotiating strategies? Do they repair actually humorous because it takes up the laws of or reproduce the ironic distance which seems to computational methods and pushes them to their have prompted them? logical conclusion, to absurdity: if we are going to

17 Science & Technology Studies XX(X)

rank entities, then why not these. While Stengers ples of some practical tensions as they played out and possibly Deleuze have a normative commit- in a series of workshops and how they were nego- ment to humour, which is imminent as opposed tiated. By negotiated I do not necessarily mean to irony which is transcendent, Vann argues that that these tensions were resolved, but also put ‘on both are possibly part of the same move and both the table’, ‘decomposed’ (Kaltenbrunner, 2014) or have the potential to “…turn mutual interlocutors folded into the eventual outputs. into equals” (Vann, 2010: 86).13 So without advo- I should clarify that due to the brief nature cating one approach over the other, I want to of these workshops, these tensions necessarily suggest that both may be productive strategies in pertained to the initial, exploratory stages of a relation to different sorts of tensions. Both retain study, rather than the consolidation of findings, the sceptical edge of much critical work on data questions of validity and reliability, which have and computational approaches but, I argue, do been the crux of mixed methods debates. More so in more productive ways than mixed methods empirical investigations should be undertaken diplomacy because they involve a breaking down on these later aspects of collaborations in order or questioning of roles. to understand how the normative commitments However, this remains only a potential in these of data scientists and computer scientists interact two later examples because is not clear that the with the concerns of anthropologists and micro- absurdities made present were allowed to cut sociologists and particularly how the divergent both ways. In the second example, we did not quite end-products of research are negotiated between allow the graph to trouble our assumptions (only disciplines. after the fact) and in the third example a computa- In any case before these more equitable or tional guise was adopted but not for long enough long term collaborations can proceed, ‘interpre- to produce any defensible knowledge from it. It tivist’ researchers might need a stronger sense was still not clear in these instances that these of different roles they can take on in the proceed- researchers have allowed computational methods ings and how they could make use of digital and disciplines to inform their own perspectives, tools in ways that address their own wariness or or to paraphrase Woolgar (1983, 262), “…make scepticism. What I hope to have accomplished [data] science talk to sociology rather than the in this paper is to encourage these researchers other way around.” to examine their own baggage and normative commitments and approach these computational Conclusion tools with ‘humour’ or ‘irony’, as the situation demands. Thus interdisciplinary collaborations The purpose of this paper has been to consider could be thought of in terms of what Woolgar empirically some of the possible disciplinary or (1983) calls ‘irony as a project’, but this time a methodological tensions arising from the rapid collective one, aimed at opening up (but not proliferation of digital data and computational settling) the problem of what makes accounts of approaches to analysing it. This is important reality adequate and for whom. because, while disciplines like anthropology, STS, sociology and HCI have made many legitimate political and methodological critiques of certain Acknowledgements forms of computational analysis, they are not I would like to thank Brit Ross Withereik, Torben always in a position to influence their develop- Jensen and Minna Ruckenstein for advice on an ment. This may be a result of the ironic distance earlier version of this paper as well as Alexandra effected through these critiques, or of the way dif- Supper and the anonymous reviewers for ferent existing frameworks for collaboration keep improving the final product considerably. I also these disciplines in their lanes, muffling dissenting wish to thank the Swedish Riksbankens Jubile- views. I argued that understanding what these umsfond for funding this work. Most importantly, tensions are composed of, and thus how they I thank the participants of the workshops from might be overcome, might require some poking whom I learned a great deal. and probing to bring them out. I gave three exam-

18 Moats

References Abbott A (2016) Processual Sociology. Chicago: University of Chicago Press. Abreu A and Acker A (2013) Context and Collection: A Research Agenda for Small Data. iConference 2013 Proceedings (ed Shamber, Linda ) Fort Worth, TX, USA 12-15 February, 2013: 549-554. Grandville, MI: iSchools. Adams V (2016) Metrics: What Counts in Global Health. Durham, NC, USA: Duke University Press. Anstead N (2017) Data-driven campaigning in the 2015 United Kingdom general election. The International Journal of Press/Politics 22(3): 294–313. Barocas S (2012) The price of precision: Voter microtargeting and its potential harms to the democratic process. In: Proceedings of the first edition workshop on Politics, elections and data, Maui, HI, USA, November 2012. 31–36. New York, NY: ACM. Barry A, Born G and Weszkalnys G (2008) Logics of interdisciplinarity. Economy and Society 37(1): 20–49. Bell G (2011) Unpacking cars: Doing anthropology at Intel. Anthro Notes: Museum of Natural History Publica- tion for Educators 32(2): 1–20. Benjamin R (2019) Race after Technology: Abolitionist Tools for the New Jim Code. Cambridge: Polity. Berry D (2012) Introduction. In: Berry D (ed) Understanding Digital Humanities. London: Palgrave MacMillan, pp. 1-20. Birkbak A (2016) Caring for Publics: How Media Contribute to Issue Politics. Aalborg, Denmark: Aalborg Univer- sitetsforlag. Available at: https://vbn.aau.dk/en/publications/caring-for-publics-how-media-contribute- to-issue-politics (accessed 2 April 2020). Blok A and Pedersen MA (2014) Complementary social science? Quali-quantitative experiments in a Big Data world. Big Data & Society 1(2): 1-6. Blok A, Carlsen HB, Jørgensen TB, et al. (2017) Stitching together the heterogeneous party: A complemen- tary social data science experiment. Big Data & Society 4(2): 1-15 boyd d and Crawford K (2012) Critical questions for big data. Information, Communication & Society 15(5): 662–679. Burrows R and Savage M (2014) After the crisis? Big Data and the methodological challenges of empirical sociology. Big Data & Society 1(1): 1-6 Callon M (1998) The Laws of the Markets. Oxford: Blackwell Publishing. Callon M, Law J and Rip A (eds) (1986) Mapping the Dynamics of Science and Technology. London, UK: Springer. Card SK, Mackinlay JD and Shneiderman B (1999) Readings in Information Visualization: Using Vision to Think. Burlington, MA, USA: Morgan Kaufmann. Clavarino AM, Najman JM and Silverman D (1995) The quality of qualitative data: Two strategies for analyzing medical interviews. Qualitative Inquiry 1(2): 223–242. Coopmans C (2014) Visual Analytics as Artful Revelation. In: Coopmans C, Vertesi J, Lynch M and Woolgar S (eds) Representation in Scientific Practice Revisited. Cambridge, MA: MIT Press, pp. 37–59. Costas R, Zahedi Z and Wouters P (2015) Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Infor- mation Science and Technology 66(10): 2003–2019. Danowski JA (2009) Inferences from word networks in messages. In: Krippendorff KH and Bock MA (eds) The Content Analysis Reader. London: Sage, pp. 421–429.

19 Science & Technology Studies XX(X)

Daston L and Galison P (2007) Objectivity. New York, NY, USA: Zone Books. Denzin NK (2010) Moments, mixed methods, and paradigm dialogs. Qualitative Inquiry 16(6): 419–427. Denzin NK (2012) Triangulation 2.0. Journal of Mixed Methods Research 6(2): 80–88. Dourish P (2006) Implications for design. In: Proceedings of the SIGCHI conference on Human Factors in computing systems (Grinter R and Rodden T eds). Montréal, Québec, Canada April 2006, pp. 541–550. New York, NY: ACM. Downey GL and Zuiderent-Jerak T (2016) Making and doing: Engagement and reflexive learning in STS. In: Felt U, Fouché R, Miller CA, et al. (eds) The Handbook of Science and Technology Studies. 4th ed. MIT Press, pp. 223-254. Edwards PN (2010) A Vast Machine: Computer Models, Climate Data, and the Politics of Global Warming. Cambridge, MA: MIT Press. Gerlitz C and Helmond A (2013) The like economy: Social buttons and the data-intensive web. New Media & Society 15(8): 1348–1365. Gieryn TF (1983) Boundary-work and the demarcation of science from non-science: Strains and interests in professional ideologies of scientists. American Sociological Review 48(6): 781–795. Gilbert PR (2015) Trouble in para-sites: Deference and influence in the ethnography of epistemic elites. Anthropology in Action 22(3): 52–62. Glaser B and Strauss AL (1967) The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago: Aldine Publishing. Gould SJ (2011) The Hedgehog, the Fox, and the Magister’s Pox: Mending the Gap between Science and the Humanities. Cambridge, MA: Harvard University Press. Hammersley M (1992) Deconstructing the qualitative-quantitative divide. In: Brannen J (ed.) Mixing Methods: Qualitative and Quantitative Research. London: Routledge. Iliadis A and Russo F (2016) Critical data studies: An introduction. Big Data & Society 3(2): 1-7. Irani L (2015) Hackathons and the Making of Entrepreneurial Citizenship. Science Technology and Human Values 40(5): 799–824. Jensen TE (Forthcoming) The Slalom Method: Steering the course between ethnographic field work and digital methods. Jensen TE (2013) Techno Anthropology: A new move in science and technology studies. STS Encounters- DASTS working paper series 5(1): 1–22. Kaltenbrunner W (2014) Decomposition as practice and process: Creating boundary objects in computa- tional humanities. Interdisciplinary Science Reviews 39(2): 143–161. Kaltenbrunner W (2015) Scholarly labour and digital collaboration in literary studies. Social Epistemology 29(2): 207–233. Latour B, Mauguin P and Teil G (1992) A note on socio-technical graphs. Social Studies of Science 22(1): 33–57. Law J (2004) After Method: Mess in Social Science Research. London: Routledge. Lazer D, Pentland AS, Adamic L, et al. (2009) Life in the network: the coming age of computational social science. Science 323(5915): 721. Leonelli S, Rappert B and Davies G (2017) Data Shadows: Knowledge, Openness, and Absence. Science, Tech- nology, & Human Values 42(2): 191–202. Lippert I and Verran H (2018) After Numbers? Innovations in Science and Technology Studies’ Analytics of Numbers and Numbering. Science & Technology Studies 31(4): 2–12.

20 Moats

Loukissas Y and Pollock A (2017) After Big Data Failed: The Enduring Allure of Numbers in the Wake of the 2016 US Election. Engaging Science, Technology, and Society 3(0): 16–20. Lynch M (1988) The externalized retina: selection and mathematization in the visual documentation of objects in the life sciences. Human Studies 11(2): 201–234. MacKenzie DA, Muniesa F and Siu L (2007) Do Economists Make Markets?: On the Performativity of Economics. Princeton, NJ: Princeton University Press. Madsen AK and Munk AK (2019) Experiments with a data-public: Moving digital methods into critical proximity with political practice. Big Data & Society 6(1): 1-14. Manovich L (2012) Trending: the promises and the challenges of big social data. In: Gold MK (ed) Debates in the Digital Humanities. Minneapolis, MN: University Of Minnesota Press. Marres N (2012) The redistribution of methods: On intervention in digital social research, broadly conceived. The Sociological Review 60(S1): 139–165. Marres N (2017) Digital Sociology: The Reinvention of Social Research. Cambride, UK: Polity. Marres N and Moats D (2015) Mapping controversies with social media: The case for symmetry. Social Media + Society 1(1): 1–17. Maxwell JA (2010) Using numbers in qualitative research. Qualitative inquiry 16(6): 475–482. Moats D and Seaver N (2019) ‘You Social Scientists Love Mind Games’: Experimenting in the ‘divide’ between data science and critical algorithm studies. Big Data & Society 6(1): 1-11 Mohr JW (1998) Measuring meaning structures. Annual Review of Sociology 24(1): 345–370. Munk AK and Jensen TE (2015) Revisiting the Histories of Mapping. Ethnologia Europaea 44(2): 31. Munk AK, Tommaso V and Meunier A (2016) Data Sprints: A Collaborative Format in Digital Controversy Mapping. In: Vertesi J and Ribes D (eds) Digital Sts: A Fieldguide for Science and Technology Studies. Princeton, NJ: Princeton University Press, pp 472-496. Munk AK, Madsen AK and Jacomy M (2019) Thinking through the Databody. In: Nicewonger NE and Elam M (eds) Designs for Experimentation and Inquiry: Approaching Learning and Knowing in Digital Transformation. London, UK: Routledge, pp 110–128. Murthy D (2008) Digital ethnography an examination of the use of new technologies for social research. Sociology 42(5): 837–855. Neff G, Tanweer A, Fiore-Gartland B, et al. (2017) Critique and contribute: A practice-based framework for improving critical data studies and data science. Big Data 5(2): 85–97. Neyland D (2016) Bearing account-able witness to the ethical algorithmic system. Science, Technology, & Human Values 41(1): 50–76. Pantzar M, Ruckenstein M and Mustonen V (2017) Social rhythms of the heart. Health Sociology Review 26(1): 22–37. Pasquale F (2015) The Black Box Society: The Secret Algorithms That Control Money and Information. Cambridge, MA: Harvard University Press. Rabinow P, Marcus GE, Faubion JD, et al. (2008) Designs for an Anthropology of the Contemporary. Raleigh, NC: Duke University Press. Rogers R (2013) Digital Methods. Cambridge, MA, USA: MIT Press. Rogers R and Marres N (2000) Landscaping climate change: A mapping technique for understanding science and technology debates on the World Wide Web. Public Understanding of Science 9(2): 141–163.

21 Science & Technology Studies XX(X)

Ruppert E, Harvey P, Lury C, et al. (2015) Socialising big data: from concept to practice. CRESC Working Paper Series (138). Available at: http://research.gold.ac.uk/11614/ (accessed 24 November 2016). Schutt R and O’Neil C (2013) Doing Data Science: Straight Talk from the Frontline. Sebastopol, CA: O’Reilly Media. Seaver N (2015) Bastard algebra. In: Boellstorff T and Maurer B (eds) Data, Now Bigger and Better! Chicago, IL: Prickly Paradigm Press, pp. 27–46. Shapin S (1984) Pump and circumstance: Robert Boyle’s literary technology. Social Studies of Science 14(4): 481–520. Silverman D (1985) Qualitative Methodology and Sociology: Describing the Social World. Aldershot: Gower Pub Co. Slota SC, Hoffman AS, Ribes D, et al. (2020) Prospecting (in) the data sciences. Big Data & Society 7(1): 1-12. Smith DE (1978) `K is Mentally Ill’ the Anatomy of a Factual Account. Sociology 12(1): 23–53. Smith-Morris C (2016) When numbers and stories Collide: RCTs and the search for ethnographic fidelity in the veterans administration. Metrics: What counts in global health. Durham, NC: Duke University Press, pp. 181–202. Snow CP (1998) The Two Cultures. Cambridge: Cambridge University Press. Stengers I (2000) The Invention of Modern Science. Minneapolis, MN: University of Minnesota Press. Stengers I (2005) The cosmopolitical proposal. In: Latour B and Weibel P (eds) Making Things Public: Atmo- spheres of Democracy. Cambridge, MA: MIT Press, pp. 994–1003. Suchman L (1987) Plans and Situated Actions: The Problem of Human-Machine Communication. Cambridge: Cambridge University Press. Tashakkori A and Teddlie C (2010) Sage Handbook of Mixed Methods in Social & Behavioral Research. Abingdon: Sage. Uprichard E (2011) Dirty data: Longitudinal classification systems. The Sociological Review 59: 93–112. Uprichard E (2013) Focus: big data, little questions? Discover Society Available at: http://www.discoverso- ciety.org/2013/10/01/focus-big-data-little-questions/ (accessed 30 September 2014). van Eck N and Waltman L (2009) Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 84(2): 523–538. Vann K (2010) Irony and Humour, Toward a Deleuzian Science Studies. In: Jensen CB and Rödje K (eds) Deleuzian Intersections: Science, Technology, Anthropology. New York, NY: Berghahn Books, pp. 83-100. Venturini T and Latour B (2010) The Social Fabric: Digital Traces and Quali-Quantitative Methods. In: Proceed- ings of Future En Seine, Paris, France 2010. Paris, France: Editions Future en Seine, pp. 30–15. Venturini T, Laffite NB, Cointet J-P, et al. (2014) Three maps and three misunderstandings: A digital mapping of climate diplomacy. Big Data & Society 1(2): 1–19. Vertesi J, Ribes D, Forlano L, et al. (2016) Engaging, designing, and making digital systems. In: Felt U, Fouché R, Miller CA, et al. (eds) The Handbook of Science and Technology Studies. 4th ed. MIT Press, pp. 169–193. Vinkhuyzen E and Cefkin M (2016) Developing socially acceptable autonomous vehicles. In: Ethnographic Praxis in Industry Conference Proceedings, Hoboken, NJ, USA, 2016, pp. 522–534. Wiley Online Library. Wattenberg M and Viégas FB (2008) The word tree, an interactive visual concordance. IEEE transactions on visualization and computer graphics 14(6): 1221–1228. Wilkie A, Michael M and Plummer-Fernandez M (2015) Speculative method and Twitter: Bots, energy and three conceptual characters. The Sociological Review 63(1): 79–101.

22 Moats

Woolgar S (1983) Irony in the Social Study of Science In: Knorr-Cetina KD and Mulkay M (eds) Science Observed: Perspectives on the Social Study of Science. London: Sage, pp 239-266. Woolgar S (1990) Configuring the user: The case of usability trials. The Sociological Review 38(1_suppl): 58–99. Wouters P, Beaulieu A, Scharnhorst A, et al. (2013) Virtual Knowledge: Experimenting in the Humanities and the Social Sciences. Cambridge, MA: MIT Press. Wyatt S, Milojević S, Park HW, et al. (2017) The intellectual and practical contributions of scientometrics to STS. In: Felt U, Fouché R, Miller CA, et al. (eds) Handbook of Science and Technology Studies. Cambridge, MA: MIT Press, pp. 87–112. Zuiderent-Jerak T (2015) Situated Intervention: Sociological Experiments in Health Care. Cambridge, MA: MIT Press. Notes 1 I am reluctant to definitively name the two sides of this supposed conflict because it can take on many forms and both ‘sides’ of the conflict are not as monolithic as they seem. Throughout this paper I have attempted to stick to the specific discipline, methods or roles being negotiated in particular situations. 2 By corporate they have in mind Suchman’s celebrated work for Xerox which helped shape the discipline of HCI (Suchman, 1987; see also Bell, 2011), by critical they give the example of Star’s work bringing out the politics of digital infrastructures, by inventive they have in mind forms of ‘making and doing’ (Downey and Zuiderent-Jerak, 2016) and by inquiry they invoke more speculative, open-ended investi- gations (Wilkie et al., 2015). 3 As Denzin (2010) notes, mixed methods have, more often than not, involved quantitative researchers employing simplistic, impoverished versions qualitative methods (like interviews and participant observation) in the service of quantitative methods. 4 While there is not space to engage with nearly a half-century of debates about these frameworks, it is important to note than many of the most influential of them were developed in a time defined by different methods (sample surveys, interviews and ethnography) and different disciplinary tensions (“the paradigm wars”) and they might require more fundamental rethinking in an age of messy, found data culled from social media platforms, open government databases (Burrows and Savage, 2014) and automated types of analysis like Natural Language Processing, Machine Learning, network graphs and other types of visualisations. 5 There are plenty of approaches within anthropology which offer strategies for rethinking relations between the ethnographer and her informants. Marcus and other have developed the notion of “para- sites” which describe forms of epistemic collaboration with expert communities, however, crucial to this programme is one of deferral to the epistemic expertise of informants (Gilbert, 2015). Co-labora- tories (Rabinow et al., 2008) have been involved in questioning the role of extended fieldwork (with a single author) as the primary method of anthropology, but have rarely resulted in anything like the STS examples of anthropologists taking up quantitative tools for their own ends. 6 It is worth pointing out that in Kaltenbrunner’s (2014) study, the participants rethinking of their working relationship began when the author shared with them some early STS reflections, which seem to authorise these more radical moves. 7 Visualisations also however raise other sorts of concerns about literacy, and what they make invisible as well as visible (Coopmans, 2014; Kennedy et al., 2016). 8 Interestingly, the terms “qualitative” and “quantitative” were rarely invoked in any of the workshops, except in these moments of switching between approaches.

23 Science & Technology Studies XX(X)

9 One participant confessed that for several days after the workshop, she continued to think like a computer, constantly looking for things to count. 10 This approach of course ignores relationships between words or between sentences, and, notably, sarcasm. It would certainly need tweaking in relation to medicine where positive outcomes can be related to negative words like “disease”, “death”, “surgery”. “Murder” is taken as an extremely negative word even if the sentence was “I could murder a pizza”. 11 This is why, thinking of these graphs as a bad simplification of close reading is “too simple” to borrow Lynch’s (1988) phrasing. Lynch’s point is that we should not think of scientific representations which are presented as simplifications of other images or phenomena as reductions, but as active transforma- tions into something else. 12 This is not diplomatic in the sense used by Stengers in ‘The Cosmopolitical Proposal’ (2005), which suggests a participant who is themselves put at risk in the scenario. 13 More specifically, Vann suggests, the Sadistic form of Irony and the Masocistic form of humour are both humourous because there are responses to a situation (modernity) in which the law is not grounded.

24