Theory Choice in Physics

Faculty of Arts and Philosophy Centre for Logic and Philosophy of Science Director: Prof. Dr. E. Weber

Peter Rubbens

Promotor: Dr. R. De Langhe

Dissertation submitted to obtain the grade of Postgraduate in Logic, History and Philosophy of Science

Academic Year 2014–2015 Toelating tot bruikleen

“De auteur geeft de toelating deze scriptie voor consultatie beschikbaar te stellen en delen van de scriptie te kopi¨erenvoor persoonlijk gebruik. Elk ander gebruik valt onder de beperkingen van het auteursrecht, in het bijzonder met betrekking tot de verplichting de bron uitdrukkelijk te vermelden bij het aanhalen van resultaten uit deze scriptie.”

“The author gives the permission to use this thesis for consultation and to copy parts of it for personal use. Every other use is subject to the copyright laws, more speciﬁcally the source must be extensively speciﬁed when using from this thesis.”

Peter Rubbens, January 2015

i Contents

Toelating tot bruikleeni

Table of Contents ii

1 Introduction1

2 From Theory Choice to Theory Search: The Essential Tension between Exploration and Exploitation in Science5

3 Evaluation of the OPERA Collaboration and the Faster than Light Neu- trino Anomaly 16

4 Conclusion 32

Bibliography 36

ii Chapter 1

Introduction

This thesis contains the final product of two papers written during the “Postgraduate Stud- ies in the Logic, History and Philosophy of Science”. In this chapter I will first introduce these two papers separately, after which I will outline their relevance concerning their respective fields, to end with a note on what both papers can mean for each other.

The ﬁrst paper is titled “From Theory Choice to Theory Search: The Essential Tension between Exploration and Exploitation in Science” (FTCTS) and has been written in collaboration with dr. Rogier De Langhe [7]. It will appear in a special issue concerning Thomas Kuhn of the edited volume “Boston Studies in the Philosophy and History of Science”.

FTCTS is part of the research that tries to model scientific revolutions by means of agent- based-modeling. It uses the complex-systems-approach, in which it perceives a scientific community along with its dynamics as a complex system. As Newman notes, there is no technical and precise definition of a complex system, but the definition he uses is quite apt: “a system composed out of many interacting parts, such that the collective behaviour of those parts together is more than the sum of their individual behaviours” [16, p. 1]. The collective behaviour is dubbed emergent behaviour.

FTCTS proposes a model which shows that it is possible for scientists, who solely interact on a local level, to undergo a scientific revolution at the collective level, the entire scientific community. The mechanism behind this phenomenon is the balance scientists try to find between exploitation and exploration of a theory or paradigm. The few scientists who try to explore alternative theories instead of exploiting the existing ones are the ones who pave the way for other scientists to follow in a later stage. In this way theory choice becomes an activity of theory search; a minority of scientists explore theories until they find a theory

1 2

which fit their merits better. When this point is reached, the entire scientific community switches to this alternative theory. As this switch is quite sudden and consists out of quasi the entire scientific community, we are allowed to dub this non-cumulative break as a scientific revolution. The model is remarkably robust.

The first research considering the modelling of scientific revolutions originates in the work of John D. Sterman in 1985 [20]. In this paper Sterman attempts to test the dynamic consistency of Kuhn’s theory by formalizing it and subsequently simulating it on the computer. The main variable in the model of Sterman is a scientist’s confidence in a paradigm (CP ). CP = 1 represents total commitment, CP = 0 total rejection. CP is determined by the relative number of accumulated anomalies (RA) and the rate of progress of the paradigm (RSP ). Based on CP , scientists adhere or leave paradigms. With this model, Sterman is able to let a majority of a virtual scientific community adhere a certain paradigm for a significant amount of time, after which the number of adherents suddenly declines and another paradigm emerges. The lifetime of such a ‘dominant’ paradigm varies.

This research gained new insights after Wittenberg uttered some critiques to Sterman’s research in 1992 [25]. Wittenberg argues that Sterman exceeds “a modeler’s prerogative of reinterpretation”. Furthermore he is of the opinion that there are a number of method- ological problems when trying to model Kuhnian science. However, Sterman states in the same edition of “System Dynamics Review” that he disagrees with Wittenberg [21].1

Hereafter Sterman and Wittenberg joined forces, which results in a new version of the model in 1999 [22]. New theories are now stochastically and endogenously created and by means of positive feedback loops can unobservable microlevel perturbations become important; this makes the dynamics path-dependent and a self-organizing evolutionary system, typical features of complex systems.

Recently, there has been an attempt by Bornholdt et al. where the emergence and decline of paradigms is modelled by means of agent-based modelling [4]. Agents, in this case scientists, are able to adhere a certain paradigm, but a memory requirement makes sure a scientist cannot go back to a previous paradigm. By means of cooperative eﬀects, paradigms who have more adherents will attract more scientists. However, there is an

1Other responses in “System Dynamics Review” to the original model of Sterman are the one by Yaman Barlas who states that it is not fundamentally but practically impossible to model Kuhnian revolutions [3], and Michael J. Radzicki who suggests to introduce noise rather than aggregation in order to represent the idiosyncrasies of scientists [18]. 3 exogenous parameter α which introduces new paradigms to which scientists are also able to switch to. This model results in a regular pattern of global paradigm shifts.2

FTCTS diﬀers from previous researches in the following sense: all predescribed models use a global parameter to guide a scientiﬁc community to a new theory where FTCTS does not, FTCTS is based on pure local interactions. Moreover, it encompasses a majority of characteristics which previous models also displayed: FTCTS gives rise to non-cumulative breaks in a self-organising and endogenous way.

Chapter3 consists of a paper originally developed for the course “Scientiﬁc Explanation” and is titled “Evaluation of the OPERA Collaboration and the Faster than Light Neutrino Anomaly” (EVAL) [19]. It is displayed in the format as it has been submitted to the journal “Synthese”.3

In November 2011 the OPERA collaboration released results which seemed to originate from neutrinos which are able to travel faster than the speed of light c. It took almost a year, until July 2012, before it was clear the results were erroneous due to defects in the experimental set-up. This paper is an attempt to evaluate the scientiﬁc practice of the OPERA collaboration.

The ﬁrst part of EVAL consists out of a schematic overview of the communication by the OPERA collaboration combined with a sketch of the complexity of the experiment. It ends with a brief explanation of how the OPERA collaboration got to their erroneous measurements.

The second part of EVAL tries to evaluate the scientific practice of the OPERA collaboration. In order to do this properly, the toolbox developed by Weber, Van Bouwel and De Vreese is used [24, section 3.7]. This toolbox evaluates a scientific practice on five different levels: explanation-seeking questions and epistemic interests, the appropriate format of an explanation, explanation and levels of rationality, abstraction and amount of details in explanation and irrelevant premises.

Originally it might seem there is not much overlap between the two papers. However, the OPERA episode might be a suitable case-study of scientists choosing between the roles

2However, it can be shown that the memory requirement is unrealistic and on top of that not even necessary to achieve this behaviour [13]. 3Synthese rejected the paper on the basis that they ﬁnd the philosophical basis too thin. I will now try to submit it to “Studies in History and Philosophy of Science Part B: Studies in History and Philosophy of Modern Physics”. In the meanwhile a preprint is published on the “PhilSci-Archive”. 4

of on the one hand explorers and on the other exploiters. During the OPERA episode the majority of the physics community kept up a skeptical stance towards the OPERA results and kept exploiting the paradigm in which they were working in. Only a few scientists started exploring other possible paths, of which the most important one is that of superluminality. Superluminality considers phenomena which appear to travel faster than the speed of light. Giovanni Amelino-Camelia points out that their already existed earlier work on superluminality [1]. This work emerged from research performed on quantum- gravity. Some studies argue that there will be a departure from Einstein his theory of relativity ath the Planck-length. Superluminality is a possibility in these cases.

Because of the initial results of OPERA, research was stimulated to focus on superluminal neutrinos. Considering superluminal neutrinos are possible, further research had to be undertaken to on the one hand conﬁrm superluminal neutrinos are actually possible and on the other ﬁnd the limits within which superluminal neutrinos are possible to measure.

For example, Gian Giudice, Sergey Sibiryakov, and Alessandro Strumia proved that superluminal neutrinos would result in anomalies for the velocities of electrons and muons [8]. By means of cosmic ray data such anomalies could already be ruled out, which contradicts the OPERA results. Another example stems from the work of Andrew Cohen and Sheldon Glashow. They predicted that superluminal neutrinos would lose energy by producing electrons and positrons by processes which are analogue to Cherenkov radiation [5]. However, this radiation was not found during the OPERA experiment.4

In other words, exploring research implied more and more that the OPERA results had to be erroneous. More generally, as the OPERA episode clearly illustrates the tension between exploring new ﬁelds of science and the majority of scientists who choose to exploit the existing ones, this is a ﬁrst illustration of how the OPERA episode can be of importance for the complex-systems-approach in the philosophy of science. I will return to this matter more elaborately in chapter4.

4The ICARUS experiment, also located at Gran Sasso, made use of the same CNGS beam during 2010 and 2011. They also did not register any form of radiation which could be caused by superluminality [2]. Chapter 2

From Theory Choice to Theory Search: The Essential Tension between Exploration and Exploitation in Science

5 From Theory Choice to Theory Search: The Essential Tension between Exploration and Exploitation in Science

December 16, 2013

Rogier De Langhe and Peter Rubbens

Introduction

Theory choice is one of the most important problems in philosophy of science. Some argue that choice for one theory over another is rational if the procedure that led up to that choice was rational (e.g. Karl Popper (1963) proposed a methodology based on falsifiability), while others have argued that choice for a theory is rational because of certain properties of that theory itself, called “scientific virtues” (e.g. Henri Poincare (1905)’s defence of the virtue of simplicity). However a standard assumption is that the theories from which to choose already exist. A typical assumption in 20th century philosophy of science has been to restrict itself to the “context of justification”, treating theories as given.(Reichenbach 1938) This assumption reduces the problem of theory choice to a problem of choice under risk. Although at the moment of choice it is unknown which theory will eventually be right, a comparison of theories’ past performance allows to select the one which is most probable to be successful in the future. However, if no scientist ever chooses to search for a new theory, scientific progress would soon come to a halt. The starting point of this paper is therefore that at least some scientists must search for new theories rather than keep on developing the existing ones. By shifting the focus of scientific rationality from choice among given alternatives to finding a balance between the exploitation of existing theories and the exploration of new theories, the activity that scientists engage in is no longer one of passive choice but of active search. By framing the problem of theory choice as search, we argue that a more realistic thematization ensues of the problem that practicing scientists are confronted with: the question of whether to expand on existing theories or start working on an entirely new theory.

This expansion of the problem of theory choice entails that the set of alternatives from which a choice must be made is no longer given but infinite. Despite the fact that the number of questions that can be asked about the world is infinite, we observe that scientists nevertheless work together without any centralized control and with only local information available. It is then all but a miracle that these independent scientists succeed in collective construction of theories. Conversely, once a collectively shared set of questions and methods has been accepted, it is difficult to see how individual agents could unilaterally succeed in changing

1 it. To be successful, a community of scientists must therefore ﬁnd the right balance between the exploitation of existing theories and the creation of new theories. The problem of theory choice then becomes one of theory search, and how scientists can rationally decide between both.

Expanding the problem of theory choice to theory search turns the problem of theory choice under risk into a problem of choice under uncertainty. Because the set of alternatives is infinite, the criteria a good theory must meet cannot be specified independently in advance. And because the properties of the new theories are unknown when the choice for their development is made, no algorithm can be specified for their selection. Uncertainty therefore entails ana- lytical intractability. Nevertheless we know from many domains in life that successful action is still possible under uncertainty on the basis of heuristics. A heuristic only tells agents how to look, not what to find and thereby guides the decision process without determining it. It is less specified than an algorithm, but it is this lack of specificity which makes it robust against choice for unknown alternatives. As such, heuristics are not inferior to algorithms, but a different solution to a different problem.

Possibly it is this lack of algorithmic treatment which explains why philosophers have been so keen on maintaining the unrealistic restriction to the context of justification. Thomas Kuhn (1970) is a famous exception. In response to the ensuing uncertainty, Kuhn suggested that theory choice is based on heuristics rather than on an algorithm. He described five common scientific virtues (accuracy, consistency, scope, simplicity and fruitfulness) as “criteria that influence decisions without specifying what those decisions must be.”(Kuhn 1977, 330) How- ever Kuhn was unable to specify how possibly such values could lead decentralized scientists to produce collectively successful science as we know it.1

Here we define a successful scientific community as a community which finds a rational balance between exploration and exploitation. Using the novel tool of agent based-modeling, we show that a succesful scientific community can emerge from agents choosing based on a simple heuristic applied to local information.

1 Heuristic: exploitation and exploration

Consider a community of N (1,...,n) scientists. Each turn, each scientist makes a contribution C (c1,...,cN ) to a theory S(s1,...,sM ); mark that N is a constant of the system, however, M may vary as the system evolves. A heuristic for individual theory search must rely only on information locally available to the agent and must be suﬃciently general to be applicable across large ranges of possible choice situations. The balance between exploration and exploitation is a general consideration applicable to any conceivable alternative.

Exploitation consists in an allocation of scientiﬁc labor to an existing theory. The more scientists exploit the same theory, the higher the beneﬁt of specialization becomes because

1“Even those who have followed me this far will want to know how a value-based enterprise of the sort I have described can develop as a science does, repeatedly producing powerful new techniques for prediction and control. To that question, unfortunately, I have no answer at all [....] The lacuna is one I feel acutely”(Kuhn 1977, 332-3)

2 Figure 1: Moore neighbourhood, H = 8. Figure used fromPolhill et al. (2010). scientists can specialize in narrower subproblems and specialized tools can be developed.2 As a consequence, a local proxy for the beneﬁts of exploitation is the number of adopters of a theory. More precisely, the “adoption” A of a theory s is the sum of the number of scientists that contribute to it, where H denotes the size of the local neighbourhood of a scientist.

H As(t)= ai,s(t). (1) Xi=0

In ﬁgure 1 an example of a local neighbourhood is given, a so-called Moore neighbourhood, for which H = 8.

Exploration consists in an allocation of scientiﬁc labor to a new theory. The less articulated a theory, the higher the innovative value of contributing to that theory. We will assume that each scientist makes one contribution at each turn. As a consequence, a local proxy for the beneﬁts of exploration is the inverse of the number of contributions made to a theory. More precisely, the “production” P as the sum of contributions to a theory s is the sum of adopters through time t: t Ps(t)= As(t′)dt′. (2) Z0

The relation between adoption and production as specified here dynamically captures the trade-off between exploitation and exploration. More exploitation means less exploration, and similarly the number of adopters to a theory increases the specialization benefits from exploiting it but decreases the novelty of exploring it.

Since exploration and exploitation jointly determine the utility of a contribution to a theory, this utility can then be expressed as follows:

α As (t) Us(t)= . (3) Ps(t)

2The insight that division of labor increases productivity by fostering specialization is as old as Adam Smith (2003) and marked the birth of modern economics.

3 The parameter α denotes the output elasticity of coordination, which is a function of the (for the purpose of this paper exogenous) state of technology. Us is backward-looking because it evaluates the utility of the last contribution to a theory. But if scientists would always choose to develop that theory which is best developed, then soon enough no alternative theories would ever be developed because any new theory would always be less developed than the existing ones. Scientists can only be expected to develop new theories if their focus is not backward-looking, but forward-looking. Therefore the utility of the next contribution should be considered:

α (As(t) + 1) Us′(t)= . (4) Ps(t) + 1

2 Dynamics: a battle of perspectives

In the face of unknown alternatives, scientists must take into account the actions of others. For an adopter of a theory, the trade-off between exploration and exploitation entails a tension in the value of a new adopter to that theory: in the short term it increases the number of agents with whom labor can be divided and therefore the exploitation part of the value of that theory, but in the long term it leads to a higher production within that theory and thereby decreases the exploratory value of contributions to that theory. However for each individual agent acting locally, the benefits of a new adopter outweighs their cost, and therefore agents try to persuade each other to adopter their theories.3 As such we specify the dynamics of the model as driven by the desire of agents to persuade each other: every turn N randomly assigned agents are selected to be convinced by one of their neighbors. The probability that this convincing attempt is successful (“conversion”) is proportional to the utility of the respective theories from the perspective the adopter of that theory.4 As such there is what we call a “battle of perspectives” by which theories are confronted based on the value the respective defender of that theory attributes to it. The battle of perspectives has three possible outcomes: stick to ones theory i (Ps), be convinced by the convincing neighbor’s theory j (Pc) and contribute to an entirely new theory k (Pn). Our definition of the utility of a contribution to a theory (see eq.4) allows us to quantify the probability of each of these outcomes:5

Usi Ps = , (5) Usi + Usj + 1

Usj Pc = , (6) Usi + Usj + 1

1 Pn = . (7) Usi + Usj + 1 3It is only in the long term that these individual decisions will collectively exhaust that theory and lead to its collapse. 4Each agent only knows the utility of his own theory based on his local information from the neighborhood and when a convincing attempt is made the probability of success is proportional to the utility of their respective theories. 5Note that the utility of contributing to an entirely new theory is always exactly 1 because an undeveloped theory has no adopters and no production so both A and P are 0 and eq.4 always equals 1 irrespective of α.

4 Time-Series 12000 Number of scientists in dominant theory Number of explorers 10000 8000 6000 4000 2000 0 0.0035 Incentive new theory 0.003 0.0025 0.002 0.0015 0.001 0.0005 0 0 2000 4000 6000 8000 10000 t (simulatiestappen)

Figure 2: Time-series in which the largest number of scientists who work within the same theoryand the incentive of a new paradigm has been visualized for L = 100 and α = 7.

3 Simulation results

A heuristic for individual theory search was developed and the probabilities for the resulting dynamics were quantiﬁed. The only information lacking to understand the consequences of this model is that on adoption. Adoption in the model will vary as a consequence of the probabilities of the endogenously created theories, and those probabilities are in turn determined by adoption. As a consequence the system is permanantly out of equilibrium and its dynamics cannot be speciﬁed by detmerining equilibria but only by studying the resulting process. The study of aggregate patterns emerging from the interactions of individual agents is possible using the technique of agent-based modeling. Assume periodic boundary conditions and let agents interact on a two-dimensional grid with size L; this means we have N = L2 agents. Each agent sees only his Moore-neighborhood and tracks each turn the number of adopters to each theory and their total production. Initially all agents adopt the same theory.

Fig.2 shows the evolution of adoption to the dominant theory in a typical run of the model. A few observations can already be made:

Although there is only one initial theory, novel theories are created endogenously. • Although the number of possible alternative theories is all but inﬁnite, cooperation on a • single theory emerges and novel theories are only created as existing ones are exhausted. The model thus self-organizes to ﬁnd a dynamic balance between locking in to a single theory and a situation where there are as many theories as there are scientists.

All action in the model is taken by individuals. The fact that theory change occurs • proves that individuals have the capacity to unilaterally initiate theory change.

Disruption of consensus (“crisis”) is of varying size and length, although typically shorter • than the length of consensus.

5 Average Number of Scientists in Dominant Cluster 12000 L = 50 L = 75 L = 100

10000

8000

6000

4000 Average Number of Scientists 2000

0 0 2 4 6 8 10 α

Figure 3: The average size of the largest cluster of scientists who work within the theory that counts the most scientists for various α and diﬀerent dimensions L.

6 The system exhibits a cross-over from essentially competitive to cooperative as α increases. In ﬁg.3 we show the average size of the dominant theory for various α and diﬀerent dimensions of L. Variation of all sizes can be observed around α = 6.5, perhaps suggesting critical point. Communities with α below this point are characterized by continuous presence of multiple competing theories, while communities in which α is higher are characterized by the alternating monopoly of a single theory separated by shorter periods of crisis.

The evolution of the model is driven by the ever-changing incentive structure for the three alternatives available to each agent in the model. In particular, the probability of contributing to a new theory is inversely proportional to the sum of the utility of contributing to the existing theories. As a consequence the incentive for agents to create new theories is not given but varies endogenously with the dynamics of the model. The bottom of ﬁg. 2 shows how the incentive structure of the model, represented by the utility of contributing to a new theory, coevolves with the very contributions it regulates: the longer the period of consensus, the higher the probability that agents create a new theory; conversely the probability of creating a new theory is lowest when consensus on a new theory has just emerged. The ﬁgure clearly shows two separate phases: a normal phase in which the incentive structure is stable and predictable, and a revolutionary phase in which the incentive structure is unstable and unpredictable.

It is clear that these two phases in the incentive structure result in very different ratios of explorers and exploiters. These respective phases are characterized by statistically different properties, allowing for a quantitative separation of these phases based on the distribution of their production. During a normal phase characterized by consensus on a single theory, production follows a markedly different distribution than during a revolutionary phase characterized by the absence of consensus.

Let an explorer be a contributor to a new theory and an exploiter a contributor to an existing theory. We will show that we are able to make a clear distinction between the two sorts of phases. We will do this by means of the distribution of the total production, defined as the total amount of contributions made to a certain theory s, visualized in fig. 4. In this figure the distribution of the total number of contributions per theory S is visualized. This is done for three different values of α, one in which competitivity mainly determines the behavior of scientists (α = 5), one in which coordination mainly determines the behavior of scientists (α = 8) and one in which both determine the behavior of scientists (α = 6.5). The top row shows the complete distribution of total production, the second row the production produced by explorers and the third row the production produced by the exploiters.

For α = 5 we see that the contributions of exploiters have a marginal influence on the total distribution. However, they play a part in the dynamics, and it would be wrong to claim that there are no exploiters when α is low. When α = 6.5, contributions of both explorers and exploiters matter to the total distribution. When looking at the productions of only explorers, it appears that they would sometimes also act in a coherent way. However, the bump can be explained by fig. 3. For α = 6.5, we see that we can have clusters of all sizes. This means that multiple large clusters are able to coexist, nevertheless, with our current definition of explorers, we do not take into account larger clusters who are not the largest cluster. These large clusters however give rise to the previously mentioned bump. Whether these scientists

7 α =5 α =6.5 α =8 1.0 106 105 104 3 100.8 102 101 100 6 100.6 ) 105 104 3

C/S 10

( 102

P 0.4 101 100 106 105 100.24 103 102 101 100.00 0.0102 103 104 10 0.25 106 107 102 10 0.43 104 105 106 0.6107 102 103 10 0.84 105 106 107 1.0 C/S

Figure 4: Distribution of total production for α = 5, 6.5, 8; L = 100. The ﬁrst row visualizes all contributions, the second those of explorers and the third those of exploiters.

8 can be percieved as explorers or not is a discussion less important for the purposes of this paper.

When coordination is strong, which is the case for α = 8, the distribution splits itself in two, in which the explorers are represented by the left distribution and the exploiters by the right. It is clear from this that explorers follow a different distribution than exploiters, which is globally visible only when the coordination is strong. From we this we can conclude that scientists are able to behave in two significantly different ways; either a scientist contributes to a well-established theory in which the gain is obvious because a lot of his peers are contributing to the same theory or a scientist contributes to a non-established en less-known theory, in which he explores the possibilities of that theory.

It is interesting to note that α plays an important role in visualizing both distributions. By gradually increasing α, the distribution of exploiters becomes more and more clear, and ultimately both distributions are visible and completely disconnected from each other.

Although scientists have the possibility to contribute to different theories, we can distinguish periods in which the whole scientific community unite themselves contributing to one theory. This happens in a self-organizing way, in which the model finds a dynamic balance between locking in to a single theory and a situation where there are as many theories as there are scientists. On top of that, new theories are created endogenously and only when the existing ones seem to get exhausted.

4 Conclusion

In conclusion, in the absence of centralized control, with only limited information and using nothing but a simple heuristic, the interactions of scientists result in a robust pattern of intermittent theory exploitation and exploration with shifts between them occurring at the rational point in time. Individual theory choice in this model is not a choice between existing alternatives, but a process of search which finds a dynamic balance between the actual and the possible; between given alternatives and creating new alternatives. As such the theories created define the possibility space for subsequent theory choice, all in the same model. It was shown in this paper that this heuristic results in a self-organizing balance between tradition and innovation, where theories are created as they are needed. From the interactions between individuals using only a local rule of thumb that strikes a balance between exploration and exploitation (both the benefits of specialization and the race for priority are taken into account) can emerge a community that finds a dynamic balance between the exploitation of existing theories and the exploration of new theories.

References

Thomas Kuhn. The Structure of Scientiﬁc Revolutions, 2nd Ed. Chicago University Press, Chicago, 1970.

Thomas Kuhn. The Essential Tension. Chicago University Press, Chicago, 1977.

Henri Poincare. Science and Hypothesis. Walter Scott Publishing, 1905.

9 J. Gary Polhill, Lee-Ann Sutherland, and Nicholas M. Gotts. Using qualitative evidence to enhance an agent-based modelling system for studying land use change. Journal of Artiﬁcial Societies and Social Simulation, 13(2):10, 2010. ISSN 1460-7425. URL http: //jasss.soc.surrey.ac.uk/13/2/10.html.

Karl Popper. Conjectures and Refutations. Routledge, 1963.

Hans Reichenbach. Experience and Prediction. University of Chicago Press, 1938.

Adam Smith. Wealth of Nations. Bantam Classics, New York, 2003. [1776].

10 Chapter 3

Evaluation of the OPERA Collaboration and the Faster than Light Neutrino Anomaly

16 Noname manuscript No. (will be inserted by the editor)

Evaluation of the OPERA Collaboration and the Faster than Light Neutrino Anomaly

Received: date / Accepted: date

Abstract On 22 September 2011 the OPERA collaboration published a paper which communicated their results concerning the measurement of the neutrino velocity, which appeared to have exceeded the speed of light. If confirmed, this would imply a huge anomaly for the theory of relativity and physics in general. It took until July 2012 for the OPERA collaboration to figure out this was due to an internal error in the experimental set-up. It made spokesperson Antonio Ereditato and physics coordinator Dario Autiero eventually resign. In the meanwhile there was a lot of attention from both scientists as the media, however the OPERA collaboration is yet to be properly evaluated. This paper aims at evaluating the scientific practice of the OPERA collaboration by considering the following two questions. How did the OPERA collaboration address this apparent anomaly and have the OPERA scientists performed as professional scientists should or not? Keywords Scientific explanation OPERA collaboration Evaluation of scientific practice Scientific communication· · ·

1 Introduction

On 22 September 2011 the OPERA collaboration published a paper on the arXiv where they state that they have measured a neutrino (ν) velocity which appears to exceed the speed of light c [Adam et al(2011a)].1 Up until that day we had not been able to measure a phenomenon in physics with which it was possible to exceed c. In other words, this would imply the ﬁrst serious anomaly for the theory of relativity.

1 The arXiv is a collection of electronic preprints of scientific papers which are free accessi- ble. The papers stem from various fields, physics, mathematics, computer science, statistics, quantitative biology and quantitative finance. Those who are interested can take a look at http://arxiv.org/. 2

This result was received with a lot of attention from the media [Brown, K. and Khan, A.(2011), Brumfield, G.(2011),Hooker, B.(2011),Matson, J.(2011)]. The physics community was also aware of the result, but remained skeptical. One of the reasons for this is that previous experiments in which the neutrino velocity was determined had always been in accordance with the theory of relativity. It took until July 2012 for the OPERA collaboration to conclude that their measurements resulted from an error in the experimental set-up. Spokesperson Antonio Ereditato and physics coordinator Dario Autiero resigned after the publication of their error, which seems to imply a certain form of unprofessionalism. This paper consists out of two goals. The first one is to determine how the OPERA collaboration addressed the apparent anomaly. This will be illustrated by giving a brief sketch of the complexity of the experiment in section 3 after which I will analyze their working papers and communications to the press in section 4. Second I will try to answer the question whether the OPERA collaboration can be called unprofessional or not. This will be done by evaluating their scientific practice in a systematic way, by using the toolbox developed by Weber, Van Bouwel and De Vreese in order to evaluate a scientific practice properly. This will be discussed in section 5. However, in order to be able to get a full grip on the matter I will start with some necessary background information.

2 Background Information

I will first give a short introduction in the theories of physics which are needed to understand the experiment. I will start with the theory of relativity. One of the postulates of Einstein’s special theory of relativity holds that there is a finite speed at which information can be transferred, which in fact is the speed of light c; this quantity has in vacuum the approximate value of 3 108 m/s. Particles that are massless will move with the speed of light. Particles× that are not will move at a speed v < c. In other words, it is not possible to exceed the speed of light. In physics we distinguish four different fundamental forces: the gravita- tional force, the electromagnetic force, the strong interaction and the weak interaction. The neutrino can be seen as a signature of the weak interaction. It belongs to the family of leptons, of which neutrinos are the uncharged ones. The charged leptons are the electrons (e), muons (µ) and tauons (τ), which all carry a negative charge. Accordingly, neutrinos can have three different flavors: the electron-neutrino (νe), the muon-neutrino (νµ) and the tau-neutrino (ντ ). Neutrinos can perhaps be seen as one of the fundamental particles in physics of which we still know the least. Within the Standard Model (SM), the established model in particle physics which describes subatomic particles and their interactions, neutrinos are massless. This would imply according to the theory of relativity that they move at the speed of light. Title Suppressed Due to Excessive Length 3

However, in more recent experiments physicists have registered certain phenomena which would imply that neutrinos do have a mass, albeit still a very small mass. One of these phenomena are the so-called neutrino-oscillations: neutrinos are able to oscillate between different flavors or in other words, one is able to measure neutrinos which appear to have a different flavor later on. This is however only possible when neutrinos do have a mass. This is where the OPERA experiment becomes important. The original goal of the OPERA experiment was to perform a direct measurement of the νµ ντ oscillation. The measurement of the neutrino velocity was initially an additional→ goal, meant as a confirmation and fine-tuning of previous results. These results carry the following information. In 1979 Kalbfleish et al. were able to measure the maximum deviation of the velocity of movement of a 5 neutrino vν compared to c:(v c)/c < 4 10− [Kalbleish et al(1979)]. The neutrinos coming from the SN1987A− supernova× yielded a maximum deviation 9 of v c /c < 2 10− [Longo(1987)]. The MINOS collaboration reported in 2007| − a measurement| × of (v c)/c = (5.1 2.9) 105 [Adamson et al(2007)]; in other words, all these results− are in agreement± × with the theory of relativity. As stated, the OPERA collaboration reported a different result. They found a six times statistical deviation of the neutrino velocity compared to c, namely that (v c)/c = (2.48 0.28 (stat.) 0.30 (sys.)) 105 [Adam et al(2011a)]. This would− imply a serious± anomaly± for the theory× of relativity.

3 Complexity of Experiment

One should not underestimate the complexity of contemporary scientific experiments. It is an idealized image that contemporary scientists are both theorists as well as experimentalists. This romantic image of science no longer holds. Set- ting up and designing experiments and the technology that goes along with it calls for a whole different kind of expertise than research in theoretical sciences does. Communication between the two fields can therefore be cumbersome. However, this section is not intended as a discussion of this topic. I do would like to give an overview of the complexity of the experimental set-up of the OPERA experiment in order to illustrate that various errors can be concealed at different levels of the experiment and that it therefore can take up a lot of time to expose them. I will highlight various topics in the same sequence as the OPERA collaboration builds up its papers discussing the neutrino velocity measurement, see [Adam et al(2011a),Adam et al(2011b),Adam et al(2012a)] and [Adam et al(2012b)].2

2 The ﬁnal design of the theoretical experimental set-up from which it was built can be found in [Acquafredda et al(2009)]. 4

3.1 Experimental Set-Up

The OPERA detector and the CNGS neutrino beam. The neutrinos are produced at the CERN Super Proton Synchrotron (SPS) at Genève. In the synchrotron protons are accelerated up to high speeds and lead to carbon targets, after which they will decay into kaons and pions.3 The positively charged pions and kaons are energy-selected and lead to the laboratory at Gran Sasso. These particles will decay in muons and muon-neutrinos in a 1000 m long vacuum pipe after which the muons will be filtered out. This results in the CERN Neutrino beam to Gran Sasso (CNGS neutrino beam).4 The majority of the beam now consists out of muon-neutrinos.5 These neutrino’s are detected by the OPERA detector which is able not only to locate neutrino interactions in its target, but also to measure the arrival of time of neutrinos. Principle of the neutrino time of flight measurement. In order to measure the neutrino velocity, one needs te measure the time of flight of neutrinos (TOFν ) and compare it to the time of flight assuming the speed of light (TOF ), which results in the deviation δt = TOF TOF . One is not able c c − ν to measure TOFν at the single interaction level because it is not clear which proton will result in the production of a neutrino. However by measuring time distributions of protons for each sample of which neutrino interactions are observed in the detector at CERN one can obtain the probability density function (PDF) of the time of emission of the neutrinos at CERN. These distributions can then be compared to the time distributions at OPERA. The timing measurements were performed by GPS receivers and Cesium (Cs) atomic clocks at both ends of the CNGS beam. These are needed in order te be able to have an accurate relative time tagging. The OPERA collaboration calculated side-effects in order to check whether their were deviations and if so, if they needed to compensate for them. For example the reference point for the baseline measurement at CERN is 743.4 m upstream of the target. This results in a negligible correction of 0.007 ns [Adam et al(2012b), p. 6]. Another example is a difference between the time base of the CERN and OPERA GPS receivers, which was measured to be (2.3 0.9.) ns.6 ± Measurement of the neutrino baseline. Another important feature in the determination of δt is accurate knowledge of the neutrino baseline between the CERN and Gran Sasso facilities. As we need to use relativity theory, these coordinates need to be known within the same global geodesy reference frame. Special attention has been made to relative distances in the experimental set-ups at both CERN and OPERA. The determination of the total

3 Kaons and pions belong to the meson family, which are constituted by two quarks. 4 A more thorough elaboration concerning the construction of the CNGS beam can be found in [Acquafredda et al(2009), section 2]. 5 There is a 2.1% contamination with muon-antineutrinos (ν ¯µ’s), and contaminations of less than 1% with electron-neutrinos and electron-antineutrinos; see [Adam et al(2012b), p. 3 - 4]. 6 See [Feldmann et al(2011)] for a full report on this compensation. Title Suppressed Due to Excessive Length 5 distance results in a distance of 730534.61 0.20 m.7 “The 20 cm uncertainty is dominated by the 8.3 km underground link± between the outdoor GPS bench- marks and the benchmark at the OPERA detector”[Adam et al(2012b), p. 9]. There has also been a correction for tidal eﬀects. The GPS receivers are able to continuously monitor tiny movements of the Earth’s crust. Neutrino event timing. One cannot simply start a stopwatch at CERN and stop it when a neutrino arrives at OPERA. Several instruments are needed to capture various travel times and interactions between particles. Furthermore, these instruments need to communicate with each other. For example, the CERN timing chain is characterized by three delays, the OPERA timing chain is even more complex. One also needs to take into account the rotation of the Earth around its axis, the so-called Sagnac eﬀect. Therefore the timing of the several neutrino events is far from an easy mission.

3.2 Data

The following two topics consider the experimental set-up at a different level. No longer will I speak of the experimental set-up, measurements, communication between devices and side-effects scientists have to deal with, but now I will consider the data scientists have gathered after performing their experiment. Two steps have to be considered in dealing with data. Firstly, data needs to be filtered in a proper way, and secondly it needs to be analyzed. Data selection. Some data is contaminated with effects OPERA scientists would like to exclude from their experimental set-up. These numbers have to be left out ot the analysis. Other data has to be sorted in different categories. A classification algorithm has been developed in order to do so.8 In total 15223 neutrino interactions have been analyzed. They do not include 5% of the preselected events, these are classified as ‘noise’. Data analysis. Data has to be analyzed in order to be able to make justified conclusions. Firstly this means the right corrections have to be actually carried out; a number of these corrections have already been mentioned in the previous section. Secondly, one has to apply the right statistics for the various components of the experiment. For example one has to perform a maximum likelihood procedure for the proton extractions at CERN. Another example is the comparison of the proton PDF and the neutrino time distributions to show that they are statistically equivalent. Thirdly, one needs to consider systematic effects. An example in the OPERA set-up is the check whether there are density variations at the target of the detectors the particles collided with and whether this yields an effect. The effect is negligible [Adam et al(2012b), p. 24].

7 See [Colosimo et al(2011)] for a full report. 8 See [Bertolin and Tran(2009)] for a full report. 6

4 Communication by the OPERA collaboration

This section will consider the way in which the results were communicated. In the first subsection I will consider the four versions of the paper “Measurement of the neutrino velocity with the OPERA detector in the CNGS beam” which they have published on the arXiv and of which the final version is published in JHEP (Journal of High Energy Physics); I will do this in chronological order. All versions largely have the same outline as presented in section 3. In the second subsection I will briefly highlight some comments made in press releases considering this experiment. This will be used in order to be able to properly evaluate the scientific practice of the OPERA collaboration, which will be done in section 5.

4.1 OPERA Timeline

4.1.1 version 1

The first communication about their results dates from 22 September 2011 [Adam et al(2011a)]. After summing results from previous experiments considering measurements of the neutrino velocity, it deals with the entire experimental set-up and data selection and analysis of the OPERA experiment. In the section ‘final results’ they communicate their experimental results. They state that they were able to measure (v c)/c one order of magnitude − better than previous measurements. They find that δt = TOFν TOFc = (60.7 6.9 (stat.) 7.4 (sys.)) ns and that (v c)/c = (2.48 0.28− (stat.) 0.30 (sys.))± 105 [Adam± et al(2011a), p. 22]. In− other words, these± results yield± a significant× statistical deviation from the upper limit c. This would imply a serious anomaly for the theory of relativity. However, the OPERA collaboration does not speak of an anomaly; they call it an “early arrival time of CNGS muon neutrinos with respect to the one computed assuming the speed of light in vacuum”[Adam et al(2011a), p. 19]. Already this early on the OPERA scientists performed an additional test in order to check the energy dependence of their results. They split their data in two bins of nearly equal statistics for all their internal events. They find for the +7.3 low- and high-energy bins respectively that δt = (54.7 18.4 (stat.) 6.9 (sys.)) +7.3 ± − ns and δt = (68.1 19.1 (stat.) 6.9 (sys.)) ns, from which they conclude that they are not able± to find clues towards− a possible energy dependence. The OPERA collaboration conclude their work with the following sentence: “In conclusion, despite the large significance of the measurement reported here and the robustness of the analysis, the potentially great impact of the result motivates the continuation of our studies in order to investigate possible still uknown systematic effects that could explain the observed anomaly. We deliberately do not attempt any theoretical or pheomenological interpretation of the results”[Adam et al(2011a), p. 22]. Title Suppressed Due to Excessive Length 7

4.1.2 version 2

The second publication appears 17 November 2011 [Adam et al(2011b)]. In the meanwhile the OPERA collaboration was able to improve the CNGS timing system and the OPERA detector. However, the new found result is still not +8.3 in agreement with the theory of relativity: δt = (57.8 7.8 (stat.) 5.9(sys.)) ns. In order to be more accurate, the OPERA collaboration± performed− an alternative analysis in which they now calculated a maximum likelihood function built by associating each neutrino interaction to its waveform instead of the global PDF. Spitefully, this has lead to a compatible value of δt = +9.6 (54.5 5.0 (stat.) 7.2 (sys.)) ns [Adam et al(2011b), p. 25]. ± − They performed an additional test in order to exclude possible systematic and statistical biases because of the use of proton waveforms as PDF for the distribution of neutrino arrival times within the two extractions. Again they ﬁnd a compatible result of δt = (62.1 3.7) ns. This makes them end with exactly the same sentence as written in± the previous paragraph.

4.1.3 version 3/4

The third and fourth publication only differ one day in publication date, as they don’t differ much from each other, they will be discussed together (see [Adam et al(2012a)] for the third version and [Adam et al(2012b)] for the fourth). Version four appeared on the arXiv on 12 July 2012. The OPERA sci- +8.3 entists state that they have found the following result: δt = (6.5 7.4 (stat.) 8.0 (sys.)) +3.4 6 ± − ns, which gives (v c)/c = 2.7 3.1(stat.) 3.3(sys.) 10− [Adam et al(2012b), p. 30]. − ± − × An alternative analysis in which the likelihood function is built by associating each neutrino interaction to its waveform instead of using the global +9.4 PDF gives a value of δt = (3.5 5.6 (stat.) 9.1 (sys.)) ns. The search for an energy dependence yields a null± effect. An additional− test to exclude possible systematic and statistical biases because of the use of the proton waveforms as PDF for the distributions of the neutrino arrival times within the two extractions leads also to compatible results: δt = 1.9 3.7 ns (TT-distribution) and δt = 0.8 3.5 ns (RPC).9 − ± − ± These results leads the OPERA collaboration to conclude the following: “After several months of additional studies, with the new results reported in this paper, the OPERA Collaboration has completed the scrutiny of the originally reported neutrino velocity anomaly by identifying its instrumental sources and coming to a coherent interpretation scheme”[Adam et al(2012b), p. 30].

9 The additional tests made use of the data registered by the Target Tracker (TT) on the one hand and the Resistive Plate Chambers (RPC) on the other, which results in two values. See for more information [Adam et al(2012b), section 9]. 8

4.1.4 Final Paper and Further Results

In 2012 they published this version in the JHEP [Adam et al(2012)]. In 2013 there appeared a follow-up in which they were able to measure the neutrino velocity with even higher accuracy [Adam et al(2013)]. In this way we can conclude the OPERA collaboration eventually accomplished their two premised goals of establishing the νµ ντ channel and measuring the neutrino velocity with high accuracy. →

4.2 Communication to the Press

Another way in which the OPERA collaboration communicated their results to the media was by means of press releases alongside the papers discussed in subsection 4.1.10 I will briefly highlight some interesting quotations from the original press release which will be used in section 5 to evaluate the scientific practice of the OPERA collaboration. According to the original press release, the OPERA collaboration communicated their results in order to invoke a broader scrutiny of their results. “When an experiment finds an apparently unbelievable result and can find no artefact of the measurement to account for it, it’s normal procedure to invite broader scrutiny, and this is exactly what the OPERA collaboration is doing, it’s good scientific practice,” as CERN Research Director Sergio Bertolucci stated. Spokesperson Ereditato makes a similar comment: “After many months of studies and cross checks we have not found any instrumental effect that could explain the result of the measurement. While OPERA researchers will continue their studies, we are also looking forward to independent measurements to fully assess the nature of this observation.” However, both also make statements in which they speak of a possible impact on physics. Bertolucci states the following: “If this measurement is confirmed, it might change our view of physics, but we need to be sure that there are no other, more mundane, explanations. That will require independent measurements.” Spokesperson Ereditato makes a similar statement: “The po- tential impact on science is too large to draw immediate conclusions or attempt physics interpretations. My first reaction is that the neutrino is still surprising us with its mysteries.” Already in February 2012 one can read that the OPERA has distinguished two effects who might have had their impact on their results, however one would lead to an overestimate of the result, the other one the an underestimate. The press release states the OPERA collaboration has informed its funding agencies and host laboratories of these effects. In June 2012 it is communicated that the OPERA measurements are in agreement with those of other experiments which have also measured the neu-

10 A summary of these releases by CERN can be found here: http://press.web.cern.ch/press-releases/2011/09/ opera-experiment-reports-anomaly-flight-time-neutrinos-cern-gran-sasso Title Suppressed Due to Excessive Length 9 trino velocity (Borexino [Sanchez et al(2012)], ICARUS [Antonello et al(2012)] and LVD [Agafonova et al(2012)]). Bertolucci states that “[a]lthough this result isn’t as exciting as some would have liked, it is what we all expected deep down. The story captured the public imagination, and has given people the op- portunity to see the scientific method in action an unexpected result was put up for scrutiny, thoroughly investigated and resolved in part thanks to collaboration between normally competing experiments. That’s how science moves forward.” This information will be used in order to evaluate the scientific practice of the OPERA collaboration. However, I will first briefly illustrate how the effects mentioned above resulted in the erroneous results made by the OPERA collaboration.

4.3 Explanation of the Apparent Anomaly

One can distinguish two errors which the OPERA scientists did not know prior to the first two publications. The first is a time shift due to an improper connection of an optical cable. This reduces the amount of light received by the optical/electrical receiver of the Master Clock, which makes the Master Clock giving a pulse with a delay. This makes neutrinos appear to have traveled in less time than they actually have which results in apparent fast neutrinos. However, there also seemed to be a time drift in the other direction. This was due to an incorrect calibration of the Master Clock. Intuitively one could argue that both effects would cancel each other out. The time drift was however not big enough to compensate for the time shift. After additional tests and simulations, the OPERA collaboration was able to explain what went wrong and how these effects resulted in erroneous measurements.

5 Evaluation of Scientiﬁc Practice

The evaluation of the scientific practice of the OPERA collaboration will happen by means of section 3.7 in the work “Scientific Explanation” by Weber, Van Bouwel and De Vreese [Weber, E. and Van Bouwel, J. and De Vreese, L.(2013)]. They consider five clusters of evaluative questions, each centered around a different theme. The first cluster centers around ‘explanation-seeking questions’, the second around the ‘format’ in which the explanation is given, the third deals with the ‘ontological level’, the fourth with the ‘level of abstraction’ and the fifth and last considers the use of ‘irrelevant criteria’.

5.1 EQ1: Explanation-Seeking Questions and Epistemic Interests

This topic considers the questions asked by a scientific practice. Not every question that can be asked is equally interesting. Two questions from the 10 cluster which are especially important to evaluate the OPERA practice are the following: – “What are the interesting explanation-seeking questions about this phenomenon?” – “Do scientists in this discipline ask interesting explanation-seeking questions?”11 A first guideline in a search for interesting explanation-seeking questions is the following. Weber et al. suggest to search for mutually exclusive properties P and P ∗. The guideline suggests then that: “Suppose that object x has property P at time t. Then the question “Why does x have property P , rather than the ideal property P ∗?” is an interesting explanation-seeking question”[Weber, E. and Van Bouwel, J. and De Vreese, L.(2013), p. 63]. If we let property P be the ability to travel faster than the speed of light, the obvious interesting explanation-seeking question becomes the following contrastive question: “Why do the OPERA neutrinos appear to have moved faster than c while neutrinos in previous experiments appear to have c as an upper limit and in this way are in agreement with the well-established theory of relativity?”12 This is perhaps the most interesting question that can be asked about the experiment. It is remarkable that the OPERA collaboration does not ask a single question in all four versions of their paper about their measurement of the neutrino velocity. Regarding their conclusions they wrote in the different versions of their paper, they do seem to realize that they have encountered a phenomenon which is extraordinary. That is why we may conclude they are aware of the interesting explanation-seeking question concerning their results. Espe- cially as they speak in the final sentence of their final paper of completing “the scrutiny of the originally reported neutrino velocity anomaly”, they finally explicitly address their original result as an anomaly. It is only then that it is explicitly clear they did try to answer the former explanation-seeking question. The fact this is evident so late on in the research might imply a first hint of unprofessionalism.

5.2 EQ2: The Appropriate Format of an Explanation

The second cluster deals with the diﬀerent formats explanations can have. The most suitable evaluative question to ask in this case is the following:

– “Do scientists in this discipline give answers that have an appropriate format?”

11 As we are evaluating the OPERA collaboration, I will interpret in the following sections “scientists in this discipline” as “scientists of the OPERA collaboration”. 12 Weber et al. call this an I’-type question, as it invokes an ideal state considering two objects who are not the same but belong to the same category, see[Weber, E. and Van Bouwel, J. and De Vreese, L.(2013), p. 64]. Title Suppressed Due to Excessive Length 11

The OPERA collaboration gives a very thorough description of their experimental set-up and the way they selected and processed their data. They also seem to consider a great account of systematic effects and motivate approxi- mations and steps in their analysis thoroughly or by referring to other more extensive reports. They use an etiological account for interactions betweens particles and interactions between particles and detectors.13 All four versions have the same format. They are all papers published on the arXiv, but it goes further than that. They all have the same outline, and large parts of the paper are identically the same. Only version 1 differs from the others as it discusses its results under the section ‘data-analysis’, after which the paper ends with the section ‘conclusions’. The other versions have a separate section called ‘(final) results’ followed by a section which discusses an additional test called ‘test with a short-bunch wide-spacing beam’ after which the section ‘conclusions’ follows. It is odd that the section in version 2 is called ‘final results’ instead of ‘results’. This seems to indicate that the collaboration has reached some kind of end point in their research process. It seems troublesome that they use the same outline and the same parts of text in different versions of the paper. It is hard to spot the differences and one must actually search to find the explanation for the apparent anomaly. Because they only address the apparent anomaly properly in the final section in the final version of the paper, one gets only in the end the feeling that they are fully committed on exposing the reasons for their results. One might actually perceive that they want to hide their errors, which is something that should never happen. As they are fully aware of the meaning and impact of their results, they should add a straightforward and brief explanation which addresses the causes. When analyzing some of their communications to the press, we can conclude that they did not speak of an anomaly but are aware of the controversy of their results. The invoked scrutiny is at place. However, the ‘what-if-our- results-are-established scenarios’ in section 4.2 are not, especially not when communicating with the media. This should only be put forward when the results are effectively established. Furthermore, when some time later on the phenomenon is attributed to instrumental errors, even though the OPERA collaboration made sure the effect could not come from instrumental effects, it is only justified that your scientific practice is being questioned. It would be unfair to call them unprofessional purely based on this part of the communication, but one might understand why others would. Another feature which wasn’t discussed yet, is the question why the OPERA collaboration did not wait any longer to publish their results as they don’t have a competing collaboration. For example, in the search for the Higgs-Boson there are two competing collaborations who carry out the task, the ATLAS collaboration and the CMS collaboration. This has been done in order to prac-

13 See [Weber, E. and Van Bouwel, J. and De Vreese, L.(2013), p. 48-49] for a more detailed discussion considering the etiological account. 12 tice science in an efficient manner, as the scientific enterprise is built around the priority rule; only the first who achieves a result is credited.14 However, this is not the case for the OPERA collaboration. In this way, there was no need to publish fast and they could have carried out their research a bit more in all peace and quiet.

5.3 EQ3: Explanations and Levels of Reality

The third cluster of evaluation considers different levels of reality where the explanation can be situated. We can evaluate the OPERA collaboration by means of the following question: – “Do scientists in this discipline consider explanations at different levels of reality and do they make justified decisions about the level to be used?” One can say that the OPERA collaboration has put a lot of effort in the proper application of various theories and methods. For example, they carried out an additional test considering the energy dependence of neutrinos; in the second version of their paper they associated each neutrino to its waveform instead of the global PDF, which is a lot more accurate. However, one may get the feeling that they did not put enough effort in screening and making sure the experimental hardware works in the way they think it works. It took them until the last versions to come up with the proper explanation, published ten months after the firs appearance of the paper. Malfunction- ing hardware appears quite unprofessional after having performed additional checks and making sure you applied your theory right, even more when you stated you could not attribute any instrumental malfunctioning to the explain- ing of the results.

5.4 EQ4: Abstraction and Amount of Detail in Explanations

This cluster deals with the amount of information included in the explanation. Which information is causally relevant and which information can be excluded? The question to evaluate the OPERA collaboration is the following: – “Is the level of detail of the explanation adequate?” According to Weber et al., an explanation is adequate if it contains information which makes a diﬀerence. The description of the experimental set-up and statistical data-analysis is elaborate, but not overly detailed. They deal cautiously with the amount of details and often refer to additional notes or other literature for those who want to know more about a certain feature of which the motivation is not essential to the experiment.

14 For a schematic overview of the priority rule, see for example [Strevens(2003), p. 56 - 58]. Title Suppressed Due to Excessive Length 13

However, as already earlier stated, the explanation of the apparent anomaly should be a lot more clearer when reading their ﬁnal paper. They should have addressed the causation more properly and perhaps include a distinct section in which they elaborate on their explanation. They did however test the time shift and drift thoroughly, and were able to explain their previous results afterwards. The last cluster of evaluation (EQ5) deals with irrelevant premises. As the OPERA collaboration did not use any, it is clear that this cluster of evaluation does not have to be used.

6 Conclusion

How do we evaluate the practice of the OPERA collaboration? Scientific experiments are nowadays very complex. A large number of scientists have to cooperate in the hope to accomplish in this case two goals: determination of the first oscillation in the νµ ντ channel and the measurement of the neutrino velocity with an higher→ accuracy than previous experiments were able. And in the end the OPERA collaboration managed to accomplish then. They have built a complex experimental set-up which was able to do the task and performed a very thorough statistical analysis, for which they should be credited. They were open about their results and invoked broader scrutiny at a time they were distraught about the follow-up of their work. Based on EQ1 it is clear the OPERA collaboration realizes what the interesting explanation-seeking question is concerning their results. However, considering EQ2, they should have addressed the anomaly in their papers more properly. One really has to search for the explanation to this question; a distinct section or perhaps even a paper dealing solely with the explanation would be at place. There was also no need for hurrying in publishing their results as they had no competitors. Analyzing the press releases they spoke with proper care about their results, however they did speak of possible consequences in case their results would be established. This however is conjecture. All in all we can say that based on EQ2 it is justified to dub the format in which the OPERA collaboration addressed the anomaly as inappropriate. Even more, we also can state the OPERA collaboration acted unprofessional based on EQ3. Eventually it was clear the anomaly was caused by two internal errors in the experimental set-up. Theories and methods are open for other scientists as well to evaluate and sometimes adjust or redefine, but the experimental set-up is not. When it then becomes clear the phenomenon is nevertheless due to errors in the internal business of the OPERA collaboration only a long time after they have invoked a broader scrutiny, it is justified to call the OPERA collaboration unprofessional. When invoking broader scrutiny other scientists need to be sure the experimental set-up operates the way it was communicated it operates. Otherwise no scientist is able to perform proper science. 14

In the end we might say that it is still science that triumphs, as the apparent anomaly did get its proper explanation and we furthermore were able to solve yet another puzzle in the area of experimental subatomic particle physics.

References

Acquafredda et al(2009). Acquafredda R, et al (2009) The OPERA Experiment in the CERN to Gran Sasso Neutrino Beam. JOURNAL OF INSTRUMENTATION 4 Adam et al(2011a). Adam T, et al (2011a) Measurement of the Neutrino Velocity with the OPERA Detector in the CNGS Beam URL http://arxiv.org/abs/1109.4897v1, 1109.4897 Adam et al(2011b). Adam T, et al (2011b) Measurement of the Neutrino Velocity with the OPERA Detector in the CNGS Beam URL http://arxiv.org/abs/1109.4897v2, 1109.4897 Adam et al(2012a). Adam T, et al (2012a) Measurement of the Neutrino Velocity with the OPERA Detector in the CNGS Beam URL http://arxiv.org/abs/1109.4897v3, 1109.4897 Adam et al(2012b). Adam T, et al (2012b) Measurement of the Neutrino Velocity with the OPERA Detector in the CNGS Beam URL http://arxiv.org/abs/1109.4897v4, 1109.4897 Adam et al(2012). Adam T, et al (2012) Measurement of the Neutrino Velocity with the OPERA Detector in the CNGS Beam. Journal Of High Energy Physics (10) Adam et al(2013). Adam T, et al (2013) Measurement of the Neutrino Velocity with the OPERA Detector in the CNGS Beam Using the 2012 Dedicated Data. Journal Of High Energy Physics (1) Adamson et al(2007). Adamson P, et al (2007) Measurement of neutrino velocity with the minos detectors and numi neutrino beam. Phys Rev D 76:072,005 Agafonova et al(2012). Agafonova NY, et al (2012) Measurement of the Velocity of Neu- trinos from the CNGS Beam with the Large Volume Detector. PHYSICAL REVIEW LETTERS 109(7) Antonello et al(2012). Antonello M, et al (2012) A Search for the Analogue to Cherenkov Radiation by High Energy Neutrinos at Superluminal Speeds in ICARUS. Physics Let- ters B 711(3-4):270–275 Bertolin and Tran(2009). Bertolin A, Tran N (2009) Opcarac: An algorithm for the classification of the neutrino interactions recorded by the opera experiment. OPERA Pub- lic Note 100, URL http://operaweb.lngs.infn.it:2080/Opera/publicnotes/note132. pdf Bettini(2008). Bettini A (2008) Introduction to Elementary Particle Physics. Cambridge University Press Brown, K. and Khan, A.(2011). Brown, K and Khan, A (2011) Faster Than Light? CERN Findings Bewilder Scientists. Los Angeles Times Brumfield, G.(2011). Brumfield, G (2011) Particles Break Light-Speed Limit. Nature News Colosimo et al(2011). Colosimo G, et al (2011) Determination of the cngs global geodesy. OPERA Public Note 132, URL http://operaweb.lngs.infn.it:2080/Opera/ publicnotes/note132.pdf Feldmann et al(2011). Feldmann T, et al (2011) Relative calibration of the gps time link between cern and lngs, report calibration cern-lngs 2011. OPERA Public Note 134, URL http://operaweb.lngs.infn.it:2080/Opera/publicnotes/note134.pdf Hooker, B.(2011). Hooker, B (2011) OPERA Experiment Reports Anomaly in Flight Time of Neutrinos. Fermilab Today Kalbleish et al(1979). Kalbleish G, et al (1979) Experimental Comparison of Neutrino, An- tineutrino and Muon Velocities. Physical Review Letters 43(19):1361–1364 Longo(1987). Longo MJ (1987) Tests of relativity from sn1987a. Physical Review D 36:3276–3277 Matson, J.(2011). Matson, J (2011) Faster-Than-Light Neutrinos? Physics Luminaries Voice Doubts. Scientific American Title Suppressed Due to Excessive Length 15

Sanchez et al(2012). Sanchez P, et al (2012) Measurement of CNGS muon neutrino speed with Borexino. PHYSICS LETTERS B 716(3-5):401–405 Strevens(2003). Strevens M (2003) The Role of the Priority Rule in Science. Journal of Philosophy 100(2):55–79 Weber, E. and Van Bouwel, J. and De Vreese, L.(2013). Weber, E and Van Bouwel, J and De Vreese, L (2013) Scientiﬁc Explanation. Springer Chapter 4

Conclusion

As I stated in chapter1, there is no direct connection between FTCTS and EVAL. However, as I tried to illustrate, the OPERA episode might be a suitable case study of scientists trying to find the balance between exploring new theories and exploiting existing ones. Perhaps it might even be a suitable case-study for inquiries in the philosophy of science in general, as it is both surveyable as significant. I will outline future research in which the OPERA episode is apt to play a part, but first I will motivate why the OPERA episode can be used as a suitable case study.

As Amelino-Camelia notes there appeared more than two hundred OPERA inspired papers on the arXiv during the six months of debate considering the OPERA results [1].1 This is a signiﬁcant amount, large enough to perform a quantitative analysis. Yet it is still small enough to perform qualitative research as well. It is possible to approach this episode ‘historically’ in a sense that each OPERA inspired paper can be analysed individually. This is why the OPERA episode is quite unique in its kind and seems well-suited as a case study for analysing scientiﬁc dynamics.

The basic thought for possible future research is that a scientific community can be per- ceived as a complex system [6]. In this way we are in need of local interaction rules, the first principles of the system, as the system displays emergent behaviour. These principles are what one might call the fundamentals of the system. The research I therefore would like to suggest is a search for the first principles that describe the dynamics of our current scientific system.

1This is why according to Amelino-Camelia the next fundamental revolution in physics will not be recorded in conventional journals, as the arXiv is an open source archive to publish preprints of papers online, which makes no use of a blinded peer review system.

32 33

To make it plausible there actually exists such a set of principles one should first try to find these principles in theoretical models in philosophy of science. An example of such a model is the “Structure of Scientific Revolutions” by Thomas S. Kuhn [9, 10]. As FCTCS has shown, it is possible to model Kuhnian dynamics as complex system. The local rules that govern the global behaviour can then be extrapolated as the first principles of the system. Of course, it is in a sense easier achievable to model a theoretical complex system. However, it is needed to in fact show that it is possible to retrieve such principles; this justifies the complex-system-approach. It will also become important in a later stage, when one will be able to compare the theoretical principles with the empirically derived principles, but more of that later on.

What FCTCS shows is that scientists who try to find a balance between exploring new theories versus exploiting existing ones give rise to a Kuhnian scientific system. That both roles in science are intrinsically different is motivated by the fact that the explorers follow another distribution than the exploiters [7]. This gives rise to a principle that in order to generate non-cumulative breaks there is need of a minority of scientists who perform exploring research. The fact that a model based on pure local dynamics gives rise to a system that behaves globally as Kuhn describes it, suits the complex-systems-approach.

This research can be extended and one can try to use the complex-systems-approach to other models in the philosophy of science. I would like to suggest two of these models. The first one is the concept of a “research programme” by Imre Lakatos [11]. A research programme consists out of a hard core of theoretical assumptions which cannot be altered. Leaving these assumptions would result in a switch to another research programme. The core of a research programme is protected by a set of auxiliary hypotheses, which can be changed. Lakatos put this concept forward in an attempt to combine Popper’s falsifica- tionism and Kuhn’s concept of scientific revolutions.

The second one I would like to suggest is the concept of a “research tradition” put forward by Larry Laudan [12], which is an alternative to Lakatos his research programmes. A research tradition is in a sense a more loose concept as it is less clearly defined than a “paradigm’ ’or “research programme”. Laudan offers also a rational rule for scientists to choose between different traditions. Scientists should choose that tradition which makes the most progress.

One can see the immediate connection between these models, originating at Karl Popper and Thomas Kuhn. It is still not clear which theory is more apt to describe actual scientiﬁc 34 dynamics. Using the complex-system-approach might resolve this issue. Another reason I highlight these models is that all these philosophers have a technical background in mathematics or physics, which has its reﬂection in their models.2

As a second step one can now use the complex-systems-approach in order to analyse current scientific dynamics empirically. If so, one should now search for that set of first principles which make up the empirical scientific dynamics.

A number of such inquiries have already taken place and are in favour of the complex- systems approach. Newman for example shows that several collaboration networks form so- called small worlds, networks in which randomly chosen pairs of scientists can be connected by only a short path of fellow collaborators [14]. This could be an essential feature of a scientiﬁc community. Furthermore he shows that scientiﬁc communities exhibit a high level of clustering. This means that two scientists are much more likely to collaborate if they have a third collaborator in common. This might show that it is important for scientists to introduce fellow collaborators to each other.

In a more recent study Newman was able to predict which papers would be highly cited [17]. Forerunners in a certain field appear to gain significantly more citations, partly because of the first-mover effect [15] and partly because they have had longer to accumulate citations. In this way it might show that it is profitable to be one of the first in the field.

However there is still a long way to go in order to arrive at a full set of principles which describes the actual scientiﬁc dynamics. But as De Langhe notes, as we now have ‘Big Data’ at our disposal, and even more, the computational capacities to deal with it, we should now be able to retrieve these principles in a more full concept [6].

Newman’s research and FTCTS go hand in hand and form an argument for each other. However they also comply to a concept which is known in the philosophy of science as the “priority rule”, first put forward by Robert Merton.3 This rule holds that the first and only the first who accomplishes a certain result in science is credited.

Exploring work will, when successful, result in high rewards, according to both FTCTS and Newman’s work: theoretically such a paper might result in a non-cumulative break and empirically such a paper will have a high impact. Or in other words, the rewards

2Thomas S. Kuhn and Larry Laudan are both physicists, whereas Imre Lakatos is both a physicist and a mathematician. 3See [23] by Michael Strevens for a concise overview of the priority rule and its usage in current science. 35 are high when you are the first to perform successful research in an area which is not the established area. This means that we can state that the first-mover effect or the priority rule acts as a first principle in the current scientific dynamics.

This is the third step I would like to suggest in future research. To put it more generally, after one has completed the first and second step I described earlier, one should now search for concepts known in philosophy of science which comply to the first principles localized in both theoretical as empirical scientific dynamics. If such principles are still unknown, one should try to put new concepts forward. In this way a framework can be created of new and existing concepts which accurately describe theoretical and empirical scientific dynamics.

Even more, with both sets of principles at our hand, we are now capable to compare them. In this way we are now able to ﬁnally state which models in the philosophy of science are appropriate to describe scientiﬁc dynamics. Bibliography

[1] G. Amelino-Camelia. Phenomenology of Philosophy of Science: OPERA Data. ArXiv e-prints, Jun 2012.

[2] M. Antonello et al. A Search for the Analogue to Cherenkov Radiation by High Energy Neutrinos at Superluminal Speeds in ICARUS. Physics Letters B, 711(3-4):270–275, May 2012.

[3] Y. Barlas. On the Very Idea of a System Dynamics Model of Kuhnian Science - Comments. System Dynamics Review, 8(1):43–47, Win 1992.

[4] S. Bornholdt, M. H. Jensen, and K. Sneppen. Emergence and Decline of Scientiﬁc Paradigms. Physical Review Letters, 106(5):058701, Feb 2011.

[5] A. G. Cohen and S. L. Glashow. Pair Creation Constrains Superluminal Neutrino Propagation. Physical Review Letters, 107(18), Oct 2011.

[6] R. De Langhe. The Kuhnian Paradigm. Topoi - An International Review of Philoso- phy, 32(1):65–73, Apr 2013.

[7] R. De Langhe and P. Rubbens. From Theory Choice to Theory Search: The Essen- tial Tension between Exploration and Exploitation in Science. In Devlin, W.J. and Bokulich, A., editor, Kuhn’s Structure of Scientiﬁc Revolutions - 50 Years On, volume 311 of Boston Studies in Philosophy and History of Science. Springer, 2015.

[8] G. F. Giudice, S. Sibiryakov, and A. Strumia. Interpreting OPERA Results on Su- perluminal Neutrino. Nuclear Physics B, 861(1):1–16, Aug 2012.

[9] T.S. Kuhn. The Structure of Scientiﬁc Revolutions. University of Chicago Press, 1962.

[10] T.S. Kuhn. The Structure of Scientiﬁc Revolutions. University of Chicago Press, 1970.

36 Bibliography 37

[11] I. Lakatos. Criticism and the Methodology of Scientiﬁc Research Programmes. Pro- ceedings of the Aristotelian Society, 69:149–186, 1968.

[12] L. Laudan. Progress and Its Problems: Towards a Theory of Scientiﬁc Growth. Rout- ledge & Kegan Paul, 1977.

[13] J. Leliaert. Modellering van de Dynamiek van Wetenschappelijke Paradigma’s. Mas- ter’s thesis, Ghent University, 2012.

[14] M.E.J. Newman. The Structure of Scientiﬁc Collaboration Networks. Proceedings of the National Academy of Sciences of the United States of America, 98(2):404–409, Jan 2001.

[15] M.E.J. Newman. The First-Mover Advantage in Scientiﬁc Publication. EPL, 86(6), Jun 2009.

[16] M.E.J. Newman. Complex Systems: A Survey. American Journal of Physics, 79:800– 810, 2011.

[17] M.E.J. Newman. Prediction of Highly Cited Papers. EPL, 105(2), Jan 2014.

[18] M.J. Radzicki. On the Very Idea of a System Dynamics Model of Kuhnian Science - Reﬂections. System Dynamics Review, 8(1):49–53, Win 1992.

[19] Peter Rubbens. Evaluation of the opera collaboration and the faster than light neutrino anomaly, Nov 2014.

[20] J.D. Sterman. The Growth of Knowledge : Testing a Theory of Scientiﬁc Revolutions with a Formal Model. Technological Forecasting & Social Change : An International Journal, 28(2):99–123, 1985.

[21] J.D. Sterman. On the Very Idea of a System Dynamics Model of Kuhnian Science - Response. System Dynamics Review, 8(1):35–42, Win 1992.

[22] J.D. Sterman and J. Wittenberg. Path Dependence, Competition, and Succession in the Dynamics of Scientiﬁc Revolution. Organization Science, 10(3):322–341, May-Jun 1999.

[23] M. Strevens. The Role of the Priority Rule in Science. Journal of Philosophy, 100(2):55–79, Feb 2003. Bibliography 38

[24] Weber, E. and Van Bouwel, J. and De Vreese, L. Scientiﬁc Explanation. Springer, 2013.

[25] J. Wittenberg. On the Very Idea of a System Dynamics Model of Kuhnian Science. System Dynamics Review, 8(1):21–33, Win 1992.