Research Methods Course Objectives

Scientific Method ■ Understand the nature and practice of science in general Tim Weyrich ■ http://www.cs.ucl.ac.uk/staff/t.weyrich/ Learn how to identify and test Based on slides by Daniel Alexander scientific hypotheses ■ Understand publishing processes ■ Perform a literature review for your Master’s project ■ Obtain practice in written and oral presentation

1 2

Course Objectives Overview

■ A bit of philosophy ■ Learn how to work like a scientist (after all, it’s an MSc…) ■ Some technical material – Statistical hypothesis testing • Frequentist and Bayesian – Experiment design • Modelling and sampling, experiment design. ■ Handling live subjects – Data from humans (questionnaires) – Ethics in science ■ Mechanisms of scientific discourse ■ Oral and written presentation

3 4

Assessment Today’s Outline

■ Coursework ■ What is this thing called science? • 2 pieces, 20% ■ Facts and experiments ■ Written Literature Review ■ Induction and Deduction • max.(!) 10,000 words, 60% ■ Falsifiability ■ Final Presentation on literature and ■ Paradigms and revolutions planned research ■ Other theories of science • 15 minutes, 20%

5 6 What is this thing called science? What is this thing called science?

■ Knowledge derived from facts! ?

7 8

Facts and experiments Facts and experiments

■ What are the facts ■ What are the facts

9 10

Facts and experiments Facts and experiments

■ What are the facts ■ What are the facts ■ What people observe depends highly ■ What experiments do you do? on their experience ■ Influence of theory on: • Botanist • What experiments • Radiologist • Experimental setup ■ Observations of facts are fallible

11 12 Deduction and induction Deduction and induction

■ How do we derive knowledge from ■ Deductive logic is truth preserving facts 1. All lectures on philosophy are boring ■ Deductive logic: Move from given 2. This lecture is on philosophy premises (by definition true) to 3. Therefore, this lecture is boring conclusions 1. All humans have 2 noses ■ Inductive logic: Move from particular 2. I am human facts to general theory 3. Therefore, I have 2 noses

13 14

Deduction and induction Deduction and induction

■ When can we apply inductive logic ■ Inductive logic • N is large • Metal x expanded when heated at t 1 1 • N is varied • Metal x expanded when heated at t 2 2 • The theory does not contradict any • Metal x3 expanded when heated at t3 existing evidence • Therefore, all metals expand when heated ■ Inductive logic as applied by the Christmas turkey (Bertrand Russel)

15 16

Deduction and induction Deduction and induction

■ When can we apply inductive logic ■ Further problems with inductionism • N is large • How can the existence of unobservable • N is varied concepts (DNA, atoms) be derived from • The theory does not contradict any observable facts existing evidence • How can the correctness of inductionism ■ Problematic!! be induced? • When is N large enough • What should we vary • There are always exceptions

17 18 Deduction and induction Falsifiability

Theories ■ Karl Popper (1902 – 1994) ■ Worried about all kinds of ‘scientific theories” which were backed up by facts (Marxism, Freud)

Facts New ■ These theories can be made to acquired predictions fit any data ■ But do they rule out anything? • Rainbow example

19 20

Falsifiability Falsifiability

■ Science makes definite predictions ■ Eddington’s test of general relativity in 1919 ■ Science is therefore falsifiable ■ Observations are guided by and presuppose theories ■ Theories are tentative constructions created by scientists to overcome problems existing with previous theories

21 22

Falsifiability Falsifiability

■ A hypothesis is falsifiable if there ■ Falsifiable statements exists a logically possible observation 1. It never rains on Wednesdays statement that is inconsistent with it 2. Either it is raining or it is not raining ■ Example of falsification 3. All points on a Euclidean circle are • Theory: All swans are white equidistant from the centre • Discovery of Australia 4. All metals expand when heated • Observation of black swans 5. Heavenly objects fall in straight lines • Theory is falsified towards the earth

23 24 Falsifiability Falsifiability

■ Falsifiable statements Is string theory a valid scientific theory? 1. Luck is possible in speculation in "For more than a generation, physicists have been chasing a willo-the-wisp called string theory. The beginning of this chase sports marked the end of what had been three-quarters of a century of progress. Dozens of string-theory conferences have been held, hundreds of new Ph.D.s have been minted, and thousands of 2. When a ray of light is reflected from a papers have been written. Yet, for all this activity, not a single new testable prediction has been made, not a single theoretical plane mirror, the angle of incidence is puzzle has been solved. In fact, there is no theory so far—just a set of hunches and calculations suggesting that a theory might equal to the angle of reflection exist. And, even if it does, this theory will come in such a bewildering number of versions that it will be of no practical use: a Theory of Nothing.

http://www.newyorker.com/archive/2006/10/02/061002crat_atlarge

25 26

Falsifiability Falsifiability

■ A caricature of Adler’s psychology ■ Degrees of falsifiability • All behaviour is guided by feelings of a) Mars moves in an ellipse around the sun inferiority of some kind b) All planets move in ellipses around the • Situation: Man is standing next to a sun treacherous river and suddenly a child ■ The set of potential falsifiers for a) is falls in. a subset of the set of potential • What does he do? falsifiers for b).

27 28

Falsifiability Falsifiability

■ The enterprise of science involves the Inductionism Falsificationism proposal of highly falsifiable Only theories that are Any theory hypotheses, followed by deliberate true are scientific (speculative or not) and tenacious attempts to falsify which is falsifiable is them. useful for the advancement of our ■ Science is trial and error. knowledge ■ Falsification is the major growth path The more precise a theory is the more falsifiable it is and thus the better

29 30 Falsifiability Falsifiability

Progress of science ■ Sophisticated falsification ■ Set of problems ■ Competing theories ■ Set of hypotheses to deal with them ■ New theory is accepted above ■ Falsification of many of these hypotheses previous ones if: • It is more falsifiable than the previous ■ Further testing of the remaining successful hypotheses one • It predicts a new phenomenon not taken ■ New problems arise into account in previous theories ■ Etc…

31 33

Falsifiability Falsifiability

■ The discovery of Neptune ■ The proposal of highly risky new • The orbit of Uranus did not fit with theories is not always a good thing Newtonian mechanics ■ Progress is especially made if: • A new planet was proposed which 1. A bold conjecture is confirmed. disturbed Uranus’s orbit • A range of possible orbits for the new 2. A cautious conjecture is falsified planet was derived ■ Confirmation does have a role in the • The new planet (Neptune) was observed (intial) acceptance of theories in the range of predicted orbits!

35 36

Limitations of falsifiability Limitations of falsifiability

■ Fact is inconsistent with theory ■ Example: Tycho Brahé’s refutation of • Experimentalist: “Theory is wrong” Copernicus • Theorist: “Measurement is wrong” ■ If the earth orbits the sun, the position ■ Observations are theory-driven and of the stars should differ during the fallible. So if O is observed, which course of a year. falsifies T, then this will only mean ■ He did not observe this because he that O|T is false ■ The observation of O is underestimated the distance to these supplemented by auxiliary theory other stars (measurement apparatus)

37 38 Limitations of falsifiability Paradigms and revolutions

■ Can we back up falsifiability by ■ Thomas Kuhn (1922 – 1996) historical examples ■ “The structure of scientific revolutions” ■ From Copernicus to Newton…

pre-science – normal science – crisis – revolution (paradigm shift) – normal science – crisis - …

40 41

Paradigms and revolutions Paradigms and revolutions

■ Scientists normally work within a ■ Kuhn’s definition of a scientific paradigm. paradigm: ■ Paradigm: • what is to be observed and scrutinised • Sets the standard for legitimate work • the kind of questions that relate to the subject • Coordinates ‘puzzle-solving’ • how these questions are to be • Distinguishes science from non-science structured • Has explicitly stated fundamental laws • how the results of scientific and theoretical assumptions investigations should be interpreted

42 43

Paradigms and revolutions Paradigms and revolutions

■ Paradigm: ■ Normal scientists don’t spend much • Has standard ways to of applying these time worrying about the paradigm laws ■ Anomalies do not falsify the paradigm • E.g.: Apply Newton’s law to predict but are new puzzles to be solved motion of planets, pendulums and within the paradigm billiard balls ■ Scientists are taught within the • Has also something to say about the paradigm: instruments of experimentation. • Solving standard problems • Perform a piece of research under supervision (PhD)

44 46 Paradigms and revolutions Paradigms and revolutions

■ Crisis and revolution ■ Function of normal science: ■ Existence of increasing number of anomalies which are ■ Enables direction of energy towards • Fundamental broadening the scope of the current • Long-standing paradigm. ■ Occurrence of rival paradigms ■ Paradigms are interpreted differently ■ New paradigm by different research groups => 1. is a new description of the world with new concepts Diversification of strategies within the 2. Asks new questions paradigm. 3. Lives in a ‘different world’

47 48

Paradigms and revolutions Paradigms and revolutions

■ Limitations of Kuhn’s theory: ■ Moving away from the exact ■ Not clear how different paradigms can sciences: be compared and how the best one ■ Psychology can be chosen • Mentalism ■ Is Kuhn a relativist? • Behaviourism ■ He didn’t think so… • Cognitive Neuroscience ■ Description of scientific practice instead of theory?

49 50

Other theories of science Other theories of science

■ Paul Feyerabend (1924 – 1994) ■ Bayesianism ■ Against Method: Outline of an ■ New experimentalism Anarchistic Theory of Knowledge ■ Constructive empericism ■ No distinction between science and non-science ■ “Anything goes” as scientific method ■ Science does not deserve its privileged status in Western society

51 52 Summary

■ The norms of scientific method • Hypothesis generation • Data gathering • Hypothesis non-rejection/rejection ■ An ‘ideology’ – a set of rules that are supposed to be followed ■ In practice this happens only at a very small scale – data that threatens the paradigm is usually suppressed in some way ■ But adherence to norms of scientific method constitutes an idea to follow in practice ■ (If not, your papers won’t be published anyway!) ■ ‘Scientific truth’ is a social convention.

53