C82MST: Statistical methods
What is good science?
Tobias Bast, School of Psychology, University of Nottingham Is Psychology a science?
1)Yes!
2) No!
3)Don’t know . . . Is Psychology a science? Is science better than other approaches to understand and explain the world?
“Science is much closer to myth than a scientific philosophy is prepared to admit. It is one of the many forms of thought that have been developed by man, and not necessarily the best. It is conspicuous, noisy, and impudent, but it is inherently superior only for those who have already decided in favour of a certain ideology, or who have Paul Feyerabend 1924-1994 accepted it without ever having examined its advantages and its limits.”
Quoted in Schick (1997) What is science?
“ Ask a scientist what he conceives the scientific method to be, an he will adopt an expression that is at once solemn and shifty-eyed: solemn because he feels he ought to declare an opinion; shifty-eyed because he is wondering how to conceal the fact that he has no opinion to Sir Peter Medawar 1915-1987 declare.” Quoted in Schick (1997) What is science? “Ideas and techniques of scientific investigation”:
•“[T]he first thing really is, before you begin you must not know the answer. . . The question of doubt and uncertainty is Richard Feynman necessary to begin.” 1918-1988 •“[T]he next thing is to look for evidence, and the scientific method is to begin with trials.”
•“But another way and a very important one that should not be neglected and that is very vital is to put together ideas to try to enforce a logical consistency among the various things that you know.”
•“After we look for the evidence we have to judge the evidence; it’s not right to pick only what you like, but to take all the evidence, to try to maintain some objectivity about the thing . . .” “Disinterest” in reporting results is important, i.e. “that they are not reported in such a way as to try to influence the reader into an idea that’s different than what the evidence indicates.”
•“Authority may be a hint as to what the truth is, but is not the source of information. As long as it’s possible, we should disregard authority whenever the observation disagrees with it.” Feynman (1964) What is and what should be the role of scientific culture in modern society? In: Feynman (1999) The pleasure of finding things out The scientific method (in a nut shell)
•Wanting to find things out!
Francis Bacon •Collect empirical evidence through observation 1561-1626 and/or experiments
•Induce a hypothesis based on the evidence (i.e., conclude a general prediction from past regular occurrences)
•Test hypothesis by collecting additional Galileo Galilei 1564-1642 empirical evidence (additional observations and/or experiments)
•Accept, modify, or reject hypothesis
Isaac Newton 1643-1727 Scientific proof a) Scientists can prove that a hypothesis is right. b) Scientists can prove that a hypothesis is wrong. c) Neither a) nor b) is correct. Notes of caution . . .
• Why should the past resemble the future?
•Strictly spoken, there can be no definite empirical ‘proof’ of a hypothesis: that we have observed confirmative evidence many times does not exclude the possibility of evidence to the contrary!
David Hume 1711-1776 Critical attitude and attempted falsification
•“All [scientific] laws and theories are conjectures or tentative hypothesis” (rather than dogmas!)
•Empirical evidence can never ‘prove’ a theory. “Only the falsity of a theory can be inferred from the empirical evidence . . .”
•“The method of science is criticism, i.e. attempted falsifications.” This is “the method of trial and error – of conjecture and refutation: of boldly proposing our Karl Popper theories; of trying our best to show that these are 1902-1994 erroneous; and of accepting them tentatively if our critical efforts are unsuccessful.”
• ‘Attempted falsification’ suggested as criterion for demarcation of science from other activities (including ‘pseudo-science’).
Popper (1963) Science: Conjectures and refutations Normal science, paradigms, and scientific revolutions
•‘Normal’ science is characterized by ‘paradigms’, i.e. sets of theoretical assumptions, methodological approaches, and research directions shared by most scientists.
•During ‘normal’ phases, scientists work on the basis of such ‘paradigms” and try to confirm and elaborate widely-accepted theoretical assumptions; findings contradicting such assumptions (‘anomalies’) tend to Thomas Kuhn be ignored or explained away. 1902-1994 •An accumulation of ‘anomalies’ leads to a scientific ‘crisis’, i.e. faltering confidence in a ‘paradigm’.
•In response to crisis, science progresses through ‘scientific revolution’, i.e. a ‘paradigm shift’.
Thomas Kuhn (1962) The structure of scientific revolutions So, what are the key features of science . . .?
“Science is different from many another human enterprise -- not, of course, in its practitioners being influenced by the culture they grew up in, nor in sometimes being right and sometimes wrong (which are common to every human activity), but in its passion for Carl Sagan 1934-1996 framing testable hypotheses, in its search for definitive experiments that confirm or deny ideas, in the vigor of its substantive debate, and in its willingness to abandon ideas that have been found wanting.” Carl Sagan (1995) The demon-haunted world: science as a candle in the dark
What makes good science?
•Interesting and relevant question
•Interesting hypothesis/theory
•Powerful methodology
•Thorough and careful data collection and analysis
•Interesting results
•Ethical standards? (Next lecture) Psychological research recognised with a Nobel Prize 1904. Ivan P. Pavlov (Physiology or Medicine) "In recognition of his work on the physiology of digestion, through which knowledge of vital aspects of the subject has been transformed and enlarged" 1906. Santiago Ramón y Cajal and Camillo Golgi (Physiology or Medicine) "In recognition of their work on the structure of the nerve cell" 1949. Edgar Moniz (Physiology or Medicine, with Walter Rudolf Hess) "For his discovery of the therapeutic value of leucotomy in certain psychoses" 1961. Georg von Békésy (PhM) "For his discoveries of the physical mechanism of stimulation within the cochlea" 1973. Karl von Frisch, Konrad Lorenz and Nikolaas Tinbergen (PhM) "For their discoveries concerning organization and elicitation of individual and social behavior patterns" 1978. Herbert Simon (Economics) "For his groundbreaking work on bounded rationality principle bridging psychology and economics" 1981. Roger W. Sperry & David H. Hubel and Torsten N. Wiesel (PhM) "For his work on the functional specialization of the cerebral hemispheres“ “For their discoveries concerning information processing in the visual system.“ 2000. Arvid Carlsson, Paul Greengard and Eric R. Kandel (PhM) “for their discoveries concerning signal transduction in the nervous system” 2002. Daniel Kahneman,( Economics, with Vernon Smith) "For his groundbreaking work on applying psychological insights to economics theory, particularly in the areas of judgement and decision-making under uncertainty" 2014. John O´Keefe, May Britt-Moser and Edvard I.Moser (PhM). "For their discoveries of cells that constitute a positioning system in the brain“ 2017. Richard Thaler (Economics) "for his contributions to behavioural economics." https://www.nobelprize.org/
Quality control in science: peer review Peer review: process by which research or research proposals are evaluated (‘reviewed’) by independent experts (‘peers’) to decide on publication or funding (or awards).
Parliamentary Office of Science and Technology (September 2002) Peer review. Postnote 182 https://www.parliament.uk/documents/post/pn182.pdf Impact of science
•Academic impact:
-Number of publications
-Quality of publications
-Impact indicators (citation factors)
•Social and economic impact:
-Influence on public opinion
-Influence on policies
-Applications, e.g. treatments, patents, drugs Science world powers
David A. King (2004) The scientific impact of nations. Nature 430:311-316 Science world powers – Bang for buck
David A. King (2004) The scientific impact of nations. Nature 430:311-316 https://www.theguardian.com/news/datablog/ng-interactive/2014/dec/18/university- research-excellence-framework-2014-full-rankings Some questions to ponder
•What makes psychology a science, what makes me a scientist (are there criteria that ‘demarcate’ science from other attempts to explain and understand the world)?
•How does science proceed and progress?
•Is there scientific proof?
•Is there scientific fact?
•Is there a difference between science and philosophy?
•What makes (my) science good and exciting?
•Can we measure the quality of science?
•What are the advantages and disadvantages of peer review? Some reading suggestions Schick, Jr., T (1997) The end of science. The skeptical inquirer 21.2 (http://www.csicop.org/si/show/end_of_science/) Popper K (1963) Science: conjectures and refutations (reprint on Moodle and on my webpage) Parliamentary Office of Science and Technology (September 2002) Peer review. Postnote 182 (https://www.parliament.uk/documents/post/pn182.pdf)