The Scientific Method
Total Page:16
File Type:pdf, Size:1020Kb
The Scientific Method DA2205 September 22, 2014 Recap of previous lecture • Do computer science research as scientific research because 1. Computer science can be viewed as a science: the transformation of artificial and natural information processes and/or 2. It forces you to have good research methods • Research is hard because of 1. intrinsic difficulty of discovering the unknown, 2. personal failings - self-deception, 3. bias and limitations imposed by the community. • But the scientific method has allowed for rapid and not too many misguided declarations of progress! The computer scientific method Obsolete Scientific Method Computer Scientific Method • Hypothesis • Hunch • Sequence of experiments. • 1 experiment & change all parameters • Change 1 parameter per experiment. • Discard if it doesn't support hunch • Prove/Disprove Hypothesis. • Why waste time? • Document for others to repro- We know this. duce results. Source: \How to have a bad career in research/academia" by David Patterson, Feb 2002. Good research methods help you • Identify interesting new questions you hadn't thought of. (Exploratory data analysis) • Ask questions that are scientifically meaningful. (Falsifiable hypotheses) • Avoid fixating on one favored hypothesis. (Multiple working hypotheses) • Devise evaluations that maximize what you can learn. (Experimental design) • Avoid seeing patterns in randomness. (Hypothesis tests) • Understand when you have collected sufficient evidence. (Statistical power) • Formulate broader theories. (Modeling) It discourages this scenario The Significance of Research 9-Oct-10 2 Today's lecture • Review of the scientific method - Definition of Scientific Theory, Hypothesis and Law. - Review of reasoning in research ? Deduction, Induction, Abduction - Expanding Scientific knowledge and falsification. - The Hypothetic{deductive scheme. • Scientific method and computer science • The variety of research endeavour Review of the scientific method Scientific method • The scientific method helps us do this Observation Theory World Validation Important functions of the scientific method • Validation ? Can a general statement be judged as true, false or probable? • Analysis of hierarchical structure ? Reductionism: Understand complex system by reducing them to the interactions of their parts. ? Emergence: Produce complex, interesting high-level function by combining simple low-level mechanisms in simple ways. • Causality ? Define and explore the connection between actions. Encoding of scientific knowledge or premises • Scientific Hypothesis: - Proposed explanation for an observable phenomena. • Scientific Model: - A physical, conceptual, or computer-based representation of a system. • Scientific Theory: - A comprehensive set of ideas that explains a phenomenon. • Scientific Law: - Describes a phenomena, often mathematically. Theory building should be the goal because • Theories lie at the heart of what it means to do science. ? Production of generalizable knowledge ? Serve to explain and predict ? Concepts, relationships, causal inferences • Theory provides orientation for data collection ? Theoretical perspective allows efficient observation of the world • Theories allow us to compare similar work ? Theories include precise definition for the key terms ? Theories provide a rationale for which phenomena to measure • Theories support analytical generalization ? Provide a deeper understanding of our empirical results ? ...and hence how they apply more generally ? Much more powerful than statistical generalization Reasoning within the scientific method Deduction explained • Deduction is Arguing from the general to the particular • Example All Frenchmen like red wine. Pierre is a Frenchman. =) Pierre likes red wine. • Terminology of the example: All Frenchmen like red wine. premise Pierre is a Frenchman. =) Pierre likes red wine. " " premise conclusion Virtues of deduction • A deduced conclusion is definitely true if ? premises (axioms) are true and ? reasoning carried out correctly. • An example is Pythagoras' Theorem can be deduced from Euclid's axioms. • For deductive logic: discovery process ≡ justification process • However, most mathematical theorems are not discovered solely by the exercise of deductive reasoning. • Use deductive reasoning to justify or falsify an inspiration or an intuitive belief. Limitations of deduction • Deductive reasoning uncovers what is implicit in our premises. • Unfortunately, deductive reasoning cannot bring us knowledge of the world beyond our premises. • It can however bring us awareness about our premises. • But how can we expand our knowledge??? Limitations of deduction • Deductive reasoning uncovers what is implicit in our premises. • Unfortunately, deductive reasoning cannot bring us knowledge of the world beyond our premises. • It can however bring us awareness about our premises. • But how can we expand our knowledge??? Reasoning to potential new knowledge: Induction Induction explained • Induction is Arguing from the particular to the general • Example of inductive inference: The first five eggs in the box are rotten. All six eggs have the same best-before date stamped on them. Infer the sixth egg will be rotten too. • Inductive inference does not guarantee the conclusion is true. An inductive argument • Start with observations or experimental results. • On the basis of these, start framing general principles that take the observations into account. • Illustration of its limitation( enumerative induction) ? Someone from Europe, having seen many swans, all of them white, comes to the conclusion \All swans are white". ? He anticipates the next swan to appear will also be white. ? The generalization is confirmed with every new swan that is seen. ? Then, visiting Australia, the person comes across a black swan and has to think again. An inductive argument • Start with observations or experimental results. • On the basis of these, start framing general principles that take the observations into account. • Illustration of its limitation( enumerative induction) ? Someone from Europe, having seen many swans, all of them white, comes to the conclusion \All swans are white". ? He anticipates the next swan to appear will also be white. ? The generalization is confirmed with every new swan that is seen. ? Then, visiting Australia, the person comes across a black swan and has to think again. The inductive scientific method • Generation of a possible hypothesis ? Gather evidence { if possible eliminate irrelevant factors. (somewhat contradictory as need a hypothesis to define relevancy.) ? Conclusions inferred from the evidence leads to a hypothesis. • Refinement of the hypothesis ? Experiments are devised to test out the hypothesis, based on its predictions. ? If necessary the hypothesis is modified to take into account the results of the later experiments. ? A general theory is framed from the hypothesis and its related experimental data. • Verify or falsify hypothesis ? Use this theory to make predictions. On the basis of these can confirm or disprove the theory. Reasoning to potential new knowledge: Abduction Abduction • Infer a as an explanation of b. • Abduction allows the precondition a to be inferred from the consequence b. • Deduction and abduction differ in the direction in which a rule like \a entails b" is used for inference. • Truth of the assumptions do not guarantee the truth of the conclusion. What is abduction? \...a method of reasoning in which one chooses the hypothesis that would, if true, best explain the rele- vant evidence. Abductive reasoning starts from a set of accepted facts and infers their most likely, or best, expla- nations." Example of abductive reasoning • You notice \The lawn is wet". • If \it rained last night", it would be unsurprising that \the lawn is wet." • By abductive reasoning, the possibility that it rained last night is reasonable. Example of abductive reasoning • You notice \The lawn is wet". • If \it rained last night", it would be unsurprising that \the lawn is wet." • By abductive reasoning, the possibility that it rained last night is reasonable. Note • Abducing rain last night from the observation of the wet lawn can lead to a false conclusion. • Maybe dew, lawn sprinklers, or some other process could have resulted in the wet lawn. Abductive reasoning in science • Abduction selects, among the hypotheses that are being considered, the one that best accounts for the evidence. • Abductive reasoning is closely related to the statistical method of maximum likelihood estimation. • There exist several obvious threats to validity ? Small hypotheses spaces. ? Small amounts of evidence to explain. Both are key challenges to the scientific practice. Challenges of abduction • Creating hypothesis spaces likely to contain the \true" hypothesis ? Approach { Create large hypothesis spaces. • Knowing when more valid hypotheses are missing from your hypothesis space ? Approach { Constantly evaluate your hypothesis space, and expand the hypotheses space whenever the data become extremely unlikely, given any current hypothesis. • Creating good sets of evidence to explain ? Approach { Seek diverse and independent evidence with which to evaluate hypotheses. Reasoning to potential new knowledge: The hypothetic{deductive reasoning scheme Scientific reasoning: Hypothetic{deductive scheme • Science in its discovery mode is not propelled by logic. • Scientific reasoning is a dialogue between the possible and the actual. • There are two thought process which alternate and interact the imaginative ! the critical • Imaginative