SCIENTIFIC METHODS an online book Richard D. Jarrard Dept. of Geology and Geophysics, University of Utah [email protected] © Richard D. Jarrard 2001 Scientific Methods is an online book about the techniques and processes of science and the experience of being a scientist. This book is written by a scientist for scientists. My hope is that it will be browsed by scientists (including graduate students) and read by undergraduates. Why am I publishing this book online, despite having a willing soft-cover publisher? The main reason is wider availability to readers. A typical science book has a publication run of ~2000 copies, then it goes out of print. Additional factors include educational use and ease of revision. I admit that I would have enjoyed saying that I earned ~25¢/hour by writing this book. Feel free to print a personal copy of this book. Note, however, that this book is copyrighted; it is unethical (see Chapter 10) and illegal to distribute multiple printouts or digital copies or to copy any of these files to other web sites. CONTENTS 1. Introduction 1 Overview 1 Thumbnail History of Scientific Methods 3 Myth of a Scientific Method 12 Scientific Methods 12 SCIENTIFIC TOOLBOX 2. Variables 15 Statistics 15 Errors 16 Precision > Accuracy > Reliability 17 Random and Systematic Errors 18 Representative Sampling 18 Replication and Confirmation 20 Probability 20 Sampling Distribution for One Variable 21 Histograms 22 Normal Distribution 23 Mean & Standard Deviation 23 Normal Distribution Function 24 Weighted Mean 26 95% Confidence Limits on Mean 26 How Many Measurements are Needed? 27 Propagation of Errors 28 Non-Normal Distributions 29 Normality Tests 30 Rejecting Anomalous Data 32 Median, Range, & 95% Confidence Limits 33 Examples 37 3. Induction and Pattern Recognition 42 Types of Explanation 43 Coincidence 45 Correlation 46 Examples 47 Crossplots 50 Plotting Hints 52 Extrapolation and Interpolation 53 Correlation Statistics 55 Nonlinear Relationships 58 Correlation Conclusions 60 Perspectives on Causality 60 Mill's Canons: Five Inductive Methods 64 Method of Agreement 65 Method of Difference 66 Joint Method of Agreement & Difference 67 Method of Concomitant Variations 67 Method of Residues 67 Correlation or Causality? 68 4. Deduction and Logic 71 Logic 72 Deduction vs. Induction 73 Deductive Logic 74 Classification Statements 75 Deductive Aids: Venn Diagrams and Substitution 76 Logically Equivalent Statements 78 Relationships among Statements 80 Syllogisms 82 Categorical Syllogisms 83 Hypothetical Syllogisms 85 Pitfalls: Fallacious Arguments 86 Fallacies Resulting from Problems in a Premise 88 Fallacies Employing Extraneous Other Evidence 90 Faulty Link between Premises & Conclusion 92 Case-dependent Relationship between Parts & Whole 94 5. Experimental Techniques 97 Observational versus Experimental Science 98 Seizing an Opportunity 101 Experimental Equipment 101 Prototypes and Pilot Studies 102 Troubleshooting and Search Procedures 104 Problem: Find a Needle in a Haystack 109 Problem: Search for the Top Quark 110 Tips on Experimental Design and Execution 110 Pitfalls of Experimental Design 116 Control of Variables 117 Problem: the Noisy Widgetometer 120 Computation and Information Handling 121 LIVING SCIENCE 6. The Myth of Objectivity 125 Perception: Case Studies 125 Perception, Memory, and Schemata 131 Postmodernism 135 Pitfalls of Subjectivity 137 Experimental Design 137 Experiment Execution 138 Data Interpretation 140 Publication 140 Pitfall Examples 141 Group Objectivity 143 7. Evidence Evaluation and Scientific Progress 146 Judgment Values 147 Evaluation Aids 151 Confirmation and Refutation of Hypotheses 156 Power of Evidence 157 Hypothesis Modification 159 Paradigm and Scientific Revolution 161 Pitfalls of Evidence Evaluation 164 Hidden Influence of Prior Theory on Evidence Evaluation 164 Incremental Hypotheses and Discoveries 165 'Fight or Flight' Reaction to New Ideas 165 Confusing the Package and Product 166 Pitfall Examples 166 8. Insight 168 Role of Insight in Science 169 Characteristics of Insight 170 Conditions Favoring Insight 171 Obstacles to Insight 173 The Royal Way 175 How Does Insight Work? 176 Alternative Paths to Insight 178 Unexpected Results 178 Transfer from other Disciplines 178 Breakthroughs by Amateurs: the Outsider Perspective 179 From Puzzle Solving . 180 . To Mystical Experience 181 9. The Scientist's World 183 Scientist and Lay Person 183 Science and Society 184 Science and the Arts 187 Science and Pseudoscience 187 Applied and Basic Research 189 Conflict: Applied vs. Basic Research 189 Changing Goals for Applied and Basic Research 191 Resolution: Bridging the Gap 192 Big Science versus Little Science 193 Ego and the Scientific Pecking Order 194 10. The Scientist 197 Scientists' Characteristics 197 Essential Characteristics 197 Common Characteristics 199 Cooperation or Competition? 202 Science Ethics 205 Publication 207 A Scientist's Life: Changing Motivations 210 Process and Product 211 References 214 Name Index 225 Subject Index 228 1 Chapter 1: Introduction Overview Consider the dance of science -- the dance that obsesses us so. It’s said that in viewing the night sky, the present is illusion. The stars are so dis- tant that I see them as they were millions or billions of years ago, when their light rays began the voyage to my eye. It’s said that I am infinitesimally small and transient; the stars will not miss the light my eyes have stolen. They will not notice that they have joined me in the dance. Technique and style are the framework of dance. Techniques of science are generally the easy part; many are deliberately and systematically taught. For example, throughout our many years of schooling we refine skills such as fact gathering and mathematical analysis. We learn other scien- tific techniques -- such as statistics, deductive logic, and inductive logic -- in classes that lack the perspective of scientists’ needs. Some techniques are more intangible: critical thinking and analysis, pattern recognition, and troubleshooting of experimental technique. Scientists are not merely technicians; an equally crucial part of the dance is style: how do scientists combine rationality and insight, or skepticism and inno- vation; how do scientists interact, and what motivates their obsession? These skills seldom are taught explicitly. Instead, they are implicit in the scientific apprenticeship, an excellent but often incomplete educational process. Who of us has mastered all of the techniques of science? I certainly have not; researching and writing this book have shown me that. Of course, when I recognize that an aspect of my scientific methods is deficient, I am enough of a professional to seek a remedy. More often, I, like Konrad Lorenz’s [1962] water-shrew, am not even aware of what is missing: The water shrew dashes through its territory at incredible speed, by following the familiar path. “To them, the shortest line is always the accustomed path.” Lorenz de- cided to explore the extent of this habit by removing a stone from a water-shrew’s path. When it came racing along, it jumped over the nonexistent stone. It paused in bafflement, backed up and jumped ‘over’ it again, then finally reconnoitered the anomaly. How often do we leap missing stones? * * * Consider the science of science. Let’s turn our gaze on our lives, looking beyond the surface interplay of experiment and theory. What are we scientists doing, and what tools are we using? We’ve left such introspection to philosophers, but their goals differ from ours. They deal in abstracts: what rules do scientists follow, and how should the process of science change? We sci- entists generally prefer the more pragmatic approach of just doing, not talking about doing. Are we too busy, or too confident in our established routines, to analyze what we are doing? Why are virtu- ally all of the books on scientific methods written by philosophers of science, rather than by scien- tists? 2 “It is inevitable that, in seeking for its greatest unification, science will make itself an object of scientific investigation.” [Morris, 1938] * * * This book was originally intended as ‘How to do science’, or ‘How to be a scientist’, providing guidance for the new scientist, as well as some reminders and tips for experienced researchers. Such a book does not need to be written by the most expert or most famous scientist, but by one who likes to see the rules of play laid out concisely. It does need to be written by a working scien- tist, not by a philosopher of science. The first half of the book, called ‘Scientist’s Toolbox’, retains this original focus on what Jerome Brumer called the structure of science -- its methodologies and logic. This objective is still present in the second half of the book, ‘Living Science’. In researching that section, however, I was fascinated by the perspectives of fellow scientists on ‘What it is like to be a scientist.’ Encountering their insights into the humanity of science, I found resonance with my already intense enjoyment of the process of science. Gaither and Cavazon-Gaither [2000] provide many additional scientific quotations on the experience of science. * * * Consider the process of science. Knowledge is the goal of science: basic research seeks reliable knowledge, and applied research seeks useful knowledge. But if knowledge were our primary goal as scientists, we would spend much of our available time in reading the literature rather than in slowly gathering new data. Science is not static knowledge; it