Fallacies in academic writing Chad Nilep There are many possible sources of fallacy – an idea that is mistakenly thought to be true, even though it may be untrue – in academic writing. The phrase logical fallacy often refers to a formal fallacy in a logical argument; that is, an invalid argument that is mistakenly thought to be valid. Often people talk about various kinds of rhetorical fallacies, writing or speech that appears to be based on an argument but is not logically entailed by premises. Research writing can also be prone to statistical fallacy: mistaken use of statistics, or conclusions based on misunder- standing statistics. Below I discuss a few common fallacies in academic writing.

Logical (formal) fallacies

Affirming the consequent. Implication is a valid form of logical argument. The truth of one idea (call it an antecedent, meaning it comes first) may imply a consequence. If the antecedent is true then the consequent (the thing that happens after or sometimes because of the antecedent) must be true. For example, if a person is murdered that person is dead. This is necessarily true because it is part of the meaning of murder. If Taro was murdered, we can conclude that he is dead. That is a valid argument from implication. But the opposite is not a valid argument: the truth of the consequent does not imply the truth of the antecedent. If Taro is dead, we cannot conclude that he was murdered; he might have died from a disease or an accident. The consequent does not imply the truth of the antecedent.

Denying the antecedent. On the other hand, if an antecedent implies a consequent and that consequent is not true, then the antecedent must not be true. Think about the murder example again: if Taro was murdered, he is dead. If he is not dead, then we know that he was not mur- dered. But again the opposite is not a valid argument: a false antecedent does not imply a false consequent. Even if we know that Taro was not murdered, we don’t know whether he is dead. He may be alive, but he may have died naturally or accidentally.

Syllogistic fallacies We discussed syllogisms in class. In a syllogism, two premises add up to a conclusion. If the syl- logism is valid and the premises are true, then the conclusion must be true. Be careful, though: there are several fallacies that can make a syllogism invalid. VALID SYLLOGISMS All P is M. Some P is M. All P is M. All M is Q. All M is Q. No M is Q. (All M is not-Q) Therefore, all P is Q. Therefore, some P is Q. Therefore P is not Q.

Undistributed middle. All syllogisms must have a middle term that is “distributed” – true for all – in at least one premise. If the middle term is not distributed, the syllogism is not valid. NOT VALID SYLLOGISMS All P is M. Some P is M. All P is M. All Q is M. Some M is Q. Some M is not Q Therefore, all P is Q. Therefore, some P is Q. Therefore P is not Q.

We’ve seen the first non-valid argument in class: “All rabbits run fast; Usain Bolt runs fast. Therefore, Usain Bolt is a rabbit.” I also mentioned an example of the second pattern: “Some students in this class are women; some women are over 60 years old. Therefore, some students in this class are over 60 years old.” Here is an example of the third pattern: “All students in this class are graduate students. Some graduate students do not study logic. Therefore, students in this class do not study logic.” None of the conclusions are proved true since the syllogisms are invalid.

Four terms. A valid syllogism must have three terms, like the ones labeled P, M, and Q above. If the conclusion or one of the premises introduces a fourth term, it is not valid. Consider this argument: “Socrates is a human. All humans are mortal. Therefore, Socrates is Japanese.” The wrongness of that syllogism is obvious. But sometimes it can be hard to spot this fallacy when the same words express two different ideas. Consider the example below. Warm beer is better than nothing. Nothing is better than true love. Therefore, warm beer is better than true love. Although this looks like a valid syllogism of the kind we have seen before, the conclusion is not true. What went wrong? Notice that “better than nothing” (無いよりまし) in the first premise does not mean the same thing as “nothing is better” (最高) in the second premise. Those two different meanings add up to a fallacy of four terms.

Affirmative conclusion from negative premises or negative conclusion from affirmative premises. Two negative premises cannot entail an affirmative conclusion. Likewise, it is not valid to add negative meaning to an affirmative syllogism.

NOT VALID SYLLOGISMS No P is M. Some P is M. No M is Q. All M is Q Therefore, all P is Q. Therefore some P is not Q.

As an example of the first pattern consider this invalid syllogism: “No rabbits are horses; no horses are birds. Therefore, all rabbits are birds.” It is not valid to draw an affirmative conclusion from negative premises. Likewise, negative conclusions cannot be drawn from premises that contain no negative statement. Maybe you can think of examples that seem to be true; for example: “Some people are doctors; all doctors are university graduates. Therefore, some people are not university graduates”. But the truth of that conclusion is only accidental. It is not a valid conclusion from the syllogism. Consider this example, which uses the same logic: “Some army officers are generals; all generals are soldiers. Therefore, some army officers are not soldiers.” That’s not valid and it’s not true. Remember: a valid syllogism allows us to deduce that the conclusion is true, but truth (matching the world) is not the same as validity (entailment from argument).

Rhetorical (informal) fallacies

Writers sometimes argue, meaning they try to convince the reader that what they say is right, without a logical argument in the sense of “a set of premises that imply the truth of, or support the likelihood of their conclusion”. Here are some examples of rhetorical fallacy. Appeal to authority. Writers may imply, or readers may assume, that something is true because a famous, smart, or respected person said it. There is, however, no necessary link between the authority of a speaker and the truth of what s/he says. Similarly a bandwagon argument suggests that if many people believe the same thing, that belief must be true. Again, there is no necessary link between popularity or fame and truth. Ad hominem. When two people or groups disagree, an ad hominem argument may appear: arguing that the person saying something is bad, so what s/he says must not be true. Just as the goodness of a speaker does not entail the truth of what s/he says, the badness of a speaker does not entail falseness. Straw man argument consists of misrepresenting either a conclusion or a logical argument in order to counter it. A scholar may reject another scholar’s conclusion by showing either that an argument was invalid or its premises are not true. If instead the scholar creates an invalid argument that reaches the same conclusion in order to reject that conclusion, he has created a straw man argument. Begging the question or circular reasoning involve making the conclusion one of the premises. Consider this argument: “Fundamentals of academic writing is a popular class. Most students like popular classes. Therefore, most students like fundamentals of academic writing.” The conclusion and the first premise mean basically the same thing, even though the words are slightly different. Therefore, it is not actually an argument; it just states what the teacher hopes is true. (Note that in everyday English the phrase beg the question is sometimes used to mean “cause to ask a question”. That is not related to this fallacy.) Appeal to ignorance refers to the assumption that something not proved to be true must be false, or that something not proved to be false must be true. Neither of these is a valid assumption; something not proved might be true or false, but it’s currently unknown. For example, there is no clear evidence that life exists on other planets, but that is not proof that no such life exists. It might not exist, or it might exist but be unknown to humans. (Note, however, that some kinds of absence can be used to support some kinds of inferences. For example if a doctor tests me for cancer cells and doesn’t find any then it is likely, though not certain, that I do not have cancer.)

And although it’s not exactly a fallacy I will also share a warning from my former professor Lise Menn: “Beware Procrustes bearing Occam’s razor”. Occam’s razor is the valid suggestion (not a fallacy) that when two theories can account for the same data, it is best to assume the simpler of the two theories. But simple theories are not always correct. Procrustes is a character from a Greek myth who made everyone fit into the same bed by either stretching them or cutting off their legs. Scholars sometimes commit an error by “cutting off” some data in order to make it fit with a simple theory. For example, there is a simple theory that explains the evolution of large human brains, but this theory cannot explain variation in the size of other human-like species’ brains. In this case a more complex theory that explains all the data might be true, even though it fails the suggestion of Occam’s razor. Statistical fallacies

Readers and writers sometimes get confused about the meaning of statistical analyses. This can lead scholars to claim things that are not supported by the analysis. Here are some statistical fallacies. Correlation is not causation. Correlation means that two values vary together. Causation means that a change in one variable causes a change in another. For example, an increase in demand for sugar can cause the price of sugar to increase. A common error is to assume that because values are correlated (for example, when one increases the other also increases) one must cause the other. Sometimes correlation is spurious or accidental. For example, rates of cancer and rates of autism have both increased over the past twenty years, but it is unlikely that these changes are related. Sometimes a third factor – called a hidden variable or a confounding variable – correlates with both variables. For example, drowning deaths are strongly correlated with ice cream sales; this is not because ice cream causes drowning but because both ice cream and swimming are most popular in summer. This is an example of a common informal logic fallacy called cum hoc ergo propter hoc “with this therefore because of this”. Correlation does not imply causation. On the other hand, it often suggests that a phenomenon might be worth inves- tigating further to see if there is some hidden meaning behind the correlation. Significance versus “significance”. In everyday usage the word significant means “important, noteworthy, or meaningful”. But in statistics significant means “unlikely to be a random error”. The value p<0.01 calculated by a t-test essentially means that if our numbers were completely random, the difference we found would be seen less than one time in 100. People sometimes misunderstand, thinking that a statistically significant result must be important or meaningful. That is not necessarily the case. For example, say we give an intelligence test to 10 men and 10 women and find that the average IQ of the men is 105 while the average of the women is 107. This difference is not significant (p=0.15). But say we give the same test to 100 men and 100 women and find the same averages: 105 for men, 107 for women. This result is statistically significant (p<0.01), even though the result is no more or less meaningful. This is because a t-test is sensitive to small differences when sample sizes are large. Statistical significance allows us to reject a null hypothesis – it tells us that there actually is a difference – but it cannot indicate how important or meaningful that difference is. Base rate fallacy. When doing statistical tests, people often ignore the “base rate” or underlying probability and focus only on the probability of the test, leading them to under- or over-estimate their findings. For example, imagine that there are 200 werewolves in Nagoya. Luckily, I have a machine that can identify werewolves. The machine is 99% accurate, meaning it will misidentify a werewolf as human only 1% of the time, and misidentify a human as werewolf only 1% of the time. I point the machine at my neighbor and it says she is a werewolf. How confident can I be that she really is a werewolf? If I ignore the base rate, I may think that there is a 99% chance that she is a werewolf. But in fact, since there are about two million humans and only two hundred werewolves in Nagoya, the machine will wrongly identify about 20,000 humans as werewolves while accurately identifying 198 werewolves. There is actually less than 1% chance that the neighbor identified as a werewolf really is one. Biased samples. It is usually impossible or at least impractical to test every person or thing relevant to our research. Therefore, we usually select a sample and then generalize the results to a larger population. Such generalization is only valid, though, if the sample is representative of the population. A random sample, in which participants are randomly selected from the whole population and subjects are randomly assigned to treatment or control groups when doing an experiment, can help ensure representative, unbiased samples. Using convenience samples, participants who are not randomly selected, can make the results less generalizable. For example, since many psychology studies are done by university professors, they often recruit university students as subjects. But since university students are not representative of the whole world (among other differences, they tend to be younger and better educated than average) the results cannot be generalized to all people. Similarly, medical research carried out in hospitals may choose subjects from among hospital patients and therefore ignore the large portion of the population who are not sick. Another form of sample bias is called self-selection or non-response bias. For example, if a sociologist sends a questionnaire to a random sample of people, some of those subjects might not answer. The sample becomes biased if the people who respond and the people who do not respond differ in some way. Similarly, if a family restaurant asks customers to rate their service it is likely that only people who are very happy or very unhappy with the restaurant will answer, so that the survey results will not represent the whole population of customers. Extrapolation fallacies. Research in many fields examines change over time (or across space, or over other variation). A common use of such results is to extrapolate – to predict that the rate of change observed in the past will continue into the future. This can be useful, but it can lead to wrong predictions. For example, during the 2012 United States presidential campaign three opinion polls in September found that candidate Mitt Romney was becoming more popular each week, while President Barrack Obama was becoming less popular. News media predicted that this trend would continue and that Romney would win the election in November. In fact though, the trend changed in October: Obama became more popular, and he eventually won the election. “Garbage-in-garbage-out” is a phrase commonly used in computer science and information technology; it can also apply to statistics. It means that the value of an analysis (what comes out) is only as good as the value of the data that goes into it. Readers and writers often assume that statistical analyses are reliable and valid without questioning the quality of the data that was analyzed or the relevance of the analysis to the research question. If the data are not accurate or appropriate, statistical methods will not make them any more reliable. Finally, a word about charts. When statistical data are presented in chart form, even accurate numbers can give a false impression. Consider these two charts. Both of them show the exact same data: an increase from 35.0% to 39.5%. In the chart on the left that change looks very big, while on the right it looks small. That apparent difference is due to the scale of the charts, from 32-40 on the right or from 0-100 on the left. Choosing one of these charts rather than the other can give a very different impression of the same descriptive statistics. Be careful to present your data accurately and fairly.