Some arguments are probably valid: Syllogistic reasoning as communication Michael Tessler, Noah D. Goodman fmtessler, [email protected] Department of Psychology, Stanford University Abstract Syllogistic reasoning lies at the intriguing intersection of natu- ral and formal reasoning, of language and logic. Syllogisms comprise a formal system of reasoning yet use natural lan- guage quantifiers, and invite natural language conclusions. How can we make sense of the interplay between logic and language? We develop a computational-level theory that con- siders reasoning over concrete situations, constructed proba- bilistically by sampling. The base model can be enriched to consider the pragmatics of natural language arguments. The model predictions are compared with behavioral data from a recent meta-analysis. The flexibility of the model is then ex- plored in a data set of syllogisms using the generalized quan- tifiers most and few. We conclude by relating our model to two extant theories of syllogistic reasoning – Mental Models and Probability Heuristics. Keywords: Reasoning; language; QUD; Bayesian model Figure 1: How will the reasoner interpret the experimenter’s Consider for a moment that your friend tells you: “Every- argument? one in my office has the flu and, you know, some people with this flu are out for weeks.” Do you respond with “Everyone in your office has the flu.” Do you respond with “Pardon me, is no relation between A & C which is true in every situa- there is no inference I can draw from what you just said.” tion in which the premises are true. This is the case with Or do you respond “I hope your officemates are not out for the argument above. Often in these cases, however, people weeks and I hope you don’t get sick either.” are perfectly comfortable drawing some conclusion. A recent The first response – while true – does not go beyond the meta-analysis of syllogistic reasoning showed that over the premises; the second response attempts to go beyond the population, the proper production of no valid conclusion re- premises by strict classical logic, and fails; the final response sponses for invalid arguments ranged from 76% to 12%. For goes beyond the premises, to offer a conclusion which is valid arguments, the accuracy of producing valid conclusions probably useful and probably true. This cartoon illustrates a ranged from 90% to 1% (Khemlani & Johnson-Laird, 2012): critical dimension along which cognitive theories of reason- people do not seem to find drawing deductively valid conclu- ing differ: whether the core and ideal of reasoning is deduc- sions particularly natural. tive validity or probabilistic support. A separate dimension Perhaps because of this divergence between human behav- concerns the extent to which principles of natural language— ior and deductive logic, syllogistic reasoning has been a topic pragmatics and semantics—are necessary for understanding of interest in cognitive psychology for over a hundred years reasoning tasks. In this paper, we explore the idea that the (Storring,¨ 1908), and before that in philosophy, dating back formalism of probabilistic pragmatics can provide insight into to Aristotle. Syllogisms are undoubtedly logical; indeed, the how people reason with syllogisms. syllogistic form was the only formal system of logic for mil- The form of the argument above resembles a syllogism: lennia. At the same time, the arguments use natural language a two-sentence argument used to relate two properties (or quantifiers and invite natural language conclusions; precisely terms: A, C) via a middle term (B); the relations used in syl- pinning down the meaning and use of quantifiers has been an logisms are quantifiers. Fit into a formal syllogistic form, this ongoing area of inquiry since Aristotle (e.g. Horn, 1989). argument would read: Many theories of syllogistic reasoning take deduction as All officemates are out with the flu a given and try to explain reasoning errors as a matter of Some out with the flu are out for weeks noise during cognition. Errors, then, may arise from im- Therefore, some officemates are out for weeks proper use of deductive rules or biased construction of log- ical models. Many other kinds of reasoning, however, can be The full space of syllogistic arguments is derived by shuffling well-explained as probabilistic inference under uncertainty. the ordering of the terms in a sentence (“All A are B” vs. “All Probability theory provides a natural description of a world B are A”) and changing the quantifier (all, some, none, not in which you don’t know exactly how many people are in the all1). Most syllogisms have no valid conclusion, i.e. there that in sentence-form, the last two quantifiers are typically presented 1For distinctiveness, we will refer to the quantifiers as such. Note as “No A are B” and “Some A are not B”, respectively. hallway outside your door, or whether or not the lion is going Here the helper function all-true simply checks that all ele- to charge. We suggest that combining probabilistic reason- ments of a list are true, i.e. that all the As are indeed Bs. ing with natural language semantics and pragmatics is a use- The function map applies the given function —(lambda ...)— to ful approach in that knowledge can describe distributions on each element of the list objects. Similarly we can define some, possible situations, and these distributions can be updated by none, not-all to have their standard meanings. For a first test sentences with new information. In this formalism, deduction of the model, we assume sets are non-empty, i.e. all and none emerges as those arguments which are always true and syllo- cannot be trivially true. gistic reasoning becomes a process of determining that which The key observation to connect these truth-functional is most probable, relevant, and informative. meanings of quantifier expressions to probability distribu- tions over situations is that an expression which assigns a A pragmatic Bayesian reasoner model Boolean value to each situation can be used for probabilis- Our model begins with the intuition that people reason prob- tic conditioning. That is, these quantifier expressions can be abilistically about situations populated by objects with prop- used to update a prior belief distribution over situations into a erties. To represent this type of richly structured model, we posterior belief distribution. For syllogistic reasoning we are must go beyond propositional logic and its probabilistic coun- interested not in the posterior distribution over situations per terpart, Bayesian networks. We instead build our model us- se, but the distribution on true conclusions that these situa- ing the probabilistic programing language Church (Goodman, tions imply. In Church this looks like: Mansinghka, Roy, Bonawitz, & Tenenbaum, 2008), a kind (query of higher-order probabilistic logic in which it is natural to (define objects (list ’o1 ’o2 ... ’on)) ... defineA,B,C... describe distributions over objects and their properties. For ... define all, some, no, not-all . background and details on this form of model representation, (define conclusion (conclusion-prior)) see http://probmods.org. conclusion Situations are composed of n objects: (and (conclusion A C) (premise-one A B) (define objects (list ’o1 ’o2 ... ’on)) (premise-two B C))) (Ellipses indicate omissions for brevity, otherwise models are The first arguments to a query function are a generative specified via runnable Church code2.) Properties A, B, and C model: definitions or the background knowledge with which of these objects are represented as functions from objects to a reasoning agent is endowed. Definitions for which a prior the property value. We assume properties are Boolean, and is stipulated (e.g. conclusion) denote aspects of the world over so property values can be true or false. We assume no a priori which the agent has uncertainty. The second argument, called information about the meaning of the properties and thus they the query expression, is the aspect of the computation about are determined independently: which we are interested; it is what we want to know. The fi- (define A (mem (lambda (x) (flip br)))) nal argument, called the conditioner, is the information with (define B (mem (lambda (x) (flip br)))) which we update our beliefs; it is what we know. We assume (define C (mem (lambda (x) (flip br)))) that the prior distribution over conclusions (and premises, be- Note that the operator mem memoizes these functions, so that low) is uniform. a given object has the same value each time it is exam- ined within a given situation, even though it is initially de- Recursion and pragmatics termined probabilistically (via flip). Previous probabilistic We have suggested viewing syllogistic reasoning as a case models (Oaksford & Chater, 1994) have invoked a principle of communication, and this in turn suggests that reasoning of rarity from the observation that properties are relatively should go beyond the semantics of language, to its pragmat- rare of objects in the world3. For us, this simply means the ics. base rate, br, of properties is small. Following the rational speech-act (RSA) theory (Goodman We interpret quantifiers as truth-functional operators, con- & Stuhlmuller,¨ 2013; Frank & Goodman, 2012), we imag- sistent with standard practice in formal semantics. A quan- ine a reasoner who receives premises from an informative tifier (e.g. all) is a function of two properties (e.g. A and B) speaker. The speaker conveys information about which only which maps to a truth value by consulting the properties of she has access – in RSA, her access was a current state of the objects in the current situation. For instance: the world. By being informative with respect to the world- (define all state, the speaker is able to communicate enriched meanings (lambda(AB) (e.g. scalar implicature – that “some” may also imply “not (all-true (map(lambda (x) (if (A x) (B x) true)) objects)))) all”).