sign in sign up
<<
Home , Inductive reasoning, Prediction, Statistical inference

Applied Lecture 4 part 1 –

Marcin Szczuka

Institute of Informatics, The University of Warsaw

Monographic lecture, Spring semester 2018/2019

Marcin Szczuka (MIMUW) Applied Logic 2019 1 / 30 Lecture plan

1 Introduction

2 Incomplete inductive reasoning Types of inductive reasoning

3 Towards Inductive Logic

Marcin Szczuka (MIMUW) Applied Logic 2019 2 / 30 Marcin Szczuka (MIMUW) Applied Logic 2019 3 / 30 Deduction vs. induction

All of the reasoning systems () we have seen so far were based on deduction. The worked with use of certain rules that allowed derivation of consequences from axioms. In particular, deductive systems are closed with respect to creation of new notions and inference of true consequences. Reasoning based on pure deduction is very rare in real life. In most cases they are limited to precise, mathematical models of (theoretical , , informatics, ..). A great example of work based on rigorous is the monumental work by Euclid of Alexandria, the Elements. In real-life scenarios it is usually hard to preserve the strict rules of deduction and the inferred conclusions are not always absolutely true. Therefore, in complement to deductive reasoning in real-life situation we frequently use also induction and/or abduction.

Marcin Szczuka (MIMUW) Applied Logic 2019 4 / 30 Inductive reasoning

In simplest terms, inductive reasoning can be seen as inference

from particular to general or from examples to rules.

Inductive reasoning is by imprecise. Inductive reasoning is built on human capability of finding patterns and rules on the basis of of a finite (and possibly incomplete and imperfect) . For example, on the basis of obserwation any “reasonable” thinker will consider the following sentence to be empirically proven. All crows are black, but not all cats are black.

Marcin Szczuka (MIMUW) Applied Logic 2019 5 / 30 Nuances of induction

The inference on the basis of empirical observation is as old as scientific reasoning itself. However, due to the lack of formalised approach, up to the end of middle ages the Aristotelean deductive reasoning was considered the only “true” method for proving. While considered briefly the inductive method, but limited only to the primitive complete enumerative induction. Nowadays the reasoning by induction is much more precise and regularised. First important questions regarding and practicality of simplified, enumerative approach to induction were posed by (1561-1626). Bacon proposed more practical eliminative induction (induction by elimination). The principle of eliminative induction was formulated more precisely by American philosopher John Stuarta Mill in A System of Logic, Ratiocinative and Inductive (1843) in the form of five so called Mill’s methods (cannons).

Marcin Szczuka (MIMUW) Applied Logic 2019 6 / 30 Nuances of induction

After Bacon, eliminative induction principle was disputed by (1748). pitched in too. Hume, in his philosophical works about cognition and , proposed a novel view on induction and provided a constructive critical Hume’s propositions became a cornerstone of the modern understanding of inductive reasoning. Hume divides all reasoning into demonstrative, by which he deductive, and probabilistic, by which he means the generalization of . Today the idea that our of the world is not complete and precise is commonplace. However, in Hume’s era it sounded like a shocking , especially in light of recently formulated foundations of Newtonian physics. Contemporary understanding of inductive diverted from the Hume’s and Kant’s ideas towards inductive logic. Nowadays, instead of answering the question “what justifies truthfulness of the ?” the researchers concentrate on the question “why a given statement is possible/probable?”. A prominent representative of this kind of approach is (amongst others) . Marcin Szczuka (MIMUW) Applied Logic 2019 7 / 30 Types of inductive reasoning

Complete induction Complete induction (complete enumerative induction, exhaustive induction) is a reasoning method that establishes truthfulness of a rule (proposition) by checking all possible cases when the rule applies. Complete induction is in a certain, deductive method. It eliminates possible by exhaustive enumeration of all positive cases. In most non-trivial situations the complete induction method is highly inefficient. A special case on complete induction is the commonly used to prove mathematical theorems, especially in discrete domains. Somewhat against its , mathematical induction is a deductive technique.

Marcin Szczuka (MIMUW) Applied Logic 2019 8 / 30 Eliminative induction

“...when you have eliminated all which is impossible, then whatever remains, however improbable, must be the .”

Sherlock Holmes

ŹAfter: Arthur Conan Doyle, The Blanched Soldier

Marcin Szczuka (MIMUW) Applied Logic 2019 9 / 30 Types of inductive reasoning

Eliminative induction The simplest eliminative induction scheme (Bacon) is based on building a list of mutually exclusive hypotheses and then eliminating all but one of them through empiric . J.S. Mill extended the eliminative induction principle by setting five rules for elimination of hypotheses (Mills methods). Mill’s methods allow for (partial) formalisation of inference by establishing relationships of the type “ Cause A yields result a” on the basis of series of . For example, the direct method of agreement (first method) facilitates the following reasoning: Situation 1: We record appearance of causes A, B, C and results a, b, c. Situation 2: We record appearance of causes A, D, E and results a, d, e. Conclusion: We eliminate non-repeating (not directly agreeing) observations and we get “Cause A yields result a”.

Marcin Szczuka (MIMUW) Applied Logic 2019 10 / 30 Types of inductive reasoning

Incomplete induction Incomplete induction (incomplete enumerative induction) is the method that establishes a general rule (proposition) on the basis of a finite number of statements (observations) that confirm the rule. We from sample about general regularities. Incompleteness of this reasoning scheme is a manifestation of the nature of reality that we attempt to describe. In real life we almost never are able to observe sufficiently many (or all) possible situations. Incompleteness also means, that previously constructed may need to be modified (completed) as new observations emerge. For example, Einstein’s relativity extends Newtonian mechanics. Incomplete induction is one of the most basic tools for all experimental . Many disciplines have developed frameworks for dealing with introduced through use of incomplete induction, for example error calculus, sta

Marcin Szczuka (MIMUW) Applied Logic 2019 11 / 30 Lecture plan

1 Introduction

2 Incomplete inductive reasoning Types of inductive reasoning

3 Towards Inductive Logic

Marcin Szczuka (MIMUW) Applied Logic 2019 12 / 30 The issue of induction

The issue of incomplete inductive reasoning has been considered by researchers for centuries. The discussion about validity and necessity of its use in describing the universe can be traced back to (3-2 century B.C.). Throughout the ages some of the most illustrious minds, including Bacon, Cartesius, Kant, Newton, Mill, Hume and others, addressed the issue. In contemporary discourse on induction as a to discovery important additions were made by eminent philosophers of sciece, including , Wesley C. Salmon and David Miller.

Construction of a logical system based on incomplete inductive reasoning poses a challenge. On the one hand, such logic should extend deductive systems by addition of a mechanism for deriving conclusion that may not be absolutely true. On the other hand we are not willing to part with the essential property of deductive systems: True guarantee true conclusions.

Marcin Szczuka (MIMUW) Applied Logic 2019 13 / 30 Criterion of Adequacy

In order for inductive logic to be considered useful we usually expect that it has a mechanism for establishing a level of support for conclusions expressed in it. This mechanism measures the degree of influence of premises’ truthfulness on validity of the conclusion. We expect that the measure used by this mechanism satisfies the Criterion of Adequacy (CoA). CoA - Criterion of Adequacy As accumulates, the degree to which the collection of true evidence statements comes to support a , as measured by the logic, should tend to indicate that false hypotheses are probably false and that true hypotheses are probably true.

Marcin Szczuka (MIMUW) Applied Logic 2019 14 / 30 Peculiarities of induction

To be able to use inductive reasoning principles (inductive logic) properly it is necessary to eliminate the possibility of creating paradoxical results or sophismata. Inductive proof of immortality Fact 1 – n Many times (n  1) I heard that somebody died. Fact n + 1 Every time I heard that somebody died – it was not me. conclusion There are no observations supporting the fact of me dying. Hence, I am immortal.

Obviously, this reasoning is a . It does not take into account any negative as well as considers only a part of positive information that by no “decent” measure can be considered complete. However, in practical applications of inductive reasoning it is always prudent to introduce safeguards against nonsense.

Marcin Szczuka (MIMUW) Applied Logic 2019 15 / 30 Lecture plan

1 Introduction

2 Incomplete inductive reasoning Types of inductive reasoning

3 Towards Inductive Logic

Marcin Szczuka (MIMUW) Applied Logic 2019 16 / 30 Types of inductive reasoning

In everyday practice of using inductive inference we apply some standard schemes (methods). Among them are:

1 Inductive generalisation.

2 Statistical .

3 Simple/direct induction.

4 from .

5 .

6 . Etiology. NOTE: can be considered as a (very) special case of simple/direct induction.

Marcin Szczuka (MIMUW) Applied Logic 2019 17 / 30 Inductive generalisation

Inductive generalisation is a method that proceeds from a about a sample to a conclusion about the whole population. Rule Premise: In sample p taken from population P proportion q of cases meets the condition A. Conclusion: The proportion q of the population P meets the condition A.

Note, that at the do not concern ourselves with the size and representativeness of the sample or the value of p. In real-life scenarios the influence of these parameters has to be carefully checked in order to obtain valid outcome.

Marcin Szczuka (MIMUW) Applied Logic 2019 18 / 30 Statistical syllogism

Syllogism is an inference technique that uses two premises, which share a common element, to produce a conclusion that consist of two elements that appear in exactly one of premises. A statistical syllogism proceeds from a premise about entire population to a conclusion about an individual. Rule Premises: – In population P proportion q of cases meet the condition A. – NEW case s is in P . Conclusion: There is a which corresponds to q that s meets A.

Reasoning with use of statistical syllogism is prone to of type , typical for all .

Marcin Szczuka (MIMUW) Applied Logic 2019 19 / 30 Fallacia dicto simpliciter

Errors in reasoning (fallacia) of secundum quid type appear when we use syllogisms in an improper way. For example, a type of syllogism introduced by Aristotle: All men are mortal (major premise) and Socrates is a man (minor premise) hence we may validly conclude that Socrates is mortal. In case of statistical syllogism we may encounter two sub-types of the fallacy.

1 – Fallacia a dicto simpliciter ad dictum secundum quid – inference of particular conclusion from general rule while ignoring important limitation, e.g., “If there are so many lazy students then there must be some lazy students in the room right now”.

2 Reverse accident – Fallacia a dicto secundum quid ad dictum simpliciter – inference of general statement from particular one by omission of important specifying (narrowing) condition, e.g., “If it is allowed to kill in self-defense then killing is OK”.

Marcin Szczuka (MIMUW) Applied Logic 2019 20 / 30 Simple/direct induction

Direct (simple) induction proceeds from a premise about a group of examples (part of population) to a conclusion about another, previously unseen individual. Rule Premises: – In population P proportion q of known instances meets condition A. – NEW case s is in P . Conclusion: There is a probability which corresponds to q that s meets A.

In this particular example the rule for simple induction is a composition of rules for generalisation and statistical syllogism. The conclusion from generalisation becomes the first premise in statistical syllogism.

Marcin Szczuka (MIMUW) Applied Logic 2019 21 / 30 Argument from analogy

Similarity in some aspects determine similarity in other aspects. Reguła Premises: – Cases (objects) s and t agree on conditions A, B, C. – Case (object) s meets condition D. Conclusion: It is very likely that t meets D.

Argument from analogy is frequently used in reasoning as well as scientific, legal and philosophical. A limited and strictly regulated version of this reasoning method is in the basis of a branch of AI known as Case Based Reasoning (CBR).

Marcin Szczuka (MIMUW) Applied Logic 2019 22 / 30 Prediction

A prediction draws a conclusion about a new () case by observation of collected (in the past) sample. Rule Premise: I dotychczas zaobserwowanej population P seen so far the proportion q of cases meets condition A. Conclusion: Newly observed s meets A with probability proportional to q.

Prediction is one of most frequently used methods of inductive reasoning. When regularised and formalised it is a basis of major fields of CS/AI such as Machine and Knowledge Discovery in Databases (KDD).

Marcin Szczuka (MIMUW) Applied Logic 2019 23 / 30 Etiology – causality

Etiology (α´ιτιoλoγ´ια) is a branch of dealing with causality (investigating causes) of phenomena, processes or , especially in domains such as criminology or disease control. In view of inductive reasoning, especially the part that is used in computer science, investigation of is frequently reduced to: Finding causality from In the simplest case, let us consider two facts (two variables) X and Y . Usually, we assume that X and Y are time-dependent. We check, using available data, which of relations X → Y or Y → X treated as a hypothesis (potential conclusion) has more supporting evidence in data. Particular methods and diffre in the way they establish support.

Marcin Szczuka (MIMUW) Applied Logic 2019 24 / 30 Lecture plan

1 Introduction

2 Incomplete inductive reasoning Types of inductive reasoning

3 Towards Inductive Logic

Marcin Szczuka (MIMUW) Applied Logic 2019 25 / 30 Expectations for inductive logic

From a (quasi-)formal system that we dare to call inductive logic we expect:

1 Fulfillment of the Criterion of Adequacy (CoA).

2 Ensuring, that the degree of confidence in the inferred conclusion is no greater than the confidence of the premises and inference rules.

3 Ability to clearly discern between proper conclusions (hypotheses) and nonsensical ones (vide: proof of immortality). Additional expectation is the intuitive interpretation. However, is not always helpful, as demonstrated in the next slide.

Marcin Szczuka (MIMUW) Applied Logic 2019 26 / 30 Monty Hall Paradox

Called that to commemorate the host of quiz show “Let’s Make a Deal” running for many years on TV in the US. It’s not really a paradox. It is rather a demonstration that our intuitive understanding of “statistical/probabilistic” rules is frequently shallow and error-prone. Monty Hall Paradox The Player faces three doors. Behind each door is either a prize (e.g. new car) or one of two goats. The Host asks the Player where (behind which door) is the prize. Then, the Host (who knows where the prize is) opens one of the doors not chosen by the Player. Behind that door is the goat. Now, the Host asks the Player if he wants to change the initial door choice. What should the Player decide in order to maximise his chance of winning the prize. Should he stick to the first choice or make switch?

The answer is so counter-intuitive that even Paul Erdős did not believe in it until in 1995 he was shown a proof with use of a decision tree and computer .

Marcin Szczuka (MIMUW) Applied Logic 2019 27 / 30 Inductive = statistical?

Frequently, practical inductive reasoning systems are based on elements of probabilistic and/or . If all proper precautions are taken, this is not a bad approach. Historically, probabilistic approach was one of the first properly formalised and practically used. One of the most frequently used is Bayesian (probabilistic) reasoning. This approach is sometimes – a liitle bit excessively – called Bayesian LOGic (BLOG). If we identify support, or plausibility measures with empirical probability we may utilise all of the very nice and powerful “machinery” of , hence obtaining a proper formal reasoning scheme. However, when using probabilistic interpretations in inductive reasoning one has to be cautious, as they tend to diverge from the intuitive understanding of “natural” induction.

Marcin Szczuka (MIMUW) Applied Logic 2019 28 / 30 Uncertain reasoning

As mentioned earlier, most models of inductive inference represent incomplete uncertain reasoning. In majority of cases they are also non-monotonic, i.e., with appearance of new evidence (new observations) the conclusions drawn inductively before may be eliminated (contradicted). Reasoning in the presence of uncertainty is widely recognised and investigated in many branches of scientific investigations. Some significant approaches include: Plausibility relations – Relacje wiarygodności; Dempster-Shafer functions – Funkcje przekonań Dempstera-Shafera; Qualitative probability relations – Jakościowe relacje prawdopodobieństwa; Probability functions – Funkcje probabilistyczne; Possibility functions in Fuzzy Logic – Funkcje possybilistyczne (rozmyte); functions – Funkcje rankujące (sic!).

Marcin Szczuka (MIMUW) Applied Logic 2019 29 / 30 Uncertain reasoning

Figure below present some approaches to uncertain reasoning. Arrows indicate “strength” of approach – from more to less general.

Qualitative probability relations

Plausibility Dempster-Shafer Probabilistic relations belief functions functions

Possibilitic functions (fuzzy)

Ranking functions

Marcin Szczuka (MIMUW) Applied Logic 2019 30 / 30