Topic #5: Probability

Informally, probable is one of several words applied to uncertain events or knowledge, being closely related in meaning to likely, risky, hazardous, and doubtful. Chance, odds, and bet are other words expressing similar notions. Just as the theory of mechanics assigns precise definitions to such everyday terms as work and force, the theory of probability attempts to quantify the notion of probable. Probability always lies between 0 and 1. If probability is equal to 1 then that event is certain to happen and if the probability is 0 then that event will never occur. Concepts

There is essentially one set of mathematical rules for manipulating probability; these rules are listed under "Formalization of probability" below. (There are other rules for quantifying uncertainty, such as the Dempster-Shafer theory and possibility theory, but those are essentially different and not compatible with the laws of probability as they are usually understood.) However, there is ongoing debate over what, exactly, the rules apply to; this is the topic of probability interpretations.

The general idea of probability is often divided into two related concepts:

Aleatory probability, which represents the likelihood of future events whose occurrence is governed by some random physical phenomenon. This concept can be further divided into physical phenomena that are predictable, in principle, with sufficient information (see Determinism), and phenomena which are essentially unpredictable. Examples of the first kind include tossing dice or spinning a roulette wheel; an example of the second kind is radioactive decay. Epistemic probability, which represents one's uncertainty about propositions when one lacks complete knowledge of causative circumstances. Such propositions may be about past or future events, but need not be. Some examples of epistemic probability are to assign a probability to the proposition that a proposed law of physics is true, and to determine how "probable" it is that a suspect committed a crime, based on the evidence presented.

It is an open question whether aleatory probability is reducible to epistemic probability based on our inability to precisely predict every force that might affect the roll of a die, or whether such uncertainties exist in the nature of reality itself, particularly in quantum phenomena governed by Heisenberg's uncertainty principle. Although the same mathematical rules apply regardless of which interpretation is chosen, the choice has major implications for the way in which probability is used to model the real world. Formalization of probability

Like other theories, the theory of probability is a representation of probabilistic concepts in formal terms-that is, in terms that can be considered separately from their meaning. These formal terms are manipulated by the rules of mathematics and logic, and any results are then interpreted or translated back into the problem domain. There have been at least two successful attempts to formalize probability, namely the Kolmogorov formulation and the Cox formulation. In Kolmogorov's formulation, sets are interpreted as events and probability itself as a measure on a class of sets. In Cox's formulation, probability is taken as a primitive (that is, not further analyzed) and the emphasis is on constructing a consistent assignment of probability values to propositions. In both cases, the laws of probability are the same, except for technical details;

1. a probability is a number between 0 and 1; 2. the probability of an event or proposition and its complement must add up to 1; and 3. the joint probability of two events or propositions is the product of the probability of one of them and the probability of the second, conditional on the first.

The reader will find an exposition of the Kolmogorov formulation in the probability theory article, and of the Cox formulation in the Cox's theorem article. See also the article on probability axioms.

The probability of an event is generally represented as a real number between 0 and 1, inclusive. An impossible event has a probability of exactly 0, and a certain event has a probability of 1, but the converses are not always true: probability 0 events are not always impossible, nor probability 1 events certain. The rather subtle distinction between "certain" and "probability 1" is treated at greater length in the article on "almost surely".

Most probabilities that occur in practice are numbers between 0 and 1, indicating the event's position on the continuum between impossibility and certainty. The closer an event's probability is to 1, the more likely it is to occur.

For example, if two mutually exclusive events are assumed equally probable, such as a flipped or spun coin landing heads-up or tails-up, we can express the probability of each event as "1 in 2", or, equivalently, "50%" or "1/2".

Probabilities are equivalently expressed as odds, which is the ratio of the probability of one event to the probability of all other events. The odds of heads-up, for the tossed/spun coin, are (1/2)/(1 - 1/2), which is equal to 1/1. This is expressed as "1 to 1 odds" and often written "1:1".

Odds a:b for some event are equivalent to probability a/(a+b). For example, 1:1 odds are equivalent to probability 1/2, and 3:2 odds are equivalent to probability 3/5.

There remains the question of exactly what can be assigned probability, and how the numbers so assigned can be used; this is the question of probability interpretations. There are some who claim that probability can be assigned to any kind of an uncertain logical proposition; this is the Bayesian interpretation. There are others who argue that probability is properly applied only to random events as outcomes of some specified random experiment, for example sampling from a population; this is the frequentist interpretation. There are several other interpretations which are variations on one or the other of those, or which have less acceptance at present. Distributions

A probability distribution is a function that assigns probabilities to events or propositions. For any set of events or propositions there are many ways to assign probabilities, so the choice of one distribution or another is equivalent to making different assumptions about the events or propositions in question.

There are several equivalent ways to specify a probability distribution. Perhaps the most common is to specify a probability density function. Then the probability of an event or proposition is obtained by integrating the density function. The distribution function may also be specified directly. In one dimension, the distribution function is called the cumulative distribution function. Probability distributions can also be specified via moments or the characteristic function, or in still other ways. A distribution is called a discrete distribution if it is defined on a countable, discrete set, such as a subset of the integers. A distribution is called a continuous distribution if it has a continuous distribution function, such as a polynomial or exponential function. Most distributions of practical importance are either discrete or continuous, but there are examples of distributions which are neither.

Important discrete distributions include the discrete uniform distribution, the Poisson distribution, the binomial distribution, the negative binomial distribution, and the Maxwell-Boltzmann distribution.

Important continuous distributions include the normal distribution, the gamma distribution, the Student's t-distribution, and the exponential distribution.