Appendix A Review of Probability Theory A.1 Introduction: What Is Probability? What’s in a word? The words “probably” and “probability” are used commonly in everyday speech. We all know how to interpret expressions such as “It will proba- bly rain tomorrow,” or “Careless smoking probably caused that fire,” although the meanings are not particularly precise. The common usage of “probability” has to do with how closely a given statement resembles truth. Note that in common usage, it may be impossible to verify whether the statement is true or not; that is, the truth may not be knowable. Informally, we use the terms “probable” and “probability” to express a likelihood or chance of truth. While these common usages of the term “probability” are effective in communi- cating ideas, from a mathematical point of view, they lack the precision and stan- dardization of terminology to be particularly functional. Thus scientists and math- ematicians have developed various theories of probability to address the needs of scientific analysis and decision making. We will use a particular theory that has its origins in the early twentieth century and is now (by far) the most widely used theory of probability. This theory provides a formal structure (entities, definitions, axioms, etc.) that allows us to use other well-developed mathematical concepts (limits, sums, averages, etc.) in a way that remains consistent with our understanding of physi- cal principals. All theories have limitations. Our theory of probability, for instance, will not help us answer questions like, “What is the probability that individual X is guilty of a crime?” or “What is the probability that pigs will fly?” Fortunately, a well-developed theory has well-defined limitations, and we should be able to identify when we have overstepped the bounds of scientific validity. As we discuss these concepts, keep in mind that it is “probably” inevitable that we will at times encounter conflicts between the colloquial meanings of words and their formal mathematical definitions. These conflicts are natural and are no cause for alarm! © Springer International Publishing Switzerland 2016 325 M. Sánchez-Silva and G.-A. Klutke, Reliability and Life-Cycle Analysis of Deteriorating Systems, Springer Series in Reliability Engineering, DOI 10.1007/978-3-319-20946-3 326 Appendix A: Review of Probability Theory A.2 Random Experiments and Probability Spaces: The Building Blocks of Probability Our theory of probability begins with the concept of a random experiment. The idea is that we intend to perform an experiment that results in (precisely) one of a group of outcomes. We use the term random experiment because we cannot be certain in advance about the outcome. That is, we can identify all possible outcomes of the experiment, but we do not know in advance which particular outcome will occur. The experiment is assumed to be repeatable, in the sense that we could recreate the exact conditions of the experiment. If we repeat the experiment, however, we are not guaranteed that the same outcome will occur. To effectively describe the random experiment, we must be able to: (i) identify its outcomes, (ii) characterize the information available to us about the outcome of the experiment, and (iii) quantify the likelihood that the experiment results in a particular incident. In mathematical terminology, a random experiment will be identified with (actually, is equivalent to) a probability space. A probability space consists of three entities: a sample space (we will call it ), an event space (we’ll call it F ), and a probability measure (we’ll call it P). Let us discuss each of these entities in turn. A.2.1 Sample Space Formally, we define the sample space to be the collection of all possible outcomes. Elements of the sample space are distinct and exhaustive (i.e., on any given perfor- mance of the experiment, one and only one outcome occurs), and we can think of the sample space as a set of distinct points. The sample space may be discrete (countable or denumerable) or continuous (uncountable or nondenumerable), likewise, it may be finite or infinite. Example A.1 The experiment consists of tossing a coin three times consecutively. Assuming that we do not allow the possibility of a coin landing on its side (H heads or T Tails), the sample space can be identified as {(HHH), (HHT), (HTH), (THH), (HTT), (THT), (TTH), (TTT)}. The sample space is discrete and finite. Example A.2 The experiment consists of two players (A and B) playing hands of poker for $1 per hand. Each player begins with $5, and the game continues until one of the players is bankrupt. Here the sample space can be identified as all sequences of the elements A and B such that the number of one letter does not exceed the number of the other letter by more than 5. The sample space is discrete and infinite. Example A.3 The experiment consists of measuring the diameter of every 5th steel cylinder that leaves a manufacturing line. The sample space consists of sequences of real numbers; it is continuous and infinite. Appendix A: Review of Probability Theory 327 To reiterate, a sample space is a set of outcomes; it obeys the typical rules that obtain with sets (unions, intersections, complements, differences, etc.). A.2.2 Event Space The second element of a probability space is a collection of so-called events F . Events themselves consist of particular groups of outcomes. Thus the set of events is a collection of subsets of the sample space. Events can be thought of as characteristics of outcomes that can be identified once the experiment has been performed; that is, they are the “information scale” at which we can view the results of an experiment. In many, but not all, experiments, we can identify individual outcomes of an experi- ment; in some experiments we can identify only certain characteristics of individual outcomes. Thus the event space characterizes the information that we have available to us about the outcomes of a random experiment; it is the mesh or filter through which we can view the outcomes. Some terminology: we say that “an event has occurred” if the outcome that occurred is contained in that event. The specification of the event space is not completely arbitrary; in order to main- tain consistency, we need to instill some structure (rules) on the event space. The structure makes perfect intuitive sense. First, if we are able to observe that a partic- ular group of outcomes occurred, we should be able to observe that the same group of outcomes did not occur. This means that if a set of outcomes F is in the event space, then the set of outcomes F (the complement of F) is also in the event space. Secondly, if we are able to determine if a set of outcomes F1 occurred, and we are able to determine if a group of outcomes F2 occurred, then we should be able to determine if either F1 or F2 occurred. That is, if F1 and F2 are in the event space, then F1 ∪ F2 must be in the event space. Finally, we must be able to observe that some outcome occurred; that is, itself must be an event. Note that since is in the event space, so is φ, the empty set (also called the impossible event). With these rules for the event space, the smallest event space that we can work with is F ={,φ}. Example A.4 Suppose the random experiment is as in ExampleA.1, and suppose that we are able to observe the outcome of each individual coin toss. Then the event space consists of all subsets of the sample space (the power set of the sample space). Example A.5 Now suppose the random experiment is as in ExampleA.1, except that we are able to observe only the outcome of the last toss. Then the event space consists of ,φ, and the sets {(HHH), (HTH), (THH), (TTH)} and {(HHT), (HTT), (THT), (TTT)}. Note that an event can be determined either by listing its elements or by stating a condition that its elements must satisfy; e.g., if the sample space of our experiment is as in ExampleA.1,theset{(HHT), (HTH), (THH), (HTT)} and the statement “two heads occurred” determine the same event. 328 Appendix A: Review of Probability Theory A.2.3 Probability Measure The final element of our probability space is an assignment of probabilities for each event in the event space. Such an assignment is described by a function P that assigns a value to each event. This value represents our belief in the likelihood that the experiment will result in an event’s occurrence. The choice of this function quantifies our knowledge of the randomness of the experiment. It is important to remember that a probability measure lives on (assigns values to) events rather than outcomes,but remember, also, that there are certain situations where individual outcomes can also be events; such events are called atomic events. Definition 55 A sample space of a random experiment is the set of all possible outcomes of the experiment. Definition 56 An event space F of a random experiment is a collection of subsets of the sample space that satisfy • is in F • If F is in F , then F is in F • If F1 and F2 are in F , then F1 ∪ F2 is in F . Definition 57 A probability measure P for a random experiment is a function that assigns a numerical value to each event in an event space such that • If F is an event, 0 ≤ P(F) ≤ 1.