Appendix A Notation and Definitions All notation used in this work is “standard”, and in most cases it conforms to the ISO conventions. (The notable exception is the notation for vectors.) I have opted for simple notation, which, of course, results in a one-to-many map of notation to object classes. Within a given context, however, the over- loaded notation is generally unambiguous. I have endeavored to use notation consistently. This appendix is not intended to be a comprehensive listing of definitions. The subject index, beginning on page 377, is a more reliable set of pointers to definitions, except for symbols that are not words. General Notation Uppercase italic Latin and Greek letters, A, B, E, Λ, and so on, are generally used to represent either matrices or random variables. Random variables are usually denoted by letters nearer the end of the Latin alphabet, X, Y , Z, and by the Greek letter E. Parameters in models (that is, unobservables in the models), whether or not they are considered to be random variables, are generally represented by lowercase Greek letters. Uppercase Latin and Greek letters, especially P , in general, and Φ, for the normal distribution, are also used to represent cumulative distribution functions. Also, uppercase Latin letters are used to denote sets. Lowercase Latin and Greek letters are used to represent ordinary scalar or vector variables and functions. No distinction in the notation is made between scalars and vectors; thus, β may represent a vector, and βi may represent the ith element of the vector β. In another context, however, β may represent a scalar. All vectors are considered to be column vectors, although we may write a vector as x =(x1,x2,...,xn). Transposition of a vector or a matrix is denoted by a superscript T. Uppercase calligraphic Latin letters, F, V, W, and so on, are generally used to represent either vector spaces or transforms. 313 314 APPENDIX A. NOTATION AND DEFINITIONS Subscripts generally represent indexes to a larger structure; for example, th xij may represent the (i, j) element of a matrix, X. A subscript in paren- theses represents an order statistic. A superscript in parentheses represents an (k) th iteration, for example, xi may represent the value of xi at the k step of an iterative process. The following are some examples: th xi The i element of a structure (including a sample, which is a multiset). th x(i) The i order statistic. x(i) The value of x at the ith iteration. Realizations of random variables and placeholders in functions associated with random variables are usually represented by lowercase letters correspond- ing to the uppercase letters; thus, may represent a realization of the random variable E. A single symbol in an italic font is used to represent a single variable. A Roman font or a special font is often used to represent a standard operator or a standard mathematical structure. Sometimes, a string of symbols in a Roman font is used to represent an operator (or a standard function); for example, exp represents the exponential function, but a string of symbols in an italic font on the same baseline should be interpreted as representing a composition (probably by multiplication) of separate objects; for example, exp represents the product of e, x, and p. A fixed-width font is used to represent computer input or output; for exam- ple, a = bx + sin(c). In computer text, a string of letters or numerals with no intervening spaces or other characters, such as bx above, represents a single object, and there is no distinction in the font to indicate the type of object. Some important mathematical structures and other objects are: IR The field of reals or the set over which that field is defined. IR d The usual d-dimensional vector space over the reals or the set of all d-tuples with elements in IR. d IR + The set of all d-tuples with positive real elements. APPENDIX A. NOTATION AND DEFINITIONS 315 IC The field of complex numbers or the set over which that field is defined. ZZ The ring of integers or the set over which that ring is defined. IG(n) A Galois field defined on a set with n elements. C0,C1,C2,... The set of continuous functions, the set of functions with con- tinuous first derivatives, and so forth. √ i The imaginary unit −1. Computer Number Systems Computer number systems are used to simulate the more commonly used num- ber systems. It is important to realize that they have different properties, however. Some notation for computer number systems follows. IF The set of floating-point numbers with a given precision, on a given computer system, or this set together with the four operators +, -, *, and /. IF is similar to IR in some useful ways; it is not, however, closed under the two basic operations, and not all reciprocals of the elements exclusive of the additive identity exist, so it is clearly not a field. II The set of fixed-point numbers with a given length, on a given computer system, or this set together with the four operators +, -, *, and /. II is similar to ZZ in some useful ways; it is not, however, closed under the two basic operations, so it is clearly not a ring. emin and emax The minimum and maximum values of the exponent in the set of floating-point numbers with a given length. min and max The minimum and maximum spacings around 1 in the set of floating-point numbers with a given length. or mach The machine epsilon, the same as min. [·]c The computer version of the object ·. NaN Not-a-Number. 316 APPENDIX A. NOTATION AND DEFINITIONS Notation Relating to Random Variables A common function used with continuous random variables is a density function, and a common function used with discrete random variables is a probability function. The more fundamental function for either type of random variable is the cumulative distribution function, or CDF. The CDF of a random variable X, denoted by PX (x) or just by P (x), is defined by P (x) = Pr(X ≤ x), where “Pr”, or “probability”, can be taken here as a primitive (it is defined in terms of a measure). For vectors (of the same length), “X ≤ x” means that each element of X is less than or equal to the corresponding element of x. Both the CDF and the density or probability function for a d-dimensional random variable are defined over IRd. (It is unfortunately necessary to state that “P (x)” means the “function P evaluated at x”, and likewise “P (y)” means the same “function P evaluated at y” unless P has been redefined. Using a different expression as the argument does not redefine the function despite the sloppy convention adopted by some statisticians—including myself sometimes!) The density for a continuous random variable is just the derivative of the CDF (if it exists). The CDF is therefore the integral. To keep the notation sim- ple, we likewise consider the probability function for a discrete random variable to be a type of derivative (a Radon–Nikodym derivative) of the CDF. Instead of expressing the CDF of a discrete random variable as a sum over a countable set, we often also express it as an integral. (In this case, however, the integral is over a set whose ordinary Lebesgue measure is 0.) A useful analog of the CDF for a random sample is the empirical cumulative distribution function, or ECDF. For a sample of size n, the ECDF is n 1 P (x)= I −∞ (x ) n n ( ,x] i i=1 for the indicator function I(−∞,x](·). Functions and operators such as Cov and E that are commonly associated with Latin letters or groups of Latin letters are generally represented by that letter in a Roman font. Pr(A) The probability of the event A. pX (·) The probability density function (or probability function), or or PX (·) the cumulative probability function, of the random variable X. pXY (·) The joint probability density function (or probability function), or PXY (·) or the joint cumulative probability function, of the random vari- ables X and Y . APPENDIX A. NOTATION AND DEFINITIONS 317 pX|Y (·) The conditional probability density function (or probability or PX|Y (·) function), or the conditional cumulative probability function, of the random variable X given the random variable Y (these functions are random variables). pX|y(·) The conditional probability density function (or probability or PX|y(·) function), or the conditional cumulative probability function, of the random variable X given the realization y. Sometimes, the notation above is replaced by a similar notation in which the arguments indicate the nature of the distribution; for example, p(x, y)orp(x|y). pθ(·) The probability density function (or probability function), or or Pθ(·) the cumulative probability function, of the distribution charac- terized by the parameter θ. Y ∼ DX (θ) The random variable Y is distributed as DX (θ), where X is the name of a random variable associated with the distribution, and θ is a parameter of the distribution. The subscript may take forms similar to those used in the density and distribution functions, such as X|y, or it may be omitted. Alternatively, in place of DX , a symbol denoting a specific distribution may be used. An example is Z ∼ N(0, 1), which means that Z has a normal distribution with mean 0 and variance 1. CDF A cumulative distribution function. ECDF An empirical cumulative distribution function. i.i.d. Independent and identically distributed. (i) d (i) X → X The sequence of random variables X or Xi converges in dis- d or Xi → X tribution to the random variable X.