Appendix 445 12.4 Calculus review This section is a very brief summary of Calculus skills required for reading this book. 12.4.1 Inverse function Function g is the inverse function of function f if g(f(x)) = x and f(g(y)) = y for all x and y where f(x) and g(y) exist. 1 Notation Inverse function g = f − 1 (Don’t confuse the inverse f − (x) with 1/f(x). These are different functions!) To find the inverse function, solve the equation f(x) = y. The solution g(y) is the inverse of f. For example, to find the inverse of f(x)=3+1/x, we solve the equation 1 3 + 1/x = y 1/x = y 3 x = . ⇒ − ⇒ y 3 − The inverse function of f is g(y) = 1/(y 3). − 12.4.2 Limits and continuity A function f(x) has a limit L at a point x0 if f(x) approaches L when x approaches x0. To say it more rigorously, for any ε there exists such δ that f(x) is ε-close to L when x is δ-close to x0. That is, if x x < δ then f(x) L < ε. | − 0| | − | A function f(x) has a limit L at + if f(x) approaches L when x goes to + . Rigorously, for any ε there exists such N that f∞(x) is ε-close to L when x gets beyond N∞, i.e., if x > N then f(x) L < ε. | − | Similarly, f(x) has a limit L at if for any ε there exists such N that f(x) is ε-close to L when x gets below ( N), i.e.,−∞ − if x < N then f(x) L < ε. − | − | Notation lim f(x) = L, or f(x) L as x x0 x x0 → → lim→ f(x) = L, or f(x) L as x x → → ∞ lim→∞ f(x) = L, or f(x) L as x x →−∞ → → −∞ 446 Probability and Statistics for Computer Scientists Function f is continuous at a point x0 if lim f(x) = f(x0). x x0 → Function f is continuous if it is continuous at every point. 12.4.3 Sequences and series A sequence is a function of a positive integer argument, f(n) where n = 1, 2, 3,.... Sequence f(n) converges to L if lim f(n) = L n →∞ and diverges to infinity if for any M there exists N such that f(n) >M when n>N. A series is a sequence of partial sums, n f(n) = ak = a1 + a2 + . + an. kX=1 Geometric series is defined by n an = Cr , where r is called the ratio of the series. In general, n rm rn+1 Crn = f(n) f(m 1) = C − . − − 1 r kX=m − For m = 0, we get n 1 rn+1 Crn = C − . 1 r kX=0 − A geometric series converges if and only if r < 1. In this case, | | ∞ Crm ∞ C lim Crn = and Crn = . n →∞ 1 r 1 r kX=m − kX=0 − A geometric series diverges to if r 1. ∞ ≥ 12.4.4 Derivatives, minimum, and maximum Derivative of a function f at a point x is the limit f(y) f(x) f ′(x) = lim − y x y x → − provided that this limit exists. Taking derivative is called differentiation. A function that has derivatives is called differentiable. Appendix 447 d Notation f (x) or f(x) ′ dx Differentiating a function of several variables, we take partial derivatives denoted as ∂ ∂ f(x1,x2,...), f(x1,x2,...), etc.. ∂x1 ∂x2 The most important derivatives are: m m 1 (x )′ = mx − x x (e )′ = e (ln x)′ = 1/x C′ = 0 (f + g)′(x) = f ′(x) + g′(x) Derivatives (Cf)′(x) = Cf ′(x) (f(x)g(x))′ = f ′(x)g(x) + f(x)g′(x) f(x) ′ f ′(x)g(x) f(x)g′(x) = − g(x) g2(x) for any functions f and g and any number C To differentiate a composite function f(x) = g(h(x)), we use a chain rule, d Chain rule g(h(x)) = g′(h(x))h′(x) dx For example, d 1 ln3(x) = 3 ln2(x) . dx x Geometrically, derivative f ′(x) equals the slope of a tangent line at point x; see Figure 12.1. Computing maxima and minima At the points where a differentiable function reaches its minimum or maximum, the tangent line is always flat; see points x2 and x3 on Figure 12.1. The slope of a horizontal line is 0, 448 Probability and Statistics for Computer Scientists f(x) 6 slope = 0 slope = f (′x 4 ) ) 1 (x f′ slope = 0 slope = - x1 x2 x3 x4 x FIGURE 12.1: Derivative is the slope of a tangent line. and thus, f ′(x) = 0 at these points. To find out where a function is maximized or minimized, we consider – solutions of the equation f ′(x) = 0, – points x where f ′(x) fails to exist, – endpoints. The highest and the lowest values of the function can only be attained at these points. 12.4.5 Integrals Integration is an action opposite to differentiation. A function F (x) is an antiderivative (indefinite integral) of a function f(x) if F ′(x) = f(x). Indefinite integrals are defined up to a constant C because when we take derivatives, C′ = 0. Notation F (x) = f(x) dx Z An integral (definite integral) of a function f(x) from point a to point b is the difference of antiderivatives, b f(x) dx = F (b) F (a). − Za Improper integrals b ∞ ∞ f(x) dx, f(x) dx, f(x) dx a Z Z−∞ Z−∞ Appendix 449 are defined as limits. For example, b ∞ f(x) dx = lim f(x) dx. b Za →∞ Za The most important integrals are: xm+1 xm dx = for m = 1 m + 1 6 − R 1 x− dx = ln(x) Indefinite R ex dx = ex integrals R (f(x) + g(x))dx = f(x)dx + g(x)dx R Cf(x)dx = CR f(x)dx R Rfor any functions f and g Rand any number C 2 3 4 For example, to evaluate a definite integral 0 x dx, we find an antiderivative F (x) = x /4 and compute F (2) F (0) = 4 0 = 4. A standard way to write this solution is − − R 2 x4 x=2 24 04 x3dx = = = 4. 4 4 − 4 Z0 x=0 Two important integration skills are integration by substitution and integration by parts. Integration by substitution An integral often simplifies when we can denote a part of the function as a new variable (y). The limits of integration a and b are then recomputed in terms of y, and dx is replaced by dy dx dx = or dx = dy, dy/dx dy whichever is easier to find. Notice that dx/dy is the derivative of the inverse function x(y). Integration dx f(x) dx = f(x(y)) dy by substitution dy Z Z For example, 2 6 y=6 6 3 1 1 e e− e3xdx = ey dy = ey = − = 134.5. 1 3 3 3 y= 3 3 Z− Z− − 450 Probability and Statistics for Computer Scientists Here we substituted y = 3x, recomputed the limits of integration, found the inverse function x = y/3 and its derivative dx/dy = 1/3. In the next example, we substitute y = x2. Derivative of this substitution is dy/dx = 2x: 2 4 4 y=4 4 2 dy 1 1 e 1 x ex dx = x ey = eydy = ey = − = 26.8. 2x 2 2 2 Z0 Z0 Z0 y=0 Integration by parts This technique often helps to integrate a product of two functions. One of the parts is integrated, the other is differentiated. Integration f ′(x)g(x)dx = f(x)g(x) f(x)g′(x)dx by parts − Z Z Applying this method is reasonable only when function (fg′) is simpler for integration than the initial function (f ′g). x In the following example, we let f ′(x) = e be one part and g(x) = x be the other. Then x f ′(x) is integrated, and its antiderivative is f(x) = e . The other part g(x) is differentiated, and g′(x) = x′ = 1. The integral simplifies, and we can evaluate it, x exdx = x ex (1)(ex)dx = x ex ex. − − Z Z Computing areas Area under the graph of a positive function f(x) and above the interval [a, b] equals the integral, b (area from a to b) = f(x)dx. Za Here a and b may be finite or infinite; see Figure 12.2. Gamma function and factorial Gamma function is defined as ∞ t 1 x Γ(t) = x − e− dx for t > 0. Z0 Taking this integral by parts, we obtain two important properties of a Gamma function, Γ(t + 1) = tΓ(t) for any t > 0, Γ(t + 1) = t! = 1 2 . t for integer t. · · · Appendix 451 6f(x) This This area area equals equals b ∞ f(x)dx f(x)dx c Za Z - a b c x FIGURE 12.2: Integrals are areas under the graph of f(x). 12.5 Matrices and linear systems A matrix is a rectangular chart with numbers written in rows and columns, A11 A12 A1c A A · · · A A = 21 22 · · · 2c ··· ··· ··· ··· A A A r1 r2 · · · rc where r is the number of rows and c is the number of columns. Every element of matrix A is denoted by Aij , where i [1, r] is the row number and j [1, c] is the column number.