1. Mathematical Preliminaries
1.1. Sets . A set is a well-defined collection of elements. Example: A = {a, b, c} is a set with three elements. a A means that the element a is in the set A d A means that the element d is not in the set A. . denotes the null set or empty set, i.e., the set with no element. . set A is a subset of set B if a B for all a A. This is denoted by A B, or B A. . union of sets: A B = {x: x A or x B}. . intersection of sets: A B = {x: x A and x B}. . A and B are disjoint sets if A B = . . the complement of set A in set B is the set B\A = {x: x B and x A}. . Cartesian product: Let x X and y Y, and let (x, y) be an ordered pair. Then (x, y) XY, where XY is the Cartesian product of X and Y.
1.2. Logic Consider two propositions P and Q. If P implies Q, then P is a sufficient condition for Q, and Q is a necessary condition for P. This is denoted by P Q. If P implies Q and Q implies P, then “P holds if and only if Q holds”, P and Q are equivalent, and P is a necessary and sufficient condition for Q. This is denoted by P Q. Let {not P} denote the statement that P is not true. Contrapositive: If P implies Q, then {not Q} implies {not P}.
Example 1: Let x be a real number, P = {x > 0} and Q = {x2 > 0}. Then, P Q and {not Q} {not P} hold, but {Q P} is false. Example 2: Let P = {x > 0} and Q = {x3 > 0}. Then P Q, and {not P} {not Q} hold.
1.3. Numbers . natural numbers: N = {1, 2, 3, ...} . integers: Z = {..., -3, -2, -1, 0, 1, 2, 3, ...} . rational numbers: Q = {a/b: a Z, b Z, b 0} . irrational numbers: 21/2, 31/2, e, , ... . real numbers: R = {x: x is rational or irrational}
R+ = {x: x R, x 0}
R++ = {x: x R, x > 0} . complex numbers: C = {a + b i: a R, b R, i = (-1)1/2}, where a is the real part, and b is the imaginary part n . the n-fold Cartesian product of R: R = R...R = {(x1, x2, ..., xn): xi R, i = 1, ..., n}, where xi is the i-th coordinate of x = (x1, x2, ..., xn)
1.4. Functions Let X and Y be sets. Definition: f: X Y is a mapping associating each element of X with an element of Y. X is the domain of f 2
f(x) is the image of x under f f(X) = {f(x): x X} is the image of X under f . a function: if only one point in Y is associated with each point in X . a correspondence: if more than one point in Y can be associated with each point in X . inverse function: x = f-1(y) if and only if y = f(x) . a function f: X Y is onto if f(X) = Y. It means that the equation f(x) = y has at least one solution for each y. . if f(x) and f-1(y) are both single-valued, then f is one-to-one. It means that the equation f(x) = y has at most one solution for each y. . composite function: h = g(f(x)) = g f is the composition of f with g satisfying f: A B C, g: C D, g f: A D.
1.5. Bounds Let S R. . S is bounded from above (from below) if there exists a R (b R) such that x a (x b) for all x S. Then, a is an upper bound of S, and b is a lower bound of S. . The least upper bound (lub) or supremum (sup) of S is the upperbound of S such that there does not exist a smaller upper bound. It is denoted by sup(S). . The supremum of S is called a maximum (max) of S if sup(S) S. It is denoted by max(S). . The greatest lower bound (glb) or infimum (inf) of S is the lower bound of S such that there does not exist a larger lower bound. It is denoted by inf(S). . The infimum of S is called a minimum (min) of S if inf(S) S. It is denoted by min(S).
. Property: If S R and S has an upperbound, then S has a supremum. If S R and S has a lowerbound, then S has an infimum.
1.6. Vector Space Consider a set V. L1- associative law: x + (y + z) = (x + y) + z, for all x, y, z, V L2- identity: there exists 0 V such that x + 0 = x for all x V L3- inverse: there exists (-x) V such that x + (-x) = 0 for all x V L4- commutative law: x + y = y + x for all x, y V L5- associative law: (x) = () x for all , R, and for all x V L6- identity: there exists 1 V such that 1x = x for all x V L7- distributive law: (x + y) = x + y for all R, and for all x, y V L8- distributive law: ( + )x = x + x for all , R, and for all x V L9- closure: x V and y V implies that (x + y) V L10- closure: x V and R implies that (x) V.
Definition: A set V is vector space (or linear space) if it satisfies L1-L10. Then x V is called a vector.
Examples: Rn, or Cn is each a vector space.
1.7. Norms and distances Consider a function d(x, y) satisfying: M1: d(x, y) = 0 if and only if x = y M2: d(x, y) + d(y, z) d(z, x) M3: d(x, y) 0 for all x, y 3
M4: d(x, y) = d(y, x).
Definition: For a given set X, if a function d: XX R satisfies M1-M4, then: X is a metric space, denoted by (X, d) d is a metric d(x, y) is the distance between points x and y.
Examples: 2 1/2 d1(x, y) = [i (xi - yi) ] = Euclidian distance, denoted by ||x - y|| d2(x, y) = maxi |xi - yi|
d3(x, y) = i |xi - yi|
Note: Topology consists in studying the properties of sets that are independent of the distance measure chosen.
Definition: Let V be a vector space. A real value function N: V R is called a norm on V if: N(x) 0 for all x V N(x) = 0 if and only if x = 0 N(r x) = |r| N(x) for all r R and x V, and N(x + y) N(x) + N(y) for all x, y V.
2 1/2 Example: N(x) = d1(x, 0) = [i (xi) ] = ||x|| is the Euclidian norm of x in R.
Rn, with Euclidian norm and Euclidian metric, is a normed vector space.
Every normed vector space is a metric space with respect to the induced metric defined by d1(x, y) = ||x - y||.
1.8. Convex Sets Let X be a vector space (e.g., X = Rn).
Definition: A set S X is convex if any x, y S implies that ( x + (1-) y) S, for all R, 0 1.
Note: ( x + (1-) y) is called a linear combination of x and y.
Properties: . Any intersection of convex sets is convex.
. Let Si, i = 1, ..., m, be convex sets in vector space X. Then:
. (iI i Si) = {x: x = i=1,…,m i xi, xiSi, iR, i = 1, …, m} is a convex set.
. (S1S2...Sm) = i=1,…,m (Si) is a convex set.
1.9. Compact Sets Let S Rn.
n Definition: An open ball about x0 R with radius r R, r > 0, is defined as:
Br(x0) = {x: x S, d(x, x0) < r}, where d(x, x0) is the Euclidian distance between points x and x0.
n Definition: An open set S R is a set S such that, for each x S, there exists an open ball Br(x) completely contained in S. 4
. The union of open sets is open. . A finite intersection of open sets is open.
Definition: The interior of a set S, denoted by int(S), is the union of all open sets contained in S.
. A set S is open if and only if S = int(S).
Definition: A set S Rn is closed if the set (Rn\S) is open.
. The intersection of closed sets is closed. . A finite union of closed sets is closed.
Definition: The closure of a set S, denoted by cl(S), is the intersection of all closed sets containing S.
. A set S is closed if and only if S = cl(S).
Definition: The boundary of a set S Rn is the set cl(S)cl(Rn/S).
Definition: A set S is bounded if there exists an open ball with a finite radius which contains S.
Definition: A collection of open sets (S)A in a metric space X is said to be an open cover of a n given set S R if S A S.
The open cover (S)A of S is said to admit a finite subcover if there exists a finite
subcollection (S)F such that S F S.
Definition 1: A set S Rn is compact if and only if it is closed and bounded.
Definition 2: A subset S of a metric space X is compact if and only if every open cover of S has a finite subcover.
Note: The definition 2 of compactness applies to sets in any metric space, while definition 1 applies only to sets in Rn.
1.10. Sequences Let (X, d) be a metric space (e.g., X = Rn), and let S X.
Definition: A sequence {xj: j = 1, ..., } in S converges to y if, for any > 0, there exists a
positive integer j’ such that j j’ implies d(y, xj) < .
This is denoted by y = limj {xj}, where y is the limit of {xj}.
Note: It does not follow that y = limj {xj} S. Examples of convergent series: n e = limn (1 + 1/n) , where n {1, 2, 3, …} 2.71828… More generally, for x R, x n e = limn {(1 + x/n) , where n {1, 2, 3, …} x This defines the exponential function, e : R R++. It satisfies e0 = 1 5
ex+y = ex ey (ex)y = exy for x, y R. x And for y > 0, y = e ln(y) = x. This defines the logarithmic function, ln(y): R++ R, as the inverse function of ex. It satisfies ln(1) = 0 ln(e) = 1
ln(x y) = ln(x) + ln(y), and ln(x/y) = ln(x) - ln(y), for x, y R++ y ln(x ) = y ln(x), for x R++ , y R.
Definition: A sequence {xj: j = 1, ..., } in S is a Cauchy sequence if for any > 0, there exists a
positive integer j’ such that, for any i, j j’, d(xi, xj) < .
Definition: If every Cauchy sequence in a metric space is also a convergent sequence, then the metric space is said to be complete.
n . A sequence {xj: j = 1, …,} in R is a Cauchy sequence if and only if it is a convergent n sequence, i.e. if and only if there is y R such that limj {xj} y.
By the above definition, this implies that Rn is complete (although not all metric spaces are complete).
Definition: Let m(j) be an increasing function: m: {1, 2, 3, ...} {1, 2, 3, ...}, such that m(k+1)
> m(k). Given a sequence {xj: j = 1, 2, ..., }, {xm(j): m = 1, ..., } is a subsequence of
{xj: j = 1, 2, ..., }.
. A set S Rn is closed if and only if every convergent sequence of points in S converges to a point in S.
. A set S Rn is compact if and only if every sequence in S has a convergent subsequence whose limit is in S.
n . A sequence {xj: j = 1, 2, ..., } in R converges to y if and only if every subsequence of {xj: j = 1, ..., } converges to y.
. Every bounded sequence contains a convergent subsequence.
Definition: A sequence {xj: j = 1, 2, ..., } is (strictly) increasing if, for all m > n, xm (>) xn for all n.
A sequence {xj: j = 1, 2, ..., } is (strictly) decreasing if, for all m > n, xm (<) xn for all n . Let X R, X . If X is bounded from above (below), there exists an increasing (decreasing) sequence in X converging to sup(X) (inf(X)).
Definition: Assume that are allowed as limits of a sequence.
The lim sup of the sequence {xj: j = 1, 2,…} in R is defined as limj {aj: j = 1, 2 …},
where aj = sup{xj, xj+1, xj+2, …}. It is denoted by limj supkj xk, or simply by lim supj xj. 6
The lim inf of the sequence {xj: j = 1, 2,…} in R is defined as limj {bj: j = 1, 2 …},
where bj = inf{xj, xj+1, xj+2, …}. It is denoted by limj infkj xk, or simply by lim inf j xj.
. A sequence xj in R converges to a limit y R if and only if y = lim supj xj = lim infj xj.