Lecture 3
Econ 2001
2015 August 12 Lecture 3 Outline
1 Metric and Metric Spaces 2 Norm and Normed Spaces 3 Sequences and Subsequences 4 Convergence 5 Monotone and Bounded Sequences
Announcements: - Friday’sexam will be at 3pm, in WWPH 4716; recitation will be at 1pm. The exam will last an hour. Vector Space Over the Reals
Definition A vector space V over R is a 4-tuple (V , R, +, ) where V is a set, + : V V V is vector addition, : R V V is scalar multiplication,· satisfying: × → · × → 1 Closure under addition: x, y V , (x + y) V ∀ ∈ ∈ 2 Associativity of +: x, y, z V , (x + y) + z = x + (y + z) ∀ ∈ 3 Commutativity of +: x, y V , x + y = y + x ∀ ∈ 4 Existence of vector additive identity: !0 V s.t. x V , x + 0 = 0 + x = x ∃ ∈ ∀ ∈ 5 Existence of vector additive inverse: x V !y V s.t. x + y = y + x = 0. Denote this unique y as ( x), and define∀ ∈ x ∃y as∈ x + ( y). − − − 6 Distributivity of scalar multiplication over vector addition: α R x, y V , α (x + y) = α x + α y ∀ ∈ ∀ ∈ · · · 7 Distributivity of scalar multiplication over scalar addition: α, β R x V , (α + β) x = α x + β x ∀ ∈ ∀ ∈ · · · 8 Associativity of : α, β R, x V , (α β) x = α (β x) · ∀ ∈ ∀ ∈ · · · · 9 Multiplicative identity: x V , 1 x = x ∀ ∈ · Fun With Vector Spaces
Supose we have a vector space V over R and x V . ∈ What is 0 x =? · distributivity of scalar muliplication existence of vector additive identity 0 x = (0+0) x = 0 x+0 x = 0 · · · · where the first zero is a scalar while the last is a vector. z}|{ z}|{ Using this result: 0 = 0 x = (1 1) x = 1 x + ( 1) x = x + ( 1) x · − · · − · − ·
So we conclude ( 1) x = ( x). − · − which is not something implied by the nine properties of a vector space. Distance We work with abstract sets (typically vector spaces) and our main objective is to be able to talk about their properties as well as properties of functions defined over them. For example, we would like to define notions like limits and convergence, continuity of a function from one of these sets to another, and so on. Continuity of a function f : R R is defined as → ε > 0 δ > 0 such that x y < δ f (x) f (y) < ε ∀ ∃ | − | ⇒ | − | How can we construct a similar definition when the domain is some abstract set X ? ε, δ, and f ( ) are scalars, but we do not know how to measure the distance between two· points. So, next we define function that measures “distance” between any two points in a set. using this function, one can express the idea that two points are “close”(say within δ of each other). Notice that the even more general case in which f is a function from X to Y does not complicate things. First, we define distance for abstract settings; next, specialize to vector spaces. Metric Spaces and Metrics
Definition
A metric d on a set X is a function d : X X R+ satisfying × → 1 d(x, y) 0, with d(x, y) = 0 x = y x, y X . ≥ ⇔ ∀ ∈ 2 d(x, y) = d(y, x) x, y X (symmetry). ∀ ∈ 3 triangle inequality: d(x, z) d(x, y) + d(y, z) x, y, z X ≤ ∀ ∈
A metric space (X , d) is a set X with a metric d defined on it.
The value d(x, y) is interpreted as the distance between x and y. This definition generalizes the concept of distance to abstract spaces. Why “triangle inequality”? y
x %& z → Notice that a metric space need not be a vector space. Metrics: Examples
Example The discrete metric on any given set X is defined as 0 if x = y d(x, y) = 1 if x = y 6 It trivially satisfies 1. and 2., and as for triangle inequality: 1 2. ≤ Example 1 2 2 2 The l metric (taxicab) on R is defined as d : R R R+: × → d(x, y) = x1 y1 + x2 y2 | − | | − |
Example 2 2 2 The l ∞ metric on R is defined as d : R R R+: × → d(x, y) = max x1 y1 , x2 y2 {| − | | − |} Metrics and Applications Definition In a metric space (X , d), a subset S X is bounded if ⊆ x X and β R such that s S, d(s, x) β ∃ ∈ ∈ ∀ ∈ ≤ Given a metric space, boundedness is a propety of subsets (it is relative!).
Definition In a metric space (X , d), we define the diameter of a subset S X by ⊆ diam (S) = sup d(s, s0): s, s0 S { ∈ } A metric can be used to define a “ball of a given radius around a point ”.
Definition In a metric space (X , d), we define an open ball of center x and radius ε as
Bε(x) = y X : d(y, x) < ε { ∈ } and a closed ball of and center x and radius ε as
Bε[x] = y X : d(y, x) ε { ∈ ≤ } Distance Between Sets
We can also define the distance from a point to a set: d(A, x) = inf d(a, x) a A ∈ and the distance between sets: d(A, B) = inf d(B, a) a A ∈ = inf d(a, b): a A, b B { ∈ ∈ } Note that d(A, B) is not a metric (why?). Norms and Normed Spaces The following defines a vector’slength (its size). Definition
Let V be a vector space. A norm on V is a function : V R+ satisfying k · k → 1 x 0 x V . k k ≥ ∀ ∈ 2 x = 0 x = 0 x V . k k ⇔ ∀ ∈ 3 triangle inequality: x + y x + y x, y V k k ≤ k k k k ∀ ∈ 4 αx = α x α R, x V k k | |k k ∀ ∈ ∀ ∈ x x y Why is this also called “triangle” inequality? 0 %&x + y y → x &%y Definition A normed vector space is a vector space over R equipped with a norm.
There are many different norms: if is a norm on V , so are 2 and 3 and k for any k > 0. k · k k · k k · k k · k Normed Spaces
Can one relate norm and distance on a given normed vector space? In Rn, the standard norm is defined as 1 n 2 z = z2 k k i i=1 ! X while the Euclidean distance between two vectors x and y n 2 2 2 2 d(x, y) = (x1 y1) + (x2 y2) + ... + (xn yn) = (xi yi ) − − − v − u i=1 p uX t Norm and Euclidean distance Here, the distance between x and y is the length of the vector given by their difference: d(x, y) = x y k − k
This idea generalizes. Norm and Distance
In any normed vector space, the norm can be used to define a distance.
Theorem
Let (V , ) be a normed vector space. Let d : V V R+ be defined by k · k × ⇒ d(v, w) = v w k − k Then (V , d) is a metric space.
In a normed vector space, we define a metric using the norm.
Remember that x : V R+ is the lenght of a vector x V . k k → ∈ Here x = v w. − Hence, the distance between two vectors is the length of the difference vector. Proof that (V , d) is a Metric SpaceI
We must verify that the d defined satisfies the three properties of a metric. First property: d(x, y) 0, with d(x, y) = 0 x = y x, y X ≥ ⇔ ∀ ∈ Proof. Let v, w V . By definition, d(v, w) = v w 0. Moreover ∈ k − k ≥ d(v, w) = 0 v w = 0 ⇔ k − k v w = 0 by def of ⇔ − k · k (v + ( w)) + w = w by vector additive inverse ⇔ − v + (( w) + w) = w by associative property ⇔ − v + 0 = w by vector additive inverse ⇔ v = w ⇔ Proof that (V , d) is a Metric SpaceII
For symmetry, we need to show that d(x, y) = d(y, x) x, y X ∀ ∈ Proof. Let v, w V . ∈ d(v, w) = v w k − k = 1 v w | − |k − k = ( 1)(v + ( w)) by def of k − − k k · k = ( 1)v + ( 1)( w) k − − − k = v + w by the first slide k − k = w + ( v) k − k = w v k − k = d(w, v) Proof that (V , d) is a Metric SpaceIII
The triangle inequality is d(x, z) d(x, y) + d(y, z) x, y, z X . ≤ ∀ ∈ Proof. Let u, w, v V . ∈ d(u, w) = u w k − k = u + ( v + v) w k − − k = u v + v w k − − k u v + v w by triangle for ≤ k − k k − k k · k = d(u, v) + d(v, w)
Summary Since we have shown that d( ) = satisfies the three properites of a metric on V , we conclude that (V , d) is· a metrick · k space. Normed Vector Spaces: Examples
Rn admits many norms The n-dimensional Euclidean space Rn supports the following norms the standard (Euclidean) norm:
n 2 x 2 = (xi ) = √x x k k v · u i=1 uX where (the “dot product” of vectors)t is defined as: · n
x y = x1y1 + x2y2 + ... + xnyn = xi yi · i=1 n X the L1 norm: x 1 = xi k k | | i=1 1 p p X p p the L norm: x = ( x1 + ... + xn ) k k∞ | | | | the maximum norm, or sup norm, or L∞ norm:
x = max x1 ,..., xn k k∞ {| | | |} Normed Vector Spaces: Results
Theorem (Cauchy-Schwarz Inequality)
If v, w Rn, then ∈ n 2 n n 2 2 vi wi v w ≤ i i i=1 ! i=1 ! i=1 ! X X X This can also be written as v w 2 v 2 w 2 k · k ≤ k k k k or v w v w k · k ≤ k kk k where is the standard norm. k · k Prove this as an exercise. Different Norms Different norms on a given vector space yield different geometric properties. In Rn, a closed ball around the origin of radius ε is: n Bε [0] = x R : d(x, 0) ε = 0 X : x ε { ∈ ≤ } { ∈ k k ≤ } Draw a ball in R2 ( x R2 : x = 1 ) in 3 different cases: { ∈ k k } standard norm ( ) 1 2 L norm ( 1 ) sup-norm ( ) k·k k·k k·k∞
2 2 x 2 = x1 + x2 = 1, x 1 = x1 + x2 = 1, x = max x1 , x2 = 1 k k k k | | | | k k∞ {| | | |} p Equivalent Norms
Do different norm imply different results when applied to ideas like convergence, continuity and so on? Hopefully not.
Definition
Two norms and ∗ on the same vector space V are said to be Lipschitz-equivalentk · k (ork · equivalent k ) if m, M > 0 such that x V , ∃ ∀ ∈ m x x ∗ M x k k ≤ k k ≤ k k Equivalently, m, M > 0 such that x V , x = 0, ∃ ∀ ∈ 6 x ∗ m k k M ≤ x ≤ k k Equivalent Norms
Equivalent norms define the same notions of convergence and continuity. For topological purposes, equivalent norms are indistinguishable.
Compare Different Norms
Let and ∗ be equivalent norms on the vector space V , and fix x V . k · k k · k ∈ Let
Bε(x, ) = y V : x y < ε and Bε(x, ∗) = y V : x y ∗ < ε k·k { ∈ k − k } k·k { ∈ k − k } Then, for any ε > 0,
B ε (x, ) Bε(x, ∗) B ε (x, ) M k · k ⊆ k · k ⊆ m k · k ε balls are ‘close’to each other.
Result In Rn (or any finite-dimensional normed vector space), all norms are equivalent. Roughly, up to a difference in scaling, for topological purposes there is a unique norm in Rn . Sequences of Real Numbers
Definition A sequence of real numbers is a function f : N R. −→ Notation
If f (n) = an, for n N, we denote the sequence by the symbol an n∞=1, or ∈ { } sometimes by a1, a2, a3,... . { } The values of f , that is, the elements an, are called the terms of the sequence.
Examples
Let an = n then an ∞ = 1, 2, 3,... { }n=1 { } Let a1 = 1, an+1 = an + 2, n > 1 then an n∞=1 = 1, 3, 5,... n ∀ { } { } Let an = ( 1) , n 1 then an ∞ = 1, 1, 1, 1,... − ≥ { }n=1 {− − } Limit of a Sequence of Real Numbers
Definition
A sequence of real numbers an is said to converge to a point a R if { } ∈ ε > 0 there exists N(ε) N such that n N an a < ε ∀ ∈ ≥ ⇒ | − | The sequence converges to a limit point if for every strictly positive epsilon, there exists a natural number (possibly dependent on epsilon) such that all the points in the sequence on and after that number are at most epsilon away from the limit point.
Notation
When an converges to a we usually write { } an a −→ or lim an = a n →∞
Generalize to arbitrary sequences next. Sequences
The objects along the sequence are now elements of some set.
Definition Let X be a set. A sequence is a function f : N X . −→ Notation
We denote the sequence by the symbols xn ∞ , or sometimes by xn . { }n=1 { } When X = Rn each term in the sequence is a vector in Rn. Convergence To define convergence, X must be a metric space so that one talk about points being close to each other using the metric to measure distance.
Definition
Let (X , d) be a metric space. A sequence xn converges to x X if { } ∈ ε > 0 N(ε) N such that n > N(ε) d(xn, x) < ε ∀ ∃ ∈ ⇒ Notation
When xn converges to x we write { } xn x or lim xn = x n → →∞
For any strictly positive epsilon, there exists a number such that all points in the sequence from then on are at most epsilon away from the limit point. Similar to real numbers: we replaced the standard distance with a metric.
Remark The limit point must be an element of X . Uniqueness of Limits for Real Numbers
Theorem
Let an be a sequence of real numbers that converges to two real numbers b and { } b0; then, b = b0.
Proof.
By contradiction. Assume (wlog) that b > b0 and let 1 ε = (b b0) > 0. 2 −
By definition, there are N(ε) and N0(ε) such that
n N(ε) b an < ε and n N0(ε) b0 an < ε ∀ ≥ | − | ∀ ≥ | − | But this is impossible since it implies that
2ε = b b0 b an + b0 an < 2ε − ≤ | − | | − | for n > max N(ε), N0(ε) . { } Uniqueness of Limits
Theorem (Uniqueness of Limits)
In a metric space (X , d), if xn x and xn x0, then x = x0. → → Proof.
By contradiction. Suppose xn is a sequence in X , xn x, xn x0, and x = x0. { } d(x,x0) → → 6 Since x = x0, d(x, x0) > 0. Let ε = 6 2 n > N(ε) d(xn, x) < ε Then, there exist N(ε) and N0(ε) such that ⇒ . n > N0(ε) d(xn, x0) < ε ⇒ Choose n > max N(ε), N0(ε) Then { } d(x, x0) d(x, xn) + d(xn, x0) ≤ < ε + ε = 2ε
= d(x, x0)
d(x, x0) < d(x, x0) a contradiction. Subsequences
If xn is a sequence and n1 < n2 < n3 < then xn is called a subsequence. { } ··· { k } Definition A subsequence of the sequence described by f from N to X is the sequence given by a composite function f g : N X ◦ −→ where g : N N −→ and g(m) > g(n) whenever m > n.
A subsequence is formed by taking some of the elements of the parent sequence in the same order.
Example 1 1 1 1 1 1 1 1 1 xn = , so xn = 1, , , , , ,... . If nk = 2k, then xn = , , ,... . n { } 2 3 4 5 6 { k } 2 4 6 Monotone Sequences of Reals
Definitions
A sequence of real numbers xn is increasing if xn+1 xn for all n. { } ≥ A sequence of real numbers xn is decreasing if xn+1 xn for all n. { } ≤ We say strictly increasing or decreasing if the above inequalities are strict.
Definition A sequence of real numbers is monotone if it is either increasing or decreasing.
Definition
If xn is a sequence of real numbers, xn tends to infinity (written xn or { } { } → ∞ limn xn = ) if →∞ ∞ K R N(K) such that n > N(K) xn > K ∀ ∈ ∃ ⇒ Similarly define xn or lim xn = . → −∞ −∞ Facts about Convergence of Sequences of Reals
Convergence and Subsequences
an converges to b if and only if every subsequence of an converges to b. { } { } Convergence Operations
Let an and bn be sequences such that lim an = a and lim bn = b. Then n n { } { } →∞ →∞ 1 lim (an + bn) = a + b n →∞ 2 for c , lim can = ca, R n ∈ →∞ 3 lim anbn = ab n →∞ k k 4 lim (an) = a n →∞ 1 1 5 lim = provided an = 0 (n = 1, 2, 3,... ), and a = 0 n an a →∞ 6 6 an a 6 lim = provided bn = 0 (n = 1, 2, 3,... ), and b = 0 n bn b →∞ 6 6 Boundedness for Sequences of Real Numbers
Definition
The sequence of real numbers an is bounded above if { } m R such that an m n N ∃ ∈ ≤ ∀ ∈ Definition
The sequence of real numbers an is bounded below if { } m R such that an m n N ∃ ∈ ≥ ∀ ∈ Boundedness and Convergence
Theorem
If a sequence of real numbers an converges then it is bounded. { } Proof.
Suppose an a. −→ Then, there is an integer N such that n > N implies an a < 1. | − | Define r as follows:
r = max 1, a1 a ,..., aN a { | − | | − |} Then an a r for n = 1, 2, 3,... . | − | ≤ Boundedness and Convergence
Theorem
Let an , bn , and cn be sequences of real numbers such that { } { } { }
an bn cn, n N ≤ ≤ ∀ ∈ and we have that both an a, cn a −→ −→ Then bn a −→ Proof in the Problem Set. Monotonicity and Convergence Theorem A monotone sequence of real numbers is convergent if and only if it is bounded.
Proof. We have already shown that a convergent sequence is bounded (monotone or not), so we need only prove that a bounded and monotone sequence converges. Suppose an is bounded and increasing. Let a = sup an : n N . { } { ∈ } We want a to be the limit of an ; so, we need to show that { } ε > 0, N N such that a an < ε, n N ∀ ∃ ∈ | − | ∀ ≥ For any ε > 0, a ε is not an upper bound of an ; thus N such that − { } ∃ aN > a ε . − Since an is increasing, for every n N we have aN an Putting{ these} observations together ≥ ≤
a ε < aN an a − ≤ ≤ where the last inequality follows from an a, n. Thus, ≤ ∀ an a < ε, n N | − | ∀ ≥ Prove the case in which an is bounded and decreasing as an exercise. { } A Monotone Subsequence Always Exist
Theorem (Rising Sun Lemma) Every sequence of real numbers has a monotone subsequence.
Why the name? S ◦←←←←←←←←←←←← U • • • •◦←←←←←←←← N • • • • • ◦ ← ← ← ••• •
Proof. Exercise Bolzano-Weierstrass Theorem
Theorem (Bolzano-Weierstrass) Every sequence of real numbers that is bounded has a convergent subsequence.
Proof. This is done by using the Rising Sun Lemma and the previous theorem: every real sequence has a monotone subsequence and a monotone sequence is convergent if and only if it is bounded Tomorrow
We start defining open and closed sets, then we move to functions and define limits and continuity.
1 Open and Closed Set 2 Limits of functions 3 Continuity