EE263 Autumn 2015 S. Boyd and S. Lall Orthogonality 1 Inner product T hx; yi := x1y1 + x2y2 + ··· + xnyn = x y important properties: I hαx; yi = αhx; yi I hx + y; zi = hx; zi + hy; zi I hx; yi = hy; xi I hx; xi ≥ 0 I hx; xi = 0 () x = 0 n T f(y) = hx; yi is linear function : R ! R, with linear map defined by row vector x 2 Cauchy-Schwarz inequality and angle between vectors n for any x; y 2 R jxTyj ≤ kxkkyk n I (unsigned) angle between vectors in R defined as xTy θ = (x; y) = cos−1 \ kxkkyk x y T θ x y 2 y kyk T I thus x y = kxkkyk cos θ 3 Special cases T I x and y are aligned: θ = 0; x y = kxkkyk; (if x 6= 0) y = αx for some α ≥ 0 T I x and y are opposed: θ = π; x y = −kxkkyk (if x 6= 0) y = −αx for some α ≥ 0 T I x and y are orthogonal: θ = π=2 or −π=2; x y = 0 denoted x ? y 4 Angles interpretation of xTy > 0 and xTy < 0 T I x y > 0 means \(x; y) is acute T I x y < 0 means \(x; y) is obtuse x x y y xT y > 0 xT y < 0 5 Halfspaces a halfspace with outward normal vector y, and boundary passing through 0 H = fx j xTy ≤ 0g T {x | x y ≤ 0} 0 y 6 Orthogonal complements n for any set S ⊂ R , the orthogonal complement is S? = fx j xTy = 0 for all y 2 Sg ? I S is always a subspace ? I S is the set of all vectors x, each of which is orthogonal to every vector in S 7 Orthonormal set of vectors n set of vectors fu1; : : : ; ukg ⊂ R is I normalized if kuik = 1, i = 1; : : : ; k (ui are called unit vectors or direction vectors) I orthogonal if ui ? uj for i 6= j I orthonormal if both slang: we say `u1; : : : ; uk are orthonormal vectors' but orthonormality (like inde- pendence) is a property of a set of vectors, not vectors individually in terms of U = [u1 ··· uk], orthonormal means T U U = Ik 8 Orthonormality an orthonormal set of vectors is independent T I to see this, multiply Ux = 0 by U I hence fu1; : : : ; ukg is an orthonormal basis for span(u1; : : : ; uk) = range(U) T I warning: if k < n then UU 6= I (since its rank is at most k) (more on this matrix later . ) 9 Orthonormal basis for Rn A matrix U is called orthogonal if U is square and U TU = I n I the set of columns u1; : : : ; un is an orthonormal basis for R I (you'd think such matrices would be called orthonormal, not orthogonal) −1 T T I it follows that U = U , and hence also UU = I, i.e., n X T uiui = I i=1 10 Expansion in orthonormal basis suppose U is orthogonal, so x = UU Tx, i.e., n X T x = (ui x)ui i=1 T I ui x is called the component of x in the direction ui T I a = U x resolves x into the vector of its ui components I x = Ua reconstitutes x from its ui components n X I x = Ua = aiui is called the (ui-) expansion of x i=1 11 Geometric interpretation if U has orthonormal columns then transformation w = Uz I preserves norm of vectors, i.e., kUzk = kzk I preserves angles between vectors, i.e., \(Uz; Uz~) = \(z; z~) I we say U is isometric, it preserves distances 12 Example: Rotation 2 rotation by θ in R is given by cos θ − sin θ y = U x; U = θ θ sin θ cos θ since e1 7! (cos θ; sin θ), e2 7! (− sin θ; cos θ) x2 e2 θ e1 x1 13 Example: Reflection reflection across line x2 = x1 tan(θ=2) is given by cos θ sin θ y = R x; R = θ θ sin θ − cos θ since e1 ! (cos θ; sin θ), e2 ! (sin θ; − cos θ) x2 e2 θ e1 x1 14.