Norm and Inner Products in Rn Math 130 Linear Algebra Kcvk = |C| Kvk D Joyce, Fall 2015 When C Is a Real Number and V Is a Real Vector

Home , Scalar multiplication

In many ways, norms act like absolute values. For instance, the norm of any vector is nonnegative, and the only vector with norm 0 is the 0 vector. Like absolute values, norms are multiplicative in the sense that Norm and inner products in Rn Math 130 Linear Algebra kcvk = |c| kvk D Joyce, Fall 2015 when c is a real number and v is a real vector. So far we’ve concentrated on the operations of There’s also a triangle inequality for norms addition and scalar multiplication in Rn and, more generally, in abstract vector spaces. kw − vk ≤ kwk + kvk. There are two other algebraic operations on Rn H we mentioned early in the course, and now it’s time ¡ H ¡ HHw − v to look at them in more detail. H ¡ HH One of them is the length of a vector, more com- ¡ Hj v 1 monly called the norm of a vector. The other is a ¡ ¡ w kind of multiplication of two vectors called the in- ¡ ner product or dot product of two vectors. There’s a ¡ connection between norms and inner products, and we’ll look at that connection. Here’s a geometric argument for the triangle in- Today we’ll restrict our discussion of these con- equality. If you draw a triangle with one side being cepts to Rn, but later we’ll abstract these concepts the vector v, and another side the vector w, then to define inner product spaces in general. the third side is w−v. Then the triangle inequality just says that one side of a triangle is less than or The norm, or length, kvk of a vector v. Con- 2 equal to the sum of the other two sides. Equality sider a vector v = (v1, v2) in the plane R . By the holds when the vectors v and w point in the same Pythagorean theorem of plane geometry, the dis- direction. tance k(v , v )k between the point (v , v ) and the 1 2 1 2 Later, we’ll prove the triangle inequality alge- origin (0, 0) is braically. q 2 2 k(v1, v2)k = v1 + v2. The inner product hv|wi of two vectors. This Thus, we define the length or norm of a vector v = are also commonly called a dot product and denoted (v , v ) as being 1 2 with the alternate notation v · w. q 2 2 We’ll start by defining inner products alge- kvk = k(v1, v2)k = v + v . 1 2 braically, then see what they mean geometrically. The norm of a vector is sometimes denoted |v| The inner product hv|wi of two vectors v and w rather than kvk. in Rn is the sum of the products of corresponding Norms are defined for Rn as well coordinates, that is,

kvk = k(v1, v2, . . . , vn)k v hv|wi = h(v1, v2, . . . , vn)|(w1, w2, . . . , wn)i q u n n 2 2 2 uX 2 X = v1 + v2 + ··· + vn = t vk. = v1w1 + v2w2 + ··· vnwn = vkwk k=1 k=1

1 Notice right away that we can interpret the The inner product of two vectors and the square of the length of the vector as an inner prod- cosine of the angle between them. For this uct. Since discussion, we’ll restrict our attention to dimension 2 since we know a lot of plane geometry. 2 2 2 2 kvk = v1 + v2 + ··· + vn, The law of cosines for oblique triangles says that given a triangle with sides a, b, and c, and angle θ therefore between sides a and b, kvk2 = hv|vi. Because of this connection between norm and inner ¡HH H c product, we can often reduce computations involv- ¡ HH ¡ HH ing length to simpler computations involving inner ¡ H a¡ products. ¡ The inner product acts like multiplication in a ¡ b θ lot of ways, but not in all ways. First of all, the in- ¡ ner product of two vectors is a scalar, not another vector. That means you can’t even ask if it’s as- c2 = a2 + b2 − 2ab cos θ. sociative because the expression hhu|vi|wi doesn’t Now, start with two vectors v and w, and place even make sense; hu|vi is a scalar, so you can’t take them in the plane with their tails at the same point. its inner product with the vector w. Let θ be the angle between these two vectors. The But aside from associativity, inner products act vector that joins the head of v to the head of w is a lot like ordinary products. For instance, inner w − v. Now we can use the law of cosines to see products are commutative: that hu|vi = hv|ui.

H Also, inner products distribute over addition, ¡ H ¡ HHw − v H hu|v + wi = hu|vi + hu|wi, ¡ HH ¡ Hj 1 v¡ and over subtraction, ¡ w ¡θ ¡ hu|v − wi = hu|vi − hu|wi, and the inner product of any vector and the 0 vec- kw − vk2 = kvk2 + kwk2 − 2kvk kwk cos θ. tor is 0 We can convert the distances to inner products to hv|0i = 0. simplify this equation.

These last few statements can be summarized by Now, if we subtract kvk2 + kwk2 from both sides saying that inner products are linear in each coor- of our equation, and then divide by −2, we get dinate, or that inner products are bilinear operations. hv|wi = kvk kwk cos θ.

2 That gives us a way of geometrically interpreting u = the inner product. We can also solve the last equa- 3 4 tion for cos θ, >> norm(u) hv|wi cos θ = , kvk kwk ans = which will allow us to do trigonometry by means of 5 linear algebra. Note that >> v = [5 12] hv|wi θ = arccos . kvk kwk v = 5 12 Orthogonal vectors. The word “orthogonal” is synonymous with the word “perpendicular,” but for >> norm (v) some reason is preferred in many branches of math- ematics. We’ll write w ⊥ v if the vectors w and v ans = are orthogonal, or perpendicular. 13

>> dot(u,v)

ans = BBM B 63 1 v B B w >> costheta = dot(u,v)/(norm(u)*norm(v)) B B costheta = Two vectors are orthogonal if the angle between 0.9692 them is 90◦. Since the cosine of 90◦ is 0, that means >> acos(costheta) w ⊥ v if and only if hw|vi = 0 ans = Vectors in Matlab. You can easily ﬁnd the 0.2487 length of a vector in Matlab; where the length Thus, the angle between the vectors (3, 4) and of a vector is called its norm. Let’s ﬁnd the length (5, 12) is 0.2487 radians. of two vectors and the angle between them using the formula Math 130 Home Page at hv|wi http://math.clarku.edu/~ma130/ θ = arccos . kvk kwk

Note that arccosines are computed with the acos function, and inner products with the dot function

>> u = [3 4]