Multilinear Algebra

Home , Multilinear algebra

Multilinear Algebra

Davis Shurbert

University of Puget Sound

April 17, 2014

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 1 / 21 Overview

1 Basics Multilinearity Dual Space

2 Tensors Tensor Product p Basis of Tq (V )

3 Component Representation Kronecker Product Components Comparison

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 2 / 21 Deﬁnition A function f : V × U 7→ W , where V , U, and W are vector spaces over a ﬁeld F , is bilinear if for all x, y in V and all α, β in F

f (αx1 + βx2, y) = αf (x1, y) + βf (x2, y), and

f (x, αy1 + βy2) = αf (x, y1) + βf (x, y2).

Basics Multilinearity Multilinear Functions

Deﬁnition A function f : V 7→ W , where V and W are vector spaces over a ﬁeld F , is linear if for all x, y in V and all α, β in F

f (αx + βy) = αf (x) + βf (y).

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 3 / 21 Basics Multilinearity Multilinear Functions

Deﬁnition A function f : V 7→ W , where V and W are vector spaces over a ﬁeld F , is linear if for all x, y in V and all α, β in F

f (αx + βy) = αf (x) + βf (y).

Deﬁnition A function f : V × U 7→ W , where V , U, and W are vector spaces over a ﬁeld F , is bilinear if for all x, y in V and all α, β in F

f (αx1 + βx2, y) = αf (x1, y) + βf (x2, y), and

f (x, αy1 + βy2) = αf (x, y1) + βf (x, y2).

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 3 / 21 Basics Multilinearity Multilinear Functions

Deﬁnition A function f : V 7→ W , where V and W are vector spaces over a ﬁeld F , is linear if for all x, y in V and all α, β in F

f (αx + βy) = αf (x) + βf (y).

Deﬁnition A function f : V × U 7→ W , where V , U, and W are vector spaces over a ﬁeld F , is bilinear if for all x, y in V and all α, β in F

f (αx1 + βx2, y) = αf (x1, y) + βf (x2, y), and

f (x, αy1 + βy2) = αf (x, y1) + βf (x, y2).

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 3 / 21 Basics Multilinearity Multilinear Functions

Deﬁnition A function f : V 7→ W , where V and W are vector spaces over a ﬁeld F , is linear if for all x, y in V and all α, β in F

f (αx + βy) = αf (x) + βf (y).

Deﬁnition A function f : V × U 7→ W , where V , U, and W are vector spaces over a ﬁeld F , is bilinear if for all x, y in V and all α, β in F

f (αx1 + βx2, y) = αf (x1, y) + βf (x2, y), and

f (x, αy1 + βy2) = αf (x, y1) + βf (x, y2).

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 3 / 21 Basics Multilinearity Multilinear Functions

Deﬁnition A function f : V 7→ W , where V and W are vector spaces over a ﬁeld F , is linear if for all x, y in V and all α, β in F

f (αx + βy) = αf (x) + βf (y).

Deﬁnition A function f : V × U 7→ W , where V , U, and W are vector spaces over a ﬁeld F , is bilinear if for all x, y in V and all α, β in F

f (αx1 + βx2, y) = αf (x1, y) + βf (x2, y), and

f (x, αy1 + βy2) = αf (x, y1) + βf (x, y2).

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 3 / 21 Basics Multilinearity Multilinear Functions

Deﬁnition A function f : V 7→ W , where V and W are vector spaces over a ﬁeld F , is linear if for all x, y in V and all α, β in F

f (αx + βy) = αf (x) + βf (y).

Deﬁnition A function f : V × U 7→ W , where V , U, and W are vector spaces over a ﬁeld F , is bilinear if for all x, y in V and all α, β in F

f (αx1 + βx2, y) = αf (x1, y) + βf (x2, y), and

f (x,α y1 + βy2) = αf (x, y1) + βf (x, y2).

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 3 / 21 Basics Multilinearity Multilinear Functions

Deﬁnition s A function f : V1 × · · · × Vs 7→ W , where {Vi }i=1 and W are vector spaces over a ﬁeld F , is s-linear if for all xi , yi in Vi and all α, β in F

f (v1, . . . , αxi + βyi ,..., vs ) =

αf (v1,..., xi ,..., vs ) + βf (v1,..., yi ,..., vs ),

for all indices i in {1,..., s}.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 4 / 21 Basics Multilinearity Multilinear Functions

Deﬁnition s A function f : V1 × · · · × Vs 7→ W , where {Vi }i=1 and W are vector spaces over a ﬁeld F , is s-linear if for all xi , yi in Vi and all α, β in F

f (v1, . . . , αxi + βyi ,..., vs ) =

αf (v1,..., xi ,..., vs ) + βf (v1,..., yi ,..., vs ),

for all indices i in {1,..., s}.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 4 / 21 Basics Multilinearity Multilinear Functions

Deﬁnition s A function f : V1 × · · · × Vs 7→ W , where {Vi }i=1 and W are vector spaces over a ﬁeld F , is s-linear if for all xi , yi in Vi and all α, β in F

f (v1,...,α xi + βyi ,..., vs ) =

αf (v1,..., xi ,..., vs ) + βf (v1,..., yi ,..., vs ),

for all indices i in {1,..., s}.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 4 / 21 Any inner product deﬁned on V is such a function, as

hαv1 + βv2, ui = αhv1, ui + βhv2, ui, and

hv, αu1 + βu2i = αhv, u1i + βhv, u2i

Fix all inputs of f except the i th input, if f is linear as a function of this input, then f is multilinear.

In other words, define fî (x) = f (v1,..., vi−1, x, vi+1 ..., vs ) , then f is s-linear iff fî is linear for all i in {1,..., s}.

Example We already know of a bilinear function from V × V 7→ R.

Basics Multilinearity Multilinear Functions

How do we test if a function f is linear?

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 5 / 21 Any inner product deﬁned on V is such a function, as

hαv1 + βv2, ui = αhv1, ui + βhv2, ui, and

hv, αu1 + βu2i = αhv, u1i + βhv, u2i

In other words, define fî (x) = f (v1,..., vi−1, x, vi+1 ..., vs ) , then f is s-linear iff fî is linear for all i in {1,..., s}.

Example We already know of a bilinear function from V × V 7→ R.

Basics Multilinearity Multilinear Functions

How do we test if a function f is linear? Fix all inputs of f except the i th input, if f is linear as a function of this input, then f is multilinear.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 5 / 21 Any inner product deﬁned on V is such a function, as

hαv1 + βv2, ui = αhv1, ui + βhv2, ui, and

hv, αu1 + βu2i = αhv, u1i + βhv, u2i

Example We already know of a bilinear function from V × V 7→ R.

Basics Multilinearity Multilinear Functions

How do we test if a function f is linear? Fix all inputs of f except the i th input, if f is linear as a function of this input, then f is multilinear.

In other words, define fî (x) = f (v1,..., vi−1, x, vi+1 ..., vs ) , then f is s-linear iff fî is linear for all i in {1,..., s}.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 5 / 21 Any inner product deﬁned on V is such a function, as

hαv1 + βv2, ui = αhv1, ui + βhv2, ui, and

hv, αu1 + βu2i = αhv, u1i + βhv, u2i

Basics Multilinearity Multilinear Functions

How do we test if a function f is linear? Fix all inputs of f except the i th input, if f is linear as a function of this input, then f is multilinear.

In other words, define fî (x) = f (v1,..., vi−1, x, vi+1 ..., vs ) , then f is s-linear iff fî is linear for all i in {1,..., s}.

Example We already know of a bilinear function from V × V 7→ R.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 5 / 21 Basics Multilinearity Multilinear Functions

How do we test if a function f is linear? Fix all inputs of f except the i th input, if f is linear as a function of this input, then f is multilinear.

In other words, define fî (x) = f (v1,..., vi−1, x, vi+1 ..., vs ) , then f is s-linear iff fî is linear for all i in {1,..., s}.

Example We already know of a bilinear function from V × V 7→ R. Any inner product deﬁned on V is such a function, as

hαv1 + βv2, ui = αhv1, ui + βhv2, ui, and

hv, αu1 + βu2i = αhv, u1i + βhv, u2i

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 5 / 21 Basics Multilinearity Multilinear Functions

How do we test if a function f is linear? Fix all inputs of f except the i th input, if f is linear as a function of this input, then f is multilinear.

In other words, define fî (x) = f (v1,..., vi−1, x, vi+1 ..., vs ) , then f is s-linear iff fî is linear for all i in {1,..., s}.

Example We already know of a bilinear function from V × V 7→ R. Any inner product deﬁned on V is such a function, as

hαv1 + βv2, ui = αhv1, ui + βhv2, ui, and

hv, αu1 + βu2i = αhv, u1i + βhv, u2i

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 5 / 21 Basics Multilinearity Multilinear Functions

How do we test if a function f is linear? Fix all inputs of f except the i th input, if f is linear as a function of this input, then f is multilinear.

In other words, define fî (x) = f (v1,..., vi−1, x, vi+1 ..., vs ) , then f is s-linear iff fî is linear for all i in {1,..., s}.

Example We already know of a bilinear function from V × V 7→ R. Any inner product deﬁned on V is such a function, as

hαv1 + βv2, ui = αhv1, ui + βhv2, ui, and

hv, αu1 + βu2i = αhv, u1i + βhv, u2i

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 5 / 21 Basics Multilinearity Multilinear Functions

How do we test if a function f is linear? Fix all inputs of f except the i th input, if f is linear as a function of this input, then f is multilinear.

In other words, define fî (x) = f (v1,..., vi−1, x, vi+1 ..., vs ) , then f is s-linear iff fî is linear for all i in {1,..., s}.

Example We already know of a bilinear function from V × V 7→ R. Any inner product deﬁned on V is such a function, as

hαv1 + βv2, ui = αhv1, ui + βhv2, ui, and

hv, αu1 + βu2i = αhv, u1i + βhv, u2i

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 5 / 21 Basics Multilinearity Multilinear Functions

How do we test if a function f is linear? Fix all inputs of f except the i th input, if f is linear as a function of this input, then f is multilinear.

In other words, define fî (x) = f (v1,..., vi−1, x, vi+1 ..., vs ) , then f is s-linear iff fî is linear for all i in {1,..., s}.

Example We already know of a bilinear function from V × V 7→ R. Any inner product deﬁned on V is such a function, as

hαv1 + βv2, ui = αhv1, ui + βhv2, ui, and

hv,α u1 + βu2i = αhv, u1i + βhv, u2i

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 5 / 21 Example s For any collection of vector spaces {Vi }i=1, and any collection of linear functions fi : Vi 7→ R, the function s Y f (v1,... vs ) = (fi (vi )) i=1 is s-linear.

Basics Multilinearity More Examples

Example When treated as a function of the columns (or rows) of an n × n matrix, the determinant is n-linear.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 6 / 21 Basics Multilinearity More Examples

Example When treated as a function of the columns (or rows) of an n × n matrix, the determinant is n-linear.

Example s For any collection of vector spaces {Vi }i=1, and any collection of linear functions fi : Vi 7→ R, the function s Y f (v1,... vs ) = (fi (vi )) i=1 is s-linear.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 6 / 21 Consider the set of all linear functions f : V 7→ R, denoted L(V : R). What does this set look like? R is a vector space of dimension 1, so L(V : R) is the set of all linear transformations from an N-dimensional vector space to a 1-dimensional vector space. We can thus represent every element of L(V : R) as a 1 × N matrix, otherwise known as a row vector.

Deﬁnition ∗ L(V : R) is the dual space of V , and is denoted V .

Basics Dual Space Dual Space

Fix a vector space V over R, where dim(V ) = N.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 7 / 21 What does this set look like? R is a vector space of dimension 1, so L(V : R) is the set of all linear transformations from an N-dimensional vector space to a 1-dimensional vector space. We can thus represent every element of L(V : R) as a 1 × N matrix, otherwise known as a row vector.

Deﬁnition ∗ L(V : R) is the dual space of V , and is denoted V .

Basics Dual Space Dual Space

Fix a vector space V over R, where dim(V ) = N. Consider the set of all linear functions f : V 7→ R, denoted L(V : R).

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 7 / 21 R is a vector space of dimension 1, so L(V : R) is the set of all linear transformations from an N-dimensional vector space to a 1-dimensional vector space. We can thus represent every element of L(V : R) as a 1 × N matrix, otherwise known as a row vector.

Deﬁnition ∗ L(V : R) is the dual space of V , and is denoted V .

Basics Dual Space Dual Space

Fix a vector space V over R, where dim(V ) = N. Consider the set of all linear functions f : V 7→ R, denoted L(V : R). What does this set look like?

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 7 / 21 We can thus represent every element of L(V : R) as a 1 × N matrix, otherwise known as a row vector.

Deﬁnition ∗ L(V : R) is the dual space of V , and is denoted V .

Basics Dual Space Dual Space

Fix a vector space V over R, where dim(V ) = N. Consider the set of all linear functions f : V 7→ R, denoted L(V : R). What does this set look like? R is a vector space of dimension 1, so L(V : R) is the set of all linear transformations from an N-dimensional vector space to a 1-dimensional vector space.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 7 / 21 Deﬁnition ∗ L(V : R) is the dual space of V , and is denoted V .

Basics Dual Space Dual Space

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 7 / 21 Basics Dual Space Dual Space

Deﬁnition ∗ L(V : R) is the dual space of V , and is denoted V .

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 7 / 21 v ∗ denotes an arbitrary element of V ∗, rather than the conjugate transpose of v ∈ V . We keep this distinction in order to preserve generality. Elements of V are vectors while elements of V ∗ are covectors (V ∗)∗ is identical to V . Notation: hv ∗, vi denotes the value of v ∗ evaluated at v. For our purposes, consider it the inner product of v and (v ∗)T .

Basics Dual Space Dual Space

V ∗ =∼ V , as they are both vector spaces of dimension N.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 8 / 21 We keep this distinction in order to preserve generality. Elements of V are vectors while elements of V ∗ are covectors (V ∗)∗ is identical to V . Notation: hv ∗, vi denotes the value of v ∗ evaluated at v. For our purposes, consider it the inner product of v and (v ∗)T .

Basics Dual Space Dual Space

V ∗ =∼ V , as they are both vector spaces of dimension N. v ∗ denotes an arbitrary element of V ∗, rather than the conjugate transpose of v ∈ V .

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 8 / 21 Elements of V are vectors while elements of V ∗ are covectors (V ∗)∗ is identical to V . Notation: hv ∗, vi denotes the value of v ∗ evaluated at v. For our purposes, consider it the inner product of v and (v ∗)T .

Basics Dual Space Dual Space

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 8 / 21 (V ∗)∗ is identical to V . Notation: hv ∗, vi denotes the value of v ∗ evaluated at v. For our purposes, consider it the inner product of v and (v ∗)T .

Basics Dual Space Dual Space

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 8 / 21 Notation: hv ∗, vi denotes the value of v ∗ evaluated at v. For our purposes, consider it the inner product of v and (v ∗)T .

Basics Dual Space Dual Space

V ∗ =∼ V , as they are both vector spaces of dimension N. v ∗ denotes an arbitrary element of V ∗, rather than the conjugate transpose of v ∈ V . We keep this distinction in order to preserve generality. Elements of V are vectors while elements of V ∗ are covectors (V ∗)∗ is identical to V .

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 8 / 21 Basics Dual Space Dual Space

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 8 / 21 p Tq (V ) forms a vector space under natural operations, as the cartesian product of n vector spaces over F forms a vector space over F × · · · × F . 0 ∗ 1 1 ∼ T1 (V ) = V , T0 (V ) = V , and T1 (V ) = L(V : V ). That is, lower order tensors are the 1 and 2 dimensional arrays we usually work with.

Tensors Tensors

Deﬁnition A tensor of order (p, q) is a (p + q)-linear map

∗ ∗ T : V × · · · × V × V × · · · × V 7→ R. | {z } | {z } p times q times

p We denote the set of all order (p, q) tensors on V as Tq (V )

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 9 / 21 0 ∗ 1 1 ∼ T1 (V ) = V , T0 (V ) = V , and T1 (V ) = L(V : V ). That is, lower order tensors are the 1 and 2 dimensional arrays we usually work with.

Tensors Tensors

Deﬁnition A tensor of order (p, q) is a (p + q)-linear map

∗ ∗ T : V × · · · × V × V × · · · × V 7→ R. | {z } | {z } p times q times

p We denote the set of all order (p, q) tensors on V as Tq (V )

p Tq (V ) forms a vector space under natural operations, as the cartesian product of n vector spaces over F forms a vector space over F × · · · × F .

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 9 / 21 That is, lower order tensors are the 1 and 2 dimensional arrays we usually work with.

Tensors Tensors

Deﬁnition A tensor of order (p, q) is a (p + q)-linear map

∗ ∗ T : V × · · · × V × V × · · · × V 7→ R. | {z } | {z } p times q times

p We denote the set of all order (p, q) tensors on V as Tq (V )

p Tq (V ) forms a vector space under natural operations, as the cartesian product of n vector spaces over F forms a vector space over F × · · · × F . 0 ∗ 1 1 ∼ T1 (V ) = V , T0 (V ) = V , and T1 (V ) = L(V : V ).

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 9 / 21 Tensors Tensors

Deﬁnition A tensor of order (p, q) is a (p + q)-linear map

∗ ∗ T : V × · · · × V × V × · · · × V 7→ R. | {z } | {z } p times q times

p We denote the set of all order (p, q) tensors on V as Tq (V )

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 9 / 21 Example ∗ ∗ Consider the function (v ⊗ v ): V × V 7→ R, deﬁned as

(v ⊗ v ∗)(u∗, u) = hu∗, vihv ∗, ui

Recall that this is a special case of our earlier example, as (v ⊗ v ∗) is the product of two linear functions.

Tensors Tensor Product

Given any two vectors v ∗ ∈ V ∗ and v ∈ V , we can construct a tensor of order (1, 1).

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 10 / 21 Recall that this is a special case of our earlier example, as (v ⊗ v ∗) is the product of two linear functions.

Tensors Tensor Product

Given any two vectors v ∗ ∈ V ∗ and v ∈ V , we can construct a tensor of order (1, 1). Example ∗ ∗ Consider the function (v ⊗ v ): V × V 7→ R, deﬁned as

(v ⊗ v ∗)(u∗, u) = hu∗, vihv ∗, ui

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 10 / 21 Recall that this is a special case of our earlier example, as (v ⊗ v ∗) is the product of two linear functions.

Tensors Tensor Product

Given any two vectors v ∗ ∈ V ∗ and v ∈ V , we can construct a tensor of order (1, 1). Example ∗ ∗ Consider the function (v ⊗ v ): V × V 7→ R, deﬁned as

(v ⊗ v ∗)(u∗, u) = hu∗, vihv ∗, ui

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 10 / 21 Recall that this is a special case of our earlier example, as (v ⊗ v ∗) is the product of two linear functions.

Tensors Tensor Product

Given any two vectors v ∗ ∈ V ∗ and v ∈ V , we can construct a tensor of order (1, 1). Example ∗ ∗ Consider the function (v ⊗ v ): V × V 7→ R, deﬁned as

(v ⊗ v ∗)(u∗, u) = hu∗, vihv ∗, ui

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 10 / 21 Tensors Tensor Product

Given any two vectors v ∗ ∈ V ∗ and v ∈ V , we can construct a tensor of order (1, 1). Example ∗ ∗ Consider the function (v ⊗ v ): V × V 7→ R, deﬁned as

(v ⊗ v ∗)(u∗, u) = hu∗, vihv ∗, ui

Recall that this is a special case of our earlier example, as (v ⊗ v ∗) is the product of two linear functions.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 10 / 21 Deﬁnition p j p For any collection of vectors {vi }i=1, and vectors {v }j=1, their tensor product is the function

1 q ∗ ∗ v1 ⊗ · · · ⊗ vp ⊗ v ⊗ · · · ⊗ v : V × · · · × V × V × · · · × V 7→ R, | {z } | {z } p times q times

deﬁned as

1 q 1 p (v1 ⊗ · · · ⊗ vp ⊗ v ⊗ · · · ⊗ v )(u ,... u , u1,... uq) 1 p 1 q = hu , v1i · · · hu , vpihv , u1i · · · hv , uqi

Tensors Tensor Product

There is a very natural extension of the operator ⊗ to allow any number of vectors and covectors.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 11 / 21 Tensors Tensor Product

There is a very natural extension of the operator ⊗ to allow any number of vectors and covectors. Deﬁnition p j p For any collection of vectors {vi }i=1, and vectors {v }j=1, their tensor product is the function

1 q ∗ ∗ v1 ⊗ · · · ⊗ vp ⊗ v ⊗ · · · ⊗ v : V × · · · × V × V × · · · × V 7→ R, | {z } | {z } p times q times deﬁned as

1 q 1 p (v1 ⊗ · · · ⊗ vp ⊗ v ⊗ · · · ⊗ v )(u ,... u , u1,... uq) 1 p 1 q = hu , v1i · · · hu , vpihv , u1i · · · hv , uqi

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 11 / 21 Tensors Tensor Product

1 q ∗ ∗ v1 ⊗ · · · ⊗ vp ⊗ v ⊗ · · · ⊗ v : V × · · · × V × V × · · · × V 7→ R, | {z } | {z } p times q times deﬁned as

1 q 1 p (v1 ⊗ · · · ⊗ vp ⊗ v ⊗ · · · ⊗ v )(u ,... u , u1,... uq) 1 p 1 q = hu , v1i · · · hu , vpihv , u1i · · · hv , uqi

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 11 / 21 Tensors Tensor Product

1 q ∗ ∗ v1 ⊗ · · · ⊗ vp ⊗ v ⊗ · · · ⊗ v : V × · · · × V × V × · · · × V 7→ R, | {z } | {z } p times q times deﬁned as

1 q 1 p (v1 ⊗ · · · ⊗ vp ⊗ v ⊗ · · · ⊗ v )(u ,... u , u1,... uq) 1 p 1 q = hu , v1i · · · hu , vpihv , u1i · · · hv , uqi

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 11 / 21 In general, not all tensors are simple. p However, we can use simple tensors to build a basis of Tq (V ).

Theorem N ∗ For any basis of V , B = {ei }i=1, there exists a unique dual basis of V j N relative to B, denoted {e }j=1 and deﬁned as ( j j 1, if i = j he , ei i = δ = . i 0, if i 6= j

p Tensors Basis of Tq (V )

Tensors formed from the tensor product of vectors and covectors are called simple tensors.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 12 / 21 p However, we can use simple tensors to build a basis of Tq (V ).

Theorem N ∗ For any basis of V , B = {ei }i=1, there exists a unique dual basis of V j N relative to B, denoted {e }j=1 and deﬁned as ( j j 1, if i = j he , ei i = δ = . i 0, if i 6= j

p Tensors Basis of Tq (V )

Tensors formed from the tensor product of vectors and covectors are called simple tensors. In general, not all tensors are simple.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 12 / 21 Theorem N ∗ For any basis of V , B = {ei }i=1, there exists a unique dual basis of V j N relative to B, denoted {e }j=1 and deﬁned as ( j j 1, if i = j he , ei i = δ = . i 0, if i 6= j

p Tensors Basis of Tq (V )

Tensors formed from the tensor product of vectors and covectors are called simple tensors. In general, not all tensors are simple. p However, we can use simple tensors to build a basis of Tq (V ).

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 12 / 21 p Tensors Basis of Tq (V )

Tensors formed from the tensor product of vectors and covectors are called simple tensors. In general, not all tensors are simple. p However, we can use simple tensors to build a basis of Tq (V ).

Theorem N ∗ For any basis of V , B = {ei }i=1, there exists a unique dual basis of V j N relative to B, denoted {e }j=1 and deﬁned as ( j j 1, if i = j he , ei i = δ = . i 0, if i 6= j

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 12 / 21 The size of this basis is N(p+q). Simpliﬁed proof in my paper, but our relation of linear dependence is nasty (p + q nested sums).

p Tensors Basis of Tq (V )

Theorem N j N For any basis {ei }i=1 of V , and the corresponding dual basis {e }j=1 of V ∗, the set of simple tensors

j1 jq {ei1 ⊗ · · · ⊗ eip ⊗ e ⊗ · · · ⊗ e }

p q for all combinations of {ik }k=1 ∈ {1,..., N} and {jz }z=1 ∈ {1,..., N}, p forms a basis of Tq (V ).

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 13 / 21 Simpliﬁed proof in my paper, but our relation of linear dependence is nasty (p + q nested sums).

p Tensors Basis of Tq (V )

Theorem N j N For any basis {ei }i=1 of V , and the corresponding dual basis {e }j=1 of V ∗, the set of simple tensors

j1 jq {ei1 ⊗ · · · ⊗ eip ⊗ e ⊗ · · · ⊗ e }

p q for all combinations of {ik }k=1 ∈ {1,..., N} and {jz }z=1 ∈ {1,..., N}, p forms a basis of Tq (V ).

The size of this basis is N(p+q).

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 13 / 21 p Tensors Basis of Tq (V )

Theorem N j N For any basis {ei }i=1 of V , and the corresponding dual basis {e }j=1 of V ∗, the set of simple tensors

j1 jq {ei1 ⊗ · · · ⊗ eip ⊗ e ⊗ · · · ⊗ e }

p q for all combinations of {ik }k=1 ∈ {1,..., N} and {jz }z=1 ∈ {1,..., N}, p forms a basis of Tq (V ).

The size of this basis is N(p+q). Simpliﬁed proof in my paper, but our relation of linear dependence is nasty (p + q nested sums).

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 13 / 21 Can be represented by 2-dimensional array, but we consider this product to be a list of lists, table of lists, list of tables, table of tables, ect.

Yes, but ﬁrst... Deﬁnition

For two matrices Am×n and Bp×q, the Kronecker product of A and B is deﬁned as   [A]1,1B [A]1,2B ··· [A]1,nB  [A]2,1B [A]2,2B ··· [A]2,nB  A ⊗ B =    . . .. .   . . . .  [A]m,1B [A]m,2B ··· [A]m,nB

Component Representation Kronecker Product Kronecker Product

Can we use this basis to ﬁnd a component representation of tensors in p Tq (V )?

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 14 / 21 Can be represented by 2-dimensional array, but we consider this product to be a list of lists, table of lists, list of tables, table of tables, ect.

Component Representation Kronecker Product Kronecker Product

Can we use this basis to find a component representation of tensors in p Tq (V )? Yes, but first... Definition

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 14 / 21 Component Representation Kronecker Product Kronecker Product

Can we use this basis to find a component representation of tensors in p Tq (V )? Yes, but first... Definition

Can be represented by 2-dimensional array, but we consider this product to be a list of lists, table of lists, list of tables, table of tables, ect.

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 14 / 21 For vectors, this is exactly how we deﬁne components (hv, ei i = [v]i ). If T is a simple tensor, then the (p + q)-dimensional array formed by Ai1,...,ip is equal to the Kronecker product of the vectors and covectors j1,...,jq which make up T .

Component Representation Components Components as Basis Images

Deﬁnition p In general, we deﬁne the components of T ∈ Tq (V ) to be the (p + q)-indexed scalars

i ,...,i A 1 p = A(ei1 ,..., eip , e ,..., e ). j1,...,jq j1 jq

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 15 / 21 If T is a simple tensor, then the (p + q)-dimensional array formed by Ai1,...,ip is equal to the Kronecker product of the vectors and covectors j1,...,jq which make up T .

Component Representation Components Components as Basis Images

Deﬁnition p In general, we deﬁne the components of T ∈ Tq (V ) to be the (p + q)-indexed scalars

i ,...,i A 1 p = A(ei1 ,..., eip , e ,..., e ). j1,...,jq j1 jq

For vectors, this is exactly how we deﬁne components (hv, ei i = [v]i ).

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 15 / 21 Component Representation Components Components as Basis Images

Deﬁnition p In general, we deﬁne the components of T ∈ Tq (V ) to be the (p + q)-indexed scalars

i ,...,i A 1 p = A(ei1 ,..., eip , e ,..., e ). j1,...,jq j1 jq

For vectors, this is exactly how we deﬁne components (hv, ei i = [v]i ). If T is a simple tensor, then the (p + q)-dimensional array formed by Ai1,...,ip is equal to the Kronecker product of the vectors and covectors j1,...,jq which make up T .

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 15 / 21 1,1 1 A1 = A(e , e1, e1) 1 1 1 = h 1 0 , ih 2 1 , ih 1 3 , i 1 0 0 = 2 2,1 1 A1 = A(e , e2, e1) = 1 1,2 1 A1 = A(e , e1, e2) = 6 . .

Component Representation Comparison Example

1 For V = 2, consider the vectors u = , v ∗ = 2 1 , and R 1 w ∗ = 1 3 . Let A = u ⊗ v ∗ ⊗ w ∗ and consider

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 16 / 21 Component Representation Comparison Example

1 For V = 2, consider the vectors u = , v ∗ = 2 1 , and R 1 w ∗ = 1 3 . Let A = u ⊗ v ∗ ⊗ w ∗ and consider

1,1 1 A1 = A(e , e1, e1) 1 1 1 = h 1 0 , ih 2 1 , ih 1 3 , i 1 0 0 = 2 2,1 1 A1 = A(e , e2, e1) = 1 1,2 1 A1 = A(e , e1, e2) = 6 . .

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 16 / 21 1 u ⊗ v ∗ ⊗ w ∗ = ⊗ 2 1 ⊗ 1 3 1 2 1 = ⊗ 1 3 2 1 2 1 3 1 1 3 = 2 1 3 1 1 3 2 6 1 3 = , 2 6 1 3

Component Representation Comparison Example

Or, we can take the Kronecker product u ⊗ v ∗ ⊗ w ∗ to get

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 17 / 21 Component Representation Comparison Example

Or, we can take the Kronecker product u ⊗ v ∗ ⊗ w ∗ to get

1 u ⊗ v ∗ ⊗ w ∗ = ⊗ 2 1 ⊗ 1 3 1 2 1 = ⊗ 1 3 2 1 2 1 3 1 1 3 = 2 1 3 1 1 3 2 6 1 3 = , 2 6 1 3

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 17 / 21 Component Representation Comparison

Either way, we get

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 18 / 21 Component Representation Comparison

Either way, we get

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 18 / 21 Component Representation Comparison References

1. INTRODUCTION TO VECTORS AND TENSORS, Linear and Multilinear Algebra, Volume 1. Ray M. Bowen, C.-C. Wang. Updated 2010. (Freely available at: http://repository.tamu.edu/bitstream/handle/ 1969.1/2502/IntroductionToVectorsAndTensorsVol1.pdf?s..)

2. Matrix Analysis for Scientists and Engineers, Alan J. Laub. (Relevant chapter at http://www.siam.org/books/textbooks/OT91sample.pdf)

3. Notes on Tensor Products and the Exterior Algebra, K. Purbhoo. Updated 2008 (Lecture notes, available at http://www.math.uwaterloo.ca/ kpurbhoo/spring2012- math245/tensor.pdf)

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 19 / 21 Component Representation Comparison

5. Tensor Spaces and Exterior Algebra. Takeo Yokonuma. English version translated 1991.

7. Multilinear Algebra. Werner Greub. Springer, updated 2013.

8. Eigenvalues of the Adjacency Tensor on Products of Hypergraphs. Kelly J. Pearson, Tan Zhang. October, 2012 http://www.m- hikari.com/ijcms/ijcms-2013/1-4-2013/pearsonIJCMS1-4-2013.pdf

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 20 / 21 Component Representation Comparison

The End

Davis Shurbert (UPS) Multilinear Algebra April 17, 2014 21 / 21