Modelling 1 SUMMER TERM 2020
′ 푓 퐱 푥1 = 퐯, 퐛1 = 2 퐯, 퐛1 푥1 = 1.5
′ 퐛1 퐛1 푥2 = 1 ′ 푥2 = 퐯, 퐛ퟐ 퐛2 ′ 퐛2 = 2 퐯, 퐛ퟐ
ADDENDUM Co- and Contravariance
Michael Wand · Institut für Informatik · . Covariance & Contravariance Representing Vectors
Two operations in linear algebra 푓 퐱 ▪ Contravariant: Linear combination of vectors 푥1 = 3 푛 푥1 퐛 퐯 = 푥푖퐛푖 → 퐯 ≡ ⋮ 1 푥2 = 2 푥푛 푖=1 퐛2 ▪ Covariant:
Projection on vectors (w/scalar product) 푥1 = 퐯, 퐛1 퐯, 퐛1 퐯 ≡ ⋮ 퐯, 퐛푛 퐛1 푥2 = 퐯, 퐛ퟐ 퐛2 Where is the difference?
Change of basis 푓 퐱 ▪ Contravariant: 푥1 = 1.5
▪ Keep same output vector: 퐛1 −1 푥2 = 1 퐛푖 → 퐓퐛푖 requires 퐱 → 퐓 퐱 퐛2 ▪ Covariant: ′ 퐱, 퐛1 푥1 = 퐯, 퐛1 푓 퐱 = ⋮ = 2 퐯, 퐛1 퐱, 퐛푛
▪ Keep same output vector: ′ 퐛1 ′ 퐛 → 퐓퐛 requires 퐱 → 퐓퐱 푥2 = 퐯, 퐛ퟐ 푖 푖 ′ 퐛2 = 2 퐯, 퐛ퟐ Awesome Video
Tensors, Co-/Contra-Variance ▪ „Tensors Explained Intuitively: Covariant, Contravariant, Rank“ Physics Videos by Eugene Khutoryansky https://www.youtube.com/watch?v=CliW7kSxxWU Covariance & Contravariance
Linear map 퐟: 푉1 → 푉2 Matrix representation (standard basis) 퐌 ∈ ℝ푑1×푑2 Change of basis | | | | 퐁 = 퐛 1 ⋯ 퐛 1 , 퐁 = 퐛 2 ⋯ 퐛 2 1 1 푑1 2 1 푑2 | | | |
New matrix representation (bases 퐁1, 퐁2)
−1 푑1×푑2 퐁2 퐌퐁1 ∈ ℝ Covariance & Contravariance
Situation covariant 퐟: 푉1 → 푉2 퐱 퐈 퐌 퐈 mapping M
−1 퐁1 퐁2 퐌퐁1 퐁2 퐲 contravariant Transformation law
▪ Input vectors 퐱 (퐌퐱): 퐱 퐁1 = 퐁1퐱 퐈 (covariant) −1 ▪ Output vectors 퐲 = 퐌퐱: 퐲 퐁2 = 퐁2 퐲 퐈 (contravariant) Covariance & Contravariance
Situation covariant 퐟: 푉1 → 푉2 퐱 퐈 퐌 퐈 mapping M
−1 퐁1 퐁2 퐌퐁1 퐁2 퐲 contravariant Transformation law
▪ Input vectors 퐱 (퐌퐱): 퐱 퐁1 = 퐁1퐱 퐈 (covariant) −1 ▪ Output vectors 퐲 = 퐌퐱: 퐲 퐁2 = 퐁2 퐲 퐈 (contravariant) Covariance & Contravariance
−1 푓 퐱 ← 퐁2 퐌퐁1 ← 퐱
−1 퐁2 퐌퐁1 transforms row-vectors
−1 퐁2 퐌 퐁1
transforms column-vectors Scalar Product covariant 퐱 퐲 General scalar product 푑 퐱, 퐲 ∈ ℝ mapping M 퐱, 퐲 = 퐱T 퐐 퐲, T 퐐 = 퐐 , 퐐 > 0 ⟨퐱, 퐲⟩ scalar (no coordinate system) Change of basis 퐈 퐁 퐱, 퐲 = 퐱T 퐐 퐲 퐱 → 퐁퐱, 퐈 퐲 → 퐁퐲 T 퐓 퐱, 퐲 퐁 = 퐱 ⋅ 퐁 ⋅ 퐐 ⋅ 퐁 ⋅ 퐲 Three shades of dual PCA, SVD, MDS Inputs and Outputs Input (“covariant”) side of the matrix
Output (“contravariant”) side of the matrix
Squaring a Matrix ▪ Possibility 1: 퐀 ⋅ 퐀푇 ⋅ =
▪ Possibility 2: 퐀푇 ⋅ 퐀 ⋅ = A Story about Dual Spaces VT X U D 1 0 0 0 0 0 = 0 2 0 0 0 0 0 0 3 0 0 0 SVD 0 0 0 4 0 0
XT X U D2 UT 1² 0 0 0 0 2² 0 0 = PCA 0 0 3² 0 0 0 0 4²
T X V D2 VT
X 1² 0 0 0 0 0 0 2² 0 0 0 0 0 0 3² 0 0 0 = MDS 0 0 0 4² 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Tensors: Multi-Linear Maps Tensors
General notion: Tensor ▪ Tensor: multi-linear form*) with 푟 ∈ ℕ input vectors
퐓: 푉1, … , 푉푛, 푉푛+1, … , 푉푟 → 퐹 (usually: field 퐹 = ℝ)
▪ “Rank 푟” tensor ▪ Linear in each input (when keeping the rest constant) ▪ Each input can be covariant or contravariant ▪ (푛, 푚) tensor ▪ 푟 = 푛 + 푚 ▪ 푛 – contravariant inputs ▪ 푚 – covariant inputs
*) i.e.: multi-linear mapping / function Tensors
Representation ▪ Represented as r-dimensional array 푡푖1,푖2,…,푖푛 푗1,푗2,…,푗푚 ▪ n – contravariant inputs (“indices”) ▪ m – covariant inputs (“indices”) ▪ Mapping rule 퐓 퐯 1 , … , 퐯 푛 , 퐰 1 , … , 퐰 푚 ≔ ⋯ ⋯ 푣 1 ⋯ 푣 푛 푤 1 ⋯ 푤 푚 푡푖1,푖2,…,푖푛 푖1 푖푛 푗1 푗푚 푗1,푗2,…,푗푚 푖1=0,…,푛푖1 푖푛=0,…,푛푖푛 푗1=0,…,푛푗1 푗푚=0,…,푛푗푚
(Note: writing the application of 퐓 as multi-linear mapping here) Tensors
Remarks ▪ No difference between co-/contravariant dimensions in terms of numerical representation ▪ Generalization of matrix
Example 푧1 푥1 푦1 퐓 , , 푧2 = 42푥1푦1푧1 + 23푥1푦1푧2 + ⋯ + 16푥2푦2푧3 푥2 푦2 푧3 ▪ Purely linear polynomial in each input parameter when all others remain constant. ▪ 3D array - 2 × 2 × 3 combinations of coefficients Einstein Notation
Example: Quadratic polynomial ℝ3 → ℝ3 푝푗 퐱 = 퐱T퐀퐱 + 퐛퐱 + c 3 3 3 푗 푗 푗 푗 푝 = 푥푘푥푙 푎푘푙 + 푥푘푏푘 + c 푘=1 푙=1 푘=1
Tensor notation
▪ Input: 푥푖, 푖 = 1. . 3 ▪ Quadratic form (Matrix) 퐀: 푎푘푙 푗 ▪ Output: 푝 , 푗 = 1. . 3 ▪ Linear form (Co-Vector) 퐛: 푏푘 ▪ Constant 푐 Einstein Notation
Example: Quadratic polynomial ℝ3 → ℝ3 푝푗 퐱 = 퐱T퐀퐱 + 퐛퐱 + c
Einstein notation (implicit sums over common indices) 푗 푗 푗 푗 푝 = 푥푘푥푙푎푘푙 + 푥푘푏푘 + c
Tensor notation
▪ Input: 푥푖, 푖 = 1. . 3 ▪ Quadratic form (Matrix) 퐀: 푎푘푙 푗 ▪ Output: 푝 , 푗 = 1. . 3 ▪ Linear form (Co-Vector) 퐛: 푏푘 ▪ Constant 푐 Further Examples
Examples ▪ (푛, 푚)-tensor ▪ n contravariant “indices” ▪ m covariant “indices” ▪ Matrix: (1,1)-tensor ▪ Scalar product: (0,2)-tensor ▪ Vector: (1,0)-tensor ▪ Co-vector: (0,1)-tensor ▪ Geometric vectors: (1,0)-tensors Covariant Derivatives?
Examples ▪ Geometric vectors: (1,0) tensors ▪ Derivatives*) / gradients / normal vectors: (0,1) tensors *) to be precise: ▪ Spatial derivatives co-vary for changes of the basis of the space – 푓: ℝ푛 → ℝ, 푓 퐱 = 퐲, ⇒ 훻푓 is covariant (0,1). – Examples: Gradient vector ▪ Derivatives of vector functions by unrelated dimensions remain contravariant d – 푓: ℝ → ℝ푛, 푓 푡 = 퐲, ⇒ 푓 remains contravariant (1,0). d푡 – Examples: velocity, acceleration 푛 푚 ▪ Mixed case: 푓: ℝ → ℝ , 훻푓 = 퐽푓 is a (1,1)-tensor (1,1) Example: Plane Equation
Plane equation(s) ▪ Parametric: 퐱 = 휆1퐫1 + 휆2퐫2 + 퐨 ▪ Implicit: 퐧, 퐱 − 푑 = 0
Transformation 퐱 → 퐓퐱 ▪ Parametric: 퐓퐱 = 퐓 휆1퐫1 + 휆2퐫2 + 퐨 = 휆1퐓퐫1 + 휆2퐓퐫2 + 퐓퐨 ▪ Implicit: 퐧, 퐓퐱 − 푑 = 퐧T퐓 퐱 − 푑0 More Structure?
Connecting ▪ Integrals ▪ Derivatives ▪ In higher dimensions ▪ And their transformation rules
“Exterior Calculus” ▪ Unified framework ▪ Beyond this lecture (take a real math course :-) ) Vectors & Covectors in Function Spaces Remark: Function Spaces
Discrete vector spaces ▪ Picking entries by index is a linear operation ▪ Can be represented by projection to vector (multiplication with “co-vector”)
Example
▪ 퐱 = (푥1, 푥2, 푥3, 푥4, 푥5)
▪ 퐱 ↦ 푥4 is a linear maps ▪ Represented by 0,0,0,1,0 , 퐱
▪ “Linear form”: 퐱 ↦ 0,0,0,1,0 , 퐱 , ퟎ T ퟎ in short, ⋅, 0,0,0,1,0 , shorter: 0,0,0,1,0 = ퟎ ퟏ ퟎ Linear Forms in Function Spaces
In function spaces ▪ Picking entries by x-axis is a linear operation ▪ Cannot be represented by projection to another function (multiplication with “co-vector”)
푔 1, if 푥 = 4 푔 푥 = ቊ 푓 0, elsewhere
Example න 푓 푥 푔 푥 푑푥 = 0 ℝ ▪ 푓: ℝ → ℝ, 푓 푥 = sin 푥 ▪ 퐿: ℝ → ℝ → ℝ, 퐿: 푓 ↦ 푓 4.0 is a linear map ▪ A function 푔 with 푔, 푓 = 푓 4.0 does not exist Dirac’s “Delta Function”
f(x) f(x)
x x
f(x) f(x)
න 푓 푥 푑푥 = 1 x x Ω
Dirac Delta “Function” ▪ ℝ 훿 푥 푑푥 = 1, zero everywhere but at 푥 = 0 ▪ Idealization (“distribution”) – think of very sharp peak Distributions
Distributions ▪ Adding all linear forms to the vector space ▪ All linear mappings from the function space to ℝ ▪ This makes the situation symmetric ▪ 훿 is a distribution, not a (traditional) function
Formalization ▪ Different approaches (details beyond our course) ▪ Limits of “bumps” ▪ Space of linear-forms (“co-vectors”, “dual functions”) ▪ Difference of complex functions on Riemann sphere (exotic)