My PhD Notes Nicholas Rotella August 12, 2012 Forward Congratulations! You've (somehow) managed to stumble across my vast repository of knowledge on all sorts of fun and exciting topics somehow related to the work of this robotics PhD student. This document contains all of my notes since I started my PhD in 2012. Many portions have come from my study of various internet sources including others' personal notes; I found these to be extremely useful in my studies and so decided to put my notes online as well. Other portions have been transcribed by hand into LaTeX from texts and interspersed with notes and addendums which (I hope) provide additional insight or a different interpretation of the material (I'm working on adding references where applicable). In all cases, I do not profit off of the material found in this document; it is merely an informal place for me to collect interesting notes and thoughts for later use. Most of my notes are the product of years of taking courses, reading texts, scouring the web and spending time working out intuitive explanations for concepts which once stumped me and putting them into my own words. This document is constantly evolving and likely contains tons of errors at any given time; in any case, I hope it will be useful to you, dear reader, in some way. i Contents Forward i 1 Calculus 1 1.1 Fundamental Theorem of Calculus . .1 1.2 Mean Value Theorem . .1 1.3 Taylor Series Expansion . .2 1.4 Rules of Integration . .2 1.5 Gradients, Jacobians and Hessians . .4 1.5.1 The Gradient . .4 1.5.2 The Jacobian . .4 1.5.3 The Hessian . .5 1.6 Data Interpolation . .5 1.6.1 Interpolating MPC Solutions . .6 2 Linear Algebra 8 2.1 Basic Theory . .8 2.1.1 Matrix Multiplication . .8 2.1.2 Schwarz Inequality . .9 2.1.3 Triangle Inequality . .9 2.1.4 Gaussian Elimination . .9 2.1.5 LU Decomposition . .9 2.1.6 Gauss-Jordan Elimination . 10 2.1.7 Matrix Inverse . 10 2.1.8 Determinant . 10 2.1.9 Cramers Rule . 12 2.1.10 Permutation Matrix . 12 2.1.11 Reduced Row Echelon Form . 13 2.1.12 Vector Space . 13 2.1.13 Basis . 13 2.1.14 Span . 13 2.1.15 Subspace . 13 2.1.16 Orthogonal Subspaces . 14 2.1.17 Rank . 14 ii CONTENTS iii 2.1.18 Column Space . 14 2.1.19 Nullspace . 15 2.1.20 Row Space . 15 2.1.21 Left Nullspace . 15 2.1.22 Fundamental Theorem of Linear Algebra/The Four Subspaces . 16 2.1.23 Projections . 16 2.1.24 Least Squares . 16 2.1.25 Orthogonal Matrix . 17 2.1.26 Gram-Schmidt . 17 2.1.27 QR Decomposition . 18 2.1.28 Eigenvalues/Eigenvectors . 18 2.1.29 Similarity Transformations . 20 2.1.30 Singular Value Decomposition . 21 2.2 Complex Linear Algebra . 23 2.2.1 Complex Inner Product Spaces . 23 2.2.2 Unitary Transformations . 23 2.2.3 Unitary Similarity . 26 2.2.4 Hermitian Transformations . 28 2.2.5 Normal Linear Transformations . 28 2.3 Nearest Orthogonal Matrix . 30 2.4 Damped Least Squares . 31 2.5 Principal Component Analysis (PCA) . 31 2.6 Cholesky Decomposition . 32 2.6.1 Generating samples from a multivariate Gaussian . 33 2.7 Statistical Significance . 33 2.7.1 Mahalanobis Distance . 33 2.7.2 Chi-squared Distribution . 34 2.8 Skew-Symmetric Matrices . 35 2.9 Positive Definiteness . 36 2.10 Cayley-Hamilton Theorem . 36 2.10.1 Cayley-Hamilton Theorem (General Proof) . 37 2.11 Quadratic Forms, Norms, and Singular Values . 39 2.11.1 Vector Norms . 39 2.11.2 Matrix Norms . 39 2.11.3 Quadratic Forms . 40 2.12 Condition Number . 41 2.13 Least Squares and Related Decompositions . 42 2.13.1 Normal Equations . 42 2.13.2 QR Decomposition . 42 2.13.3 Singular Value Decomposition . 43 2.14 Conjugate Gradient Method . 44 2.15 Underdetermined Systems . 45 2.16 Projections Onto Subspaces . 46 CONTENTS iv 3 Differential Geometry 48 3.1 Curves, Surfaces and Manifolds . 48 3.1.1 Smoothness . 48 3.1.2 Functions on Manifolds . 49 3.2 Quaternions and Rotations . 49 3.2.1 Interpolating Rotations (SLERP) . 50 3.3 Lie Groups . 51 3.4 Principal Geodesic Analysis . 51 3.4.1 Principal Component Analysis (Review) . 52 3.4.2 Extension of PCA to manifolds . 54 3.4.3 Statistics of Manifold Data . 55 3.4.4 Geodesic Subspaces and Projection . 57 4 Dynamics 58 4.1 Intro to Analytical Dynamics . 58 4.2 Constraints . 59 4.2.1 Holonomic Constraints . 59 4.2.2 Nonholonomic Constraints . 60 4.3 Gauss' Principle . 61 4.4 The Fundamental Equation . 62 5 Systems and Control Theory 63 5.1 System Representations . 63 5.1.1 Properties of Linear Systems: . 64 5.1.2 Linearization of State Equations . 65 5.1.3 Transfer Functions of a State-Space Realization . 65 5.1.4 Realization of SISO Transfer Functions . 66 5.1.5 Equivalent Realizations . 70 5.2 Solutions of Linear Systems . 71 5.2.1 LTV Systems and The Peano-Baker Series . 71 5.2.2 Properties of the State Transition Matrix . 72 5.2.3 Solution of the Forced System . 73 5.3 Solution of LTI Systems . 74 5.3.1 The Matrix Exponential . 75 5.3.2 Discretization of LTI Systems . 78 5.4 Stability . 79 5.4.1 Internal Stability of Linear Systems . 79 5.4.2 Types of Stability . 80 5.4.3 The Routh Stability Criterion . 82 5.4.4 Lyapunov Stability Theory . 82 5.4.5 BIBO Stability . 85 5.5 Controllability and Observability . 86 5.5.1 Controllability . 87 5.5.2 Observability . 88 CONTENTS v 5.5.3 Duality . 89 5.5.4 Lyapunov Stability (updated) . ..