DOCSLIB.ORG

Lecture Notes for EE263

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Lecture Notes for EE263 Lecture Notes for EE263 Stephen Boyd Introduction to Linear Dynamical Systems Autumn 2007-08 Copyright Stephen Boyd. Limited copying or use for educational purposes is fine, but please acknowledge source, e.g., “taken from Lecture Notes for EE263, Stephen Boyd, Stanford 2007.” Contents Lecture 1 – Overview Lecture 2 – Linear functions and examples Lecture 3 – Linear algebra review Lecture 4 – Orthonormal sets of vectors and QR factorization Lecture 5 – Least-squares Lecture 6 – Least-squares applications Lecture 7 – Regularized least-squares and Gauss-Newton method Lecture 8 – Least-norm solutions of underdetermined equations Lecture 9 – Autonomous linear dynamical systems Lecture 10 – Solution via Laplace transform and matrix exponential Lecture 11 – Eigenvectors and diagonalization Lecture 12 – Jordan canonical form Lecture 13 – Linear dynamical systems with inputs and outputs Lecture 14 – Example: Aircraft dynamics Lecture 15 – Symmetric matrices, quadratic forms, matrix norm, and SVD Lecture 16 – SVD applications Lecture 17 – Example: Quantum mechanics Lecture 18 – Controllability and state transfer Lecture 19 – Observability and state estimation Lecture 20 – Some final comments Basic notation Matrix primer Crimes against matrices Solving general linear equations using Matlab Least-squares and least-norm solutions using Matlab Exercises EE263 Autumn 2007-08 Stephen Boyd Lecture 1 Overview • course mechanics • outline & topics • what is a linear dynamical system? • why study linear systems? • some examples 1–1 Course mechanics • all class info, lectures, homeworks, announcements on class web page: www.stanford.edu/class/ee263 course requirements: • weekly homework • takehome midterm exam (date TBD) • takehome final exam (date TBD) Overview 1–2 Prerequisites • exposure to linear algebra (e.g., Math 103) • exposure to Laplace transform, differential equations not needed, but might increase appreciation: • control systems • circuits & systems • dynamics Overview 1–3 Major topics & outline • linear algebra & applications • autonomous linear dynamical systems • linear dynamical systems with inputs & outputs • basic quadratic control & estimation Overview 1–4 Linear dynamical system continuous-time linear dynamical system (CT LDS) has the form dx = A(t)x(t)+ B(t)u(t), y(t)= C(t)x(t)+ D(t)u(t) dt where: • t ∈ R denotes time • x(t) ∈ Rn is the state (vector) • u(t) ∈ Rm is the input or control • y(t) ∈ Rp is the output Overview 1–5 × • A(t) ∈ Rn n is the dynamics matrix × • B(t) ∈ Rn m is the input matrix × • C(t) ∈ Rp n is the output or sensor matrix × • D(t) ∈ Rp m is the feedthrough matrix for lighter appearance, equations are often written x˙ = Ax + Bu, y = Cx + Du • CT LDS is a first order vector differential equation • also called state equations, or ‘m-input, n-state, p-output’ LDS Overview 1–6 Some LDS terminology • most linear systems encountered are time-invariant: A, B, C, D are constant, i.e., don’t depend on t • when there is no input u (hence, no B or D) system is called autonomous • very often there is no feedthrough, i.e., D = 0 • when u(t) and y(t) are scalar, system is called single-input, single-output (SISO); when input & output signal dimensions are more than one, MIMO Overview 1–7 Discrete-time linear dynamical system discrete-time linear dynamical system (DT LDS) has the form x(t + 1) = A(t)x(t)+ B(t)u(t), y(t)= C(t)x(t)+ D(t)u(t) where • t ∈ Z = {0, ±1, ±2,...} • (vector) signals x, u, y are sequences DT LDS is a first order vector recursion Overview 1–8 Why study linear systems? applications arise in many areas, e.g. • automatic control systems • signal processing • communications • economics, finance • circuit analysis, simulation, design • mechanical and civil engineering • aeronautics • navigation, guidance Overview 1–9 Usefulness of LDS • depends on availability of computing power, which is large & increasing exponentially • used for – analysis & design – implementation, embedded in real-time systems • like DSP, was a specialized topic & technology 30 years ago Overview 1–10 Origins and history • parts of LDS theory can be traced to 19th century • builds on classical circuits & systems (1920s on) (transfer functions . ) but with more emphasis on linear algebra • first engineering application: aerospace, 1960s • transitioned from specialized topic to ubiquitous in 1980s (just like digital signal processing, information theory, . ) Overview 1–11 Nonlinear dynamical systems many dynamical systems are nonlinear (a fascinating topic) so why study linear systems? • most techniques for nonlinear systems are based on linear methods • methods for linear systems often work unreasonably well, in practice, for nonlinear systems • if you don’t understand linear dynamical systems you certainly can’t understand nonlinear dynamical systems Overview 1–12 Examples (ideas only, no details) • let’s consider a specific system x˙ = Ax, y = Cx with x(t) ∈ R16, y(t) ∈ R (a ‘16-state single-output system’) • model of a lightly damped mechanical system, but it doesn’t matter Overview 1–13 typical output: 3 2 1 0 −1 −2 −3 0 50 100 150 200 250 300 350 t 3 2 1 0 y y −1 −2 −3 0 100 200 300 400 500 600 700 800 900 1000 t • output waveform is very complicated; looks almost random and unpredictable • we’ll see that such a solution can be decomposed into much simpler (modal) components Overview 1–14 t 0.2 0 −0.2 0 50 100 150 200 250 300 350 1 0 −1 0 50 100 150 200 250 300 350 0.5 0 −0.5 0 50 100 150 200 250 300 350 2 0 −2 0 50 100 150 200 250 300 350 1 0 −1 0 50 100 150 200 250 300 350 2 0 −2 0 50 100 150 200 250 300 350 5 0 −5 0 50 100 150 200 250 300 350 0.2 0 −0.2 0 50 100 150 200 250 300 350 (idea probably familiar from ‘poles’) Overview 1–15 Input design add two inputs, two outputs to system: x˙ = Ax + Bu, y = Cx, x(0) = 0 × × where B ∈ R16 2, C ∈ R2 16 (same A as before) 2 problem: find appropriate u : R+ → R so that y(t) → ydes = (1, −2) simple approach: consider static conditions (u, x, y constant): x˙ =0= Ax + Bustatic, y = ydes = Cx solve for u to get: −1 −1 −0.63 ustatic = −CA B ydes = 0.36 Overview 1–16 let’s apply u = ustatic and just wait for things to settle: 0 −0.2 1 −0.4 u −0.6 −0.8 −1 −200 0 200 400 600 800 t1000 1200 1400 1600 1800 0.4 0.3 2 0.2 u 0.1 0 −0.1 −200 0 200 400 600 800 t1000 1200 1400 1600 1800 2 1.5 1 1 y 0.5 0 −200 0 200 400 600 800 t1000 1200 1400 1600 1800 0 −1 2 −2 y −3 −4 −200 0 200 400 600 800 t1000 1200 1400 1600 1800 . takes about 1500 sec for y(t) to converge to ydes Overview 1–17 using very clever input waveforms (EE263) we can do much better, e.g. 0.2 0 1 −0.2 u −0.4 −0.6 0 10 20 t30 40 50 60 0.4 2 0.2 u 0 −0.2 0 10 20 t30 40 50 60 1 0.5 1 y 0 −0.5 0 10 20 t30 40 50 60 0 −0.5 2 −1 y −1.5 −2 −2.5 0 10 20 t30 40 50 60 . here y converges exactly in 50 sec Overview 1–18 in fact by using larger inputs we do still better, e.g. 5 1 0 u −5 −5 0 5 10t 15 20 25 1 0.5 2 0 u −0.5 −1 −1.5 −5 0 5 10t 15 20 25 2 1 1 0 y −1 −5 0 5 10t 15 20 25 0 −0.5 2 −1 y −1.5 −2 −5 0 5 10t 15 20 25 . here we have (exact) convergence in 20 sec Overview 1–19 in this course we’ll study • how to synthesize or design such inputs • the tradeoff between size of u and convergence time Overview 1–20 Estimation / filtering u w y H(s) A/D • signal u is piecewise constant (period 1 sec) • filtered by 2nd-order system H(s), step response s(t) • A/D runs at 10Hz, with 3-bit quantizer Overview 1–21 1 ) t 0 ( u −1 0 1 2 3 4 5 6 7 8 9 10 1.5 ) 1 t ( 0.5 s 0 0 1 2 3 4 5 6 7 8 9 10 1 ) t ( 0 w −1 0 1 2 3 4 5 6 7 8 9 10 1 ) t 0 ( y −1 0 1 2 3 4 5 6 7 8 9 10 t problem: estimate original signal u, given quantized, filtered signal y Overview 1–22 simple approach: • ignore quantization • design equalizer G(s) for H(s) (i.e., GH ≈ 1) • approximate u as G(s)y . yields terrible results Overview 1–23 formulate as estimation problem (EE263) . 1 0.8 0.6 (dotted) 0.4 ) t ( 0.2 ˆ u 0 −0.2 −0.4 (solid) and −0.6 ) t −0.8 ( u −1 0 1 2 3 4 5 6 7 8 9 10 t RMS error 0.03, well below quantization error (!) Overview 1–24 EE263 Autumn 2007-08 Stephen Boyd Lecture 2 Linear functions and examples • linear equations and functions • engineering examples • interpretations 2–1 Linear equations consider system of linear equations y1 = a11x1 + a12x2 + ··· + a1nxn y2 = a21x1 + a22x2 + ··· + a2nxn . ym = am1x1 + am2x2 + ··· + amnxn can be written in matrix form as y = Ax, where y1 a11 a12 ··· a1n x1 y2 a21 a22 ··· a2n x2 y = A = x = .
Load more

Recommended publications
  • Quiz7 Problem 1
    Quiz 7 Quiz7 Problem 1. Independence The Problem. In the parts below, cite which tests apply to decide on independence or dependence. Choose one test and show complete details. 1 ! 1 ! (a) Vectors ~v = ;~v = 1 −1 2 1 0 1 1 0 1 1 0 1 1 B C B C B C (b) Vectors ~v1 = @ −1 A ;~v2 = @ 1 A ;~v3 = @ 0 A 0 −1 −1 2 5 (c) Vectors ~v1;~v2;~v3 are data packages constructed from the equations y = x ; y = x ; y = x10 on (−∞; 1). (d) Vectors ~v1;~v2;~v3 are data packages constructed from the equations y = 1 + x; y = 1 − x; y = x3 on (−∞; 1). Basic Test. To show three vectors are independent, form the system of equations c1~v1 + c2~v2 + c3~v3 = ~0; then solve for c1; c2; c3. If the only solution possible is c1 = c2 = c3 = 0, then the vectors are independent. Linear Combination Test. A list of vectors is independent if and only if each vector in the list is not a linear combination of the remaining vectors, and each vector in the list is not zero. Subset Test. Any nonvoid subset of an independent set is independent. The basic test leads to three quick independence tests for column vectors. All tests use the augmented matrix A of the vectors, which for 3 vectors is A =< ~v1j~v2j~v3 >. Rank Test. The vectors are independent if and only if rank(A) equals the number of vectors. Determinant Test. Assume A is square. The vectors are independent if and only if the deter- minant of A is nonzero.
    [Show full text]
  • Adjacency and Incidence Matrices
    Adjacency and Incidence Matrices 1 / 10 The Incidence Matrix of a Graph Definition Let G = (V ; E) be a graph where V = f1; 2;:::; ng and E = fe1; e2;:::; emg. The incidence matrix of G is an n × m matrix B = (bik ), where each row corresponds to a vertex and each column corresponds to an edge such that if ek is an edge between i and j, then all elements of column k are 0 except bik = bjk = 1. 1 2 e 21 1 13 f 61 0 07 3 B = 6 7 g 40 1 05 4 0 0 1 2 / 10 The First Theorem of Graph Theory Theorem If G is a multigraph with no loops and m edges, the sum of the degrees of all the vertices of G is 2m. Corollary The number of odd vertices in a loopless multigraph is even. 3 / 10 Linear Algebra and Incidence Matrices of Graphs Recall that the rank of a matrix is the dimension of its row space. Proposition Let G be a connected graph with n vertices and let B be the incidence matrix of G. Then the rank of B is n − 1 if G is bipartite and n otherwise. Example 1 2 e 21 1 13 f 61 0 07 3 B = 6 7 g 40 1 05 4 0 0 1 4 / 10 Linear Algebra and Incidence Matrices of Graphs Recall that the rank of a matrix is the dimension of its row space. Proposition Let G be a connected graph with n vertices and let B be the incidence matrix of G.
    [Show full text]
  • A Brief Introduction to Spectral Graph Theory
    A BRIEF INTRODUCTION TO SPECTRAL GRAPH THEORY CATHERINE BABECKI, KEVIN LIU, AND OMID SADEGHI MATH 563, SPRING 2020 Abstract. There are several matrices that can be associated to a graph. Spectral graph theory is the study of the spectrum, or set of eigenvalues, of these matrices and its relation to properties of the graph. We introduce the primary matrices associated with graphs, and discuss some interesting questions that spectral graph theory can answer. We also discuss a few applications. 1. Introduction and Definitions This work is primarily based on [1]. We expect the reader is familiar with some basic graph theory and linear algebra. We begin with some preliminary definitions. Definition 1. Let Γ be a graph without multiple edges. The adjacency matrix of Γ is the matrix A indexed by V (Γ), where Axy = 1 when there is an edge from x to y, and Axy = 0 otherwise. This can be generalized to multigraphs, where Axy becomes the number of edges from x to y. Definition 2. Let Γ be an undirected graph without loops. The incidence matrix of Γ is the matrix M, with rows indexed by vertices and columns indexed by edges, where Mxe = 1 whenever vertex x is an endpoint of edge e. For a directed graph without loss, the directed incidence matrix N is defined by Nxe = −1; 1; 0 corresponding to when x is the head of e, tail of e, or not on e. Definition 3. Let Γ be an undirected graph without loops. The Laplace matrix of Γ is the matrix L indexed by V (G) with zero row sums, where Lxy = −Axy for x 6= y.
    [Show full text]
  • Graph Equivalence Classes for Spectral Projector-Based Graph Fourier Transforms Joya A
    1 Graph Equivalence Classes for Spectral Projector-Based Graph Fourier Transforms Joya A. Deri, Member, IEEE, and José M. F. Moura, Fellow, IEEE Abstract—We define and discuss the utility of two equiv- Consider a graph G = G(A) with adjacency matrix alence graph classes over which a spectral projector-based A 2 CN×N with k ≤ N distinct eigenvalues and Jordan graph Fourier transform is equivalent: isomorphic equiv- decomposition A = VJV −1. The associated Jordan alence classes and Jordan equivalence classes. Isomorphic equivalence classes show that the transform is equivalent subspaces of A are Jij, i = 1; : : : k, j = 1; : : : ; gi, up to a permutation on the node labels. Jordan equivalence where gi is the geometric multiplicity of eigenvalue 휆i, classes permit identical transforms over graphs of noniden- or the dimension of the kernel of A − 휆iI. The signal tical topologies and allow a basis-invariant characterization space S can be uniquely decomposed by the Jordan of total variation orderings of the spectral components. subspaces (see [13], [14] and Section II). For a graph Methods to exploit these classes to reduce computation time of the transform as well as limitations are discussed. signal s 2 S, the graph Fourier transform (GFT) of [12] is defined as Index Terms—Jordan decomposition, generalized k gi eigenspaces, directed graphs, graph equivalence classes, M M graph isomorphism, signal processing on graphs, networks F : S! Jij i=1 j=1 s ! (s ;:::; s ;:::; s ;:::; s ) ; (1) b11 b1g1 bk1 bkgk I. INTRODUCTION where sij is the (oblique) projection of s onto the Jordan subspace Jij parallel to SnJij.
    [Show full text]
  • EUCLIDEAN DISTANCE MATRIX COMPLETION PROBLEMS June 6
    EUCLIDEAN DISTANCE MATRIX COMPLETION PROBLEMS HAW-REN FANG∗ AND DIANNE P. O’LEARY† June 6, 2010 Abstract. A Euclidean distance matrix is one in which the (i, j) entry specifies the squared distance between particle i and particle j. Given a partially-specified symmetric matrix A with zero diagonal, the Euclidean distance matrix completion problem (EDMCP) is to determine the unspecified entries to make A a Euclidean distance matrix. We survey three different approaches to solving the EDMCP. We advocate expressing the EDMCP as a nonconvex optimization problem using the particle positions as variables and solving using a modified Newton or quasi-Newton method. To avoid local minima, we develop a randomized initial- ization technique that involves a nonlinear version of the classical multidimensional scaling, and a dimensionality relaxation scheme with optional weighting. Our experiments show that the method easily solves the artificial problems introduced by Mor´e and Wu. It also solves the 12 much more difficult protein fragment problems introduced by Hen- drickson, and the 6 larger protein problems introduced by Grooms, Lewis, and Trosset. Key words. distance geometry, Euclidean distance matrices, global optimization, dimensional- ity relaxation, modified Cholesky factorizations, molecular conformation AMS subject classifications. 49M15, 65K05, 90C26, 92E10 1. Introduction. Given the distances between each pair of n particles in Rr, n r, it is easy to determine the relative positions of the particles. In many applications,≥ though, we are given only some of the distances and we would like to determine the missing distances and thus the particle positions. We focus in this paper on algorithms to solve this distance completion problem.
    [Show full text]
  • Incidence Matrices and Interval Graphs
    Pacific Journal of Mathematics INCIDENCE MATRICES AND INTERVAL GRAPHS DELBERT RAY FULKERSON AND OLIVER GROSS Vol. 15, No. 3 November 1965 PACIFIC JOURNAL OF MATHEMATICS Vol. 15, No. 3, 1965 INCIDENCE MATRICES AND INTERVAL GRAPHS D. R. FULKERSON AND 0. A. GROSS According to present genetic theory, the fine structure of genes consists of linearly ordered elements. A mutant gene is obtained by alteration of some connected portion of this structure. By examining data obtained from suitable experi- ments, it can be determined whether or not the blemished portions of two mutant genes intersect or not, and thus inter- section data for a large number of mutants can be represented as an undirected graph. If this graph is an "interval graph," then the observed data is consistent with a linear model of the gene. The problem of determining when a graph is an interval graph is a special case of the following problem concerning (0, l)-matrices: When can the rows of such a matrix be per- muted so as to make the l's in each column appear consecu- tively? A complete theory is obtained for this latter problem, culminating in a decomposition theorem which leads to a rapid algorithm for deciding the question, and for constructing the desired permutation when one exists. Let A — (dij) be an m by n matrix whose entries ai3 are all either 0 or 1. The matrix A may be regarded as the incidence matrix of elements el9 e2, , em vs. sets Sl9 S2, , Sn; that is, ai3 = 0 or 1 ac- cording as et is not or is a member of S3 .
    [Show full text]
  • Dimensionality Reduction
    CHAPTER 7 Dimensionality Reduction We saw in Chapter 6 that high-dimensional data has some peculiar characteristics, some of which are counterintuitive. For example, in high dimensions the center of the space is devoid of points, with most of the points being scattered along the surface of the space or in the corners. There is also an apparent proliferation of orthogonal axes. As a consequence high-dimensional data can cause problems for data mining and analysis, although in some cases high-dimensionality can help, for example, for nonlinear classification. Nevertheless, it is important to check whether the dimensionality can be reduced while preserving the essential properties of the full data matrix. This can aid data visualization as well as data mining. In this chapter we study methods that allow us to obtain optimal lower-dimensional projections of the data. 7.1 BACKGROUND Let the data D consist of n points over d attributes, that is, it is an n × d matrix, given as ⎛ ⎞ X X ··· Xd ⎜ 1 2 ⎟ ⎜ x x ··· x ⎟ ⎜x1 11 12 1d ⎟ ⎜ x x ··· x ⎟ D =⎜x2 21 22 2d ⎟ ⎜ . ⎟ ⎝ . .. ⎠ xn xn1 xn2 ··· xnd T Each point xi = (xi1,xi2,...,xid) is a vector in the ambient d-dimensional vector space spanned by the d standard basis vectors e1,e2,...,ed ,whereei corresponds to the ith attribute Xi . Recall that the standard basis is an orthonormal basis for the data T space, that is, the basis vectors are pairwise orthogonal, ei ej = 0, and have unit length ei = 1. T As such, given any other set of d orthonormal vectors u1,u2,...,ud ,withui uj = 0 T and ui = 1(orui ui = 1), we can re-express each point x as the linear combination x = a1u1 + a2u2 +···+adud (7.1) 183 184 Dimensionality Reduction T where the vector a = (a1,a2,...,ad ) represents the coordinates of x in the new basis.
    [Show full text]
  • Contents 5 Eigenvalues and Diagonalization
    Linear Algebra (part 5): Eigenvalues and Diagonalization (by Evan Dummit, 2017, v. 1.50) Contents 5 Eigenvalues and Diagonalization 1 5.1 Eigenvalues, Eigenvectors, and The Characteristic Polynomial . 1 5.1.1 Eigenvalues and Eigenvectors . 2 5.1.2 Eigenvalues and Eigenvectors of Matrices . 3 5.1.3 Eigenspaces . 6 5.2 Diagonalization . 9 5.3 Applications of Diagonalization . 14 5.3.1 Transition Matrices and Incidence Matrices . 14 5.3.2 Systems of Linear Dierential Equations . 16 5.3.3 Non-Diagonalizable Matrices and the Jordan Canonical Form . 19 5 Eigenvalues and Diagonalization In this chapter, we will discuss eigenvalues and eigenvectors: these are characteristic values (and characteristic vectors) associated to a linear operator T : V ! V that will allow us to study T in a particularly convenient way. Our ultimate goal is to describe methods for nding a basis for V such that the associated matrix for T has an especially simple form. We will rst describe diagonalization, the procedure for (trying to) nd a basis such that the associated matrix for T is a diagonal matrix, and characterize the linear operators that are diagonalizable. Then we will discuss a few applications of diagonalization, including the Cayley-Hamilton theorem that any matrix satises its characteristic polynomial, and close with a brief discussion of non-diagonalizable matrices. 5.1 Eigenvalues, Eigenvectors, and The Characteristic Polynomial • Suppose that we have a linear transformation T : V ! V from a (nite-dimensional) vector space V to itself. We would like to determine whether there exists a basis of such that the associated matrix β is a β V [T ]β diagonal matrix.
    [Show full text]
  • Eigenvalues of Graphs
    Eigenvalues of graphs L¶aszl¶oLov¶asz November 2007 Contents 1 Background from linear algebra 1 1.1 Basic facts about eigenvalues ............................. 1 1.2 Semide¯nite matrices .................................. 2 1.3 Cross product ...................................... 4 2 Eigenvalues of graphs 5 2.1 Matrices associated with graphs ............................ 5 2.2 The largest eigenvalue ................................. 6 2.2.1 Adjacency matrix ............................... 6 2.2.2 Laplacian .................................... 7 2.2.3 Transition matrix ................................ 7 2.3 The smallest eigenvalue ................................ 7 2.4 The eigenvalue gap ................................... 9 2.4.1 Expanders .................................... 10 2.4.2 Edge expansion (conductance) ........................ 10 2.4.3 Random walks ................................. 14 2.5 The number of di®erent eigenvalues .......................... 16 2.6 Spectra of graphs and optimization .......................... 18 Bibliography 19 1 Background from linear algebra 1.1 Basic facts about eigenvalues Let A be an n £ n real matrix. An eigenvector of A is a vector such that Ax is parallel to x; in other words, Ax = ¸x for some real or complex number ¸. This number ¸ is called the eigenvalue of A belonging to eigenvector v. Clearly ¸ is an eigenvalue i® the matrix A ¡ ¸I is singular, equivalently, i® det(A ¡ ¸I) = 0. This is an algebraic equation of degree n for ¸, and hence has n roots (with multiplicity). The trace of the square matrix A = (Aij ) is de¯ned as Xn tr(A) = Aii: i=1 1 The trace of A is the sum of the eigenvalues of A, each taken with the same multiplicity as it occurs among the roots of the equation det(A ¡ ¸I) = 0. If the matrix A is symmetric, then its eigenvalues and eigenvectors are particularly well behaved.
    [Show full text]
  • Linear Algebra
    Linear Algebra July 28, 2006 1 Introduction These notes are intended for use in the warm-up camp for incoming Berkeley Statistics graduate students. Welcome to Cal! We assume that you have taken a linear algebra course before and that most of the material in these notes will be a review of what you already know. If you have never taken such a course before, you are strongly encouraged to do so by taking math 110 (or the honors version of it), or by covering material presented and/or mentioned here on your own. If some of the material is unfamiliar, do not be intimidated! We hope you find these notes helpful! If not, you can consult the references listed at the end, or any other textbooks of your choice for more information or another style of presentation (most of the proofs on linear algebra part have been adopted from Strang, the proof of F-test from Montgomery et al, and the proof of bivariate normal density from Bickel and Doksum). Go Bears! 1 2 Vector Spaces A set V is a vector space over R and its elements are called vectors if there are 2 opera- tions defined on it: 1. Vector addition, that assigns to each pair of vectors v ; v V another vector w V 1 2 2 2 (we write v1 + v2 = w) 2. Scalar multiplication, that assigns to each vector v V and each scalar r R another 2 2 vector w V (we write rv = w) 2 that satisfy the following 8 conditions v ; v ; v V and r ; r R: 8 1 2 3 2 8 1 2 2 1.
    [Show full text]
  • Linear Algebra with Exercises B
    Linear Algebra with Exercises B Fall 2017 Kyoto University Ivan Ip These notes summarize the definitions, theorems and some examples discussed in class. Please refer to the class notes and reference books for proofs and more in-depth discussions. Contents 1 Abstract Vector Spaces 1 1.1 Vector Spaces . .1 1.2 Subspaces . .3 1.3 Linearly Independent Sets . .4 1.4 Bases . .5 1.5 Dimensions . .7 1.6 Intersections, Sums and Direct Sums . .9 2 Linear Transformations and Matrices 11 2.1 Linear Transformations . 11 2.2 Injection, Surjection and Isomorphism . 13 2.3 Rank . 14 2.4 Change of Basis . 15 3 Euclidean Space 17 3.1 Inner Product . 17 3.2 Orthogonal Basis . 20 3.3 Orthogonal Projection . 21 i 3.4 Orthogonal Matrix . 24 3.5 Gram-Schmidt Process . 25 3.6 Least Square Approximation . 28 4 Eigenvectors and Eigenvalues 31 4.1 Eigenvectors . 31 4.2 Determinants . 33 4.3 Characteristic polynomial . 36 4.4 Similarity . 38 5 Diagonalization 41 5.1 Diagonalization . 41 5.2 Symmetric Matrices . 44 5.3 Minimal Polynomials . 46 5.4 Jordan Canonical Form . 48 5.5 Positive definite matrix (Optional) . 52 5.6 Singular Value Decomposition (Optional) . 54 A Complex Matrix 59 ii Introduction Real life problems are hard. Linear Algebra is easy (in the mathematical sense). We make linear approximations to real life problems, and reduce the problems to systems of linear equations where we can then use the techniques from Linear Algebra to solve for approximate solutions. Linear Algebra also gives new insights and tools to the original problems.
    [Show full text]
  • Notes on the Proof of the Van Der Waerden Permanent Conjecture
    Iowa State University Capstones, Theses and Creative Components Dissertations Spring 2018 Notes on the proof of the van der Waerden permanent conjecture Vicente Valle Martinez Iowa State University Follow this and additional works at: https://lib.dr.iastate.edu/creativecomponents Part of the Discrete Mathematics and Combinatorics Commons Recommended Citation Valle Martinez, Vicente, "Notes on the proof of the van der Waerden permanent conjecture" (2018). Creative Components. 12. https://lib.dr.iastate.edu/creativecomponents/12 This Creative Component is brought to you for free and open access by the Iowa State University Capstones, Theses and Dissertations at Iowa State University Digital Repository. It has been accepted for inclusion in Creative Components by an authorized administrator of Iowa State University Digital Repository. For more information, please contact [email protected]. Notes on the proof of the van der Waerden permanent conjecture by Vicente Valle Martinez A Creative Component submitted to the graduate faculty in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Major: Mathematics Program of Study Committee: Sung Yell Song, Major Professor Steve Butler Jonas Hartwig Leslie Hogben Iowa State University Ames, Iowa 2018 Copyright c Vicente Valle Martinez, 2018. All rights reserved. ii TABLE OF CONTENTS LIST OF FIGURES . iii ACKNOWLEDGEMENTS . iv ABSTRACT . .v CHAPTER 1. Introduction and Preliminaries . .1 1.1 Combinatorial interpretations . .1 1.2 Computational properties of permanents . .4 1.3 Computational complexity in computing permanents . .8 1.4 The organization of the rest of this component . .9 CHAPTER 2. Applications: Permanents of Special Types of Matrices . 10 2.1 Zeros-and-ones matrices .
    [Show full text]