CONTROL ROOM ACCELERATOR PHYSICS

Day 2 Review of 1/27/14 USPAS 2

Outline

1. Introduction

2. Vector spaces

3. Matrices and linear operators

4. Eigenvalues and eigenvectors

5. Diagonalization

6. Singular-value decomposition 1/27/14 USPAS 3 Introduction Motivation

• Linear algebra provides many practical techniques for analysis and solution of linear (and nonlinear) systems. We utilize these established methods in our applications.

• This material should be a review, but we formalize it then show it in the context of accelerator control applications 1/27/14 USPAS 4 Introduction Linear Algebra: Definition and Description

• Linear algebra is the branch of concerned with the study of vectors, vector spaces (also called linear spaces), linear maps between vector spaces (also called linear transformations, linear operators), and systems of linear equations.

• Here we will think of “linear algebra” loosely as matrices • Matrices are “tangible”, • They are computer friendly • Represent linear relations (maps) between finite dimensional vector spaces (e.g., the space of correctors and the space of BPMs)

• Our objective here is to review some basic facts about matrices and demonstrate applications to accelerator control 1/27/14 USPAS 5 Introduction Basic Notation Z Ø The set of integers {…, -1, 0, +1, …}

R Ø The set of real numbers

Rn Ø The n times Cartesian product of R, or the set of “n-tuples” Ø Vectors of the form x = (x1, …,xn)

Rm×n Ø The set of m×n matrices having real number elements

L : Rm → Rn Ø Linear operators mapping elements from Rm to elements in Rn

n×n GL(n, R) Ø The set of elements in R having nonzero Ø These are the invertible matrices (has an inverse for multiplication) 1/27/14 USPAS 6 Vector Spaces Definition and Description • Definition: A over the field R is a set V along with two operations, vector addition + and multiplication •, such that for any vectors u, v, w ∈ V and scalars r, s ∈ R we have the axioms:

• Associativity of addition: u + (v + w) = (u + v) + w • Commutativity of addition: u + v = v + u • Identity element of addition: ∃ 0 ∈ V such that v + 0 = v for all v ∈ V. • Inverse elements of addition: • For every v ∈ V, there exists an element −v ∈ V, called the additive inverse of v, such that v + (−v) = 0 • Distributivity of multiplication w.r.t. vector addition: a(u + v) = au + av ugly details details ugly • Distributivity of multiplication w.r.t. field addition: (a + b)v = av + bv • Compatibility of scalar and field multiplication: a(bv) = (ab)v • Identity element of multiplication: 1v = v, where 1∈ R is the identity.

• Informally, “any of elements in V is also an element in V “ • That is, If u, v ∈ V and r, s ∈ R then ru + sv ∈ V basic idea idea basic

Anything that fits the above requirements is a vector space, and can be treated by linear algebraic methods. 1/27/14 USPAS 7 Vector Spaces The Space Rn We can consider the space Rn as the natural extension of the more familiar vectors in Euclidean 3 space.

This is the most common vector space for computer solution.

rp 3 z p = ( p ,0,0) + (0, p ,0) + (0,0, p ) R p Addition of vectors x y z = ( px , py , pz )

Multiplication by rp = r( p , p , p ) y x y z scalars = (rp x ,rp y ,rp z ) x

r ∈ R Vector addition x + y = (x1 + y1,...,xn + yn ) n x = (x1,..., xn ) ∈ R n y = (y1,..., yn ) ∈ R rx = (rx1,...,rxn ) Extension to Rn 1/27/14 USPAS 8 Vector Spaces Examples • Although Rn is the vector space of our primary concern, there are many others

• The set F(R) of real valued functions on the real line R, {f : R → R} • The set F([0,1]) of real functions on interval [0,1], {f : [0,1] → R} • Note F([0,1]) is a vector subspace of F(R)

∞ • The set R of infinite sequences {r1, r2, r3, …}, ri ∈ R

• The set F(Z+) of real functions on the positive integers Z+ {f : Z+ → R} ∞ • Vector spaces R and F(Z+) are isomorphic (the same thing)

• The set Cn(R) of continuous functions on R {f : R → R | dnf/dxn <∞} • The set C∞(R) of smooth functions on R {f : R→R | dnf/dxn <∞ all n}

• Note the subspace structure C∞(R) ⊆ Cn(R) ⊆ F(R) 1/27/14 USPAS 9 Vector Spaces Sets

• A basis for vector space V is a (possibly infinite) subset {e1, e2, e3, …} of V such that any v ∈ V can be expressed as a linear combination of basis vectors. • If v ∈ V then ∃ r1, r2, r3, … ∈ R such that v = r1e1 + r2e2 + r3e3 + …

• All basis sets of V have the same cardinality, called the dimension of V

• dim V = |{e1, e2, e3, …}|

• Examples: • (1,…,0), (0, 1, …,0), …, (0, 0, …,1) form the standard basis for Rn.

• {1, x, x2, …, xn-1} is a basis for the subspace of f in C∞([0,1]) such that dnf/dxn = 0

• The eigenvectors of any symmetric, pos. def. matrix A ∈ Rn×n form a basis of Rn.

All these • The first n eigenfunctions {ψ1, ψ2, …, ψn} of quantum mechanical operator A form a basis vector for the subspace containing the first n observables of A. spaces are isomorphic! • {1, cos 2πft, cos 4πft, …, cos (n-1)πft} is a basis for the set of all even, band-limited functions with period 1/f, and cutoff frequency nf.

(As a Hilbert space they are not) 1/27/14 USPAS 10 Linear Operators Between Vector Spaces Matrix Representations A L : U → V between vector spaces U and V has the defining property

L(rx+sy) = rL(x) + sL(y) for vectors x,y ∈ U and scalars r,s ∈ R.

(Linear maps respect the algebra of vector spaces!)

• Given any basis set {ei} ⊆ U, by linearity L is completely defined by where it maps basis vectors {ei} ∈ U, that is by {L(ei)} ∈ V • Any u ∈ U can be expressed as u = r1e1+…+rmem some {ri} ⊆ R • Then L(u) = L(r1e1+…+rmem) = r1L(e1) + … + rmL(em) • Is {L(e1), …, L(em)} a basis set for V ?

• A matrix A in Rm×n is, in a natural way, a linear map between the vector spaces Rm and Rn under the usual

• Once again Au = A(r1e1+…+rmem) = r1Ae1 + … + rmAem n • When is {Ae1, …, Aem} a basis set for R ? 1/27/14 USPAS 11

Beamline A Matrices in Accelerator Control Maps Between Sensors and Actuators • A matrix A in Rm×n can represent a linear map between the vector spaces Rm and Rn

• Accelerator Control Example: • Elements of Rm are corrector magnet strengths (m mag.’s) • Elements of Rn are BPM readbacks (n BPMs) • Then A in Rm×n is the response matrix

• Note three cases concerning response matrix A: A • m > n Domain Rm is “bigger” than range Rn

(more correctors than BPMs) magnets Corrector Monitors Position Beam • m < n Domain Rm is “smaller” than range Rn (more BPMs than correctors)

• m = n The matrix A is square (equal numbers of correctors and BPMs) n u ∈ Rm y ∈ R 1/27/14 USPAS 12 Matrices in Accelerator Control Analyzing Relationship Between Sensors and Actuators n Assume A has full throughout ker A ≡ {x ∈ R | Ax = 0} Im A ARm Case 1: m>n ker A ≠ 0 ≡ We can get to all beam positions. A But, corrector settings are not 0 0 unique (and they can “fight” each m n other). R R

Case 2: m

Case 3: m=n A ker A = 0, Im A = Rn 0 We can steer to all beam positions. 0 Each has a unique corrector setting. Rm Rn = Rm 1/27/14 USPAS 13 Matrices in Accelerator Control Matrices are Ubiquitous in Beam Physics Determining the Propagating the beam perturbations in the closed coordinates zi at one orbit of a ring when location of the beamline to changing a dipole corrector another (downstream) values. v location zf. BPMj (x,y,z) Qh S1 Qh D1 Qv S2 Qv

zi 0 0 zf

DHi D2

u u qh (ush1,usv1) qv (ush2,usv2) ⎛ ∂BPM1 ∂BPM1 ⎞ ⎜  ⎟ 6×6 ⎜ ∂DH1 ∂DH m ⎟ z f = Φ(u)zi Φ ∈ R A = ⎜   ⎟ ⎜ ⎟ ∂BPM n ∂BPM n Transfer matrices Response matrices ⎜  ⎟ ⎝ ∂DH1 ∂DH m ⎠ 1/27/14 USPAS 14 Matrix Decomposition Eigenvalues, , Diagonalization

• Because matrices model important aspects of beam physics and accelerator control, it is instructive to look at their structure

• The eigenvectors and eigenvalues of a matrix decompose the vector spaces into the natural “modes” of the system

• Matrix factorization techniques decompose a matrix into constituent factors with special structure

• Matrix diagonalization is a special type of factorization which identifies the eigen modes of a matrix

• We cover two explicit methods of factoring a matrix, eigenvalue decomposition and singular-value decomposition, both are diagonalization processes.

• We focus on response matrices throughout the discussion, however, the material also applies to transfer matrices, linearized dynamics, coupled equations, etc. 1/27/14 USPAS 15 Eigenvectors and Eigenvalues The Natural Modes of A • For a square matrix A in Rn×n we can often find special vectors e in Rn so that Ae = λe

where λ ≠ 0 is a scalar (either real or complex) • Any such vector e is called an eigenvector of A • Any such scalar λ is called an eigenvalue of A

• A does not change the direction of e, only the length! • A acts like an amplifier on e with gain λ

• Eigenvectors e1, e2, … are the natural, (decoupled), modes of A • What if we could decompose all of Rn into eigenvectors of A? (see homework) 1/27/14 USPAS 16 Eigenvectors and Eigenvalues Example: Examining Orbit Position System Response Say we find an eigenvector e = (e1, e2, …, en) for the Correctors ⇒ A ⇒ BPMs response matrix A so that BPM1 Ae = λe e1 DH1 λe1 (hor) A • Drive each dipole corrector BPM DHi with the strength ei at the 2 e e2 DH2 (hor) λ 2 eigenvector coordinate i e λe : • Each BPM behaves as if it is : i ⋅ ⋅ directly connected to DHi

BPMn en DH λen • The response (beam positions) n (hor) are simply amplified by eigenvalue λ Beamline Called a natural (uncoupled) mode of the system 1/27/14 USPAS 17 Matrix Examples:

• Algebraic analogy: • The integers Z have prime factorizations, e.g., 42 = 2•3•7 • Real numbers have only trivial factorizations (every real number is a factor of any other real number).

• Matrix A in Rm×n admits many factorizations (or decompositions) • A = LU (m=n), L upper triangular, U lower triangular (“LU” decomposition) • A = QR , Q ∈ SO(m), R symmetric, positive definite (polar decomp.) • A = TΛT-1 (m=n), T ∈ GL(n, R), Λ ∈ diag(Rn×n) (eigenvalue decomp.) • A = UDVT , U ∈ SO(m) , D ∈ diag(Rn×n), V ∈ SO(n) (sing. value decomp.)

• We cover eigenvalue decomposition and the more general decomposition in detail. 1/27/14 USPAS 18 Matrix Factorization Factoring into Natural Modes Sometimes a square matrix A in Rn×n can be factored as −1 A = TΛT n×n Where T is in GL(n,R) (invertible) and Λ = diag{λ1,…,λn} in R . • Λ is called the spectral matrix (with spectrum {λ1,…,λn}) • T is called the modal matrix and is composed of eigenvectors • When this condition is satisfied, i.e. when A = TΛT-1, we say A is diagonalizable

• The matrix T describes the coupling modes between the correctors and the BPMs. • The matrix Λ describes the gains between these natural couplings • For example….

1/27/14 USPAS 19 Matrix Factorization Decoupling Beam Position Response • Our corrector space DH is isomorphic to Rn, that is, DH ≅ Rn

• The vector x ∈ DH represents the set of corrector strengths x1, x2, …, xn

• Our response space BPM is isomorphic to Rn, that is, BPM ≅ Rn

• The vector y ∈ BPM represents the set of beam positions y1, y2, …, yn

• The response matrix A ∈ Rn×n relates y to x, that is, y = Ax ⎛ ∂BPM ∂BPM ⎞ ⎜ 1  1 ⎟ ⎜ ∂DH1 ∂DH m ⎟ • If A factors as A = TΛT-1, then Λ = T-1AT A = ⎜   ⎟ ⎜ ⎟ ∂BPM n ∂BPM n ⎜  ⎟ -1 -1 ⎜ ⎟ • Set ξ ≡ T x and η ≡ T y ⎝ ∂DH1 ∂DH m ⎠

yielding η = Λξ

Beam or the n scalar equations x y

ξ T T-1 η ηi =λiξi for i = 1, …, n A The restrictive part here is requiring the same number of correctors and BPMs Design of a response decoupler 1/27/14 Matrix Diagonalization Decoupling Beam Position Response (Special Case m=n) Matrix diagonalization decouples our corrector space DH ≅ Rn and our response space BPM ≅ Rn into the natural modes of (square) response matrix A

• Again, transform our inputs x to the uncoupled inputs ξ ≡ T-1x

n th • Let si ≡ (0,…, 1,…, 0) ∈ R be the i standard basis vector

• Let x = ei and note (accept) ei ≡ Ts i = coli T ∈ DH th -1 • Ts i is the i eigenmode. Or equivalently, T ei = si transforms eigenvectors to standard basis vectors. • Recall each x ∈ DH decomposes as x = ξ1e1 + … + ξnen (Hwk - prove this!)

• Output vector yi = λiei = λiTs i ∈ BPM is the response to eigenvector input ei. Λ T-1 T ξ1 λ1 λ1ξ1 x1 y ξ λ λ ξ 1 x 2 2 2 2 y x 2 2 y y = TΛT−1x

ξn λn λnξn xn yn recoupler DH decoupler BPM “uncoupled 20 beam” 1/27/14 USPAS 21 Singular Value Decomposition (SVD) Generalizing Eigenvalue Decomposition Any matrix A ∈ Rm×n may be factored as follows:

A = UDVT

m×m m×n n×n where U ∈ R , D = diag{σ1,…,σn} ∈ R , V ∈ R

• The numbers {σ1,…,σn} are the singular values of A • They are non-negative real numbers • They are generalizations of eigenvalues for square matrices 1/2 1/2 • Specifically, ! T # and ! T # σ i = "λi (A A)$ for n ≤ m σ i = "λi (AA )$ for n ≥ m

• Matrix U and V have special properties • VTV = I ∈ Rn×n , that is, it is orthogonal, in set SO(n) • UTU = I ∈ Rm×m , that is, it is orthogonal, in set SO(m) • (note UTU = I implies UT = U-1)

NOTE: No requirements on the number of correctors and BPMs 1/27/14 USPAS 22 Singular Value Decomposition (SVD) Generalizing Diagonalization • The columns of V are called the right singular vectors of A • The columns of U are called the left singular vectors of A

Again consider the matrix-vector equation

y = Ax = UDVT x

and perform the substitutions • ξ = VTx • η = UTy

We then have the (almost) equivalent decouple system

Like the case of a diagonalizable A, this η = Dξ equation is completely decoupled ηi = σiξi 1/27/14 USPAS 23 Singular Value Decomposition (SVD) Generalizing Diagonalization Returning to the corrector/BPM example with singular-value decomposition D T DH V U BPM σ ξ x ξ1 σ1 1 1 1 y1 ξ σ σ2ξ2 x 2 2 y Clearly this situation is analogous to 2 2 diagonalizable case, however…

ξm σ m σmξm We must be careful! xm ym

0 y • Some inputs may not be connected. m+1 0 ym+2

n • We have outputs {y} R that are unreachable! 0 ⊆ yn • This example was for the case where mn “Dead inputs” {x} ⊆ Rm that do nothing

SVD is an important part of Model-Independent Analysis in high-level beam control 1/27/14 USPAS 24 Singular Value Decomposition Summary • We can factor any matrix A ∈ Rm×n as A = UDVT • This factorization alone provides an enormous amount of information about a system represented by

y = Ax

• Matrix V tells you coupling of inputs (i.e., the xi) • Corrector combinations that have effect

• Matrix U tells you coupling of outputs (i.e., the yi) • BPMs combinations that are likely/unlikely

• The D provides …

• The gains (i.e., the σi) for the system • Internal system degeneracy – “zero” singular value 1/27/14 USPAS 25 Matrix Exponential A Matrix Function

• Functions of (square) matrices are common in analysis • For example, sin(A), log(A), exp(A), for A ∈ Rn×n • These functions may seem strange, but they are well-defined by the Taylor series for their scalar function sibling (matrix powers are well-defined)

• Of particular importance for us is the matrix exponential etA • The scalar t representing time • Occurs frequently in linear dynamical systems as the natural response • Defined according to Taylor series t 2 t3 etA ≡ I + tA + A2 + A3 +… 2! 3! • Show etA is well defined for all A ∈ Rn×n given that ea is well defined for all a ∈ R

• Say A is diagonalizable where A = TΛT-1 • Show that etA = TetΛT-1 • The exponential etΛ is easy to compute, etΛ = diag{etλ1,…, etλn} • Jacobi identity det eA = etr A 1/27/14 USPAS 26 Control Theory A Preview Given a system (a “plant”) G, control theory starts with the basic questions: Is the system… • Stable: is the output y bounded for bounded inputs u? • Is ||y||<∞ for ||u||<∞ ?

• Observable: can we deduce internal state x by observing the inputs u and outputs y?

• Controllable: can we steer the system to any arbitrary output y? • (There exists a u(t) to do so?)

Then we can ask how to stabilize, how to observe, how to control Linear, continuous-time plant G Plant G Internal State x x(t) = A(t)x(t)+ B(t)u(t) x x u B Σ ∫ C y y(t) = C(t)x(t) input output

A 1/27/14 USPAS 27 Linear Beam Optics A Preview Linear systems and the matrix exponential play a crucial part in linear beam optics Focusing Quadrupole n and, consequently, the XAL online model

• In linear beam optics beamline elements are modeled by matrices Φ ∈ Sp(6). • Sp(6) is The group of 6x6 symplectic matrices.

• These matrices are formed from the exponential of another matrix G

• The matrix G represents the equations of motion z(s) = Φn (s)z0

• The XAL online model is based upon these Φ (s) ≡ esGn ideas n

zʹ(s) = Gn (s)z(s) 1/27/14 USPAS 28 Linear Algebra Summary

• Matrices can be treated as linear operators between finite dimensional vector spaces, in particular, the spaces Rn

• A square matrix A often has eigenvalues and eigenvectors that characterize the action of A upon vector space Rn

• If a matrix A can be diagonalized as A = TΛT-1 then its action can be completely decoupled

• Any matrix A ∈ Rm×n may be factored according to A = UDVT where D is the matrix of singular values • This factorization, although not as straightforward, also characterizes the action of A upon Rn (and domain Rm) 1/27/14 USPAS 29

Supplementary Material

• More details on Linear Algebra 1/27/14 USPAS 30 Matrix Diagonalization Factoring into Natural Modes Sometimes a square matrix A in Rn×n can be factored as −1 A = TΛT n×n Where T is in GL(n,R) and Λ = diag{λ1,…,λn} in R . • Λ is called the spectral matrix (with spectrum {λ1,…,λn}) • T is called the modal matrix • When this condition is satisfied, i.e., when A = TΛT-1, we say A is diagonalizable

n×n Fact: If a square matrix A in R has n unique eigenvalues {λ1, …,λn} then it can be factored as above

• In fact, a suitable T can be formed by augmenting all the eigenvectors as columns. Specifically,

T = (e1 e2  en ) 1/27/14 USPAS 31 Matrix Diagonalization Interpretation Matrix diagonalization decouples Rn into the natural modes of A • Instead of working in Rn, we work in the space T-1Rn !

• To make this less abstract consider equation y = Ax and, for example, • Think of A as a multiple-input, multiple-output, coupled amplifier. • Instead of using parameters x and y, use ξ ≡ T-1x and η ≡ T-1y • Everything decouples as ξi = λiηi (transform back when you’re done)

x T-1 ξ Λ η T y −1 y = TΛT x x λ1 1 y1 1 1 λ − − x 2 y T y = ΛT x 2 -1 2 λ2 η = Λξ λn xn yn n −1 n n T T-1 1/27/14 USPAS 32 Matrix Diagonalization Special Case: Symmetric, Positive T Fact: A positive-definite (λi > 0, for each i), symmetric (A = A ), square matrix A in Rn×n can always be diagonalized as A = RΛRT Where R is in the special orthogonal group SO(n) in Rn×n.

For any element R of SO(n) in Rn×n • RRT = I where I is the • From the above, R-1 = RT, e.g., just like a rotation in 3-space • det R = 1

A appears as a hyper-ellipsoid

with semi-axes {λ1,…,λn} rotated by a (generalized) angle R in hyper-Euclidean n space R 1/27/14 USPAS 33 Singular Value Decomposition Generalizing Diagonalization

Any matrix A ∈ Rm×n may be factored as follows:

T Note the similarity with the A = UDV special case when A is symmetric and positive-definite. m×n n×n where U ∈ R , D = diag{σ1,…,σn}, V ∈ SO(n) ∈ R

• The numbers {σ1,…,σn} are the singular values of A • They may be any (complex) number, including zero! • They are generalizations of eigenvalues for square matrices

• Matrix U has the special property that it is “partially orthogonal” • UTU = I ∈ Rn×n • Note that VTV = I ∈ Rn×n because V ∈ SO(n) 1/27/14 USPAS 34 Singular Value Decomposition Generalizing Diagonalization • The columns of V are called the right singular vectors of A • The columns of U are called the left singular vectors of A

Again consider the matrix-vector equation

y = Ax = UDVT x

and perform the substitutions • ξ = VTx • η = UTy

We then have the (almost) equivalent equation

= D Like the case of a diagonalizable A , this = η ξ equation is completely decoupled ηi σiξi 1/27/14 USPAS 35

Vector Space: Basis Sets

• A basis for vector space V is a (possibly infinite) subset {e1, e2, e3, …} of V such that any v ∈ V can be expressed as a linear combination of basis vectors. • If v ∈ V then ∃ r1, r2, r3, … ∈ R such that v = r1e1 + r2e2 + r3e3 + …

• All basis sets of V have the same cardinality, called the dimension of V

• dim V = |{e1, e2, e3, …}|

• Examples: • (1,…,0), (0, 1, …,0), …, (0, 0, …,1) form the standard basis for Rn.

• {1, x, x2, …, xn-1} is a basis for the vector subspace {f ∈ C∞([0,1]) | dnf/dxn = 0}

• {1, cos 2πft, cos 4πft, …, cos (n-1)πft} is a basis for the set of all even, band-limited All these functions with period 1/f, and cutoff frequency nf. vector • The eigenvectors of any symmetric, pos. def. matrix A ∈ Rn×n form a basis of Rn. spaces are isomorphic!