DOCSLIB.ORG

Least Squares Solutions to Linear Systems of Equations

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Least Squares Solutions to Linear Systems of Equations 1 2 Least Squares with Examples in Signal Processing The `sum of squares' of x is denoted by kxk2, i.e., Ivan Selesnick March 7, 2013 NYU-Poly 2 X 2 T kxk2 = jx(n)j = x x: (3) These notes address (approximate) solutions to linear equations by least n squares. We deal with the `easy' case wherein the system matrix is full The `energy' of a vector x refers to kxk2. rank. If the system matrix is rank deficient, then other methods are 2 In these notes, it is assumed that all vectors and matrices are real-valued. needed, e.g., QR decomposition, singular value decomposition, or the In the complex-valued case, the conjugate transpose should be used in place pseudo-inverse, [2,3]. of the transpose, etc. In these notes, least squares is illustrated by applying it to several basic problems in signal processing: 2 Overdetermined equations 1. Linear prediction Consider the system of linear equations 2. Smoothing y = Hx: 3. Deconvolution If there is no solution to this system of equations, then the system is 4. System identification `overdetermined'. This frequently happens when H is a `tall' matrix (more rows than columns) with linearly independent columns. 5. Estimating missing data In this case, it is common to seek a solution x minimizing the energy of For the use of least squares in filter design, see [1]. the error: 2 1 Notation J(x) = ky − Hxk2: We denote vectors in lower-case bold, i.e., Expanding J(x) gives 2 x 3 1 T 6 7 J(x) = (y − Hx) (y − Hx) (4) 6 x2 7 x = 6 : 7 : (1) = yT y − yT Hx − xT HT y + xT HT Hx (5) 6 : 7 4 : 5 = yT y − 2yT Hx + xT HT Hx: (6) xN We denote matrices in upper-case bold. The transpose of a vector or matrix Note that each of the four terms in (5) are scalars. Note also that the scalar in indicated by a superscript T , i.e., xT is the transpose of x. xT HT y is the transpose of the scalar yT Hx, and hence xT HT y = yT Hx. The notation kxk2 refers to the Euclidian length of the vector x, i.e., Taking the derivative (see AppendixA), gives q @ kxk = jx j2 + jx j2 + ··· + jx j2: (2) J(x) = −2HT y + 2HT Hx 2 1 2 N @x 1 For feedback/corrections, email [email protected]. Setting the derivative to zero, Matlab software to reproduce the examples in these notes is available on the web or from the author. http://eeweb.poly.edu/iselesni/lecture_notes/ @ J(x) = 0 =) HT Hx = HT y Last edit: November 26, 2013 @x 1 Let us assume that HT H is invertible. Then the solution is given by Set the derivatives to zero to get: T −1 T 1 x = (H H) H y: x = HT µ (11) 2 This is the `least squares' solution. y = Hx (12) 2 T −1 T min ky − Hxk2 =) x = (H H) H y (7) x Plugging (11) into (12) gives In some situations, it is desirable to minimize the weighted square error, 1 T P 2 y = HH µ: i.e., n wn rn where r is the residual, or error, r = y − Hx, and wn are 2 positive weights. This corresponds to minimizing kW1=2(y − Hx)k2 where 2 Let us assume HHT is invertible. Then W is the diagonal matrix, [W]n;n = wn. Using (7) gives T −1 1=2 2 T −1 T µ = 2(HH ) y: (13) min kW (y − Hx)k2 =) x = (H WH) H Wy (8) x where we have used the fact that W is symmetric. Plugging (13) into (11) gives the `least squares' solution: 3 Underdetermined equations x = HT (HHT )−1y: Consider the system of linear equations We can verify that x in this formula does in fact satisfy y = Hx by plugging y = Hx: in: T T −1 T T −1 If there are many solutions, then the system is `underdetermined'. This Hx = H H (HH ) y = (HH )(HH ) y = y X frequently happens when H is a `wide' matrix (more columns than rows) with linearly independent rows. So, In this case, it is common to seek a solution x with minimum norm. That 2 T T −1 min kxk2 s.t. y = Hx =) x = H (HH ) y: (14) is, we would like to solve the optimization problem x 2 In some situations, it is desirable to minimize the weighted energy, i.e., min kxk2 (9) x P 2 n wn xn, where wn are positive weights. This corresponds to minimizing such that y = Hx: (10) 1=2 2 kW xk2 where W is the diagonal matrix, [W]n;n = wn. The derivation Minimization with constraints can be done with Lagrange multipliers. So, of the solution is similar, and gives define the Lagrangian: 1=2 2 −1 T −1 T −1 min kW xk2 s.t. y = Hx =) x = W H HW H y 2 T x L(x; µ) = kxk2 + µ (y − Hx) (15) Take the derivatives of the Lagrangian: @ This solution is also derived below, see (25). L(x) = 2x − HT µ @x 4 Regularization @ In the overdetermined case, we minimized ky − Hxk2. In the underde- L(x) = y − Hx 2 @µ 2 termined case, we minimized kxk2. Another approach is to minimize the 2 2 2 weighted sum: c1 ky − Hxk2 + c2 kxk2. The solution x depends on the Setting the derivative to zero, ratio c =c , not on c and c individually. 2 1 1 2 @ A common approach to obtain an inexact solution to a linear system is J(x) = 0 =) HT Hx + λAT Ax = HT y @x to minimize the objective function: =) (HT H + λAT A) x = HT y J(x) = ky − Hxk2 + λ kxk2 (16) 2 2 So the solution is given by where λ > 0. Taking the derivative, we get x = (HT H + λAT A)−1HT y @ J(x) = 2HT (Hx − y) + 2λx @x So, Setting the derivative to zero, 2 2 min ky − Hxk2 + λ kAxk2 x @ T T (18) J(x) = 0 =) H Hx + λx = H y T T −1 T @x =) x = (H H + λA A) H y =) (HT H + λI) x = HT y Note that if A is the identity matrix, then equation (18) becomes (17). So the solution is given by 6 Constrained least squares x = (HT H + λI)−1HT y Constrained least squares refers to the problem of finding a least squares solution that exactly satisfies additional constraints. If the additional So, constraints are a set of linear equations, then the solution is obtained as 2 2 T −1 T min ky − Hxk2 + λ kxk2 =) x = (H H + λI) H y (17) follows. x The constrained least squares problem is of the form: This is referred to as `diagonal loading' because a constant, λ, is added 2 T min ky − Hxk2 (19) to the diagonal elements of H H. The approach also avoids the problem x T T of rank deficiency because H H + λI is invertible even if H H is not. In such that Cx = b (20) addition, the solution (17) can be used in both cases: when H is tall and when H is wide. Define the Lagrangian, 5 Weighted regularization 2 T L(x; µ) = ky − Hxk2 + µ (Cx − b): A more general form of the regularized objective function (16) is: The derivatives are: J(x) = ky − Hxk2 + λ kAxk2 2 2 @ L(x) = 2HT (Hx − y) + CT µ where λ > 0. Taking the derivative, we get @x @ @ J(x) = 2HT (Hx − y) + 2λAT Ax L(x) = Cx − b @x @µ 3 Setting the derivatives to zero, 6.1 Special cases @ Simpler forms of (23) are frequently useful. For example, if H = I and L(x) = 0 =) x = (HT H)−1(HT y − 0:5CT µ) (21) @x b = 0 in (23), then we get @ 2 L(x) = 0 =) Cx = b (22) min ky − xk2 s.t. Cx = 0 @µ x (24) −1 =) x = y − CT CCT Cy Multiplying (21) on the left by C gives Cx, which from (22) is b, so we have If y = 0 in (23), then we get T −1 T T C(H H) (H y − 0:5C µ) = b 2 min kHxk2 s.t. Cx = b x (25) −1 or, expanding, =) x = (HT H)−1 CT C(HT H)−1CT b T −1 T T −1 T C(H H) H y − 0:5C(H H) C µ = b: If y = 0 and H = I in (23), then we get Solving for µ gives 2 T T −1 min kxk2 s.t. Cx = b =) x = C CC b (26) x −1 µ = 2 C(HT H)−1CT C(HT H)−1HT y − b which is the same as (14). Plugging µ into (21) gives 7 Note T −1 T T T −1 T −1 T −1 T x = (H H) H y − C C(H H) C C(H H) H y − b The expressions above involve matrix inverses. For example, (7) involves T −1 Let us verify that x in this formula does in fact satisfy Cx = b, (H H) . However, it must be emphasized that finding the least square solution does not require computing the inverse of HT H even though the T −1 T Cx = C(H H) H y− inverse appears in the formula. Instead, x in (7) should be obtained, in −1 T T CT C(HT H)−1CT C(HT H)−1HT y − b practice, by solving the system Ax = b where A = H H and b = H y.
Load more

Recommended publications
  • 8.5 Least Squares Solutions to Inconsistent Systems
    8.5 Least Squares Solutions to Inconsistent Systems Performance Criterion: 8. (f) Find the least-squares approximation to the solution of an inconsistent system of equations. Solve a problem using least-squares approximation. (g) Give the least squares error and least squares error vector for a least squares approximation to a solution to a system of equations. Recall that an inconsistent system is one for which there is no solution. Often we wish to solve inconsistent systems and it is just not acceptable to have no solution. In those cases we can find some vector (whose components are the values we are trying to find when attempting to solve the system) that is “closer to being a solution” than all other vectors. The theory behind this process is part of the second term of this course, but we now have enough knowledge to find such a vector in a “cookbook” manner. Suppose that we have a system of equations Ax = b. Pause for a moment to reflect on what we know and what we are trying to find when solving such a system: We have a system of linear equations, and the entries of A are the coefficients of all the equations. The vector b is the vector whose components are the right sides of all the equations, and the vector x is the vector whose components are the unknown values of the variables we are trying to find. So we know A and b and we are trying to find x. If A is invertible, the solution vector x is given by x = A−1 b.
    [Show full text]
  • Linear Algebra I
    Linear Algebra I Gregg Waterman Oregon Institute of Technology c 2016 Gregg Waterman This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. The essence of the license is that You are free: to Share to copy, distribute and transmit the work • to Remix to adapt the work • Under the following conditions: Attribution You must attribute the work in the manner specified by the author (but not in • any way that suggests that they endorse you or your use of the work). Please contact the author at [email protected] to determine how best to make any attribution. Noncommercial You may not use this work for commercial purposes. • Share Alike If you alter, transform, or build upon this work, you may distribute the resulting • work only under the same or similar license to this one. With the understanding that: Waiver Any of the above conditions can be waived if you get permission from the copyright • holder. Public Domain Where the work or any of its elements is in the public domain under applicable • law, that status is in no way affected by the license. Other Rights In no way are any of the following rights affected by the license: • Your fair dealing or fair use rights, or other applicable copyright exceptions and limitations; ⋄ The author’s moral rights; ⋄ Rights other persons may have either in the work itself or in how the work is used, such as ⋄ publicity or privacy rights. Notice For any reuse or distribution, you must make clear to others the license terms of this • work.
    [Show full text]
  • Underdetermined and Overdetermined Linear Algebraic Systems
    Underdetermined and Overdetermined Linear Algebraic Systems ES100 March 1, 1999 T.S. Whitten Objectives ● Define underdetermined systems ● Define overdetermined systems ● Least Squares Examples Review B 500 N = − ° = ∑ Fx 0; 500 FBC sin 45 0 = ° − = ∑ Fy 0; FBC cos 45 FBA 0 FBC FBA − sin 45° 0 F − 500 ⋅ BC = ° 1cos42445 4341 1F2BA3 0 coefficients variables Review cont. − sin 45° 0 F − 500 ⋅ BC = ° 1cos42445 4341 1F2BA3 0 coefficients variables The system of matrices above is of the form: Ax = b and can be solved using MATLAB left division thus, x = A\b × results in a 1 2 matrix of values for FBC and FBA Review Summary ● A system of two Equations and two unknowns may yield a unique solution. ● The exception is when the determinant of A is equal to zero. Then the system is said to be singular. ● The left division operator will solve the linear system in one step by combining two matrix operations ●A\B is equivalent to A-1*B Graphical Representation of Unique vs. Singular Systems unique solution y singular system x Underdetermined Systems ● A system of linear equations is may be undetermined if; 1 The determinant of A is equal to zero A = 0 2 The matrix A is not square, i.e. the are more unknowns than there are equations + + = x x 3y 2z 2 1 3 2 2 ⋅ y = x + y + z = 4 1 1 1 4 z Overdetermined Systems ● The converse of an underdetermined system is an overdetermined system where there are more equations than there are variables ● This situation arises frequently in engineering.
    [Show full text]
  • Solutions of Overdetermined Linear Problems (Least Square Problems)
    Chapter 3 Solutions of overdetermined linear problems (Least Square problems) 1. Over-constrained problems See Chapter 11 from the textbook 1.1. Definition In the previous chapter, we focused on solving well-defined linear problems de- fined by m linear equations for m unknowns, put into a compact matrix-vector form Ax = b with A an m m square matrix, and b and x m long column vectors. We focussed on using⇥ direct methods to seek exact solutions− to such well-defined linear systems, which exist whenever A is nonsingular. We will re- visit these problems later this quarter when we learn about iterative methods. In this chapter, we look at a more general class of problems defined by so- called overdetermined systems – systems with a larger numbers of equations (m) than unknowns (n): this time, we have Ax = b with A an m n matrix with m>n, x an n-long column vector and b an m long column vector.⇥ This time there generally are no exact solutions to the problem.− Rather we now want to find approximate solutions to overdetermined linear systems which minimize the residual error E = r = b Ax (3.1) || || || − || using some norm. The vector r = b Ax is called the residual vector. − Any choice of norm would generally work, although, in practice, we prefer to use the Euclidean norm (i.e., the 2-norm) which is more convenient for nu- merical purposes, as they provide well-established relationships with the inner product and orthogonality, as well as its smoothness and convexity (we shall see later).
    [Show full text]
  • Math 253 - Homework 2 Due in Class on Wednesday, February 19
    Math 253 - Homework 2 Due in class on Wednesday, February 19 Write your answers clearly and carefully, being sure to emphasize your answer and the key steps of your work. You may work with others in this class, but the solutions handed in must be your own. If you work with someone or get help from another source, give a brief citation on each problem for which that is the case. Part I While you are expected to complete all of these problems, do not hand in the problems in Part I. You are encouraged to write complete solutions and to discuss them with me or your peers. As extra motivation, some of these problems will appear on the weekly quizzes. 1. Practice Problems: (a) Section 1.3: 1, 2 2. Exercises: (a) Section 1.3: 1-13 odd, 19, 21, 25, 27, 29, 33 Part II Hand in each problem separately, individually stapled if necessary. Please keep all problems together with a paper clip. n 1. The center of mass of v1;:::; vk in R , with a mass of mi at vi, for i = 1; : : : ; k and total mass m = m1 + ··· + mk, is given by m v + ··· + m v m m v¯ = 1 1 k k = 1 v + ··· + k v : m m 1 m k n The centroid, which can be thought of as the geometric center, of v1;:::; vk in R is the center of mass with mi = 1 for each i = 1; : : : ; k, i.e. v + ··· + v 1 1 c¯ = 1 k = v + ··· + v : k k 1 k k 243 2−13 2 1 3 2−53 2 1 3 Pk (a) Let v1 = 415 ; v2 = 4 0 5 ; v3 = 4 3 5 ; v4 = 4 0 5 ; v5 = 4−45 : Let uk = i=1 vi.
    [Show full text]
  • Investigating an Overdetermined System of Linear Equations by Using Convex Functions
    Hacettepe Journal of Mathematics and Statistics Volume 46 (5) (2017), 865 874 Investigating an overdetermined system of linear equations by using convex functions Zlatko Pavi¢ ∗ y and Vedran Novoselac z Abstract The paper studies the application of convex functions in order to prove the existence of optimal solutions of an overdetermined system of lin- ear equations. The study approaches the problem by using even convex functions instead of projections. The research also relies on some spe- cial properties of unbounded convex sets, and the lower level sets of continuous functions. Keywords: overdetermined system, convex function, global minimum. 2000 AMS Classication: 15A06, 26B25. Received : 30.08.2016 Accepted : 19.12.2016 Doi : 10.15672/ HJMS.2017.423 1. Introduction We consider a system of m linear equations with n unknowns over the eld of real numbers given by a11x1 + ::: + a1nxn = b1 . (1.1) . .. : am1x1+ ::: + amnxn = bm Including the matrices 2 a11 : : : a1n 3 2 x1 3 2 b1 3 . (1.2) 6 . .. 7 6 . 7 6 . 7 A = 4 . 5 ; x = 4 . 5 ; b = 4 . 5 ; am1 : : : amn xn bm the given system gets the matrix form (1.3) Ax = b: ∗Department of Mathematics, Mechanical Engineering Faculty in Slavonski Brod, University of Osijek, Croatia, Email: [email protected] yCorresponding Author. zDepartment of Mathematics, Mechanical Engineering Faculty in Slavonski Brod, University of Osijek, Croatia, Email: [email protected] 866 n m Identifying the matrix A with a linear operator from R to R , the column matrix x n m with a vector of R , and the column matrices Ax and b with vectors of R , the given system takes the operator form.
    [Show full text]
  • System of Linear Equations - Wikipedia, the Free Encyclopedia
    System of linear equations - Wikipedia, the free encyclopedia http://en.wikipedia.org/wiki/System_of_linear_equations System of linear equations From Wikipedia, the free encyclopedia In mathematics, a system of linear equations (or linear system) is a collection of linear equations involving the same set of variables. For example, is a system of three equations in the three variables x, y, z. A solution to a linear system is an assignment of numbers to the variables such that all the equations are simultaneously satisfied. A solution to the system above is given by A linear system in three variables determines a collection of planes. The intersection point is the solution. since it makes all three equations valid.[1] In mathematics, the theory of linear systems is the basis and a fundamental part of linear algebra, a subject which is used in most parts of modern mathematics. Computational algorithms for finding the solutions are an important part of numerical linear algebra, and play a prominent role in engineering, physics, chemistry, computer science, and economics. A system of non-linear equations can often be approximated by a linear system (see linearization), a helpful technique when making a mathematical model or computer simulation of a relatively complex system. Very often, the coefficients of the equations are real or complex numbers and the solutions are searched in the same set of numbers, but the theory and the algorithms apply for coefficients and solutions in any field. For solutions in an integral domain like the ring of the integers, or in other algebraic structures, other theories have been developed.
    [Show full text]
  • Low-Rank Approximation
    Ivan Markovsky Low-Rank Approximation Algorithms, Implementation, Applications January 19, 2018 Springer vi Preface Preface Low-rank approximation is a core problem in applications. Generic examples in systems and control are model reduction and system identification. Low-rank approximation is equivalent to the principal component analysis method in machine learning. Indeed, dimensionality reduction, classification, and information retrieval problems can be posed and solved as particular low-rank approximation problems. Sylvester structured low-rank approximation has applications in computer algebra for the decoupling, factorization, and common divisor computation of polynomials. The book covers two complementary aspects of data modeling: stochastic esti- mation and deterministic approximation. The former aims to find from noisy data that is generated by a low-complexity system an estimate of that data generating system. The latter aims to find from exact data that is generated by a high com- plexity system a low-complexity approximation of the data generating system. In applications, both the stochastic estimation and deterministic approximation aspects are present: the data is imprecise due to measurement errors and is possibly gener- Simple linear models are commonly used in engineering despite of the fact that ated by a complicated phenomenon that is not exactly representable by a model in the real world is often nonlinear. At the same time as being simple, however, the the considered model class. The development of data modeling methods in system models have to be accurate. Mathematical models are obtained from first princi- identification and signal processing, however, has been dominated by the stochastic ples (natural laws, interconnection, etc.) and experimental data.
    [Show full text]
  • Computing the Strict Chebyshev Solution of Overdetermined Linear Equations
    MATHEMATICS OF COMPUTATION, VOLUME 31, NUMBER 140 OCTOBER 1977, PAGES 974-983 Computing the Strict Chebyshev Solution of Overdetermined Linear Equations By Nabih N. Abdelmalek Abstract. A method for calculating the strict Chebyshev solution of overdetermined systems of linear equations using linear programming techniques is described. This method provides: (1) a way to determine, for the majority of cases, all the equations belonging to the characteristic set, (2) an efficient method to obtain the inverse of the matrix needed to calculate the strict Chebyshev solution, and (3) a way of recog- nizing when an element of the Chebyshev solution equals a corresponding element of the strict Chebyshev solution. As a result, in general, the computational effort is con- siderably reduced. Also the present method deals with full rank as well as rank defi- cient cases. Numerical results are given. 1. Introduction. Consider the overdetermined system of linear equations (1) Ca = f, where C is a given real n x m matrix of rank k < m < n and / is a given real «-vector. Let E denote the set of n equations (1). The Chebyshev solution (C.S.) of system (1) is the OT-vectora* = (a*) which minimizes the Chebyshev norm z, (2) z = max\r¡(a)\, i E E, where r, is the rth residual in (1) and is given by (3) a-,.= cixax+ ••• + cimam - f¡, i E E. Let us denote the C.S. a* by (a*)c s . It is known that if C satisfies the Haar condition, the C.S. (a*)c s is unique. Otherwise it may not be unique.
    [Show full text]
  • ECEU692 Intro to SSI Course Notes Some Selected Topics in Linear
    EECE5698 SP 2011 DHB Set 2 D.H. Brooks ECEU692 Intro to SSI Course Notes Some selected topics in linear algebra for SSI (2) Prof. Brooks Northeastern University Spring 2011 This talk will relate to parts of • Chapter 5 • Appendix B EECE5698 SP 2011 DHB Set 2 D.H. Brooks Outline of those Selected Topics • Some subsurface problems that lead to linear systems of equations • Some different ways of looking at n equations in n unknowns: in particular, some geometric interpretations • The geometry of when square linear systems have: – exact solutions, – no solutions, and/or – non-unique solutions • Problems where you have more equations than unknowns • Problems where you have more unknowns than equations • A quick discussion of solving linear systems in Matlab • The Singular Value Decomposition to analyze linear systems 2 EECE5698 SP 2011 DHB Set 2 D.H. Brooks Outline of these Notes: Linear Systems that are not Square • What about errors in the model? • Overdetermined systems: motivation and geometry • “Solving” overdetermined systems via least-squares • What about when you can’t make enough measurements ? • Underdetermined systems: motivation and geometry • A few words on underdetermined systems • A few words on solving linear systems in Matlab 3 EECE5698 SP 2011 DHB Set 2 D.H. Brooks Reflectance Spectroscopy Again: Errors in our Assumptions In the reflectance spectroscopy problem, we have several sources of error: • We may not know, or be able to account for, all the chromophores present, e.g. – some chromophore is in the medium we did not expect, or – some chromophore is in the medium that we thought would be negligible but in fact is not • We may not have exact values for the spectra, e.g.
    [Show full text]
  • Arxiv:Math/0206212V3
    QUANTUM COHOMOLOGY VIA D-MODULES Martin A. Guest Dedicated to Graeme Segal on the occasion of his 60th birthday Abstract. We propose a new point of view on quantum cohomology, motivated by the work of Givental and Dubrovin, but closer to differential geometry than the existing approaches. The central object is a D-module which “quantizes” a commutative algebra associated to the (uncompactified) space of rational curves. Under appropriate conditions, we show that the associated flat connection may be gauged to the flat connection underlying quantum cohomology. This method clarifies the role of the Birkhoff factorization in the “mirror trans- formation”, and it gives a new algorithm (requiring construction of a Groebner basis and solution of a system of o.d.e.) for computation of the quantum product. Quantum cohomology first arose in physics, and its (mathematically conjectural) prop- erties were supported by physical intuition. A rigorous mathematical definition came later, based on deep properties of certain moduli spaces. We shall propose another point of view on quantum cohomology, closer in spirit to differential geometry. The main ingredient in our approach is a flat connection, considered as a holonomic D-module (or maximally overdetermined system of p.d.e.). This object itself is not new: Givental’s “quantum cohomology D-module” is already well known ([Gi1]), and the asso- ciated flat connection appears in Dubrovin’s theory of Frobenius manifolds ([Du]). But, in the existing literature, the D-module plays a subservient role, being a consequence of arXiv:math/0206212v3 [math.DG] 6 Oct 2004 the construction of the Gromov-Witten invariants and the quantum cohomology algebra.
    [Show full text]
  • Overdetermined Elliptic Systems
    FoCM manuscript No. (will be inserted by the editor) Overdetermined Elliptic Systems Katsiaryna Krupchyk1, Werner M. Seiler2, Jukka Tuomela1 1 Dept. of Mathematics, University of Joensuu, Finland, e-mail: [email protected], [email protected] 2 Interdisciplinary Centre for Scientific Computing, Universitat¨ Heidelberg, 69120 Heidel- berg, Germany, e-mail: [email protected] Abstract We consider linear overdetermined systems of partial differential equa- tions. We show that the introduction of weights classically used for the definition of ellipticity is not necessary, as any system that is elliptic with respect to some weights becomes elliptic without weights during its completion to involution. Fur- thermore, it turns out that there are systems which are not elliptic for any choice of weights but whose involutive form is nevertheless elliptic. We also show that reducing the given system to lower order or to an equivalent one with only one unknown function preserves ellipticity. Key words Overdetermined system, partial differential equation, symbol, ellip- tic system, completion, involution 1 Introduction The definition of ellipticity for general overdetermined systems is quite rarely found in the literature, one accessible exception being the encyclopaedia article [15, Def. 2.1]. Without the general definition one may encounter conceptual prob- lems already in very simple situations. For instance, consider the transformation of the two-dimensional Laplace equation uxx + uyy = 0 to the first order system (this is discussed in the recent textbook [42, Example 2.10]): ux = v , uy = w , vx + wy = 0 . The transformed system is not elliptic, although it is obviously equivalent to La- place’s equation.
    [Show full text]