Mohammed Zaidi A. Alqarni, Ph.D., December 2019 APPLIED MATHEMATICS PRECONDITIONERS FOR PDE-CONSTRAINED OPTIMIZATION PROBLEMS (86 pages) Dissertation Advisor: Jing Li Finding the solution for PDE-Constrained Optimization Problems can be accomplished in two ways: optimize-then-discretize and discretize-then-optimize. Both techniques lead to either a linear or non-linear system of equations. To obtain the optimal solution (to solve the system of equations), it's more practical to use numerical methods than analytical ones. Numerical methods can be direct or iterative. Generally, direct solvers tend to be more costly than iterative ones in terms of memory utilization and computational time. More- over, they are less flexible in taking advantage of multiple processors in parallel computers than iterative methods. Preconditioners are essential for the performance of iterative methods. They are bene- ficial in diminishing the overall computational time as a reduction in iteration counts and ideally their computing cost. Over the past few decades, a stack of preconditioners has been developed. Here, we propose preconditioners that reduce both the number of iterations and the computational time for a specific type of PDE-Constrained Optimization problems, that is optimal control problems. Subsequently, for the sake of clarity, we apply those precondi- tioners to these problems. PRECONDITIONERS FOR PDE-CONSTRAINED OPTIMIZATION PROBLEMS A dissertation submitted to Kent State University in partial fulfillment of the requirements for the degree of Doctor of Philosophy by Mohammed Zaidi A. Alqarni December 2019 c Copyright All rights reserved Except for previously published materials Dissertation written by Mohammed Zaidi A. Alqarni B.S., King Khalid University, 2011 M.A., Kent State University, 2015 Ph.D., Kent State University, 2019 Approved by Jing Li , Chair, Doctoral Dissertation Committee Lothar Reichel , Members, Doctoral Dissertation Committee Xiaoyu Zheng Qiang Guan Jong-Hoon Kim Accepted by Andrew M. Tonge , Chair, Department of Mathematical Sciences James L. Blank , Dean, College of Arts and Sciences TABLE OF CONTENTS TABLE OF CONTENTS . iv LIST OF FIGURES . vii LIST OF TABLES . viii ACKNOWLEDGMENTS . x 1 Introduction . 1 2 Partial Differential Equations and Finite Element Method . 4 2.1 Poisson's Equation . .5 2.2 (ADR) Advection-Diffusion-Reaction Equation . .9 3 Solving System of Linear Equations . 15 3.1 Direct Methods . 15 3.1.1 LU Factorization . 15 3.1.2 Cholesky Factorization . 16 3.2 Krylov Subspace Methods . 17 3.2.1 Generalized Minimal Residual Method (GMRES) . 18 3.2.2 Minimal Residual Method (MINRES) . 20 3.2.3 Conjugate Gradients Method (CG) . 21 3.3 Preconditioned Krylov Subspace Methods . 22 3.3.1 Preconditioned GMRES (PGMRES) . 23 3.3.2 Preconditioned MINRES (PMINRES) . 23 3.3.3 Preconditioned CG (PCG) . 24 iv 4 Domain Decompsition . 25 4.1 One-Level Overlapping Schwarz Methods . 25 4.2 Two-Level Overlapping Schwarz Methods . 30 4.3 Convergence Analysis . 35 5 PDE-Constrained Optimization Problems . 37 5.1 Solutions of PDE-Constrained Optimization Problems . 38 5.1.1 Discretize-then-Optimize . 38 5.1.2 Optimize-then-Discretize . 38 5.2 Discretization of the PDE-Constrained Optimization Problems . 38 5.2.1 Constraints as Poisson's Equation . 38 5.2.2 Constraints as Advection-Diffusion-Reaction Equation . 41 6 Preconditioners for Saddle-Point Systems . 44 6.1 Constraint by Poisson's Equation . 44 6.1.1 A Spectrally Equivalent Approximation . 46 6.1.2 A Tridiagonal Approximation . 48 6.1.3 Incomplete Cholesky Approximation . 48 6.2 Constraint by Advection-Diffusion-Reaction . 49 6.2.1 A Spectrally Equivalent Approximation . 51 6.2.2 Incomplete Cholesky Approximation . 52 7 Numerical Experiments . 54 7.1 Optimal Control with Poisson's Constriants . 54 7.2 Optimal Control with Advection-Diffusion-Reaction Constraints . 58 8 Conclusion and Future Work . 72 8.1 Conclusion . 72 8.2 Future Work . 73 v BIBLIOGRAPHY . 74 vi LIST OF FIGURES 1.1 Mach contours and streamlines for an aeroelastic simulation on the surface of an aircraft at Mach 0.9 [11]. .1 1.2 The front and back of a stem before and after applying a force to get the desired shape [17]. .2 2.3 Illustration of a piecewise linear basis function in 1D. .7 2.4 Illustration of a piecewise bilinear basis function in 2D. .7 4.5 Two overlapping subdomains. 27 4.6 Coarse and fine grids. 30 4.7 Interpolation of a node on the coarse grid for n = 7 and 3 × 3 subdomains. 32 7.8 Target temperature y............................... 55 7.9 State vs. control for φ = 2 × 100......................... 56 7.10 State vs. control for φ = 2 × 10−2......................... 56 7.11 State vs. control for φ = 2 × 10−4......................... 57 7.12 State vs. control for φ = 2 × 10−8......................... 57 7.13 Solution of the state for φ = 10−1 without stabilization. 61 7.14 Solution of the state for φ = 10−1 with stabilization. 62 7.15 State vs. control for φ = 2 × 100......................... 62 7.16 State vs. control for φ = 2 × 10−2......................... 63 7.17 State vs. control for φ = 2 × 10−4......................... 63 7.18 State vs. control for φ = 2 × 10−8......................... 63 vii LIST OF TABLES 7.1 MINRES itration counts and elapsed time for P, P1 , and P2 for different values of φ and h and for tol = 10−6....................... 58 h h h 7.2 MINRES itration counts and elapsed time for P , P1 , and P2 for different values of φ and h and for tol = 10−6....................... 59 T ri T ri T ri 7.3 MINRES itration counts and elapsed time for P , P1 , and P2 for different values of φ and h and for tol = 10−6....................... 60 ich ich ich 7.4 MINRES itration counts and elapsed time for P , P1 , and P2 for different values of φ and h and for tol = 10−6....................... 61 ich 7.5 Applying two-level overlapping Schwarz method in P1 , with δ = 2h..... 62 7.6 GMRES iteration counts and elapsed time for P, P1 , and P2 for different values of φ and h and for tol = 10−6....................... 64 h h h 7.7 GMRES iteration counts and elapsed time for P , P1 , and P2 for different values of φ and h and for tol = 10−6....................... 65 ich ich ich 7.8 GMRES iteration counts and elapsed time for P , P1 , and P2 for different values of φ and h and for tol = 10−6....................... 66 ich 7.9 Applying two-level overlapping Schwarz method in P1 , with δ = 2h using GMRES. 66 7.10 MINRES iteration counts and elapsed time for P, P1 , and P2 for different values of φ and h and for tol = 10−6....................... 67 h h h 7.11 MINRES iteration counts and elapsed time for P , P1 , and P2 for different values of φ and h and for tol = 10−6....................... 68 ich ich ich 7.12 MINRES iteration counts and elapsed time for P , P1 , and P2 for different values of φ and h and for tol = 10−6....................... 69 viii ich 7.13 Applying two-level overlapping Schwarz method in P1 , with δ = 2h using MINRES. 69 h ich 7.14 Applying GMRES with the preconditioners P2, P2 and P2 for the corre- sponding third blocks. 70 7.15 Applying MINRES with the preconditioners P , P h and P ich for the given third blocks. 70 h ich 7.16 Applying MINRES with the preconditioners P2, P2 and P2 for the corre- sponding third blocks. 71 ix ACKNOWLEDGMENTS All praise is due to Allah Who bestowed upon me the opportunity and ability to write this dissertation. Thereafter, I would like to express my gratitude and appreciation to the knowledgeable Dr. Jing Li for his excellent guidance, patience, and providing me with an auspicious atmosphere for conducting research. Many thanks to the Graduate School of Mathematical Sciences at Kent State University, especially the committee members for their input, valuable discussions and accessibility. I am indebted to King Khalid University, Saudi Arabia for their financial support and awarding me the scholarship for my studies here in the US. Words cannot express how grateful I am to my mother for all the sacrifices she made on my behalf, whose prayer has sustained me thus far. To my amazing family, and especially my wife, who continues to believes in me: Undoubtedly, this accomplishment would not have been possible without your love and support. I will never forget the sacrifices you made, leaving your friends, family and home; the weddings you missed to serve our humble family. This achievement is as much yours as it is mine. Last but not least, what would life be without precious and supportive friends? It is because of you that this journey was enjoyable and entertaining. It would be a shame to forget my editor who was knowledgeable and accessible. Finally, peace and prayers be upon our Master Muhammad, his family and companions, Amin. x CHAPTER 1 Introduction With the abundant development of technology, computers are used practically everywhere nowadays. Numerical computing is an example of a computer's benefit and is useful for solving sophisticated practical problems, like in the case of Partial Differential Equations (PDEs). These equations are widely used in many practical fields. Generally, their appear- ance comes from modeling physical phenomena. Thus, solving such substantial equations is a significant challenge, especially in the absence of their analytical solutions. Most of these challenges will appear in the implementation of the approximating approaches to these solu- tions { for example, the requirement of large memory and the cost of excessive computations. In general, the approximating techniques will provide (e.g., in this dissertation) an extensive system of linear equations that are preferred to be solved iteratively to overcome the lack of memory space and to be preconditioned to minimize the computational cost.