DOCSLIB.ORG

Randomized Iterative Methods with Alternating Projections

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Randomized Iterative Methods with Alternating Projections Randomized Iterative Methods with Alternating Projections Hua Xianga∗ Lin Zhangb† a School of Mathematics and Statistics, Wuhan University, Wuhan, 430072, P.R. China b Institute of Mathematics, Hangzhou Dianzi University, Hangzhou 310018, P.R. China. September 24, 2018 Abstract We use a unified framework to summarize sixteen randomized iterative methods including Kaczmarz method, coordinate descent method, etc. Some new iterative schemes are given as well. Some relationships with mg and ddm are also discussed. We analyze the convergence properties of the iterative schemes by using the matrix integrals associated with alternating projectors, and demonstrate the convergence behaviors of the randomized iterative methods by numerical examples. Keywords. linear systems, randomized iterative methods, Kaczmarz, co- ordinate descent, alternating projection 1 Introduction Solving a linear system is a basic task in scientific computing. Given a real matrix A Rm×n and a real vector b Rm, in this paper we consider the following consistent∈ linear system ∈ Ax = b. (1) For a large scale problem, iterative methods are more suitable than direct meth- ods. Though the classical iterative methods are generally deterministic [1, 3], arXiv:1708.09845v1 [math.NA] 31 Aug 2017 in this paper, we consider randomized iterative methods, for example, Kacz- marz method, the coordinate descent (cd) method and their block variants. Motivated by work in [4], we give a very general framework to such randomized iterative schemes. We need two matrices Y Rm×l and Z Rn×l, where l 6 min m,n , and usually l is much smaller than∈ m and n. In∈ most practical cases,{ only} one of ∗Corresponding author (H. Xiang). E-mail: [email protected]. H. Xiang is supported by the National Natural Science Foundation of China (No.11571265, No.11471253) and NSFC- RGC (No.11661161017). †E-mail: [email protected]. L. Zhang is supported by Natural Science Foundation of Zhe- jiang Province of China (LY17A010027) and National Natural Science Foundation of China (No.11301124). 1 the matrices Y and Z is given, and the other one is determined consequently. Suppose that Y and Z are of full column rank. Let E Y T AZ, which is invertible in most cases, otherwise we use the pseudoinverse≡ in the following. For convenience, we define Ξ := ZE†Y T = Z(Y T AZ)†Y T , where denotes the (Moore-Penrose) pseudoinverse. In this† paper, we consider the following iterative scheme: xk+1 = xk + Ξ(b Axk), (2) − where xk (k = 0, 1, ) is the kth approximate solution, and x0 is the initial guess. This is our general··· framework under consideration. Let T := I ΞA = I Z(Y T AZ)†Y T A. Then (2) can be rewritten as − − xk+1 = T xk + Ξb. (3) Assume that x∗ is the true solution of (1), and define the kth iteration error ek = xk x∗. Then we have the error propagation: − ek+1 = (I ΞA)ek = Tek. (4) − Besides the formulations (2) and (3), the iteration scheme can be viewed in seemingly different but equivalent ways as pointed out in [4]. From an algebraic viewpoint, xk+1 is the solution of Y T Ax = Y T b and x = xk + Zy. From a geometric viewpoint, xk+1 is the random intersect of two affine spaces. That is, xk+1 = (x∗ + Null(Y T A)) (xk + Range(Z)). { We} can easily check that∩ ΞA and T are projectors. If Y or Z is different in each step k, then the scheme is nonstationary and T depends on k, denoted by Tk. Choosing Y or Z properly in each step, we can recover the classical Kaczmarz method, and the coordinate descent method, etc. We will show how to achieve this in Section 2. Furthermore, from the error propagation, we have k+1 0 e = Tk T e , (5) ··· 0 where T0, ,Tk are a series of different alternating projectors applied to the initial iteration··· error e0. We show that under such alternating projections the final iteration error approaches zero in the sense of probability in Section 3. The convergence is proved by using the matrix integrals associated projections, without the assumption on the matrix positive definiteness in [4]. Numerical examples are given in Section 4. 2 Iterative schemes We consider the iterative methods under the unified framework of (2) or (3). Depending on the choice of parameter matrices Y and Z, we divide the iterative schemes into three types for convenience. In the following we always assume that the parameter matrix G is symmetric positive definite (spd). (1) Type-k methods. In this type, Y is given. We set Z = GAT Y , then consequently ΞA = ZE†Y T A = GAT Y (Y T AGAT Y )†Y T A. These are row action methods, and the Kaczmarz algorithm is the typical method of this type. 2 For this type, as pointed in [4], the iterative scheme can be expressed from a sketching viewpoint as k+1 k T T x = arg min x x −1 , s.t., Y Ax = Y b. || − ||G And also it can be rewritten from the optimization viewpoint as follows. k+1 ∗ k x = arg min x x −1 , s.t., x = x + Zy. || − ||G (2) Type-c methods. These are column action methods, and the coordinate descent algorithm is the typical method of this type. In this type, Z is given. we set Y = GAZ, then ΞA = ZE†Y T A = Z(ZT AT GAZ)†ZT AT GA. Similarly, for this type methods we can reexpress the iterative scheme from a sketching viewpoint as k+1 k T T x = arg min x x G, s.t., Y Ax = Y b, || − || b where A is supposed to be of full column rank, and then G := AT GA is spd. And also we can reformulate from the optimization viewpoint b k+1 ∗ k x = arg min x x G, s.t., x = x + Zy. || − || b (3) Type-s methods. This type of iterative schemes is applied to the symmet- ric cases. In this type, we choose Y = Z, then E = ZT AZ,Ξ= Z(ZT AZ)†ZT , and ΞA = Z(ZT AZ)†ZT A. For the spd case, under the settting G = A−1, type-k and type-c methods will be the same, and both reduce to type-s meth- ods. In the following, by choosing specific Y or Z, we recover the classical well- known iteration schemes, and also obtain some new ones. The iteration schemes are summarize in Tables 1 and 2. We will derive and discuss them one by one. 2.1 Kaczmarz and row methods The Kaczmarz method, also known as the algebraic reconstruction technique (art), is a typical row action method, dated back to the Polish mathematician Stefan Kaczmarz in 1937 and reignited by Strohmer and Vershynin [11]. The advantage of Kaczmarz method and the following cd lies in the fact that at a time they only need access to individual rows (or columns) rather than the entire coefficient matrix. Due to its simplicity, it has numerous applications in image reconstruction, signal processing, etc. In the Kaczmarz algorithm, the approximate solution x is projected onto the hyperplane determined by a random row of the matrix A and the respective T T element of the right hand side b. Let A = (α1, , αm). That is, αi is the ··· T T αi x bi ith row of A. The ith component of (1) reads α x = bi, i.e., = (i = i ||αi|| ||αi|| 1, ,m). For an approximation x not in the plane, the distance of this point ··· αT x x to the plane is bi i αi , which is used to update the approximate ||αi|| − ||αi|| ||αi|| solution. Then the iterative scheme reads: T bi αi x x x + − 2 αi, (6) ← αi || || 3 Table 1: Matrices in the randomized iterative methods No. Y Z E = Y T AZ Ξ= ZE†Y T T = I ΞA T T T T T T † T T− T T † T T K1 ei A ei e AA ei A ei(e AA ei) e I A ei(e AA ei) (A ei) i i i − i K2 ω AT ω ωT AAT ω AT ω(ωT AAT ω)†ωT I AT ω(ωT AAT ω)†ωT A T T T T T T † T − T T T † T K3 IR A IR I AA IR A IR(I AA IR) I I A IR(I AA IR) I A R R R − R R K4 Ω AT Ω ΩT AAT Ω AT Ω(ΩT AAT Ω)†ΩT I AT Ω(ΩT AAT Ω)†ΩT A T T T T T T † T − T T T † T K5 IR GA IR IR AGA IR GA IR(IR AGA IR) IR I GA IR(IR AGA IR) IR A K6 Ω GAT Ω ΩT AGAT Ω GAT Ω(ΩT AGAT Ω)†ΩT I − GAT Ω(ΩT AGAT Ω)†ΩT A T T T T † T − T T † T C1 Aej ej e A Aej ej(e A Aej ) (Aej ) I ej (e A Aej) (Aej ) A j j − j C2 Aω ω ωT AT Aω ω(ωT AT Aω)†ωT AT I ω(ωT AT Aω)†ωT AT A T T T T † T T − T T † T T C3 AIC IC I A AIC IC (I A AIC ) I A I IC (I A AIC ) I A A C C C − C C C4 AΩΩ ΩT AT AΩ Ω(ΩT AT AΩ)†ΩT AT I Ω(ΩT AT AΩ)†ΩT AT A T T T T † T T − T T † T T C5 GAIC IC I A GAIC IC (I A GAIC ) I A G I IC (I A GAIC ) I A GA C C C − C C C6 GAΩΩ ΩT AT GAΩ Ω(ΩT AT GAΩ)†ΩT AT G I Ω(ΩT AT GAΩ)†ΩT AT GA T T † T − T † T S1 ei ei ei Aei ei(ei Aei) ei I ei(ei Aei) ei A S2 ω ω ωT Aω ω(ωT Aω)†ωT I − ω(ωT Aω)†ωT A T T † T − T † T S3 IC IC IC AIC IC (IC AIC ) IC I IC (IC AIC ) IC A S4Ω Ω ΩT AΩ Ω(ΩT AΩ)†ΩT I − Ω(ΩT AΩ)†ΩT A − Table 2: Randomized iterative schemes No.
Load more

Recommended publications
  • Randomized Fast Solvers for Linear and Nonlinear Problems in Data Science
    UC Irvine UC Irvine Electronic Theses and Dissertations Title Randomized Fast Solvers for Linear and Nonlinear Problems in Data Science Permalink https://escholarship.org/uc/item/7c81w1kx Author Wu, Huiwen Publication Date 2019 License https://creativecommons.org/licenses/by/4.0/ 4.0 Peer reviewed|Thesis/dissertation eScholarship.org Powered by the California Digital Library University of California UNIVERSITY OF CALIFORNIA, IRVINE Randomized Fast Solvers for Linear and Nonlinear Problems in Data Science DISSERTATION submitted in partial satisfaction of the requirements for the degree of DOCTOR OF PHILOSOPHY in Math by Huiwen Wu Dissertation Committee: Professor Long Chen, Chair Professor Jack Xin Chancellor’s Professor Hongkai Zhao 2019 © 2019 Huiwen Wu DEDICATION To my family ii TABLE OF CONTENTS Page LIST OF FIGURES vi LIST OF TABLES vii ACKNOWLEDGMENTS ix CURRICULUM VITAE x ABSTRACT OF THE DISSERTATION xi 1 Introduction 1 2 Linear and Nonlinear Problems in Data Science 5 2.1 Least Squares Problem . .5 2.2 Linear Regression . .6 2.3 Logistic Regression . 10 2.3.1 Introduction . 10 2.3.2 Likelihood Function for Logistic Regression . 12 2.3.3 More on Logistic Regression Objective Function . 14 2.4 Non-constraint Convex and Smooth Minimization Problem . 16 3 Existing Methods for Least Squares Problems 20 3.1 Direct Methods . 21 3.1.1 Gauss Elimination . 21 3.1.2 Cholesky Decomposition . 21 3.1.3 QR Decomposition . 23 3.1.4 SVD Decomposition . 24 3.2 Iterative Methods . 25 3.2.1 Residual Correction Method . 25 3.2.2 Conjugate Gradient Method . 26 3.2.3 Kaczmarz Method .
    [Show full text]
  • A Novel Greedy Kaczmarz Method for Solving Consistent Linear Systems
    A NOVEL GREEDY KACZMARZ METHOD FOR SOLVING CONSISTENT LINEAR SYSTEMS∗ HANYU LI† AND YANJUN ZHANG† Abstract. With a quite different way to determine the working rows, we propose a novel greedy Kaczmarz method for solving consistent linear systems. Convergence analysis of the new method is provided. Numerical experiments show that, for the same accuracy, our method outperforms the greedy randomized Kaczmarz method and the relaxed greedy randomized Kaczmarz method introduced recently by Bai and Wu [Z.Z. BAI AND W.T. WU, On greedy randomized Kaczmarz method for solving large sparse linear systems, SIAM J. Sci. Comput., 40 (2018), pp. A592–A606; Z.Z. BAI AND W.T. WU, On relaxed greedy randomized Kaczmarz methods for solving large sparse linear systems, Appl. Math. Lett., 83 (2018), pp. 21–26] in term of the computing time. Key words. greedy Kaczmarz method, greedy randomized Kaczmarz method, greedy strategy, iterative method, consistent linear systems AMS subject classifications. 65F10, 65F20, 65K05, 90C25, 15A06 1. Introduction. We consider the following consistent linear systems (1.1) Ax = b, where A ∈ Cm×n, b ∈ Cm, and x is the n-dimensional unknown vector. As we know, the Kacz- marz method [10] is a popular so-called row-action method for solving the systems (1.1). In 2009, Strohmer and Vershynin [18] proved the linear convergence of the randomized Kaczmarz (RK) method. Following that, Needell [13] found that the RK method is not converge to the ordi- nary least squares solution when the system is inconsistent. To overcome it, Zouzias and Freris [20] extended the RK method to the randomized extended Kaczmarz (REK) method.
    [Show full text]
  • Parallelization and Implementation of Methods for Image Reconstruction Applications in GNSS Water Vapor Tomography
    UNIVERSIDADE DA BEIRA INTERIOR Faculdade Engenharia Parallelization and Implementation of Methods for Image Reconstruction Applications in GNSS Water Vapor Tomography Fábio André de Oliveira Bento Submitted to the University of Beira Interior in candidature for the Degree of Master of Science in Computer Science and Engineering Supervisor: Prof. Dr. Paul Andrew Crocker Co-supervisor: Prof. Dr. Rui Manuel da Silva Fernandes Covilhã, October 2013 Parallelization and Implementation of Methods for Image Reconstruction ii Parallelization and Implementation of Methods for Image Reconstruction Acknowledgements I would like to thank my supervisor Prof. Paul Crocker for all his support, enthusiasm, patience, time and effort. I am grateful for all his guidance in all of the research and writing of this dissertation. It was very important to me to work with someone with experience and knowledge and that believed in my work. I also would like to thank my co-supervisor Prof. Rui Fernandes for all his support in the research and writing of this dissertation. His experience and knowledge was really helpful especially in the GNSS area and the dissertation organization. I am really grateful for all his support and availability. The third person that I would like to thank is André Sá. He was always available to help me with all my GNSS doubts, his support was really important for the progress of this dissertation. i am really thankful for his time, enthusiasm and patience. I am also very grateful to Prof. Cédric Champollion for all his support and providing the ES- COMPTE data, David Adams for his support and providing the Chuva Project data and Michael Bender for his support and for providing German and European GNSS data.
    [Show full text]
  • Randomised Algorithms for Solving Systems of Linear Equations
    Randomised algorithms for solving systems of linear equations Per-Gunnar Martinsson Dept. of Mathematics & Oden Inst. for Computational Sciences and Engineering University of Texas at Austin Students, postdocs, collaborators: Tracy Babb, Ke Chen, Robert van de Geijn, Abinand Gopal, Nathan Halko, Nathan Heavner, James Levitt, Yijun Liu, Gregorio Quintana-Ortí, Joel Tropp, Bowei Wu, Sergey Voronin, Anna Yesypenko, Patrick Young. Slides: http://users.oden.utexas.edu/∼pgm/main_talks.html Research support by: Scope: Let A be a given m × n matrix (real or complex), and let b be a given vector. The talk is about techniques for solving Ax = b: Environments considered: (Time permitting . ) • A is square, nonsingular, and stored in RAM. • A is square, nonsingular, and stored out of core. (Very large matrix.) • A is rectangular, with m n. (Very over determined system.) • A is a graph Laplacian matrix. (Very large, and sparse). n d • A is an n × n “kernel matrix”, in the sense that given some set of points fxigi=1 ⊂ R , the ij entry of the matrix can be written as k(xi; xj) for some kernel function k. ? Scientific computing: High accuracy required. ? Data analysis: d is large (d = 4, or 10, or 100, or 1 000, . ). Techniques: The recurring theme is randomisation. • Randomized sampling. Typically used to build preconditioners. • Randomized embeddings. Reduce effective dimension of intermediate steps. Prelude: We introduce the ideas around randomized embeddings by reviewing randomized techniques for low rank approximation. (Recap of Tuesday talk.) Randomised SVD: Objective: Given an m × n matrix A of approximate rank k, compute a factorisation A ≈ UDV∗ m × n m × k k × k k × n where U and V are orthonormal, and D is diagonal.
    [Show full text]
  • A Kaczmarz Method with Simple Random Sampling for Solving Large Linear Systems
    A KACZMARZ METHOD WITH SIMPLE RANDOM SAMPLING FOR SOLVING LARGE LINEAR SYSTEMS YUTONG JIANG∗, GANG WU†, AND LONG JIANG‡ Abstract. The Kaczmarz method is a popular iterative scheme for solving large, consistent system of over- determined linear equations. This method has been widely used in many areas such as reconstruction of CT scanned images, computed tomography and signal processing. In the Kaczmarz method, one cycles through the rows of the linear system and each iteration is formed by projecting the current point to the hyperplane formed by the active row. However, the Kaczmarz method may converge very slowly in practice. The randomized Kaczmarz method (RK) greatly improves the convergence rate of the Kaczmarz method, by using the rows of the coefficient matrix in random order rather than in their given order. An obvious disadvantage of the randomized Kaczmarz method is its probability criterion for selecting the active or working rows in the coefficient matrix. In [Z.Z. BAI, W. WU, On greedy randomized Kaczmarz method for solving large sparse linear systems, SIAM Journal on Scientific Computing, 2018, 40: A592–A606], the authors proposed a new probability criterion that can capture larger entries of the residual vector of the linear system as far as possible at each iteration step, and presented a greedy randomized Kaczmarz method (GRK). However, the greedy Kaczmarz method may suffer from heavily computational cost when the size of the matrix is large, and the overhead will be prohibitively large for big data problems. The contribution of this work is as follows. First, from the probability significance point of view, we present a partially randomized Kaczmarz method, which can reduce the computational overhead needed in greedy randomized Kaczmarz method.
    [Show full text]
  • Solution of General Linear Systems of Equations Using Block Krylov Based
    Solution of general linear systems of equations using blo ck Krylov based iterative metho ds on distributed computing environments LeroyAnthony Drummond Lewis Decemb er i Resolution de systemes lineaires creux par des metho des iteratives par blo cs dans des environnements distribues heterogenes Resume Nous etudions limplantation de metho des iteratives par blo cs dans des environnements mul tipro cesseur amemoire distribuee p our la resolution de systemes lineaires quelconques Dans un premier temps nous nous interessons aletude du p otentiel de la metho de du gradient con jugue classique en environnement parallele Dans un deuxieme temps nous etudions une autre metho de iterativedelameme famille que celle du gradient conjugue qui a ete concue p our resoudre simultanementdessystemes lineaires avec de multiples seconds membres et commune mentreferencee sous le nom de gradient conjugue par blo cs La complexite algorithmique de la metho de du gradient conjugue par blo cs est superieure a celle de la metho de du gradientconjugue classique car elle demande plus de calculs par iteration et necessite en outre plus de memoire Malgre cela la metho de du gradient conjugue par blo cs apparat comme etant mieux adaptee aux environnements vectoriels et paralleles Nous avons etudie trois variantes du gradient conjugue par blo cs basees sur dierents mo deles de programmation parallele et leur ecaciteaete comparee sur divers environnements distribues Pour la resolution des systemes lineaires non symetriques nous considerons lutilisation de me thodes
    [Show full text]
  • Stochastic Reformulations of Linear Systems: Algorithms and Convergence Theory
    Stochastic Reformulations of Linear Systems: Algorithms and Convergence Theory Item Type Preprint Authors Richtarik, Peter; Takáč, Martin Eprint version Pre-print Publisher arXiv Rights Archived with thanks to arXiv Download date 25/09/2021 03:25:50 Link to Item http://hdl.handle.net/10754/626554.1 Stochastic Reformulations of Linear Systems: Algorithms and Convergence Theory Item Type Article Authors Richtarik, Peter; Takãč, Martin Citation Richtárik, P., & Takáč, M. (2020). Stochastic Reformulations of Linear Systems: Algorithms and Convergence Theory. SIAM Journal on Matrix Analysis and Applications, 41(2), 487–524. doi:10.1137/18m1179249 Eprint version Publisher's Version/PDF DOI 10.1137/18M1179249 Publisher arXiv; Society for Industrial & Applied Mathematics (SIAM) Journal SIAM Journal on Matrix Analysis and Applications Rights Archived with thanks to SIAM Journal on Matrix Analysis and Applications Download date 02/06/2020 12:35:39 Item License CCBY license Link to Item http://hdl.handle.net/10754/626554.1 Stochastic Reformulations of Linear Systems: Algorithms and Convergence Theory∗ Peter Richt´ariky Martin Tak´aˇcz June 7, 2017 Abstract We develop a family of reformulations of an arbitrary consistent linear system into a stochastic problem. The reformulations are governed by two user-defined parameters: a positive definite matrix defining a norm, and an arbitrary discrete or continuous distribution over random matrices. Our re- formulation has several equivalent interpretations, allowing for researchers from various communities to leverage their domain specific insights. In particular, our reformulation can be equivalently seen as a stochastic optimization problem, stochastic linear system, stochastic fixed point problem and a probabilistic intersection problem. We prove sufficient, and necessary and sufficient conditions for the reformulation to be exact.
    [Show full text]
  • Space-Efficient and Exact System Representations for the Nonlocal
    Space-efficient and exact system representations for the nonlocal protein electrostatics problem Dissertation zur Erlangung des Grades „Doktor der Naturwissenschaften” am Fachbereich Physik, Mathematik und Informatik der Johannes Gutenberg-Universität in Mainz Thomas Kemmer geb. in Wittlich Mainz, den 14. Oktober 2020 1. Berichterstatter: Prof. Dr. Andreas Hildebrandt 2. Berichterstatter: Prof. Dr. Bertil Schmidt Tag des Kolloquiums: 09.03.2021 D77 ii Abstract The study of proteins and protein interactions, which represent the main constituents of biological functions, plays an important role in modern biology. Application settings such as the development of pharmaceutical drugs and therapies rely on an accurate description of the electrostatics in biomolecular systems. This particularly includes nonlocal electrostatic contributions of the water-based solvents in the cell, which have a significant impact on the long-range visibility of immersed proteins. Existing mathematical models for the nonlocal protein electrostatics problem can be approached with numerical standard techniques, including boundary element methods (BEM). However, the typically large, dense, and asymmetric matrices involved in discretized BEM formulations previously prevented the application of the method to realistically-sized biomolecular systems. Here, we overcome this obstacle by developing implicit yet exact representations for such BEM matrices, capturing trillions of floating-point values in only a few megabytes of memory and without loss of information. We present generalized reference implementations for our implicit matrix types alongside specialized matrix operations for the Julia and Impala programming languages and demonstrate how they can be utilized with existing linear system solvers. On top of these implementations, we develop full- fledged CPU- and CUDA-based BEM solvers for the nonlocal protein electrostatics problem and make them available to the scientific community.
    [Show full text]
  • Accelerated Projection Methods for Computing Pseudoinverse Solutions of Systems of Linear Equations
    BIT 19 11979). 145,163 ACCELERATED PROJECTION METHODS FOR COMPUTING PSEUDOINVERSE SOLUTIONS OF SYSTEMS OF LINEAR EQUATIONS AKE BJORCK and TOMMY ELFVING Abstract. Iterative methods are developed for computing the Moore-Penrose pseudoinverse solution of a linear system Ax b, where A is an m x n sparse matrix. The methods do not require the explicit formation of AT A or AAT and therefore are advantageous to use when these matrices are much less sparse than A itself. The methods are based on solving the two related systems (i) x=ATy, AAly=b, and (ii) AT Ax=A1 b. First it is shown how the SOR­ and SSOR-methods for these two systems can be implemented efficiently. Further, the acceleration of the SSOR-method by Chebyshev semi-iteration and the conjugate gradient method is discussed. In particular it is shown that the SSOR-cg method for (i) and (ii) can be implemented in such a way that each step requires only two sweeps through successive rows and columns of A respectively. In the general rank deficient and inconsistent case it is shown how the pseudoinverse solution can be computed by a two step procedure. Some possible applications are mentioned and numerical results are given for some problems from picture reconstruction. 1. IntrodDction. Let A be a given m x n sparse matrix, b a given m-vector and x = A + b the Moore-Penrose pseudoinverse solution of the linear system of equations (1.1) Ax b. We denote the range and nullspace of a matrix A by R(A) and N(A) respectively.
    [Show full text]
  • Bibliography on the Kaczmarz Method the Original Paper of Kaczmarz
    January, 23rd, 2021 Andrzej Cegielski Institute of Mathematics University of Zielona Góra ul. Szafrana 4a, 65-516 Zielona Góra, Poland e-mail: [email protected] Bibliography on the Kaczmarz method The original paper of Kaczmarz [Kac37] S. Kaczmarz, Angenäherte Auflösung von Systemen linearer Gleichungen (Przyblizone· rozwi ¾azywanie uk÷adówrówna´n liniowych), Bull. Intern. Acad. Polonaise Sci. Lett., Cl. Sci. Math. Nat. A, 35 (1937) 355-357. English translation: S. Kaczmarz, Approximate solution of systems of linear equations, International Journal of Control, 57 (1993) 1269-1271. Publications which cite [Kac37] [1] J. Abaffy, E. Spedicato, A generalization of Huang’smethod for solving systems of linear algebraic equations, Bolletino dell’UnioneMatematica Italiana, 3B (1984) 517-529. [2] J. Abaffy, E. Spedicato, ABS Projection Algorithms: Mathematical Techniques for Linear and Nonlinear Equations, Halstead Press, Chichester UK, 1989. [3] M.I.K. Abir, Iterative CT Reconstruction from Few Projections for the Nondestructive Post Irradiation Examination of Nuclear Fuel Assemblies, Ph.D. Thesis, Missouri University of Science and Technology, Rolla, Missouri, USA, 2015. [4] A. Aboud, A Dualized Kaczmarz Algorithm in Hilbert and Banach Space, Ph.D. Thesis, Iowa State University, Ames, Iowa, USA, 2019. [5] M.F. Adams, A low memory, highly concurrent multigrid algorithm, arXiv:1207.6720v3 (2012). [6] M. Aftanas, Through Wall Imaging with UWB Radar System, Ph.D. Thesis, Technical University of Koice, Koice, Slovakia, 2009. [7] A. Agaskar, Problems in Signal Processing and Inference on Graphs, Ph.D. Thesis, Harvard University, Cambridge, MA, USA, 2015. [8] A. Agaskar, C. Wang, Y.M. Lu, Randomized Kaczmarz algorithms: Exact MSE analysis and optimal sampling probabilities, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Atlanta, GA, 3-5 Dec.
    [Show full text]
  • A MATLAB Package of Algebraic Iterative Reconstruction Methods
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Elsevier - Publisher Connector Journal of Computational and Applied Mathematics 236 (2012) 2167–2178 Contents lists available at SciVerse ScienceDirect Journal of Computational and Applied Mathematics journal homepage: www.elsevier.com/locate/cam AIR Tools — A MATLAB package of algebraic iterative reconstruction methodsI Per Christian Hansen ∗, Maria Saxild-Hansen Department of Informatics and Mathematical Modelling, Technical University of Denmark, DK-2800 Lyngby, Denmark article info a b s t r a c t Article history: We present a MATLAB package with implementations of several algebraic iterative Received 29 October 2010 reconstruction methods for discretizations of inverse problems. These so-called row action methods rely on semi-convergence for achieving the necessary regularization of the MSC: problem. Two classes of methods are implemented: Algebraic Reconstruction Techniques 65F22 (ART) and Simultaneous Iterative Reconstruction Techniques (SIRT). In addition we provide 65Y15 a few simplified test problems from medical and seismic tomography. For each iterative Keywords: method, a number of strategies are available for choosing the relaxation parameter and the ART methods stopping rule. The relaxation parameter can be fixed, or chosen adaptively in each iteration; SIRT methods in the former case we provide a new ``training'' algorithm that finds the optimal parameter Semi-convergence for a given test problem. The stopping rules provided are the discrepancy principle, the Relaxation parameters monotone error rule, and the NCP criterion; for the first two methods ``training'' can be Stopping rules used to find the optimal discrepancy parameter. Tomographic imaging ' 2011 Elsevier B.V.
    [Show full text]
  • AIR Tools a MATLAB Package of Algebraic Iterative Reconstruction Techniques
    AIR Tools A MATLAB Package of Algebraic Iterative Reconstruction Techniques Version 1.2 for Matlab 8.0 Per Christian Hansen Maria Saxild-Hansen Department of Applied Mathematics and Computer Science Building 324, Technical University of Denmark DK-2800 Lyngby, Denmark October 2014 Abstract This collection of MATLAB software contains implementations of several Algebraic Iterative Reconstruction methods for discretizations of inverse problems. These so-called row action methods rely on semi-convergence for achieving the necessary regularization of the problem. Two classes of methods are implemented: Algebraic Reconstruction Techniques (ART) and Simultaneous Iterative Reconstruction Techniques (SIRT). In addition we provide a few simplified test problems from medical and seismic tomography. For each iterative method, a number of strategies are available for choosing the relaxation parameter and the stopping rule. The relaxation parameter can be fixed, or chosen adaptively in each iteration; in the former case we provide a new \training" algorithm that finds the optimal parameter for a given test problem. The stopping rules provided are the discrepancy principle, the monotone error rule, and the NCP criterion; for the first two methods \training" can be used to find the optimal discrepancy parameter. The corresponding manuscript is: • P. C. Hansen and M. Saxild-Hansen, AIR Tools { A MATLAB Package of Algebraic Iterative Reconstruction Techniques, Journal of Computational and Applied Mathemat- ics, 236 (2012), pp. 2167{2178; doi:10.1016/j.cam.2011.09.039. We have included the most common algebraic iterative reconstruction methods in the package { but we left out block versions of the methods, which are better suited for other programming languages than MATLAB.
    [Show full text]