Branch-And-Bound with Peer-To-Peer for Large-Scale Grids Alexandre Di Costanzo

Total Page:16

File Type:pdf, Size:1020Kb

Branch-And-Bound with Peer-To-Peer for Large-Scale Grids Alexandre Di Costanzo Branch-and-Bound with Peer-to-Peer for Large-Scale Grids Alexandre Di Costanzo To cite this version: Alexandre Di Costanzo. Branch-and-Bound with Peer-to-Peer for Large-Scale Grids. Networking and Internet Architecture [cs.NI]. Université Nice Sophia Antipolis, 2007. English. tel-00221450 HAL Id: tel-00221450 https://tel.archives-ouvertes.fr/tel-00221450 Submitted on 28 Jan 2008 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. THÈSE DE DOCTORAT École Doctorale « Sciences et Technologies de l’Information et de la Communication » de Nice - Sophia Antipolis Discipline Informatique UNIVERSITÉ DE NICE - SOPHIA ANTIPOLIS FACULTÉ DES SCIENCES BRANCH-AND-BOUND WITH PEER-TO-PEER FOR LARGE-SCALE GRIDS par Alexandre DI COSTANZO Thèse dirigée par Denis CAROMEL au sein de l’équipe OASIS, équipe commune de l’I.N.R.I.A. Sophia Antipolis, du C.N.R.S. et du laboratoire I3S soutenue le 12 octobre 2007 Jury: Président du Jury Franck CAPPELLO Directeur de recherche, LRI-INRIA Futurs Orsay Rapporteurs Raphaël COUTURIER Professeur, IUT de Belfort Montbéliard Dennis GANNON Professeur, Indiana University El-Ghazali TALBI Professeur, LIFL-INRIA Futurs Lille Examinateur Manuel CLERGUE Maître de Conférences, I3S UNSA-CNRS Invité industriel François LABURTHE Directeur R&D, Amadeus Directeur de thèse Denis CAROMEL Professeur, Université de Nice Sophia Antipolis ii iii To Magali, to my family, – Alexandre iv Acknowledgments A PhD thesis is a great experience for working on very stimulating topics, challeng- ing problems, and for me perhaps the most important to meet and collaborate with extraordinary people. This remarkable research environment wasn’t possible without the determination of Isabelle Attali and Denis Caromel who both created the best con- ditions for my PhD. Unfortunately, Isabelle tragically died with her two young sons in the tsunami, Sri Lanka, December 2004. I thank you for the great opportunities you and Denis gave to me. First and foremost, I thank my advisor, Denis Caromel for accepting me in the OASIS project. I am grateful to him for letting me pursue my research interests with sufficient freedom, while being there to guide me all the same. I thank also Françoise Baude for advices she gave me along my thesis. My greetings also go to Raphaël Couturier, Dennis Gannon, and El-Ghazali Talbi who honored me by accepting to review this thesis. I am also very grateful to the other members of the jury, Franck Cappello, Manuel Clergue, and François Laburthe. I would like again to greatly thank Dennis Gannon for reviewing this thesis and for accepting to be a member of the jury, it is an immense honor for me. Of course, I am very thankful to him for his kindness when he invited me at Indiana University to work with his team. During this visit, I met great people and learned a lot. My sojourn gave me an extraordinary motivation to accomplish my thesis. I am still thinking a lot about this experience. I am grateful, too, to the legions of OASIS people who are not only colleague but friends. In a random order, I thank Laurent Baduel, Arnaud & Virginie Contes, Chris- tian G. Delbé, Matthieu Morel, Ludovic Henrio, Fabrice Huet, Cédric Dalmasso, Clé- ment Mathieu, and others. A special thanks to all OASIS’s Chilean people Tomas Barros, Marcela Rivera, Javier Bustos, and specially Mario Leyton and Antonio Cansado who reviewed some parts of this thesis. Also I am grateful to Ian Stokes-Rees for his incredible review of my thesis. I say good luck to the next: Guillaume, Mario, Antonio, Paul (my ping-pong profes- sor), and Marcela. Without forgetting assistants who during years help me to solve all administrative annoyances. Thanks to Claire Senica, Patricia Lachaume, Christiane Gallerie, and Pa- tricia Maleyran. During my PhD my happiness was to meet many people some of the them are now v vi Chapter 0. Acknowledgments more than friends. A special acknowledgment goes to Guillaume Chazarain – alias the Geek – my office- mate. I learned a lot with your research/technical discussion and thanks for having taking care of my desktop computer. And good luck because you are the next PhD. I keep the hope than one day you leave the Linux side for the Mac side. I would like to thank Vincent Cavé who I have a debt. Thank you Vincent, to have taken care of the n-queens while I was staying in the USA. Many thanks to Igor Rosenberg for his unconditional friendship, his support, and to remember me to focus on finishing my PhD. This thesis could not have been written without Magali’s help and support during these last months. For that I am deeply grateful to her. Last but certainly not the least, I owe a great deal to my family for providing me with emotional support during my PhD. My parents have been very supportive; I must acknowledge them for their help and support. Thanks, folks! Contents Acknowledgments v List of Figures xi List of Tables xii I Thesis 1 1 Introduction 3 1.1 Problematics . 3 1.2 Objectives and contributions . 4 1.3 Overview . 5 2 Principles and Positioning 7 2.1 Grid: Principles and Positioning . 7 2.1.1 Principles of Grid computing . 7 2.1.2 Positioning with respect to general Grid architecture . 9 2.2 Peer-to-Peer: Principles and Positioning . 12 2.2.1 Principles of peer-to-peer systems . 12 2.2.2 Comparing peer-to-peer and Grid computing . 17 2.2.3 Overview of existing peer-to-peer systems . 18 2.2.3.1 Global computing platforms . 19 2.2.3.2 Peer-to-Peer platforms . 20 2.2.3.3 Synthesis . 21 2.2.4 Requirements for computational peer-to-peer infrastructure . 22 2.3 Branch-and-Bound: Principles and Positioning . 25 2.3.1 Principles of branch-and-bound algorithm . 25 2.3.1.1 Solving the traveling salesman problem with branch-and- bound . 26 2.3.1.2 Parallel branch-and-bound . 28 2.3.2 Overview of existing branch-and-bound framework for Grids . 31 2.3.2.1 Branch-and-bound . 32 2.3.2.2 Skeletons . 33 2.3.2.3 Divide-and-conquer . 33 2.3.3 Requirements for Grid branch-and-bound . 34 2.4 Context: ProActive Grid middleware . 36 2.4.1 Active objects model . 37 2.4.2 The ProActive library: principles, architecture, and usages . 37 2.4.2.1 Implementation language . 37 vii viii CONTENTS 2.4.2.2 Implementation techniques . 38 2.4.2.3 Communication by messages . 39 2.4.2.4 Features of the library . 40 2.5 Conclusion . 41 3 Desktop Peer-to-Peer 43 3.1 Motivations and objectives . 43 3.2 Design and architecture . 45 3.2.1 First contact: bootstrapping . 45 3.2.2 Discovering acquaintances . 45 3.2.3 Asking resources . 47 3.2.4 Peer and node failures . 48 3.3 Integration within ProActive . 49 3.4 Long running experiment . 51 3.4.1 The n-queens problem . 51 3.4.2 Permanent desktop Grid: INRIA Sophia P2P Desktop Grid . 52 3.4.3 Long running experiment . 53 3.5 Application to numerical hydrodynamic simulation . 56 3.5.1 Motivations and context . 56 3.5.2 River modeling using finite element method . 57 3.5.3 The TELEMAC system parallel version with ProActive . 59 3.5.4 Experiments . 62 3.5.5 Analysis . 62 3.6 Conclusion . 63 4 Branch-and-Bound: A Communicating Framework 65 4.1 Motivations and objectives . 65 4.2 Design and architecture . 67 4.2.1 Entities and their roles . 67 4.2.2 The public library . 69 4.2.3 The task life-cycle . 71 4.2.4 Search tree strategies . 71 4.2.5 Communications . 72 4.2.6 Load-balancing . 73 4.2.7 Exceptions and fault-tolerance . 74 4.3 Implementation . 75 4.4 Grid’BnB user code example: solving flow-shop . 75 4.4.1 The flow-shop problem . 75 4.4.2 Solving flow-shop with Grid’BnB . 75 4.4.2.1 The algorithm . 75 4.4.2.2 The root task . 76 4.5 Experiments with flow-shop . 78 4.5.1 Single cluster experiments . 78 4.5.2 Large-scale experiments . 80 4.6 Conclusion . 83 CONTENTS ix 5 Mixing Clusters and Desktop Grid 87 5.1 Motivations and objectives . 87 5.2 Design and architecture . 89 5.2.1 Sharing clusters with the peer-to-peer infrastructure . 89 5.2.2 Crossing firewalls . 89 5.2.3 Managing dynamic group of workers with Grid’BnB . 89 5.3 Large-scale experiments: mixing desktops and clusters . 90 5.3.1 Environment of experiments . 91 5.3.2 Experiments with n-queens . 91 5.3.3 Experiments with flow-shop . 92 5.4 Peer-to-Peer Grid experiments . 93 5.4.1 Experiment design . 94 5.4.2 Experiments results . 94 5.5 Conclusion . 96 6 Advanced Features and Deployment 103 6.1 Context: ProActive deployment framework . 103 6.1.1 Deployment model . 104 6.1.2 Retrieval of resources . 104 6.1.2.1 Creation-based deployment . 105 6.1.2.2 Acquisition-based deployment .
Recommended publications
  • A Learning-Based Iterative Method for Solving Vehicle
    Published as a conference paper at ICLR 2020 ALEARNING-BASED ITERATIVE METHOD FOR SOLVING VEHICLE ROUTING PROBLEMS Hao Lu ∗ Princeton University, Princeton, NJ 08540 [email protected] Xingwen Zhang ∗ & Shuang Yang Ant Financial Services Group, San Mateo, CA 94402 fxingwen.zhang,[email protected] ABSTRACT This paper is concerned with solving combinatorial optimization problems, in par- ticular, the capacitated vehicle routing problems (CVRP). Classical Operations Research (OR) algorithms such as LKH3 (Helsgaun, 2017) are inefficient and dif- ficult to scale to larger-size problems. Machine learning based approaches have recently shown to be promising, partly because of their efficiency (once trained, they can perform solving within minutes or even seconds). However, there is still a considerable gap between the quality of a machine learned solution and what OR methods can offer (e.g., on CVRP-100, the best result of learned solutions is between 16.10-16.80, significantly worse than LKH3’s 15.65). In this paper, we present “Learn to Improve” (L2I), the first learning based approach for CVRP that is efficient in solving speed and at the same time outperforms OR methods. Start- ing with a random initial solution, L2I learns to iteratively refine the solution with an improvement operator, selected by a reinforcement learning based controller. The improvement operator is selected from a pool of powerful operators that are customized for routing problems. By combining the strengths of the two worlds, our approach achieves the new state-of-the-art results on CVRP, e.g., an average cost of 15.57 on CVRP-100. 1 INTRODUCTION In this paper, we focus on an important class of combinatorial optimization, vehicle routing problems (VRP), which have a wide range of applications in logistics.
    [Show full text]
  • The Generalized Simplex Method for Minimizing a Linear Form Under Linear Inequality Restraints
    Pacific Journal of Mathematics THE GENERALIZED SIMPLEX METHOD FOR MINIMIZING A LINEAR FORM UNDER LINEAR INEQUALITY RESTRAINTS GEORGE BERNARD DANTZIG,ALEXANDER ORDEN AND PHILIP WOLFE Vol. 5, No. 2 October 1955 THE GENERALIZED SIMPLEX METHOD FOR MINIMIZING A LINEAR FORM UNDER LINEAR INEQUALITY RESTRAINTS GEORGE B. DANTZIG, ALEX ORDEN, PHILIP WOLFE 1. Background and summary. The determination of "optimum" solutions of systems of linear inequalities is assuming increasing importance as a tool for mathematical analysis of certain problems in economics, logistics, and the theory of games [l;5] The solution of large systems is becoming more feasible with the advent of high-speed digital computers; however, as in the related problem of inversion of large matrices, there are difficulties which remain to be resolved connected with rank. This paper develops a theory for avoiding as- sumptions regarding rank of underlying matrices which has import in applica- tions where little or nothing is known about the rank of the linear inequality system under consideration. The simplex procedure is a finite iterative method which deals with problems involving linear inequalities in a manner closely analogous to the solution of linear equations or matrix inversion by Gaussian elimination. Like the latter it is useful in proving fundamental theorems on linear algebraic systems. For example, one form of the fundamental duality theorem associated with linear inequalities is easily shown as a direct consequence of solving the main prob- lem. Other forms can be obtained by trivial manipulations (for a fuller discus- sion of these interrelations, see [13]); in particular, the duality theorem [8; 10; 11; 12] leads directly to the Minmax theorem for zero-sum two-person games [id] and to a computational method (pointed out informally by Herman Rubin and demonstrated by Robert Dorfman [la]) which simultaneously yields optimal strategies for both players and also the value of the game.
    [Show full text]
  • An Efficient Three-Term Iterative Method for Estimating Linear
    mathematics Article An Efficient Three-Term Iterative Method for Estimating Linear Approximation Models in Regression Analysis Siti Farhana Husin 1,*, Mustafa Mamat 1, Mohd Asrul Hery Ibrahim 2 and Mohd Rivaie 3 1 Faculty of Informatics and Computing, Universiti Sultan Zainal Abidin, Terengganu 21300, Malaysia; [email protected] 2 Faculty of Entrepreneurship and Business, Universiti Malaysia Kelantan, Kelantan 16100, Malaysia; [email protected] 3 Department of Computer Sciences and Mathematics, Universiti Teknologi Mara, Terengganu 54000, Malaysia; [email protected] * Correspondence: [email protected] Received: 15 April 2020; Accepted: 20 May 2020; Published: 15 June 2020 Abstract: This study employs exact line search iterative algorithms for solving large scale unconstrained optimization problems in which the direction is a three-term modification of iterative method with two different scaled parameters. The objective of this research is to identify the effectiveness of the new directions both theoretically and numerically. Sufficient descent property and global convergence analysis of the suggested methods are established. For numerical experiment purposes, the methods are compared with the previous well-known three-term iterative method and each method is evaluated over the same set of test problems with different initial points. Numerical results show that the performances of the proposed three-term methods are more efficient and superior to the existing method. These methods could also produce an approximate linear regression equation to solve the regression model. The findings of this study can help better understanding of the applicability of numerical algorithms that can be used in estimating the regression model. Keywords: steepest descent method; large-scale unconstrained optimization; regression model 1.
    [Show full text]
  • Revised Simplex Algorithm - 1
    Lecture 9: Linear Programming: Revised Simplex and Interior Point Methods (a nice application of what we have learned so far) Prof. Krishna R. Pattipati Dept. of Electrical and Computer Engineering University of Connecticut Contact: [email protected] (860) 486-2890 ECE 6435 Fall 2008 Adv Numerical Methods in Sci Comp October 22, 2008 Copyright ©2004 by K. Pattipati Lecture Outline What is Linear Programming (LP)? Why do we need to solve Linear-Programming problems? • L1 and L∞ curve fitting (i.e., parameter estimation using 1-and ∞-norms) • Sample LP applications • Transportation Problems, Shortest Path Problems, Optimal Control, Diet Problem Methods for solving LP problems • Revised Simplex method • Ellipsoid method….not practical • Karmarkar’s projective scaling (interior point method) Implementation issues of the Least-Squares subproblem of Karmarkar’s method ….. More in Linear Programming and Network Flows course Comparison of Simplex and projective methods References 1. Dimitris Bertsimas and John N. Tsisiklis, Introduction to Linear Optimization, Athena Scientific, Belmont, MA, 1997. 2. I. Adler, M. G. C. Resende, G. Vega, and N. Karmarkar, “An Implementation of Karmarkar’s Algorithm for Linear Programming,” Mathematical Programming, Vol. 44, 1989, pp. 297-335. 3. I. Adler, N. Karmarkar, M. G. C. Resende, and G. Vega, “Data Structures and Programming Techniques for the Implementation of Karmarkar’s Algorithm,” ORSA Journal on Computing, Vol. 1, No. 2, 1989. 2 Copyright ©2004 by K. Pattipati What is Linear Programming? One of the most celebrated problems since 1951 • Major breakthroughs: • Dantzig: Simplex method (1947-1949) • Khachian: Ellipsoid method (1979) - Polynomial complexity, but not competitive with the Simplex → not practical.
    [Show full text]
  • A Non-Iterative Method for Optimizing Linear Functions in Convex Linearly Constrained Spaces
    A non-iterative method for optimizing linear functions in convex linearly constrained spaces GERARDO FEBRES*,** * Departamento de Procesos y Sistemas, Universidad Simón Bolívar, Venezuela. ** Laboratorio de Evolución, Universidad Simón Bolívar, Venezuela. Received… This document introduces a new strategy to solve linear optimization problems. The strategy is based on the bounding condition each constraint produces on each one of the problem’s dimension. The solution of a linear optimization problem is located at the intersection of the constraints defining the extreme vertex. By identifying the constraints that limit the growth of the objective function value, we formulate linear equations system leading to the optimization problem’s solution. The most complex operation of the algorithm is the inversion of a matrix sized by the number of dimensions of the problem. Therefore, the algorithm’s complexity is comparable to the corresponding to the classical Simplex method and the more recently developed Linear Programming algorithms. However, the algorithm offers the advantage of being non-iterative. Key Words: linear optimization; optimization algorithm; constraint redundancy; extreme points. 1. INTRODUCTION Since the apparition of Dantzig’s Simplex Algorithm in 1947, linear optimization has become a widely used method to model multidimensional decision problems. Appearing even before the establishment of digital computers, the first version of the Simplex algorithm has remained for long time as the most effective way to solve large scale linear problems; an interesting fact perhaps explained by the Dantzig [1] in a report written for Stanford University in 1987, where he indicated that even though most large scale problems are formulated with sparse matrices, the inverse of these matrixes are generally dense [1], thus producing an important burden to record and track these matrices as the algorithm advances toward the problem solution.
    [Show full text]
  • Lecture Notes on Iterative Optimization Algorithms
    Charles L. Byrne Department of Mathematical Sciences University of Massachusetts Lowell December 8, 2014 Lecture Notes on Iterative Optimization Algorithms Contents Preface vii 1 Overview and Examples 1 1.1 Overview . 1 1.2 Auxiliary-Function Methods . 2 1.2.1 Barrier-Function Methods: An Example . 3 1.2.2 Barrier-Function Methods: Another Example . 3 1.2.3 Barrier-Function Methods for the Basic Problem 4 1.2.4 The SUMMA Class . 5 1.2.5 Cross-Entropy Methods . 5 1.2.6 Alternating Minimization . 6 1.2.7 Penalty-Function Methods . 6 1.3 Fixed-Point Methods . 7 1.3.1 Gradient Descent Algorithms . 7 1.3.2 Projected Gradient Descent . 8 1.3.3 Solving Ax = b ................... 9 1.3.4 Projected Landweber Algorithm . 9 1.3.5 The Split Feasibility Problem . 10 1.3.6 Firmly Nonexpansive Operators . 10 1.3.7 Averaged Operators . 11 1.3.8 Useful Properties of Operators on H . 11 1.3.9 Subdifferentials and Subgradients . 12 1.3.10 Monotone Operators . 13 1.3.11 The Baillon{Haddad Theorem . 14 2 Auxiliary-Function Methods and Examples 17 2.1 Auxiliary-Function Methods . 17 2.2 Majorization Minimization . 17 2.3 The Method of Auslander and Teboulle . 18 2.4 The EM Algorithm . 19 iii iv Contents 3 The SUMMA Class 21 3.1 The SUMMA Class of Algorithms . 21 3.2 Proximal Minimization . 21 3.2.1 The PMA . 22 3.2.2 Difficulties with the PMA . 22 3.2.3 All PMA are in the SUMMA Class . 23 3.2.4 Convergence of the PMA .
    [Show full text]
  • Iterative Methods for Solving Optimization Problems
    ITERATIVE METHODS FOR SOLVING OPTIMIZATION PROBLEMS Shoham Sabach ITERATIVE METHODS FOR SOLVING OPTIMIZATION PROBLEMS Research Thesis In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Shoham Sabach Submitted to the Senate of the Technion - Israel Institute of Technology Iyar 5772 Haifa May 2012 The research thesis was written under the supervision of Prof. Simeon Reich in the Department of Mathematics Publications: piq Reich, S. and Sabach, S.: Three strong convergence theorems for iterative methods for solving equi- librium problems in reflexive Banach spaces, Optimization Theory and Related Topics, Contemporary Mathematics, vol. 568, Amer. Math. Soc., Providence, RI, 2012, 225{240. piiq Mart´ın-M´arquez,V., Reich, S. and Sabach, S.: Iterative methods for approximating fixed points of Bregman nonexpansive operators, Discrete and Continuous Dynamical Systems, accepted for publi- cation. Impact Factor: 0.986. piiiq Sabach, S.: Products of finitely many resolvents of maximal monotone mappings in reflexive Banach spaces, SIAM Journal on Optimization 21 (2011), 1289{1308. Impact Factor: 2.091. pivq Kassay, G., Reich, S. and Sabach, S.: Iterative methods for solving systems of variational inequalities in reflexive Banach spaces, SIAM J. Optim. 21 (2011), 1319{1344. Impact Factor: 2.091. pvq Censor, Y., Gibali, A., Reich S. and Sabach, S.: The common variational inequality point problem, Set-Valued and Variational Analysis 20 (2012), 229{247. Impact Factor: 0.333. pviq Borwein, J. M., Reich, S. and Sabach, S.: Characterization of Bregman firmly nonexpansive operators using a new type of monotonicity, J. Nonlinear Convex Anal. 12 (2011), 161{183. Impact Factor: 0.738. pviiq Reich, S.
    [Show full text]
  • Topic 3 Iterative Methods for Ax = B
    Topic 3 Iterative methods for Ax = b 3.1 Introduction Earlier in the course, we saw how to reduce the linear system Ax = b to echelon form using elementary row operations. Solution methods that rely on this strategy (e.g. LU factorization) are robust and efficient, and are fundamental tools for solving the systems of linear equations that arise in practice. These are known as direct methods, since the solution x is obtained following a single pass through the relevant algorithm. We are now going to look at some alternative approaches that fall into the category of iterative methods . These techniques can only be applied to square linear systems ( n equations in n unknowns), but this is of course a common and important case. 45 Iterative methods for Ax = b begin with an approximation to the solution, x0, then seek to provide a series of improved approximations x1, x2, … that converge to the exact solution. For the engineer, this approach is appealing because it can be stopped as soon as the approximations xi have converged to an acceptable precision, which might be something as crude as 10 −3. With a direct method, bailing out early is not an option; the process of elimination and back-substitution has to be carried right through to completion, or else abandoned altogether. By far the main attraction of iterative methods, however, is that for certain problems (particularly those where the matrix A is large and sparse ) they are much faster than direct methods. On the other hand, iterative methods can be unreliable; for some problems they may exhibit very slow convergence, or they may not converge at all.
    [Show full text]
  • On Some Iterative Methods for Solving Systems of Linear Equations
    Computational and Applied Mathematics Journal 2015; 1(2): 21-28 Published online February 20, 2015 (http://www.aascit.org/journal/camj) On Some Iterative Methods for Solving Systems of Linear Equations Fadugba Sunday Emmanuel Department of Mathematical Sciences, Ekiti State University, Ado Ekiti, Nigeria Email address [email protected], [email protected] Citation Fadugba Sunday Emmanuel. On Some Iterative Methods for Solving Systems of Linear Equations. Computational and Applied Mathematics Journal. Vol. 1, No. 2, 2015, pp. 21-28. Keywords Abstract Iterative Method, This paper presents some iterative methods for solving system of linear equations Jacobi Method, namely the Jacobi method and the modified Jacobi method. The Jacobi method is an Modified Jacobi Method algorithm for solving system of linear equations with largest absolute values in each row and column dominated by the diagonal elements. The modified Jacobi method also known as the Gauss Seidel method or the method of successive displacement is useful for the solution of system of linear equations. The comparative results analysis of the Received: January 17, 2015 two methods was considered. We also discussed the rate of convergence of the Jacobi Revised: January 27, 2015 method and the modified Jacobi method. Finally, the results showed that the modified Accepted: January 28, 2015 Jacobi method is more efficient, accurate and converges faster than its counterpart “the Jacobi Method”. 1. Introduction In computational mathematics, an iterative method attempts to solve a system of linear equations by finding successive approximations to the solution starting from an initial guess. This approach is in contrast to direct methods, which attempt to solve the problem by a finite sequence of some operations and in the absence of rounding errors.
    [Show full text]
  • Matrices and Matroids for Systems Analysis (Algorithms And
    Algorithms and Combinatorics Volume 20 Editorial Board R.L. Graham, La Jolla B. Korte, Bonn L. Lovasz,´ Budapest A. Wigderson, Princeton G.M. Ziegler, Berlin Kazuo Murota Matrices and Matroids for Systems Analysis 123 Kazuo Murota Department of Mathematical Informatics Graduate School of Information Science and Technology University of Tokyo Tokyo, 113-8656 Japan [email protected] ISBN 978-3-642-03993-5 e-ISBN 978-3-642-03994-2 DOI 10.1007/978-3-642-03994-2 Springer Heidelberg Dordrecht London New York Library of Congress Control Number: 2009937412 c Springer-Verlag Berlin Heidelberg 2000, first corrected softcover printing 2010 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Cover design: deblik, Berlin Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) Preface Interplay between matrix theory and matroid theory is the main theme of this book, which offers a matroid-theoretic approach to linear algebra and, reciprocally, a linear-algebraic approach to matroid theory.
    [Show full text]
  • Lecture 20 1 Linear Equations
    U.C. Berkeley — CS270: Algorithms Lecture 20 Professor Vazirani and Professor Rao Scribe: Anupam Last revised Lecture 20 1 Linear Equations Gaussian elimination solves the linear system Ax = b in time O(n3). If A is a symmetric positive semidefinite matrix then there are iterative methods for solving linear systems in A. Each iteration in an iterative method amounts to matrix vector multiplications, the number of iterations for convergence to a vector at distance from the true solution for these methods is bounded in terms of the condition number κ = λmax , λmin (i) Steepest descent: O(κ log 1/) √ (ii) Conjugate gradient: O( κ log 1/). Preconditioning tries to improve the performance of iterative methods by multiplying by a matrix B−1 such that the system B−1Ax = B−1b is ‘easy’ to solve. Trivially A is the best preconditioner for the system, but we are looking for a matrix that is ‘easy’ to invert. Intuitively finding a preconditioner is like finding a matrix that is like A but easy to invert. We will review the iterative methods, then use a low stretch spanning tree as a precon- ditioners for solving Laplacian linear systems, and finally see that a preconditioner obtained by adding some edges to the tree preconditioner achieves a nearly linear time solution. 1.1 Steepest Descent The steepest descent method is the analog of finding the minima of a function by following n the derivative in higher dimensions. The solution to Ax = b is the point in R where the 1 T quadratic form f(x) = 2 x Ax − bx + c achieves the minimum value.
    [Show full text]
  • 6. Iterative Methods for Linear Systems
    TU M ¨unchen 6. Iterative Methods for Linear Systems The stepwise approach to the solution... by minimization Miriam Mehl: 6. Iterative Methods for Linear Systems The stepwise approach to the solution... by minimization, January 14, 2013 1 TU M ¨unchen 6.3. Large Sparse Systems of Linear Equations II – Minimization Methods Formulation as a Problem of Minimization • One of the best known solution methods, the method of conjugate gradients (cg), is based on a different principle than relaxation or smoothing. To see that, we will tackle the problem of solving systems of linear equations using a detour. ; • In the following, let A 2 Rn n be a symmetric and positive definite matrix, i.e. A = AT and xT Ax > 0 8x 6= 0. In this case, solving the linear system Ax = b is equivalent to minimizing the quadratic function 1 f (x) := xT Ax − bT x + c 2 for an arbitrary scalar constant c 2 R. • As A is positive definite, the hyperplane given by z := f (x) defines a paraboloid in + Rn 1 with n-dimensional ellipsoids as isosurfaces f (x) = const:, and f has a global minimum in x. The equivalence of the problems is obvious: 1 1 f 0(x) = AT x + Ax − b = Ax − b = −r(x) = 0 , Ax = b : 2 2 Miriam Mehl: 6. Iterative Methods for Linear Systems The stepwise approach to the solution... by minimization, January 14, 2013 2 TU M ¨unchen A Simple Two-Dimensional Example Compute the intersection point of two lines given by 2x = 0 and y = 0. This corresponds to the linear system y 2 0 x 0 y=0 x = : 0 1 y 0 2x=0 This is equivalento to minimizing the quadratic function 1 f (x; y) = −x2+ y 2: 2 graph of f (x) isolines f (x) = c Miriam Mehl: 6.
    [Show full text]