z b r¡'e')- NUMERICAL METHODS FOR TOEPLITZ MATRTCES by Douglas Robin Sweet, B. Sc. (Hons) , Dip. Cornp. Sc. \ A thesis subniitt.ed for the clegree of Doctor of Philosopþ in the DeparEnent of Colrrputing Scienc.e, Universj-ty of Adelaide Mav 1982 I TABLE OF CONTENTS VOLIIME I Page TABLE OF CONTENTS i SIJMMARY v SIGNED STATEMEM vii AC KNOI,I¡LE D G ET,f ENTS viii NOTATION ix GLOSSARY OF ALGORITHMS xvii I INTRODUCTION 1. Toeplitz natrices I 2. Applications of Toeplitz and related natrices 2 3. Airns of 'the thesis 5 4^ Previous work on Toeplitz matrices 7 5. Main results 11 6. Organization of the thesis 13 2 FAST TOEPI,ITZ FACTORIZATION ALGORITHMS - ALTERNATIVE DERIVATIONS 1. Introduction 15 2. The Bareiss algorithm L7 3. Computing the factors of ? using the Bareiss algorithrn (BNA) 23 4. Derivation of FTF algorithms using rank-1 updates 28 5. Conclusion 37 3 THE RELATION BETWEEN TOEPLITZ ELIMINATION AND INVERSION ALGORTTI+IS 1. Introduction 38 2. The Trench-Zohar algorithn 39 3. The extended Bareiss syrûnetric algorithm 42 4. The al.ternative Bareiss symmetric algorithnr (ABSA) - Toeplitz inversiorr 47 5. Conclusion 52 11 Page 4 ERROR ANALYSIS OF BAREISSIS ALGQRITIßI 1. IntroductÌon 53 2, Bareiss aLgorithm ' niscellaneous resuLts 54 3. Bareiss algorithrn - increase in error bound 66 4. Bareiss al-gorithm - backward error analysis 7T 5. Conclusion 79 5 THE PIVOTED BAREISS ALGORTTHM 1. Introduction 80 2. Pivoting procedirres for the Bareiss algorithm 81 3. Pivoting strategies 114 4. fncrease in error bound - simple pivoting strategy 134 5. Introduction of Pivoting into the Bareiss algorithm (BSA) 158 6. Conclusion 159 6 NUT{ERICAL ASPECTS OF TOEPLITZ FACTOF{IZATION AND INVERSION 1. Introduction r40 2. Toeplitz factotization - increase in error bound r4l 3. Toeplitz factotization - backwa::cl error analysis r42 4. Relation bettveen the pivoted Bareiss algorithm and Toeplitz factorization 155 5. Pivoted Toeplitz factottzation - ayoidance of increase in error bound 159 6. Toeplitz inversion - increase in error bound 160 7. Pivoted extended Bareiss symmetric algorithnt 161 B. The pivoted alternative Bareiss synrnetric algorithm and pivoted Trench-Zohar algorithnt 162 9. Avoidance of error groL{th in PABSA or PTZA 169 10. Pivot selectj.on strategY t73 11. Conclusion 174 111 Page 7 FAST TOEPLITZ ORTHOGONLIZATION l. Introduction r75 2, The Gill.Go1ub'Murray-Saunder algorithn (GGMS) 178 3, Fast Toeplitz orthogonalization, version 1 (FTOI) 184 4. CaleuLation of Q and.R explicitly (FT02) 198 5. Conclusion 206 8 FAST 1'OEPLTTZ ORTHOGONALIZATION II 1. Introduction 207 2. Fast Toeplitz orthogonalization, version 3 (FT03) 208 3. Calculation of both 4 and .R explicitly : FT04 219 4. The use of fast Givens transforms in Toeplitz orthogonalization 225 5. Toeplitz orthogonalization - somê extensions 240 6. Conclusion 247 9 THE SINGULAR VALUE DECOMPOSITION OF TOEPLITZ IVIATRICES 1. Introduction ?.48 2. The Golub-Reinsch algorithm 249 3. Teoplitz SVD by Fast Toeptritz orthogonalization 253 4. Toeptitz SVD by Gram-Schmidt bidiagonalization 257 5. Conclusion 265 10. CONCLUSION 266 REFERENCES 268 1V .1 Ì l'¡ VOLUME II Èìt APPENDIX ^ PROGRAIqS A.ND RESULTS Page No. I t t str4tegy I A.1 PBAI - Pivoted Bareiss a]-gorithn, r 't PBA2 - Plvoted Bareiss algorithn, stTategy 2 A.B '' algorithn, strategy 3 A.2I ù PBAS - Pivoted Bareiss 1 t PBAFAC - Pivoted Toeplitz factorization A.31 I '¡I PTZA1 - Pivoted Trench-Zohar algorithm, strategy L A. 41 FT01 - Fast Toeplitz orthogonalization, version 1 4.48 FT02 - Fast Toeplitz orthogonalization, version 2 A.54 FT02S - Fast Toeplitz orthogonaLization, version 25 A.61 FT03 - Fast Toeplitz orthogonalization, version 5 A.72 FT04 - Fast Toeplitz orthogonalization, version 4 4.78 FT05 - Fast Toeplitz orthogonalization, version 5 A. 85 Miscellaneous lr'fatrix Routines ^.92 ,i I I v I Li¡ t- stIvu"lARY This thesis is rnainly concerned with net.hods for solving ToepLitz l"inear algebra problerns ¡n 0fuZ ) rnultiplications/divisions. There are three main airns (i) to find new connexions between known aLgorithns, and re-derive sone of these using clifferent approaches (ii) to derive new results concerning the numerical stability of sone algorithns, and to nodify these algorithns to inprove their numerical performance (iii) to derive fast Toeplitz algoritfuns for new applica- tions, such as the orthogonal deconposition and singular value decompo- siti on. In Chapter 2, fast Toeplitz faetorization algorithrns (FTFts) are :re-derived frorn the Bareiss algorithm for solving Toeplitz systenìs and also fron algr:rithms for performi-ng rank-1 updates of factors. In Chapter 3, the Bareiss algorithm is related to the Trench Algorithrn for Toeplitz invelsion. Several new results regarding the propagation of rounding errors in the Bareiss algorithn are derived in Chapter 4. A pivoting scheme is proposed in Chapter 5 to improve the numerical performance of the Bareiss algorithm. In Chapter 6, erïor analyses are performed on FTFrs and some pivoting is incorporated to improve their nunerical perfornance. The results of Chapter 3 are used to adapt the Bareis-s pivoting procedure to the Trench algorithm. v1 Methsds are proposed in Chapter 7 to conpute the QR factoriza- tion of a Toepritz natrix in 0fu2) oPerations' Thi5 ¿lgerithn uses the shift-invariance property of the Toeplitz natrix and a known procedure for updating the QR factors of a general natrix.. In Chapter B, methods are described for speeding up the aLgorithns of the previous chapter, and several extensions are proposed, . In Chapter 9, two algorithns are proposed to conpute the singular-va1ue decomposition (SVD) of a ToepLitz natrix in fewer operations than for a general matrix. The first algorithm ¡s O(ns ) but depending on the dimension of the problem requir:es up to 80% fewer operations than for general SVD algorithns. The second possibly unstable nethod has complexity .) 0(n' Log d. A modification of this method is proposed which nay enable the singular values to be calculated stably. I l va]- DECLARATION This thesis contains no rnaterial which has been accepted for the award of any other degree or diplona in any university and, to the best of my knowledge and belief, contains no naterial previously published or written by another person, except where due reference is nade in the text of the thesis. D.R. SWEET viii ACKNOI{LEDGEMENTS I wish to express ny sincere gratitude to Dr. Sylvan Elhay, for his encouraging support, his help in planning the thesis, and the rnany hours he has spent in paÌnstakingl-y checking the rigour of the nathenratics, and the many suggestions he has nade to improve the presenta- tion. To Dr. Jaroslav l(autsky, I give thanks for several stimulating discussions and also for his assj.stance while my supervisor was on study leave. Thanks, are due to nt¡' enployer, the Department of Defence, for giving me leave to pursue my studíes, To my colieagues at the Defence Research Centre, Salisbury, and especially to Graham Mountford, I am obliged for their encouragernent. The financial support provicled by the Australian Government under the Commonwealth Postgraduate Awards Schene is gratefully acknowiedged. To Tracey Young, special thanks are due for her patience and skili over the last ferv lnonths in typing the manuscript. I also recognize the assistance of Fiona Adlington in dr:awing the diagrans, and of Eljzabeth Hender:son and Dawn Danvent for their help in the typing. For her patience, encouragement and urging over the past fevr years I pay tribute to my wife Vanessa. 1X NOTATION General. Matrix and Vector Notation FY ¡or Exp lanation a, A vector with elements Qy oZ, . r. a^, whete m is the order of a A The natrix with elements a¿¡t 'ç-Lf ,. .,m;i=Lrn, where m and n are the lrunber of rows and colunlns respectively in ,4 Row i and column i of A Elements i to k of gi, i elenents h to i of ?.j A. Rows i to i of columns k to L of A 1,: J , K: L j columns to L of A Ai,i ';t 'u'' Rows i to of A; k A" The kth leading submatrix of ,4 (unless otherwise K indicated oT rAT The transpose of a, A A--1 The inverse of .4 R g The reverse of a., i.e if a: (arr...ro^)T, P-\T then a":(dmr.. roL) RT a, The reverse-transPose of ø tl1 9 A-' The -secondary transpose of ,4 (transpose about the secondary diagonal) .AT2=EATE (see below for E) -m dLq,gla,ij A diagonal matrix with diagonal eLements a?aZ'...am 1 det A The detenninant of ¡1 ll/ll; llall A natrix-norm of ¡l; a corresponding subordinate vector-ncrm of ø (if aPPl icable) ll¿ll t llall The p-norn of .4 and g P p x 9)'EÞo¡ Expl.anation A k;i The displaced leading subrnatrix of Á with order k and displacenent 7'; fot i>0, Ak,j=OO;krt+¡:k+j; j<o' f'ot Ak;¡=At+li l:k+l¡l,t:t< rrEastr?; rrSoutht') oTr, otu Sane as Ak;l ; Ak;-l, (E: ^9: ,5 4,¡tok;ì (At,k,1;k^i-' gl,k,i) ; Ast;ì.:l'",,:,t:'r) ?e llg¿.l lla..ll oö' oi , 1 A* The Toeplitz part of A dma,* Tfflrn-t url a. j ,- O : the ¿th diagonal above the nain diagonal J of the Toeplitz paït of A; i < 0 ¿ the liltn diagonal below the main diagonal of the Toeplitz part of .4. a@b The cyclic convolution of a and b m A A^ A^-1 . ,. A', where {AU} are square natrices of JT 1, æJ the same order (not leading submatrices of A) Ã,4 The computed value of A,a -A-A, 6Á, ôa The errors in .ãrã ; a-ã respectively aa/ a reL'A, reL â, I o" | | | ; I I ii o ¡ 4i " ¿¡ eond A The conclition nunber of .4, defined as ll¿ll lln-lll T A Toeplitz natrix { A block-vector with blockt O(i) blockt A A A block-rnatrix with (¿j ) Block-row í and bLock-col.unn of A 4(¿.