A Matrix Handbook for Statisticians
Total Page:16
File Type:pdf, Size:1020Kb
A MATRIX HANDBOOK FOR STATISTICIANS George A. F. Seber Department of Statistics University of Auckland Auckland, New Zealand BICENTENNIAL BICENTENNIAL WILEY-INTERSCIENCE A John Wiley & Sons, Inc., Publication This Page Intentionally Left Blank A MATRIX HANDBOOK FOR STATISTICIANS THE WlLEY BICENTENNIAL-KNOWLEDGE FOR GENERATIONS Gachgeneration has its unique needs and aspirations. When Charles Wiley first opened his small printing shop in lower Manhattan in 1807, it was a generation of boundless potential searching for an identity. And we were there, helping to define a new American literary tradition. Over half a century later, in the midst of the Second Industrial Revolution, it was a generation focused on building the future. Once again, we were there, supplying the critical scientific, technical, and engineering knowledge that helped frame the world. Throughout the 20th Century, and into the new millennium, nations began to reach out beyond their own borders and a new international community was born. Wiley was there, expanding its operations around the world to enable a global exchange of ideas, opinions, and know-how. For 200 years, Wiley has been an integral part of each generation's journey, enabling the flow of information and understanding necessary to meet their needs and fulfill their aspirations. Today, bold new technologies are changing the way we live and learn. Wiley will be there, providing you the must-have knowledge you need to imagine new worlds, new possibilities, and new opportunities. Generations come and go, but you can always count on Wiley to provide you the knowledge you need, when and where you need it! n WILLIAM J. PESCE PETERBOOTH WILEY PRESIDENT AND CHIEF ExmzunvE OFFICER CHAIRMAN OF THE BOARD A MATRIX HANDBOOK FOR STATISTICIANS George A. F. Seber Department of Statistics University of Auckland Auckland, New Zealand BICENTENNIAL BICENTENNIAL WILEY-INTERSCIENCE A John Wiley & Sons, Inc., Publication Copyright 0 2008 by John Wiley & Sons, Inc. All rights reserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at m.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 11 1 River Street, Hoboken, NJ 07030, (201) 748-601 1, fax (201) 748-6008, or online at http://www.wiley.comgo/permission. Limit of LiabilitylDisclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic format. For information about Wiley products, visit our web site at www.wiley.com. Wiley Bicentennial Logo: Richard J. Pacific0 Library of Congress Cataloging-in-Publieation Data: Seber, G. A. F. (George Arthur Frederick), 1938- A matrix handbook for statisticians I George A.F. Seber. p.; cm. Includes bibliographical references and index. ISBN 978-0-471-74869-4 (cloth ) 1. Matrices. 2. Statistics. I. Title. QA188.S43 2007 5 12.9'4346~22 2007024691 Printed in the United States of America. 10987654321 CONTENTS Preface xvi 1 Notation 1.1 General Definitions 1.2 Some Continuous Univariate Distributions 1.3 Glossary of Notation 2 Vectors, Vector Spaces, and Convexity 7 2.1 Vector Spaces 7 2.1.1 Definitions 7 2.1.2 Quadratic Subspaces 9 2.1.3 Sums and Intersections of Subspaces 10 2.1.4 Span and Basis 11 2.1.5 Isomorphism 12 2.2 Inner Products 13 2.2.1 Definition and Properties 13 2.2.2 Functionals 15 2.2.3 Orthogonality 16 2.2.4 Column and Null Spaces 18 2.3 Projections 20 V vi CONTENTS 2.3.1 General Projections 20 2.3.2 Orthogonal Projections 21 2.4 Metric Spaces 25 2.5 Convex Sets and Functions 27 2.6 Coordinate Geometry 31 2.6.1 Hyperplanes and Lines 31 2.6.2 Quadratics 31 2.6.3 Areas and Volumes 32 3 Rank 35 3.1 Some General Properties 35 3.2 Matrix Products 37 3.3 Matrix Cancellation Rules 39 3.4 Matrix Sums 40 3.5 Matrix Differences 44 3.6 Partitioned and Patterned Matrices 46 3.7 Maximal and Minimal Ranks 49 3.8 Matrix Index 51 4 Matrix Functions: Inverse, Transpose, Trace, Determinant, and Norm 53 4.1 Inverse 53 4.2 Transpose 54 4.3 Trace 54 4.4 Determinants 57 4.4.1 Introduction 57 4.4.2 Adjoint Matrix 59 4.4.3 Compound Matrix 61 4.4.4 Expansion of a Determinant 61 4.5 Permanents 63 4.6 Norms 65 4.6.1 Vector Norms 65 4.6.2 Matrix Norms 67 4.6.3 Unitarily Invariant Norms 73 4.6.4 M,N-Invariant Norms 77 4.6.5 Computational Accuracy 77 5 Complex, Hermitian, and Related Matrices 79 5.1 Complex Matrices 79 5.1.1 Some General Results 80 5.1.2 Determinants 81 5.2 Hermitian Matrices 82 CONTENTS vii 5.3 Skew-Hermitian Matrices 83 5.4 Complex Symmetric Matrices 84 5.5 Real Skew-Symmetric Matrices 85 5.6 Normal Matrices 86 5.7 Quaternions 87 6 Eigenvalues, Eigenvectors, and Singular Values 91 6.1 Introduction and Definitions 91 6.1.1 Characteristic Polynomial 92 6.1.2 Eigenvalues 95 6.1.3 Singular Values 101 6.1.4 Functions of a Matrix 103 6.1.5 Eigenvectors 103 6.1.6 Hermitian Matrices 104 6.1.7 Computational Methods 105 6.1.8 Generalized Eigenvalues 106 6.1.9 Matrix Products 107 6.2 Variational Characteristics for Hermitian Matrices 108 6.3 Separation Theorems 111 6.4 Inequalities for Matrix Sums 116 6.5 Inequalities for Matrix Differences 119 6.6 Inequalities for Matrix Products 119 6.7 Antieigenvalues and Antieigenvectors 122 7 Generalized Inverses 125 7.1 Definitions 125 7.2 Weak Inverses 126 7.2.1 General Properties 126 7.2.2 Products of Matrices 130 7.2.3 Sums and Differences of Matrices 132 7.2.4 Real Symmetric Matrices 132 7.2.5 Decomposition Methods 133 7.3 Other Inverses 134 7.3.1 Reflexive (912) Inverse 134 7.3.2 Minimum Norm (914) Inverse 134 7.3.3 Minimum Norm Reflexive (9124) Inverse 135 7.3.4 Least Squares (913) Inverse 136 7.3.5 Least Squares Reflexive (9123) Inverse 137 7.4 Moore-Penrose (91234) Inverse 137 7.4.1 General Properties 137 7.4.2 Sums of Matrices 143 viii CONTENTS 7.4.3 Products of Matrices 143 7.5 Group Inverse 145 7.6 Some General Properties of Inverses 145 8 Some Special Matrices 147 8.1 Orthogonal and Unitary Matrices 147 8.2 Permutation Matrices 151 8.3 Circulant, Toeplitz, and Related Matrices 152 8.3.1 Regular Circulant 152 8.3.2 Symmetric Regular Circulant 155 8.3.3 Symmetric Circulant 156 8.3.4 Toeplitz Matrix 158 8.3.5 Persymmetric Matrix 159 8.3.6 Cross-Symmetric (Centrosymmetric) Matrix 160 8.3.7 Block Circulant 160 8.3.8 Hankel Matrix 161 8.4 Diagonally Dominant Matrices 162 8.5 Hadamard Matrices 164 8.6 Idempotent Matrices 166 8.6.1 General Properties 166 8.6.2 Sums of Idempotent Matrices and Extensions 170 8.6.3 Products of Idempotent Matrices 175 8.7 Tripotent Matrices 175 8.8 Irreducible Matrices 177 8.9 Triangular Matrices 178 8.10 Hessenberg Matrices 179 8.11 Tridiagonal Matrices 180 8.12 Vandermonde and Fourier Matrices 183 8.12.1 Vandermonde Matrix 183 8.12.2 Fourier Matrix 184 8.13 Zero-One (0,l) Matrices 186 8.14 Some Miscellaneous Matrices and Arrays 187 8.14.1 Krylov Matrix 187 8.14.2 Nilpotent and Unipotent Matrices 188 8.14.3 Payoff Matrix 188 8.14.4 Stable and Positive Stable Matrices 189 8.14.5 P-Matrix 191 8.14.6 Z- and M-Matrices 191 8.14.7 Three-Dimensional Arrays 194 CONTENTS ix 9 Non-Negative Vectors and Matrices 195 9.1 Introduction 195 9.1.1 Scaling 196 9.1.2 Modulus of a Matrix 197 9.2 Spectral Radius 197 9.2.1 General Properties 197 9.2.2 Dominant Eigenvalue 199 9.3 Canonical Form of a Non-negative Matrix 20 1 9.4 Irreducible Matrices 202 9.4.1 Irreducible Non-negative Matrix 202 9.4.2 Periodicity 207 9.4.3 Non-negative and Nonpositive Off-Diagonal Elements 208 9.4.4 Perron Matrix 209 9.4.5 Decomposable Matrix 210 9.5 Leslie Matrix 210 9.6 Stochastic Matrices 212 9.6.1 Basic Properties 212 9.6.2 Finite Homogeneous Markov Chain 213 9.6.3 Countably Infinite Stochastic Matrix 215 9.6.4 Infinite Irreducible Stochastic Matrix 215 9.7 Doubly Stochastic Matrices 216 10 Positive Definite and Non-negative Definite Matrices 219 10.1 Introduction 219 10.2 Non-negative Definite Matrices 220 10.2.1 Some General Properties 220 10.2.2 Gram Matrix 223 10.2.3 Doubly Non-negative Matrix 223