Calculus with Applications and C++ Programs

Willi-Hans Steeb

World Scientific Matrix Calculus and Kronecker Product with Applications and C++ Programs This page is intentionally left blank Matrix Calculus and Kronecker Product with Applications and C++ Programs

Willi-Hans Steeb International School of Scientific Computing, Rand Afrikaans University

in collaboration with Tan Kiat Shi National University of Singapore

WorM Scientific Singapore • New Jersey 'London 'Hong Kong Published by World Scientific Publishing Co. Pte. Ltd. PO Box 128, Farrer Road, Singapore 912805 USA office: Suite IB, 1060 Main Street, River Edge, NJ 07661 UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE

Library of Congress Cataloging-in-Publication Data Steeb, W.-H. Matrix calculus and Kronecker product with applications and C++ programs / Willi-Hans Steeb, in collaboration with Tan Kiat Shi. p. cm. Includes bibliographical references and indexes. ISBN 9810232411 1. Matrices. 2. Kronecker products. 3. Matrices--Data processing. 4. Kronecker products ■- Data processing. 5. C++ (Computer program language) I Shi, Tart Kiat. II Title. QA188.S663 1997 530.15?9434-dc21 97-26420 CIP

British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library.

Copyright

For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher.

This book is printed on acid-free paper

Printed in Singapore by Uto-Print Preface The Kronecker product of matrices plays an important role in mathematics and in ap­ plications found in theoretical physics. Such applications are signal processing where the Fourier and Hadamard matrices play the central role. In group theory and ma­ trix the Kronecker product also comes into play. In statistical mechanics we apply the Kronecker product in the calculation of the partition function and free energy of spin and Fermi systems. Furthermore the spectral theorem for finite dimensional hermitian matrices can be formulated using the Kronecker product. The so-called quantum groups rely heavily on the Kronecker product. Most books on linear algebra "and matrix theory investigate the Kronecker product only superficially. This book gives a comprehensive introduction to the Kronecker product of matrices together with its software implementation in C++ using an object-oriented design.

In chapter 1 we give a comprehensive introduction into matrix algebra. The basic def­ initions and notations are given in section 1.1. The trace and determinant of square matrices are introduced and their properties are discussed in section 1.2. The eigen­ value problem plays a central role in physics. Section 1.3 is devoted to this problem. Projection matrices and projection operators are important in Hilbert space theory and quantum mechanics. They are also used in group theoretical reduction in finite group theory. Section 1.4 discusses these matrices. In signal processing Fourier and Hadamard matrices play a central role for the fast Fourier transform and fast Hadamard transform, respectively. Section 1.5 is devoted to these matrices. Transformations of matrices are described in section 1.6. The invariance of the trance and determinant are also dis­ cussed. Finite groups can be represented as permutation matrices. These matrices are investigated in section 1.7. The vec operator describes an important connection between matrices and vectors. This operator is also important in connection with the Kronecker product. Section 1.8 introduces this operator. The different vector and matrix norms are defined in section 1.9. The relationships between the different norms are explained. We also describe the connection with the eigenvalues of the matrices. The exponential function of a square matrix is useful in many applications, for example Lie groups, Lie transformation groups and for the solution of systems of ordinary differential equations. Sequences of vectors and matrices are introduced in section 1.10 and in particular the exponential function is discussed. Groups are studied in section 1.11. A number of their properties are given. Section 1.12 introduces Lie algebras. Applications in quantum theory are given in section 1.13. We assume that the Hamilton operator is given by a Hermitian matrix. We investigate the time evolution of the wave function (Schrodinger equation) and the time evolution of a matrix (Heisenberg equation of motion). VI PREFACE

Sections 2.1 to 2.3 in chapter 2 give an introduction to the Kronecker product. In partic­ ular, the connection with is discussed. In Section 2.4 permutation matrices are discussed. Section 2.5 is devoted to the trace and determinant of a matrix and their relation to the Kronecker product. The eigenvalue problem is studied in section 2.6. We calculate the eigenvalues and eigenvectors of Kronecker products of matrices. In order to investigate the spectral representation of Hermitian matrices we introduce projection matrices in section 2.7. Section 2.8 describes the spectral representation of Hermitian matrices using the Kronecker product. Fourier and Hadamard matrices are important in spectral analysis, such as fast Fourier transforms. These matrices are intro­ duced in section 2.9 and their connection with the Kronecker product is described.The direct sum and the Kronecker product are studied in section 2.10. Section 2.11 is de­ voted to the vec-operator and its connection with the Kronecker product. Groups and the Kronecker product are investigated in Sections 2.12 and 2.13. In particular the matrix representation of groups is described. The inversion of partitioned matrices is discussed in section 2.14.

In chapter 3 we study applications in statistical mechanics, quantum mechanics, Lax representation and signal processing for the Kronecker product. First we introduce Pauli spin matrices and give some applications in section 3.1. The eigenvalue problem of the two point Heisenberg model is solved in detail. The one-dimensional Ising model is solved in section 3.2. Fermi systems are studied in section 3.3. We then study the dimer problem in section 3.4. The two-dimensional Ising model is solved in section 3.5. In section 3.6 the one-dimensional Heisenberg model is discussed applying the famous Yang-Baxter relation. Quantum groups are discussed in section 3.7. Section 3.8 describes the connection of the Kronecker product with the Lax representation for ordinary differential equations. Signal processing and the Kronecker product is discussed in section 3.9.

The tensor product can be considered as an extension of the Kronecker product to infinite dimensions. Chapter 4 gives an introduction into the tensor product and some applications. The Hilbert space is introduced in section 4.1 and the tensor product in section 4.2. Sections 4.3 and 4.4 give two applications. In the fist one we consider a spin- orbit system and the second one a Bose-spin system. For the interpretation of quantum mechanics (system and the measuring apparatus) the tensor product and Kronecker product is of central importance. We describe this connection in section 4.5.

In chapter 5 the software implementations are given. An object-oriented design is used. A vector and matrix class (abstract data types) are introduced together with all the operations, i.e. trace, determinat, matrix multiplication, Kronecker product etc.. The classes are written on a template basis so that every appropiate data type can be used with these classes, for example, integer, double, verylong, rational, symbolic. A number of applications are given. PREFACE vii

In most sections a large number of examples and problems serve to illustrate the mathe­ matical tools. The end of a proof is indicated by 4. The end of an example is indicated by*.

If you have any comments about the book please send them to us at: email: [email protected]

Web page: http://zeus.rau.ac.za/steeb/steeb.html

Acknowledgements

I am grateful to John and Catharine Thompson for careful proof-reading of the manuscript and checking a number of results. I also thank Dr. Fritz Solms for discussions on object-oriented programming and C++. Finally, I appreciate the help of the Lady from Shanghai Zhao Gui Zhu. This page is intentionally left blank Contents

1 Matrix Calculus 1 1.1 Definitions and Notation 1 1.2 Trace and Determinant 7 1.3 Eigenvalue Problem . 13 1.4 Projection Matrices 17 1.5 Fourier and Hadamard Matrices 20 1.6 Transformation of Matrices . 26 1.7 Permutation Matrices . 29 1.8 Vec Operator 32 1.9 Vector and Matrix Norms 34 1.10 Sequences of Vectors and Matrices . 39 1.11 Groups 42 1.12 Lie Algebras 50 1.13 Application in Quantum Theory 53

2 Kronecker Product 55 2.1 Definitions and Notations . 55 2.2 Basic Properties 61 2.3 Matrix Multiplication . . 65 2.4 Permutation Matrices . 71 2.5 Trace and Determinant . . . 74 2.6 Eigenvalue Problem 78 2.7 Projection Matrices 82 2.8 Spectral Representation 84 2.9 Fourier and Hadamard Matrices 89 2.10 Direct Sum 93 2.11 Vec Operator . 96 2.12 Groups . . 99 2.13 Group Representation Theory . 102 2.14 Inversion of Partitioned Matrices 106 X CONTENTS

3 Applications 109 3.1 Spin Matrices . . 109 3.2 One-Dimensional Ising Model 117 3.3 Fermi Systems . 122 3.4 Dimer Problem 133 3.5 Two-Dimensional Ising Model 141 3.6 One-Dimensional Isotropic Heisenberg Model . 160 3.7 Quantum Groups 170 3.8 Lax Representation 176 3.9 Signal Processing 179

4 Tensor Product 187 4.1 Hilbert Spaces 187 4.2 Hilbert Tensor Products of Hilbert Spaces 195 4.3 Spin and Statistics for the n-Body Problem . 199 4.4 Exciton-Phonon Systems 202 4.5 Interpretation of Quantum Mechanics 204

5 C++ Software Implementation 209 5.1 Abstract Data Type and C++ 209 5.2 The Vector Class 211 5.2.1 Abstraction 211 5.2.2 Templates 212 5.2.3 Data Fields 212 5.2.4 Constructors 212 5.2.5 Operators . 213 5.2.6 Member Functions and Norms 216 5.2.7 Streams 217 5.3 The Matrix Class 218 5.3.1 Abstraction 218 5.3.2 Data Fields 219 5.3.3 Constructors 219 5.3.4 Operators . . 220 5.3.5 Member Functions and Norms 221 5.4 Header File Vector Class 226 5.5 Header File Matrix Class 236

Bibliography 249

Index 252 Symbol Index

z the set of integers N the set of positive integers: natural numbers Q the set of rational numbers E) the set of real numbers R+ non negative real numbers C the set. of complex numbers Rn the n dimensional real linear space C the n dimensional complex linear space J :=v^T u,v column vectors in Cn 0 zero vector in C"

<■) Kronecker product (direct product, tensor product) m direct sum II.II norm (.) scalar product (inner product) in Cn tr trace det determinant I unit matrix (identity matrix) A'' of the matrix A •y permutation P permutation matrix U unitary matrix II projection matrix

[■ ] commutator [,h anti-commutator fij* Kronecker delta 1 if j = ft and 0 if j ^ k A eigenvalue G group a spin ax,

xt Chapter 1 Matrix Calculus

1.1 Definitions and Notation

We assume that the reader is familiar with some basic terms in linear algebra such as vector spaces, linearly dependent vectors, matrix addition and matrix multiplication.

Throughout we consider matrices over the field of complex numbers C or real number R. Let z 6 C with z = x + iy and x, y 6 R. Then z = x — iy. In some cases we restrict the underlying field to the real numbers R. The matrices are denoted by A, B, C, D, X, Y. The matrix elements (entries) of the matrix A are denoted by a}k- For the column vectors we write u, v, w. The zero column vector is denoted by 0. Let A be a matrix. Then AT denotes the transpose and A is the complex conjugate matrix. We call A* the adjoint matrix, where A* := AT A special role is played by the n x n matrices, i.e. the square matrices. In this case we also say the matrix is of order n. /„ denotes the n x n unit matrix (also called identity matrix). The zero matrix is denoted by 0.

Let V be a of finite dimension n, over the field R of real numbers, or the field C of complex numbers. If there is no need to distinguish between the two, we will speak of the field K of scalars. A basis of V is a set { ej, e2,..., en } of n linearly independent vectors of V, denoted by (e,-)"=1. Every vector v e V then has the unique reDresentation

the scalars vt, which we will sometimes denote by (v)i, being the components of the vector v relative to the basis (e);. As long as a basis is fixed unambiguously, it is thus always possible to identify V with Kn In matrix notation, the vector v will always be represented by the column vector 2 CHAPTER 1. MATRIX CALCULUS while vT and v* will denote the following row vectors:

where a is the complex conjugate of a. The row vector vr is the transpose of the column vector v, and the row vector v* is the conjugate transpose of the column vector v.

Definition. Let Cn be the familiar n-dimensional vector space. Let u,v 6 C". Then the scalar (or inner) product is defined as

Obviously and

In matrix notation the scalar product can be written as

Definition. Two vectors u, v £ C" are called orthogonal if

Example. Let

Then (u,v) = 0.

The scalar product induces a norm of u defined by

In section 1.9 a detailed discussion of norms is given.

Definition. A vector u 6 C" is called normalized if (u, u) = 1.

Example. The vectors 1.1. DEFINITIONS AND NOTATION 3 are normalized. &

Let V and W be two vector spaces over the same field, equipped with bases (e,,)"=1 and (ft)™ n respectively. Relative to these bases, a linear transformation

A:V -* W is represented by the matrix having m rows and n columns

The elements a^ of the matrix A are defined uniquely by the relations

Equivalently, the jth column vector

of the matrix A represents the vector Ae, relative to the basis (f;)2=i- We call

(an ai2 ■ ■ ■ ain) the ith row vector of the matrix A. A matrix with m rows and n columns is called a matrix of type (m, n), and the vector space over the field K consisting of matrices of type (m, n) with elements in K is denoted by Amin. A column vector is then a matrix of type (m, 1) and a row vector a matrix of type (1, n). A matrix is called real or complex according whether its elements are in the field R or the field C. A matrix A with elements ay is written as A = (ai:l) the first index i always designating the row and the second, j, the column.

The null matrix (also called the zero matrix) is represented by the symbol 0.

Definition. Given a matrix A e An,n(C), the matrix A" e Ai,m(C) denotes the adjoint of the matrix A and is defined uniquely by the relations

which imply that {A*)ij = a}l.