On the Covariance Completion Problem Under a Circulant Structure
Total Page:16
File Type:pdf, Size:1020Kb
1 On the Covariance Completion Problem under a banded inverse with zero values corresponding to the location Circulant Structure of unspecified elements, cf. [4]. The purpose of the present note is to develop a simple in- Francesca P. Carli and Tryphon T. Georgiou dependent argument that explains this result and, at the same time, shows that the algebraic constraint for the completion to be circulant is automatically satisfied in all cases, i.e., for Abstract| Covariance matrices with a circulant structure arise any number of missing bands as well as for any number of ar- in the context of discrete-time periodic processes and their sig- bitrary missing elements in a block-circulant structure. More nificance stems also partly from the fact that they can be diag- onalized via a Fourier transformation. This note deals with the specifically, the proof of the key result relies on the observation problem of completion of partially specified circulant covariance that circulant and block-circulant matrices are stable points of matrices. The particular completion that has maximal deter- a certain group. The group action preserves the value of the de- minant (i.e., the so-called maximum entropy completion) was terminant. Hence, the maximizer of the determinant, which is considered in Carli etal. [2] where it was shown that if a single unique and has the Dempster property [4], will generate an or- band is unspecified and to be completed, the algebraic restriction bit under the group-action that preserves the specified elements that enforces the circulant structure is automatically satisfied and that the inverse of the maximizer has a band of zero values in their original locations (since these are compatible with the that corresponds to the unspecified band in the data|i.e., it has circulant structure). The values at the unspecified locations will the Dempster property. The purpose of the present note is to be varying and thus generating more than one maximizer unless develop an independent proof of this result which in fact extends they are also compatible with the circulant structure. Since the naturally to any number of missing bands as well as arbitrary maximizer is unique, it follows that the maximizer is circulant. missing elements. More specifically, we show that this general fact is a direct consequence of the invariance of the determinant We note that the importance of invariance in establishing an under the group of transformations that leave circulant matri- alternative proof was already anticipated in a remark in [2]. ces invariant. A description of the complete set of all positive The present note goes on to highlight certain connections be- extensions of partially specified circulant matrices is also given tween the structure of all positive extensions for partially speci- and certain connections between such sets and the factorization fied circulant matrices and factorizations of certain polynomials of certain polynomials in many variables, facilitated by the cir- in many variables. More specifically, since circulant m × m- culant structure, is highlighted. block matrices can be diagonalized by a Fourier transforma- tion, they can be thought of as matrix-valued functions on the I. Introduction cyclic group Z=(nZ). Therefore, positivity of a partially speci- The present work has been motivated by a recent study by fied such matrix gives rise to n, m-order curves, that delineate Carli, Ferrante, Pavon and Picci [2] on discrete-time periodic the admissible completion set. These curves represent factors processes. In particular, they studied reciprocal random pro- of the determinant viewed as a polynomial in the unspecified cesses on the cyclic group with n elements [11], [12]. Applica- coefficients|in case m = 1 they are lines and the solution set tions in the digital age abound, not least because of the com- a polytope. Thus, a Fourier transformation allows factoriza- putational simplicity in working with such processes and their tion of polynomials which, without the knowledge that they can representations ([10]). be written as the determinant of a circulant matrix with vari- The subject in [2] was the completion of partially specified co- able entries, would be challenging or impossible using rational variances for such a process x` on the discrete finite \time-set" techniques and elimination theory (e.g., Galois theory, Gr¨obner f0; 1; : : : ; n − 1g. The assumption of (second-order) stationar- bases, symbolic manipulations). ity R := Efx x∗g = Efx x∗ g for all k; ` 2 =(n ), The outline of the material is as follows. In Section II we in- ` 0 ` k (k+`)mod n Z Z together with the periodicity R` = R` mod n dictates that the troduce certain facts about circulant matrices. In Section III we covariance matrix discuss the Dempster property (Theorem 4), namely the prop- erty of the inverse of the maximum entropy completion to have 2 ∗ ∗3 R0 R1 R2 :::R2 R1 zero entries in places where the covariance matrix is unspecified. ∗ ∗ ∗ 6R1 R0 R1 R3 R27 In the same section we present our main result (Theorem 5) for 6 ∗ ∗ ∗ ∗7 6R2 R1 R0 R4 R37 maximum-determinant completions of matrices with a circulant R := 6 7 ; (1) 6 . .. 7 structure. Finally Section IV contains two examples which pro- 6 . 7 6 7 vide insight into the structure of the completion-set, and high- 4R2 R3 R4 R0 R15 R R R :::R∗ R light a connection with the factorization of certain polynomials 1 2 3 1 0 in many variables. has the block-circulant structure [3], [8]. In practice, lack of II. Technical preliminaries & notation data may result in missing values in estimating statistics, i.e., m×m some of the R 's above. We work in the space C of (m × m) complex-valued ma- ` m×m To this end, for the completion of general statistics it is com- trices. As usual, for any a 2 C , the complex-conjugate- ∗ mon practice (e.g., see [4], [9], [5]) to select values that maxi- transpose (adjoint) is denoted by a . At times the size of ma- mize the determinant of a covariance matrix. The reason can trices is (nm×nm), in which case these are typically of the form n×n m×m be traced to the fact that, for Gaussian statistics, this gives b ⊗ a with b 2 C , a 2 C and ⊗ denoting the Kronecker the maximum likelihood distribution. Specializing to the case product, i.e., these are \block-matrices." of circulant covariances, Carli etal. [2] showed that when there We define the circulant (up) (n × n)-shift is a single band of unspecified values, the constraint that en- 20 1 ::: 03 forces the circulant structure when maximizing the determinant 6. 7 is automatically satisfied, and thereby, the maximizer shares the 6. .. 7 S := 6 7 ; property of maximizers for more general problems in having a 40 0 15 1 0 ::: 0 Supported by the National Science Foundation and by the Air Force Office of Scientific Research. and the notation In for the (n × n)-identity matrix. Clearly 2 n k S = In, and as is well-known (and easy to check), S has the In case the inverse exists, then it is (eigenvalue-eigenvector) decomposition −1 ` 0 −1 1 −1 2 −1 a(S) = (U ⊗ Im) diagfa(w ) ; a(w ) ; a(w ) ; SkU = UW k; n−1 −1 ´ ∗ : : : ; a(w ) g (U ⊗ Im): pq where U is the Fourier-matrix with elements Up;q = w for p −j 2π j := −1, w = e n and Define the notation W = diagf1; w; w2; : : : ; wn−1g: Ek = diagf0;:::; 0; 1; 0;:::; 0g Note that diag{·} denotes a diagonal (or, possibly, block- diagonal) matrix with its entries listed within the brackets. We with a 1 at the kth diagonal entry. Since (S ⊗ Im)(U ⊗ Im) = −1 ∗ now consider the space of (n × n)-circulant matrices (U ⊗ Im)(W ⊗ Im) while W = W , we have that ( n−1 ) ` −1´ ∗ X k (S ⊗ Im) a(S) (S ⊗ Im) = (U ⊗ Im)(W ⊗ Im) Cn := a(S) := S ak j ak 2 ; C n−1 ! k=0 X k −1 −1 ∗ × Ek ⊗ a(w ) (W ⊗ Im)(U ⊗ Im) and the space of (n × n)-circulant m-block matrices k=0 ( n−1 ) n−1 ! X k −1 ∗ X k m×m = (U ⊗ I ) E ⊗ a(w ) (U ⊗ I ) Cn;m := a(S) := S ⊗ ak j ak 2 C : m k m k=0 k=0 −1 It can be seen that a (block-)circulant matrix is completely de- = a(S) : fined by its first (block-)row, and also that a circulant matrix ∗ Therefore, the inverse is also circulant/block-circulant by a(S) is Hermitian if a0 = a as well as 0 Proposition 1. ∗ j n k ak = an−k for k = 1;::: : Corollary 3: 2A Hermitian matrix a(S) 2 Cn;m is non-negative 2 definite if and only if the following m×m matrices are non-negative, Thus, when n is even, ab n c needs to be Hermitian as well for 2 n−1 a(S) to be Hermitian. The cone of Hermitian non-negative el- −j2π`=n X −j2π`k=n + + a(e ) = e ak ≥ 0; for ` = 0; : : : ; n − 1: ements in these two spaces will be denoted by Cn and Cn;m, k=0 respectively. E.g., Proof: Follows readily from equation (3). + ∗ Cn := fa(S) ≥ 0 j a(S) = a(S) 2 Cng 2 Of course, in the above, if m = 1 then a(x) is a polynomial where ≥ 0 denotes non-negative definiteness while 0 denotes the and det(a(S)) = Qn a(e−j2π`=n). However, in general, zero matrix of suitable size. Next, we provide certain basic facts `=1 about circulant/block-circulant matrices.