FrontREVIEW:Comput ARTICLE:Sci: DOI State of the Art and Prospects of Structured Sensing Matrices in Compressed Sensing Kezhi Li 1, Shuang Cong 2;∗ 1ACCESS Center, Royal Institute of Technology (KTH) Stockholm, 10044, Sweden 2Dept. of Automation, University of Science and Technology of China Hefei, 230027, China c Higher Education Press and Springer-Verlag Berlin Heidelberg 201X Abstract Compressed sensing (CS) enables people to ac- least twice the highest frequency present in a signal (so-called quire the compressed measurements directly and recover Nyquist rate) to retain signal information intact [1, 2]. The sparse or compressible signals faithfully even when the sam- Shannon’s theorem solves the problem in theory perfectly, yet pling rate is much lower than the Nyquist rate. However, the unfortunately it is not omnipotent. In many applications such pure random sensing matrices usually require huge memory as remote surveillance or spectroscopy, sampling in the result for storage and high computational cost for signal reconstruc- with Nyquist rate is expensive, or even physically impossible. tion. Many structured sensing matrices have been proposed So as long as the recovery performance achieves an accept- recently to simplify the sensing scheme and the hardware im- able level, people want to build devices which are capable plementation in practice. Based on the restricted isometry of acquiring samples at a necessary rate as low as possible. property and coherence, couples of existing structured sens- In some other applications, such as imaging system or video ing matrices are reviewed in this paper, which have special processing, sampling a large number of measurements seems structures, high recovery performance, and many advantages feasible. However, because of the limited storage space and such as the simple construction, fast calculation and easy using of advanced compression techniques, people often dis- hardware implementation. The number of measurements and card the most received data, and just save a small amount of arXiv:1408.1391v1 [cs.IT] 7 Aug 2014 the universality of different structure matrices are compared. the compressed data (e.g., JEPG). Apparently it will waste Keywords Compressed sensing, structured sensing matri- lots of valuable sensing resources since the entire data are ces, RIP, coherence; sampled at first. Aiming at solving above problems, the compressed sens- ing (short for CS) theory [3–7] has become one of the hottest 1 Introduction research areas in signal processing since 2006. The research of CS has been growing very fast and it focuses on acquiring In the digital revolution, people are now employing various and reconstructing sparse or compressible signals. By using signal processing techniques and new sensing systems in gen- CS, compressed measurements can be acquired directly and eral electronic products with ever-increasing resolution and one may recover the original sparse or compressible signal fidelity. The conventional manners of sampling signals, im- faithfully even when the sampling rate is much lower than the ages, videos, or other data obey the celebrated Shannon’s the- Nyquist rate. An N-length signal x is regarded as sparse if x orem, that requires to sample a signal at a sampling rate at has K nonzero values and K N. Compressible x means that x can be well-approximated by another sparse signal f Received month dd, yyyy; accepted month dd, yyyy in certain domain Ψ by using only K nonzero coefficients: E-mail: scong@ustc:edu:cn x = Ψf; jfj0 = K. Normally the traditional compression tech- Kezhi LI et al. State of the Art and Prospects of Structured Sensing Matrices in Compressed Sensing niques2 preserve the values and locations of the largest co- in the construction, calculation or hardware implementation. efficients, such as JPEG, JPEG2000, MPEG. While CS has For many of them, corresponding fast recovery algorithms more efficient sensing or sampling protocols that capture the have been developed by exploiting their specific structures. essential information content embedded in the original sig- Here we put our emphasis on how the structured matrices are nal and obtain the condensed data straightforwardly. More generated and what are their recovery performances. Based precisely, these protocols are nonadaptive linear transforms, on these, the number of measurements and the universality which can be represented by well-designed matrices, called of different structure matrices are compared. The paper will sensing matrices Φ. These matrices should be incoherent to help readers to understand the characteristics of popular sens- the sparsifying matrix Ψ of the compressible signal. With ing matrices well and may inspire them to explore or pursuit the measurements and the sensing matrix, the process of ex- more efficient sensing schemes in the CS area. act reconstructing signals from a subset of measurements can The remaining paper is organized as follows. In Section be implemented by solving a nonlinear optimization prob- 2, we describe the core concepts of this paper: sensing ma- lem. The approaches to solving the nonlinear problems are trices, and introduce two prevalent criteria that examine the named reconstruction/recovery algorithms. From a mathe- effectiveness of sensing matrices: restricted isometry prop- matical point of view, compressed sensing is also deemed as erty and coherence. In Section 3, couples of structured sens- a technique of finding sparse solutions to underdetermined ing matrices are analyzed. The overview of applications is linear systems. discussed in Section 4. Finally, the prospects of structured The CS theory is a revolution in both the theory of reliable sensing matrices are discussed in Section 5 followed by the signal sampling and physical design of sensors. Since the conclusions in Section 6. original signal can be sensed from fewer linear projections rather than acquired in its initial domain, the sensing matri- ces play an important role in the CS framework. The prop- 2 Sensing Matrices erty of the sensing matrices affects the number of necessary In our real world, normally the useful signals are not random. measurements and the recovery performance directly. Early Images, videos or voices often contain specific structures and researchers proved that a random projection is one of the best strong correlation among pixels, frames or samples. These solutions [5, 8]. The projection matrices are generated by or- structures and correlations are the assumptions behind the thogonalizing measured vectors uniformly and independently sparse representation theory. Given an N-dimensional sig- on a unit sphere. In addition, sensing matrices consisting of nal x, Ψ denotes the sparsifying transform basis for x, where independent and identically distributed (i.i.d) entries drawn throughout this paper we assume that Ψ is an N × N normal- from a Gaussian or Bernoulli distribution also perform well ized unitary matrix satisfying Ψ∗Ψ = NI . So x can easily in both theory and practice [3, 4]. Though the problem has N be decomposed by means of a linear superposition of K ele- been solved in mathematics, there still exist many obstacles mentary components: to overcome. One main drawback of the pure random sensing matrices is that they require huge storage-memory, namely XK M × N entries to recover a length-N signal, and high com- x = fk k: (1) k=1 putational cost for signal reconstruction. Moreover, the diffi- culty of hardware implementation also makes them expensive which can be rewritten in the form of matrix multiplication in practice. as To simplify the sensing scheme, many structured sensing x = Ψf; (2) matrices have been proposed in recent years. In this paper, where f is a length N sparse vertical vector with K nonzero after explaining some terminologies such as restricted isom- values, K N. Typical transforms Ψ include discrete etry property and coherence, we give an introduction to cou- Fourier transform (DFT), discrete cosine transform (DCT) ples of existing structured sensing matrices, including sub- and Discrete Wavelet Transform (DWT). Sometimes the sampled incoherent bases, random Toeplitz matrices, random number of nonzero values in f is larger than K. In this case demodulator matrices, random convolution matrices, struc- people usually encode the most significant K non-zero entries turally random matrices, convolutional matrices using se- of f and disregard the rest, which is also the core principle quences and some other structured sensing matrices. These of the image compression standard JPEG (using DCT) and matrices have special structures which equip them efficiency JPEG2000 (using DWT). Front. Comput. Sci. 2014, X(X): XXXXX Now assuming a length-N signal x as defined in (1), the satisfy the RIP with parameters (K; δ) (δ 2 (0; 1)) if 3 data acquisition process can be described as (1 − δ)kfk2 ≤ kΘfk2 ≤ (1 + δ)kfk2; for all f 2 Γ; (4) y = Φx = ΦΨf = Θf; (3) where Γ represents the set of all length-N vectors with K non- where the measurement y represents an M × 1 sampled vec- zero coefficients. tor, Φ is an M × N measurement/sensing matrix, Θ = ΦΨ. (3) is the kernel equation of the sensing process. The sens- Generally speaking, RIP requires the sensing matrix to act ing process with a random Gaussian measurement matrix Φ as a near isometry on the set of all K-sparse signals. It is con- and a DCT matrix Ψ is illustrated in Figure 1, in which there sistent with the thought of spreading energy behind random are four columns of Ψ that correspond to nonzero fi coeffi- sensing matrices. So measurement y preserves the energy cients; the measurement vector y is a linear combination of that does not shrink or expand too much comparing with the these columns. The CS theory considers problems based on original signal x. If Θ satisfies the RIP, many reconstruction the fundamental equation (3). These problems can be sum- algorithms can be used to recover any K-sparse signal f from marized as how to design efficient sensing matrix Φ, and how M measurements Θf, such as Basis Pursuit (BP) or Matching to recover x given y and Φ.