Measurement, Quantization, Consistency

This famous strobe photograph taken by Dr. Harold “Doc” Edgerton at MIT in 1957 captures the transition of a milk drop as it splashes from a single drop into a continuous ring and then coalesces into droplets forming the points of a coronet. Copyright Harold and Ester Edgerton Family Foundation. Reproduced with permission.

n this article we present a signal processing frame- impose quantum mechanical constraints that we find work that we refer to as quantum signal processing useful and to avoid those that are not. This framework (QSP) [1] that is aimed at developing new or mod- provides a unifying conceptual structure for a variety of Iifying existing signal processing algorithms by bor- traditional processing techniques and a precise mathe- rowing from the principles of matical setting for developing gener- quantum mechanics and some of its Yonina C. Eldar and alizations and extensions of interesting axioms and constraints. algorithms, leading to a potentially However, in contrast to such fields as Alan V. Oppenheim useful paradigm for signal processing quantum computing and quantum with applications in areas including information theory, it does not inherently depend on the frame theory, quantization and sampling methods, detec- physics associated with quantum mechanics. Conse- tion, parameter estimation, covariance shaping, and quently, in developing the QSP framework we are free to multiuser wireless communication systems.

12 IEEE SIGNAL PROCESSING MAGAZINE NOVEMBER 2002 1053-5888/02/$17.00©2002IEEE Fig. 1 presents a general overview of the key elements These examples are just several of many that under- in quantum physics that provide the basis for the QSP score the fact that even in signal processing contexts that framework and an indication of the key results that have are not constrained by the physics, exploiting laws of na- so far been developed within this framework. In the re- ture can inspire new methods for algorithm design and mainder of this article, we elaborate on the various ele- may lead to interesting, efficient, and effective process- ments in this figure. ing techniques. Three fundamental inter-related underlying principles of quantum mechanics that play a major role in QSP as Overview presented here are the concept of a measurement, the Many new classes of signal processing algorithms have principle of measurement consistency, and the principle been developed by emulating the behavior of physical of quantization of the measurement output. In addition, systems. There are also many examples in the signal pro- when using quantum systems in a communication con- cessing literature in which new classes of algorithms have text, other principles arise such as inner product con- been developed by artificially imposing physical con- straints. QSP is based on exploiting these various straints on implementations that are not inherently sub- principles and constraints in the context of signal process- ject to these constraints. In this article, we survey a new ing algorithms. class of algorithms of this type that we refer to as QSP.A In a broad sense, the terms measurement, measure- fully detailed treatment is contained in [1] as well as in the ment consistency, and output quantization are well various journal and conference papers referred to known in signal processing, although not with the same throughout this article. Among the many well-known ex- meaning or precise mathematical interpretation and con- amples of digital signal processing algorithms that are de- straints as in quantum mechanics. rived by using physical phenomena and constraints as In signal processing, the term measurement can be metaphors are wave digital filters [2]. This class of filters given a variety of precise or imprecise interpretations. relies on emulating the physical constraints of passivity However, as discussed further in this article, in quantum and energy conservation associated with analog filters to mechanics measurement has a very specific definition and achieve low sensitivity in coefficient variations in digital meaning, much of which we carry over to the QSP frame- filters. As another class of ex- amples, the fractal-like as- pects of nature [3] and Quantum related modeling have in- Physics spired interesting signal pro- cessing paradigms that are not constrained by the asso- Quantum Generalized Quantum ciated physics. These include Measurement Measurement fractal modulation [4], which emulates the fractal characteristic of nature for Measurement Measurement Measurement Vector communicating over a par- Quantization Consistency Orthogonality ticular class of unreliable channels, and the various ap- proaches to image compres- Quantum Detection sion based on synthetic Quantum generation of fractals [5]. Physics Quantum Likewise, the chaotic behav- Signal Processing ior of certain features of na- ture have inspired new QSP Inner Product Combined QSP classes of signals for secure Measurement Constraints Measurement communications, remote sensing, and a variety of Subspace Coding Least-Squares Randomized Algorithms other signal processing ap- Probabilistic Quantization Inner Product Shaping Frames plications [6]-[9]. Other ex- CS Matched Filter Subspace Detectors amples of algorithms using General Sampling Framework MMSE Covariance Shaping physical systems as a simile CS Least-Squares Estimation include solitons [10], ge- CS Multiuser Detection netic algorithms [11], simu- lated annealing [12], and s 1. QSP framework, its relationship to quantum physics, and key results. In the figure, CS stands neural networks [13]. for covariance shaping.

NOVEMBER 2002 IEEE SIGNAL PROCESSING MAGAZINE 13 work. Similarly, in signal processing, quantization is tra- terms of a set of measurement vectors µ that span mea- i ditionally thought of in fairly specific terms. In quantum surement subspaces S in H. The laws of quantum me- i mechanics, quantization of the measurement output is a chanics impose the constraint that the vectors µ must be i fundamental underlying principle, and applying this orthonormal and therefore also linearly independent. A principle, along with the quantum mechanical notions of measurement of this form is referred to as a rank-one measurement and consistency, leads to some potentially quantum measurement. In the more general case, the intriguing generalizations of quantization as it is typically quantum measurement is described in terms of a set of viewed in signal processing. projection operators P onto subspaces S of H, where i i Measurement consistency also has a precise meaning from the laws of quantum mechanics these projections in quantum mechanics, specifically that repeated applica- must form a complete set of orthogonal projections. Such tions of a measurement must yield the same outcome. In a measurement is referred to as a subspace quantum mea- signal processing, a similar consistency concept is the ba- surement. In quantum mechanics, the outcome of a mea- sis for a variety of classes of algorithms including signal surement is inherently probabilistic, with the estimation, interpolation, and quantization methods. Some early examples of consistency as it typically arises in probabilities of the outcomes of any measurement deter- signal processing are the interpolation condition in filter mined by the vector representing the underlying state of design [14] and the condition for avoiding intersymbol the system at the time of the measurement. The measure- interference in waveforms for pulse amplitude modula- ment collapses (projects) the state of the quantum system tion [15]. More recent examples include perfect recon- onto a state that is compatible with the measurement out- struction filter banks [16], [17], multiresolution and come. In general the final state of the system is different wavelet approximations [18], [19], and sampling meth- than the state of the system prior to the measurement. ods in which the perfect reconstruction requirement is re- Measurement consistency is a fundamental postulate of placed by the less stringent consistency requirement quantum mechanics, i.e., repeated measurements on a [20]-[24]. Here again, viewing measurement consis- system must yield the same outcomes; otherwise we tency in a broader framework motivated by quantum me- would not be able to confirm the output of a measure- chanics can lead to some new and interesting signal ment. Therefore the state of the system after a measure- processing algorithms. ment must be such that if we instantaneously remeasure In the next section, we summarize the basic principles the system in this state, then the final state after this sec- of measurement, consistency,and quantization as they re- ond measurement will be identical to the state after the late to quantum mechanics. We also outline the key ele- first measurement. ments and constraints that are associated with the use of Quantization of the measurement outcome is a direct quantum states for communication in the context of what consequence of the consistency requirement. Specifically, is commonly referred to as the quantum detection prob- the consistency requirement leads to a class of states re- lem [25]. We then indicate how these principles and con- straints imposed by the physics are emulated and applied ferred to as determinate states of the measurement [26]. in the framework of QSP. These are states of the quantum system for which the measurement yields a known outcome with probability one and are the states that lie completely in one of the measurement subspaces S . Furthermore, even when the i Quantum Systems state of the system is not one of the determinate states, af- In developing the QSP framework, it is convenient to dis- ter performing the measurement the system is quantized cuss both signal processing and quantum mechanics in a to one of these states, i.e., is certain to be in one of these vector space setting. Specifically we consider an arbitrary states, where the probability of being in a particular deter- Hilbert (inner product) spaceH with inner product 〈〉xy, minate state is a function of the inner products between for any vectors xy, in H. Typically we will refer to ele- the state of the system and the determinate states. See ments of H as vectors or signals interchangeably. In this “Quantum Measurement” for more details. section we summarize the key elements of physical quan- As an example of a rank-one quantum measurement, tum systems that are emulated in QSP,including those as- sociated with quantum detection. suppose that the measurement is defined by two orthonormal measurement vectors µ and µ and the 1 2 state of the system is given by x =+(/12 )µµ ( 32 / ) . 12 As outlined further in “Quantum Measurement,” the Measurement A quantum system in a pure state is characterized by a measurement will project the state x onto one of the mea- surement vectors µ or µ , where the probability of pro- normalized vector in H. Information about a quantum 1 2 jecting onto µ is px()|114=〈 ,µ 〉 |2 = / and the system is extracted by subjecting the system to a quantum 1 1 probability of projecting onto µ is measurement. 2 px()|234=〈 ,µ 〉 |2 = / . The measurement process is illus- A quantum measurement is a nonlinear (probabilistic) 2 mapping that in the simplest case can be described in trated in Fig. 2.

14 IEEE SIGNAL PROCESSING MAGAZINE NOVEMBER 2002 Quantum Measurement

S φ∈S φ=φ standard (von Neumann) measurement in quan- measurement spaces i . Indeed, if i , then Pi A tum mechanics is defined by a collection of pro- and from (5), pi()=1. ∈I jection operators {,Pii }onto subspaces The quantum measurement can be formulated in SH⊆∈ I I {,}i i , where denotes an index set and the terms of a probabilistic mapping between H and the index i ∈I corresponds to a possible measurement determinate states. Given a space of states X and an outcome. The laws of quantum mechanics impose the observation space Y,aprobabilistic mapping from X ∈I constraint that the operators {,Pii }form a com- to Y is a function f: XW×→ Y, where W is the sam- plete set of orthogonal projections so that with()*⋅ de- ple space of an auxiliary chance variable noting the adjoint of the corresponding = W Wx() { , pWX| ( wx |)}with a probability distribution transformation, for any ik, ∈I, W ∈X pwxWX| (|)on that in general depends on x . Note that a deterministic mapping f: XY→ is a spe- = * PPii; cial case of a probabilistic mapping in which the auxil- (1) iary chance variable is deterministic; i.e., it has one 2 = outcome w w.p. 1. In this case the function f is inde- PPii; (2) pendent of W. A rank-one quantum measurement corresponding =≠ µ ∈∈HI PPi k 0, if i k; to orthonormal measurement vectors {,}i i (3) SH⊆∈ I that span subspaces{,}i i can be viewed as a probabilistic mapping between H and the determi- ∑ PI= H . i∈I i nate states that is (4) s φ∈S 1) a deterministic identity mapping for i ; Conditions (3) and (4) imply that the measure- s 2) a probabilistic mapping for nondeterminate states S φ ment subspaces i are orthogonal and that their direct that maps to a normalized vector in the direction of H φ ∈I sum is equal to . the orthogonal projection Pi for some value i , φ =〈µ φ〉∈I If the state vector is , then from the rules of quan- where if({k , , k }, wi ). tum mechanics, the probability of observing the ith Here f: HW×→ Iis a probabilistic mapping be- outcome is tween elements φ of H and indices i ∈I, which de- pends on a chance variableW with a discrete alphabet =〈 φφ〉 pi() Pi , . WI= such that the probability of outcome w ∈W (5) i depends on the input φ only through the inner prod- ucts {,,µ φ∈k I }. Specifically, the probability of out- Since the state is normalized, k µ φ 2 come w i is |,|i .Ifw i is observed, then =〈φφ 〉 =〈φφ〉= 〈〉∈µ I = ∑∑pi() P , , 1. fxkwi({k , , },i ) . i i i A subspace quantum measurement corresponding In the simplest case, the projection operators are to a complete set of orthogonal projection operators =µ µ* ∈I SH⊆∈ I rank-one operators Piiifor some nonzero vectors {,Pii }onto subspaces {,}i i can be µ ∈∈HI {,}i i . We refer to such measurements as viewed as a probabilistic mapping between H and the rank-one quantum measurements. Then (3) and (4) determinate states that is µ ∈I imply that the measurement vectors{,i i }form an s 1) a deterministic identity mapping for φ∈S ; H φ i orthonormal basis for . If the state vector is , then s 2) a probabilistic mapping for nondeterminate the probability of observing the ith outcome is states that maps φ to a normalized vector in the direc-

2 tion of the orthogonal projection P φ for some value pi()=〈µ , φ〉 . i i i ∈I, where if=〈φφ〉∈({ PP , , kI }, w ). (6) kk i Here f: HW×→ Iis a probabilistic mapping be- For a rank-one quantum measurement defined by tween elements φ of H and indices i ∈I that depends µ WI= orthonormal measurement vectors i , it follows from on a chance variableW with a discrete alphabet φ=µ = ∈W (1) that if i for some i, then pi() 1 and output i is such that the probability of outcome w i depends obtained with probability one (w.p. 1). The states on the input φ only through the inner products φ=µ 〈φ φ〉∈I {}i are therefore called the determinate states of {,PPkk , k }. Specifically, the probability of out- 〈φ φ〉 the measurement. More generally, the determinate come w i is PPii, .Ifw i is observed, then φφ ∈I = states are the states that lie completely in one of the fPP({kk , , k }, wi ) i.

NOVEMBER 2002 IEEE SIGNAL PROCESSING MAGAZINE 15 Quantum Detection Necessary and sufficient conditions for an optimum The constraints imposed by the physics on a quantum measurement minimizing the probability of detection er- measurement lead to some interesting problems within ror are known [25], [27]-[29]. However, except in some the framework of quantum mechanics. In particular, an particular cases [25], [30]-[33], obtaining a closed-form interesting problem that arises when using quantum analytical expression for the optimal measurement di- states for communication is a quantum detection problem rectly from these conditions is a difficult and unsolved [25], which we outline here. As we discuss later in a sig- problem. nal processing context, this problem suggests a variety of In [32], an alternative approach is taken based on new or generalized algorithms within the QSP frame- choosing a squared-error criterion and determining a work based on specific ways of imposing inner product measurement that minimizes this criterion. Specifically, constraints. the measurement vectors µ are chosen to be orthogonal i In a quantum detection problem a sender conveys clas- and closest in a least-squares (LS) sense to the given set of sical information to a receiver using a quantum-mechani- state vectors φ so that the vectors µ are chosen to mini- i i cal channel. The sender represents messages by preparing mize the sum of the squared norms of the error vectors the quantum channel in a pure quantum state drawn from e =−φµ , as illustrated in Fig. 3. The optimal measure- iii a collection of known states φ . The receiver detects the i ment is referred to as the LS measurement. information by subjecting the channel to a quantum mea- The LS measurement problem has a simple closed- surement with measurement vectors µ that are con- i form solution with many desirable properties. Its con- strained by the physics to be orthogonal. If the states are struction is relatively simple; it can be determined directly not orthogonal, then no measurement can distinguish from the given collection of states; it minimizes the prob- perfectly between them. In the general context of quan- ability of detection error when the states exhibit certain tum mechanics, a fundamental problem is to construct symmetries [32]; it is “pretty good” when the states to be measurements optimized to distinguish between a set of distinguished are equally likely and almost orthogonal nonorthogonal pure quantum states. In the context of a [34]; it achieves a probability of error within a factor of communications system, we would like to choose the two of the optimal probability of error [35]; and it is as- measurement vectors to minimize the probability of de- ymptotically optimal [36]. tection error. In this context this problem is commonly Thus, in the context of quantum detection the con- referred to as the quantum detection problem. straints of the physics lead to the interesting problem of choosing an optimal set of orthogonal vectors. Bor- rowing from quantum detection, a central idea in QSP applications is to impose orthogonality or more general 1 µ inner product constraints on algorithms and then use the p(1)= 4 2 √ x =+1 µµ3 2 122 LS measurement and the results derived in the context of quantum detection to design optimal algorithms subject p µ (2)= 3 1 to these constraints. 4 Original Output State State Quantum Signal Processing s 2. Illustration of a rank-one quantum measurement. As mentioned previously, the QSP framework draws heavily on the notions of measurement, consistency, and quantization as they relate to quantum systems. Further- more it borrows from and generalizes the inner product µ 2 constraint, specifically orthogonality,that quantum phys- φ 2 ics imposes on measurement vectors. However, the QSP e 2 framework is broader and less restrictive than the quan- tum measurement framework since in designing algo- µ rithms we are not constrained by the physical limitations 1 e1 of quantum mechanics. φ 1 In quantum mechanics, systems are “processed” by performing measurements on them. In signal processing, signals are processed by applying an algorithm to them. Therefore, to exploit the formalism and rich mathemati- cal structure of quantum physics in the design of algo- rithms we first draw a parallel between a quantum s 3. Two-dimensional example of the LS measurement. The vec- mechanical measurement and a signal processing algo- µ tors i are chosen to be orthonormal and to minimize rithm by associating a signal processing measurement ∑∑〈ee,, 〉= 〈φ−µµ φ− 〉. i ii i iiii with a signal processing algorithm. Wethen apply the for-

16 IEEE SIGNAL PROCESSING MAGAZINE NOVEMBER 2002 malism and fundamental principles of quantum measure- in quantum mechanics, we require that if we remeasure ment to the definition of the QSP measurement. An the outcome signal, then the new outcome will be equal algorithm is then described by a QSP measurement, with to the original outcome. additional input and output mappings if appropriate. In analogy with the measurement in quantum me- This conceptual framework is illustrated schematically in chanics, a rank-one QSP measurement (ROM) M onH is Fig. 4. defined by a set of measurement vectors q that span i The QSP framework is primarily concerned with the one-dimensional subspaces S inH. Since we are not con- i design of the QSP measurement, borrowing from the strained by the physics of quantum mechanics, these vec- principles, axioms, and constraints of quantum physics tors are not constrained to be orthonormal nor are they and a quantum measurement. As we will show, the QSP constrained to be linearly independent. Nonetheless, in measurement depends on a specific set of measurement some applications we will find it useful to impose an parameters, so that this framework provides a convenient orthogonality constraint. A subspace QSP measurement and useful setting for deriving new algorithms by choos- on H is defined by a set of projection operators E onto i ing different measurement parameters, borrowing from subspaces S in H. Here again, since we are not con- i the ideas of quantum mechanics. Furthermore, since the strained by the physics, the projection operators and the QSP measurement is defined to have a mathematical subspaces S are not constrained to be orthogonal. The i structure similar to a quantum measurement, the mathe- measurement of a signal x is denoted by Mx(). matical constraints imposed by the physics on the quan- Measurement consistency in our framework is formu- tum measurement can also be imposed on the QSP lated mathematically as measurement leading to some intriguing new signal pro- cessing algorithms. MMx(())()= Mx. (7)

Note that by our definition of measurement, if x is a QSP Measurement signal in a signal space H then Mx()is also a signal in H The quantum-mechanical principles of measurement, and can therefore be remeasured. consistency,and quantization lead to the definition of the Quantization of the measurement outcome is imposed QSP measurement. by requiring that the outcome signal Mx()is one of a set Measurement of a signal in the QSP framework corre- of signals determined by the measurement M. Spe- sponds to applying an algorithm to a signal. As indicated cifically, in analogy with the quantum mechanical deter- in the previous section and in Fig. 4, one class of algo- minate states we define the set of determinate signals, rithms in the QSP frame- work consists of a QSP measurement (i.e., an algo- rithm) applied in a signal Quantum Metaphor Signal Processing Physics Algorithm space that may perhaps be a remapping of the overall in- put and output signal spaces. For example, if the signal we wish to process is a sequence of scalars, then we Quantum QSP may first map the scalar val- Measurement Measurement ues into vectors in a higher dimensional space, which corresponds to the input mapping in Fig. 4. We then measure the vector represen- tation. The measurement Input QSP Output outcome is a signal in the Mapping Measurement Mapping same signal space as the measured signal, which is then mapped to the algo- Signal Processing Algorithm rithm output using an out- put mapping as illustrated in s Fig. 4, so that the measure- 4. Illustration of a class of algorithms within the QSP framework. In this framework quantum ment output represents the physics is used as a metaphor to design new signal processing algorithms by drawing a parallel between a signal processing algorithm and a quantum mechanical measurement. One class of output of the algorithm, algorithms is designed by constructing a QSP measurement borrowing from the principles of a which in turn may be a sig- quantum measurement, which is then translated into a signal processing algorithm using appro- nal or any other element. As priate input and output mappings.

NOVEMBER 2002 IEEE SIGNAL PROCESSING MAGAZINE 17 chosen is determined by the mapping f that depends on M the input x only through the inner products 〈〉=xq,/34 1 q1 and 〈〉=xq,/12. The measurement process is illus- 3 1 1 1 2 fM (), xq=  +  q 4 2 2 122 trated in Fig. 5. q2 Our definition of a ROM is very similar to the defini- tion of a rank-one quantum measurement, with two main Measurement Measurement differences. First, we allow for an arbitrary mapping f M Input Output in (8); as described in “Quantum Measurement,” in quantum mechanics f is unique and is the probabilistic M mapping in which q is chosen with probability|,〈〉xq |2 . s 5. Illustration of a ROM. i i Second, the measurement vectors are not constrained to which are the signals that lie completely in one of the be orthonormal nor are they constrained to be linearly in- measurement subspaces S . dependent, as in quantum mechanics. The properties of a i The measurement M is then defined to preserve the rank-one quantum measurement and a ROM are summa- two fundamental properties of a quantum measurement. rized in Table 1. s The measurement outcome is always equal to one of the determinate signals. Subspace Measurements s For every input signal x to a QSP measurement, (7) is The definition of a subspace QSP measurement parallels satisfied. that of a ROM and borrows from the definition of a subspace quantum measurement. Rank-One QSP Measurements A subspace QSP measurement M defined by a set of A ROM M defined by a set of measurement vectors q measurement projections E that span measurement i i that span one-dimensional measurement subspaces S in subspaces S in H is a nonlinear mapping between H and i i H is in general a nonlinear mapping betweenH and the set the set of determinate signals of M where if x is a determi- nate signal then Mx()== Ex x, and otherwise of determinate signals of M. The measurement subspace i S is the one-dimensional subspace that contains all vec- Mx()= Exwhere i i tors that are multiples of q . Therefore, the determinate i signals in this case are all signals of the form aq for i=〈 f(){} ExEx, 〉. i Mkk (9) some index iand scalar a. With E denoting a projection i onto S , the measurement is defined such that if x is a Here f is a (possibly probabilistic) mapping between i M determinate signal then Mx()== Ex x, and otherwise the input signal x and the set of indices that depends on i Mx()= Exwhere the input x only through the inner products{,}〈〉ExEx . i kk A special case of a subspace QSP measurement is the if=〈〉(){} xq, . case in which the measurement is defined by a single Mk (8) projection. Then Mx()= Exfor all x and the subspace Here f is a mapping between the input signal x QSP measurement reduces to a linear projection opera- M and the set of indices that may be probabilistic (see tor. We refer to such a measurement as a simple “Quantum Measurement”) and that depends on the subspace measurement. input x only through the inner products between x and the measurement vectors q , which are a subset i Table 1. Comparison Between of the determinate signals. For example, we may Rank-One Measurements. choose fx()=〈〉arg max xq , . As another example, Mkk we may choose fx()=〈〉−〈〉arg max( qxqq , , ). This Quantum QSP Mkkkk Measurement Measurement mapping f chooses the vector q that minimizes the M k distance xq− between x and each of the measurement k Input x Vector in H Vector in H vectors, so that it maps x to the closest vector q . k Measurement Note that since Exis in S for any x, the outcome Orthonormal Any vectors in H i i vectors q Mx()is always a determinate signal of M, and since for k any determinate signal x, Mx()= x, this definition of a Function of measurement satisfies the required properties. Function of <>xq, k <>xq, k As an example of a ROM, suppose that the measure- Selection rule T ment input is xq=+(/12 ) (/ 12 ) qwhere q =[]10 Deterministic or 121 Probabilistic and q =[.55 . ]T with[]⋅ T denoting the conjugate trans- probabilistic 2 pose, are two measurement vectors. Then the measure- ment output will be either a vector in the direction of q Multiple of q Multiple of q for one 1 Output k k or a vector in the direction of q . The particular output for one value k value k 2

18 IEEE SIGNAL PROCESSING MAGAZINE NOVEMBER 2002 The subspace QSP measurement is very similar to a jections) in a Hilbert space H. This basic strategy is illus- subspace quantum measurement, with three main differ- trated in Fig. 6. If the signal~x to be processed does not lie ences: we allow for an arbitrary mapping f , the mea- inH, then we first map it into a signal x inH using a map- M surement projections are not constrained to be ping TX . To obtain the algorithm output we measure the orthogonal, and the measurement subspaces are not con- representation x of the signal to be processed. If x isade- strained to be orthogonal. terminate signal of M, then the measurement outcome is The properties of a subspace quantum measurement and yMxx==() . Otherwise we approximate x by a determi- a subspace QSP measurement are summarized in Table 2. nate signal y using a mapping f . If appropriate, the M measurement outcome y may be mapped to the algo- rithm output ~y using a mapping T . Algorithm Design in the QSP Framework Y By choosing different input and output mappings TX and T and different measurement parameters fq, , Within the QSP framework, the QSP measurement plays Y Mi and E , and using the QSP measurement framework of a central role in the design of signal processing algo- i rithms. In this framework, signals are processed by either Fig. 6, we can arrive at a variety of new and interesting subjecting them to a QSP measurement or by using some processing techniques. of the QSP measurement parameters fq, , and E but Mi i not directly applying the measurement, as described in Modifying Known Algorithms the following sections. As demonstrated in [1], many traditional detection and processing techniques fit naturally into the framework of Designing Algorithms Fig. 6. Examples include traditional and dithered Based on QSP Measurements quantization, sampling methods, matched-filter detec- Algorithm Design tion, and multiuser detection. Once an algorithm is de- To design an algorithm using a QSP measurement we scribed in the form of a QSP measurement, modifications first identify the measurement vectors q in a ROM, or and extensions of the algorithm can be derived by chang- i the measurement projection operators E in a subspace ing the measurement parameters f , q , and E . Thus, i M i i QSP measurement, that specify the possible measure- the QSP framework provides a unified conceptual struc- ment outcomes. For example, in a detection scenario the ture for a variety of traditional processing techniques and measurement vectors may be equal to the transmitted sig- a precise mathematical setting for generating new, poten- nals or may represent these signals in a possibly different tially effective, and efficient processing methods by modi- space. As another example, in a scalar quantizer the mea- fying the measurement parameters. surement vectors may be chosen as a set of vectors that To modify an existing algorithm represented by a map- represent the scalar quantization levels. In a subspace ping T using the QSP framework, we first cast the algo- QSP measurement, the measurement projection opera- rithm as a QSP measurement M, i.e., we choose an input tors may be projections onto a set of subspaces used for mapping TX and an output mapping TY if appropriate, and the measurement parameters fq, , and E .We signaling. We then embed the measurement vectors (pro- Mi i then systematically change some of these parameters, re- sulting in a modified measurement M′, which can then be Table 2. Comparison Between translated into a new signal processing algorithm repre- Subspace Measurements. sented by a mapping T′. The modifications we consider result from either imposing some of the additional con- Quantum QSP Measurement Measurement straints of quantum mechanics on the measurement pa- rameters of M or from relaxing some of these constraints Input x Vector in H Vector in H which we do not have to impose in signal processing. These basic steps are summarized in Fig. 7. Measurement Can be Typical modifications of the parameters that we con- Orthogonal projections Ek nonorthogonal sider include s using a probabilistic mapping f Measurement Can be M Orthogonal s imposing inner product constraints on the measure- subspaces S nonorthogonal k ment vectors q i Function of Function of

<>Exkk, Ex <>Exkk, Ex Selection rule ~ x y ~ Deterministic or x TX M TY y Probabilistic probabilistic

{,,}f MiqE i Multiple of Ex Multiple of Ex Output k k for one value k for one value k s 6. Designing algorithms using a QSP measurement.

NOVEMBER 2002 IEEE SIGNAL PROCESSING MAGAZINE 19 ~ ~ measurement vectors are restricted to have a certain inner 1. Original Algorithm x T y product structure, as in quantum mechanics. However, ~ x y ~ since we are not limited by physical laws, we are not con- 2. QSP Representation x TX M TY y fined to an orthogonality constraint. As part of the QSP ~ x y ~ framework, we develop methods for choosing a set of 3. Modify Parameters x TX M ′ TY y measurement vectors that “best” represent the signals of ~ ~ 4. Obtain New Algorithm x T ′ y interest and have a specified inner product structure [37], [1]; these methods rely on ideas and results we obtained s 7. Using the QSP measurement framework to modify existing in the context of quantum detection [32], which unlike signal processing algorithms. QSP are subject to the constraints of quantum physics. Specifically, we construct measurement vectors q with a i s using nonorthogonal (oblique) projections E in place i given inner product structure that are closest in an LS of orthogonal projections. sense to a given set of vectors s , so that the vectors q are i i chosen to minimize the sum of the squared norms of the error vectors eqs=−. These techniques are referred to as iii Designing Algorithms LS inner product shaping. Further details on LS inner prod- Using the Measurement Parameters Another class of algorithms we develop results from pro- uct shaping are summarized in “Least Squares Inner Prod- cessing a signal with some of the measurement parame- uct Shaping.” ters and then imposing quantum mechanical constraints As we show, the concept of LS inner product shaping directly on these parameters. For example, we may view can be used to develop effective solutions to a variety of any linear processing of a signal as processing with a set of problems that result from imposing a deterministic or measurement vectors and then imposing inner product stochastic inner product constraint on the algorithm and constraints on these vectors. Using the ideas of quantum then designing optimal algorithms subject to this con- detection we may then design linear algorithms that are straint. In each of these problems we either describe the optimal subject to these inner product constraints. algorithm as a QSP measurement and impose an inner To generate new algorithms or modify existing ones, product constraint on the corresponding measurement we describe the algorithm as processing by one of the vectors or we consider linear algorithms on which the in- measurement parameters and then modifying these pa- ner product constraints can be imposed directly.We dem- rameters using one of the three modifications outlined at onstrate that, even for problems without inherent inner the end of the previous section. product constraints, imposing such constraints in combi- In the remainder of this section we discuss each of nation with LS inner product shaping leads to new pro- these modifications and indicate how they will be applied cessing techniques in diverse areas including frame to the development of new processing methods. theory, detection, covariance shaping, linear estimation, and multiuser wireless communication, which often ex- hibit improved performance over traditional methods. Probabilistic Mappings The QSP framework naturally gives rise to probabilistic and randomized algorithms by letting f be a probabilis- Oblique Projections M tic mapping, emulating the quantum measurement. We In a quantum measurement defined by a set of projection expand on this idea below in the context of quantization. operators, the rules of quantum mechanics impose the Further examples are developed in [1]. However, the full constraint that the projections must be orthogonal. In potential benefits of probabilistic algorithms in general QSP we may explore more general types of measure- resulting from the QSP framework remain an interesting ments defined by projection operators that are not re- area of future study. stricted to be orthogonal, i.e., oblique projections [38]-[40]. An oblique projection is a projection operator E satis- Imposing Inner Product Constraints fying EE2 = that is not necessarily Hermitian, i.e., the One of the important elements of quantum mechanics is range space and null space of E are not necessarily or- that the measurement vectors are constrained to be thogonal spaces. The notation E denotes an oblique US orthonormal. This constraint leads to some interesting projection with range space U and null space S.IfSU= ⊥ , problems such as the quantum detection problem de- then E is an orthogonal projection onto U. An oblique US scribed previously. A fundamental problem in quantum projection E can be used to decompose x into its com- US mechanics is to construct optimal measurements subject ponents in two disjoint vector spaces U and S that are not to this constraint that best represent a given set of state constrained to be orthogonal, as illustrated in Fig. 8. vectors. In analogy to quantum mechanics, an important (Two subspaces are said to be disjoint if the only vector feature of QSP is that of imposing particular types of con- they have in common is the zero vector.) straints on algorithms. The QSP framework provides a Oblique projections are used within the QSP frame- systematic method for imposing such constraints: the work to develop new classes of frames, effective subspace

20 IEEE SIGNAL PROCESSING MAGAZINE NOVEMBER 2002 Least-Squares Inner Product Shaping

≤≤ uppose we are given a set of m vectors {,simi 1 } matrix if and only if the set is cyclic.) We may then wish S H 〈〉 〈〉≤≤ in a Hilbert space , with inner product xy, for to specify the values {,hh1 k ,1 k m }(possibly up to ∈H any xy, . The vectors si span an n-dimensional a scale factor), which corresponds to specifying the subspace. If the vectors are linearly independent, then eigenvalues of R, or we may choose these values, nm= ; otherwise nm< . Our objective is to construct a equivalently the eigenvalues ofR, so that the LS error is ≤≤ set of optimal vectors {,himi 1 }with a specified in- minimized. ner product structure, from the given vectors In [1] we consider both the case in which R is fixed ≤≤ {,simi 1 }. Specifically, we seek the vectors hi that and the case in which the eigenvalues of R are chosen ε are “closest” to the vectors si in the LS sense. Thus, the to minimize the LS error LS . As we show, for fixedR the vectors are chosen to minimize LS inner product shaping problem has a simple closed m form solution; by contrast, if the eigenvalues of R are ε =〈− −〉, LS ∑ shshiiii, not specified, then there is no known analytical solu- i =1 (10) tion to the LS inner product shaping problem for arbi- trary vectors s . An iterative algorithm is developed in subject to the constraint i [1]. In the simplest case where R is a specified rank-r 〈〉=2 matrix, with rnm==, the LS vectors are the “col- hhi , kik cr, (11) umns” of the (possibly infinite) matrix

> ~ */−−12~ * 12 / for some c 0 and constants rik . HSSS==αα()RRRR SSS(), We may wish to constrain the constant c in (11) or (12) ε may choose c such that the LS error LS is minimized.

Similarly, withR denoting the matrix with ikth element where S is the matrix of columns si .Ifc in (11) is speci- α=~ rik , we may wish to constrain the elements rik of R,or fied then c, and if c is chosen to minimize the LS er- we may choose R to have a specified structure so that ror then the eigenvectors of R are fixed, but choose the Tr()()SS*/R 12 eigenvalues to minimize the LS error. α=~ , Tr()R For example, we may wish to construct a set of or- (13) thogonal vectors, so that R is a diagonal matrix with eigenvector matrix equal to I, but choose the norms of where Tr(⋅ )denotes the trace of the corresponding ma- the vectors, i.e., the eigenvalues of R, to minimize the trix. LS error. As another example, we may wish to con- If rn= and NN()S = ()R where N ()⋅ denotes the 〈〉 struct a cyclic set hi so that hhi , k depends only on null space of the corresponding transformation, then − 〈〉≤≤ kimod m. In this case {,hhi k ,1 i m }is a cyclic † † 〈〉≤≤ ==~αα*/12~ */ 12 permutation of {,hh1 k ,1 k m }for all k, and the HSSS()()()RRRR SSS(), Gram matrix HH* is a circulant matrix diagonalized (14) by a DFT matrix, so that the eigenvectors of R are fixed. (In [37] we show that a vector set has a circulant Gram where()⋅ † denotes the Moore-Penrose pseudoinverse.

detectors, and a general sampling framework for sam- utility of dithering techniques is limited by the computa- pling and reconstruction in arbitrary spaces. tional complexity associated with generating a random process with an arbitrary joint probability distribution. A probabilistic quantizer can be used to effectively realize a dither signal with an arbitrary joint probability Applications of Rank-One Measurements distribution, while requiring only the generation of QSP Quantization one uniform random variable per input. By introduc- By emulating the quantum measurement, in [1] we de- ing memory into the probabilistic selection rule we velop a probabilistic quantizer and show that it can be derive a probabilistic quantizer that shapes the used to efficiently implement a dithered quantizer. quantization noise. In dithered quantization a random signal called a dither signal is added to the input signal prior to quantization [41]-[44]. Dithering techniques have be- Covariance Shaping Matched Filter Detection come commonplace in applications in which data is As an example of the type of procedure we may follow in quantized prior to storage or transmission. However, the using the concept of LS inner product shaping and opti-

NOVEMBER 2002 IEEE SIGNAL PROCESSING MAGAZINE 21 mal QSP measurements to derive new processing meth- impact in performance when the noise is Gaussian. These ods, in [45] and [46] we consider a generic detection results are encouraging since they suggest that if the re- problem where one of a set of signals is transmitted over ceiver is designed to operate in different noise environ- a noisy channel. When the additive noise is white and ments or in an unknown noise environment, then we may Gaussian, it is well known (see, e.g., [15] and [47]) that prefer using the modified detectors since for certain the receiver that maximizes the probability of correct de- non-Gaussian noise distributions these detectors may re- tection is the matched filter (MF) receiver. If the noise is sult in an improvement in performance over an MF detec- not Gaussian, then the MF receiver does not necessarily tor, without significantly degrading the performance if maximize the probability of correct detection. However, the noise is Gaussian. it is still used as the receiver of choice in many applica- In Fig. 9 we illustrate the performance advantage with tions since the optimal detector for non-Gaussian noise one simulation from [45]. In this figure we plot the mean is typically nonlinear (see, e.g., [48] and references therein) and depends on the noise distribution, which 1 may not be known. Orthogonal Matched Filter By describing the MF detector as a QSP measurement 0.9 Matched Filter and imposing an inner product constraint on the mea- 0.8 surement vectors, we derive a new class of receivers con- 0.7 sisting of a bank of correlators with correlating signals that are matched to a set of signals with a specified inner 0.6

d product structure R and are closest in an LS sense to the P 0.5 transmitted signals. These receivers depend only on the 0.4 transmitted signals, so that they do not require knowl- 0.3 edge of the noise distribution or the channel sig- nal-to-noise ratio (SNR). We refer to these receivers as 0.2 covariance shaping MF receivers [1]. In the special case in 0.1 which RI= , the receiver consists of a bank of correlators 0 with orthogonal correlating signals that are closest in an 0 5 10 15 20 25 30 LS sense to the transmitted signals and is referred to as the Number of Signals in Transmitted Constellation orthogonal matched filter (OMF) receiver [45], [46]. Alternatively, we show that the modified receivers can s 9. Comparison between the OMF and MF detectors in Beta-dis- be implemented as an MF demodulator followed by an tributed noise, as a function of the number of signals in the optimal covariance shaping transformation that opti- transmitted constellation. The parameters of the distribution are ab==01. , and the SNR is 0 dB. The dashed line is the mally shapes the correlation of the outputs of the MF mean Pd using the OMF detector, and the solid line is the prior to detection. This equivalent representation leads to mean P using the MF detector. The vertical bars indicate the the concept of minimum mean-squared error (MMSE) d standard deviation of the corresponding Pd . covariance shaping, which we consider in its most general form in [1] and [49]. Simulations presented in [45] and [46] show that when the noise is non-Gaussian this ap- 1 proach can lead to improved performance over conven- 0.9 tional MF detection in many cases, with only a minor 0.8

0.7 S 0.6 Pd 0.5 U 0.4

0.3 ExUS 0.2 x 0.1 Orthogonal Matched Filter Matched Filter 0 S –10 –5 0 5 10 15 20 SNR [dB] ExSU s 10. Comparison between the OMF and MF detectors for trans- mitted constellations of seven signals in Gaussian noise, as a

function of SNR. The dashed line is the mean Pd using the OMF

detector, and the solid line is the mean Pd using the MF detec- s 8. Decomposition of x into its components in U and in S given tor. The vertical bars indicate the standard deviation of the cor-

by ExUS and ExSU , respectively. responding Pd .

22 IEEE SIGNAL PROCESSING MAGAZINE NOVEMBER 2002 and standard deviation of the probability of correct detec- shaping transformation T satisfying (15). While in some tion P for the OMF detector with RI= in applications certain conditions might be imposed on the d Beta-distributed noise, as a function of the number of sig- transformation such as causality or symmetry, with the nals in the transmitted constellation. The dimension m of exception of the work in [46], [45], [52], [53], [49], and the signals in the constellation is equal to the number of [54], which explicitly relies on the optimality properties signals and the samples of the signals are mutually inde- developed in [1], there have been no general assertions of pendent zero-mean Gaussian random variables with vari- optimality for various choices of a linear shaping transfor- ance 1/ m, scaled to have norm one. The vertical bars mation. In particular, the shaped vector may not be indicate the standard deviation of P . The results in the “close” to the original data vector. If this vector under- d figure were obtained by generating 500 realizations of goes some noninvertible processing or is used as an esti- signals. For each particular signal realization, we deter- mator of some unknown parameters represented by the mined the probability of correct detection for the detec- data, then we may wish to choose the covariance shaping tors in both types of noise by recording the number of transformation so that the shaped output is close to the successful detections over 500 noise realizations. From original data in some sense. the figure it is evident that the OMF detector outper- Building upon the concept of LS inner product shap- forms the MF detector, particularly when the probability ing, we propose choosing an optimal shaping transfor- of correct detection with the MF is marginal. The relative mation that results in a shaped vector b that is as close as improvement in performance of the OMF detector over possible to the original vector a in an MSE sense, which the MF detector increases for increasing constellation we refer to as MMSE covariance shaping. Specifically, size. In Fig. 10 we plot the mean of P for the OMF detec- d among all possible transformations we seek the one that tor and the MF detector in Gaussian noise, for transmit- minimizes the total MSE given by ted constellations of seven signals in Gaussian noise, as a m function of SNR. Again, the vertical bars indicate the ε =−=−−Ea()( b )2 E() ()()ab* ab, MSE ∑ ii standard deviation of P . i=1 (16) d subject to (15), where a and b are the ith components of i i Optimal Covariance Shaping a and b, respectively. Drawing from the quantum detection problem, we can The solution to the MMSE covariance shaping prob- develop new classes of linear algorithms that result from lem is developed in [1]; further details are summarized in imposing a deterministic or stochastic inner product con- “MMSE Covariance Shaping.” This new concept of straint on the algorithm, i.e., a covariance constraint, and MMSE shaping can be useful in a variety of signal pro- then using the results we obtained in the context of quan- cessing methods that incorporate shaping transforma- tum detection to derive optimal algorithms subject to this tions in which we can imagine using an optimal constraint. In particular, we may extend the concept of LS procedure that shapes the data but at the same time mini- inner product shaping suggested by the quantum detec- mizes the distortion to the original data. tion framework to develop optimal algorithms that mini- mize a stochastic mean-squared error (MSE) criterion subject to a covariance constraint. Linear Estimation As an example of this approach, in [1] we exploit the As another example of an algorithm suggested by the concept of LS inner product shaping to develop a new quantum detection framework, where we use the ideas viewpoint towards whitening and other covariance shap- of LS inner product shaping to design an optimal linear ing problems. algorithm subject to a stochastic inner product con- Suppose we have a random vector a that lies in the straint, in [54] we derive a new linear estimator for the m unknown deterministic parameters in a linear model. m-dimensional space C with covariance C a , and we want to shape the covariance of the vector a using a shap- The estimator is chosen to minimize an MSE criterion, ing transformation T to obtain the random vector bTa= , subject to a constraint on the covariance of the estima- where the covariance matrix of b is given by CR= c 2 tor. This new estimator is defined as the covariance shap- b for some c >0 and some covariance matrix R. Thus we ing LS (CSLS) estimator. seek a transformation T such that Many estimation problems can be represented by the linear model yHxw=+, where His a known matrix, x is CTCT==*2c R, (15) b a a vector of unknown deterministic parameters to be esti-

mated, and w is a random vector with covariance C w .A for some c >0. common approach to estimating the parameters x is to re- Data shaping arises in a variety of contexts in which it strict the estimator to be linear in the data y and then find is useful to shape the covariance of a data vector either the linear estimate of x that results in an estimated data prior to subsequent processing or to control the spectral vector that is as close as possible to the given data vector y shape after processing [50], [51]. As is well known, given in a (weighted) LS sense, so that it minimizes the total a covariance matrix C a , there are many ways to choose a squared error in the observations [55]-[58]. It is well

NOVEMBER 2002 IEEE SIGNAL PROCESSING MAGAZINE 23 MMSE Covariance Shaping

n [1] we show that the MMSE covariance shaping in signal processing applications incorporating whitening Iproblem can be interpreted as an LS inner product (see, e.g., [57], [50], and [51]). shaping problem, so that the MMSE shaping transfor- Another interesting case is when the random vector a is = mation can be found by applying results derived in that white so thatCIa and the problem is to optimally shape its

context. In the simplest case in which Ca and R are in- covariance. The MMSE transformation in this case is vertible, the optimal shaping transformation is given 12/ by TR=α , (21) T==αα R() C R−−12// () RC 12 R, aa where if c is fixed then α=c, and if c is chosen to minimize (17) the MSE then where if c in (15) is specified then α=c, and if c is cho- 12/ α=Tr()R sen to minimize the MSE then . Tr()R (22) Tr((CR )12/ ) α= a . Tr()R We may also consider a weighted MMSE covariance shap- (18) ing problem in which the shaping T is chosen to minimize a

12/ weighted MSE. Thus we seek a transformation T such that Here ()⋅ denotes the unique nonnegative-definite bTA= has covarianceCR= c2 for some c > 0and such that square root of the corresponding matrix. b When RI= so that T is a whitening transformation, εw =−E((abAab )* ( − )), (17) reduces to MSE (23)

= α −12/ TCa is minimized, where A is some nonnegative-definite (19) Hermitian weighting matrix. In the simplest case in whichCa , R, and A are all invertible, the weighted MMSE covariance and (18) becomes shaping transformation is

1 12/ α= −12/ Tr()Ca . = α m TRACARA()a , (20) (24)

It is interesting to note that the MMSE whitening where if c is specified then α=c, and if c is chosen to mini- transformation has the additional property that it is the mize the weighted MSE then unique symmetric whitening transformation (up to a Tr((RAC A )12/ ) factor of ±1) [103]. It is also proportional to the α= a . Mahalanobis transformation, which is frequently used Tr()RA (25) known that among all possible unbiased linear estima- The CSLS estimator, denoted by x$ , is a biased es- CSLS tors, the LS estimator minimizes the variance [56]. How- timator directed at improving the performance of the ever, this does not imply that the resulting variance or traditional LS estimator at low to moderate SNR by MSE is small, where the MSE of an estimator is the sum choosing the estimate to minimize the (weighted) total of the variance and the squared norm of the bias. In par- error variance in the observations subject to a constraint ticular, a difficulty often encountered when using the LS on the covariance of the estimation error, so that we con- estimator to estimate the parameters x is that the error in trol the dynamic range and spectral shape of the covariance of the estimation error. Thus, xGy$ = is estimating x can have a large variance and a covariance CSLS structure with a very high dynamic range. This is due to chosen to minimize the fact that in many cases the data vector y is not very ε =′−′E()()()yHGyC* −1 yHGy ′−′, sensitive to changes in x, so that a large error in estimating CSLS w (26) x may translate into a small error in estimating the data vector y, in which case the LS estimate may result in a where yy′= − E() y, subject to the constraint that the poor estimate of x. This effect is especially predominant covariance of the error in the estimate x$ , which is CSLS at low to moderate SNR, where the data vector y is typi- equal to the covariance of the estimate x$ , is propor- CSLS cally affected more by the noise than by changes in x; the tional to a given covariance matrix R. ThusG must satisfy exact SNR range will depend on the properties of the * 2 model matrix H. GCw G= c R, (27)

24 IEEE SIGNAL PROCESSING MAGAZINE NOVEMBER 2002 Covariance Shaping Least-Squares Estimation

f the model matrix H has full column rank, then the SNR values below this threshold the CSLS estimator yields Icovariance shaping LS estimator that minimizes (26) a lower MSE than the LS estimator, for all values of x. subject to (27) is given by If the output covariance is chosen to be equal to σR, =σ2 where CCw for some covariance matrix C, then it can $ = β */*−−112 − 1 be shown that there is a threshold SNR value so that for xy,CSLS ()RH Cww H RH C (28) SNR values below this threshold, the MSE of the CSLS esti- where if c in (27) is specified thenβ=c, and if c is chosen to mator is always smaller than the MSE of the LS estimator, ζσ= 2 2 minimize the error then regardless of the value of x. Specifically, let x /(m ) denote the SNR per dimension. Then with BHCH= *1− , */−112 β=Tr((RH Cw H ) ) γσ= arg max , and σ denoting the eigenvalues of − . k k * 1 (29) 12/* 12 / Tr()RH Cw H QII=−((RB ) ) (( RB ) − ), the MSE of the CSLS estima- tor is less than or equal to the MSE of LS estimator for The CSLS estimator can also be expressed as an LS esti- $ ζζ≤ , where mator followed by a weighted MMSE covariance shaping WC transformation. Specifically, suppose we estimate the pa- −1 − ζ$ = Tr()BR Tr () $ $ =+~ WC . (32) rameters x using the LS estimate xLS . Since xxwLS σγ where ww~ = ()H*C−−11 H H*C , the covariance of the ww ζ$ ~ ~ $ The bound WC is a worst case bound, since it corre- noise component w in xLS is equal to the covariance of xLS , = σ211* −− sponds to the worst possible choice of parameters, namely denoted C $ , which is given by CHCH$ ()w .Toim- x LS x LS when the unknown vector x is in the direction of the prove the performance of the LS estimator, we may shape eigenvector of Q corresponding to the eigenvalue σ .In the covariance of the noise component in the estimator γ practice the CSLS estimator will outperform the LS estima- x$ . Thus we seek a transformation W such that the LS ζ$ $$ tor for higher values of SNR than WC . = $ covariance matrix of xxW LS , denoted by C x, satisfies * 2 Since we have freedom in designing R, we may always CWCW$$==c R, for some c > 0. To minimize the dis- x x LS ζ$ > $ choose R so that WC 0. In this case we are guaranteed tortion to the estimator xLS , we choose the transformation W that minimizes the weighted MSE that there is a range of SNR values for which the CSLS esti- mator leads to a lower MSE than the LS estimator for all $$* −1 $$ choices of the unknown parameters x. E()(xx′ −′WC)($ xx′ − W′ ) . LS LS x LS LS (30) LS For example, suppose we wish to design an estimator with covariance proportional to some given covariance As we show in [54], the resulting estimator x$ is equal to matrix Z, so that RZ= a for some a > 0. If we choose x$ , so that the CSLS estimator can be determined by first CSLS a < Tr()/()BZ−1 Tr , then we are guaranteed that there is an finding the LS estimator x$ and then optimally shaping its LS SNR range for which the CSLS estimator will have a lower covariance. The CSLS estimator with fixed scaling can also MSE than the LS estimator for all values of x. be expressed as a matched correlator estimator followed In specific applications it may not be obvious how to by MMSE shaping. choose a particular proportionality factor a. In such cases, we If the noise w in the model yH=+x wis Gaussian with may prefer using the CSLS estimator with optimal scaling. In zero-mean and covariance C , then the CSLS estimator w this case, the scaling is a function of R and therefore cannot achieves the CRLB for biased estimators with bias B equal be chosen arbitrarily, so that in general we can no longer to the bias of the CSLS estimator, which is given by guarantee that there is a positive SNR threshold, i.e., that

− there is always an SNR range over which the CSLS performs B =−()β()RH*/ C112 H Ix. w (31) better than the LS estimator. However, as shown in [54], in the special case in which R = I, there is always such an SNR β Thus, from all estimators with bias given by (31) for some λ ≤≤ range. Specifically, with {,k 1 km }denoting the m m and R, the CSLS estimator minimizes the variance. eigenvalues ofBHCH= *1− , and αλ= ()/∑ 12/ ()∑ λ , k =1 k k =1 k While it would be desirable to analyze the MSE of the the MSE of the CSLS estimator is less than or equal to the CSLS estimator for more general forms of bias, we cannot ζζ≤ MSE of LS estimator for WC , where directly evaluate the MSE of the CSLS estimator since the m (/1 m )∑ λα−12− bias, and consequently the MSE, depend explicitly on the ζ = k =1 k WC 2 , 12/ (33) unknown parameters x. Instead, we may compare the MSE αλγ −1 of the CSLS estimator with the MSE of the LS estimator. The 2 γαλ=−12/ analysis in [54] indicates that there are many cases in and argmax k 1 . ζ which the CSLS estimator performs better than the LS esti- Here again WC is a worst case bound; in practice the CSLS mator in a MSE sense, for all values of the unknown pa- estimator will outperform the LS estimator for higher values ζ rameters x. Specifically, for a variety of choices of the of SNR than WC . Examples presented in [54] indicate that in a ζ output covarianceR, there is a threshold SNR, such that for variety of applications WC can be pretty large.

NOVEMBER 2002 IEEE SIGNAL PROCESSING MAGAZINE 25 where c >0 is a constant that is either specified or chosen noise variance. As can be seen from the figure, in this ex- to minimize the error (26). ample the CSLS estimator significantly outperforms the The CSLS estimator x$ is developed in [1] and LS estimator. In general, the performance advantage us- CSLS [54]. Some of its properties are discussed in “Covariance ing the CSLS estimator will depend on the properties of

Shaping Least-Squares Estimation.” the model matrix Hand the noise covariance C w , as indi- Various modifications of the LS estimator under the cated by the analysis in [54] and in “Covariance Shaping linear model assumption have been previously proposed Least-Squares Estimation.” in the literature. Among the more prominent alternatives are the ridge estimator [59] (also known as Tikhonov regularization [60]) and the shrunken estimator [61]. In Multiuser Detection [1] we show that both the ridge estimator and the Based on the concept of CSLS estimation, we propose a shrunken estimator can be formulated as CSLS estima- new class of linear receivers for synchronous code-divi- tors, which allows us to interpret these estimators as the sion multiple-access (CDMA) systems, which we refer to estimators that minimize the total error variance in the as the covariance shaping multiuser (CSMU) receivers. observations, from all linear estimators with the same These receivers depend only on the users’ signatures and covariance. do not require knowledge of the channel parameters. As shown in [54], the CSLS estimator has a prop- Nonetheless, over a wide range of these parameters the erty analogous to the property of the LS estimator. Spe- performance of these receivers can approach the perfor- cifically, it achieves the Cramer-Rao lower bound mance of the linear MMSE receiver which is the optimal (CRLB) for biased estimators [56], [62], [63] when linear receiver that assumes knowledge of the channel pa- the noise is Gaussian. This implies that for Gaussian rameters and maximizes the output signal-to-interference noise, there is no linear or nonlinear estimator with a ratio. These receivers generalize the recently proposed or- smaller variance, or MSE, and the same bias as the thogonal multiuser receiver [52]. CSLS estimator. Consider the problem of detecting information trans- Analysis of the MSE of the CSLS estimator [54] dem- mitted by each of the users in an m-user CDMA system. onstrates that the covariance of the estimation error can Each user transmits information by modulating a signa- be chosen such that over a wide range of SNR, the CSLS ture sequence. The discrete-time model for the received estimator results in a lower MSE than the traditional LS signal y is given by [64] estimator, for all values of the unknown parameters. Sim- ulations presented in [1] and [54] strongly suggest that ySAbw=+, (34) the CSLS estimator can significantly decrease the MSE of where S is the nm× matrix of columns s and s is the the estimation error over the LS estimator for a wide k k range of SNR values. As an example, in Fig. 11 we plot length-n signature vector of the kth user, A = diag(AA ,K , ) where A >0 is the received ampli- the MSE in estimating a set of AR parameters in an 1 m k tude of the kth user’s signal, b is a vector of elements b ARMA model contaminated by white noise, using both k where b ∈−{,11 }is the bit transmitted by the kth user, the CSLS with R = I and the LS estimators from 20 noisy k observations of the channel, averaged over 2000 noise re- and w is a white Gaussian noise vector with zero mean 2 2 2 and covariance CI=σ . alizations, as a function of −10log σ where σ is the w Given the received signal y the problem is to detect the information bits b of the different users. One approach k 100 consists of estimating the vector xAb= and then detect- CSLS ing the kth symbol as bx$ =sgn()where x is the kth com- LS kk k ponent of x. 10−1 Estimating x using an LS estimator results in the well-known decorrelator receiver, first proposed by Lupas and Verdu [65]. Alternatively, we may estimate x 10−2 using the CSLS estimator, which results in a new receiver

MSE which we refer to as the CSMU receiver. The form of this 10−3 receiver is discussed in “Covariance Shaping Multiuser Detection.” From the general properties of the CSLS estimator [1], 10−4 [54], it follows that the CSMU receiver can be imple- mented as a decorrelator receiver followed by a (weighted) 10−5 MMSE covariance shaping transformation that optimally –5 0 5 10 15 20 25 30 35 shapes the covariance of the output of the decorrelator. SNR [dB] The choice of shaping can be tailored to the specific set of s 11. Mean-squared error in estimating the AR parameters using signatures. It can also be shown that this receiver is equiva- the LS estimator and the CSLS estimator. lent to an MF receiver followed by an MMSE covariance

26 IEEE SIGNAL PROCESSING MAGAZINE NOVEMBER 2002 shaping transformation. Finally, the CSMU receiver can Covariance Shaping Multiuser Detection also be implemented as a correlation demodulator with correlating signals with inner product matrix R that are he CSMU detector results from estimating x = Ab closest in an LS sense to the signature vectors. Tin (34) using the CSLS estimator, which in the case In the special case in which RI= , the shaping transfor- of linearly independent signature vectors leads to the mation is a whitening transformation, and the CSMU re- estimator ceiver is equivalent to a decorrelator receiver followed by MMSE whitening. This receiver has been referred to as the orthogonal multiuser receiver [22], [53]. By allowing x$ = ()RS*/* S−12 RS y, CSLS for other choices of R we can improve the performance (35) over both the decorrelator and the orthogonal multiuser receiver for a wide range of channel parameters. whereR is any positive-definite Hermitian matrix. Note To demonstrate the performance advantage in using $ that the scaling of xCSLS will not effect the detector out- the CSLS estimator, we consider the case in which the sig- put and therefore can be chosen arbitrarily. The result- nature vectors are chosen as pseudonoise sequences cor- ing receiver cross-correlates y with each of the responding to maximal-length, shift-register sequences Q = */−12 columns qk of SR() S SR to yield the outputs [64], [66], so that d = qy* .Thekth users’ bit is then detected as $kk bd= sgn(). 1,;lk= kk ss* =  kl −≠1 /,nlk . (41) In the large system limit when both the number of us- ers and the length of the signature vectors goes to infinity The shaping R is chosen as a circulant matrix with param- with fixed ratio β=nm/ , it can be shown that the SINR γ at the output of the CSMU detector withRI= converges to eter ρ: a deterministic limit when random Gaussian signatures 1 ρρ⋅⋅⋅ ρ and accurate power control are used. Specifically, in [53]   we show that ρρ1 ⋅⋅⋅ ρ R = . ms..  MO M γ →   ρρ⋅⋅⋅ ρ 1 (42) 1 − 2 1 ηηη[]()+−−−EK()121 ηη / η() ηη / 1− 212 12 1 12 Fig. 12 compares the theoretical probability of bit error πβ22 ζ+ 91() of the CSMU receiver in the case of ten users with ρ=035.

and with accurate power control so that Am =1 for all m. (36) The corresponding curves for the decorrelator, MF, and linear MMSE receivers are plotted for comparison. The where1/ ζ is the received SNR, MMSE receiver is the linear receiver that maximizes the signal-to-interference ratio and, unlike the decorrelator, π/2 dt 1 dx Kk()= ∫∫= MF,and CSMU receivers, requires knowledge of the chan- 0 111− kt22sin0 (−−xkx222 )( ) (37) nel parameters. We see that the CSMU receiver performs better than the decorrelator and the MF and performs sim- 22 π/2 1 1− kx ilarly to the linear MMSE receiver, even though it does not Ek()=−∫∫1 k22 sin tdt = dx 0 0 1− x2 rely on knowledge of the channel parameters. (38) In [67] we develop methods to analyze the output sig- nal-to-interference+noise ratio (SINR) of the CSMU receiver are the complete elliptic integrals of the first and sec- in the large system limit. We show that the SINR converges to ond kinds, respectively [104], and a deterministic limit and compare this limit to the known SINR limits for the decorrelator, MF,and linear MMSE receiv- 2 ers [67]-[69]. The analysis suggests that this modified receiver ηβ=−()1 1 can lead to improved performance over the decorrelator and (39) MF receiver and can approach the performance of the linear MMSE receiver over a wide range of channel parameters with- 2 ηβ=+ out requiring knowledge of these parameters. 2 ()1 . (40)

m.s. Applications of Subspace Measurements The notation → denotes convergence in the mean-squared (L2 ) sense [105]. Simple Subspace Measurements A simple subspace QSP measurement is equivalent to a linear projection operator. Numerous signal processing

NOVEMBER 2002 IEEE SIGNAL PROCESSING MAGAZINE 27 and detection algorithms based on orthogonal projec- larger system. Alternatively, we can view a generalized tions have been developed. Algorithms based on quantum measurement as a combination of a standard oblique projections have received much less attention in measurement followed by an orthogonal projection the signal processing literature. Recently, oblique pro- onto a lower space. jections have been applied to various detection prob- Drawing from the quantum mechanical POVM, as lems [70], [71], to signal and parameter estimation part of the QSP framework we also consider combined [72], to computation of wavelet transforms [73], and to QSP measurements. The QSP analogue of a quantum the formulation of consistent interpolation and sam- POVM is a ROM followed by a simple subspace QSP pling methods [20], [74]. measurement corresponding to an orthogonal projection In [20] the authors develop consistent reconstruction operator. Since the QSP framework does not depend on algorithms for sampled signals, in which the recon- the physics associated with quantum mechanics, we may structed signal is in general not equal to the original signal extend the notion of a (physically realizable) POVM to but nonetheless yields the same samples. Using a simple include other forms of combined QSP measurements, subspace QSP measurement corresponding to an oblique where we perform any two measurements successively. projection operator, in [23] and [24] the results of [20] As developed further in [1], such measurements lead to a are extended to a broader framework that can be applied variety of extensions and rich insights into frames, to new to arbitrary subspaces of an arbitrary Hilbert space, as classes of frames, and to the concept of oblique frame ex- well as arbitrary input signals. The algorithms developed pansions. This framework also leads to subspace MF de- yield perfect reconstruction for signals in a subspace of H, tectors and randomized algorithms for improving and consistent reconstruction for arbitrary signals, so that worst-case performance. this framework includes the more restrictive perfect re- construction theories as special cases. This framework leads to some new sampling theorems and can also be Combined Measurements and Tight Frames used to construct signals from a certain class with pre- Emulating the quantum POVM leads to combined mea- scribed properties [75]. For example, we can use this surements where a ROM is followed by an orthogonal framework to construct a finite-length signal with speci- projection onto a subspace U. Such measurements are fied low-pass coefficients or an odd-symmetric signal characterized by an effective set of measurement vectors. with specified local averages. In [80] we show that the family of possible effective mea- surement vectors in U is equal to the family of rank-one POVMs on U and is precisely the family of (normalized) Subspace Coding and Decoding tight frames for U. Subspace measurements also lead to interesting and po- Frames are generalizations of bases which lead to re- tentially useful coding and decoding methods for com- dundant signal expansions [81], [82]. A frame for a munication-based applications over a variety of channel Hilbert space U is a set of not necessarily linearly inde- models. In particular, in [1] a subspace approach for pendent vectors that spans U and has some additional transmitting information over a noisy channel is pro- properties. Frames were first introduced by Duffin and posed, in which the information is encoded in disjoint Schaeffer [81] in the context of nonharmonic Fourier se- subspaces. Todetect the information, we design a receiver ries and play an important role in the theory of nonuni- based on a subspace QSP measurement and show that for form sampling [81]-[83]. Recent interest in frames has a certain class of channel models this receiver implements been motivated in part by their utility in analyzing wave- a generalized likelihood ratio test. Although the discus- let expansions [84], [85]. A tight frame is a special case of sion constitutes a rather preliminary exploration of such a frame for which the reconstruction formula is particu- coding techniques, it represents an interesting and poten- larly simple and is reminiscent of an orthogonal basis ex- tially useful model for communication in many contexts. pansion, even though the frame vectors in the expansion In particular, decoding methods suggested by the QSP are linearly dependent. framework may prove useful in the context of recent ad- Exploiting the equivalence between tight frames and vances in multiple-antenna coding techniques [76], [77]. quantum POVMs, we develop frame-theoretic analogues of various quantum-mechanical concepts and results [80]. In particular, motivated by the construction of opti- Combined Measurements mal LS quantum measurements [32], we consider the An interesting class of measurements in quantum me- problem of constructing optimal LS tight frames for a chanics results from restricting measurements to a subspace U from a given set of vectors that span U. subspace in which the quantum system is known a pri- The problem of frame design has received relatively lit- ori to lie. This leads to the notion of generalized mea- tle attention in the frame literature. A popular frame con- surements, or positive operator-valued measures struction from a given set of vectors is the canonical frame (POVMs) [78], [79]. It can be shown that a general- [86]-[89], first proposed in the context of wavelets in ized measurement on a quantum system can be imple- [90]. The canonical frame is relatively simple to con- mented by performing a standard measurement on a struct, can be determined directly from the given vectors,

28 IEEE SIGNAL PROCESSING MAGAZINE NOVEMBER 2002 and plays an important role in wavelet theory [18], [91], [92]. In [80] we show that the canonical frame vectors Three principles of quantum are proportional to the LS frame vectors. This relationship between combined measurements mechanics that play a major and frames suggests an alternative definition of frames in role in QSP are the concept of terms of projections of a set of linearly independent sig- nals in a larger space. This perspective provides additional a measurement, the principle insights into frames and suggests a systematic approach of measurement consistency, for generating new classes of frames by changing the properties of the signals or changing the properties of the and the principle of projection. quantization of the measurement output. Geometrically Uniform Frames In the context of a single QSP measurement, we have seen that imposing inner product constraints on the measure- trarily, due to the extra degrees of freedom offered by the ment vectors of a ROM leads to interesting new process- use of frames that allow us to construct frames with pre- ing techniques. Similarly, in the context of combined scribed properties [84], [102]. Furthermore, if the mea- measurements imposing such constraints leads to the def- surements are quantized prior to reconstruction, then as inition of geometrically uniform frames [93]. This class of we show the average power of the reconstruction, error frames is highly structured, resulting in nice computa- using this redundant procedure can be reduced by as tional properties, and possesses strong symmetries that much as the redundancy of the frame in comparison with may be advantageous in a variety of applications such as the nonredundant procedure. channel coding [94]-[96] and multiple description source coding [97]. Further results regarding these frames are developed in [93]. Acknowledgments We are extremely grateful to Prof. G.D. Forney for his many thoughtful comments and valuable suggestions Consistent Sampling and that greatly influenced the formulation of the QSP Oblique Dual Frame Vectors We also explore extensions of frames that result from framework and many of the specific results that follow choosing an oblique projection operator onto U. In this from it. This work has been supported in part by the case the the measurement is described in terms of two Army Research Laboratory (ARL) Collaborative Tech- sets of effective measurement vectors, where the first nology Alliance through BAE Systems, Inc. subcontract set forms a frame for U and the second set forms what RK7854; BAE Systems Cooperative Agreement we define as an oblique dual frame. The frame operator RP6891 under Army Research Laboratory Grant corresponding to these vectors is referred to as an DAAD19-01-2-0008; BAE Systems, Inc. through Con- oblique dual frame operator and is a generalization of the tract RN5292; Texas Instruments through the TI Lead- well-known dual frame operator [86]. As we show in [23], these frame vectors have properties that are very similar to those of the conventional dual frame vectors. 100 CSMU Receiver However, in contrast with the dual frame vectors, they -1 Decorrelator are not constrained to lie in the same space as the origi- 10 MF MMSE nal frame vectors. Thus, using oblique dual frame vec- 10-2 tors we can extend the notion of a frame expansion to include redundant expansions in which the analysis 10-3 frame vectors and the synthesis frame vectors lie in dif- 10-4 ferent spaces. Based on the concept of oblique dual frame vectors, in 10-5

[23] we develop redundant consistent sampling proce- Probability of Error 10-6 dures with (almost) arbitrary sampling and reconstruc- tion spaces. By allowing for arbitrary spaces, the 10-7 sampling and reconstruction algorithms can be greatly 10-8 simplified in many cases with only a minor increase in ap- 0 2 4 6 8 101214161820 proximation error [20], [21], [98]-[101]. By using SNR [dB] oblique dual frame vectors, we can further simplify the sampling and reconstruction processes while still retain- s 12. Probability of bit error with ten users, ρ = 0.35, and accu- ing the flexibility of choosing the spaces almost arbi- rate power control, as a function of SNR.

NOVEMBER 2002 IEEE SIGNAL PROCESSING MAGAZINE 29 ership University Consortium; and by IBM through an [5] M.F. Barnsley and A.D. Sloan, “A better way to compress images,” BYTE, IBM Fellowship. vol. 1, no. 12, pp. 215-223, Jan. 1988. [6] T.L. Carroll and L.M. Pecora, “Synchronizing chaotic circuits,” IEEE Yonina C. Eldar received the B.Sc. degree in physics Trans. Circuits Syst., vol. 38, pp. 453-456, Apr. 1991. (summa cum laude) in 1995 and the B.Sc. degree in [7] L.M. Pecora and T.L. Carroll, “Driving systems with chaotic signals,” Phys. electrical engineering (summa cum laude) in 1996 both Rev. A, vol. 44, pp. 2374-2383, Aug. 1991. from Tel-Aviv University (TAU), Tel-Aviv, Israel, and [8] K.M. Cuomo, A.V. Oppenheim, and S.H. Strogatz, “Synchronization of the Ph.D. degree in electrical engineering and computer Lorenz-based chaotic circuits with applications to communications,” IEEE science in 2001 from the Massachusetts Institute of Trans. Circuits Syst. II, vol. 40, pp. 626-639, Oct. 1993.

Technology (MIT), Cambridge. From January 2002 to [9] “Special issue on applications of chaos in modern communications sys- July 2002 she was a postdoctoral fellow at the Digital tems,” IEEE Trans. Circuits Syst. I, vol. 48, Dec. 2001. Signal Processing Group at MIT. She is currently a se- [10] A.C. Singer, A.V. Oppenheim, and G.W. Wornell, “Detection and esti- nior lecturer in the Department of Electrical Engi- mation of multiplexed soliton signals,” IEEE Trans. Signal Processing, vol. neering at the Technion-Israel Institute of Technology, 47, pp. 2768-2782, Oct. 1999.

Haifa. Her current research interests are in the general [11] K.S. Tang, K.F. Man, S. Kwong, and Q. He, “Genetic algorithms and areas of quantum detection and signal processing. In their applications,” IEEE Signal Processing Mag., pp. 22-37, Nov. 1996. 1998 she held the Rosenblith Fellowship for study in [12] M. Pirlot and R.V.V. Vidal, “Simulated annealing: A tutorial,” Contr. electrical engineering at MIT, and in 2000 she held an Cybern., vol. 25, no. 1, pp. 9-31, 1996. IBM Research Fellowship. [13] S.S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed. Upper Saddle River, NJ: Prentice-Hall, 1999. Alan V.Oppenheim received the S.B. and S.M. degrees [14] H.W. Schüssler and P. Steffen, “Some advanced topics in filter design,” in in 1961 and the Sc.D. degree in 1964, all in electrical Advanced Topics in Signal Processing, J.S. Lim and A.V. Oppenheim, Eds. Englewood Cliffs, NJ: Prentice-Hall, 1988. engineering, from the Massachusetts Institute of Tech- nology (MIT). He is also the recipient of an honorary [15] J.G. Proakis, Digital Communications, 3rd ed. New York: McGraw-Hill, 1995. doctorate from Tel Aviv University in 1995. In 1964, he joined the faculty at MIT,where he currently is Ford [16] P.P. Vaidyanathan, Multirate Systems and Filter Banks. Englewood Cliffs, Professor of Engineering and a MacVicar Faculty Fel- NJ: Prentice-Hall, 1992. low. His research interests are in the general area of sig- [17] G. Strang and T. Nguyen, Wavelets and Filter Banks. Wellesley, MA: nal processing and its applications. He is coauthor of Wellesley-Cambridge, 1996. the textbooks Discrete-Time Signal Processing and Sig- [18] S.G. Mallat, “A theory of multiresolution signal decomposition: The nals and Systems. He is also editor of several advanced wavelet representation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 11, books on signal processing. He is a member of the Na- pp. 674-693, 1989. tional Academy of Engineering, a Fellow of the IEEE, [19] M. Vetterli and J. Kovacevic, Wavelets and Subband Coding. Englewood and a member of Sigma Xi and Eta Kappa Nu. He has Cliffs, NJ: Prentice-Hall, 1995. been a Guggenheim Fellow and a Sackler Fellow at Tel [20] M. Unser and A. Aldroubi, “A general sampling theory for nonideal ac- Aviv University. He has also received a number of quisition devices,” IEEE Trans. Signal Processing, vol. 42, pp. 2915-2925, awards for outstanding research and teaching includ- Nov. 1994. ing the IEEE Education Medal, the IEEE Centennial [21] M. Unser, “Sampling—50 years after Shannon,” Proc. IEEE, vol. 88, pp. Award, the Society Award, the Technical Achievement 569-587, Apr. 2000. Award, and the Senior Award of the IEEE Society on [22] Y.C. Eldar and A.V. Oppenheim, “Nonredundant and redundant sam- Acoustics, Speech and Signal Processing. He has also pling with arbitrary sampling and reconstruction spaces,” in Proc. 2001 received a number of awards at MIT for excellence in Workshop on Sampling Theory and Applicatations, SampTA’01, May 2001, teaching, including the Bose Award and the Everett pp. 229-234. Moore Baker Award. [23] Y.C. Eldar, “Sampling and reconstruction in arbitrary spaces and oblique dual frame vectors,” J. Fourier Analys. Appl., to be published.

[24] Y.C. Eldar, “Sampling without input constraints: Consistent reconstruc- References tion in arbitrary spaces,” Sampling, Wavelets and Tomography, A. Zayed and J.J. Benedetto, Eds., to be published. [1] Y.C. Eldar, “Quantum signal processing,” Ph.D. dissertation, Mass. Inst. Technol., Dec. 2001. Available: http://allegro.mit.edu/dspg/publica- [25] C.W. Helstrom, Quantum Detection and Estimation Theory. New York: tions/TechRep/index.html Academic, 1976.

[2] A. Fettweis, “Wave digital filters: Theory and practice,” Proc. IEEE, vol. 74, [26] D.G. Griffiths, Introduction to Quantum Mechanics. Upper Saddle River, pp. 270-316, Feb. 1986. NJ: Prentice-Hall, 1995.

[3] B.B. Mandelbrot, The Fractal Geometry of Nature. San Francisco, CA: Free- [27] A.S. Holevo, “Statistical decisions in quantum theory,” J. Multivar. Anal., man, 1982. vol. 3, pp. 337-394, Dec. 1973.

[4] G.W. Wornell and A.V. Oppenheim, “Wavelet-based representations for a [28] H.P. Yuen, R.S. Kennedy, and M. Lax, “Optimum testing of multiple hy- class of self similar signals with application to fractal modulation,” IEEE potheses in quantum detection theory,” IEEE Trans. Inform. Theory, vol. Trans. Inform. Theory, vol. 38, pp. 785-800, Mar. 1992. IT-21, pp. 125-134, Mar. 1975.

30 IEEE SIGNAL PROCESSING MAGAZINE NOVEMBER 2002 [29] Y.C. Eldar, A. Megretski, and G.C. Verghese, “Designing optimal quan- [52] Y.C. Eldar and A.V. Oppenheim, “Orthogonal multiuser detection,” Sig- tum detectors via semidefinite programming,” IEEE Trans. Inform. Theory, nal Processing, vol. 82, pp. 321-325, 2002. to be published. [53] Y.C. Eldar and A.M. Chan, “An optimal whitening approach to linear [30] M. Charbit, C. Bendjaballah, and C.W. Helstrom, “Cutoff rate for the multiuser detection,” submitted for publication. -ary PSK modulation channel with optimal quantum detection,” IEEE [54] Y. C. Eldar and A. V. Oppenheim, “Covariance shaping least-squares esti- Trans. Inform. Theory, vol. 35, pp. 1131-1133, Sept. 1989. mation,” IEEE Trans. Signal Processing, to be published. [31] M. Ban, K. Kurokawa, R. Momose, and O. Hirota, “Optimum measure - [55] T. Kailath, Lectures on Linear Least-Squares Estimation. Wein, NY: ments for discrimination among symmetric quantum states and parameter Springer, 1976. estimation,” Int. J. Theor. Phys., vol. 36, pp. 1269-1288, 1997. [56] S.M. Kay, Fundamentals of Statistical Signal Processing: Estimation Theory. [32] Y.C. Eldar and G.D. Forney, Jr., “On quantum detection and the Upper Saddle River, NJ: Prentice-Hall, 1993. square-root measurement,” IEEE Trans. Inform. Theory, vol. 47, pp. 858-872, Mar. 2001. [57] C.W. Therrien, Discrete Random Signals and Statistical Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, 1992. [33] Y.C. Eldar, A. Megretski, and G.C. Verghese, “Optimal detection of sym- metric mixed quantum states,” in preparation. [58] J.A. Cadzow, “Signal processing via least squares modeling,” IEEE ASSP Mag., pp. 12-31, Oct. 1990. [34] P. Hausladen and W.K. Wootters, “A ‘pretty good’ measurement for dis- tinguishing quantum states,” J. Mod. Opt., vol. 41, pp. 2385-2390, 1994. [59] A.E. Hoerl and R.W. Kennard, “Ridge regression: Biased estimation for nonorthogonal problems,” Technometrics, vol. 12, pp. 55-67, Feb. 1970. [35] H. Barnum and E. Knill, “Reversing quantum dynamics with near-opti- mal quantum and classical fidelity,” [Online]. Available: [60] A.N. Tikhonov and V.Y. Arsenin, Solution of Ill-Posed Problems. Washing- http://xxx.lanl.gov/abs/quant-ph/0004088 ton, DC: Winston, 1977. [61] L.S. Mayer and T.A. Willke, “On biased estimation in linear models,” [36] P. Hausladen, R. Josza, B. Schumacher, M. Westmoreland, and W.K. Technometrics, vol. 15, pp. 497-508, Aug. 1973. Wootters, “Classical information capacity of a quantum channel,” Phys. Rev. A, vol. 54, pp. 1869-1876, Sept. 1996. [62] C.R. Rao, “Minimum variance and the estimation of several parameters,” Proc. Cambridge Phil. Soc., pp. 280-283, 1946. [37] Y.C. Eldar, “Least-squares inner product shaping,” Linear Algebra Applicat., vol. 348, pp. 153-174, May 2002. [63] H.L. Van Trees, Detection, Estimation, and Modulation Theory, Part I. New York: Wiley, 1968. [38] K. Hoffman and R. Kunze, Linear Algebra, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1971. [64] S. Verdu, Multiuser Detection, Cambridge, U.K.: Cambridge Univ. Press, 1998. [39] A. Aldroubi, “Oblique projections in atomic spaces,” Proc. Amer. Math. Soc., vol. 124, no. 7, pp. 2051-2060, 1996. [65] R. Lupas and S. Verdu, “Linear multiuser detectors for synchronous code-division multiple-access channels,” IEEE Trans. Inform. Theory, vol. [40] S. Kayalar and H.L. Weinert, “Oblique projections: Formulas, algo- 35, pp. 123-136, Jan. 1989. rithms, and error bounds,” Math. Contr. Signals Syst., vol. 2, no. 1, pp. 33-45, 1989. [66] S. Ulukus and R.D. Yates, “Optimum multiuser detection is tractable for synchronous CDMA systems using m-sequences,” IEEE Commun. Lett., [41] S.P. Lipshitz, R.A. Wannamaker, and J. Vanderkooy, “Quantization and vol. 2, pp. 89-91, Apr. 1998. dither: A theoretical survey,” J. Audio Eng. Soc., vol. 40, no. 5, pp. 355-375, May 1992. [67] Y.C. Eldar and A.M. Chan, “On the asymptotic large system performance of the decorrelator,” IEEE Trans. Inform. Theory, to be published. [42] R.A. Wannamaker, S.P. Lipshitz, J. Vanderkooy, and J.N. Wright, “A theory of nonsubtractive dither,” IEEE Trans. Signal Processing, vol. 48, pp. [68] D.N.C. Tse and S.V. Hanly, “Linear multiuser receivers: Effective inter- 499-516, Feb. 2000. ference, effective bandwidth and user capacity,” IEEE Trans. Inform. Theory, vol. 45, pp. 641-657, Mar. 1999. [43] R.A. Wannamaker, “The theory of dithered quantization,” Ph.D. disserta- tion, Univ. Waterloo, Waterloo, Canada, 1997. [69] S. Verdu and S. Shamai (Shitz), “Spectral efficiency of CDMA with ran- dom spreading,” IEEE Trans. Inform. Theory, vol. 45, pp. 622-640, Mar. [44] R.M. Gray and T.G. Stockham, “Dithered quantizers,” IEEE Trans. In- 1999. form. Theory, vol. 39, pp. 805-812, May 1993. [70] R.T. Behrens and L.L. Scharf, “Signal processing applications of oblique [45] Y.C. Eldar, A.V. Oppenheim, and D. Egnor, “Orthogonal and projected projection operators,” IEEE Trans. Signal Processing, vol. 42, pp. orthogonal matched filter detection,” Signal Processing, June 2002, sub- 1413-1424, June 1994. mitted. [71] Y.C. Eldar, “On geometrical properties of the decorrelator,” IEEE [46] Y.C. Eldar and A.V. Oppenheim, “Orthogonal matched filter detection,” Commun. Lett., vol. 6, pp. 16-18, Jan. 2002. in Proc. Int. Conf. Acoust., Speech, Signal Processing (ICASSP-2001), Salt [72] R.T. Behrens and L.L. Scharf, “Parameter estimation in the presence of Lake City, UT, 2001, vol. 5, pp. 2837-2840. low-rank noise,” in Proc. 22nd Asilomar Conf. Signals Syst. Comput., Pacific [47] L.L. Scharf, Statistical Signal Processing: Detection, Estimation and Time Se- Grove, CA, Nov. 1988. ries Analysis. Reading, MA: Addison-Wesley, 1991. [73] M.J. Vrhel, C. Lee, and M. Unser, “Rapid computation of the continuous [48] T. Kailath and V. Poor, “Detection of stochastic processes,” IEEE Trans. wavelet transform by oblique projections,” IEEE Trans. Signal Processing, Inform. Theory, vol. 44, pp. 2230-2259, Oct. 1998. vol. 45, pp. 891-900, Apr. 1997.

[49] Y.C. Eldar and A.V. Oppenheim, “MMSE whitening and subspace whit- [74] C. Lee, M. Eden, and M. Unser, “High-quality image resizing using ening,” submitted for publication. oblique projection operators,” IEEE Trans. Image Processing, vol. 7, pp. 670-692, May 1998. [50] E.M. Friel and K.M. Pasala, “Direction finding with compensation for a near field scatterer,” in Proc. Int. Symp. Ant. Propagat. Soc., 1995, pp. [75] Y.C. Eldar, “Constructing signals with prescribed properties,” IEEE Signal 106-109. Processing Lett., to be published.

[51] R.J. Piechocki, N. Canagarajah, and J.P. McGeehan, “Improving the di- [76] B.M. Hochwald and W. Sweldens, “Space-time modulation for unknown rection-of-arrival resolution via double code filtering in WCDMA,” in Proc. fading,” in Digital Wireless Communication, R.M. Rao, S.A. Dianat, and First Int. Conf. 3G Mobile Commun. Technol., Mar. 2000, pp. 204-207. M.D. Zoltowski, Eds., SPIE Proc., vol. 3708, June 1999.

NOVEMBER 2002 IEEE SIGNAL PROCESSING MAGAZINE 31 [77] B.M. Hochwald and T.L. Marzetta, “Unitary space-time modulation for [92] M. Unser and A. Aldroubi, “Families of multiresolution and wavelet multiple-antenna communications in rayleigh flat fading,” IEEE Trans. In- spaces with optimal properties,” Numer. Funct. Anal. Optimiz., vol. 14, pp. form. Theory, vol. 46, pp. 543-564, 2000. 417-446, 1993.

[78] M.A. Neumark, “On a representation of additive operator set functions,” [93] Y.C. Eldar and H. Bölcskei, “Geometrically uniform frames,” IEEE Trans. C. R. (Doklady) Acad. Sci. URSS (N.S.), vol. 41, pp. 359-361, 1943. Inform. Theory, to be published.

[79] A. Peres, “Neumark’s theorem and quantum inseparability,” Found. Phys., [94] G.D. Forney, Jr., “Geometrically uniform codes,” IEEE Trans. Inform. vol. 20, no. 12, pp. 1441-1453, 1990. Theory, vol. 37, pp. 1241-1260, Sept. 1991.

[80] Y.C. Eldar and G.D. Forney, Jr., “Optimal tight frames and quantum [95] G. Ungerboeck, “Channel coding with multilevel/phase signals,” IEEE measurement,” IEEE Trans. Inform. Theory, vol. 48, pp. 599-610, Mar. Trans. Inform. Theory, vol. 28, pp. 55-67, 1982. 2002. [96] D. Raphaeli, “On multidimensional coded modulations having uniform [81] R.J. Duffin and A.C. Schaeffer, “A class of nonharmonic Fourier series,” error property for generalized decoding and flat-fading channels,” IEEE Trans. Amer. Math. Soc., vol. 72, pp. 314-366, 1952. Trans. Commun., vol. 46, pp. 34-40, 1998. [82] R.M. Young, An Introduction to Nonharmonic Fourier Series. New York: [97] V.K. Goyal, J. Kovavcevic, and J.A. Kelner, “Quantized frame expansions Academic, 1980. with erasures,” Appl. Comp. Harmonic Anal., vol. 10, pp. 203-233, May [83] J.J. Benedetto, “Irregular sampling and frames,” in Wavelets—A Tutorial 2001. in Theory and Applications, C.K. Chui, Ed. Boca Raton, FL: CRC, 1992, [98] M. Unser, “Splines: A perfect fit for signal and image processing,” IEEE pp. 445-507. Signal Processing Mag., pp. 22-38, Nov. 1999. [84] C.E. Heil and D.F. Walnut, “Continuous and discrete wavelet trans- forms,” SIAM Rev., vol. 31, no. 4, pp. 628-666, Dec. 1989. [99] M. Unser and J. Zerubia, “Generalized sampling: Stability and perfor- mance analysis,” IEEE Trans. Signal Processing, vol. 45, pp. 2941-2950, [85] I. Daubechies, “The wavelet transform, time-frequency localization and Dec. 1997. signal analysis,” IEEE Trans. Inform. Theory, vol. 36, pp. 961-1005, Sept. 1990. [100] T. Blu and M. Unser, “Quantitative Fourier analysis of approximation techniques: Part I—Interpolators and projectors,” IEEE Trans. Signal Pro- [86] I. Daubechies, Ten Lectures on Wavelets. Philadelphia, PA: SIAM, 1992. cessing, vol. 47, pp. 2783-2795, Oct. 1999. [87] H. Bölcskei, “Oversampled filter banks and predictive subband coders,” [101] T. Blu and M. Unser, “Quantitative Fourier analysis of approximation Ph.D. dissertation, Vienna Univ. Technol., Nov. 1997. techniques: Part II—Wavelets,” IEEE Trans. Signal Processing, vol. 47, pp. [88] H. Bölcskei and A.J.E.M. Janssen, “Gabor frames, unimodularity, and 2796-2806, Oct. 1999. window decay,” J. Fourier Analys. Applicat., vol. 6, no. 3, pp. 255-276, 2000. [102] A. Aldroubi, “Portraits of frames,” Proc. Amer. Math. Soc., vol. 123, pp. 1661-1668, 1995. [89] A.J.E.M. Janssen and H. Bölcskei, “Equivalence of two methods for con- structing tight Gabor frames,” IEEE Signal Processing Lett., vol. 7, pp. [103] J.J. Atick and A.N. Redlich, “Convergent algorithm for sensory receptive 79-82, Apr. 2000. filed development,” Neurak Comp., vol. 5, 1993.

[90] Y. Meyer, “Ondelettes et functions splines,” Dec. 1986, Seminaire EDP, [104] I.S. Gradshteyn and I.M. Ryzhik, Table of Integrals, Series and Products. Ecole Polytechn., Paris, France. New York: Academic, 1980.

[91] I. Daubechies, “Orthonormal bases of compactly supported wavelets,” [105] K.L. Chung, A Course in Probability Theory. San Diego, CA: Academic, Comm. Pure Appl. Math., pp. 909-996, 1988. 2001.

32 IEEE SIGNAL PROCESSING MAGAZINE NOVEMBER 2002