Measuring the Signal-To-Noise Ratio in Magnetic Resonance Imaging: a Caveat� Deniz Erdogmusa;∗, Erik G

Signal Processing 84 (2004) 1035–1040 www.elsevier.com/locate/sigpro

Measuring the signal-to-noise ratio in magnetic resonance imaging: a caveat Deniz Erdogmusa;∗, Erik G. Larssona, Rui Yana, Jose C. Principea, Je/rey R. Fitzsimmonsb

aDepartment of Electrical and Computer Eng., University of Florida, Gainesville, FL 32611, USA bDepartment of Radiology, University of Florida, Gainesville, FL 32610, USA

Received 30 October 2003; received in revised form 10 February 2004

Abstract

The validity of the signal-to-noise ratio (SNR) as an objective quality measure for biomedical images has been the subject of a long-standing debate. Nevertheless, the SNR is the most popularly used measure both for assessing the quality of images and for evaluating the e/ectiveness of image enhancement and signal processing techniques. In this correspondence, we illustrate that under certain conditions the SNR can be changed by a nonlinear transformation, and also that it is often hard to measure objectively. Therefore, the issue is not only how well the SNR correlates with image quality as perceived by a human observer (which has been the primary subject of earlier debate), but also that SNR is questionable from a quantitative measurement point of view. ? 2004 Elsevier B.V. All rights reserved.

Keywords: Magnetic resonance imaging; Signal processing; Signal-to-noise ratio

1. Introduction for the study of other parts of the nervous system (suchas thespinal cord), as well as various joints, Magnetic resonance imaging (MRI) is a notable the thorax, the pelvis and the abdomen. Because of medical imaging technique that has proven to be par- the recent interest in signal processing for improving ticularly valuable for examination of the soft tissues the image quality, which is often quantiÿed by the in the body (such as the brain), and it has become estimated signal-to-noise ratio (SNR), it is imperative an instrumental tool for the diagnosis of stroke and to understand how this measure can be a/ected by other signiÿcant diseases as well as for pinpointing nonlinear signal processing operations. 1 the focus of diseases such as epilepsy [6]. It is also Most MR imaging scenarios are limited by the SNR considered to be an extremely important instrument in the reconstructed image. In particular, although it is

This work was partially supported by the NSF Grant 1 Although the physical ratio-of-amplitudes is commonly used ECS-0300340. to quantify SNR in the MRI literature, in this paper, we will ∗ Corresponding author. assume the engineering convention of ratio-of-powers. There is a E-mail address: [email protected] (D. Erdogmus). square-relationship between the former and the latter.

0165-1684/$ - see front matter ? 2004 Elsevier B.V. All rights reserved. doi:10.1016/j.sigpro.2004.03.006 1036 D. Erdogmus et al. / Signal Processing 84 (2004) 1035–1040 sometimes argued that improving SNR beyond 20 dB 2. The SNR is diagnostically not signiÿcant for static imaging (see, e.g., [3,7,15]), an improvement in SNR can always be As we illustrate in this section, the major drawback translated into an increase in acquisition speed and of the SNR as a quality measure is that it is not in- therefore be used to reduce the imaging cost and mo- variant to nonlinear transformations. Consider an ob- tion artifacts. Therefore, improving the SNR in MR servation model of the form images has become extremely critical for reducing mo- x = s + e; (1) tion artifacts and in applications where imaging speed is a major concern. Suchapplications include imaging where s is a signal of interest, and e is noise. We of dynamic processes, such as the heart [13]. Also, assume that both s and e are random variables. Also, since an improvement in SNR can signiÿcantly cut throughout this paper we assume for simplicity that all imaging times, it can increase the cost-e/ectiveness of signals and noise are real valued, 2 and that the noise MRI equipment in a hospital environment as well as is zero mean. 3 The SNR in x is given by decrease breath-holding durations and other discom- E{s2} forts for patients. SNR = ; (2) x E{e2} Evaluation of the quality of a real-world image is often a subjective task, and perhaps due to the absence where E{·} stands for statistical expectation. of more sophisticated indicators, the SNR appears to Let us consider the following nonlinear transforma- be one of the most popularly used measures of the tion of x: quality of an MR image. In general the SNR does ∞ f(k)(s)ek not measure bias errors (which are often signiÿcant), y = f(x)=f(s + e)=f(s)+ ; (3) k! and furthermore there is not always a clear correlation k=1 between the SNR and the image quality as visually where k! is the factorial of k and f(k)(x)isthekth perceived by a human observer, which is more related derivative of f(x), assuming that all derivatives of to the contrast in a broad sense (see, e.g., [14, Chapter f(x) are well deÿned. The SNR in y is equal to 7] for a discussion of visual image quality). E{f2(s)} In this communication we illustrate how SNR can SNR = y ∞ (k) k 2 be manipulated by nonlinear operations on the signals, E{( k=1 f (s)e =k!) } and that it is sometimes also diGcult to measure ob- E{f2(s)} jectively. The goal of this paper is, therefore, to em- ≈ ; (4) 2 phasize the fact that caution should be exercised when E{f (s)}E{e2} the SNR measured from an MR image is used as a where by convention f(s)=f(1)(s) and f2(s)= quality measure, or as an indicator for the improve- (f(s))2, and where the approximation is valid when ment o/ered by a signal processing algorithm that SNR 1. We conclude that SNR ¿ SNR exactly possibly employs a nonlinear operation at some stage x y x when of processing. Examples of suchalgorithmsinclude E{f2(s)} image enhancement procedures using median ÿlters ¿E{s2} (5) or nonlinear anisotropic ÿltering techniques (see, e.g., E{f2(s)} [4,16]). and therefore nonlinear transformations can improve In this study, we assume that a real-valued MR im- the SNR in a signal, provided that the function f(x) age is already obtained from the raw k-space data and and the statistical distribution of s are suchthat( 5) that necessary corrections to reduce phase distortions holds. may have been applied as discussed in [8,9,12]. How- ever, since the results on the distortion of SNR under nonlinear operations are true in general, similar ef- 2 All results extend to the complex case as well. 3 fects are expected to occur if nonlinear techniques are Note that if magnitude images are considered, the noise is not zero mean, but the analysis here could be extended to such employed when reconstructing images from the raw cases. See, e.g., [2,5,12] for more discussion on the statistics of k-space data. the noise in MR images. D. Erdogmus et al. / Signal Processing 84 (2004) 1035–1040 1037

In general, the conditions on f(x), under which (5) 3. Measuring the SNR holds, depend on the distribution of s. However, we can easily study a few special cases. If, for example, In the previous section we have seen that for signals f(x)=x2, then f(x)=2x and hence withcertain properties, a nonlinear transformation can change the SNR. Next, we discuss the diGculties as- SNR E{f(s)2} E{e2} E{s4} y = = : (6) sociated with measuring the SNR (see also [10]). Let 2 2 2 2 SNRx E{f (s)}E{e2} E{s } 4(E{s }) us, for simplicity consider a signal consisting of two regions, one area with N samples of a signal {s } We conclude that SNRy ¿ SNRx if and only if s s n of interest, and one region n consisting of Nn sam- 4 2 2 E{s } ¿ 4(E{s }) : (7) ples {en} that are known to be pure zero-mean noise, and which are independent of the signal. Hence, the For zero-mean random signals s,(7) holds exactly signal observed at a pixel n can be written as when sn + en;n∈ s; E{s4}−3(E{s2})2 x = (9) Ä(s)= ¿ 1; (8) n (E{s2})2 en;n∈ n: 2 2 where Ä(s)istheKurtosis of s. (For a Gaussian dis- The SNR in {xn} is SNRx = E{sn}=E{en}. tribution, Ä(s) = 0; distributions for which Ä(s) ¿ 0 For a given image, the SNR is usually estimated by are called super-Gaussian, and distributions for which using a moment-based estimator of the form Ä(s) ¡ 0 are called sub-Gaussian.) This means that 2 (1=Ns) xn SNR = n∈s ; (10) if the probability distribution of the image is highly x 2 (1=Nn) x super-Gaussian (most natural images are in this class), n∈n n i.e., denser around the mean and heavier at the tails where Ns and Nn are the numbers of pixels in the signal (e.g., Laplacian) then the square-operation (such as and the noise region, respectively. For a reasonably the one used in creating magnitude images) could de- high SNRx and for a large number of measured pixels, ceivingly demonstrate an improvement in SNR. we have that An interesting question is whether it is possible to 2 2 (1=Ns) n∈ xn E{(sn + en) } ÿnd a function f(x) suchthatSNR y ¿ SNRx regard- SNR = s ≈ x 2 2 (1=Nn) x E{e } less of the distribution of s. Without loss of generality, n∈n n n consider a unit-power signal s (i.e., E{s2} = 1). Then E{s2} + E{e2} from (5) SNR ¿ SNR if f2(s) ¿f2(s) for all s. = n n = SNR +1≈ SNR ; (11) y x 2 x x This can be achieved if we choose f(s)=as where a E{en} satisÿes 1=e ¡ |a| ¡e, because in that case |ln |a ¡ 1 where we used the assumption that the noise en has 2 2s 2 2s 2 which implies that f (s)=a ¿ (ln |a|) a =f (s). zero mean and is independent of sn (this equation was Hence, a nonlinearity in the form of an exponential discussed in more detail by Henkelman [8]). Note function withexponent a in the range 1=e ¡ |a| ¡e from (11) that the measured SNR is always larger than will improve the SNR for any s withunit power. An the true SNR. However, when SNRx is high, estimat- interesting consequence of this result concerns log– ing it via (10) in general gives reliable results. magnitude images. In some cases, to improve the con- We next discuss how the measured SNR can change trast and the dynamic range, a logarithmic nonlinearity when the signal {xn} is transformed via a quadratic might be applied to the constructed image. According nonlinear function. 4 For illustration purposes, we as- to the above analysis, we conclude that the original sume that s is constant (i.e., sn = s is a deterministic image (in linear scale) will have a higher SNR than the log-scale image, although the contrast of the latter is signiÿcantly better especially for low-signal power 4 The square-nonlinearity is assumed due to its simplicity and its regions. This demonstrates how insuGcient SNR is in common occurence in signal processing techniques and magnitude operations. However, similar analyses could be carried out for representing the visual clues that human perception other possible types of nonlinear operations encountered in the looks for when assessing image quality. processing. 1038 D. Erdogmus et al. / Signal Processing 84 (2004) 1035–1040 quantity) throughout s, and that we form a new signal signal, and the measured SNR in the transformed sig- {yn} according to nal (as deÿned via (13)), for some di/erent values of 1=2. The true SNR in the transformed signal y is 2 n yn = xn: (12) approximately 6 dB lower than the SNR in the origi- 2 From the analysis in Section 2 we know that for a nal signal xn, when 1= is high. (We can see that the measured SNR of x converges to the true SNR in this constant signal, SNRy ≈ SNRx=4 and hence the SNR n case; cf. (11).) This 6 dB di/erence in SNR between in {yn} is less than that in {xn}. (This is natural since 2 2 1 the sign, or the phase for complex data, is lost when yn and xn corresponds to the theoretical value of 4 de- the transformation (12) is applied.) Nevertheless, the scribed in Section 2. On the other hand, the measured SNR in {y } appears much larger than the true SNR, SNR in {yn},asmeasured via (10) can be muchlarger n which corroborates the ÿndings of Section 3. than the SNR measured from the original image {xn}. To understand why this is so, consider the measured Example 2 (Cat spinal cord): We analyze data from SNR in {yn}, assuming that SNRx1: a cat spinal cord using a 4:7 T MRI scanner (ob- 2 (1=Ns) n∈ yn tained withTR = 1000 ms ; TE = 15 ms, FOV = 10 × SNR = s y 2 (1=Nn) y 5cm; matrix = 120 × 120, slice thickness = 2 mm, n∈n n sweep width= 26 khz, 1 average) [ 1]. The data col- 2 2 2 (1=Ns) [s +2sen + en] lected from a phased array of four coils is combined = n∈s 4 (1=Nn) n∈ en using the sum-of-squares (SoS) technique to yield a n 5 reconstructed image. Let yk be the observed pixel 4 (1=Ns) s value from coil k: ≈ n∈s : (13) 4 (1=Nn) en n∈n yk = ck + nk ;k=1; 2; 3; 4; (14) 2 This expression essentially behaves as (SNRx) . where is the (real-valued) object density (viz. the Therefore, we expect the measured SNR in {yn} to MR contrast), ck is the (complex-valued) sensitivity be much larger than it actually is; i.e., the squaring in associated withcoil k for the image voxel under con- (12) makes the signal appear to an observer as if it sideration, and nk is zero-mean complex-valued noise. were muchless noisy. The SoS reconstruction for this voxel is obtained via 6 4 2 ˆ = |yk | : (15) 4. Illustration k=1 We consider two di/erent nonlinear operations on We provide two examples to illustrate the phenom- the SoS reconstruction: natural logarithm and me- ena discussed in the previous sections. In the ÿrst ex- dian ÿltering (MF). The former nonlinear operation ample, we consider a simulated constant signal em- simply generates a new image by modifying the bedded in zero-mean Gaussian noise. In the second pixel-by-pixel values by applying the log function. example, we use real MRI data from a cat spinal cord. The latter one is a standard nonlinear image processing technique that is robust to outliers, which is often Example 1 (Step function in noise): We consider a used to improve SNR. In median ÿltering, eachpixel signal consisting of two segments and , during s n value is simply replaced by the median of the values which the signal level is equal to 10 and 0, respec- of its neighboring pixels (here we use a 5 × 5 region tively, embedded in white Gaussian noise with vari- centered at the pixel of interest). ance 2. The signal along with its noisy version are shown in Fig. 1(a) for 2 =1. In Fig. 1(b) we show the signal after the nonlinear transformation (12). Finally, 5 It is known that the SNR of the SoS reconstructed image in Fig. 1(c) we show the true SNR for the original asymptotically approaches that of the best linear unbiased reconstruction image [11]. signal, the measured SNR for the original signal (as 6 The image voxel value estimate ˆ corresponds to the signal deÿned via (10)), the true SNR for the transformed x in the arguments presented in the previous sections. D. Erdogmus et al. / Signal Processing 84 (2004) 1035–1040 1039

14 180

12 160

10 140

8 120

6 100

4 80 original signal 2 transformed signal 60

0 40

−2 20

−4 0 0 100 200 300 400 500 600 700 800 0 100 200 300 400 500 600 700 800 (a)t (b) t

60 true SNR in the original signal:x n measured SNR in the original signal:x 50 n true SNR in the transformed signal:y n measured SNR in the transformed signal:y 40 n

SNR(dB) 20

−10 0 1 2 3 10 10 10 10 (c) 1/σ2

Fig. 1. Synthetic data example. (a) Original noisy step function signal xn, (b) transformed (squared) signal yn, and (c) the true and the measured SNR levels.

In Fig. 2 we present the images provided by SoS, make the SNR appear higher or lower in a manner log-SoS, and MF-SoS as well as their corresponding that is uncorrelated with the perceived image quality. local SNR estimates using the reference noise region Therefore the SNR must be used with careful judge- shown in the upper-right corner. Observe that although ment as a quality measure when evaluating images the MF-SoS image exhibits an improved SNR com- processed by certain signal processing procedures. In pared to the original SoS, the image actually looks this communication we have illustrated how general worse. On the other hand, the SNR of the log-SoS im- nonlinear operations on the (magnitude) images could age decreased, yet the dynamic range and the signal alter the experimentally measured SNR. An especially contrast in low-power regions are improved. alarming observation is that for certain nonlinear operations the true SNR increases, whereas the experimentally measured SNR decreases (and vice versa). 5. Concluding remarks Another problem that observed is that the SNR may be improved at the cost of a decrease in perceived SNR is a popular measure of quality in phased-array quality. MRI reconstruction. Nevertheless, it is hard to mea- Our observations call for an extended debate on sure objectively in a practical set-up. It is also easy objective quality measures that are invariant (at least to manipulate, because nonlinear transformations can to some extent) to nonlinear operations on the signals. 1040 D. Erdogmus et al. / Signal Processing 84 (2004) 1035–1040

References

[1] B.L. Beck, S.J. Blackband, Phased array imaging on a 4.7T/33cm animal researchsystem, Rev. Sci. Instrum. 72 (11) (2001) 4292–4294. [2] C.D. Constantinides, E. Atalar, E.R. McVeigh, Signal- to-noise measurements in magnitude images from NMR phased arrays, Magn. Reson. Med. 38 (1997) 852–857. [3] W.A. Edelstein, G.H. Glover, C.J. Hardy, R.W. Redington, The intrinsic signal-to-noise ratio in NMR imaging, Magn. Reson. Med. 3 (1986) 604–618. [4] G. Gerig, O. Kubler, R. Kikinis, F.A. Jolesz, Nonlinear anisotropic ÿltering of MRI data. IEEE Trans. Medical Imaging 11 (2) (1992) 221–232. [5] H. Gudbjartsson, S. Patz, The Rician distribution of noisy MRI data, Magn. Reson. Med. 34 (1995) 910–914. [6] E.M. Haacke, R.W. Brown, M.R. Thompson, R. Venkatesan, Magnetic Resonance Imaging—Physical Principles and Sequence Design, Wiley, New York, 1999. [7] C.E. Hayes, P.B. Roemer, Noise correlations in data simultaneously acquired from multiple surface coil arrays, Magn. Reson. Med. 16 (1990) 181–191. [8] R.M. Henkelman, Measurement of signal intensities in the presence of noise in MR images, Med. Phys. 12 (1985) 232 –233. [9] R.M. Henkelman, M.J. Bronskill, Artifacts in magnetic resonance imaging, Reviews of Magn. Reson. Med. 2 (1987) 1–126. [10] L. Kaufman, D.M. Kramer, L.E. Crooks, D.A. Ortendahl, Measuring signal-to-noise ratios in MR imaging, Radiology 173 (1989) 265–267. [11] E.G. Larsson, D. Erdogmus, R. Yan, J.C. Principe, J.R. Fitzsimmons, SNR-optimality of sum-of-squares reconstruction for phased-array magnetic resonance imaging, J. Magn. Reson. 163 (1) (2003) 121–123. [12] G. McGibney, M.R. Smith, An unbiased signal-to-noise ratio measure for magnetic resonance images, Med. Phys. 20 (4) (1992) 1077–1078. [13] E. McVeigh, C. Ozturk, Imaging myocardial strain, IEEE Signal Process. Mag. Nov (2001) 44–56. [14] W.K. Pratt, Digital Image Processing, Wiley, New York, 1978. Fig. 2. Reconstruction images and their SNR performance. (a) [15] P.B. Roemer, W.A. Edelstein, C.E. Hayes, S.P. Souza, O.M. SoS, (b) SNR of SoS, (c) logarithm of SoS, (d) SNR of logarithm Mueller, The NMR phased array, Magn. Reson. Med. 16 of SoS, (e) median ÿltered SoS, (f) SNR of median ÿltered SoS. (1990) 192–225. [16] M.A. Schulze, Biomedical image processing with morphology-based nonlinear ÿlters, Ph.D. Dissertation, The Althoughwedo not propose sucha new quality mea- University of Texas at Austin, 1994. sure here, we should note that one possible class of [17] Q. Zhao, J.C. Principe, J. Fitzsimmons, M. Bradley, such measures here, we should note that one possi- P. Lang, Functional magnetic resonance imaging data analysis ble class of suchmeasures includes quantities derived with information-theoretic approaches, in: P.M. Pardalos, J.C. Principe (Eds.), Biocomputing, Kluwer Academic from information theory (see, e.g., [17] for ideas along Publishers, Dordrecht, 2002, pp. 159–173. these lines).