<p> FPGA IMPLEMENTATION OF ADAPTIVE MEDIAN FILTER FOR IMAGE IMPULSE NOISE SUPPRESSION</p><p>ABSTRACT This paper introduces, a new Index Terms-Field-programmable gate intelligent hardware module suitable for the arrays (FPGAs), impulse noise, median computation of an adaptive median filter is filter, real-time filtering. presented for the first time. The function of Introduction the proposed circuit is to detect the Two applications of great existence of impulse noise in an image importance in the area of image processing neighborhood and apply the operator of the are noise filtering and image enhancement. median filter only when it is necessary. These tasks are an essential part of any The noise detection procedure can be image processor whether the final image is controlled so that a range of pixel values is utilized for visual interpretation or for considered as impulse noise. In this way, automatic analysis. The aim of noise the blurring of the image in process is filtering is to eliminate noise and its effects avoided, and the integrity of edge and on the original image, while corrupting the detail information is preserved. image as little as possible. To this end, Experimental results with real images nonlinear techniques (like the median and, demonstrate the improved performance. in general, order statistics filters) have The proposed digital hardware structure been found to provide more satisfactory is capable to process gray-scale images results in comparison to linear methods. of 8-bit resolution and is fully pipelined, For this reason, a number of non-linear whereas parallel processing is used in order filters, which utilize correlation among to minimize computational time. In the vectors using various distance measures. presented design, a 3x3 or 5x5 pixel image However, these approaches are typically neighborhood can be selected for the implemented uniformly across an image, computation of the filter output. However, and they also tend to modify pixels that are the system can be easily expanded to undisturbed by noise, at the expense of accommodate windows of larger sizes. The blurred and distorted features. proposed digital structure was designed, In this paper, an intelligent compiled and simulated using the hardware structure of an adaptive median Modelsim and Synthesized in Xilinx filter (AMF) suitable for impulse noise VERTEX-11. For the implementation of suppression for gray-scale images is the system the PF10K200SRC240-1 field- presented for the first time. The function of programmable gate array device of the the proposed circuit is to detect first the FLEX10KE device family is utilized, and it existence of noise in the image window and can be used in industrial imaging apply the corresponding median filter only applications where fast processing is when necessary. The noise detection required. The typical clock frequency is 65 level procedure can be controlled so that MHz. a range of pixel values (and not only the Fig. 1: Block diagram of the adaptive filtering method fixed values 0 and 255, but also salt-and- device family suitable for designs that pepper noise) is considered as impulse require high densities and high I/O noise. The main advantage of this adaptive count, is utilized. The total number of the approach is that the blurring of the image in system inputs and outputs are 44 and eight process is avoided and the integrity of edge pins, respectively and the percentage of the and detail information is preserved. logic cells utilized is (40 inputs for the Moreover, the utilization of the median input data and four in- puts for the clock filter is done in a more efficient way. and the control signals required) and the Experimental results demonstrate the percentage of the logic cells utilized is improved performance of the AMF. The 99%. The typical clock frequency is 65 proposed digital hardware structure is MHz and the system can be used for real- capable of processing gray-scale images time imaging applications where fast of 8-bit resolution and performs both processing is of the utmost importance. As positive and negative impulse noise an example, the time required to perform removal. A moving window of a 3x3 and filtering of a grayscale image of 260x244 5x5 pixel image neighborhood can be pixels is approximately 7.6 ms. selected. Furthermore, the system is Adaptive median filter design directly expandable to accommodate larger The most common method used for size gray-scale images. The architecture impulse noise suppression for gray-scale chosen is based on a sequence of four basic and color images is the median filter. functional pipelined stages and parallel Impulse noise exists in many practical processing is used within each stage. The applications and can be generated by proposed structure was implemented various sources, including many man-made using field-programmable gate arrays phenomena such as unprotected switches, (FPGAs) , which offer an attractive industrial machines, and car ignition combination of low-cost, high- systems. Images are often corrupted by performance, and apparent flexibility. The impulse noise due to a noisy sensor or presented digital circuit was designed, channel transmission errors. This type of compiled and simulated using the noise is the classical salt-and-pepper noise MAX + PLUS II Programmable Logic for grayscale images. Development System by Altera The output of a median filter at a Corporation. For the realization of the point x of an image f depends on the values system the EPF10K200SRC240-1 FPGA of the image points in the neighborhood of device of the FLEX10KE device family, a x. This neighborhood is determined by a window W that is located at point x of f and an be modified. The result is the including n points x1, x2,….., xn of f. the threshold value for the detection of a noise median can be determined when the pixel, denoted as fthreshold (x) and is limited to number of points included in W is odd i.e., a positive (negative) integer threshold when n = 2k+1. the n values f(x1), f(x2),…, value. f(xn) of the n points x1, x2, ….., xn are placed 3) The value of the central pixel is in ascending order forming the set of compared to fthreshold (x), and the central ordered values {f1, f2,….., fn}, in which f1 ≤ pixel is considered to be noise, when its f2 ≤.…, ≤ fn. The median is defined as he value is greater (less) than the threshold (k+1)th value of the set {f1, f2,….,fn}, med = value fthreshold (x). fk+1. 4) When the central pixel is considered to The basic disadvantage of the be noise, it is substituted by the median application of the median filter is the value of the neighborhood, fk+1 , blurring of the image in process. In the which is the normal operation of the general case, the filter is applied median filter. In the opposite case, the uniformly across an image, modifying value of the central pixel is not altered and pixels that are not contaminated by the procedure is repeated for the next noise. In this way, the pixel values of the neighborhood window. input image are altered, leading thus to an A block diagram of the adaptive overall degradation of the image and to filtering procedure previously described is blurred or distorted features. depicted in Fig. 1. An example of the The proposed adaptive median filter application of the noise detection algorithm can be utilized for impulse noise for the cases of impulse and signal suppression for gray-scale images. Its dependent noise is illustrated in Fig. 2(a) function is to detect the existence of noise and (b), respectively. In Fig. 2(a), a typical in the image window and apply the 3x3 pixel neighborhood window of a gray- corresponding median filter only when scale image is depicted. The central pixel of necessary. The noise detection scheme for the window occupies an extreme value the case of positive (negative) noise is as (pixel value = 252) compared to the values follows. of the surrounding image points (pixel 1) For a neighborhood window W that is values ranging from 32 to 42). For this located at point x of the image f, the reason, the central pixel is considered to be maximum (minimum) pixel value of the n- impulse noise and it should be eliminated. 1 surrounding points of the neighborhood is In the specific example, the noise detection computed, denoted as fmax (x) (fmin(x)), scheme is applied as follows. In the first excluding the value of the central pixel at step, we find the maximum value of the point x. surrounding pixels, fmax (x) = 42. If we consider the parameter α = 5, then threshold value fthreshold (x) = fmax (x) * 5 = 210. The central pixel value is 252 > fthreshold (x) and, therefore, is considered to be a noisy pixel. Finally, it is substituted by the median value of the neighborhood. The same discussion applies to the example of Fig. 2(b), which is the case of signal- dependent noise. The same procedure is Fig. 2: Noise detection algorithm (a) Impulse followed, and the noisy pixel is noise (b) Signal-dependent noise successfully detected for a parameter value 2) The value fmax (x) (fmin(x)) is multiplied α = 2. by a parameter α which is a real number</p><p>Fig. 3: Impulse noise removal. (a) Original image “Café.” (b) Image corrupted by 5% positive and negative impulse noise. (c) Median filter result using 3x3 pixel window. (d) AMF result using 3x3 pixel window. (e) Median filter result using 5x5 pixel window. (f) AMF result using 5x5 pixel window.</p><p>Two major remarks about the adjustable positive threshold value (for presented adaptive algorithm should be example 240) can be used as a limit of made. First, the value of the parameter α is fthreshold (x). In this way, fthreshold (x) = 240, of great importance, since it controls whereas the central pixel value is 252 > the operation of the circuit and the fthreshold (x), and the central pixel is result of the overall procedure for successfully detected as impulse noise. The different noise cases. Second, an meaning of this normalization procedure is appropriate positive and negative that pixels occupying values between a threshold value must be utilized for the range of the impulsive values (and not only case of impulse noise, when fthreshold (x) ≥ pixels with values 0 and 255) should be 255. For example, in the case of Fig. 2(a), considered as noisy pixels. if we consider the parameter α = 8, then Hardware architecture fthreshold (x) = fmax (x) * 8 = 336 and the fthreshold The proposed architecture is based (x) is limited to the value 255, fthreshold (x) = on a sequence of pipeline stages in order to 255. The central pixel value is 252 < fthreshold reduce computational time. Parallel (x) and the central pixel is erroneously not processing has been employed to further considered to be impulse noise. An accelerate the process. For the computation of the filter output, a 3x3 or image neighborhood, only 3 or 5 pixel 5x5 pixel image neighborhood can be values stored in the memory are altered. selected. The outputs of this unit are rows of pixel The structure of the adaptive filter values (3 or 5, respectively), which are the comprises four basic functional units, the inputs to the median computation unit. moving window unit, the median Median computation unit computation unit, the noise detection unit, In this stage, the median value of and the output selection unit. The input the image neighborhood is computed in data of the system are the gray-scale values order to substitute the central pixel value, if of the pixels of the image neighborhood, necessary. In this unit, the min/max value the value of the parameter α, and the of the neighborhood is also computed, used positive and negative threshold values. in the noise detection process. For the Additionally, two control signals required computation of both the median and the for the selection of the operation of the min/max value a 24-input sorter is utilized, system (negative/positive noise the central pixel value is not included. In suppression) and the neighborhood size this way, the complexity of the design is (3x3 or 5x5) are also utilized. reduced since no additional min/max Moving window unit modules are utilized. The modules of the The pixel values of the input image, sorter used only in the case of the 5x5 pixel denoted as “IMAGE_INPUT [7..0],” are neighborhood are enabled by the “en5x5” imported into this unit in serial. The value control signal. A CS block is a max/min of the parameter is denoted as module; its first output is the maximum of “MOD_VALUE[7..0]” and the positive and the inputs and its second output the negative threshold values as POS/NEG minimum. The implementation of a CS THRESHOLD respectively. The parameter block includes a comparator and two α is a real number, 5 and 3 bits are used for multiplexers and is depicted in Fig. 4. the representation of the integral and the The outputs of the sorter, denoted as fractional part, respectively. The “OUT_0[7..0]”...“OUT_23[7..0],”produce “NEG/POS” control signal is used to a “sorted list” of the 24 initial pixel values. determine the noise type. When The output OUT_0[7..0] is the minimum “NEG/POS” is equal to “0” (“1”) the pixel value for both 3x3 and 5x5 pixel circuit operation is negative (positive) noise image window. The sorter outputs suppression. “OUT_3[7..0]” and “OUT_4[7..0]” and the For the moving window central pixel value are utilized for the operation, a 3x3 (5x5) pixel serpentine type computation of the median value for the memory is used, consisting of 9 (25) 3x3 pixel neighborhood, denoted as registers, illustrated in Fig. 6. In this way, “MEDIAN_3x3 [7..0].” when the window is moved into the next</p><p>Fig. 4: Implementation of CS block Fig. 5: Hardware structure of the figure denoted. An additional 2-to-1 multiplexer is utilized to select the positive or negative threshold to which the THRESHOLD_VALUE should be normalized, controlled by the “NEG/POS” control signal. A comparator is used to compare the THRESHOLD_VALUE to the positive or negative threshold and a multiplexer to select the corresponding output threshold value, denoted as THRESHOLD Fig. 6: Schematic diagram of a 3x3 pixel Output selection unit serpentine memory The final stage of the design is Noise detection unit the output selection unit. In this unit, The task of the noise detection unit is the appropriate output value for the to compute the threshold value for the performed operation is selected. For the detection of a noise pixel, fthreshold (x), and to selection of the output value the limit this value to the positive (negative) corresponding threshold value for the threshold. Initially, the min/max value of image neighborhood, “THRESHOLD,” is the neighborhood is selected, and for that used. The value “THRESHOLD ” is reason, the values “OUT_0[7..0],” compared to the central pixel value, “OUT_7[7..0]” and “OUT_23[7..0]” (min denoted in the circuit as and max values, respectively) are “CENTRAL_PIXEL.” imported into a multiplexer. The selection Depending on the result of the is based on the values of the “NEG/POS” comparison, the central pixel is considered control signals. In the next step, the output to be contaminated by noise or not. For the of the multiplexer is multiplied by the case of positive (negative) noise, if the parameter α (8 bits) using a multiplier central pixel value is greater (less) than the module, the resultant 16-bit value is corresponding threshold value, then the </p><p>Fig.7: Complete simulation results for the circuit</p><p> central pixel is positive (negative) noise and has to be eliminated. Floor plan placement of the top module The “FILTER_OUTPUT” is the output of this the adaptive filter. Note the way that the input data is converted to 3-pixel rows every three clock pulses, and that for a sliding of the window to the next image neighborhood only three pixel values of the memory are changed. The modification of the system to accommodate windows of larger sizes is done in a straightforward way, requiring only a small number of changes. More specifically, in the first unit, the size of the serpentine memory used for the serial input of the image data and the corresponding number of multiplexers increase following a square law. In the second unit, the sorter module should be modified, whereas no changes are required in the last two units. Chip View of Top Module</p><p>RTL View</p><p>Synthesized top level report Device Utilization for 2V80cs144 successfully simulated in FPGAs.The *********************************** typical system clock frequency is 65 MHz. Resource Used Avail Utilization REFERENCES ------[1] W. K. Pratt, Digital Image Processing. IOs 19 92 20.65% New York: Wiley, 1991. Function Generators 608 1024 [2] G. R. Arce, N. C. Gallagher, and T. 59.38% Nodes, “Median filters: Theory and CLB Slices 304 512 59.38% applications,” in Advances in Dffs or Latches 161 1300 12.38% Computer Vision and Image ------Processing, T. Huang, Ed. Using wire table: xcv2-80-6_wc Greenwich, CT: JAI, 1986. Clock Frequency Report [3] T. A. Nodes and N. C. Gallagher Jr., Clock : Frequency “The output distribution of median type ------filters,” IEEE Trans. Commun., vol. clk : 44.6 MHz COM-32, pp. 532–541, May 1984. [4] T. Sun and Y. Neuvo, “Detail- Conclusion preserving median based filters in This paper presents a new design of image processing,” Pattern Recognit. an adaptive median filter, which is capable Lett., vol. 15, pp. 341–347, Apr. 1994. of performing impulse noise suppression [5] E. Abreau, M. Lightstone, S. K. Mitra, for 8-bit grayscale images using a 3x3 or and K. Arakawa, “A new effici-cient 5x5 pixel neighborhood. The proposed approach for the removal of impulse circuit detects the existence of noise in noise from highly corrupted images,” the image neighborhood and applies the IEEE Trans. Image Processing, vol. 5, corresponding median filter only when it is pp. 1012–1025, June 1996. necessary. The noise detection procedure is [6] K. S. Ali, “Digital circuit design using controllable, and, thus, pixel values other FPGAs,” in Proc. 19th ICCIE, 1996, than the two extreme ones can be pp. 127–129. considered as impulse noise, provided that [7] K. E. Batcher, “Sorting networks they are significantly different from the and their applications,” in Proc. central pixel value. In this way, the blurring AFIPS-SJCC, 1968, pp. 307–314. of the image is avoided. [8] I. Pitas and A.N. Venetsanopoulos The system is suitable for real-time Nonlinear Digital Filters: Principles imaging applications where fast processing and Applications. Boston, is required. Moreover, the design of the MA: Kluwer, 1990. circuit can be easily modified to [9] R.M Hodgson, D.G. Bailey, M. J. accommodate larger size windows. In this Naylor, A.L. M. Ng, and S.J. McCneil, case, only small modifications are required “Properties, Implementations and in the first two units, mostly regarding the applications of rank filters”, Image Vis. size of the serpentine memory and the Comput., vol.3,pp. 3-14,1985. sorter module. [10] E. R. Dougherty and P. Laplante, The proposed digital hardware Introduction to Real time Imaging. structure was designed, compiled and Bellingham, WA:SPIE,1995.</p>