BGU - Computational Vision Course - Student Project Page

Image noise filtering using Neural Networks

Final project by

Arie Ohana, 33771635

[email protected] Introduction

Scanning images, photographing and other image related operations almost always involve the corruption of the output image due to the addition of noise. When related to imaging, noise is a high frequency random perturbation in the image pixels. You can think of noise as the subtle background hiss you hear at a stereo system at high volume.

There are many methods to achieve noise elimination and reduction and total elimination can rarely be found. Moreover, one of the biggest problems in noise reduction is that the more noise we would like to reduce, the blurrier the image becomes. Blurring an image as a way of reducing noise isn’t always such a good idea, a blurred image makes it hard to distinguish between small objects in the image, not to mention the uneasiness observing that image.

Noise comes in different flavors… Each one can be formulated and characterized. There is an additive noise, speckle and others. In this project we tried to reduce the Salt and Pepper noise only. Different types of noise will be reduced by this project's software in exactly the same method. Further discussion of this can be seen further in this paper.

This project offers to reduce image noise using a Neural Network. A neural network is a computing paradigm that is loosely modeled after cortical structures of the brain. It consists of interconnected processing elements called neurons that work together to produce an output function. The output of a neural network relies on the cooperation of the individual neurons within the network to operate. A given neural network achieves its goal by a learning process. When it is given a sufficient number of inputs and their desired outputs respectively, the network will adjust itself, by correcting the current weights on every input, according to a predefined formula. Running this process for a large enough number of iterations will make the network express some function that can map each input to its output, using the network's weights. The more examples this network is shown and the more versatile the examples are, the larger the network's ability is to generalize the problem. Neural Networks

A neural network, as explained earlier, models a function by a learning process. By introducing exemplars to the network, it can fine tune itself in an iterative process, so that eventually the network will converge into a steady state, and this steady state behavior is the function we want to model. By looking at the given input and its desired output, it's often hard to come up with a general function to map those input and outputs. Instead, the learning process of the neural network will "somehow" find a pretty close approximation to that unknown function.

There are several methods of building a neural network. The simplest is by using a single neuron, also called a perceptron, which models a single neuron in the brain. Its abilities, as you probably guessed, are therefore very limited, it can be used to model logic functions such as logical OR and logical AND. Any function that can be linearly separated can be implemented using a single neuron. A function that cannot be linearly separated needs a more sophisticated structure of the network. A more sophisticated structure means using more neurons, and more layers. Generally, we can implement every function using a network that has at least two hidden layers. A hidden layer is a sequence of neurons that don’t meet the input or output directly.

A perceptron as a basic neural network for implementing a linearly seperatable function

How many layers should the network consist of? How many neurons in each layer? What are the initial weights on the neurons' inputs? In order to answer those entire questions one must first estimate the difficulty of the problem at hand. For instance, filtering noise in an image is obviously not a simple task, its pretty innocent to assume that a single perceptron might implement the function. Moreover, it's safe to assume that we'll need at least to hidden layers since filtering is a complex task. The initial weights we usually use are randomly chosen small figures close to zero (usually in the interval [-1, 1]).

The correction of the weights in the learning process might be done in several methods. The one used in this application is the Backpropagation algorithm. The algorithm introduces an exemplar to the network, and while doing this, we measure the output on each neuron, not just the output layer. Using a few formulations we can calculate the correction delta that needs to be applied to each neuron. Those formulations are relatively simple and are based on progress of the input in the network until the output layer is met, and the correction is done in reverse – form output to input, hence the name Backpropagation. The formulations can be found in the literature. Approach and Method

The project consists of two parts. In the first part we learn our network to specialize at reducing noise. Then we receive a matrix of weights from this learning process which defines our network. The second part evolves processing a given input (a noisy image) with that matrix by letting the network perform its function but without learning (as opposed to the first part).

First stage / Learning

Learning (or training) the network to reduce noise is done by presenting the network a pixel P(corrupt, x, y) and all its 8 neighbors taken from the corrupted image and "telling" it that the desired output for that specific input is the pixel P(source, x, y). This is done for every pixel in every image in the training set (shown below). In our project a pixel means a value in the range of [0, 255].

The process begins with writing a general neural network. The network is a 10-50-1, i.e.:

1. The first layer is the input layer, the layer that will read 8 neighbors of a given pixel and the pixel itself from the noisy image, and the same coordinates pixel in the source image (10 neurons). 2. The second layer consists of 50 neurons that have 10 inputs each, form the previous layer. Experiments showed that using 50 neurons for that layer is sufficient. 3. The last layer is a single neuron that receives 50 inputs from the second layer and simply calculates a weighted sum on its inputs.

So the network layout is: Every neuron outputs its result through a sigmoid function. The sigmoid function has the form: f(x) = 1 / (1 + e^-2.7x), as often used in NN (neural networks). In the very same way we could use 5x5 or 7x7 as the neighbors instead of 3x3, which is used in this project. Using MATLAB, the addition of Salt and Pepper noise to a given image is as easy as calling the function imnoise, so we can easily produce a set of training images: a clean source image and a corrupted noisy image with a density of 0.05. So part of that training set was:

The trick is to use different images as the learning set, meaning images that are loosely related, and will represent a large percent of any image that can ever be presented to the network to filter. Moreover, the order of the inputs at the learning stage is also crucial, its highly recommended to show the network the training set but to give it a random pixel from a random image at every iteration, so the network wont converge too fast to a local optimum point, or ruining the fine-tuned weights achieved from previous learned images.

Second stage / Filtering an image

When learning is over, the network is ready to start filtering images. As mentioned, at first MATLAB was used to add noise to images so that the training set can be created. MATLAB also dumped that noisy image to a simple file containing those 9 pixels from the corrupted along with the desired pixel from the clean one. Then I used a java software to implement the network, and at the end the java program produced a matrix of weights representing the neural network. Then I used MATLAB again to receive a noisy image, the weights matrix from java and by simulating the neural network operation, MATLAB filtered the noisy image. The only service used directly from MATLAB was the imnoise function, all other operations were written as part of the project.

Since MATLAB has a demo for filtering noise from images, it was a good idea to incorporate the neural network algorithm into the existing demo so we make some observations and comparisons. Results

Running the network on images used in the MATLAB demo produced: Filtering an image using the neural network project in the same demo.

Some examples were run outside MATLAB, mainly to test some simple and dichotomy images, for instance:

A dichotomy noisy image Filtered image using NN Conclusions

General

It is pretty obvious that the existing methods for filtering Salt and Pepper noise are preferable than the one presented in this project. Although when presented a simple, dichotomies image the NN method seems to produce a pretty good result.

Results improvement

The neural network for filtering noise can be improved for better performance by:

1. Fine tuning the middle layer – adding or removing neurons, testing images and then deciding the optimized number of neurons. 2. Adding hidden layers – in this implementation, we have a single layer between the input and the output layers. Adding another layer or two might improve filtering dramatically. 3. Presenting the network a much larger and richer training set.

Analyzing the network

The method by which our network chose to filter a noisy image was simply to blur it a little and to brighten it. All the above examples show clearly the higher level of brightness the NN produces. In the example below we can observe that the NN was also blurring the image as a mean for filtering: When filtering is done right, we expect the distribution of the grayscale values of the source (cleared) image not to be that different from the distribution of that in the filtered image. A dichotomy image, much like those presented above, is filtered in a reasonable way when comparing to other type of images, since the network damaged the grayscale distribution less:

The histogram of a typical image Grayscale histogram of the image as (the guitar above). produced by the NN. The damage is pretty large.

The histogram of a dichotomy image (the bars above). The histogram the NN produced which very similar to the source.

The grayscale was normalized to be in the range of [0, 1]. The resulting histogram of the grayscale Filtering noise from an image containing values. dichotomy and gradient properties. Future work

Since noise filtering methods operate on pixels in the image that are not necessarily noise, they damage the image even in regions in the image that noise wasn't present in the source image. A neural network approach for filtering noise can improve that; we can build a neural network that specializes at recognizing noise. This kind of NN is to have impressive performance since its output is a binary value (noise / no- noise) and network as such can specialized faster and easier. Given an image to be filtered, the NN will show where the presence of noise is, and then we could apply some method for filtering noise (e.g. Averaging, Adaptive, NN or others) that will filter locally the noise. This way we avoid "filtering" noise that does not exist and by this we damage the image minimally. Of course, the NN can also be wrong here and then but the average percent of error is reasonably low when dealing with that kind of network (provided the NN is well trained, which is mainly an issue of time).

References

 Fundamentals of Neural Networks, Laurene Fausett  Building Neural Networks, David Skapura  A nonlinear filtering technique for digitized images degraded by film-grain noise, Warren Willman