Chapter 2 Origination of Hot Spot Noise
2.2 Template Noise
Since in the real world, we can only acquire the contaminated image that corrupted by hot spot like noise, it is impossible for us to figure out the performance of our designing from comparing original clean image and the processed image. To solve this problem, we construct a model to generate image noise that resembles the hot spot like noise and the algorithm is produced by observing the hot spot like noise in the corrupted images as the following steps [10].
1) First we inspect the hot pixel typical pattern model contained in a real world image, the blob nearly contains 3 â 2 pixels and hence a 3 â 2 noise mask has been chosen to be the hot pixel blob in an image.
2) Then we choose a window size of 3 â 3 blocks, each contains a sub-block blob of size 3 â 2 pixels. Let the blob containing the hot pixel to be in the center block of the window, there are 54 pixels in the extended window.
3) Using the column vector of 3 â 2 hot pixel blob as the center of an extended window of size 9 â 6 pixels, calculate the average u of this extended window by
u =Px(iàs,jàt)N ì
ìì(s,t) ∈ pixel in the extended window (1)
where W is a window of size 9 â 6 pixels. Therefore, the noise pixels blob is n(i, j) = x(i, j) à u(i, j)|(i, j) ∈ pixels in the hot pixel blob, and there are six pixels in the hot pixel blob.
4) After having the simplest noise model, we add the noise blob into the original picture. Noise n(i, j) is add to an original image f(i, j) [14] to generate an image f1(i, j) corrupted by hot pixel noise by
f1(i, j) = f(i, j) + n(i, j) (2)
5) From the steps above, the noisy images were originated from the original images.
Although we have already added the above noise in the template image, the contaminated image is still a little different from the ideal noise model. The noise pixel n(i, j) is needed to be adjusted for producing more closely replica of the true hot pixel noise. We observe that the hot pixel noise is somewhat proportional to the gray level of the local value of the pixel. To reflect this, we add the inversely factors pu(i, j)
and the random number ra(i, j) to the noise more eminent in the dark area than usual. By this setting, if u(i, j) is small, then u(i,j)
Following the steps, the hot pixel template noise can be easily added to a known image. Thus, we can have both the noisy and original images to assess the noise reduction performance in term of the quantification measures. An original image and its hot pixel noisy images are shown in Fig. 2.2 and Fig. 2.3. Fig. 2.2 is the original
image “Lena.” Fig. 2.3 is the noisy image with six mixed hot pixel noise patterns.
Fig. 2.2. Example of original image “Lena.”
Fig. 2.3. Zoomed “Lena” with six mixed hot pixel noise patterns.
Chapter 3 Decision-Based General Vector Directional Filter for Hot Pixel Removal
The conventional filters for the image processing are mostly directly applying in the spatial domain by operating a specific or a set of functions individually on the independent channel of the given image. For the single channel image processing, different spatial filter has different effect respectively, but they are not good enough for the multi-channel image processing [11]. Separately processing on each channel and then reconstruct the multi-channel image seems to be less of consistency and fail to utilize the inherent correlation that is usually presented in multi-channel images.
The general vector directional filter (GVDF) [11]-[13] provides a better approach than the spatial filters, those with inherent drawback, and considers the multi-channel image as a pack of vectors so as to take directional processing and magnitude processing into account. With this separation, GVDF can achieve good filtering on the color image, multi-channel image, for various noise source models.
3.1 General Vector Directional Filter
GVDF processes the input image in the concept of vector. A vector is formed in the color space [11], [15], and [16] by three-color components of an input pixel of the processing image, thus vector direction and magnitude are generated. GVDF can be divided into two components, the first component is the directional processing and the other is magnitude processing. In the first component of GVDF filter, the distance criterion is adopted by the angle difference and the stage aim in this step is to eliminate the atypical directions in the operating vectors. In this thesis, the introduced
directional processing is applying by a general vector directional filter, and its definition is as below [11].
Definition: The output of the generalized vector directional filter (GVDF), for input fi, i = 1, 2, ..., n implies the same ordering to the corresponding fis
f(1)6 f(2)6 ... 6 f(k)6 ... 6 f(n).
The first
k
terms of the ordered sequence f(i) constitute the output of the GVDF.GVDF outputs set of vectors whose angle difference from others in one processing are smaller. It trims out the unwanted or the more contaminated-likely pixel that particularly has atypical vector direction to the most of the other vectors. In other words, the output set of GVDF in the mask filtering can preserve the chromatic and intensity tendency in the local processing region since chromaticity and intensity for a color vector are highly correlated to the vector’s direction and its magnitude in the color cube. Indeed, chromatic difference between two color vectors is affected by the angle and the magnitude difference of the corresponding two color vectors.
Furthermore magnitude difference causes the intensity disparity. Thus, in this component, we want to ensure the output pixel’s chromaticity is not going too far away from its neighborhood.
In the second component of GVDF filter, the output of magnitude processing is obtained from specific gray-scale filter such as: the ë–trimmed mean [16], [17], the morphological open-close [18], and the multistage max/median [19]. Here the gray-scale median filter [16], [20] is introduced on the processing image. From this filter, we can appropriately substitute the possible outlier pixel with one that has more centered intensity in the output of component one , and that prevent from miss employing the substitute by selecting the pixel with more extreme intensity.
3.2 Decision-based Vector Directional Filter
Generally speaking, GVDF has good performance on the multi-channel image processing. When it is applied on the purpose of removing hot spot noise in the image processing, however, the output performance is barely satisfied. Moreover, it possibly changes too much details to retrieve the corrupted image. That is to say GVDF fails to preserve the most pixels in the original input image and it violates the original intention of our filter design. Consequently, we develop a decision-based vector directional filter (DBVDF) to enhance the performance of the GVDF.
The algorithm of DBVDF involves 2 criterions and 5 stages. Two criterions for hot spot noise detection comes from the concept of using information of four neighborhood adjoin pixels, as shown in Fig. 3.1., to determine whether the processing pixel is to filter or not. Decisions made in these two criterions are
responsible for fast processing and high detection rate of proposed filter. The parameters in these two criterions are obtained from observing and summarizing.
Besides, three of five stages have thresholds as decision. The 3rd stage is applied GVDF only, though the trimming length in the directional processing of GVDF is also a selected decision too. The whole DBVDF procedure is simply represented in the flowchart, as shown in Fig. 3.2. Thereafter, we will interpret the contents stage by stage explicitly.
Two Criterions:
Our proposed algorithm exploits low computation characteristic of two criterion to achieve the fast filtering and high noise detection rate. The definitions of Matrix Cross, Matrix Cros12, Matrix Cross23 , Matrix Cross1234 and ò0, all in the intensity sense, help to clearly describe the algorithm of adopted criterion in the first two components of DBVDF. It is noticed that the principal content in the criterion 1 is aiming at fast hot spot noise detection. The parameters in the description are experienced and experimental results. In addition, it is unnecessary to adjust these parameters for different processing images since the performances change slightly. In the criterion 2, there are four situations to comment. Different numbers of contaminated-likely neighborhood needs to be considered respectively. Indeed, the criterion 1 owns more severe restriction than criterion.
Definition:
b) x(i,j)>=1.2âCross12 AND Cross(3)>=220 AND Cross(4)>=220 c) Cross(2)>=210 AND Cross(3)>=210 AND Cross(4)>=210 d) ò0>=1.35 AND Not{Cross(1)>=210 AND Cross(2)>=210 AND
Cross(3)>=210 AND Cross(4)>=210}
Stage 1:
In the region of our 3 â 3 moving processing mask, the intensity of every pixel is not supposed to vary too much because of the characteristic of nature light [1]. That is to say that intensities in limited-area local region seem to be somewhat continuous.
Thus, we use the sum of intensity difference as a criterion to restrict the filtering so that the processing area with little intensity variation will be bypassed. It’s not only decreasing our image processing time but preserving the most uncontaminated part of the original input. Consequently, the output resembles the image without noise much more and the rate of correction gains high.
The threshold picked in this stage arises from the contents in the previous listed reasons. Considering a 3 â 3 moving processing mask as in Fig. 3.3, decision
Over chromaticity
Fig. 3.2. Flow chart of decision-based vector directional filter.
Bypass
restrains as ò1 [21], if ò1 is greater than the threshold picked in this stag, then the proposed decision mechanism is meant to regard the processing pixel as an outlier one.
Stage 2:
As the assumption in the stage 1, the color difference in the neighborhood of
moving processing mask is supposed not to alter abruptly. In these two stages, we are expected to process comparatively normal contaminated-images with no abnormal intensity and color difference bounces. Once we filter a particular input that has a lot of edges in its detail, however, so long as the threshold selected in this stage be enlarged enough to prevent from modifying the complex color variation regions (usually refer to edges) as far as possible. Since contaminated pixels mostly are discontinuous in color with its neighbor clean pixels and usually have a great jump in every dimension of 3 color components with uncontaminated neighborhood, this stage employs the characteristic of the hot pixel noise to decide whether the moving mask is
x1 x2 x3
x4 x5 x6
x7 x8 x9
Fig. 3.3. 3 â 3 moving window mask with every entry indexed in stage 1.
going to be processing or just neglect the filtering in this pixel. This step saves lots of processing time again and even preserves the originality of the most pixels of input image further.
Considering a 3 â3 moving processing mask as in Fig. 3.2, decision restrain is selected by ò2, if ò2 is greater than the threshold picked in this stage, then the proposed decision mechanism is meant to regard the processing pixel as an outlier one.
Both of the stage 1 and stage 2 are highly related to the performing time cost. It is because that they determine whether the processing pixel is to be filtering or not.
By these two stage’s criterions, we can save the a large quantity of processing time cause proceeds on filtering the uncontaminated-likely pixels with the GVDF in the next stage would takes a huge amount of time for its vast complexity of
computations.
Stage 3:
Every pixel in the image can be seen as a vector as in Fig. 3.4. On account of color characteristic, we introduce the GVDF as a main filter in whole. The GVDF utilizes its principal viewpoint based on the color space concept and this is more close to the fundamental property in color image processing than any other filters. Although
GVDF costs considerably long time, it still has an outstanding performance in the multi-channels signal process. That’s the reason why we use the GVDF as a main filter [11] in the entire process. What is more in our design, the filter is appended the restrictions previously in this stage like those in stage 1 and stage 2, thus we can decrease the processing time effectively by the way of adequately choosing the more possible contaminated-likely pixels to handle.
The strategy here is to adopt the 3 â3 moving mask too while signal length criterion should be take into consideration so that the output signal length of GVDF is not varying far from the adjoining pixels of input. In the algorithm of GVDF, we take half numbers of input signals (that is in general) to be the trimming target. Therefore, four input pixels that deviate from the other five inputs are passed over because they are seemed to be noise-like ones.
Stage 4:
In fact, the GVDF uses a cascade of directional processing and magnitude processing that is shown in Fig. 3.5. As we describe at the previous chapter, three
Fig. 3.4. Perspective representation of the color cube.
filters have been used for the step of magnitude processing: the ë-trimmed mean [16], [17] the morphological open-close [18], and the multistage max/median [19].
However we do not expect to replace the processing pixel directly with the output of vector directional filter (VDF) [11] or output of GVDF cascaded with one of these three different kinds of filters. Instead of those filters, the mean filter in the intensity sense is adopted in the step of magnitude processing. Nevertheless it is just only a reference so that we can make decision of what pixel is selected to replace the current processing contaminated-likely pixel. In other words, decision in this stage is that we substitute the contaminated-like pixel (judged via our algorithm) by the one whose intensity is more probably closed to the original uncontaminated pixel, of course, they are all in the moving mask. It really makes the retrieval pixel with intensity more continuous nearby the processing region.
The substitution is implemented in all the three color channels, thus, the output turns to be more excellent and more continuous with neighborhood in the color sense than that directly processed by GVDF and cascaded ordinary filter.
Stage 5:
In order to restore the most uncontaminated pixel of the input image, the last criterion is to let the output of our design much more resemble the original without too much artificial alteration. It is compensating for the condition once our decision goes wrong that results in filtering the clean pixel. We set a threshold ò to achieve
this goal in this stage. First we duplicate a copy of original input image. If the chromatic difference (absolute value) of the processing pixel in the original contaminated image and the output of stage 4 is smaller than ò3, then we do not modify the local processing pixel in the copy image. In case of the difference is greater than the threshold, we substitute the processing pixel in the duplicated image by the output of stage 4. In other words, we try to preserve the most original appearance in the input image again. The threshold should not be set too low so that the image contains too much artificial modification and most of that often are somehow dissimilar from the original. On the other hand, the duplicated copy may be remain unchanged for the most part as input image if the threshold is set too high so that we lose to filter out the hot spot like noise.
To summarize our design, we develop a decision-based GVDF applying on multi-channel image processing for removing the hot pixel like noise. The GVDF with a proper decision criterion is good because it would considerably reduce the required time for processing and attain a more excellent performance than that is implemented by GVDF only. Consequently we design a five-stage decision-based GVDF that considers both the chromaticity and the intensity content. In the simulation section, we will discuss its advantage by some assessment criterion.
Fig. 3.5. Multi-channel image processing using a cascade of directional processing and magnitude processing.
Chapter 4 Simulation Results
We will show the efficiency of the DBVDF comparing against the GVMF with respect to the processing time. Moreover, quantitative measures in terms of the performance of each filter have also been provided. Of course, the experimental results in images will be shown as well.
4.1 Image with Synthetic Hot Pixel Noise
4.1.1 Experimental Results of the Decision-Based Vectorized Median Filtering
The best threshold of the decision-based Vectorized Median Filter (DBVMF) has been found around 1.40. The experimental results for the commonly adopted images
“Airplane,＂ “Lena,＂ “Peppers,＂ “Sailboat,＂ with the threshold ò=1.40 from the thesis [10] published in 2003 are shown in Fig. 4.1. We show the zoomed images so that the details could be revealed. Obviously, it can be seen that some hot spot noise remains in the filtered image and the sharpness of the image is lost over the whole image. That is, the DBVMF fails to filter all the hot spot noise in the corrupted image and diverts the attention of the viewer from the subject of the filtered image.
(a)
(b)
(c)
(d)
Fig. 4.1. (a) Zoomed “Airplane＂ filtered by the decision-based VMF with
ò= 1.40. (b) Zoomed “Lena＂ filtered by the decision-based VMF with ò= 1.40. (c) Zoomed “Peppers＂ filtered by the decision-based VMF with ò= 1.40 . (d) Zoomed“Sailboat＂ filtered by the decision-based VMF with ò= 1.40.
4.1.2 DCE AutoEnhance and Hot Pixels Eliminator
DCE AutoEnhance and Hot Pixels Eliminator [22], [23] which are famous web sites concerning hot pixel reduction in the world wide web. The algorithms of these two methods are not available but executable programs are provided in the web. It has been proved that the decision-based VMF outperforms both of these two methods. Fig.
4.2 shows the filtered Airplane, Lena, Peppers, and Sailboat images from DCE AutoEnhance and Hot Pixels Eliminator, respectively. Again, we show the zoomed images for the purpose of clearly observing the texture and color contents in the filtered images. If we compare the images filtered by these two methods with the input processing images, we can find that the color and luster alter too much in the filtered images that we might think they possibly are not the results of the processing image. The filtered images looks like being shot under different light sources or in which the contents are appeared to be in different hues and saturations. Despite the two methods filtering out nearly all the hot spot noise, they are barely satisfied for the severe drawback of losing nature color of original input processing images.
Although DBVMF, DCE AutoEnhance, and Hot Pixels Eliminator have their defect respectively, they all possess the characteristic of low processing time which is the ultimate drawback of the VDF and DBVDF.
(a)
(b)
(c)
(d)
Fig. 4.2. (a) Zoomed “Airplane＂ filtered by the DCE AutoEnhance filter. (b) Zoomed “Airplane＂ filtered by the HotPixels Eliminator filter. (c) Zoomed“Lena＂ filtered by the DCE AutoEnhance filter. (d) Zoomed “Lena＂
filtered by the HotPixels Eliminator filter.
4.1.3 Generalized Vector Directional Filter and Decision-Based Vector Directional Filter
Fig. 4.3 and Fig. 4.4 show the experimental results via GVDF and DBVDF respectively. We may see that images filtered by DBVDF has effectively reduced the hot spot noise, and better preserve the texture of the original image than GVDF. The output images of DBVDF are more realistic because of the fundamental characteristic of the DBVDF algorithm. And it might be easily perceived since images filtered by DBVDF are more natural in coloring comparing to those filtered by GVDF. Although
Fig. 4.3 and Fig. 4.4 show the experimental results via GVDF and DBVDF respectively. We may see that images filtered by DBVDF has effectively reduced the hot spot noise, and better preserve the texture of the original image than GVDF. The output images of DBVDF are more realistic because of the fundamental characteristic of the DBVDF algorithm. And it might be easily perceived since images filtered by DBVDF are more natural in coloring comparing to those filtered by GVDF. Although