3. Solving Backlight Image Problem with Fuzzy Logic and Compensation Curve
3.4. Backlight Image Compensation with a Compensation Curve
3.4.4. Turning Point or Inflection Point Searching
To determine the inflection (or Turning) point, let us observe the histograms of two representative backlight images shown in Figure 3.8 (a) and (b). There are apparently two groups in Figure 3.8 (a). The first group (group A) has lower brightness, and the second group (group B) has higher brightness. We can utilize this characteristic to get the turning point. At first, we calculate the average values of group A and group B, called Lm and Hm, respectively. We use the following steps to obtain the values of Lm and Hm.
Step 1: Using Gaussian smoothing filter to smooth the histogram of the whole image.
Step 2: Calculating a series
{ }
j L 10{
0, , ..., , ..., 1 j L 1}
a j− a a a a −
= = and series b by the
following equations:
{
( )} {
| |,| |,...,| |}
, (3.12)where TH is a threshold which is around 0.0013 according to our experimental study.
Step 3: According to series a and b to obtain the T set, T =
{
T T1, , , 2 T3 ...}
, which contains the start and end points of group A and group B:(3.13)
1 maximum intensity value of image.
Step 4: Calculating the values of Lm and Hm by T set. respectively. Next, we will utilize the BF index obtained in the backlight detection phase to determine the final Lm value, called FLm, by
FLm = Lm + Lm*B F.
From equation 3.16, we discovered that the FLm will become 2Lm. Due to Lm
being average intensity value of darkness part in the image histogram, meanwhile, and people will be regard as darkness when intensity is less than 60 according human vision property, the probability of 2Lm greater than Hm was low or impossible. This value FLm represents the highest grey value of dark area in the image histogram. We shall stretch this value to Hm such that the brightness of the dark area in the image will be enhanced. To do this, we let the y coordinate of the inflection (turning) point of the compensation curve be the value Hm and the x coordinate be the value FLm, i.e., the inflection (turning) point (x, y) = (FLm, Hm ). This also decides the shape of the whole compensation curve according to the different degree of backlight.
Figure 3.8 (b) shows the histogram of another representative backlight image.
There are three groups in this histogram, representing low brightness, median (3.15) (3.14)
(3.16)
brightness, and high brightness areas in the image, respectively. The median and high brightness areas correspond to the background of the image, and the low brightness area corresponds to the backlight part of the image. Also from figure 3.8 (b), we discover that the histogram distribution among median brightness area and high brightness area is uniform. Thus, we should stretch the backlight part of the image from the low brightness area to the median brightness area. This action could enhance intensity of low brightness area and keep the intensity between median and high brightness area. Hence, by identifying the low brightness area as group A, and the median brightness area as group B, the procedure proposed before based on the histogram of Figure 3.8 (a) can be applied here to determine the inflection (turning) point (x, y) of the compensation curve. After getting the point, the intensity of main object in the backlight image will be enhanced.
(a) (b)
Figure 3.8 Histograms of two representative backlight images.
3.5. Experimental Results and Discussions
The performance of the proposed backlight detection and compensation scheme is tested in this section. All the experiments were performed on a personal computer with Pentium III 700. The testing image base consists of 100 backlight images. Figure 3.9 (a) to (d) show some typical images in our testing image database. These images were captured by the FinePix 6800Zoom of Fujifilm camera and the resolution of each image is 640*480 pixels. In the followings, we shall demonstrate the
FLm Hm
B A
FLm Hm
A B
performance of the proposed scheme based on the four images in Figure 3.9 (a) to (d).
Performance comparisons are also made by applying the method in [26] on the same images. In [26], the author use fuzzy logic method to infer a compensation value C.
For inferring this value, they define four fuzzy indices extracted from backlight image.
These indices are average luminance of a screen (V), Importance of the background (BI), Backlight degree (B), and Excessive frontlighting degree (EF). The four fuzzy indices will produce 21 rules and their final compensation algorithm is represented as follows:
Luminance difference=Average luminance of a image-reference value
Luminance of the pixel after compensation= Luminance of the pixel before compensation-luminance difference
Figure 3.9 (a) to (d) show four original backlight images, where the first two images were captured indoor and the rest two were captured outdoor. We applied our method and the method in [26], respectively, on these four images to obtain the corresponding compensated images in Figure 3.9 (e) to (l). Figure 3.9 (e) to (h) show the compensated images by our method. Figure 3.9 (i) to (l) show the compensated images by the method proposed by M. Murakami and N. Honda in [26]. In [26], a proper fixed gray value was added to each brightness value of the whole image for brightness compensation. We noticed that the over-saturation problems occurred in Figure 3.9 (i) to (l). In these compensated images, although the brightness of the backlight part in the image has been enhanced, the brightness of the background part became too high and some detailed information was lost. For example, let’s observe Figure 3.9 (b), (f) and (j). In figure 3.9 (b), the original backlight image has the lowest brightness in the central object part, but has normal brightness in the background part.
In the compensated image in Figure 3.9 (f), the brightness of the backlight part is
enhanced properly. But in the compensated image in Figure 3.9 (j), the brightness of the backlight part is obviously saturated. Therefore, the luminance in the background of this image seems unnatural to human vision perception. The results clearly demonstrate the superiority of the proposed backlight compensation technique.
Subsequently, we compare our method to the histogram equalization method. Figure 3.10 (a) to (d) are four original backlight images and Figure 3.10 (e) to (h) show the compensated images by the histogram equalization method. These processed images appear to be over-compensated. Namely, it will produce an effect of intensity saturation in some areas. This defeat is appeared because of the process attempted to merge the adjacent gray levels together in order to flatten the histogram.Hence, the simple histogram equalization method is not suitable for solving the problems of backlight images. Finally, we also compare with an alternative method. The method is to segment the backlight image and transform the intensity of each region separately.
A general, the method of segment adopts simply image thresholding method. After getting a binarization image, it will do compensation for each region. The result is shown on Figure 3.11. We discovered that it will produce the following problems. The first, the algorithm needs to individually enhance each region. Thus, it will make a lightness inconsistency for whole image. The second, because the algorithm needs to determine the adjustment region and value corresponding to the region, it will become complex.
Figure 3.12 (a) to (d) show four original backlight images, where the first two images were captured indoor and the rest two were captured outdoor. We applied our new method and the old method in [23], respectively, on these four images to obtain the corresponding compensated images in Figure 3.12 (e) to (l).They have different compensation method and the same searching method of turning or inflection point.
The old method employs two parabolic curves to achieve compensation action. At the same turning (inflection) point, we discover that these images compensated by the new compensation method have a better effect in lightness enhancement than old method. Figure 3.12 (e) to (h) show the compensated images by our old method proposed in [23]. Figure 3.12 (i) to (l) show the compensated images by our new method.
Through foregoing experiment, we can get two important results. First, the parameters used in our proposed method such as TH and T2 etc. were obtained by a large amount of experiments, and they were fixed in our system in all the experiments.
These values are quite suitable for the general backlight images processing problems.
And, the small deviations on these values will not affect the final processed results through some more experiments. Second, we use many kinds of levels of backlight images for experiments, but none of them fails or even no bad conditions have happened. This is because we compensate only for the main object with scanty of lightness, we do not destroy the original information of the object with the normal lightness. Thus for the images after processing, the brightness of main object will be enhanced, and the remained section will also not be destroyed.
Figure 3.9 (a) to (d) are original images, (e) to (h) are the compensated images by our method, and (i) to (l) are the compensated images by the method in [27].
(a) (e)
(c)
(f) (b)
(k) (j) (i)
(g)
(d) (h) (l)
Figure 3.10 (a) to (d) are four original backlight images, (e) to (h) show the compensated images by our proposed method, and (i) to (l) show the compensated images by the histogram equalization method.
(a) (e)
(b) (f)
(c) (g)
(d) (h)
(i)
(j)
(k)
(l)
Figure 3.11The column (a) shows the original images, Column (b) shows the segmented images, Column (c) shows the compensated images by the segmentation method, and Column (d) shows the compensated images by our proposed method.
(a) (b) (c) (d)
Figure 3.12 (a) to (d) are original images, (e) to (h) are the compensated images obtained by our old method in Ref. [23], and (i) to (l) are the compensated images by our new method.
(a) (e)
(c)
(f) (b)
(k) (j) (i)
(g)
(d) (h) (l)
3.6. Concluding Remarks
We proposed a new algorithm for detection and compensation of backlight images in this chapter. We derived a new backlight index through a fuzzy inference mechanism to represent the backlight degree of an image. The brightness of the backlight image was then compensated according to a compensation curve determined by the backlight index adaptively. The proposed algorithm was tested on 100 backlight images, which contained diverse environments with various backlight degrees. Experimental results and performance comparisons clearly indicated the superiority of the proposed scheme, especially in solving the over-saturation problems existed in the current backlight compensation approaches. We now focus on integrating the proposed backlight detection and compensation scheme with the solutions of other automatic exposure problems into one intelligent algorithm for applications in real-world products such as digital still cameras and digital camcorder.