Illumination estimation of real images for color recovery

In the second experiment, we test the performance of the new algorithm on real images under natural illumination. These images are captured from Olympus DC460 still digital camera under different color temperatures. For proper control of different color temperatures, these images are captured inside a Color-Viewing (CV) box (made by GAIN Associates Inc., No. D1729). In the CV box, there are four different light sources, D 、 F 、 A and UV, producing four different color temperatures (D=6500°K, F=4200°K, A=2850°K). When capturing the images inside the CV box, we have to turn off the auto-white-balancing function in the digital camera such that the proposed approach can be fully exploited. To apply the proposed illumination estimation scheme to the captured images, we have to find the matrix that transforms the camera R, G, and B signals into CIE 1931 X, Y, and Z values at first. The 3× 3 matrix is constructed so that the final CIE’s XYZ mean square errors between the reflection print and the camera image are minimized under the constraint that the neutral colors will be kept neutral after the color transformation. Let us take the Olympus DC-460 still camera as an example. Its chromaticity transformation between the phosphor primaries and the CIE’s XYZ primaries are listed in Table 5-5. Using the relations developed for camera calibration [76], [77], we have



,

The transformation matrix A= cVDU⁻¹ is determined up to a constant factor c. The convention is to choose the constant c so that y is equal to 1 when r, g, and b are set to 1. By doing so, we obtain the transformation matrix A as



Therefore, we have



Table 5-5. Camera primaries (r, g, and b are normalized by the maximum stimulus).

Stimulus r g b x y z

Red phosphor 1.0 0.0 0.0 0.6237 0.3313 0.0450 Green phosphor 0.0 1.0 0.0 0.2640 0.6044 0.1316 Blue phosphor 0.0 0.0 1.0 0.2033 0.0892 0.7075

White 1/3 1/3 1/3 0.3501 0.3693 0.2806

Table 5-6. The average RMS errors of AgfaIT8.7/2 photographic print for the image rerendered based on different illuminant estimation algorithms.

Average RMS

Error Our Scheme Gray-World Max-RGB

Error (RMS) 5.53 8.39 10.38

Next, we can obtain the raw data of CIE’s XYZ of images by using the above transformation matrix A. The chromaticity histogram is computed at the same time.

Finally, we use our approach to estimate the surrounding illumination. By substituting the estimation result into Eq. (5-10), we can obtain the color-corrected images for the desired surrounding illuminant. Figs. 5-5~5-7 show the exemplar images in this experiment. For each scene, we show four images: (a) the raw image captured by the camera with different illuminants, (b) the rendered image using the illuminant estimated by the proposed new algorithm, (c) the rendered image using the illuminant estimated by the Gray-World algorithm, and (d) the rendered image using the illuminant estimated by the Max-RGB algorithm. Besides, in Fig. 5-5, we add a raw image captured by the camera with D65 illuminants. Fig. 5-5(a) shows the AgfaIT8.7/2 color chart captured under the F illumination, and Figs. 5-5(b)~(d) are the three images rendered for illuminant D65 by the proposed, Gray-World, and Max-RGB algorithms, respectively. It is obvious that the rendered image based on the surrounding illuminant estimation of the proposed algorithm is very close to the image captured under the D65 illumination directly. In contrast, the performance of the Gray-World and Max-RGB algorithms is worse. The quantitative performance measurement is listed in Table 5-6. Table 5-6 shows the average RMS errors of colors in all the charts between the images captured under the D65 illumination and those rendered by different illumination estimation schemes. Fig. 5-6 shows the results on the image of the Macbeth ColorChecker captured under the D50 and D75 illuminants, respectively. Fig. 5-7 is the photo scene of the campus of our university. There is a dominant color caused by the grassplot on the photo. The results show that the proposed algorithm is less sensitive to the dominant color than the other two compared algorithms.

(a) (b)

(c) (d)

(e)

Figure 5-5. (a) Raw camera image captured under the F illumination. (b)-(d) Color-corrected images for the D65 illumination by the proposed algorithm, the Gray-World algorithm, and the Max-RGB algorithm, respectively. (e) Raw camera image captured under the D65 illumination.

(a) (b) (c) (d) Figure 5-6. (a) Raw camera image captured under D50 (first row) and D75 (second

row) illuminations. (b)-(d) Color-corrected images for the D65 illumination by the proposed algorithm, the Gray-World algorithm, and the Max-RGB algorithm, respectively.

(a) (b) (c) (d) Figure 5-7. (a) Raw camera image captured under daylight and evening

illuminations with a dominant color of grassplot. (b)-(d) Color-corrected images for the D65.illumination by the proposed algorithm, the Gray-World algorithm, and the Max-RGB algorithm, respectively.

5.5. Concluding Remarks

In this chapter, a new approach to surrounding illumination estimation of an image was proposed. The proposed algorithm estimated the illuminant based on chromaticity histogram of the image. And a neural network with back-propagation (BP) learning algorithm was used to estimate the spectral power distribution of the illuminant according to the center values of the chromaticity histogram. The proposed algorithm also eliminated the interference of dominant color to illumination estimation through low-pass filtering of the chromaticity histogram. The illumination estimation based on the chromaticity histogram can avoid unrealistic assumptions on the color images and provide high efficient and robust estimation. On the other hand, the use of the BP network provides good interpolation over a small number of different illuminants and gives highly estimation accuracy. Two experiments were performed to evaluate the performance of the proposed algorithm. In the first experiment, the proposed algorithm was used to estimate the illumination of synthetic images, and the estimation RMS errors were calculated. In the second experiments, the proposed algorithm was used to estimate the spectral power distributions of the illuminants, and then the colors of the image were corrected based on the finite-dimensional linear model of surface reflectance. Two popular existing illumination estimation algorithms, the Gray-World and Max-RGB algorithms, were also applied to the same images in these two experiments. Performance comparisons have demonstrated the superiority of the proposed algorithm both in estimation accuracy and robustness for color constancy.