Illumination estimation of synthetic images

In this experiment, we used a simple feedforward neural network with two input and two output nodes to approximate the nonlinear mapping between the center of chromaticity histogram and the coefficients of basis functions of illuminants. The weights of this neural network were adjusted by using back-propagation (BP) learning rule. To train the neural network, we need a set of images under different illuminants whose spectral power distributions are known in advance. However, such spectral information is not easy to measure in natural surrounding environments. To solve this problem, we use synthetic images with known illumination distributions in this experiment to produce the required training samples for the BP network. We also produce a set of testing images using the same procedure for testing the performance of the trained network.

To synthesize the images of a scene under different illuminants, we need to have the spectral reflectance R(λ) of different colors. In this experiment, we use the

colors defined in the color chart, AgfaIT8.7/2 [75], to form the colors in our synthetic images. Although AgfaIT8.7/2 consists of only 288 colors, they are enough for representing the colors of normal pictures. With the spectral reflectance R(λ) of all 288 colors and the spectral power distribution of the standard D illuminants for different color temperatures, we can then apply the linear combination of CIE’s color matching functions to synthesize the images of a scene under different illuminants. As listed in Table 5-2 and explained in Section 5.3.2, we select 28 standard D illuminants with 28 different color temperatures falling in the range of 4000°K~25000°K as the light sources for image synthesis. On the other hand, to make the synthetic images having the scenes close to the real captured ones, we randomly chose 50 colors from the AgfaIT8.7/2 color chart to form the colors in the images for the training and testing of BP network. We then apply the algorithm proposed in Section 5.3.1 to find the center values of chromaticity histogram, (Cx, Cy), on the synthetic images. The (Cx, Cy) values of the synthetic images for each of the 28 color temperatures are collected and used as the inputs of the training network, and the coefficients of the basis functions of illuminants, (e1, e2), corresponding to each of the 28 color temperatures are used as the desired outputs of the network. The learning constant in the BP learning rule is set as 0.8, and the convergence criteria in the form of output RMS error defined below is set as 0.005.

To evaluate the learning accuracy of the BP network, we define the network output error, Er, as the RMS error on the u-v chromaticity space:

2

2 ( ˆ )

) ˆ

(u_D u_D v_D v_D

Er = ′ − + ′ − , (5-19) where (u_D′ ,v_D′ ) and (uˆ_D ,vˆ_D) are the chromaticity co-ordinates of the real (synthetic) and estimated illuminants, respectively. Since the u-v color space is a

keep the same error scale. On the other hand, because the two output values of the BP network represent the coefficients of the two basis functions of illumination, we have to convert them into the values of x-y chromaticity for computing the RMS errors in Eq. (5-19). This conversion can be achieved by the following equation from the formula of standard D illuminants:

6327

where e1 and e2 are parameters whose values are related to the chromaticity co-ordinates (xˆ_D ,yˆ_D) . The values of (xˆ_D ,yˆ_D) , e1, and e2 correlate color temperatures in the range 4000°K to 25000°K. We can then use the CIE’s formula to transform (xˆ_D ,yˆ_D) into (uˆ_D ,vˆ_D), which can be used directly to calculate the RMS errors in Eq. (5-19).

Table 5-3 lists the estimation RMS errors of the proposed algorithm under different color temperatures in the second column. The estimation RMS errors of the Max-RGB and Gray-World algorithms on the same synthetic images are also listed in the third and fourth columns of Table 5-3 for comparison. Table 5-3 clearly shows the superiority of the proposed approach over the compared ones; the proposed algorithm always produces much smaller RMS errors than the other two. Although the Max-RGB and Gray-World algorithm produces smaller RMS errors in a few synthetic images, the RMS errors produced by the proposed algorithm on these images are also quite small. These exceptional cases are possible and reasonable, since we choose different combinations of colors randomly from the AgfaIT8.7/2 color chart to form the synthetic images, and the compared algorithms might have better performance on

the chosen colors. Overall, the proposed algorithm has better and stable estimation accuracy than the compared counterparts.

Table 5-3. The average RMS errors of illumination estimation of the proposed and compared algorithms on training synthetic images for an ideal camera (number of colors is 50).

Error (RMS) Color temp.

(°K) Our

Approach Gray-World Max-RGB 4000 0.0002 0.0069 0.0035 4200 0.0024 0.0044 0.0138 4400 0.0011 0.0099 0.0068 4600 0.0014 0.0057 0.0032 4800 0.0020 0.0052 0.0046 5000 0.0023 0.0072 0.0016 5200 0.0025 0.0071 0.0015 5400 0.0012 0.0036 0.0014 5600 0.0010 0.0062 0.0048 5800 0.0020 0.0065 0.0033 6000 0.0030 0.0070 0.0050 6400 0.0023 0.0083 0.0032 6800 0.0022 0.0061 0.0062 7200 0.0013 0.0059 0.0067 7600 0.0010 0.0053 0.0021 8000 0.0025 0.0064 0.0066 8500 0.0048 0.0083 0.0033 9000 0.0011 0.0058 0.0037 9500 0.0030 0.0065 0.0034 10000 0.0014 0.0054 0.0036 11000 0.0026 0.0050 0.0153 12000 0.0025 0.0043 0.0058 13000 0.0018 0.0076 0.0020 14000 0.0024 0.0062 0.0131 15000 0.0032 0.0066 0.0020 17000 0.0019 0.0057 0.0049 20000 0.0022 0.0065 0.0035 25000 0.0036 0.0058 0.0017

Avg. Error 0.0021 0.0063 0.0049

We use the same synthesis procedure as mentioned in the above to produce another set of synthetic images for testing of the trained BP network. These testing sets of images are synthesized according to the illuminants different from those used in producing the training set of synthetic images. A total of 40 colors are randomly chosen from the 288 colors in the AgfaIT8.7/2 color chart to form a testing image.

The proposed and compared approaches are then used to estimate the illumination of each testing image. The estimation RMS errors are listed in Table 5-4, where each value in the table is the average RMS error over 20 synthetic images of a specific illuminant. The results still indicate the superiority of the proposed scheme over the other two compared algorithms even in the cases of unlearned surrounding illuminants.

Finally, we want to find out how the number of colors in an image affects the estimation accuracy of the surrounding illumination. Under a specific illuminant, we randomly choose different numbers of colors from the AgfaIT8.7/2 color chart to form the synthetic images, starting from the number of five and increasing by five colors each time until 35 colors in total. Hence, we have seven synthetic images for a specific illuminant, with the numbers of colors in the seven images being 5, 10, 15, …, 50, respectively, all chosen randomly from the AgfaIT8.7/2 color chart. For a specific illuminant and a specific number of colors, a total of 25 images are synthesized. Again, we use the proposed and compared algorithms to estimate the illumination of these synthetic images. The estimation RMS errors are shown in Fig. 5-4, where each value in the figure is the average RMS error over 175 (25x7) synthetic images of a specific illuminant. The results indicate that the estimation error of each algorithm increases as the number of colors in an image decreases. This is reasonable since higher number of colors can provide more information of spectral power distribution of the surrounding

illuminant. However, in any case, the proposed algorithm still possesses the best performance.

Table 5-4. The average RMS errors of illumination estimation of the proposed and compared algorithms on testing synthetic images for an ideal camera (number of colors is 40).

Error (RMS) Color temp.

(°K) Our

ApproachGray-WorldMax-RGB 4300 0.0025 0.0072 0.0053 5300 0.0022 0.0065 0.0043 5800 0.0040 0.0086 0.0103 6600 0.0039 0.0067 0.0038 7400 0.0053 0.0061 0.0109 8200 0.0025 0.0070 0.0048 10500 0.0029 0.0067 0.0035 Avg. Error 0.0033 0.0070 0.0061

Figure 5-4. The average RMS errors of illumination estimation of the proposed and compared algorithms on testing synthetic images with respect to different number of colors in an image.