AUTOMATIC WHITE BALANCE FOR DIGITAL STILL CAMERA

AUTOMATIC WHITE BALANCE FOR DIGITAL STILL CAMERA

Tzan-Sheng Chiou (邱贊生), Chiou-Shann Fuh (傅楸善), and Varsha Chikane Department of Computer Science and Information Engineering,

National Taiwan University, Taipei, Taiwan.

E-mail: fuh@csie.ntu.edu.tw ABSTRACT

There are many key factors that have a great effect upon the quality of digital cameras. White balance is one of the keys to the quality of digital cameras. Traditional white balance methods are powerless in some extreme conditions. In order to get more precise result we divided the problem of adjusting the white balance in three parts - Detecting white point, Judging white balance, and Adjust- ing white balance. In the present study we proposed a new white balance approach to respectively solve these main problems of white balance. The experimental results show that the proposed method has the best results in both visual and evaluative results. The complexity of the proposed system is acceptable. The proposed method can be easily applied to the digital cameras to get good effects.

Keywords: white balance, color balance.

1. INTRODUCTION

Due to the continuing decline in the cost the digital cameras are becoming increasingly popular. Affordability is needed for the continued expansion of the digital camera market, but the quality of digital color image is also the important consideration for the consumer. There are many key factors that have a great effect upon the quality of digi- tal camera. White balance is one of the keys to the quality of digital camera.

An image is composed of a number of pixels representing the captured scene. Each pixel has respective component values representing the intensity of incident light of a particular color that is detected by a color sensor. The color of an object recorded by a color sensor typically var- ies with light sources that illuminate the scene. This is due to the color temperature difference of the light sources.

Color temperature is the absolute temperature at which we heat a standard black body (metal) to make it radiate the same light as a certain light source. When a white object is illuminated with low color temperature light source, the captured image will show the object as having a slightly reddish color. Similarly, if the white object is illuminated with high color temperature light source, the object in the captured image will have a bluish color.

The human vision may not be able to distinguish the color difference caused by various light sources due to the “color constancy” of human eye [1]. Colors remain constant through recognition despite they are viewed under different light sources. In summary, the mechanism employed in digital camera to compensate the color difference caused

by various light sources is white balance. This is the main investigation.

2. BACKGROUND

The traditional methods used to adjust the white balance automatically are mentioned below,

Gray World Method

This method takes an image and scales its red, green, and blue color components to make the gray world assumption [2] hold.

Advanced Gray World Method

This method is approximately the same as the gray world method in addition to filtering out those pixels having a highly chromatic color.

Fuzzy Rules Method

In this method the image is converted from the color space to the

b color space [3] and seized the color’s characteristics in the color space for the white bal- ance adjustment. The fuzzy rules based on experimental results are described below:

RGB

rC YC

b rC YC

1. The averages of and for each segment will be weighted with small values under the conditions of high-end and low-end luminance in order to avoid being saturated and colorless.

Cr C_b

2. The averages ofC andC for each segment are weighted less for dark colors than bright colors.

r b

3. When a large object or background occupies more than one segment, its color will dominate that segment. The ratio ofC to_C will be similar among adjacent segments.

The given weighting for those segments having a uniform chromatic color is small in order to avoid over-compensa- tion on the color of the picture.

r b

If the ratio of to of the segment is approximately between –1.5 to –0.5, the probability of being a white re- gion increases, the given weighting is the largest.

Cr C_b

Perfect Reflector Method

This method exploits the perfect reflector assumption [4] to correct the image. It locates the reference white point by finding the pixel with the greatest luminance value and performs the white balance adjustment according to the reference white point.

Percentile Perfect Reflector Method

This method states five percent of component values in the image having the highest red, green, and blue values.

These values are selected from the given percentile of the histogram of that image to compute the global white point and then perform the white balance adjustment according to the global white point.

3. THE NEW WHITE BALANCE METHOD

By using the advantages of above mention methods we have proposed a new white balance method. The new white balance method consists of three components, as shown in Fig.1.

Fig. 1 The component block diagram of the new white balance method.

3.1 White Point Detecting Unit

In the White Point Detecting unit, first we find out rough and precise reference white points. For detecting rough white point, the prescribed threshold must be find out for that, we put the GretagMacbeth ColorChecker under two light sources, one represents the extremely high color tem- perature (Daylight) and the other represents the extremely low color temperatures (Horizon). Second, we used a small window inside each color patch in the bottom of the Gre- tagMacbeth ColorChecker and calculated the average of those pixels within the windows to get the values of the achromatic color patches. Third, we converted the achromatic color patches from the color space to the

bcolor space. Finally, we calculated those chromaticity values (

RGB

RGB

rC YC

2 2

b

r C

C +

rC YC

) of the achromatic color patches in the above two images and picked up the largest value as the prescribed threshold (CH , 60 in our experiments). Then all pixels in the image are converted from RGB color space to the _b color space and used the condition in which the chromaticity value of a pixel is equal to or less than a prescribed threshold (_CH ) to judge whether or not this pixel has the highly chromatic color. It is formulated as shown in Equation 1.1,

th

th

th b

r C CH

C²+ ² ≤

where ^{2 2}

b

r C

C +

th

is defined as the chromaticity value of a pixel; and _CH is defined as the prescribed threshold.

Using this equation we can easily detect highly chromatic colors and remove it. To get rough reference point, we took the average of all pixels from which we removed the highly chromatic pixels.

Now, for detecting precise reference point we use advance global point detecting method. We observe by experiments that, pixel that seems to be the most appropriate for white point have two common properties: first, pixels have the relatively high luminance value and second, pixels have the higher red, green, and blue component values. We have made use of the first property to build the related white point detecting method. Now we try to take advantage of the second property to build a new white point detecting method. In order to get pixels that have the higher red, green, and blue component values from an image, we must determine first the suitable threshold for each color com- ponent of the image. Suitable threshold for each compo- nent is generated for an image based on pixels that are in given percentile of the histogram of each color component of the image [5]. This means that the red component threshold (_R ) is defined as the eightieth percentile of the red component of the image, the green component thresh- old ( G ) is defined as the eightieth percentile of the green component of the image and the blue component threshold ( ) is defined as the eightieth percentile of the blue com- ponent of the image. To get each component threshold of the image, we pick up those pixels whose red, green, and blue values are respectively higher than the related component threshold ( ) as the reference white point. It is formulated as shown in Equation 1.2,

th

th

Bth

th th

th G B

R , ,and Start

End

Fig.1: The components block diagram of the new white balance h d

White point detecting unit White balance judging unit White balance adjusting unit

th th th

B B

G G

R R

≥

≥

≥ (1.2)

where are the respective component thresh- old. Generally, it works very well to look for the reference white point but there is possibility that the reference white point detected by this method may be the yellow and cyan.

To distinguish the pixels for appropriate white point from the similar pixels of yellow and cyan, we convert the image from the color space to the _YC

th th

th G B

R , ,and

RGB _rCb

b rC YC

b rC

color space and seize the color characteristics in the color space. The pixels that seem to be the most appropriate for white have two useful characteristics in the _YC color space. First, the absolute value of is lower than a prescribed thresh- old and the absolute value of

Cr

Cb is lower than a prescribed threshold. It is formulated as shown in Equation 1.3

r

r AB

C ≤

(1.3)

b

b AB

C ≤

where is defined as the prescribed threshold for the absolute value of and is defined as the prescribed

ABr

Cr AB_b (1.1)

threshold for the absolute value of_Cb. Second, the ratio of Crto_Cb falls into a prescribed range. It is formulated as shown in Equation 1.4,

u b r

l R

C R ≤C ≤

where is defined as the lower bound for the prescribed range, and is defined as the upper bound for the prescribed range. However, to decide those proper thresh- olds ( ), we use the same experiment performed in the rough reference white point method that is, we take images of the GretagMacbeth ColorChecker in the extremely high and extremely low color temperatures and those absolute values of C

Rl

AB Ru

AB ,b _{l u}

r , R ,andR

Cb

r

b

R

and of the achro- matic color patches for the above-mentioned images, are calculated and then we pick up the largest value as the threshold for the absolute values of (AB , 45 in our experiments) and (_AB , 45 in our experiments), then we calculate those ratios ofC to_C of the achromatic color patches in the above two images and pick up the minimum as the lower bound for the prescribed range ( , -1.25 in our experiments) and pick up the maximum as the upper bound for the prescribed range ( , -0.75 in our experi- ments).

Cb

Cr _r

R

b r

l

u

Fig. 1.1 The flow chart of the white point detecting unit.

If a pixel satisfies the above conditions then we again con- firm that this pixel is the most appropriate for white point and thus we pick it up as a number of precise reference white point. Figure 1.1 shows the operations performed in the white point detecting unit. First, calculate each compo- nent threshold used in the later operation (R ,G , and

). Second, read the values of a pixel and convert them from the color space to the _b color space.

Third, judge whether or not a pixel has the qualification for the rough reference white point. The values ( ,G , )

represent the average of the rough reference white pixels.

Fourth, judge whether or not a pixel has the qualification for the precise reference white point. The values ( ,G , ) represent the average of the precise refer- ence white pixels. Finally, judge whether all pixels of the image are read. If all pixels of the image are read, we can go into the next unit and those related reference white point data are also transferred into the next unit. Otherwise, we read the next pixel of the image and perform the iterative steps.

th

C

th

Bth RGB

RGB YC_r

Rr

precise

rough r

R Br

precise

Rp

rough

R

R

R

p

rough

R

precise

M

Ma

Bp

R

precise

a

precise≥

rough

R (1.4)

3.2 The White Balance Judging Unit

The function of the white balance judging unit lies in judging whether or not to perform the white balance ad- justment and selecting a suitable reference white point from the rough reference white point and the precise refer- ence white point for the white balance adjusting unit (see Figure 1.2). First, we pick up the reference white point data that are obtained from the white point detecting unit. Then we calculate value, which is the ratio of the rough reference white pixels to all pixels of an image and

value, which is the ratio of the precise reference white pix- els to all pixels of an image. Second, judge whether or not

is equal to or larger than a value, defined as prescribed proportion (0.2 in our experiments). The rough reference white pixels represent those pixels whose colors are not highly chromatic colors, if is higher then there is a possibility that the major portions of the image are not occupied by a highly chromatic color and on the contrary, if is lower then there is a high possibility that the major portions of the image are occupied by a highly chro- matic color. Third, judge whether or not is equal to or larger than value which is defined as the prescribed proportion (0.05 in our experiments). The precise white pixels represent pixels that are the most appropriate for white, if is high then there exists enough precise ref- erence white pixels in the image, on the contrary, if is low then there exist insufficient precise refer- ence white pixels in the image. Fourth, set the adjusting modes, and stands for different adjusting modes. If the first condition ( ) is satisfied then there is possibility that the major portions of the image are not oc- cupied by highly chromatic color and if the second condi- tion (_R ) is satisfied then there exist enough precise reference white pixels in the image. In cases where

the first ( ) and second ( ) condi-

tions are satisfied, is set to 2, in this situation use pre- cise reference white point as the reference white point. In the cases where, the first condition (_R ) is satis- fied and the second condition ( ) is not satis- fied, is set to 1, in this situation choose the minor from

rough

rough

P

precise

P

≥P

rough

P

rough

precise

R

rough precise≥P

R

precise

R

≥P

≥P

precise rough

rough P

R ≥

rough

Ma

R

u b r l

b b r r

R C C R

AB C AB C

≤

≤

≤

≤ ,

W h i t e p o i n t d e t e c t i n g u n i t

Y e s Y e s Y e s

Y e s

N o

N o

N o

N o

W h i t e b a l a n c e j u d g i n g u n i t D e t e r m i n e e a c h c o l o r c o m p o n e n t t h r e s h o l d o f a n i m a g e (Rth ,Gth ,andBth).

R e a d t h e RGB v a l u e s o f a p i x e l a n d c o m p u t e i t s r e l a t e d YCrCb v a l u e s .

th b

r C CH

C²+ ²≤

T h e p i x e l i s c o u n t e d t o c a l c u l a t e t h e r o u g h r e f e r e n c e w h i t e p o i n t (Rr ,Gr ,Br).

th th th

B B

G G

R R

≥

≥

≥

T h e p i x e l i s c o u n t e d t o c a l c u l a t e t h e p r e c i s e r e f e r e n c e w h i t e p o i n t (Rp ,Gp ,Bp).

A r e a l l p i x e l s

the rough reference white point and the precise reference white point as the reference white point.

Fig.1.2 The flow chart of the white balance judging unit.

In cases where the first condition ( ) is not sat- isfied and the second condition ( ) is satisfied,

is set to 2, this means that we use precise reference white point as the reference white point. In cases where the first condition ( ) is not satisfied and also the second condition ( ) is not satisfied, _M is set to 0, this means that we will perform nothing for the white balance adjustment. We used this mode because in white balance methods there are certain situations at which it is not necessary to adjust white balance. If we adjust white balance crudely then we get strange results and mostly it happens when the major portion of the image is occupied by the highly chromatic colors.

rough

rough P

R ≥

precise precise≥P R Ma

rough

rough P

R ≥

precise P

R ≥ _{precise a}

3.3 The White Balance Adjusting Unit

The flow chart of the white balance adjusting unit is shown in Fig.1.3. It consists of two steps, one is computing the scale factor of each color component in an image and the other is scaling the component values according to its re- spective scale factor. To compute the scale factor of each color component in an image, in the white balance judging unit we have set the different adjusting mode value to in- dicate different adjusting situations so it becomes easy to get the related scale factors of the image according to the adjusting mode value.

First, we check whether or not the green value of the reference white point is higher than a prescribed threshold.

In cases where the green value of the reference white point is lower than a prescribed threshold, the image is dark, so the meaning of performing the white balance adjustment decreases. Second we calculate the scale factor of each color component according to the rough reference white

point and the scale factor of each color component accord- ing to the precise reference white point. The rough refer- ence white point defined as ( ,G , ), where, , , and are respective component values for each color component and _Y is the luminance value of the rough reference white point. It is formulated as shown in Equa- tion 1.5.

Rr _r B_r R_r G_r Br

r

R

Compute rough and precise reference white pixels rates (Rrough and Rprecise).

rough

rough P

R ≥

precise precise P

R ≥ R_precise≥P_precise

Ma is set to 0.

Ma is set to 1.

Ma is set to 2.

Yes

Yes No

Yes No No White balance

judging unit

Fig. 1.2: The flow chart of the white balance judging unit.

White point detecting unit

White balance adjusting unit

r r

r

r

R G B

Y = 0 . 299 * + 0 . 587 * + 0 . 114 *

(1.5) The scale factors of red, green, and blue component values are defined as . Thus individually, we can easily compute the scale factor of each color compo- nent according to the rough reference white point as shown in Equation 1.6.

rgain rgain

rgain G B

R , ,

.

r r rgain

r r rgain

r r rgain

B Y B

G Y G

R Y R

/ / /

=

=

=

(1.6)

On the other side, the precise reference white point of an image is defined as ( ) where, and are respective component values for each color component.

The luminance of the precise reference white point is defined as . It is formulated as shown in Equation 1.7.

p p

p G B

R , , R ,_p G_p B_p

Yp

(1.7)

p p

p

p R G B

Y =0.299* +0.587* +0.114*

We could easily compute the scale factor of each color component according to the precise reference white point as shown in Equation 1.8.

G

p p pgain

p p pgain

p p pgain

B Y B

G Y

R Y R

/ / /

=

=

=

(1.8)

Those scale factors that we calculated above may have the extreme values (ultra-high or ultra-low). By using these scale factors if we crudely scale the component values then we get strange results. To overcome this problem, we can use a sigmoid function to relax the extreme scale factor toward an acceptable value. It is formulated as shown in Equation 1.9.

(1.9)

4 . 0 )) 25 . 1 tanh(

1 (

* 05 .

1 + − +

=

X

Y

where, X is the original scale factor and Y is the ad- justed scale factor. Second, select scale factors of the im- age according to the adjusting mode value. When is equal to 0, the white balance adjustment is stopped. When is equal to 1, we select the minor between

( ) and ( ) as the actual

scale factors. When is equal to 2, we select ( ) as the actual scale factors. Third, scale each color component according to its respective scale factor.

Ma

Ma

rgain

rgain, ,B

pgain

pgain B

R , ,

rgain

G

pgain

G

pgain pgain

pgain G B

R , , Ma

Fig.1.3 The flow chart of the white balance adjusting unit.

Com pute scale factors according to the rough reference white point (Rrgain, Grgain, Brgain)

≠0 Ma

≠1 Ma

Scale the component values according to its respective scale factor.

Stop No Yes

Yes

No W hite balance

adjusting unit

W hite balance judging unit

Finish

Select the m inor am ong (Rrgain, Grgain, Brgain) and (Rpgain, Gpgain, Bpgain) as scale factors of the image.

Use (Rpgain, Gpgain, Bpgain) as scale factors of the im age.

Com pute scale factors according to precise reference white point (Rpgain, Gpgain, Bpgain).

4. EXPERIMENTAL RESULTS

We are interested in the results of distinct white balance approaches and the differences between them, so we take those test images, under the different light sources and simulated them with gray world method, advanced gray world method, fuzzy rules method, perfect reflector method, percentile perfect reflector method, the function of automatic balance in PhotoImpact and the new white balance method. The associated light sources used in our experiment with its color temperature are listed below in Table 1.

Light Source Day- light

Cool White

INCa Horizon

Color Temp.

(Kelvin) 6575 4289 2884 2264

Table 1 Color temperature of different light sources.

The test image of GretagMacbeth ColorChecker in differ- ent light sources and corresponding results after applying new white balance method. (See Figure 2.)

Daylight

Original image taken Image, after applying New in Day-light White Balance Method:

(R_roug=1, _Rprec=0.04, _Ma=1)

Cool White

Original image taken Image, after applying New in Cool White White Balance Method:

(Rrough=0.99, Rprecise=0.11, Ma=2)

INCa

Original image taken Image, after applying New in INCa White Balance Method:

^(Rrough=0.96, Rprecise=0.03, Ma=1)

Horizon

Original image taken Image, after applying New in Horizon White Balance Method:

(Rrough=0.99, Rprecise=0.11, Ma=2)

Cool White’

Original image taken Image, after applying New in Cool White White Balance Method:

(Rrough=0.96, Rprecise=0.16, Ma=2)

Fig. 2 Visual results in the different light sources.

As we are interested in the results of distinct white balance approaches and the difference between them so here we take those test images and simulate them with gray world method ^(GWM), advanced gray world method ^(AGWM), fuzzy rules method (FRM), perfect reflector method (PRM), percentile perfect reflector method (PPRM), the function of automatic balance in PhotoImpact (UPI), and the new white balance method (NWBM). Visual results of two test images (original image and image after applying new white balance method) are shown in Figure 2. In addi-

tion to visual verification we also want to quantify the re- sult of different white balance approaches. Unfortunately, there does not exist a proper quantitative tool for white balance. Here we propose a simple evaluative method. We use a small window (a 11X11 pixel block) inside each color patch in the bottom of the GretagMacbeth Color- Checker and calculate the average of those pixels within the windows to get the RGB values of the achromatic color patches and then we convert the achromatic color patches from the color space to the _YC color space. We calculate the sum of those chromaticity values (

RGB _rCb

2 2

b

r C

C + ) of the achromatic color patches in the image and use it as the evaluation criterion for white balance. Ta- ble 2 shows the evaluative results of these seven methods for the test images shown in Figure 2.

Table 2. The evaluative results of seven methods for the test images.

5. CONCLUSION

From the experimental results we find that the traditional white balance methods can partially solve the problem of adjusting the white balance and it is hard to get a better re- sult by purely applying the traditional white balance meth- ods to digital cameras. However, in our present study we have used the advantages of traditional white balance methods, such as the white balance method based on the Gray World assumption that offers us the information about the light cast over the whole image, while the white balance method based on the Perfect Reflector assumption provides us a rough methodology to find out the reference white point from an image. Moreover, the color character- istics in the _b color space help us to precisely look for the reference white point from an image. In addition to de- tecting the reference white point from an image and ad- justing white balance accordingly, we must deal with the problem that whether or not the white balance adjustment is really required. If we ignore this problem and crudely adjust white balance, we will destroy the consistency of colors in the image and get strange results on certain situa- tions. In this paper the proposed approach not only mixes the advantages of the traditional methods to form a new white point detecting method but also carefully copes with the problem of whether or not we must perform the white balance adjustment. The simulation result shows that the

proposed method has the best results in both visual results and objective evaluation

rC YC

6. ACKNOWLEDGEMENT

This research was supported by the National Science Council of Taiwan, R.O.C., under Grants NSC 91-2711-3-319-001-SP-24 and NSC 91-2212-E-002-087, by the EeRise Corporation, EeVision Corporation, Tekom Technologies, Jeilin Technology, IAC, and Delta Electronics.

7. REFERENCES

[1] R. C. Gonzalez and R. E. Woods, Digital Image Processing, Addison Wesley, Reading, MA, 1992.

[2] M. Fedor, “Approaches to Color Balancing,”

PSYCH221/EE362course project, Department of Psychology, Stanford University, 1998.

[3] Y. C. Cheng, W. H. Chan, and Y.Q. Chen, “Auto- matic White Balance for Digital Still Camera,” IEEE Transactions on Consumer Electronics, Volume 41, pp. 460-466, 1995.

[4] J. Chiang and F. Al-Turkait, “Color Balancing Ex- periments with the HP-Photo Smart-C30 Digital Camera,” PSYCH221/EE362 course project, Depart- ment of Psychology, Stanford University, 1999.

[5] A. V. Durg and O. Rashkovskiy, “Global White Point Detection and White Balance for Color Images,” U.S.

Patent #6069972, 2000.

Evaluative values Test images

Original Manual GWM AGWM FRM PRM PPRM UPI NWBM

Daylight 79.38 11.73 53.25 54.81 53.58 21.67 43.54 34.84 13.31 Cool White 76.8 6.3 45.36 44.86 45.67 15.82 37.81 49.05 7.38 INCa 161.16 13.99 25.64 24.01 82.08 28.08 16.99 45.38 17.35 Horizon 202.71 11.94 22.45 23.51 101.94 34.5 26.15 54.2 20.14 Cool White' 77.85 5.99 22.4 22.21 43.74 39.78 23.03 81.29 6.75