AUTOMATIC WHITE BALANCE FOR DIGITAL STILL CAMERA
Tzan-Sheng Chiou (邱贊生), Chiou-Shann Fuh (傅楸善), and Varsha Chikane Department of Computer Science and Information Engineering,
National Taiwan University, Taipei, Taiwan.
E-mail: fuh@csie.ntu.edu.tw ABSTRACT
There are many key factors that have a great effect upon the quality of digital cameras. White balance is one of the keys to the quality of digital cameras. Traditional white balance methods are powerless in some extreme conditions. In order to get more precise result we divided the problem of adjusting the white balance in three parts - Detecting white point, Judging white balance, and Adjust- ing white balance. In the present study we proposed a new white balance approach to respectively solve these main problems of white balance. The experimental results show that the proposed method has the best results in both visual and evaluative results. The complexity of the proposed system is acceptable. The proposed method can be easily applied to the digital cameras to get good effects.
Keywords: white balance, color balance.
1. INTRODUCTION
Due to the continuing decline in the cost the digital cameras are becoming increasingly popular. Affordability is needed for the continued expansion of the digital camera market, but the quality of digital color image is also the important consideration for the consumer. There are many key factors that have a great effect upon the quality of dig- ital camera. White balance is one of the keys to the quality of digital camera.
An image is composed of a number of pixels representing the captured scene. Each pixel has respective component values representing the intensity of incident light of a par- ticular color that is detected by a color sensor. The color of an object recorded by a color sensor typically varies with light sources that illuminate the scene. This is due to the color temperature difference of the light sources. Color temperature is the absolute temperature at which we heat a standard black body (metal) to make it radiate the same light as a certain light source. When a white object is illu- minated with low color temperature light source, the cap- tured image will show the object as having a slightly red- dish color. Similarly, if the white object is illuminated with high color temperature light source, the object in the cap- tured image will have a bluish color.
The human vision may not be able to distinguish the color difference caused by various light sources due to the “color constancy” of human eye [1]. Colors remain constant through recognition despite they are viewed under different light sources. In summary, the mechanism employed in digital camera to compensate the color difference caused
by various light sources is white balance. This is the main investigation.
2. BACKGROUND
The traditional methods used to adjust the white balance automatically are mentioned below,
Gray World Method
This method takes an image and scales its red, green, and blue color components to make the gray world assumption [2] hold.
Advanced Gray World Method
This method is approximately the same as the gray world method in addition to filtering out those pixels having a highly chromatic color.
Fuzzy Rules Method
In this method the image is converted from the RGB color space to the YCrCb color space [3] and seized the color’ s characteristics in the YCrCbcolor space for the white bal- ance adjustment. The fuzzy rules based on experimental results are described below:
1. The averages ofCrandCbfor each segment will be weighted with small values under the conditions of high-end and low-end luminance in order to avoid being saturated and colorless.
2. The averages ofCrandCbfor each segment are weighted less for dark colors than bright colors.
3. When a large object or background occupies more than one segment, its color will dominate that segment. The ratio ofCrtoCbwill be similar among adjacent segments.
The given weighting for those segments having a uniform chromatic color is small in order to avoid over-compensation on the color of the picture.
If the ratio of Cr to Cbof the segment is approximately between – 1.5 to – 0.5, the probability of being a white region increases, the given weighting is the largest.
Perfect Reflector Method
This method exploits the perfect reflector assumption [4] to correct the image. It locates the reference white point by finding the pixel with the greatest luminance value and performs the white balance adjustment according to the reference white point.
Percentile Perfect Reflector Method
This method states five percent of component values in the image having the highest red, green, and blue values.
These values are selected from the given percentile of the histogram of that image to compute the global white point and then perform the white balance adjustment according to the global white point.
3. THE NEW WHITE BALANCE METHOD
By using the advantages of above mention methods we have proposed a new white balance method. The new white balance method consists of three components, as shown in Fig.1.
Fig. 1 The component block diagram of the new white balance method.
3.1 White Point Detecting Unit
In the White Point Detecting unit, first we found out rough and precise reference white points. For detecting rough white point, the prescribed threshold must be found out for that, we put the GretagMacbeth ColorChecker under two light sources, one represents the extremely high color tem- perature (Daylight) and the other represents the extremely low color temperatures (Horizon). Second, we used a small window inside each color patch in the bottom of the Gre- tagMacbeth ColorChecker and calculated the average of those pixels within the windows to get the RGB values of the achromatic color patches. Third, we converted the ach- romatic color patches from the RGB color space to the
b rC
YC color space. Finally, we calculated those chromaticity values ( 2 2
b
r C
C + ) of the achromatic color patches in the above two images and picked up the largest value as the prescribed threshold (CHth, 60 in our experiments). Then all pixels in the image are converted from RGB color space to the
b rC
YC color space and used the condition in which the chromaticity value of a pixel is equal to or less than a prescribed threshold (CHth) to judge whether or not this pixel has the highly chromatic color. It is formulated as shown in Equation 1.1,
th b
r C CH
C2+ 2 ≤ where 2 2
b
r C
C + is defined as the chromaticity value of a pixel; and CHth is defined as the prescribed threshold.
Using this equation we can easily detect highly chromatic colors and remove it. To get rough reference point, we took the average of all pixels from which we removed the high- ly chromatic pixels.
Now, for detecting precise reference point we use advance global point detecting method. We observe by experiments that, pixel that seems to be the most appropriate for white point have two common properties: first, pixels have the relatively high luminance value and second, pixels have the higher red, green, and blue component values. We have made use of the first property to build the related white point detecting method. Now we try to take advantage of the second property to build a new white point detecting method. In order to get pixels that have the higher red, green, and blue component values from an image, we must determine first the suitable threshold for each color com- ponent of the image. Suitable threshold for each compo- nent is generated for an image based on pixels that are in given percentile of the histogram of each color component of the image [5]. This means that the red component thre- shold (Rth) is defined as the eightieth percentile of the red component of the image, the green component threshold (G ) is defined as the eightieth percentile of the green th component of the image and the blue component threshold (Bth) is defined as the eightieth percentile of the blue component of the image. To get each component threshold of the image, we pick up those pixels whose red, green, and blue values are respectively higher than the related component threshold (Rth,Gth,andBth) as the reference white point. It is formulated as shown in Equation 1.2,
th th th
B B
G G
R R
≥
≥
≥
where Rth,Gth,andBth are the respective component thre- shold. Generally, it works very well to look for the refer- ence white point but there is possibility that the reference white point detected by this method may be the yellow and cyan. To distinguish the pixels for appropriate white point from the similar pixels of yellow and cyan, we convert the image from the RGB color space to the
b rC
YC color space and seize the color characteristics in the
b rC
YC color space.
The pixels that seem to be the most appropriate for white have two useful characteristics in the
b rC
YC color space.
First, the absolute value of Cris lower than a prescribed threshold and the absolute value of Cb is lower than a prescribed threshold. It is formulated as shown in Equation 1.3
r
r AB
C ≤
b
b AB
C ≤
(1.1)
(1.3) Start
End
Fig.1: The components block diagram of the new white balance h d
White point detecting unit
White balance judging unit
White balance adjusting unit
(1.2)
where ABr is defined as the prescribed threshold for the absolute value of Crand ABb is defined as the prescribed threshold for the absolute value ofCb. Second, the ratio of CrtoCb falls into a prescribed range. It is formulated as shown in Equation 1.4,
u b r
l R
C R ≤C ≤
whereRlis defined as the lower bound for the prescribed range, and R is defined as the upper bound for the u prescribed range. However, to decide those proper thre- sholds (ABr,ABb ,Rl ,andRu), we use the same experiment performed in the rough reference white point method that is, we take images of the GretagMacbeth ColorChecker in the extremely high and extremely low color temperatures and those absolute values of Cr and Cb of the achro- matic color patches for the above-mentioned images, are calculated and then we pick up the largest value as the threshold for the absolute values of Cr (ABr, 45 in our experiments) and Cb (ABb, 45 in our experiments), then we calculate those ratios ofCrtoCbof the achromatic color patches in the above two images and pick up the minimum as the lower bound for the prescribed range (Rl, -1.25 in our experiments) and pick up the maximum as the upper bound for the prescribed range (Ru, -0.75 in our experi- ments).
Fig. 1.1 The flow chart of the white point detecting unit.
If a pixel satisfies the above conditions then we again con- firm that this pixel is the most appropriate for white point and thus we pick it up as a number of precise reference white point. Figure 1.1 shows the operations performed in the white point detecting unit. First, calculate each compo- nent threshold used in the later operation (Rth,Gth, and
Bth). Second, read the RGB values of a pixel and convert them from the RGB color space to the
b rC
YC color space.
Third, judge whether or not a pixel has the qualification for the rough reference white point. The values (Rr,Gr,Br)
represent the average of the rough reference white pixels.
Fourth, judge whether or not a pixel has the qualification for the precise reference white point. The values (Rp,Gp,B ) represent the average of the precise refer-p ence white pixels. Finally, judge whether all pixels of the image are read. If all pixels of the image are read, we can go into the next unit and those related reference white point data are also transferred into the next unit. Otherwise, we read the next pixel of the image and perform the iterative steps.
3.2 The White Balance Judging Unit
The function of the white balance judging unit lies in judging whether or not to perform the white balance ad- justment and selecting a suitable reference white point from the rough reference white point and the precise refer- ence white point for the white balance adjusting unit (see Figure 1.2). First, we pick up the reference white point data that are obtained from the white point detecting unit. Then we calculateRroughvalue, which is the ratio of the rough reference white pixels to all pixels of an image and Rprecise value, which is the ratio of the precise reference white pixels to all pixels of an im- age. Second, judge whether or not Rroughis equal to or larger than aProughvalue, defined as prescribed proportion (0.2 in our experiments). The rough refer- ence white pixels represent those pixels whose colors are not highly chromatic colors, ifRroughis higher then there is a possibility that the major portions of the image are not occupied by a highly chromatic color and on the contrary, ifRroughis lower then there is a high possibility that the major portions of the image are occupied by a highly chromatic color. Third, judge whether or not Rprecise is equal to or larger
thanProughvalue which is defined as the prescribed
proportion (0.05 in our experiments). The precise white pixels represent pixels that are the most appro- priate for white, ifRpreciseis high then there exists enough precise reference white pixels in the image, on the contrary, ifRpreciseis low then there exist insuffi- cient precise reference white pixels in the image.
(1.4)
u b r l
b b r r
R C C R
AB C AB C
≤
≤
≤
≤ ,
W h i t e p o i n t d e t e c t i n g u n i t
Y e s Y e s Y e s
Y e s
N o
N o
N o
N o
W h i t e b a l a n c e j u d g i n g u n i t D e t e r m i n e e a c h c o l o r c o m p o n e n t t h r e s h o l d o f a n i m a g e (Rth,Gth,andBth).
R e a d t h e RGB v a l u e s o f a p i x e l a n d c o m p u t e i t s r e l a t e d YCrCb v a l u e s .
th b
r C CH
C2+ 2≤
T h e p i x e l i s c o u n t e d t o c a l c u l a t e t h e r o u g h r e f e r e n c e w h i t e p o i n t (Rr,Gr,Br).
th th th
B B
G G
R R
≥
≥
≥
T h e p i x e l i s c o u n t e d t o c a l c u l a t e t h e p r e c i s e r e f e r e n c e w h i t e p o i n t (Rp,Gp,Bp).
A r e a l l p i x e l s
Fourth, set the adjusting modes, Ma and stands for different adjusting modes. If the first condition (Rrough≥Prough) is satisfied then there is possibility that the major portions of the image are not occupied by highly chromatic color and if the second condition (Rprecise≥Pprecise) is satisfied then there exist enough precise reference white pixels in the image. In cases where the first (Rrough≥Prough) and second (Rprecise≥Pprecise) conditions are satisfied, Ma is set to 2, in this situation use precise reference white point as the reference white point. In the cases where, the first condition (Rrough≥Prough) is satisfied and the second condition (Rprecise≥Pprecise) is not satisfied, Mais set to 1, in this situation choose the minor from the rough reference white point and the precise refer- ence white point as the reference white point.
Fig.1.2 The flow chart of the white balance judging unit.
In cases where the first condition (
rough
rough P
R ≥ ) is not sa- tisfied and the second condition (Rprecise≥Pprecise) is satisfied,
Mais set to 2, this means that we use precise reference white point as the reference white point. In cases where the first condition (Rrough ≥Prough) is not satisfied and also the second condition (Rprecise≥Pprecise) is not satisfied, Ma is set to 0, this means that we will perform nothing for the white balance adjustment. We used this mode because in white balance methods there are certain situations at which it is not necessary to adjust white balance. If we adjust white balance crudely then we get strange results and mostly it happens when the major portion of the image is occupied by the highly chromatic colors.
3.3 The White Balance Adjusting Unit
The flow chart of the white balance adjusting unit is shown in Fig.1.3. It consists of two steps, one is computing the scale factor of each color component in an image and the other is scaling the component values according to its re- spective scale factor. To compute the scale factor of each color component in an image, in the white balance judging unit we have set the different adjusting mode value to in- dicate different adjusting situations so it becomes easy to get the related scale factors of the image according to the adjusting mode value.
First, we check whether or not the green value of the reference white point is higher than a prescribed threshold.
In cases where the green value of the reference white point is lower than a prescribed threshold, the image is dark, so the meaning of performing the white balance adjustment decreases. Second we calculate the scale factor of each color component according to the rough reference white point and the scale factor of each color component accord- ing to the precise reference white point. The rough refer- ence white point defined as (Rr,Gr,Br), where, Rr, Gr, and Br are respective component values for each color component and Yr is the luminance value of the rough reference white point. It is formulated as shown in Equa- tion 1.5.
r r
r
r
R G B
Y = 0 . 299 * + 0 . 587 * + 0 . 114 *
The scale factors of red, green, and blue component values are defined as Rrgain,Grgain,Brgain. Thus individually, we can easily compute the scale factor of each color compo- nent according to the rough reference white point as shown in Equation 1.6.
.
r r rgain
r r rgain
r r rgain
B Y B
G Y G
R Y R
/ / /
=
=
=
On the other side, the precise reference white point of an image is defined as (Rp,Gp,Bp) where, R ,p Gp and Bp are respective component values for each color component.
The luminance of the precise reference white point is defined as Yp. It is formulated as shown in Equation 1.7.
p p
p
p R G B
Y =0.299* +0.587* +0.114*
We could easily compute the scale factor of each color component according to the precise reference white point as shown in Equation 1.8.
p p pgain
p p pgain
p p pgain
B Y B
G Y G
R Y R
/ / /
=
=
=
Those scale factors that we calculated above may have the extreme values (ultra-high or ultra-low). By using these scale factors if we crudely scale the component values then
(1.6)
(1.7)
(1.8)
Compute rough and precise reference white pixels rates (Rrough and Rprecise).
rough
rough P
R ≥
precise precise P
R ≥ Rprecise≥Pprecise
Ma is set to 0.
Ma is set to 1.
Ma is set to 2.
Yes
Yes No
Yes No No White balance
judging unit
Fig. 1.2: The flow chart of the white balance judging unit.
White point detecting unit
White balance adjusting unit
(1.5)
we get strange results. To overcome this problem, we can use a sigmoid function to relax the extreme scale factor toward an acceptable value. It is formulated as shown in Equation 1.9.
4 . 0 )) 25 . 1 tanh(
1 (
* 05 .
1 + − +
=
X
Y
where, X is the original scale factor and Y is the ad- justed scale factor. Second, select scale factors of the im- age according to the adjusting mode value. When Ma is equal to 0, the white balance adjustment is stopped. When Ma is equal to 1, we select the minor between (Rrgain,Grgain,Brgain) and (Rpgain,Gpgain,Bpgain) as the actual scale factors. When Ma is equal to 2, we select (Rpgain,Gpgain,Bpgain) as the actual scale factors. Third, scale each color component according to its respective scale factor.
Fig.1.3 The flow chart of the white balance adjusting unit.
4. EXPERIMENTAL RESULTS
We are interested in the results of distinct white balance approaches and the differences between them, so we take those test images, under the different light sources and simulated them with gray world method, advanced gray world method, fuzzy rules method, perfect reflector me- thod, percentile perfect reflector method, the function of automatic balance in PhotoImpact and the new white balance method. The associated light sources used in our experiment with its color temperature are listed below in Table 1.
Light Source Day- light
Cool White
INCa Horizon
Color Temp.
(Kelvin) 6575 4289 2884 2264
Table 1 Color temperature of different light sources.
The test image of GretagMacbeth ColorChecker in differ- ent light sources and corresponding results after applying new white balance method. (See Figure 2.)
Daylight
Original image taken Image, after applying New in Day-light White Balance Method:
(Rroug=1, Rprec=0.04, Ma=1)
Cool White
Original image taken Image, after applying New in Cool White White Balance Method:
(Rrough=0.99, Rprecise=0.11, Ma=2)
INCa
Original image taken Image, after applying New in INCa White Balance Method:
(Rrough=0.96, Rprecise=0.03, Ma=1)
Horizon
Original image taken Image, after applying New in Horizon White Balance Method:
(Rrough=0.99, Rprecise=0.11, Ma=2)
Cool White’
Original image taken Image, after applying New in Cool White White Balance Method:
(Rrough=0.96, Rprecise=0.16, Ma=2)
(1.9)
C om pute scale factors according to the rough reference w hite point (Rrgain, Grgain, Brgain)
≠0 Ma
≠1 Ma
S cale the com ponent values according to its respective scale factor.
Stop N o
Yes
Yes
N o W hite balance
adjusting unit
W hite balance judging unit
Finish
Select the m inor am ong (Rrgain, Grgain, Brgain) and (Rpgain, Gpgain, Bpgain) as scale factors of the im age.
U se (Rpgain, Gpgain, Bpgain) as scale factors of the im age.
C om pute scale factors according to precise reference w hite point (Rpgain, Gpgain, Bpgain).
Fig. 2 Visual results in the different light sources.
As we are interested in the results of distinct white balance approaches and the difference between them so here we take those test images and simulate them with gray world method (GWM), advanced gray world method (AGWM), fuzzy rules method (FRM), perfect reflector method (PRM), percentile perfect reflector method (PPRM), the function of automatic balance in PhotoImpact (UPI), and the new white balance method (NWBM). Visual results of two test images (original image and image after applying new white balance method) are shown in Figure 2. In addi- tion to visual verification we also want to quantify the re- sult of different white balance approaches. Unfortunately, there does not exist a proper quantitative tool for white balance. Here we propose a simple evaluative method. We use a small window (a 11X11 pixel block) inside each col- or patch in the bottom of the GretagMacbeth ColorChecker and calculate the average of those pixels within the win- dows to get the RGB values of the achromatic color patches and then we convert the achromatic color patches from the RGB color space to the YCrCb color space. We calculate the sum of those chromaticity values ( 2 2
b
r C
C + ) of the achromatic color patches in the image and use it as the evaluation criterion for white balance. Table 2 shows the evaluative results of these seven methods for the test images shown in Figure 2.
Table 2. The evaluative results of seven methods for the test images.
5. CONCLUSION
From the experimental results we find that the traditional white balance methods can partially solve the problem of adjusting the white balance and it is hard to get a better re- sult by purely applying the traditional white balance me- thods to digital cameras. However, in our present study we have used the advantages of traditional white balance me-
thods, such as the white balance method based on the Gray World assumption that offers us the information about the light cast over the whole image, while the white balance method based on the Perfect Reflector assumption provides us a rough methodology to find out the reference white point from an image. Moreover, the color characteristics in the YCrCb color space help us to precisely look for the ref- erence white point from an image. In addition to detecting the reference white point from an image and adjusting white balance accordingly, we must deal with the problem that whether or not the white balance adjustment is really required. If we ignore this problem and crudely adjust white balance, we will destroy the consistency of colors in the image and get strange results on certain situations. In this paper the proposed approach not only mixes the ad- vantages of the traditional methods to form a new white point detecting method but also carefully copes with the problem of whether or not we must perform the white bal- ance adjustment. The simulation result shows that the pro- posed method has the best results in both visual results and objective evaluation
6. ACKNOWLEDGEMENT
This research was supported by the National Science Council of Taiwan, R.O.C., under Grants NSC 91-2711-3-319-001-SP-24 and NSC 91-2212-E-002-087, by the EeRise Corporation, EeVision Corporation, Tekom Technologies, Jeilin Technology, IAC, and Delta Electronics.
7. REFERENCES
[1] R. C. Gonzalez and R. E. Woods, Digital Image Processing, Addison Wesley, Reading, MA, 1992.
[2] M. Fedor, “ Approaches to Color Balancing,”
PSYCH221/EE362course project, Department of Psychology, Stanford University, 1998.
[3] Y. C. Cheng, W. H. Chan, and Y.Q. Chen, “ Auto- matic White Balance for Digital Still Camera,” IEEE Transactions on Consumer Electronics, Volume 41, pp. 460-466, 1995.
[4] J. Chiang and F. Al-Turkait, “ Color Balancing Ex- periments with the HP-Photo Smart-C30 Digital Camera,” PSYCH221/EE362 course project, De- partment of Psychology, Stanford University, 1999.
[5] A. V. Durg and O. Rashkovskiy, “ Global White Point Detection and White Balance for Color Im- ages,” U.S. Patent #6069972, 2000.
Evaluative values Test images
Original Manual GWM AGWM FRM PRM PPRM UPI NWBM
Daylight 79.38 11.73 53.25 54.81 53.58 21.67 43.54 34.84 13.31 Cool White 76.8 6.3 45.36 44.86 45.67 15.82 37.81 49.05 7.38 INCa 161.16 13.99 25.64 24.01 82.08 28.08 16.99 45.38 17.35 Horizon 202.71 11.94 22.45 23.51 101.94 34.5 26.15 54.2 20.14 Cool White' 77.85 5.99 22.4 22.21 43.74 39.78 23.03 81.29 6.75
