Metering Method

(1)

Automatic Exposure with Fuzzy Control

Chi-Cheng Yu (游啟昌) and Chiou-Shann Fuh (傅楸善) Department of Computer Science and Information Engineering

National Taiwan University, Taipei, Taiwan

[email protected] and [email protected]

ABSTRACT

In this thesis, we will present an automatic exposure method based on Nikon’s method [9] and AMP (Automatic-Multiple-Pattern) [7]. Our method will add the fuzzy control into the AMP to get smooth transition effect and subject growing function to classifies subject and background to get a better exposure

.

We also implement an easy evaluative function to estimate what is a well exposed picture in the same scene with different exposure parameters.

The experiment results show that our evaluative function selects the best exposed picture similar to most people’s selection.

1. INTRODUCTION

Nikon disclosed a patent “Three-Dimensional Multi-Pattern Photo-Metering Apparatus” on August 21, 1990 [9]. The sensor is divided into five areas as shown in Figure 1.

Figure 1: The CCD is divided into five segments.

The method based on the professional photographers’

experience under various weather conditions. AMP [7] is similar to [9], however, it does not take flash into account and it is only two-dimensional. AMP classified weather into five different conditions (W1~W5), and contrast is also classified into five conditions (C1~C5). We can construct a two-dimensional reference table as shown in Figure 2.

W1 W2 W3 W4 W5

C1 C2 C3 C4 C5

Metering Method

Figure 2: Reference table, W1~W5 is from bright weather to dark weather, and C1~C4 is from flat contrast to strong contrast. (W1: Scene involves the sun, W2: Outdoors in fine weather, W3: General outdoor scenes, W4: Sunset/twilight scenes, W5: Dark night scenes. C1: Flat contrast, C2: Some contrast, C3:

Medium contrast, C4: Strong contrast, C5: Very strong contrast

When weather, contrast, and distance information are decided, a mapping metering method is selected. In this paper, we extend the concept of [9] and [7], and apply fuzzy control and subject growing function to get a better result.

2. PROBLEMS OF AMP AND IMPROVEMENT 2.1 SUBJECT CONDITION

In [7] and [9], the CCD is divided to five segments as shown in Figure 1. Center segment is treated as the subject. Is it always this way? When we photograph, our subject is usually in the center, but not always.

Sometimes, it may slight near right, left, top or bottom.

[7] and [9] fixes the subject in center, it is suitable for most conditions, however, when subject shifts (not in the middle center), the contrast between foreground

(2)

and background may wrong.

2.2 SMOOTH TRANSITION

AMP [7] does not mention how they handle the transition between two weather or contrast conditions, if our table is fine enough to cover all conditions, there is no problem for smooth transitions. However, if we do not have a fine reference table, apply fuzzy function to achieve smooth transition is necessary.

2.3 IMPROVEMENT OF AMP

In this paper, we define three reference tables for different foreground and background contrast. Fuzzy control is applied to achieve smooth transition and subject growing function is implemented to guess where subject is.

3. OUR AUTOMATIC EXPOSURE METHOD Unfortunately, most metering methods are hardware dependent. For example, the three-dimensional matrix-metering [9] need special lens with a chip to detect the subject’s distance. In this paper, we focus on the metering method without additional hardware support. We will extend the Nikon’s AMP metering method and append fuzzy control and subject growing function to smooth the transition between conditions and guess the subject.

3.1 FUZZY WEATHER

Figure 3.3: Membership functions of different weather conditions and x-axis is LV (Light Value), and y-axis is probability.

The membership functions of the five different weather conditions are shown in Figure 3.3. Red line represents W1; blue line represents W2; green line represents W3;

cyan line represents W4; and magenta represents W5. If BMAX (luminance of the brightest segment in Figure 3.1)

>= LV 16.3, it means the probability of W1 is 1. If LV 13.5>=BMAX >= LV 12.5, it means the probability of W2 is 1. If LV 9.5>=BMAX >= LV 8.5, it means the probability of W3 is 1. If LV 4.5>=BMAX >= LV 3.5, it means the probability of W4 is 1. If LV BMIN (luminance of the darkest segment in Figure 3.1) <= LV 3.5, it means the probability of W5 is 1. When BMAX or BMIN does not fall into the range that exactly belongs to any one weather condition, in other words, they fall into the gradient line segments (gray zones). Camera will take the combination of the two different weather conditions according to their probabilities. For example, if program selects bright table (Figure 3.6) as reference table and BMAX falls into the gray zone between W1 and W2. If probability of W1 is 0.3 and probability of W2 is 0.7, the fuzzy weather will return 0.3*(AV or BL) +0.7*CW (assume contrast is flat). The priority of the five weather conditions is: W1 > W5 > W2 > W3 > W4. This is because W1 is extremely bright weather condition and it may lead to under-exposure. W5 is extremely dark and it may result in over-exposure.

3.2 FUZZY CONTRAST

Figure 3.4: Membership functions for different contrast

(3)

levels (red: C1, green: C2, blue: C3, cyan: C4, magenta: C5) and x-axis is LV (Light Value), and y-axis is probability.

Let C=BMAX-BMIN, (BMAX and BMIN is the maximum and minimum LV of the five segments shown in Figure 3.1)

Probability of very strong contrast is 1 when C>=5.5.

Probability of strong contrast is 1 when 4.5>=C>=3.5.

Probability of medium contrast is 1 when 3>=C>=2.

Probability of some contrast is 1 when 1.5>=C>1.

Probability of flat contrast is 1 when C<=1

When C falls into gray zone, it is similar to fuzzy weather.

3.3 FUZZY SUBJECT

Figure 3.5: Membership functions for different subject conditions (red: moderately bright subject, green: bright subject, blue: dark subject).

When we photograph, we usually have a subject to emphasize. In this paper, we assume subject is not fixed in center. We construct three reference tables for different subject conditions, bright subject, moderately bright subject and dark subject. These tables are shown as Figure 3.6, Figure 3.7, and Figure 3.8:

Figure 3. 6: Reference table for bright subject.

Figure 3.7: Reference table for moderately bright subject.

Figure 3.8: Reference table for dark subject.

BL: Dim light metering BH: Bright light metering CW: Center weighted AV: Average metering

After introducing the three fuzzy models (fuzzy subject, fuzzy weather, and fuzzy contrast), it shows when the light condition of a scene falls into gray zones of all the three models. We need to take a combination of eight

(4)

metering methods.

3.3 SUBJECT GROWING

When photograph, subject is what we want to emphasize.

In Figure 3.1, subject is assumed in the center. In fact, when we photograph, subject is not always in the center.

Maybe subject is in the center near left, right, bottom or top, therefore, the assumption is incorrect. In this paper, subject information is important for us to support correct contrast between foreground and background. In AMP, some scenes may fool it, because subject location is fixed.

In the section, we assume parts of the subject would in the center as Figure 3.9:

1

Figure 3.9: In initial state, segment 1 is assumed part of subject (In our experiment camera, CCD is divided into 25 segments)..

In initial state, only segment 1 is the subject. We extend subject by combine the segments (connected component of segment 1) according to the luminance. If the differences of luminance of other segments and segment 1 are less than 2.0 LV, then the segments are appended to subject. If the number of member segments of the growing connected component is less than 6 or greater than 20, the subject is defined as default setting as shown in Figure 3.1.

After subject growing process, we can get a vague foreground and vague background, therefore, more precise contrast between foreground and background could be obtained. Therefore, a suitable reference table could be selected.

4. OPTIMAL EXPOSURE SELECTION If there are two or more pictures only different in exposure, and which one is preferred by most people?

There is no tool for the decision making. It is intuitive for people. However, different people may prefer different pictures. In this section, we develop an easy method for the decision making according to histogram distribution, hue, saturation, intensity, and entropy.

4.1 IMAGE SEGMENTATION AND IMAGE INFORMATION RETRIEVAL

Picture is divided into 8x8 segments as shown in Figure 3.10. Subject is assumed to contain cyan region.

Few people photograph too bright and flat subject.

Thus, if intensity of cyan region is greater than 220, program will shift the cyan region to find an optimal 2x2 region as the new subject in yellow region. When cyan region is found, the growing process is similar to subject growing in Section 3.3

Figure 3.10: Subject is assumed to contain cyan region (In computer simulation, we divide a picture into 8x8 segments).

Step 1: Subject is extended based on the cyan region according to hue and CrCb distance. After Step 1, the rough subject region would be marked.

Step 2: Calculate means, standard deviations, hue, saturation, intensity, and intensity entropy of subject

(5)

region and global image.

Mean:

We know the closer the mean value is to 128, the higher probability people may prefer it. If the mean value is near 0 or 255, the picture may be under-exposed or over-exposed. In this thesis, we divide the range of 0 to 255 into 12 zones. Figure 3.11 shows zone numbers and their relative gray values and points and Figure 3.12 shows curve of weight function.

Zone 0 1 2 3 4 5 Gray

value

0 48 97 108 120 134

Pts 0 1 3 5 7 9 Zone 6 7 8 9 10 11 Gray

value

149 166 185 206 229 255

Pts 10 6 4 3 1 0 Figure 3.11.: Table of zone numbers and their relative gray values and points.

0 2 4 6 8 10 12

0 1 2 3 4 5 6 7 8 9 10 11

Zone

Weight(Zone)

Figure 3.12: Curve of weight function.

Standard Deviation and Entropy:

The larger the intensity standard deviation, the higher probability of high contrast of the picture may be. The larger the intensity entropy is, the more uniform distribution of gray values is. Histogram equalized images have the highest intensity entropy. We need to take standard deviation and entropy into account at the same time, because they are dependent. The maximum entropy of gray values in an image is 8 and the maximum

standard deviation is 128.

Colorful

Most people like colorful image. In application, YCrCb color space is preprocessed by hardware. Y represents the luminance information. We can distinguish subject is colorful easily from Cr and Cb information.

Figure 3.13: Red (340 degree), green (220 degree), blue (100 degree), cyan (160 degree), yellow (280 degree), and magenta (40 degree) showed on CrCb plane.

The more CrCb distance from the origin, the colorful they are.

4.2 DECISION MAKING

In this section, we will count the points of mean, standard deviation, and HSI. After subject growing, a virtual subject emerges, hence, we have subject mean, standard deviation, HSI, global mean, standard deviation, and HSI. We will put more weight in subject information.

z Subject Mean (SM): R pts.

z Subject Entropy and Standard Deviation (SED): R pts.

z Subject vivid (SH). R pts.

z Global Mean (GM): R/2 pts.

z Global Entropy and Standard Deviation (GED): R/2 pts.

SM=weight(zone of subject). (Figure 3.12)

(6)

GM=1/2*weight(zone of global image).

SED=R/2*(Entropy/8 + Standard_Deviation/127.5).

GED=R/4*(Entropy/8 + Standard_Deviation/127.5).

SH=R*(distance/133).

Total points=SM+SED+GM+GED+SH .

5. EXPERIMENT RESULT

(a) Center weighted (b) Our method Point : (a) 11.449, (b) 16.810, (c) 14.262 Vote: (a) 3, (b) 20, (c) 9.

(c) Reference camera

FIGURE 3.14: Experiment result.

(a) Center weighted (b) Our method Point : (a) 13.199, (b) 15.297, (c) 14.254 Vote : (a) 8, (b) 24, (c) 6.

(a) Center weighted (b) Our method Point : (a) 9.068, (b) 14.013, (c) 11.011 Vote: (a) 1, (b) 29, (c) 5.

(a) Center weighted (b) Our method Point : (a) 9.913, (b) 15.558, (c) 9.526

Vote: (a) 1, (b) 27, (c) 5.

6. CONCLUSION AND FUTURE WORKS We have experimented for various scenes, and some predictable conditions may fool our method. If subject does not contain the middle center segment, and growing function can not work, our method degrades fixed center metering.

6.1 PROBLEMSOF OUR METHOD 1. Assumption for subject location.

We assume subject always contain the middle center segment. This is not always established.

(7)

2. Subject growing criterion:

Subject growing is the key function directing the final exposure. In this thesis, the growing criterion is only to restrict LV (light value) difference. We can not get an accurate extended subject by using only LV information.

In our implementation, we also try to append color criterion for growing function, however, we can not get accurate color information from our camera. Therefore, we suspend the criterion for now due to color information accuracy.

6.2 FUTURE WORK

In future work, 1. color compensation, 2. Dynamic subject assumption, and 3. Append Optimal Exposure Selection (OES) to assist automatic exposure can be enhanced.

7. ACKNOWLEDGEMENT

This research was supported by the National Science Council of Taiwan, R.O.C., under Grants NSC 92-2213-E-002-072, by the EeRise Corporation, EeVision Corporation, Machvision, Tekom Technologies, Jeilin Technology, IAC, ATM Electronic, Primax Electronics, Arima Computer, and Lite-on.

8. REFERENCE

[1] D. P. Curtin, “Digital Photography, The Textbook, a Free On-Line Course” http://209.196.177.41/, 2004.

[2] S. Hayakawa, “Camera Detecting Luminance from a Plurality of Areas,” United States Patent #5,189,460, 1990.

[3] B. Hitchman, “Using the Zone System in the Field”

http://www.apogeephoto.com/mag2-6/mag2-7rh.shtml, 2004.

[4] G. J. Klir and B. Yuan, Fuzzy Sets and Fuzzy Logic:

Theory and Applications, Prentice Hall, Upper Saddle River, N.J., 1995.

[5] S. K. Nayar and T. Mitsunaga, “High Dynamic Range Imaging: Spatially Varying Pixel Exposures,”

Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, South Carolina, Vol. I, pp. 472-479, 2000.

[6] Nikon Corp., “The Nikon F5 Has 1,005 Eyes,”

http://www.nikon.co.jp/main/eng/society/tec-rep/ccd.ht m, 2004.

[7] K. Rockwell, “AMP Pattern Classification Table,”

http://www.kenrockwell.com/nikon/matrix05.htm, 2003.

[8] K. Rockwell, “What are LV and EV,”

http://www.kenrockwell.com/tech/ev.htm, 2004.

[9] T. Takagi, “Three-Dimensional-Multi-Pattern Photo-Metering Apparatus,” United States Patent

#4,951,082, 1990.

[10] G. L. Ward, H. Rushmeier, C. Piatko, “A Visibility Matching Tone Reproduction Operator for High Dynamic Range Scenes,” IEEE Transactions on Visualization and Computer Graphics, Vol. 3, No. 4, pp.

291-306, 1997.