Juian-Shian Chu(

Automatic Exposure with Center/Focus Metering

Juian-Shian Chu(朱峻賢) and Chiou-Shann Fuh(傅楸善) Department of Computer Science and Information Engineering

National Taiwan University, Taipei, Taiwan

[email protected] and [email protected]

ABSTRACT

Automatic exposure (AE) controls the light intensity of pictures. In real life, the scenes are too many and diverse to analyze. The common AE metering methods do not work for all scenes, so Nikon’s Automatic Multiple Pattern (AMP) metering algorithm [2] solves these problems. C. C. Yu’s AE metering algorithm [3] solves some problems of AMP method with fuzzy control, subject growing, and multiple reference tables. Our new metering algorithm utilizes the focus information and is simpler and smaller than AMP and C. C. Yu’s algorithms.

Moreover, C. C. Yu proposes an optimal exposure selector (OES) to choose one of pictures with different exposures and to satisfy people preference. We modify the OES factors to achieve precise results.

1. Introduction

The objective of AE is to achieve a good balance of exposure in image. AE contains metering algorithm and exposure control. First, metering algorithm estimates the amount of incident light on sensor and calculates appropriate exposure value (EV). Later, exposure control adjusts three related devices, aperture diameter, shutter speed, and sensor sensitivity based on exposure value.

2. Previous Work

Automatic Multi-Pattern (AMP) solves this problem by deciding a suitable AE metering algorithm for each different scene. In [2], researchers construct a lookup table based on a visual assessment of tens of thousands of pictures, computer analysis of the relationship between brightness patterns, optimum contrast, human evaluation, and so on. The next two sections will briefly introduce the AMP process.

2.1.1. Scene Classification

The AMP table classifies the scenes into 20 classes. In each weather and contrast condition, AMP judges the different metering methods by brightness value of center region.

Figure 1 shows the segmentation of AMP sensor.

I will analyze the AMP pattern classification in Section 3.2.2, for more detailed information about pattern classification, see [2].

Fig. 1: The five-segments sensor.

2.1.2. Cut Process

This process is required to render optimally exposed pictures especially in unfavorable lighting situations; it involves undertaking the extreme bright light cut and extreme dark light cut process, as follows:

2.1.2.1. Extreme Bright Light Cut Process The brightness value of only up to two segments can be cut. In case three or more segments exceed EV 16-1/3, the third and following segments are automatically regarded as being EV 16-1/3.

2.1.2.2. Extreme Dark Light Cut Process If the given scene has extremely low brightness values, it will be cut out. When the number of segments to be cut exceeds three, the value of the third segment is regarded as being the dark-light limit and included in the exposure computation. However, unlike the bright-light value cutout process, the brightness value of the fourth and following segments will be cut out.

2.2. AMP with Fuzzy Control 2.2.1. Subject Growing

In AMP method, subject is assumed to be in the center. In photography, subject may not always locate in the exact center, but subject will locate in the center near left, right, top, or bottom. Moreover, if the center region also includes other insignificant information, the subject information will be diluted. Subject information is very important in scene classification of AMP. If the subject is not located in the center, we will take the wrong pictures naturally. Subject growing solves this problem.

In [3], it assumes the subject always crosses the center region, and it sets the gray region as initial subject, shown in Figure 2.

Fig. 2: The gray region is subject initially.

Fig. 3: The gray regions are default subject.

After initialization, we extend subject with the new regions, adjoining gray region, by considering the following conditions of new region and subject region:

(1). The difference of luminance is less than 2.0 LV.

(2). The difference of hue is less than 30 degrees.

(3). The difference of saturation of new region is small than 0.5.

If new region conforms to three conditions, program sets it as subject region. In the end of subject growing, if the number of detected subject regions is less than 6 or more than 20, the subject is defined as default setting as the gray regions in Figure 3.

After subject growing process, we can get more precise subject regions and use them to calculate the contrast between subject and background for use of multi-reference table later.

2.2.2. Fuzzy Control

When the measured light intensity is just near the threshold between two weather or contrast conditions, AMP metering system will changing the metering algorithms with the changing of light. If the changing is fast, we will see the display screen is flickering, and we feel

jerky and uncomfortable.

Fuzzy control is the mechanism that simulates the undefined regions between defined regions. It helps to smooth the transition between each condition. C. C. Yu uses fuzzy control to smoothly transition between different weather, contrast, and subject conditions.

2.2.3. Multiple Reference Tables The final improvement is the multiple reference tables. Because we have more precise subject information after subject growing, we can analyze the relation between subject and background in detail. We use three AMP tables for each subject condition, as shown in Tables 1, 2, and 3.

Metering Methods Description:

(1). AV: Average metering.

(2). CW: Center-weighted metering.

(3). BL: Dark-light metering

Besides of center region, we consider the darker regions for enhance the detail in dark.

(4). BH: Bright-light metering

Besides center region, we consider the brighter regions to enhance the bright detail.

For more detail of process, see [3].

Table 1: Reference table for bright subject.

Table 2: Reference table for moderately bright subject.

Table 3: Reference table for dark subject.

3. Center/Focus Metering Algorithm

Although subject growing process increases the probability to guess the correct subject location, the result of subject growing is not always correct. With the subject assumed to be in center, the overall results will not be bad, because at least we emphasize the center regions where subject locates in most cases.

As described in Section 2.2.1, subject growing conditions include:

(1). Light intensity difference (2). Hue difference

(3). Saturation level

They have problems in the following cases:

(1). When the light intensity difference between subject and background is small.

(2). When the subject is colorful enough, the different parts of subject would have different hue..

(3). When the subject is monochromatic, the saturation level will ignore the subject.

(4). If the initial region contains two or more objects, one of them may be the subject, but subject is smaller.

3.2. Center/Focus Metering Algorithm We propose a Center/Focus metering algorithm containing two things:

(1). Subject detection

(2). New AE metering function

Subject detection does not set center as subject region. We consider the focus information to decide the subject is in the center, left, or right. Moreover, we propose a new AE metering function to replace AMP table. Without looking up table and saving table, the new AE metering function is simple and small.

First, metering algorithm receives light information from sensor and checks the focus information. If focus information is available, we use subject detection to choose the subject region, otherwise, we use default center region.

Finally, with subject region, we calculate the EV by new AE metering function. Next sections will describe each part in detail.

3.2.1. Subject Detection

We use the focus information, if available, to detect subject location. Because the AF focuses on the subject with high probability, we optimize the exposure quality of subject region, containing high focus value. Even if the AF does not focus on the correct subject, the focused region is still clearer than others, and it is reasonable to optimize exposure quality for clear regions.

Fig. 7: Center/Right/Left region of sensor.

First, we get the focus values of each segment from AF. We find out ten segments, which contain maximum focus values and their light intensities are less than a luminance threshold. Next, we calculate (c, l, r), the numbers of maximum segments in each region.

Figure 7 shows three gray regions of possible subject locations. The rules of region decision are:

(1). Center region: (c>4) or ((c>l) && (c>r)).

(2). Left region: (1). fails and (l>c) && (l>r).

(3). Right region: (1). fails and (r>c) &&

(r>l).

(4). Default region: otherwise.

The rules of region decision consider three cases:

(1). The center region still has the highest probability to contain subject. We detect left or right region only when center region has bad focus and light intensity.

(2). The right or left region needs rigorous condition to be subject. It should contain more focus regions than others and center fails in Case one.

(3). If above conditions both fail, then we choose center as subject region.

Finally, subject detection process passes subject region information to AE metering function.

3.2.2. Analysis of Scene Classification I analyze the AMP scene classification and C. C. Yu’s multiple reference tables to find rules

for better metering results by flowchart in Figure 8.

Fig. 8: Brief flowchart of AMP rules

Cut process works when scenes contain extreme bright or dark regions. In this case, dark-light metering is used when center region or total scene is dark, and average metering is used otherwise. If the scene contrast is low, then center-weighted metering is used. If the scene contrast is high, besides the center region, rules take more regions to metering, taking another bright regions when center is bright, taking another dark regions when center is dark. In other words, when the contrast is high, we take another region whose light intensity is judged by the light intensity of center.

With above analysis, we can use subject (center or focus region) and contrast to find suitable exposure, as described in Equation 1 for case without cut process.

regions g

interestin ratio

Contrast

Subject ratio

Contrast Exposure

_ _

) _ 1

(

×

+

×

−

=

(1)

3.2.3. AE Metering Function

First, function gets light information and subject region to calculate average luminance of each region. As in Figure 7, for different subject

regions, function calculates different Regions 1~8 and subject region. Second, it sorts Regions 1~8 and calculates the contrast by subtracting min region from max region. The contrast ratio is calculated by the transformation in Figure 9.

Contrast is concerned when it is larger than contrast threshold (3LV), and maximum contrast ratio is 40% to protect the importance of subject information. Finally, with the value of subject, contrast ratio, interesting region, and cut process, function calculates the exposure by Equations 1.

Fig. 9: Contrast ratio function.

3.2.4. Experiments 3.2.4.1. Subject Detection We consider two issues:

(1). Subject is not in the center, and center region does not have subject candidate obviously. (Figure 10)

(a) Without focus information.

(b) With focus information.

Fig. 10: Subject is not in the center.

(2). Subject is in the center, but left or right region has a high contrast object which has high focus values. (Figure 11)

(a) Without focus information.

(b) With focus information.

Fig. 11: Subject is in the center and high contrast object is near.

3.2.4.2. AE Metering Function

We experiment on TI DM320 platform and take backlit scene (Figure 12.) and high contrast scene (Figure 13.) as examples.

(a) Center-weighted metering.

(b) Average metering.

(c) Bright-light metering.

(d) Dark-light metering.

(e) New metering.

Fig. 12: Backlit scene.

(a) Center-weighted metering.

(b) Average metering.

(c) Bright-light metering.

(d) Dark-light metering.

(e) New metering.

Fig. 13: High contrast scene.

3.2.5. Conclusion

We can see the good result from experiments. The purpose of AMP is to choose the appropriate metering methods in all scenes and to prevent using bad metering methods to corresponding scenes. Our new metering function result the same exposure as AMP, and our function also prevent bad result in special scenes, such as backlit and high contrast scenes.

4. Optimal Exposure Selector

Bracketing is a technique used to take a series of images of the same scene at a variety of different exposures that “bracket” the metered exposure. In general AE bracketing method, camera will automatically take 3 or 5 frames with exposure settings between 0.3 and 2.0 EV differences. It is useful when users are not sure exactly how the image will turn out or are

worried that the scene has a wide dynamic range.

When we take pictures using AE bracketing, we next select our favorite one from bracketing pictures. OES objectively selects one favorite picture from AE bracketing pictures, which differ in exposure only. About the benefits, OES saves the effort that users need to select, and saves the memory space that bracketing pictures occupy more spaces.

To select the favorite picture, researchers analyze the factors of person perceptions, such as brightness, contrast, and colorfulness. The detailed OES process is described at next paragraph.

4.2 Optimal Exposure Selector

The OES process includes two parts, subject growing and decision making.

Fig. 14: 8x8 segments of picture.

In subject growing, picture is divided into 8x8 segments as shown in Figure 14 and we select the gray region as initial subject location first, if the light intensity of gray region is smaller than 220. Otherwise, program will shift the gray region to find another optimal 2x2 region in black regions. Second, subject regions are extended based on the gray region according to hue and saturation, and subject regions are found out.

In decision making, the researcher analyzes the factors that are related to the person perception and are influenced by exposure control. The following three factors are considered:

(1). Intensity Mean

(2). Standard Deviation and Entropy (3). Colorfulness

Moreover, we propose two factors to make result more precise.

(4). Detail

(5). Non-Saturating Ratio

1.3. Experiments

We take 30 sets of AE bracketing picture from Sony F828 and Fujifilm F601. The image sets contain the scenes of outdoor, indoor, landscape, portrait, and so on. Table 4 show the matching ratio of new OES factors and C. C.

Yu’s results

Item Matching Mismat

ching

Ratio

Intensity Mean

24 6 80%

Intensity St.

Dev.

10 20 33%

Colorfulness 6 24 20%

Detail 17 13 57%

Non-Saturat ing Ratio

21 9 70%

New OES Result

26 4 87%

C. C. Yu’s OES Result

19 11 63%

Table 4: Matching ratio of each factor, new OES result, and C. C. Yu’s OES result.

Fig. 15: Experiment Indoors and colorful.

Fig. 16: Experiment Outdoors.

As shown in experiments, the modified OES can select the better exposed pictures according to details and non-saturating ratio.

5. Conclusion and Future Work

For future work, we have some suggested research directions:

(1). Standard procedure of AE test

Standard procedure of AE test is very important to verify AE performance, and we still prefer to analyze the overall scene conditions which influence AE.

(2). Color compensation of AE metering Light sensitivities of different colors are different for human perception. If we can get the color information correctly, it greatly improves AE quality.

6. Reference

[1] Z. Rahman, D. J. Jobson, and G. A.

Woodell, “Properties and Performance of a Center/Surround Retinex,” IEEE Transactions on Image Processing, Vol. 6, No. 3, pp. 451-462, 1997.

[2] K. Rockwell, “Automatic Multi-Pattern Metering,”

http://www.kenrockwell.com/nikon/matrix 01.htm, 2000.

[3] C. C. Yu, “Automatic Exposure with Fuzzy Control,” Master Thesis, Department of Computer Science and Information Engineering, National Taiwan University, 2004.