Overlapping/ Hair-Reflecting/ Beard/ Clothes False Region

region, the covering region is removed. To remove hair reflection regions, first, a horizontal gray level projection histogram (see Fig. 2.11(b)) is created. Then, the first peak location from the top of the projection histogram is defined as an upper line which usually indicates the forehead. Each eye-like region intersecting the upper line is removed. For the beard/clothes class, we first define a bottom line with distance h/2 from the upper line, where h is the distance between the upper line and the bottom of the skin region. Then all eye-like regions below the line are removed. Fig. 2.11(d) shows the result of applying the removing procedure to Fig. 2.11(a).

The first peak

(a) A bounding rectangle of a candidate skin region.

(b) The horizontal projection histogram of the bounding rectangle in (a).

h/2

Upper line

Bottom line h

(c) The detected upper and bottom lines.

(d) The result after applying the removing procedure presented in Section 2.2.3.1 to (a).

Fig. 2.11. The result of applying the false eye-like removing procedure.

2.2.3.2 Eye Position Refining And Isolated False Region Removing

For a non-rotated human face, an eye pair should be located at a near-horizontal line. Thus for an eye-like region, if we can not find another region at its left or right, then this region is called isolated region and should be removed. However, under various environments, a true eye may not be located at the center of the extracted eye-like region (see Fig. 2.12(a)), we are then unable to know where the eye is. In order to treat this problem, we provide an eye position refining method to locate the true eye position in the eye-like region. First, we classify the frontal images of HHI

into three classes according to the lighting condition: normal, high light and low light.

Next, one image is selected randomly from each class. Considering the effect of glasses, the normal class has two images selected with/without wearing glasses. For each image, three templates are extracted (see Table 2.1). An eye area including eyelid is taken as the first template. In the first template, an area only containing eye is then taken as the second template. In the second template, an area only containing eye ball is taken as the third template. After obtaining the twelve templates, we use each template to mask each eye-like region by sliding the template pixel by pixel in the region to find the best matched area for each class. Covariance is used to measure the similarity. If an eye-like region with the located eye center does not lie in the true eye, we take three new templates from the image containing the region using the previous method. The procedure is repeated until all eye-like regions are processed. After the procedure is finished, we find that nine new templates (see Table 2.1) belonging to three new classes are extracted. These three new classes are left-side (the eye sees the left side), right-side (the eye sees the right side) and bias lighting. Since the images in HHI have different face sizes, the extracted templates will also have different sizes.

After we make the eye templates, a mentioned similarity matching procedure is adopted to find the best matched area for testing image. We use the similarity function is defined as follows.

n are the mean values for each area. Thus, we can obtain six best matched areas for each eye-like region. Then the center of the best matched area with the highest Cb/Cr value (the most possible non-skin area) is considered as the best eye center in the eye-like region (see the red points in Fig. 2.12(b)). Finally, all eye-like regions are refined to be a wh region centering at the best eye center. In the dissertation, we set w to 30, h to 10 (see the green rectangles in Fig. 2.12(b)). After an eye-like region is refined, we judge if it is an isolated region. If yes, the region will be removed. Fig.

2.13(b) shows the result of removing those refined isolated false regions.

Table 2.1 The templates for six eye classes.

Eye Type Template Images

(a) An example to show an eye not located at the center of the eye-like region.

(b) The result of applying the eye position refining procedure to (a).

Fig 2.12. The result of applying the eye position refining procedure.

(a) An example to show isolated regions.

(b) The result of removing the isolated regions in (a).

Fig. 2.13. The result of removing isolated false regions.

2.2.3.3 Forehead-hair false regions removing

Before describing the proposed forehead-hair false region removing algorithm, we will define the bounding rectangle for a face. Let R be the minimum rectangle containing all eye-like regions, then R is expanded in the vertical direction to form a square rectangle (see Fig. 2.14). The square rectangle is defined as the bounding rectangle of the face.

(a) The minimum rectangle containing all eye-like regions.

(b) The expanding result of (a).

Fig. 2.14. An example to illustrate the bounding rectangle of a face.

In general, pixels in hair or shadow regions caused by a bias lighting have lower gray levels than pixels in a face. Based on this property, we find that if the bounding rectangle of a face contains a part of lower gray region (hair or side shadow), its gray-level histogram, h(x), will be a bi-model distribution; else the histogram will be a uni-model distribution (see Fig. 2.15). To distinguish these two models, an average probability horizontal line (APL) Y = T with



 number of the intersection line segments between the APL and the histogram is two, a bi-model distribution is identified (see Fig. 2.15(a)). Otherwise, if only one line segment exists, a uni-model distribution is identified (see Fig. 2.15(b)). For a face bounding rectangle, those eye-like regions appearing in the hair area should be removed. Based on the above discussion, a procedure is proposed to remove forehead-hair false regions. To identify the hair part, we set the initial bi-level threshold t0 as average gray value of the face rectangle for a bi-model histogram. If the face is a uni-model, we set the initial threshold t0 as

 less than t0 are labeled as black pixels. However, the threshold may be improper such that over-segmentation or under-segmentation (see Fig. 2.16) will occur. To treat this problem, the threshold t0 will be adjusted according to the following procedure. After applying the bi-level thresholding, if no two eye-like regions contain over Oth (30%) black pixels, an over-segmentation is detected (see Fig. 2.16(b)). The t0 should be adjusted to be larger. The adjustment rule is to take the nearest right valley to the current t0 on the histogram as a new threshold t0. If there is no valley is found, then the t0 will be adjusted by increasing a threshold value (Nth), here Nth is set to be 5.

Bi-model (a) A bias lighting face and its corresponding histogram.

Uni-model (b) A normal lighting face and its corresponding histogram.

(a) Original image. (b) An over-segmentation obtained after applying the threshold process to (a).

(c) Original image. (d) An under-segmentation obtained after applying the threshold process to (c).

Fig. 2.16. Two examples to show the over-segmentation and under-segmentation.

On the other hand, if all eye-like regions are located in a big black area, an under-segmentation is detected (see Fig. 2.16(d)), t0 will be refined to smaller. The adjustment rule is to take the nearest left valley to the current t0 on the histogram as a new threshold t0. If there is no valley was found, then the t0 will be decreased by Nth. The above procedure is repeated until no over or under segmentation occurs. In order to prevent a dangling situation, any used valley will be removed to avoid reusing.

Then the bi-level thresholding operator with the final threshold is applied to identify the dark regions in the face bounding rectangle (see Fig. 2.17(b)). If a dark area is hair, it should be large in the face bounding rectangle. If a dark area is iris, it should be small. So, for a dark region with area larger than

4

1 face bounding rectangle, we

consider it as a hair region, and all eye-like regions in the hair region are removed. In addition, if a dark region has the ratio of the height over the width larger than a threshold 2.5, we will remove the region. On the other hand, since the iris and eyelids have lower gray level than the other part of an eye, if an eye-like region contains less than 3% dark pixels, the region is also removed. Note that any eye-like region locating at the shadow part will also be removed by the above procedure (see Fig.

2.17(c)). To resume these regions, we use a previously mentioned fact if an eye is found, then the other eye should be found at its right side or left side. For each remaining eye-like region, R, we first draw three lines, L1, L2 and L3, passing through the center of R. L1 is a horizontal line, the angles between L1 and L2 (or L3) is a predefined angle A_th (here, 10^ is used) (see Fig. 2.18(a)) for a bi-model histogram face. Then a bounded range, BR, is defined to be the area bounded by L2 and L3. Based on the bounded range, all eye-like regions in the bounded range removed by the forehead-hair false region removing procedure will be got back (see Fig. 2.18(b)). Now the remaining eye-like regions are used to form a new bounding rectangle (see the dotted rectangle in Fig. 2.18(b)) which will be used to determine the true horizontal eye line.

(a) The eye-like regions before applying the removing procedure.

(b) The result of applying the bi-level threshold to (a).

(c) The result of removing the forehead - hair false regions in (b).

Fig. 2.17. The result of applying the forehead-hair false region removing procedure.

100

 100

 L2 L1 R L3

w

w/4

(a) The bounded ranges for resuming the removed eye-like regions in the shadow area of a face.

(b) Three eye-like regions (marked by white rectangles) got back.

Fig. 2.18. An example to explain the eye-like region resuming procedure.