Experimental Results - A NOVEL METHOD FOR HORIZONTAL EYE LINE DETECTION

CHAPTER 2 A NOVEL METHOD FOR HORIZONTAL EYE LINE DETECTION

2.3 Experimental Results

In order to show the effectiveness of the proposed method, we apply the proposed method to HHI face database [19] of 206 images (see Fig. 2.20) and Champion face database [20] of 227 images (see Fig. 2.21). We also collect some other images from Internet, MPEG7 video clips, PIE database [21] to evaluate the performance. These contain face images of different racial persons under different kinds of lighting conditions (such as overhead, side and color lightings). Since one of our main applications of the method is for a drowsy driving warning system and in most cases, a driver’s head position is usually frontal or near profile. Since the head is

rarely in-plane rotated when people are drivning, in dissertation only detect the horizontal eye line and allowed in-plane rotated in Ath (10^) degrees. The system is

implemented by Java language on a laptop with Pentium(R) M processor 1.4 G Hz CPU. In Fig. 2.21, some successful results of applying our method to some faces with different skin colors. The successful results for faces with different poses and eye glasses are shown in Fig. 2.20. Even there is shadow on a face; the eye line can also be detected (see Fig. 2.22).

Fig. 2.20. A part of images from HHI face database with different poses and eye-glasses.

Fig. 2.21. A part of images from Champion face database with different skin colors.

Fig. 2.22. The detection results for persons with shadows and various lighting sources on their faces.

If the detected eye line is located under eye brow and near eyes, we consider it to be a correct detection; otherwise it is an error detection. The tolerance value is the height of an eye-like region. If no horizontal eye line is detected for a face image, we call it a missing detection. Fig. 2.23 shows the successful detection results for a set of images from the Internet, MPEG7 video clips and PIE database images. For HHI database, the correct detection rate is 95.63% and the correct detection rate on Champion database is 94.27%. Fig. 2.24 shows the error and missing detection results.

Fig. 2.23. The detection results of face images collected from the Internet, MPEG7 video clips and PIE database.

Fig. 2.24. Some error/missing examples.

CHAPTER 3 A NOVEL FACE DETECTION METHOD UNDER VARIOUS ENVIRONMENTS

In this chapter, we will propose a method to detect a face with different poses under various environments. It consists of three phases: profile feature extraction, non-profile face detection and the true eye locating.

3.1 INTRODUCTION

Detecting a face from an image is the first step for face applications. Even it has been studied for many years; detecting a face under various environments is still a challenge work. Some factors make face detection difficult. One is the variety of colored lighting sources, another is that facial features such as eyes may be partially or wholly occluded by shadow generated by a bias lighting direction; others are races and different face poses with/without glasses. Several methods [4-18] had been proposed to detect faces. Some [4-6] use eigenface, neural network and support vector machine (SVM) to detect faces of restricted poses under normal lighting condition. Hsu et al. [18] proposed a face detection algorithm that uses a nonlinear

properties are used to extract the facial features and applied to verify a face. Chow et al. [7] proposed an approach to locate a face candidate under different lighting

conditions and then applied an eigenmask based method to verify the face. However, these methods fail to locate faces with some kinds of poses (such as near-profile and profile face). Shin and Chuang [8] uses shape and contour information to locate face in a plain background and normal lighting condition. However, it is difficult to detect a face in a complex background and various lighting conditions. Wu and Zhou [9]

proposed an algorithm to locate face candidates using eye-analogue segment information. The algorithm would fail when a person wears glasses. Some methods [10-12] use mosaic, edge and mathematical morphology to detect eyes, however, these methods would fail to locate eyes when a face is under a poor lighting condition (such as bias-light source).

3.2 THE PROPOSED METHOD

As mentioned above, various lighting conditions, different head poses, glasses, races, and complex background are factors to make face detection difficult. To solve these problems, we propose a method to detect a face with/without glasses under various poses and environments. Fig. 3.1 shows the block diagram of the proposed face detector. First, we use skin colors to extract candidate skin regions. Next, shape

information is used to judge whether a skin region contains a shoulder part. If yes, a skin projection based method will be applied to remove it and the head part is labeled as a face candidate for further processing. For each face candidate, a set of geometric constrains are firstly applied to judge if the face candidate is a profile face.

If no, some eye-like rectangles are first extracted. Then, based on these eye-like rectangles, a horizontal eye line and a vertical symmetric line are located to judge if the face candidate is a non-profile face.

A color image

Fig. 3.1. The block diagram of the proposed face detector.

The remaining of this chapter is described as follows. In Section 3.2.1, a shape based algorithm is proposed to identify a profile face. Section 3.2.2 presents an algorithm to detect a non-profile face. Section 3.2.3 provides the true eye locating procedure. In Section 3.3, some experimental results based on HHI and Champion

3.2.1 Profile Detection

In this section, we will provide a method to determine if an image contains a profile face. First, the skin region detector proposed by Jing and Chen [22] is adopted to extract skin regions from an input image. Fig. 3.2 shows the extracted results from some face images with different poses taken under various lighting environments. Note that some pixels in clothes with color similar to skin are also labeled as skin. In the following, we will provide a method to remove these pixels.

(a) Three images taken under various lighting environments.

(b) The extracted skin regions (white pixels) using Jing-Chen skin region detector.

Fig. 3.2. Some results of applying Jing-Chen skin region extractor.

3.2.1.1 Shoulder Removing Procedure

Fig. 3.3 shows an example of two men, one in Fig. 3.3(a) wears a T-shirt of skin-like color; and the other in Fig. 3.3(b) wears a T-shirt of non-skin color. Figs.

3.3(c) and 3.3(d) show the results of applying Jing-Chen method to Figs. 3.3(a) and 3.3(b), respectively. In Fig. 3.3(c), pixels on shoulder are labeled as skin pixels. Here, a projection-based procedure is provided to remove the shoulder part.

(a) A man wearing a T-shirt of skin-like color.

(b) A man wearing a T-shirt of non skin-like color.

Neck point

(e) The horizontal projection of (c). (f) The horizontal projection of (d).

Fig. 3.3. An example showing two men wearing clothes with/without skin-like color.

First, we need to determine whether a skin region includes a shoulder part. In general, the shoulder of a person is wider than the head and neck. Based on this fact,

the horizontal projection of skin pixels is used to remove shoulder. Figs. 3.3(e) and 3.3(f) show the horizontal projections of skin pixels in Figs. 3.3(c) and 3.3(d), respectively. From these figures, we can see that if a shoulder exists, the number of the labeled skin pixels below the middle part of the whole skin region, L, should be larger than that of the upper one, U. Based on this fact, the inequation _s

L  is

applied to judge whether a skin region includes a shoulder part. In this dissertation, the threshold _s is 1.4.

If a skin region is determined to have a shoulder, the neck location is considered as a good cutting point to remove the shoulder part. Since the neck is thinner than the other part of a human body, it will appear at the valley of the projection histogram (see Fig. 3.3(e)). Thus we take the valley point as the neck point. In order to locate the neck point, the begin and maximum points of the projection histogram are used to form a base line named as Top-Down line (see Fig. 3.4(a)). For each histogram point below the Top-Down line, we evaluate the distance from the histogram point to the Top-Down line. The point with maximum distance is considered as the neck point (see Fig. 3.4(b)). And the horizontal line passing the neck point is considered as the cutting line. Fig. 3.4(c) shows the detected cutting line on a skin region. After locating the cutting line, all skin pixels below the cutting line are removed. Fig. 3.4(d) shows the result of shoulder removing. Now, we will decide if the remaining skin is

a profile face. There are two kinds of profiles, one is right profile; the other is left profile. Since these two kinds of profiles have the same characteristics, in the following, we will only describe how to detect a right profile, as to left profiles, a similar way can be applied.

Top-Down line Begin

point

Maximum point

(a) The Top-Down line for a projection histogram.

Neck point

Cutting line

(b) The neck point and the cutting line located.

Cutting line

Fig. 3.4. An example for removing shoulder from a skin region.

3.2.1.2 Profile Feature Extraction

There are some feature points (see Fig. 3.5(a)) on a profile. These are nose peak, nose bottom, nose top and chin point. Based on the geometric relations among these

points, we can judge if a skin region is a profile. Here, we will provide a method to extract these points.

Nose peak Nose top

Nose bottom Chin point

Nose peak

(a) Some feature points on a right profile. (b) The extracted nose peak.

Chin point Chin scan line

Nose bottom Nose bottom scan line

(d) The extracted nose bottom scan line and nose bottom.

Nose top

Nose peak (e) The extracted nose top.

Fig. 3.5. An example for illustrating the profile features.

First, the most right point of a skin region is considered as the nose peak (see Fig.

3.5(b)). Next, a vertical line V passing through the nose peak is taken. Line V is then

rotated clockwise w.r.t nose peak until the length of the intersection segment between the rotated V and the skin region is larger than a predefined threshold, the rotated V is then defined as the chin scan line. For each contour pixel of the skin region below the nose peak and at the right side of the chin scan line, the distance from the pixel to the chin scan line is evaluated. And the point with the maximum distance is considered as the chin point (see Fig. 3.5(c)).

Based on the extracted chin point, the line from the nose peak to the chin point is defined as nose bottom scan line, which will be used to locate the nose bottom.

Following the contour points from the nose peak to the chin point, the contour point with the maximum distance to the nose bottom scan line is considered as the nose bottom (see Fig. 3.5(d)). Finally, in order to locate the nose top, we draw a circle using the nose peak as its center and the distance from the nose bottom to the center as its radius, the top point of the skin region on the circle is considered as the nose top (see Fig. 3.5(e)). Based on these extracted points, we will provide a procedure to check if a skin region is a profile.

Four rules are provided to verify the extracted profile features. At first, we define some profile variables. Let H be the height of the skin region, T be the vertical distance from the nose top to the chin point, C be the vertical distance from the nose peak to the chin point, D be the distance from the nose peak to the nose bottom, and K

be the vertical distance from the nose peak to the top of the skin region. Define A to be the angle formed by the three points: nose top, nose peak and nose bottom. In order to determine the front head part of a profile face, we define a vertical line V2 passing through the nose peak and then rotate counter clockwise w.r.t nose peak until the length of the intersection segment between rotated V2 and the skin region is larger than a predefined threshold. The rotated V2 is then defined as the front-head line and B to be the angle between the front-head line and the vertical line passing through the

nose peak. Fig. 3.6(a) shows an example for the above defined variables. Based on these variables, four rules for profile checking are given as follows:

Rule 1: T > H/4. (3-1)

Rule 2: D < 0.5 C. (3-2)

Rule 3: 0.85 K < C < 1.2 K. (3-3)

Rule 4: 45⁰  A150⁰and15⁰ B45⁰. (3-4)

If all rules are satisfied, the skin region is considered as a profile face.

(a) The defined variables for a skin region. (b) The defined profile face rectangle.

Fig. 3.6. An example to show the defined profile variables and the face rectangle.

If the skin region is not considered as a profile, the skin region is rotated according to the degree sequence {+5, -5, +10, -10, +15, -15}, and the profile detection procedure is applied again. Based on the nose peak and the distance C, a rectangle named as profile face rectangle is formed, we use the rectangle to identify the profile face location. Fig. 3.6(b) shows an example of the detected face rectangle for a profile face.

3.2.2 Non-Profile Face Detection

For a skin region not determined as a profile face, a non-profile face detection procedure is then applied. Jing and Chen [22] have proposed a method to locate the horizontal eye line and some eye-like rectangles on a skin image. Each eye-like rectangle has the same height, h. Based on the output of the Jing-Chen method, we will provide a method to determine if a skin region is a non-profile face. Fig. 3.7 gives an example, Fig. 3.7 (a) is a face image and Fig. 3.7(b) shows the located horizontal eye line (marked by green color) and some eye-like rectangles using Jing-Chen’s method on Fig. 3.7(a). Based on the located horizontal eye line, we can define a checking bound, which is formed by a pair of lines with distance 1.5h from the horizontal eye line, to remove those false eye-like rectangles. In Fig. 3.7(b), we can see that only two eye-like rectangles are in the defined checking bound, and the

horizontal eye line passes through these two eye-like rectangles. We classify this layout as simple face model. For a simple face model, the only two eye-like rectangles within the checking bound are considered as the true eyes. Let de be the distance between the two eyes. A face rectangle is then defined as shown in Fig.

3.7(c). However, if a man wears glasses or a face is taken under a bias lighting source, then there will exist more than two eye-like rectangles in the checking bound (see Fig. 3.8). We classify these skin regions as complex face model. In the following, we will introduce a vertical symmetric line locator to solve the complex face model.

(a) A face image. (b) The located horizontal eye line and eye-like rectangles.

The horizontal eye line

Eye-like rectangles

The horizontal eye line

Eye-like rectangles (a) A man with glasses. (b) A man under bias lighting source.

Fig. 3.8. Two examples for the complexity face model.

3.2.2.1 Vertical Symmetric Line Location

In general, face is a nearly symmetric pattern, a symmetric vertical line exists.

There are some methods [23-25] based on this property to extract the symmetric vertical line. However, they may fail when a face is taken under biased lighting environment or is not frontal. A human face is a 3D object, different lighting source and poses will make the symmetric line location difficult. In order to overcome these factors, we will first classify the complex model into four patterns: half profile, bias lighting, near frontal and normal (see Fig. 3.9). From our observation, we found that the nose peak and forehead area always has higher lighting response than other facial regions. And the areas of eyebrows and eyes contain fewer skin color pixels and have lower gray values than other facial regions. Based on these facts, we define a vertical skin projection histogram AH(x) as Eq. (3-5) to catch these characteristics.

Note that the projection area, A, used in Eq.(3-5) is formed by extending BR wb 4 1

height (see the black rectangle in Fig. 3.9(a)), where BR is the rectangle bounding

all eye-like rectangles, wb is the width of BR. Fig. 3.9 shows some projection results from some face images taken under different poses and lighting conditions.

 

and right are the left and right bounds of the projection area. If there are only two valleys below the middle line (see Fig. 3.9(a)), and the intersection between the middle line and the histogram contains three segments, we say this is a half profile,

AH(x)

(a) A half profile pattern. (b) A bias lighting pattern.

Fig. 3.9. The vertical histogram projection patterns and detected vertical symmetric lines (continued).

AH(x)

Fig. 3.9. The vertical histogram projection patterns and detected vertical symmetric lines.

and the vertical line passing the peak between two valleys is considered as the symmetric line (see Fig. 3.9(a)). If the skin region is not a half profile, then the bias lighting testing is conducted. In order to judge the shadow ratio on a face, we collect all dark segments on the middle line. Let DL be the longest dark segment (see Fig. side is considered as the symmetric line (see Fig. 3.9(b)). In this dissertation, we set the threshold  as

wb. If the skin region is neither a half profile nor a bias lighting

pattern, we apply the near frontal testing procedure. There are two kinds of near

following procedure, we will describe how to test a left-side near frontal pattern.

Using the similar way, we can test a right-side near frontal patterns. At first, define two reference lines y_top y_mid d/2 as the top line and y_bottom  y_mid 3d/4 as the bottom line, where d is the distance from the global maximum peak of the histogram to the middle line. If there exists three valleys v1(x₁,y₁),v2(x₂,y₂) and

) , ( 3 x₃ y₃

v  satisfying the left-side rule as follows:

The left-side rule: region to be a left-side near frontal face pattern, and the vertical line passing P_2,3as the symmetric line. Fig. 3.9(c) shows an example of a man with left-side near frontal face pattern. In this dissertation, t₁ is set as 1.2. Finally, for a skin region not belonging to any above three patterns, the vertical line passing the maximum peak not near the boundary of the projection area is considered as the symmetric face line (see Fig.

3.9(d)).

3.2.2.2 False Eye-Like Rectangle Removing

As we know, a pair of human eyes is symmetrically located on the left and right sides of the symmetric line of a face. That is, if we label one eye from a side of the symmetric line, then the other eye must locate at the other side and be symmetric to the labeled eye w.r.t the symmetric line. Using this property, those false eye-like rectangles can be eliminated. Based on the vertical symmetric line, a eye-like boundary is defined to remove those false eye-like rectangles. First, for each eye-like rectangle, evaluate the distance of the rectangle to the symmetric line. Second, two farest eye-like rectangles at both sides of the symmetric line are taken, and let a be the distance of the near one.

Finally, based on the horizontal eye line and the eye-like rectangle height (h), we can define the eye-like boundary as shown in Fig. 3.10. Any eye-like rectangle out of the eye-like boundary is considered as a false eye-like rectangle and is removed. Now, we will locate the true eyes from the remaining eye-like rectangles.

The vertical symmetric line

1.5 a False eye-like

rectangles

The horizontal eye line

h 1.5 h

1.5 h Eye-like boundary a

Fig. 3.10. An example to illustrate the eye-like boundary for a skin region.

在文檔中人臉偵測之研究 (頁 48-0)