Automatic Facial Feature Extraction by Applying Genetic Algorithms
Chun-Hung L,in
1Commun. and Multimedia Lab.
Ja-Ling Wu
1~d250601 O@csie.ntu.edu.tw
wj l@cmlab.csie.ntu.edu. tw
Department of Information Engineering
Dept.
of
Comp. Sci.
C!?LInfo. Eng.
National Taiwan University
Taipei, 106, Taiwan, R. 0. C.
Albstract
An automatic facial featuve atraction algorithm is pre- sented in this paper: The algorithm is composed of two main stages: the face region estimation stage and the fea- ture extraction stage. In the face region estimation stage,
U second-chance region growing method is adopted to esti- mate the face region of a taEet image. ln the feature extrac- tion stage, genetic search algorithms are applied to extract the facial feature points within the face region. It is shown
t y simulation results that the proposed algorithm can au-
tomatically and exactly extract facialj2atures with limited computational complexity,
II
Introduction
Facial feature extraction plays an important role in many applications, such as very low bit-rate video coding [ 11 and human face recognition [21. In very low bit-rate video cod- i:ng, the variation of facial features (the so-called local mo- tion) carries much more information than the other parts of
tihe target image do. In order to obtain higher coding gain, tlhe facial feature areas are usually sepairated from the target iimage, and then special coding schemes are performed on tlhese areas. As for human face recognition, facial features are extracted as the major analyzed targets, because they compose basic characteristics of a human face.
Although lots of facial feature extraction algorithms have been proposed so far [3,4], facial feature extraction is still si h.andicap to the prescribed ,applications due to its high com- plexity. In the existed algorithms, all the pixels within the fiace region were evaluated to find the best ones, that is, they d.idn't apply any fast searching method. Genetic algorithm:;
((GAS) [5] are adopted in our work to solve the most trouble s'ome task of facial feature extraction, ,the template match-. ing. Due to its good reputation in the applications of search- ing and optimization, good template matching performance:
National Chi Nan University,
Puli, 545, Taiwan, R. 0.
C.
(in terms of speed and matching error) can be expected. Al- though the results are not guaranteed to be global optima by applying GAS in searching, the expected feature points can always be extracted, in our simulations, with the aids of some intrinsic geometric relations among facial feature points.
2 An automatic facial feature extraction al-
gorithm
Fig. 1 shows the block diagram of the proposed facial feature extraction algorithm. The system is composed of three main components: ( 1 ) pre-processing, (2) first stage
feature extracting, and (3) second stage feature extract- ing. The details are described in the following subsections.
2.1 Pre-processing
The purpose of the pre-processing stage is to find out the face region. Because the face region locates roughly on the center of a head-and-shoulder image, region growing can be performed to estimate that region by selecting the central point of the image as an initial seed. If the size of the region
grown from the initial seed is beyond a range, either a new seed should be selected, or the threshold should be adjusted, to generate another region whose size is in the range.
In order to make sure that the forehead region is con- tained in the grown region, the intensities of three extended points above the region are compared with that of a pivot point in the region. If the difference is less than a thresh- old, they will be selected as new seeds. The new regions grown from the new seeds are checked and combined with the original region. The checking process prevents from in- cluding the hair region or the background region into the face region. This second-chance region growing process avoids the possibility of losing the forehead region from the face region.
When the combined region is extracted, a minimum rect- angle is found to surround that region. The following fea- ture extraction algorithm will be performed within the so- obtained square area to find the facial features.
2.2
Genetic search of facial features
The solution space S is a 2-D image subregion, and each image pixel in the subregion is an element in S. If the coor- dinate of the central point of S is ( 2
,
c),
the ith chromosomeCi in the population is defined as
a i , k - l a i , k - 2 ... a1>1 a,>0
]
, (1)” =
[
z7
]
=[
b i , k - l bi,l;-2 . . . b;,i b;,owhere k represents the codeword size of each coordinate component whose value is dependent on the region size. If the maximum value of the width and the height of the region is tu, the value k will be [log, w1. The values of
the genes are derived from the coordinate of the chromo- some, (xi, yi), that is, a i j = 1-j mod 2, and b i , j =
1-1
mod 2, where mi = q-
2 , and ni = yi-
jj.An initial population is formed before the evolution. In most GA-based applications, the initial population is created by randomly selecting elements from the solution space. To reduce the necessary iteration number, in our ap- proach, the initial chromosomes (xi
,
yi) are selected from the solution space as(xi, yi) = ( 2 0
+
s X i , YO+
s .X),
( 2 ) x i = ( - 1 ) y i-
m(m+
1)].
y,
= ( - 1 ) y i - m(m+
l)] ’ 1 [ I - ( 1 2 d q mod 2)l+
TTml), (3) (4) 1[[a&]
mod 21-
[2rn1),
1,
and d is the Each chromosome has an associated fitness value whichd . m o z ( X N - 1 , Y N - I )
where m =
LhJ,
s = [ 4length of the region. is defined as
where $’(xi2 yi) is the matching value of the ith chromo- some, and F k is the kth maximum matching value among
all the N matching values, and 6 is a delta function. The constant k determines how many chromosomes could be selected at most as the seeds in the reproduction stage for producing a rival population. The chromosomes with larger fitness values in the current population have larger proba- bility to be selected as seeds for the next generation.
The reproduction method used in this work is similar to
the weighted roulette wheel method [ 5 ] . For each chromo-
some, an interval r, is calculated as
)
iXLzlO
f k X k = O fie f k’
f k, o
5
i5
N-
1, (6) ri =where fk is the fitness value of the kth chromosome in the population, and ‘[’ and ‘)’ denote closing and opening boundaries, respectively. When the interval of each chro- mosome has been determined, N real numbers, ai, are ran- domly generated, where 0
<
ai<
1 and 0<
i<
N - 1. The value of ai will be bounded by some r j , that is,a; E r j . The chromosome Cj is then selected as a seed
to generate the rival population. It is possible that one chro- mosome can be selected twice or more. Because N real random numbers are generated, N seeds could be selected and placed in the mating pool.
For every two seeds selected from the mating pool, two new chromosomes are produced by performing uniform crossover operations [6]. Crossover is applied only when the mutation operation is performed on the most significant genes of the chromosomes.
In the mutation stage, assume the current seed to be processed is [mi nilt, where mi = [ U : , ~ - ~ U ~ , ~ - ~ . .
and n: = [ b : , k - l b i , k - 2 . .
.
b:,o]. In the j t h generation, the two genes u ; , ~ and bi,z are varied, where z = k -2
-
( j mod k). There are eight mutation operators,{(CP,vF)
= (~~p+~,Yp+i)105
P5
71, that can be ap- plied in our implementation, i.e., u ; : ~ = a $ , z+
c,,
andb::z = b:,=
+
vF,
where p is a random integer number whosevalue is between 0 and 7.
There are N chromosomes in the mating pool after per-
forming the genetic operations. Along with the original chromosomes in the current generation, N chromosomes
are selected from these 2 N chromosomes according to their fitness values. Each chromosome can only be selected at most once. The chromosomes with larger fitness values will be picked up as the members of the population in the next generation, and go through the next iterations of the genetic evolution. Although the sorting operation is needed in the survival competition stage, the overhead is not high because the population size is usually not large in this case.
The chromosome with the maximum fitness value is se- lected from the current population as the possible solution. The possible solution might be replaced by the other ones from a generation to the other generations. The iteration will be terminated if the solution is not updated for a prede- termined period of iterations.
2.3
First stage feature extracting
Basically, the face region can be divided into three sub- regions, R,, R b , and R,. The right eye, left eye, and mouth
Because the face rotation imgle is nsnnally not large, the main feature points will not cross the boundaries of their associated subregions. By applying GAS on searching the possible feature points and jhnd the one with the best match- ing value in each subregion, the three main feature points of a human face can always be extracted.
2.4 Second stage 6eature extracting
After the main features have been extracted, the other feature points are searched in the three new subregions which can be specified according to main feature points. The points with best matching values iue extracted as the stxondary feature points.
3 Experimental results
The proposed algorithm has been implemented, on a
S U N SPARC- 10 workstation under Urtix environment, at the Communication and Multimedia Lab., National Taiwan University. Figs. 2 and 3 show some of the experimental results.
The two images in Fig. 2 were captured by a normal V8
camera and transferred into digital form by an
A/D
con- verter. There was no special lighting when the images were captured. That is, the images were captured in a normal environment without any special equipment. With the pro- posed algorithm the facial R:atures of the two images can be extracted successfully.We also applied the proposed algorithm to typical head- and-shoulder images. The sequence “Claire” is shown here. E,ach image in the sequences was processed by the pro- posed algorithm and as expected the facial features had been extracted exactly. The simulation results are shown inL Fig. 3. Notice that the eye features Cim still be extracted even though the eyes on the target images are closed.
Table 1 shows the ratios of the aiverage number of searched points to the total number of candidate points. Ac-
cording to the table, it is known that most of the unneces-
sary evaluations are avoided. Therefore, the computational complexity is largely reduced, although some overheads are required for applying GAS in the searchmg process.
4
Conclusion
proposed algorithm has been applied to lots of head-and- shoulder images. It is shown by computer simulations that the facial features can always be extracted exactly by apply- ing the proposed algorithm.
References
[ 11 M. Wollbom, “Prototype prediction for colour up- date in object-based analysis-synthesis coding,” IEEE
Trans. Circuits Syst. Kdeo Technol., Vol. 4, No. 3,
June 1994, pp. 236-245.
[2] R. Chellappa, C. L. Wilson, and S. Sirohey, “Human and machine recognition of faces: a survey,” Proceed-
ings of the IEEE, Vol. 83, No. 5, May 1995, pp. 70%
740.
[3] G. Yang, and T. S . Huang, “Human face detection in a complex background,” Pattern Recognition, Vol. 27,
No. 1, 1994, pp. 5 3 4 3 .
[4] L. C. Desilva, K. Aizawa, and M. Hatori, “Detec- tion and tracking of facial features,” in SPIE Sympo-
sium on Ksual Communications and Image Pvocess- ing, Taipei, Taiwan, May 1995, pp. 1161-1 172.
[ 5 ]
D.
E. Goldberg, Genetic Algorithms in Search, Op-timization & Machine Learning. Reading: Addison- Wesley, 1989.
[6] G. Syswerda, “Uniform crossover in genetic algo- rithms,” in Proc. 3rd Int. Con$ Genetic Algorithms
andApplications, CA, 1989, pp. 2-9.
In this paper, an automatic facial fe,ature extraction al- gorithm is proposed. In order to extiract facial features fnom a head-and-shoulder image, the face region is first es- timated by a second-chance region growing method. The facial feature points are then searched b’ased on genetic al- gorithms within the face region. The necessary search num- bers are reduced, so the whole system is very efficient. The
Sequence Tseng Chou 0.12 Miss America 0.22
1
Claire1
0.431
Ratio 0.1 1Table1.The ratios of the averaged number of searched points t o
t h e
total number ofcandidate points. Extraction 1 I ~ S O S P and Eyebrows I I I J
Figure 2. Simulation results of two captured images.
Figure 3. Simulation results (of the 8th, 31th, 71th, 76th, 83th, and 86th frame from the “Claire” sequence. Notice that the eyes in the 8th and the 83th frames are closed. Moreover, the eyebrows of the second layer images are covered by hair.
