Chapter 2 Face Detection
2.4 Post-process: A region-based clustering method
The face detector can find a lot of candidates around faces in a scanned image as shown in Fig. 2-15.
Evidently, we need to deal with the troubled problem that more than two blocks are classified as faces around a single face. A region-based clustering method is proposed to solve this problem. The method consists of two levels of clustering, one is called local scale clustering and another is called global scale clustering. The local scale clustering is used to cluster the same scale of blocks and design a simple filter to judge numbers of blocks in clusters. While numbers of blocks in a cluster are more than one, the cluster is preserved as the possible candidate of faces; otherwise, it will be discarded. The global scale clustering works after local scale clustering finished around the original detected blocks. In the end, we select the average of the corners in the global scale clusters to label the faces.
Fig. 2-15: The image after face detecting
20 Eq. (2-11), Eq. (2.12) and Eq. (2.15) are formulated decision rules of the proposed method. cluster x y
( )
, = means the block x and 1 y are in the same cluster and their bounding regions is overlapped. overlap rate x y is the _( )
,percentage of overlapped region for x and y, distance x y
( )
, is the distance of a center for x and y. Figure 2-16 (a) and (b) shows the chart of the overlapped region and the distance of a center of two blocks in the local scale clustering and the global scale clustering respectively. In Fig. 2-16 (a), the two blocks are resolved as the same cluster. In Fig. 2-16 (b), the two blocks are resolved as different clusters, because the distance of their center is not satisfied with Eq. (2.13).(a)
(b)
Fig. 2-16: The chart of the overlapped region and the distance of a center of two blocks in (a) the local scale clustering and (b) the global scale clustering
21
The two blocks are not in the same cluster in Fig. 2-16(b). In a special case as shown in Fig. 2-17, the four blocks are in the different clusters. Therefore, they are considered as faces and located in the image; the most of them are false accept blocks.
For the reason, we choose the one of them block to replace the others if they are satisfied with Eq. (2.12).
Fig. 2-17: A special case in cluster
The example of Fig. 2-15 after the local scale clustering is shown in Fig. 2-18(a) and Fig. 2-18(b) is the results of after the global scale clustering from Fig. 2-18(a).
(a) (b)
Fig. 2-18: (a) The results of the local scale clustering (b) the results of the global scale clustering
22
1 Chapter 3
Face Identification
After extracting a face from the captured image, the information of the face can be used to identify the person by the system of face identification. Two major parts of the face identification in this work are eigenfaces extraction and the adaptive probabilistic model (APM). First, we describe the details of eigenfaces extraction.
Then, the proposed adaptive probabilistic model (APM) used for modeling a client’s face is presented.
The flow chart of face identification process is shown in Fig. 3-1. First, the facial feature extractor is used to extract the facial features from faces that are received from the face detector mentioned in previous chapter. The facial feature extractor is constructed by the principle components analysis (PCA) [17] which is based on projecting the image space into a low dimensional feature space. According to extracted facial features, the faces are judged as either clients or imposters by the face identifier. The face identifier is formed with the adaptive probabilistic model (APM) and the client database. Details of the proposed methods are introduced in the following sections.
Fig. 3-1: The flow chart of face identification
23
3.1 Features
The eigenfaces technique based on principle component analysis has been widely used for pattern recognition, as well as in the field of biometrics. It is the most popular feature extraction method employed by face identification techniques [17]. The principle components analysis (PCA) techniques, also known as the Karhunen-Loeve methods, choose a dimensionality reducing projection that maximizes the scatter of all projected samples. The eigenface feature extraction based on PCA is used to obtain the most important features from the face images in our system. These features are obtained by projecting the original images into corresponding subspaces.
To begin with, we have a training set of N images, and each image consists of n elements. For example, we have N = 4000 images in our database used to compute eigenfaces. Each image has n = 24 X 24 = 576 elements. Figure 3-2 shows the chart of rearranging 24 X 24 pixel of image to 576 X 1 vectors.
Fig. 3-2: The chart of rearranging 24 X 24 pixel of image to 576 X 1 vectors
The process of obtaining a single space consists of finding the covariance matrix C of the training set and computing the eigenvectors v kk; =1, 2,...,n . The eigenvectors v corresponding to the largest eigenvalues k λk span the base of the sought subspace. Each original image can be projected into the subspace as Eq. (3.1).
24
1, 2,...,
T
k vk s k m
η = ⋅Φ = (3.1)
Where m ( m< ) is the chosen dimensionality of the image subspace and n
s s
Φ = Γ −Ψ , where Γ is an original images from the set of images that have to be s projected and Ψ is the average image of the training set. In Fig. 3-3, the average image obtained from our training set is presented. The coordinates of the projected images in the subspace, ηk;k=1, 2,...,m, can be used as a feature vector for the matching procedure.
Fig 3-3: The average face image from our database
Selecting dimensionality of the image subspace is an important topic. If m is closer to n , the degree of face identification is more precise. But it spends more computational time to project the original images into the corresponding subspace.
Hence, we have to choose the appropriately dimensionality of the image subspace for the precision and the computational time.
The content of pattern information with respect to the number of eigenvectors is shown in Fig.3-4. The more eigenvectors are used, the more pattern information can be expressed. Forty eigenvectors can express about 77 percentage of pattern information; fifty eigenvectors can express about 81 percentage of pattern information;
sixty eigenvectors can express about 84 percentage of pattern information.
Figure 3-5 denotes the detection rate corresponding to each number of eigenvectors. While the number of selected eigenvectors is greater than twenty, the
25
degree of detection rate is not obvious improved. Instead of the number of selected eigenvectors is greater than fifty, the degree of detection rate is reduced progressively.
The reason is the pattern information includes the significant information and noise.
The more eigenvectors are extracted, the more noises are extracted. Hence, the performance of the detection rate is descending by the affect of the noise.
Depending on the factor of the computational load and the detection rate, we choose fifty eigenvectors as the image subspace used for face identification.
Fig 3-4: The contents of pattern information with respect to the number of eigenvectors
Fig 3-5: The detection rate corresponding to each number of eigenvectors
26
3.2 Adaptive probabilistic model (APM)
The adaptive probabilistic model (APM) is proposed to achieve a fast and functional technique of face identification. The construction of the adaptive probabilistic model (APM) is a weighted combination of simple probabilistic functions. Hence, the design of APM is sufficient for real-time tasks. Furthermore, the proposed APM can on-line register new clients and update the clients’ information.
The capability of on-line registering new clients enhances the practicability of the proposed system. The detection rate of identification can also be improved by updating clients’ information for long-term usage of the proposed system.
The primary concept of the APM architecture is based on view-independent face identification. The model of view-independent face identification is constructed by five different head orientations from each person (Ebrahimpour et al. [21] proposes the model of face recognition. In the model, the face space, spanning from right to left profiles along the horizontal plane, is divided into five views). The view-independent model of face identification is more robust than the single view model, because the head orientation of a person is variable in real world.
Our model is designed to achieve view-independent face identification with a mixture of view-independent faces modeled by probabilistic functions. The view-independent model of face identification is constructed by five different head orientations from each client as shown in Fig. 3-6.
27
Fig 3-6: Example for five different head orientations of a client
3.2.1 Similarity measure
APM follows probabilistic constraint, that is, similarity measures of APM are designed to model the likelihood functions. The judgment of classification is relying on the degree of likelihood. For example, the similarity of a testing sample x between each registered client is computed with the likelihood functions of each client.
Then, the testing sample x is classified as the client corresponding to the biggest similarity. Eq. (3.2) shows the likelihood function. In our system, n presents the label of each client, k is the one of five head orientations, and t denotes the updating times of clients’ information. The wn k t, , is the weight of each probabilistic functions, the constraining of wn k t, , is shown in Eq. (3.3). The value of wn k, ,1 is initialized by Eq.
28
Eq. (3.5) indicates the original probabilistic functions. d is the dimension of input vectors, μn k t, , is the mean vector, and σn k t, , is the covariance matrix. Due to the assumption of Eq. (3.6) (where I is the identity matrix), the probabilistic functions in Eq. (3.5) can be simplified as Eq. (3.7). Figure 3-7 indicates the chart of initial mean vector μn k, ,1.
29
Fig 3-7: The chart of initialization of mean vector μn k, ,1
3.2.2 Parameter tuning
The magnitude of covariance matrix σn k t, , can affect the performance of APM.
For this reason, we design an experiment to find the best value of covariance matrix
, , n k t
σ form the different coefficient of covariance matrix σn k t, , .
The face database containing images of 10 persons is used for the experiment.
The database has, for each person, images of ten different head orientations. We choose five images of ten different head orientations for each person to be the training data, the other five images is used to be the testing data.
The covariance matrix σn k, ,0 is initialized by the variance of training data, because the images of each person is too less to compute the variance from the images.
When obtain the initialized covariance matrix σn k, ,0, we need to adjust the coefficient of covariance matrix σn k, ,1.
30
, ,1 1 , ,0
n k n k
σ = ×k σ (3.8)
The covariance matrix σn k, ,1 is adjusted by Eq. (3.8). The detection rate with
respect to different parameter k of the covariance matrix σn k, ,1 is shown in Fig. 3-8.
When the parameter k is larger than four and smaller than forty-three, the detection rate is obvious improved. Therefore, we choose parameter k as 5 to obtain a suitable covariance matrix σn k, ,1 for APM employed in this work.
Fig 3-8: The detection rate of different parameter k of the covariance matrix
3.2.3 Adaptive updating
The topic of adaptive updating introduces the updating functions of APM. The design of adaptive updating for APM improves the detection rate of face identification.
As the updating times increase, the functions of APM become more robust. The
31
model of APM will match more precisely the head orientations of the actual person While a client is identified correctly, the function of APM is updated immediately. By the design of adaptive updating for APM, we present the improvement of the detection rate in chapter 4.
( ) ( )
The weights of each probabilistic function are adjusted using Eq. (3.9). Where α is the learning rate for the weights and Mn k t, , is satisfied with Eq. (3.10). If
, , n k t
M is 1 for the probabilistic function, the probabilistic function is closest the test sample. In our system, n presents the label of each client, k is the one of five head orientations, and t denotes the number of time of clients information updating.
Parameters μn k t, , and σn k t, , for unmatched distributions remains the same. The parameters of the distribution which matched the new observation are updated using Eq. (3.11) and Eq. (3.12).
,where ρ is the learning rate for the mean vector and the covariance matrix.
The magnitude of the learning rate affects the efficiency of updating of APM. A big learning rate may cause likelihood functions of APM over fitted, so that the
32
performance of the detection rate is reduced. Nevertheless, the small learning rate has poor improving ability of the detection rate.
We use the ORL database which contains 40 persons to select the parameter α and ρ. Each person has ten images, five of them are used to be the training data, two of them are the testing data, and the others are the updating data. Figure 3-9 and Fig.
3-10 indicates the detection rate of different parameter α and ρ. Concerning the best detection rate in experiment results, we choose 0.05 to be the value of parameter α , and the value of parameter ρ is 0.2.
Fig 3-9: The detection rate of different parameter α
Fig 3-10: The detection rate of different parameter ρ
33
Chapter 4
Experimental Results
In this chapter, experimental results of face detection and face identification are demonstrated. We implemented our system on the platform of PC with Intel P4 2.60 GHz and 1G RAM. The development tool is Borland C++ Builder 6.0 on Window XP OS. The input images are captured from camera, and all the input images are in the resolution of 320 X 240 pixels.
In section 4.1, we present the results and performance of face detection. The results and accuracy of face identification and the comparison between the proposed method and the other methods are shown in section 4.2. Furthermore, some discussions about the proposed face detector and face identifier are made in section 4.3.
4.1 Face detection
We use the features of 2D Haar and Adaboost learning algorithm to construct our face detector. Because this system is applied in eastern country, most parts of our database are eastern faces.
Figure 4-1(a) and (b) presents the results of a single face detection and the results of multi-faces detection. The yellow block indicates that the region contains a face in the image. In the case of multi-persons with different sizes of faces, the face detector can precisely allocate regions of the faces from the image.
34
(a)
(b)
Fig. 4-1: (a) The results of a single face detection (b) the results of multi-faces detection
Table 1: The comparison between face detectors with and without lighting normalization
35
We design an experiment for the performance of face detector by estimating the detection rate and the numbers of false accept images. Detection rate and the numbers of false accept images are used to estimate the enhancement of lighting normalization.
The test set consists of 130 pictures with 276 labeled frontal faces. Table 1 illustrates the comparison between face detectors with and without lighting normalization.
Figure 4-2 and Fig. 4-3 present the detection rate and the numbers of false accept images in different threshold. We test seven different thresholds for our face detector and calculate the accuracy and the numbers of false accept images correspondingly. In our results, the performance of the face detector with lighting normalization is better than the face detector without lighting normalization.
Fig. 4-2: The detection rate in different threshold
36
Fig. 4-3: The numbers of false accept images in different threshold
OpenCV (Open Source Computer Vision Library Community) is an open source library. In recent years, OpenCV is widely applied to the field of image processing.
One of the applications is face detection. The construction of the face detector of OpenCV also uses AdaBoost algorithm. The differences between our face detector and face detector of OpenCV are the training database and the method of post-process.
Moreover, an image fitting method is employed as a lighting normalization process in our system to improve the robustness of face detection.
Figure 4-4(a), (c), (e), (g) and (i) shows the results of our face detector. Figure 4-4(b), (d), (f), (h) and (j) shows the results of OpenCV. It can be observed in Fig. 4-4 that our face detector outperforms the face detector of OpenCV, especially in cases of eastern faces.
(a) (b)
37
(c) (d)
(e) (f)
(g) (h)
(i) (j)
Fig. 4-4(a), (c), (e), (g) and (i) shows the results of our face detector, Fig. 4-4(b), (d), (f), (h) and (j) shows the results of OpenCV
38
4.2 Face identification
In this section 4.2, we divide the experimental results of face identification into two parts: off-line testing and on-line testing. The details of off-line testing are introduced in section 4.2.1, and the outcome of on-line testing is shown in section 4.2.2.
4.2.1 Off-line testing
Table 2 illustrates the comparison of methods and characteristics for face identification in this thesis and the other papers. The comparing methods consist of Eigenfaces, PCA+CN, SOM+CN and PCA+APM (the proposed method). Eigenfaces and PCA+CN are commonly used in pattern recognition. SOM+CN is a combined neural network (self organizing map and convolutional neural network) proposed in [22].
The comparing characteristics are based on training time, update ability, increase of clients and practicability. Because our system is based on the efficiency and performance in real time, the practicability is more important than other comparing characteristics. Our system can on-line register new clients and on-line update the information of clients in real time, hence the practicability is satisfied.
39
Table 2: The comparison of characteristics for the proposed method and the others
Table 3 presents the comparison of detection rate for our face identifier and other papers. The comparing methods similarly consist of Eigenfaces, PCA+CN, SOM+CN and PCA+APM (the proposed method). In our face identifier, a lighting normalization method is used to improve the detection rate, and the effect of this method is also demonstrated in Table 3.
The test set is ORL database which consists of 40 persons, each person has ten images. The training images for each person involves three, four and five images to model a person, the remnant images for each person are the testing images. By the different numbers of training images for each person, compare the accuracy for proposed method and the other papers.
40
Table 3: The comparison of detection rate of the proposed method and the others
The training images for our method are depending on the different head orientations to construct an APM for each person. In the comparing results, we select the optimum particular images to be training images, the rest of images in the database are testing images. Table 3 shows that the detection rate of our face identifier without lighting normalization is slightly below that of SOM+CN and higher than that of PCA+CN. The proposed face identifier with lighting normalization results in the highest detection rate.
The tolerable degree of the proposed system
In order to test the tolerable degree of our system, we design an experiment to measure the accuracy by different numbers of clients. The face database used in this experiment contains all face database used by previous experiments in this thesis, the ORL database (used in adaptive updating and the comparison with the other papers), the 10 persons database (applied in the selection of coefficient k for covariance matrix σn k, ,1) and the 13 persons database (applied in finding the threshold to distinguish the clients and impostors). There are 63 persons with 630 frontal faces used in this experiment.
41
Fig. 4-5: The performance of face identifier with respect to the number of clients
Figure 4-5 indicates the performance of face identifier with different number of clients. While the number of clients is below ten, the detection rate achieves the percentage of 100. The detection rate begins to drop off, when the number of clients exceeds ten. Until the number of clients achieves sixty-three, the detection rate is in the percentage of 86. It is accepted that the detection rate of face identifier achieves the percentage of 80. Therefore the tolerable degree of our system can accept more than sixty-three number of clients.
Adaptive updating
The ORL database is used to measure the performance of adaptive updating. We select five images for each person to be training images, three images as testing
The ORL database is used to measure the performance of adaptive updating. We select five images for each person to be training images, three images as testing