Chapter 3 Face Identification
3.2 Adaptive probabilistic model (APM)
3.2.3 Adaptive updating
The topic of adaptive updating introduces the updating functions of APM. The design of adaptive updating for APM improves the detection rate of face identification.
As the updating times increase, the functions of APM become more robust. The
31
model of APM will match more precisely the head orientations of the actual person While a client is identified correctly, the function of APM is updated immediately. By the design of adaptive updating for APM, we present the improvement of the detection rate in chapter 4.
( ) ( )
The weights of each probabilistic function are adjusted using Eq. (3.9). Where α is the learning rate for the weights and Mn k t, , is satisfied with Eq. (3.10). If
, , n k t
M is 1 for the probabilistic function, the probabilistic function is closest the test sample. In our system, n presents the label of each client, k is the one of five head orientations, and t denotes the number of time of clients information updating.
Parameters μn k t, , and σn k t, , for unmatched distributions remains the same. The parameters of the distribution which matched the new observation are updated using Eq. (3.11) and Eq. (3.12).
,where ρ is the learning rate for the mean vector and the covariance matrix.
The magnitude of the learning rate affects the efficiency of updating of APM. A big learning rate may cause likelihood functions of APM over fitted, so that the
32
performance of the detection rate is reduced. Nevertheless, the small learning rate has poor improving ability of the detection rate.
We use the ORL database which contains 40 persons to select the parameter α and ρ. Each person has ten images, five of them are used to be the training data, two of them are the testing data, and the others are the updating data. Figure 3-9 and Fig.
3-10 indicates the detection rate of different parameter α and ρ. Concerning the best detection rate in experiment results, we choose 0.05 to be the value of parameter α , and the value of parameter ρ is 0.2.
Fig 3-9: The detection rate of different parameter α
Fig 3-10: The detection rate of different parameter ρ
33
Chapter 4
Experimental Results
In this chapter, experimental results of face detection and face identification are demonstrated. We implemented our system on the platform of PC with Intel P4 2.60 GHz and 1G RAM. The development tool is Borland C++ Builder 6.0 on Window XP OS. The input images are captured from camera, and all the input images are in the resolution of 320 X 240 pixels.
In section 4.1, we present the results and performance of face detection. The results and accuracy of face identification and the comparison between the proposed method and the other methods are shown in section 4.2. Furthermore, some discussions about the proposed face detector and face identifier are made in section 4.3.
4.1 Face detection
We use the features of 2D Haar and Adaboost learning algorithm to construct our face detector. Because this system is applied in eastern country, most parts of our database are eastern faces.
Figure 4-1(a) and (b) presents the results of a single face detection and the results of multi-faces detection. The yellow block indicates that the region contains a face in the image. In the case of multi-persons with different sizes of faces, the face detector can precisely allocate regions of the faces from the image.
34
(a)
(b)
Fig. 4-1: (a) The results of a single face detection (b) the results of multi-faces detection
Table 1: The comparison between face detectors with and without lighting normalization
35
We design an experiment for the performance of face detector by estimating the detection rate and the numbers of false accept images. Detection rate and the numbers of false accept images are used to estimate the enhancement of lighting normalization.
The test set consists of 130 pictures with 276 labeled frontal faces. Table 1 illustrates the comparison between face detectors with and without lighting normalization.
Figure 4-2 and Fig. 4-3 present the detection rate and the numbers of false accept images in different threshold. We test seven different thresholds for our face detector and calculate the accuracy and the numbers of false accept images correspondingly. In our results, the performance of the face detector with lighting normalization is better than the face detector without lighting normalization.
Fig. 4-2: The detection rate in different threshold
36
Fig. 4-3: The numbers of false accept images in different threshold
OpenCV (Open Source Computer Vision Library Community) is an open source library. In recent years, OpenCV is widely applied to the field of image processing.
One of the applications is face detection. The construction of the face detector of OpenCV also uses AdaBoost algorithm. The differences between our face detector and face detector of OpenCV are the training database and the method of post-process.
Moreover, an image fitting method is employed as a lighting normalization process in our system to improve the robustness of face detection.
Figure 4-4(a), (c), (e), (g) and (i) shows the results of our face detector. Figure 4-4(b), (d), (f), (h) and (j) shows the results of OpenCV. It can be observed in Fig. 4-4 that our face detector outperforms the face detector of OpenCV, especially in cases of eastern faces.
(a) (b)
37
(c) (d)
(e) (f)
(g) (h)
(i) (j)
Fig. 4-4(a), (c), (e), (g) and (i) shows the results of our face detector, Fig. 4-4(b), (d), (f), (h) and (j) shows the results of OpenCV
38
4.2 Face identification
In this section 4.2, we divide the experimental results of face identification into two parts: off-line testing and on-line testing. The details of off-line testing are introduced in section 4.2.1, and the outcome of on-line testing is shown in section 4.2.2.
4.2.1 Off-line testing
Table 2 illustrates the comparison of methods and characteristics for face identification in this thesis and the other papers. The comparing methods consist of Eigenfaces, PCA+CN, SOM+CN and PCA+APM (the proposed method). Eigenfaces and PCA+CN are commonly used in pattern recognition. SOM+CN is a combined neural network (self organizing map and convolutional neural network) proposed in [22].
The comparing characteristics are based on training time, update ability, increase of clients and practicability. Because our system is based on the efficiency and performance in real time, the practicability is more important than other comparing characteristics. Our system can on-line register new clients and on-line update the information of clients in real time, hence the practicability is satisfied.
39
Table 2: The comparison of characteristics for the proposed method and the others
Table 3 presents the comparison of detection rate for our face identifier and other papers. The comparing methods similarly consist of Eigenfaces, PCA+CN, SOM+CN and PCA+APM (the proposed method). In our face identifier, a lighting normalization method is used to improve the detection rate, and the effect of this method is also demonstrated in Table 3.
The test set is ORL database which consists of 40 persons, each person has ten images. The training images for each person involves three, four and five images to model a person, the remnant images for each person are the testing images. By the different numbers of training images for each person, compare the accuracy for proposed method and the other papers.
40
Table 3: The comparison of detection rate of the proposed method and the others
The training images for our method are depending on the different head orientations to construct an APM for each person. In the comparing results, we select the optimum particular images to be training images, the rest of images in the database are testing images. Table 3 shows that the detection rate of our face identifier without lighting normalization is slightly below that of SOM+CN and higher than that of PCA+CN. The proposed face identifier with lighting normalization results in the highest detection rate.
The tolerable degree of the proposed system
In order to test the tolerable degree of our system, we design an experiment to measure the accuracy by different numbers of clients. The face database used in this experiment contains all face database used by previous experiments in this thesis, the ORL database (used in adaptive updating and the comparison with the other papers), the 10 persons database (applied in the selection of coefficient k for covariance matrix σn k, ,1) and the 13 persons database (applied in finding the threshold to distinguish the clients and impostors). There are 63 persons with 630 frontal faces used in this experiment.
41
Fig. 4-5: The performance of face identifier with respect to the number of clients
Figure 4-5 indicates the performance of face identifier with different number of clients. While the number of clients is below ten, the detection rate achieves the percentage of 100. The detection rate begins to drop off, when the number of clients exceeds ten. Until the number of clients achieves sixty-three, the detection rate is in the percentage of 86. It is accepted that the detection rate of face identifier achieves the percentage of 80. Therefore the tolerable degree of our system can accept more than sixty-three number of clients.
Adaptive updating
The ORL database is used to measure the performance of adaptive updating. We select five images for each person to be training images, three images as testing images and two images as updating images. Figure 4-6 indicates the performance of adaptive updating.
42
Fig. 4-6: The performance of adaptive updating
We use cross-validations to estimate the performance of adaptive updating. In Fig. 4-6, the upper line is the detection rate of after updating, the lower line is the detection rate of before updating. According to the results of this experiment, the face identifier with adaptive updating is obviously improved.
The threshold selection for distinguishing clients and impostors
The 26 persons’ database is used to find the threshold to distinguish the clients and impostors. We select five images for each person to be training images, and the remaining images are testing images, and the number of clients is 13. We additionally select five images from remaining 13 impostors to be testing images. The value of threshold means the value of similarity measure of APM. Figure 4-7 presents the selected threshold for distinguishing the clients and impostors.
43
Fig. 4-7: The threshold distinguishing the clients and impostors
In Fig. 4-7, FRR indicates the false reject rate and FAR indicates the false accept rate. The decreasing line is the curve of FAR and the increasing line is the curve of FRR. We utilize the intersection of FAR and FRR to be the similarity threshold.
Based on the experimental result, the value of threshold for distinguishing the clients and impostors is set as 0.000125 for on-line system tested in the following section.
4.2.2 On-line testing
Results of face identification
In this part, we show the results of on-line testing. Figure 4-8(a), (b) presents the results of a single person and multi-persons generated from the proposed face
44
identifier. First, the faces are extracted from the image by the face detector, then the face identifier recognizes the faces as registered clients or impostors.
(a)
(b)
Fig. 4-8: The testing result generated from the proposed face identifier. (a) The results of a single person (b) the results of multi-persons
On-line registering new clients
The capability of registering new clients requires capturing five images of different head orientations for a new client. Figure 4-9 is the overall clients and Fig.
4-10 shows the result of face identifier. According to the overall clients, one of two persons is an impostor and the other is the client as shown in Fig. 4-10. The following section presents the procedures of on-line registering new client for the impostor in Fig. 4-10.
First, our system has to capture five images: upward, downward, leftward,
45
rightward and frontal faces to register the information of a new client. Figure 4-11(a)~(e) present the procedures of capturing five images. Figure 4-12 shows the overview of updated clients. Figure 4-13 shows the result after registering the new client.
Fig. 4-9: The overview of clients
Fig. 4-10: The result of face identifier
46
(a)
(b)
(c)
47
(d)
(e)
Fig. 4-11: (a)~(e) present the procedures of capturing five images
Fig. 4-12: The overview of updated clients
48
Fig. 4-13: The result of registering client of face identifier
4.3 Discussion
For face detection, Table 1 shows that the detection rate of our face detector is more than eighty percent, and it is sufficient for a practical entrance system. The training and testing data are all frontal faces which contains many faces of eastern people.
However, there are still some situations that may cause the system fail. Figure 4-14 and Fig. 4-15 are the examples of system fail. Sometimes the shape and texture of non-faces is too close to the truly faces, the faces are non-frontal faces or partial occlusion or too small to be detected.
49
Fig. 4-14: Examples of system fail #1
Fig. 4-15: Examples of system fail #2
For face identification, Table 3 shows that the detection rate of our face identifier is more than ninety percent for the five training images for each person, and it is enough for many face identification applications. The training and testing data all consist of five images of different head orientations.
There are some situations making the system fail, such as false acceptance and false rejection. False acceptance means the impostor is identifying as the client by the face identifier; oppositely, false rejection means the client is identifying as the impostor. Figure 4-20 is an example of false acceptance , the impostor (Fig. 4-16(a)) is identified as the client (Fig. 4-16(b)).
50
(a)
(b)
Fig. 4-16: Examples of system fail #3: (a) the impostor (b) the client
51
Chapter 5
Conclusions and Future Work
In this thesis, we present a system for the multi-client identification using adaptive probabilistic model (APM). The design of this system is based on the principles of robustness and practicability.
For face detection, the technique of lighting normalization using histogram fitting is applied to improve the performance of face detector., and a region-based clustering method is proposed to deal with the problem of multi-candidates around the faces The experimental results show that the process of lighting normalization can actually improve the detection rate of face detector.
For face identification, adaptive probabilistic model (APM) is introduced to model the characteristic of the clients. According to the design of APM, the system can on-line register new clients and on-line update the information of the clients. The APM is composed of five images of different head orientations for each person. By the process of adaptive updating, the weights of five different poses and the matched probabilistic function are adjusted to adapt the latest information of registered clients..
The experimental results show that the proposed APM technique actually has the good performance for face identification.
To further improve the performance and the robustness of our system, some enhancements can be done in the future:
(a) A robust face detector is necessary for a practical face recognition system. If the faces are not extracted from the images correctly, the face identifier cannot work in the following process. For our system, one of the restrictions is that we cannot handle the variant poses of non-frontal faces, such as the
52
side faces. It is probably to select the variant poses of non-frontal faces be the training sets to solve this problem.
(b) It is hard to identify the face if the variation of the rotated angle for the face is too big. Before the process of face identifier, we may employ a method of face calibration to deal with this problem. Furthermore, this method of face calibration can also use in face detector.
(c) For face identification, a threshold is used to discriminate between the clients and the impostors. There is a trade-off problem on the selection of the threshold. A big threshold can prevent an imposter being identified as a client, but it also increases false rejections of clients. There are two possible solutions for this problem: one is to increase the specificity of face model, the other is to apply feature extraction methods which can extract more distinguishable features than eigenfaces.
(d) The tolerable degree of the proposed system may not be affordable for requirements of large numbers of clients, for instance, an entrance guard system for a big company with hundreds of staff. Hence the tolerable degree of the identification system should be improved in the future.
(e) This work lacks quantitative results of the whole face recognition system that consist of face detection and face identification. In the future work, this kind of experiments will be designed and implemented.
53
References
[1] A. k. Jain, R. Bolle, and S. Pankanti, “Biometrics: Personal Identification in Networked Society,” eds. Kluwer Academic, 1999.
[2] D. Zhang, “Automated Biometrics: Technologies Systems,” eds. Kluwer Academic, 2000.
[3] C.P. Papageorgiou, M. Oren, and T. Poggio, “A General Framework for Object Detection,” International Conference on Computer Vision, pp. 555-562, Jan.
1998.
[4] Viola Paul and J. Jones. Michael, “Robust Real-Time Face Detection,”
International Journal of Computer Vision 57(2), 137-154, 2004.
[5] Y. Freund and R. E. Schapire, “A decision-theoretic generalization of on-line learning and an application to boosting,” In Computational Learning Theory:
Eurocolt 95, Springer-Verlag, pp. 23-57, 1995.
[6] S. Z. Li, Q. Zhang, H. Shum, and H. J. Zhang, “FloatBoost Learning for Classification,” Neural Information Processing Systems, 2002.
[7] Rainer Lienhart and Jochen Maydt, “An Extended Set of Haar-like Features for Raid Object Detection,” IEEE ICIP 2002, Vol. 1, pp. 900-903, 2002.
[8] P. N. Belhumeur, J. Hespanha, and D. Kriegman, “Eigenfaces vs. fisherfaces:
recognition using class specific linear projection,” IEEE Trans. Pattern Anal.
Mach. Intell. 17 (7) (1997) 711-720.
[9] T. F. Cootes, J. F. Edwards, and C. J. Taylor. “Active appearance models,” IEEE Trans. Pattern Anal. Mach. Intell. 23 (6) (2001) 681-685.
[10] L. Wiskott, J. M. Fellous, and C. Von Der Malsburg, “Face recognition by elastic bunch graph matching,” IEEE Trans. Pattern Anal. Mach. Intell. 19 (1997)
54
775-779.
[11] K. W. Bowyer, K. Chang, and P. Flynn, “A Survey of approaches and challenges in 3D and multi-model 3D + 2D face recognition,” Comput. Vis. Image Understanding 101 (1) (2006) 1-15.
[12] W. Zhao, R. Chellappa, A. Rosenfeld, and P. J. Phillips, “Face recognition: a literature survey,” ACM Comput. Surv. (2003) 399-458.
[13] S. Du, and R. K. Ward, “Face recognition under pose variations,” J. Franklin Inst. 343 (2006) 596-613.
[14] D. J. Beymer, “Face recognition under varying pose,” Technical Report 1461, MIT AI Lab, Massachusetts Institute of Technology, Cambridge, MA, 1993.
[15] M. Kirby, and L. Sirovich, “Application of the KL procedure for the characterization of human faces,” IEEE Trans. Pattern Amal. Mach. Intell. 12 (1) (1990) 103-108.
[16] L. Sirovich, and M. Kirby, “Low-dimensional procedure for characterization of human faces,” J. Opt. Soc. Am. 4 (1987) 519-524.
[17] M. Turk, and A. Pentland, “Eigenfaces for recognition,” J. Cogn, Neurosci. 3 (1) (1991) 71-86.
[18] A. Sehad, H. Hocini, A. Hadid, M. Djeddi, and S. Ameur, “Face recognition under varying views,” Lect. Notes Comput. Sci. 1811 (2000) 258-267.
[19] H. Murase, and S. K. Nayar, “Visual learning and recognition of 3-D objects from appearance,” Int. J. Comput, Vis. 14 (1995) 5-24.
[20] R. C. Gonzales and R. E. Woods, “Digital Image Processing,” Addison Wesley, 1993.
[21] R. Ebrahimpour, E. Kabir, M. R. Yousefi, “Teacher-directed learning inview-independent face recognition with mixture of experts using single-view eigenspaces,” Journal of the Frankin Institute 345 (2008) 87-101.
55
[22] S. Lawrence, C. L. Giles, A. C. Tsoi ,and A. D. Back, “Face Recognition : A Convolutional Neural-Network Approach,” IEEE Transaction on Neural Networks, Vol. 8, NO.1, January 1997.