• 沒有找到結果。

不受表情影響的人臉配對特徵點抓取方法研究

N/A
N/A
Protected

Academic year: 2021

Share "不受表情影響的人臉配對特徵點抓取方法研究"

Copied!
54
0
0

加載中.... (立即查看全文)

全文

(1)

國立交通大學

應用數學系數學建模與科學計算碩士班

碩 士 論 文

不受表情影響的人臉配對特徵點抓取方法研究

A Study on Expression Invariant Feature Points

for Human Faces Matching

研 究 生 : 曾茂清

指 導 教 授 : 林松山 教授

(2)

不受表情影響的人臉配對特徵點抓取方法研究

A Study on Expression Invariant Feature Points

for Human Faces Matching

研究生 : 曾茂清 Student : Moa-Ching Tseng

指導教授 : 林松山 教授 Advisor : Dr. Song-Sun Lin

國立交通大學

應用數學系

數學建模與科學計算碩士班

碩 士 論 文

A Thesis

Submitted to Department of Applied Mathematics College of Science

Institute of Mathematical Modeling and Scientific Computing

National Chiao Tung University

in Partial Fulfillment of the Requirements

for the Degree of

Master

in

Applied Mathematics

July 2012

Hsinchu, Taiwan

(3)

不受表情影響的人臉配對特徵點抓取方法研究

研究生: 曾茂清 指導教授 : 林松山

國立交通大學

應用數學系數學建模與科學計算碩士班

摘要

本論文我們在一開始介紹了許多局部形狀特徵描述子,我們根據許多

文獻整理出了常見的一些描述子的概念、特性和缺點。接下來我們針

對臉部的正面掃描介紹許多基於不同描述子而衍生出的演算法和方

法,之後我們討論了由不同姿態與表情變化所衍生出來的問題。我們

對於正面與不同姿態的臉部影像整理出了一些結論並對於受表情影

響很大的嘴唇特徵點的提取提出一些想法,希望可以在未來加以實

現。

(4)

A Study on Expression Invariant Feature Points for

Human Faces Matching

Student: Ting-Hsuan Lien Advisor : Song-Sun Lin

Department of Applied Mathematics College of Science

Institute of Mathematical Modeling and Scientific Computing

National Chiao Tung University

July, 2012

Abstract

In this study, we give an overview of some common local shape feature

descriptors. Their concepts, properties and shortcomings are organized

according to lots of literature. We then provide a discussion of facial

feature extraction methods. Based on different local feature descriptors,

we enumerate the corresponding methods and algorithms for the frontal

facial scan. Then we discuss the problems caused by changing pose and

expression variation respectively in detail and propose some ideals to

address the problems. We conclude with a summary and promising future

research directions for solving the problem of mouth feature points

extraction.

(5)

致謝

在完成這篇論文與順利完成口試以後,首先我想要先感謝我的指

導老師林松山老師,不論是在學術方面還是在做人處事方面,老師總

是給我許多的幫助與指導,在這裡真的很謝謝老師。另外,也很謝謝

林文偉老師在論文上不吝給予我意見與指導,並且在數學上給予很多

的教導與指正;再來我還要感謝何丹奇學長在最後幾個月的時間,百

忙之中抽空指導我們,幫助我們完成畢業論文,在這裡謝謝學長的幫

忙,也恭喜學長當了爸爸,真的很謝謝。

在最後幾個月的寫論文的期間裡,很謝謝爸爸媽媽常常打電話來

關心我,叮嚀囑咐雖然要用功,但身體依然要好好照顧,在那艱困的

期間,這些話都彷彿是大補丸般的振奮我的精神;也謝謝在這段期間

一起奮鬥的吳侑燊、連、許尚...等等,大家一起共患難,在壓力下

一起開無聊玩笑,一起大吃大喝,一起大聲抱怨,一起解決困難,這

種感覺現在回想起來,其實是很奇妙也很美好的;謝謝大學同學胖

胖、阿杵、阿亮、貓咪、阿Ken、換……,在H棚7-11的純沏茶聚會

總是讓我暫時拋開了所有壓力與煩惱,謝謝大家的加油與打氣,因為

來自各方的鼓勵與支持,讓我可以撐過所有難關,順利寫完畢業論

文,感謝大家。

(6)

最後,也感謝神明的保佑,賜與我智慧與幸運,讓我有力量面對

一切,完成階段性的任務,感謝所有人、所有事、所有物,讓我成長,

讓我可以不斷地往前邁進,謝謝,謝謝,真的謝謝。

曾茂清

謹誌于交通大學

2012年7月

(7)

Contents

1 Introduction 2

2 3D Local Feature Descriptors 6

2.1 Feature Descriptors Based On Curvature Analysis . . . 6

2.2 E¤ective Energy And Distance To Local Plane . . . 14

2.3 Pro…le . . . 16

2.4 Spin Image . . . 20

2.5 Comparison . . . 24

3 Expression Invariant Facial Feature Extraction 26 3.1 Extraction of Facial Feature Points on a static Frontal Facial Scan . . . 28

3.2 A Facial Scan with Head Pose Variation . . . 36

3.3 A Facial Scan with Facial Expression Variation . . . 40 4 Conclusion and Future work 42

(8)

Chapter 1

Introduction

Human face analysis is becoming a crucial technique in computer vision application of the human face analysis range from the security assurance to the movie industry.Automatic facial feature point extraction is the most important issue in analyzing the face.Many facial analysis processing,for example,facial feature tracking,pose normalization,facial expression analysis and face recognition,require a reliable feature points extraction.For example,for 3D face recognition,the widely used iterative closet point (ICP),a rigid registration method requires accurate and robust automatic landmarking to achieve a good registration result. Lots of facial feature were discussed in the literature such as eye corners,nose bridge,nose tip,nose bases,mouth corners and chin tip.In Fig.1,all the commen facial feature points are shown.

(9)

Figure 1: Position of the 11 landmarks which are the most often used in the literature.

Although lots of researches were devoted in the extraction of facial features,these meth-ods are still fall into some limitations.The major problem in extracting the facial features is that the local geometric features such as curvature and the variation in illumination will deviate the features,which leads to unstable results.The challenge increases when the pose and epression of the face changes.For extracting facial features from the 3D surface,another challenge comes from the inaccuracy of the 3D ascquisition techniques.Noise,holes,and missing parts of the facial model will also increase the di¢culty of extraction facial fea-tures.

In the 2D colored image,the extraction of facial features from the image has been stud-ied for more than a decade.These methods can be classi…ed as appearance-based,geometric-based and structure-appearance-based,geometric-based.Appearance-appearance-based,geometric-based methods use the basis vectors to represent the face and its facial features.The commen transformations are principle components analysis (PCA) [1],[2]Gabor wavelets [3],[4],independent components analysis (ICA) [1]

(10)

and discrete cosine transform (DCT)[5].If the image is transformed to the subspace repre-sentation,it can be processed by machine learning techniques like support vector machine (SVM) [1].Geometric-based methods use prior knowledge about the face position,and con-straining the landmark search by heuristic rules that involve angles,distances,and areas [4],[5],[6].Structure-based methods use the whole candidate landmarks to …t to a model of feature locations and decide the feature point. The research of facial feature extraction in 2D has achieved a good results.However,facial feature extraction on 2D data is sensitive to illumination,pose variations,facial expression,and make up,especially illumination.

In the recent years,the works of facial feature extraction gradually focus on 3D data.Most of the reasons is the 3D shape is independent of illumination.Besides,the 3D shape which is related to the structure is also independ of make up.These two reasons basically address the main problem in 2D.Therefore,it is often thoght that the use of 3D has the potential for extraction accuracy than the use of 2D face image.

There are many facial feature extraction methods in 3D in the existing works.We can say that the comment feature points in a 3D facial scan have local shapes su¢ciently di¤erent from their neighbour’s local shapes,so they can be extracted under a range of 3D shape condition.Hence,the local shape descriptors play an important role in whole meth-ods.However,human face is a complex living organism,it contains di¤erences in expression and pose.Many existing methods are able to automatically locate the feature points based on the assumption of a frontal facial scan with natural expression.There are few meth-ods addressing the problems in the presence of large pose change and large expression variation.

In this paper,we try to make a systematic discuss on facial feature extraction.The remainder of the paper is organized as follows:Chapter 2 gives an comprehensive intro-duction of 3D local shape descriptors.Chapter 3 roughly categorizes the existing methods

(11)

into three categories:For the frontal facial scan with natural expression,facial scan with head pose and facial scan with expression variation.We will give a detail discussion for each category.Finally,Chapter 4 provides conclusions and future work.

(12)

Chapter 2

3D Local Feature Descriptors

A local feature is a kind of signal or pattern with speci…c meaning and can be distinguish easily from its neighborhood. Such feature can be de…ned by the point, edge, or small patch on the image or 3D surface domain. To extract features from an image or a 3D sur-face requires the de…nition of some local shape descriptors. In this chapter, we introduce some descriptors to identify features from a given 3D surface.

2.1

Feature Descriptors Based On Curvature

Analy-sis

Curvature is a property of the local surface.It is usually regarded as a tool of describing the curve’s degree.On a curved surface,the features we think all fall on the bending places which may be ridges,valleys or peaks.It is feasible to use curvature to characterize the features on a curved surface.This section will introduce some commen shape descriptors based on curvature.

(13)

…rst de…ne the curvature on a general curve as the reciprocal of the radius of a osculation circle at p.And if there is a plane contained p and the curve,we can see the the unit vector emanating from the point p and perpendicular to the surface is called the unit normal vector. A normal plane at p is a plane that contains the normal vector. Intersection of normal plane and the surface is a curve,The curvature of the planar curve is called normal curvature at p in the speci…c direction.The maximum and minimum normal curvatures at a point de…ne the principle curvature denoted by max and min.

In [7, 39],Gorden provides the de…nition of ridge lines and valley lines.Ridge lines is the local maxima in maxalong the line of maximum curvature and valley lines is similarly the

local minima min in along the line of minimum curvature.After computing the principle

curvatures at each point,Gorden thresholded the maximum and minimum curvature maps by setting appropriate thresholding value of extreme curvature to …nd the lines what people are really interested in.Fig.2 shows ridge and valley lines for a human face. Note how clearly the characteristic features of the face are displayed by these extrema.

Figure 2:(a)Ridge line:local maxima of in direction of maximum curvature (max  )

(14)

From the principle curvature ,we can derive two curvature measures from the princi-ple curvature, the mean curvature (H) and the Gaussian curvature (K).Gaussian curva-ture,K,is the product of principle curvatures.Formally,it is de…ned as

 = max£ min (2.1) where max and min are maximum and minimum principle curvature as previous

men-tioned.Gaussian curvature represents the total bending degree at p on the curved sur-face.Mean curvature,H,is the arithmetic average of principle curvature and is de…ned as

 = max+ min

2 (2.2)

where max and min are maximum and minimum principle curvature as previous

men-tioned.Mean curvature represents the average bending degree at p on the curved sur-face.These two curvature measures show the characteristics of the local surface around a point.For instance,Points with positive Gaussian curvature are called elliptic,points with negative Gaussian curvature are called hyperbolic and points with zero Gaussian cuvature are at planar.In 1986,Besl introduced the HK segmentation dependimg on the sign of the Gaussian and Mean curvature [13],which is calculated from the two priciple curvatures

max and min.The classi…cation of surface types based on the sign of Gaussian curvature

(15)

Table1:HK classi…cation based on the sign of the Mean and Gaussian curvatures [13]. Table1 shows there are four kinds of regions are classi…ed:(+)(+) are convex,(+)(¡) are concaves,(¡)(+) are saddle with min + min  0,(¡)(¡) are saddle with

min+ min  0[7].If we apply HK segmentation method on human face analysis,it also

mainly segmentation the human face into four kinds of regions.We can see an example in Fig.3.The red zones are elliptical concave regions,green zones are elliptical convex re-gion,yellow zones are hyperbolic concave regions and blue zones are hyperbolic convex region.

(16)

Figure 4:The HK classi…cation map and its threshold version on human face [9].

According to the prior knowledge of human face,we can know most facial feature points ususally have the property of high curvature.Colombo [9] de…ned  and  as the thresh-old values for isolating the candidate nose tips and eye corners.Fig.4 shows the threshthresh-old- threshold-ing result on human face.Although we can see the good classi…cation ability on thecom-bination of Gaussian curvature and mean curvature from …g.4,the Gaussian curvature is sensitive to the scale.It is not a good shape descriptor which should be irrelated to the scale,translation and rotation and only related to the pure shape.In the following,we will introduce shape index which is irrelated to the scale and more related to the pure shape. Koenderink and van Doorn proposed an alternative curvature representation,shape index [34].His approach decouples the shape and the magnitude of the curvedness.They de…ned the shape index,S,as a quantitative measure of the shape.The formula is as fol-lowing  = 2 £ tan ¡1(max() + min() max()¡ min() ) (2.3) where max and min are the principle curvatures of the surface.The shape index ranges

from -1 to 1.A convex surface point with equal principle curvatures has a shape index 1.A concave surface point with equal principal curvatures has a shape index of -1.A saddle surface point with principal curvatures of equal magnitude and opposite sign has

(17)

a shape index of 0. The index covers all shape except for the planar shape which has

max = min = 0causing an inderminate value of shape index.

But in 1997,Dorai and Jain found out the local information about each shape category is not maintained distinctly with Keonderink and van Doorn’s de…ntion .Hence Dorai and Jain proposed an extension de…nition of Koenderink and van Doorn’s original de…nition [15, 16] and let the shape index range from 0 to 1.The extension formulation is de…ned as follow () = 1 2 ¡ 1 tan ¡1 max() + min() max()¡ min() (2.4) where max and min are the principle curvatures of the surface.Every didstinct surface

shape corresponds to a unique value of  except the planar shape.Nine well-known shape

categories and their corresponding shape index are shown in Fig. 5.

(18)

Figure 6:Nine representative shapes on the scale [15].

And the representative shapes from each category are graphically illustrated in Fig. 6.All basic shape types based on the signs of Gaussian and mean curvatures that was adopted by Besl are included in Dorai and Jain’s framework.

The shape index of a point on curved surface is not only independent of its position and orientation in space,but also independent of its scale.In order to obtain the scale di¤erences between objects,Koenderink and van Doorn introduced the curvedness to measure that how highly or gently a curved surface is.The curvedness [15, 16] is de…ned as

() =

r

2

max() + min2 ()

2 (2.5)

It equals to zero only at a point that has no curvedness,i.e. the point is on a planar patch.The combination classi…cation of the shape index and the curvedness,the SC clas-si…cation,often used to classify the curved surface.It can be applied on human face to isolate the salient feature regions by setting a appropiate threshold value.In [35],Nair and Cavallaro use shape index and curvedness index to describe the facail features as shown

(19)

respectively in Fig. 7(a) and Fig. 7(b).

(20)

Cantzler and Fisher [13] compared the di¤erence between HK and SC curvature de-scription method by checking the classi…cation ability under the classi…cation threshold varying and the noise increasing.Their conclusion indicates that the SC classi…cation is more stable at low thresholds and can deal better with image noise in image.Therefore SC classi…cation scheme has a slight advantage when dealing with real scenes containing multiple surfaces and moderate noise.

2.2

E¤ective Energy And Distance To Local Plane

In this part,we will introduce two similar feature discriptor:E¤ective Energy (EE) [28] and Distance to local plane (DLP) [26, 27]. Both of them use the concept of neighboring points to identify the points over convex, ‡at and concave areas of the surface.

We start by introducing the concept of E¤ective Energy. For a surface point  , the E¤ective Energy of  is de…ned by the inner product of the surface normal at  and the di¤erence vector of the surface point  around  within a speci…c distance  as in

Eq. 2.6. Fig.8 reveal the details of E¤ective Energy in which we can …nd the surface point

’s neighboring points within a sphere centered at  with a propriate radius .Suppose

 is in ’s neighborhood,the de…nition of e¤ective energy (EE) is provided as follow:

 = (¡  ) ¢  =jj¡  jj cos  (2.6)

We can get a e¤ective energy set by calculating the product for each point in neigh-boring set. The result of e¤ective energy set can reveal the concaveness of  . All e¤ective energy are positive value implies the shape around the surface point is concave, and, on the contrary, all e¤ective energy are negative value implies the convex shape there. For surface point having all e¤ective energy close to 0, the shape there is almost ‡at.

(21)

Figure 8:The e¤ective energy of the neighboring points.

Figure 9:Distnce to local plane.

Next we will introduce the Distance to Local Plane descriptor (DLP) [26, 27]. It is not much di¤erent from e¤ective energy.As shown in Fig.9, For a surface point  , the

(22)

Distance to Local Plane of  is de…ned by the inner product of the plane normal at  and the di¤erence vector  from  to the average position of surface point  around 

within a speci…c distance  as in Eq. 2.7.

Suppose  is the test point,and let  = f1 2  g be  ’s neighboring points

within a sphere centered at  with radius .We can …nd a plane  which can …t the neighboring set  with a normalized normal ¡!.Then,the Distance to Local Plane of  can be provided by calculate the inner product of ¡! and  ¡ ,where  is the mean

coordinate of  ’s neighboring set .The formular is shown as follow:

(  ) = ( ¡ ) ¢ ¡! (2.7)

Similar to EE, DLP can be used to identify the concaveness of the spape around the surface point. Romero indicated that DLP is stable, computationally inexpensive and implemented with any linear algebra package in [26, 27].Both EE and DLP are simple local shape descriptors, which are independent to the rotation and the translation. Fur-thermore, they are both robust to the pose variation. However, due to the limitation of identifying only the concaveness of shape, the E¤ective Energy is used usually to the coarse identi…cation work.To the contrary,DLP can identify the shape more precisely by bounding the allowable values of DLP using the Mahalanobis metric,refernced to the mean and variance of the training data [36].

2.3

Pro…le

The shape descriptors we described in the previous two sections are the local shape de-scriptors, which is determined by analyzing the local surface region around the point.To identify some structural features such as the human face based on the above

(23)

descrip-tors may be failed due to the complex shape of human face.In this section, we introduce another shape descriptor which takes the structural information of features into account. The pro…le descriptor has been discussed sice Francis Galton’s article in Nature in 1888 [37],and applied on face recognition sice Harmon in1977 [38].On human face,the only useful facial pro…le we need is cetral pro…le.Other pro…le on human face can not su¢ciently represent the human face characteristic.If the facial pro…les can not be rep-resentation of human face,the following works,such as extracting the feature points and face recognition,could be failed or get a bad result. Fig.10 shows the example of the bad and good facial pro…le.

Figure 10(a):we choose the mid-line of the facial scan in the left and the corresponding facial pro…le is portrayed in the right.

Figure 10(b):we choose the line far from the mid-line of the facial scan in the left and the corresponding facial pro…le is portrayed in the right.

(24)

In order to extract a representative pro…le,namely cetral pro…le of human face, there are some manners to decide the intersection plane or the facial pro…le.We can roughly classify the manners into two categories.One is looking for the symmetric plane of a facial surface and the other one use the extreme valueof the coordinate to extract the facial pro…le.A representative facial pro…le seperate the face into two symmetric parts,so ,in fact,we try to look for the symmetry plane from symmetrical facial surface.Pan et al [24] propose an e¤ective approach based on alignment.Suppose there is a symmetrical facial surface.At …rst,he set an initial symmetric plane and the mirrored surface of the original facail surface can be easily obtained,as shown in Fig. 9.We can see the original facial surface with a marked point  in Fig. 11(a).The mirrored surface of the facial surface with a marked point 0 which is symmetric to point  respect to the initial symmetrc plane is shown in Fig. 11(b).Then Pan aligned two facial surfaces respective in Fig.11(a) and Fig.11(b).If the alignment of two facial surfaces is accurate,the segment

0 is perpendicular to the symmetric plane.Namely,the vector¡¡!0 is the normal of the

symmetric plane.Therefore,the symmetric plane can represent as ¡

! ¢ ¡! +  = 0 (2.8) where ¡! = ¡0

jj¡0jj is the noraml of the symmetric plane and  is a constant.Fig.11(c)

shows the aligment of both facial surfaces.

According to the prior knowledge of the human facial surface,we can discover that the facial pro…le is a set of points with maximum coordinate value.In [18],Xiaoguang Lu and Anil K. Jain found out the points on nose bridge including the nose tip are close to the mid-line and have the extreme z values in the frontal face.Hence,they looked for the position with the extreme z value for each row.Then,the column which contains the greatest number of the extreme z value points is considered as mid-line in the frontal

(25)

Figure 2.1: Figure11:Finding symmetry plane with alignment.

face.The mid-line is the facial pro…le.Fig.12 illustrates how to …nd the face mid-line.

Figure 12:Finding face mid-line. (a) The yellow marks represent the positions where the z value reaches the extremum along each row. (b) Total number of extreme z values (yellow points) in each column. (c)

The mid-line (in blue) is located by choosing the column with the maximum peak in (b) [18].

Faltemier [21] and Segudo both [30] use the concept of projection to collect the point with maximum coordinate value.Faltemier [21] projected all points in the frontal face to yz-plane and xz-plane to extract the vertical and horizontal pro…le curve respectively.And

(26)

Segudo’s rotated pro…le signatures (RPS) method in [30] projected all the points in the facial surface with any pose to xy-plane to extract the rightmost pro…le.We can consider the process of projection as a behaviour of pressing something ‡at.For example,if we want to project all the points in a facial surface to yz-plane,the process of projection seems like that we press the facial surface in x direction and let it be ‡at on yz-plane.After projection,the points in the boundary of the ‡at patch on yz-plane are the points with maximum z-coordinate value.This explain why we can use the concept of projection to extract the facial pro…le.

2.4

Spin Image

The spin image is …rst introduced by Andrew E. Johnson in.The key concept of generating the spin image is the use of oriented points.There are two kinds of oriented coordinate sys-tems.One is object-otriented coordinate systems which are coordinate systems …xed on a surface.Another is viewer-oriented coordinate systems which are based on the viewpoint of the oberserver of the surface.Johnson [31] adopted the object-oriented coordinate systems for the view independent of the description of a surface under the changing viewpoint.

An oriented point  at a surface mesh vertex can be de…ned by the 3D position of the surface vertex and a surface normal which are denoted by  and  respectively [31].By using the tangent plane  through  which is perpandicular to  and the line

through  which parallel to ,it will achieve a representation in a form ( ) based on cylindrical coordinate system where  is the perpendicular distance to  and  is the signed perpendicular distance to the plane  [29].Fig.13 illustrate the detail of the creation of cylindrical coordinate system.

(27)

Figure 13:Parameters of Spin Image

We can de…ne the transformation precedure as a projection function of the 3D points

to 2D coordinate ( ) associated with the 2D basis ( ) that corresponds to the oriented point  [18].The projection function (spin map) is shown as following [16]:

0 : 3

! 2

0()! ( ) = (pjj ¡ jj2¡ ( ¢ ( ¡ )2) ¢ ( ¡ ))

Before we introduce how to generate the spin image,there are four paremeters,bin size

,image width  ,support distance  and support angle ,need to be recommanded

…rst.Bin size is an important parameter in spin image generation.It not only determines the storage size of the spin image but has an e¤ect on the descriptiveness of the spin images.A saitable bin size can reduce the in‡uence of indivisual point position.It is better to set the bin size based on mesh resolution.The reason is that the size of the shape features

(28)

on an object and the density of points in surface mesh e¤act the mesh resolution.Johnson in [31] indicated that if the bin size was set to be one or two times the mesh resolution,the resulting spin image can properly describe the global shape.Three spin image of decreasing bin size for a point on the duckie modal are shown in Fig.14 [31].Johnson [31] let the number of rows and columns in a spin image equaled to each other and de…ned the number of rows or columns in a square spin image as image width.A image width can decide the amount of global information in a spin image.Support distnace is de…ned as the product of the image width and the bin size.The amount of global information swept ot by spin image can be determined by the support distance.The last parameter,support angle,is the maximum angle between the direction of the oriented point and the surface normal of points which can contribute to the spin image.

Figyre 14:The e¤ect of bin size on spin image appearance [31].

To create aspin image,the procedure can be introduced in detail in the following [31].First,we have to select an oriented point  from a vertex of the surface mesh.Then the spin map coordinates with respect to the oriented point are computed for each vertex on the surface mesh.The next step is to screen the vertex  which meets some criteria based on the distance from the oriented point and the support angle.For example,suppose

(29)

that the oriented point  with its position and normal is set as ( ), and there exists

another point  on the surface mesh with the position and normal as ( ).If  satis…es

cos¡1(

¢ )  , can be accumulated in the spin image respect to the oriented point

.Once the vertex  can be accumulated in the spin image,the gridding index of the bin which  is projected in is determined.The index of the bin ( ) can be computed as follow

 =b 2 ¡  c  =b c (2.9)

where  is the bin size , is the image width,( ) is the spin map coordinate and b¢c is the ‡oor operator.By using the formula we can grid the 2D points into the spin image bins.In the gridding procedure,an error is produced by the ‡oor operator.We can use the spin map coordinates,( ),and the gridding indices to do bilinear interpolation.The result of the bilinear interpolation is the contribution of a point to its surrounding grid locations.We can calculate the bilinear interpolation weights acoording to the formula [31] as follow

 = ¡   = ¡  (2.10) And we can see the total procedure of the creation of the spin image described in pseudo code in Figure 15.

(30)

Figure 15:The procedure of the creation of the spin image description [31].

Spin image can show the global propertires of any surface in an object oriented coor-dinate system rather than a viewer oriented coorcoor-dinate system.It transform the 3D points into 2D cyclindrical coordinate system.( ) Sice coordinates are measured according to the oriented point and its normal,we can infer that the spin images are rotation and translation invariant,but they are not scale invariant.If there are two surfaces with the same shape but di¤erent scale,the spin image of the two surfaces will be di¤erent.Even so,we still can see the strong robustness and good adapatability descriptiveness of the 3D shape.

2.5

Comparison

According to the analysis above,we make a comparison for the descriptors we menttioned before.We can say the most commenly used descriptor is the curvature based descriptors such as shape index [15, 16, 17, 18, 19, 20],HK classi…cation [9, 10, 11, 12, 21, 23, 25] and principle curvature [7].In [14],Ceron compared the priciple curvature,mean curva-ture,Gaussian curvature,shape index and curvedness to determine which descriptors is

(31)

the most representative descriptors.He designed two kinds of tests : the …rst one tried to determine which descriptor is the most represenrative descriptor over all the points and the second one is composed by several test depending on the facial region.The experiment results showed that the best shape descriptor points in the human face is the shape index in any test.

Except the curvature based descriptors,E¤ective Energy and Distance to Local Plane both use the relatonship between the feature point and the points in its neighborhood to identify the feature points.Because of the weak representativeness,they only can roughly identify peaklike shape,valleylike shape or planar shape.Unlike EE and DLP,spin image is a strong local descriptor.It transform the 3D coordinate into 2D cylindrical coordi-nate system.The cylindrical representation makes this local descriptor be robust to the rotation.This property let spin image have stronger descriptiveness than other descrip-tors,however,its computation cost is expensive.This shortage is an urgent problem needed to solve immediately for spin image.If we want to obtain the structural information of hu-man face,facial pro…le is a good descriptor.The facial pro…le is important to the description of the nose feature.It is comprehensively applied on the nose region extraction.But it is very sensitive to rotation caused by its structural information.In the following chapter,we will introduce many people used di¤erent algorithms or combined di¤erent descriptors to …x up the descriptors’ shortcomings and make their extraction system more robust.

(32)

Chapter 3

Expression Invariant Facial Feature

Extraction

In chapter 2,we introduced some shape descriptors of 3D surfaces. Before we try to identify the feature points from the complex data,the local shape contributes corresponding to each point on a facial surface should be computed beforehand. By understanding the priori knowledge of the human face,we can establish the discriminative criterions for each feature point on human face.The most prominent feature point is clearly the nose. The nose sticks out from the whole face, and has a roof-like shape. Moreover, the nose falls approximately in the center of the whole face. These properties make the nose tip quite easy to be extracted. Apart from the nose tip,the eye corners are also easy to identify features on human face. We can say that the eye corners has a valley-like shape in the local shape aspect. It is easy to extract the eye conners by using the curvature analysis or other local shape descriptor analysis or by identify the colors around the shape. Also, because of the invariance to the human expression,the nose tip and the eye corners (including the inner eye corners and the outer eye corners) are the most frequently extracted facial feature points. Extracting other facial feature points, such as the chin, the nose bridge,

(33)

the mouth corners, and the mouth lips, are not as easy as the salient of the nose tip and the eye corners, although they are still the facial features we are interest in. These facial features are also a¤ected easily by the varying expression and the changing of head pose. Hence the extraction of facial feature points except the nose tip and the eye corners needs the methods which are more robust to the variation of expression of the face and the pose of the head.

People have proposed many kinds of methods to extract the facial points.For example the most direct methods are only based on single shape descriptors such as Gaussian curvature and mean curvature,shape index,spin image or facial pro…le.And if the shape descriptor has weak representation on facial feature points,people will combine other de-scriptors to enhance the representation or use a heuristic framework or a cascaded …ltering to extract the facial feature points much precisely.More than that,some methods based on combining the information of 2D and 3D have been proposed and vari…ed the result of the methods based on the combination of 2D and 3D are better then the methods based on respectively in experiments.It is too hard to categorize these many methods in a system-atic way.Therefore we can re-think those methods in the opposite direction of previous view.We return to the facial scan itself.When acquiring the facial scan,we can …nd the facial scans are not always be the same because of the di¤erent head poses and di¤erent facial expression.Therefore,We can roughly categorize the facial scan into three group:the frontal facial scan,the facial scan with variantion of head pose and the facial scan with variantion of facial expression.It is no doubt that it is more easiear to localization the facial feature points on a frontal facial scan than on others.The missing facial data caused by the head pose and the variance of characteristics of feature points caused by the facial expression both a¤ect the result of the localization.Hence we will illustrate the methods of extraction in detail in the three directions and give a comparison of the methods in the

(34)

end of this chapter.

3.1

Extraction of Facial Feature Points on a static

Frontal Facial Scan

In this section,we start to introduce how to extract the facial feature points.If we limit the test facial scan must be a frontal facial scan,it means we will get a su¢cient information of the test facial scan.Many methods have been proposed to extract the facial feature points on a frontal facial scan.Most of the existing methods apply the priori knowledge of human face to the algorithm.and make use of facial geometry-based analysis to localize geometri-cally salient feature points such as nose tip,eye corners,chin tip,mouth corner,etc.And we can use the shape descriptors to describe the geometrical characteristics for each facial feature point and apply the corresponding methods to extract the facial feature points.

Many existing works use the curvature based descriptors to label their shape.HK claasi-…cation is a commen descriptor in curvature based analysis.in [12],Cheng found the inner eye corner region at …rst,and then located the nose region and the nose bridge.He used HK classi…cation labeled the surface type.However he found that if only use the HK classi…ca-tion,there are too many pit regions to …nd the corresponding inner eye corners.Therefore he removed the small pit region which involve a number of points lower than the threshold value at …rst.Then a pair of regions that has similar average value in both Y and Z can be regarded as inner eye corners.In the following,he search the region between two inner eye corners for the peak region.That is exactly the nose region.This heuristic algorithm create a good result of extracting the three feature points.Although HK classi…cation is rotation invariant.But Cheng used the frontal position to design the algorithm to localize the nose tip and inner eye corners.This makes the algorithm is sensetive to the head pose and only

(35)

useful in a frontal facial scan.In [9],we can see that Colombo also used HK classi…cation for locating three facial feature points:nose tip and inner eye corners.For improving the accuracy,Colombo used some …lters to reduce the search region and then regarded the region with the highest curvature value are the nose tip and the inner eye corners.He in-dicated that other regions,like the mouth,or cheeks,or forehead,do not present particular or simple curvature characteristics that allow robust autimatic detection.

In addition to HK classi…cation,shape index is used to label the surface type very often in curvature based analysis.Lu and Jain [18] proposed the method to locate the position of eye and mouth corners,and nose tip,based on a fusion scheme of shape index on a range image and the cornerness response on an intensity image.They also developed a heuristic method based on cross pro…le analysis to locate the nose tip more robustly on a frontal scan.By searching points with the maximum z value at each row,a column with the maximum number of points with the maximum z value is regarded as a mid-line on a facial scan as show in Fig.16.The vertical z pro…le along the mid-line can be shown as in Fig17.Lu and Jain assumed that the nose tip is close to the mid-line and the nose bridge presents a strong consecutive increase in z value.

Figure 16:Finding face mid-line. (a) The yellow marks represent the positions where the z value reaches the extremum along each row. (b) Total number of extreme z values (yellow points) in each column. (c)

(36)

Figure17:The depth (Z) pro…le along the mid-line [14].

Then they use these two properties to …nd the nose tip.After they found the nose tip,they use the nose tip to …t a stastical model of the facial features to be a prior constaint to reduce the search area for the feature points.The stastical model not only greatly reduces the cost of computation,but also enhences the accuracy of extraction

re-sults.Finally,they use the min-max rule to normalize the shape index,, and cornerness,,respectively.If the normalized shape index is denoted by 0(),the 0() can be computed at point  as

0() = ()¡ minfg maxfg ¡ minfg

(3.1) where fg is the set of shape index value for feature point in seach region.The cornerness

’s normalization is the same.At last,the …lnal score  () is computed by integrating scores from two modalities using the sum rule

(37)

The point with the highest  () in each search region is identi…ed as the corresponding feature point.From the previous process,extraction of eye corners and mouth tip can be accomplished.Lu and Jain tested their methods by 98 frontal scan with natural expression and 98 frontal scan with smile expression.Th experiment results showed that the eaxtrac-tion of nose tip by the heuristic mthods is good on a frontal scan and Lu also indicated that the combining information of 2D and 3D will improve the accuracy respectively rel-ative to 2D and 3D.But we …nd that the experiment data are all with natural or smile expression without great expression.We can not know whether their method is robust to the expression.And Lu and Jain made use of the integration of shape index from a range image and cornerness from an intesity image.Since the cornerness represent the properties in 2D,it may be in‡uenced by the illumination or the head pose.

Segundo [30] also proposed a face and facial feature detection method by making use of facial pro…le and combing 2D face segmetation on deoth images with surface curvature in-formation.He wanted to localize the eye corners,nose tip,nose base,and nose corners.Fig.18 illustrates the algorithm in a detail.Segundo used the HK classi…cation to isolate the region he was interested in at …rst.Then he computed two y-projections of the depth information which is named pro…le curve and median curve for …nding the nose tip y-coordinate.These curves are obtained by determining the maximum depth value,namely pro…lle curve,and the median depth value,namely median curve,of every set of points with the same y-coordinate from the face image.We can see the curves in Fig.19.And the y-y-coordinate with the maximum di¤erence value is the nose tip’s y-coordinate.

(38)

Figure18:Diagram of Segundo’s landmark detection approach.

Figure19:Example of Segundo’s nose tip y -coordinate detection [30].

For other feature points’ y-coordinate,Segundo followed the same procedure but com-puted on a curvature image.Hence we can get a y-porjection pro…le and this projection

(39)

pro…le presents three peaks,the eyes,nose base,and mouth.Under the nose tip y-coordinate is already known,Segundo used the relative position of nose tip and other feature points to …nd other feature points’ y-coordinate.Therefore,he indicated that the closest peak to the nose tip as being the nose base y-coordinate,the upside peak is de…ned as the eye cor-ners’ y-coordinate and the underneath peak represents the mouth y-coordinate as shown in Fig.19.The next step is to …nd the x-coordinate of all the feature points.The similar process is applied in this step but all works are around the x-projection pro…le.Segundo …rst …nd the nose tip’s x-coordinate and use this information to …nd the nose base.To …nd the x-coordinates of eye corners,Segundo computed the x-projection of the curva-ture image by calculate the percentage of pit curvacurva-ture points for every column in a set of neighbor rows centered in the eye y-coordinate;Segundo tested the whole system by totaling 2500 images of 100 subjects with high facial expression variations.The result of experiment shows a high accuracy on the frontal sacn and strongly robust to the facial expression.In [30],Segundo also compared the experiment result with Lu and Jain’s [18] methods and shows the better performance than others.But the system do not show the robustness to the head pose.

(40)

Without regard to the curvature based descriptors,Xu used e¤erctive energy to de-scribe the local distibution of neighboring points and make a hierarchical …ltering scheme based on two rules,rule 1 states that a point is the nose tip candidate only if all the components in the e¤ective energy set of the point are negative and rule 2 states that a point is the nose tip candidate only if the mean of the point’s neighbors’ EE is neg-atively smaller and the variance is positively larger than that of any other facial area [28].Fig.20 illustrates the process of detecting the nose tip.Although e¤ective energy has a weak representative for each point,Xu use a hierarchical …ltering scheme to make up the insu¢cient descriptiveness.He used three di¤erent databases to test his proposed algo-rithm.These three di¤erent databases include the facial scan with pose variation and the facial scan with expression variation.The result of experiment reach the correct detection rate up to 99.3%.Although the system is robust to the pose and expression variations,but it is certain only for nose tip.We do not sure about that the system still can perform well on detecting other feature points.

(41)

Cond [19] presented a 3D facial feature location method based on the spin image regis-tration technique.First,he wanted to isolate the candidate areas which contain the feature points.Cond assumed that the area of interest have a higher curvature.Hence he calculated the mean curvature at each point.Then he can isolate the candidate areas containing the facial feature points successfully.Once the candidate areas have been foud,the spin imge is computed for each points in the candidate areas.Since each point creates di¤erent spin images,Cond applied a support vector machine (SVM) classi…cation to compare these spin images for each point.The method was tested on a database of 51 subjects.The scans in the database include some scans with small pose variation.The small pose variation do not a¤ect the experiment result.The test results reach 99.66% on frontal scans.In[29],Cond indicated that although the spin image can be a powerful tool to represent the facial fea-tures,it requires a great computation e¤ort.How to reduce the great computation cost is an important problem that we need to solve to optimize the system.

In the previous,we have dicussed some commen methods corresponding to the shape feature descriptors included HK classi…cation,shape index,pro…le curve etc.Although the proposed methods all have good localization on frontal scan.It can not guaratee that if the facial scan with some variation whether the result of extraction of facial feature points is still good as the result in the natural facial scan.We not only introduce the concept of each method but also comprehensively discuss the possible problems in each method.Most of the problems we concern about are the e¤ect of the head pose variation and the e¤ect of the facial expression.Those problems will be discussed in the following two sections.

(42)

3.2

A Facial Scan with Head Pose Variation

In section 3.1,we limit the test scans must be frontal facial scans.But in the practical application,a facial scan with head pose is very commen,and so the head pose will result some facial data missing.Under this situation,if the methods still can maintain the high accuracy result of the Feature points extraction,we can say the methods are robust to the head poses.

From the local shape descriptors began to see,the shape descriptors such as spin im-age,DLP,Gaussian curvature,mean curvature,shape index etc. are basically pose invari-ant.But if one of them alone is not su¢ciently robust for detecting landmarks in facial datasets in a variety of poses.Thus the shape descriptor usually combine with other de-scriptors or combine an robust algorithm to make while system robust to the head pose.Xu [28]developed a system with the casecade …ltering and the e¤ective energy descriptor that we have introduced in section 3.1.The experiment result shows that the system is robust to the pose.But Xu did not illustrate vary detail,so we can not know how robust the system is.And in section 3.1,we also introduced the method Cond proposed in [29].The experiment shows the robust property to the head pose.However the rotation angle is 5± to 25± .

Apart from the shape descriptor,there are other methods robust to head pose have been proposed.Colbry and Jain [20] proposed an arbitrary pose algorithm of face veri…cation in 2005.The …rst step in the algorithm describes the detection of a core set of candidate points:the inner eyes,the nose tip and the outer eyes.They regarded that the easiest feature point to identify in arbitrary pose is the inside edge of an eye next to the bridge of the nose.This point has a shape index value which is close to zero and the area around this point has a consistent shape index value across all face images and proses.Hence the candidate of inner eye corners can be identi…ed by using these properties. And the second

(43)

easiest feature point to detect is the nose tip.They listed a criteria for …nding candidate nose points:Point closest to the scanner,point farthest to the right,point farthest to the left,point farthest from the vertical plane formed by points the points closest to the scanner and farthest to the left,point farthest from the vertical plane formed by points closest to the scanner and farthest to the right and point with the largest shape index.They used the criteria to …nd the nose tip in an arbitrary pose.The next step is using some constraints to eliminate and reduce the number of possible feature points.And the …nal step is to …nd three labeled points that correspond to three points labeled on the 3D facial model.These three points are used to calculate a transformation from the test scan to the model.Colbry used the ICP algorithm to calculates the best matching distance between the scan and the model for deciding the three feature point they want.The experiment result is not as good as the result in the frontal pose point detection.

After a short while,Lu and June [19] proposed a more robust method to the arbitrary pose.They regarded the nose tip is a distinctive point of the human face and it is insensitive to the facial expression change.If the pose of a face scan is represented by the angle of rotation,hence they suppose the nose tip still has the largest depth valus (z value) if projected onto the corrected pose direction.We can call it the directional maximum.We can see the examples of the direction maximum of the nose tip in Fig.21.We can simply state the method in the following:Lu and Jain quantized the 180 degree from -90 degrees to 90 degrees into N angles with equal angular interval (¢) just like Fig.22.Then they rotated the test scan and …nd the point with the maximum projection value along the corresponding pose direction as the nose tip candidate at all N directions.They believed that the true nos tip must be involved in the N candidate points.And next,they want to …nd a the vertical pro…le in N camdidate points which has the most similar outline to the vertical pro…le on real nose tip in the training set as shown in Fig.23.They regarded the

(44)

most silmilar nose candidate point as the nose tip detection result.If the nose tip has been found,the pose angle has been known simutaneously.Hence they rotated the test scan by the pose angle and use the same method as the method proposed in [19] to extract the eye corners and the mouth tip.Although the method Lu and Jain proposed addresses the problem of pose variations and is tested against 300 multiview scans (00

,§450) from 100

subjects,we …nd out that Lu and Jain limit the pose just rotate around the yaw.It means that the method only robust to changes in yaw for sure.Lu indicated that the changes in pitch would result in an expensive using brute force search.Hence if we can improve the search scheme,the method will be more robust to the arbitrary direction.

Figure21:Direction maximum of the nose tip [19].

(45)

Figure23:Top:extracted nose pro…les ; midle:normalized and resampled nose pro-…le;bottom : extracted pro…les overlaidis based on candidate.[19]

In the same way to analysis the facial pro…le,Faltemuer [21] proposed a method called Rotated Pro…le Signature to locate the nose tip automatically in the presence of pose or expression variation.He quantize the angle from 00 to 1800with equal angle interval 50.He

rotated the test scan 50 about the vertical axis and extracted the points with largest

x-coordinate to form the pro…le.The process is repeated until each pro…le is created in every rotate angle.Then a pro…le model are manually extracted from a single subject image to a known pro…le position.Faltemuer used the model to match every x-projection pro…le for searching the position with minimum matching score.The position with a minimum matching score is reported as the nose tip.All x-projection pro…les have their own position with a minimun matching score.Faltemuer de…ned the position with the global minimum matching score is exactly the nose tip with the pose angle.This system was tested by 7317 facial scan included 406 frontal and 6911 in various yaw and pitch angles.The experiment result reach almost 100% for only rotating about yaw.It also perform well when the angle of head up or drop needs to be lower than 450

.When the rotation angle equals to § 60±,the result is not very accurate.It is caused by too large rotation angle so the e¤ective

(46)

information is missing a lot.Therefore We can know that the method wull lead to a bad result when the rotation angle is too large.And the rotated pro…le signature only used for the nose tip extraction although its e¤ect of robust tp head pose is vary well.

According to the above analysis,we can see the existing not many methods are robust to the head pose.Although there have been some methods show great robust result,they still can not deal with vary large pose variation such as the rotation angle bigger than 60± in [21].

3.3

A Facial Scan with Facial Expression Variation

Except the pose variation,expression variation also a¤ects the result of extration of feature points.The expression,such as smile,angry,frown,open the mouth etc,will change the local shape of the face.That makes some features change their shape and even change their geometrical properties.We can …nd out that the nose tip,inner eye corners and outer eye corners barely change due to the expression and mouth is the region with the biggest change due to the exprssion.Therefore in the existing works,most of the methods claim that their system is robust to the expression variation,however,they just extract those feature points which is invariant to the expression variation.Cond [29] extracted the nose tip and the eye corners.Cheng [12] only extracted the nos tip.Colbry and Jain [20] claim that their system is robust to the expression variation.But their test scan only include two kinds of expression:natural and smile.In [18, 19],Lu and Jain also claim their method is robust to the expression but the test scan only contain natural and smile.In [30],Segundo use 4950 images of 557 subjects with variations of facial expression,resolution,pose,and other characteristics,such as di¤erent hair styles.He extracted not only nose tip and eye corners but the nose bases,and the experiment result reach a high accuracy.However the

(47)

nose base is like nose tip and eye corners,it barely move due to expression variation. For mouth feature points extraction,Gorden [39] use the priciple curvature to …nd the ridge and the valley line on human face as shown in Fig.2 and use the ridge line to …nd the feature points. Kim [40] used the principle curvature information to …nd the feature line on human face,and utilised the feature line to …nd the feature points such as mouth corners.Although,we do not …nd a paper use feature line to extract the contour of the mouth and extract the mouth corner.We think extracting the contour in real time based on the maximum curvature is robust to the expression variation and keep the shape characteristic to extract the mouth corners or mouth tips.

(48)

Chapter 4

Conclusion and Future work

In this stuty,we …rst give a widely introduction of local shape feature descriptors.The local feature descriptors make an important role in whole feature point extraction frame-work.According to di¤erent feature descriptors,there will exist a corresponding methods or algorithms to use the characteristics provided by feature descriptors to extract the saliant feature points.Although there exist many methods that have good result in ex-tractging feature points,most of the methods are limited just for the frontal facial scan.For the problem caused by the changing pose,Faltemier proposed a robust method but just for nose tip extraction and still can not address the pose change over sixty degree.For the problem caused by the expression variation,most of the existing works only extract the nose tip and eye corners which are barely impacted by the expression.The mouth corners are a¤ected by the expression a lot.The existing feature descriptors can not describe it completely in any expression.It makes the mouth corners extraction be a chanllenge in nonstatistical ways.In the future,we will combine sevaral di¤erent descriptors to extract the nose tip and eye corners and …nd out the most robust combination of the descriptors and the corresponding algorithm for solving the problems caused by the changing pose and the expression variation.We will also try to use the principle curvature to extract the

(49)

contour of mouth and then utilise the contour to extract the feature points,such as mouth corners and mouth tip,in any expression.

(50)

Bibliography

[1] Antonini (G.), Popovici (V.), Thiran (J.P.), Independent Component Analysis and Support Vector Machine for Face Feature Extraction, 4th Int. Conf. on Audio- and Video-Based Biometric Person Authentication, pp. 111-118, Guildford, UK, 2003. [2] Ryu (Y.S.), Oh (S.Y.), Automatic Extraction of Eye and Mouth Fields from a Face

Image Using Eigenfeatures and Ensemble Networks, Applied Intelligence, 17, pp.171– 185, 2002.

[3] Feris (R.S.), Gemmell (J.), Toyama (K.), Krer (V.), Hierarchical Wavelet Networks for Facial Feature Localization, in 5th IEEE Int. Conf. on Automatic Face and Ges-ture Recognition, pp. 125-130, Washington DC, USA, 2002.

[4] Shih (F.Y.), Chuan (C.), Automatic Extraction of Head and Face Boundaries and Facial Features, Information Sciences, 158, pp. 117-130, 2004.

[5] Zobel (M.), Gebhard (A.), Paulus (D.), Denzler (J.), Niemann (H.), Robust Facial Feature Localization by Coupled Features, in 4th IEEE Int. Conf. on Automatic Face and Gesture Recognition, Grenoble, France, 2000.

[6] Sobottka (K.), Pitas (I.), A Fully Automatic Approach to Facial Feature Detec-tion and Tracking, in Bigun (J.), Chollet (G.), Borgefors (G.) (eds.), Audio- and Video-based Biometric Person Authentication, LNCS, 1206, pp. 77-84, Springer

(51)

Ver-lag, 1997.Sobottka (K.), PITAS (I.), A Fully Automatic Approach to Facial Feature Detection and Tracking, in BIGUN (J.),

[7] Gorden, G. 1991. Face recognition based on depth maps and surface curvature. In SPIE Proceedings, Vol. 1570: Geometric Methods in Computer Vision. SPIE Press, Bellingham, WA 234–247.

[8] Gorden, G. 1992.Face recognition based on depth and curvature features Computer Vision and Pattern Recognition (CVPR) (June) , pp. 108–110

[9] A. Colombo, C. Cusano, and R. Schettini, “3D face detection using curvature analy-sis.” Pattern Recognition, vol. 39, pp. 444–455, 2006.

[10] K.I. Chang, K.W. Bowyer, P.J. Flynn, Adaptive rigid multi-region selection for han-dling expression variation in 3D face recognition, in: IEEE Workshop on Face Recog-nition Grand Challenge Experiments, June 2005.

[11] K.W. Bowyer, K. Chang, P.J. Flynn, A survey of 3D and multi-modal 3D + 2D face recognition, in: 17-th International Conference on Pattern Recognition, August 2004, pp. 358–361.

[12] K. I. Chang, K. W. Bowyer, and P. J. Flynn. Multiple nose region matching for 3d face recognition under varying facial expression.IEEE Transactions on Pattern Analysis and Machine Intelligence,28(10):1695–1700, 2006.

[13] H. Cantzler, R. Fisher, Comparison of HK and SC curvature description methods, in: Proceedings of the Third International Conference on 3-D Digital Imaging and Modeling, 2001, pp. 285–291.

(52)

[14] Alexander Ceron,Augusto Salazar and Flavio Prieto.Relevance analysis of 3D curvature-based shape descriptors on interest points of the face.IEEE Image Process-ing Theory,Tools ans Applications,2010.

[15] C. Dorai and A. Jain, “COSMOS—a Representation Scheme for 3D Free-Form Ob-jects,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 10, pp. 1,115-1,130, Oct. 1997

[16] C. Dorai and A.K. Jain, “COSMOS—A Representation Scheme for Free-Form Sur-faces,” Proc. Fifth Int’l Conf. Computer Vision, pp.1,024-1,029, Boston, June 1995. [17] X. Lu, D. Colbry, and A. K. Jain. Three-dimensional model based face recognition.

Int. Conf. on Pattern Recognition, 1:362–366, 2004.

[18] X. Lu and A. K. Jain, “Multimodal facial feature extraction for automatic 3D face recognition,” Dept. Comput. Sci., Michigan State Univ., East Lansing, MI, 2005. [19] X. Lu and A. K. Jain. Automatic feature extraction for multiview 3d face recognition.

Automatic Face and Gesture Recognition, pages 585–590, 2006.

[20] D. Colbry, G. Stockman, and A. K. Jain, “Detection of anchor points for 3D face veri…cation,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog. Work-shops, 2005, p. 118.

[21] T. Faltemier, K. W. Bowyer, and P. J. Flynn, “Rotated pro…le signatures for robust 3D feature detection,” in Proc. 8th IEEE Int. Conf. Autom. Face Gesture Recog., 2008, pp. 1–7.

[22] J. Y. Cartoux, J. T. Lapreste, and M. Richetin, “Face authentication or recognition by pro…le extraction from range images,” in Proc. IEEE Comput. Soc. Workshop Interpretation 3D Scenes, 1989, pp. 194–199.

(53)

[23] A. Mian, M. Bennamoun, and R. Owens. An e¢cient multimodal 2d-3d hybrid ap-proach to automatic face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(11):1927–1943,2007.

[24] G. Pan, Y. Wu, and Z. Wu, “Investigating pro…le extracted from range data for 3D face recognition,” in Proc. IEEE Int. Conf. Systems, Man and Cybernetics, 2003, vol. 2, pp. 1396–1399, vol. 2.

[25] A. Moreno, A. Sanchez, J. Velez, and F. Diaz. Face recognition using 3d surface-extracted descriptors. Irish Machine Vision and Image,2003.

[26] M. Romero and N. Pears, “Landmark Localisation in 3D Face Data”, Proc. IEEE Int. Conf. Advance Video and Signal Based Surveillance 2009, in press.

[27] M. Romero-Huertas and N. Pears, “3D facial landmark localisation by matching simple descriptors,” in Proc. 2nd IEEE Int. Conf. Biometrics,Theory, Appl. Syst., 2008, pp. 1–6.

[28] C. Xu, T. Tan, Y. Wang and L. Quan, “Combining Local Features for Robust Nose Location in 3D Facial Data”, Pattern Recognition Letters 27, 2006, 1487-1494. [29] C. Conde, L. J. Rodriguez, E. Cabello, “Automatic 3D Face Feature Points

Ex-traction with Spin Images”, Proc. International Conference on Image Analysis and Recognition, LNCS 4142, 317 – 328, 2006.

[30] M. Segundo and L. Silva,Automatic Face Segmentation and Facial Landmark De-tection in range Images,IEEE Transactions on system,man,And cybernetics-Part B:Cybernetics,Vol.40,NO.5,October 2010.

(54)

[32] A. Johnson, “Spin-images: a representation for 3-D surface matching”, Ph.D. Dis-sertation, The Robotics Institute, Carnegie Mellon University, August 1997.

[33] X. Zhang,N.Fan,X. Chen,L.H. Ran,J.W. Niu:Feature Extraction of three dimensional facial landmark,

[34] William Thimble,Using surface curvature and topological features from shape-from-shading to improve shape-from-stereo,March 13,2003

[35] P. Nair and A. Cavallaro, “Region segmentation and feature point extraction on 3D faces using a point distribution model,” in Proc. IEEE Int. Conf. Image Processing, San Antonio, TX, Sep. 2007.

[36] Romero, M. and Pears, N. (2009). Point–pair descriptors for 3D facial landmark localisation. In IEEE Int. Conf. on Biometrics: Theory, Applications and Systems (BTAS 09).

[37] Galton, F. 1888. Personal identi…cation and description. Nature, (June 21), 173–188. [38] L. D. Harmon and W. F. Hunt, “Automatic recognition of human face pro…les,”

Comput. Graphics Image Process, vol. 6, 1997

[39] G. G. Gordon and L. Vincent, Application of morphology to feature extraction for face recognition, in Proc. SPIE 1658, 1992, 151–164.

[40] T. K. Kim, S. C. Kee, and S. R. Kim, “Real-time normalization and feature extraction of 3-D face data using curvature characteristics,” in Proc. 10th IEEE Int. Workshop Robot Human Interactive Communication, 2001, pp. 74–79.

數據

Figure 1: Position of the 11 landmarks which are the most often used in the literature.
Figure 3:Facial segmentation base on sign HK classi…cation [9].
Figure 4:The HK classi…cation map and its threshold version on human face [9].
Figure 5:Nine well known shape categories and their correspomding shape index [15].
+6

參考文獻

相關文件

include domain knowledge by specific kernel design (e.g. train a generative model for feature extraction, and use the extracted feature in SVM to get discriminative power).

• More important utterances have higher weights corresponding to the prosodic feature P.  Unsupervised learn important

In this paper, we study the local models via various techniques and complete the proof of the quantum invariance of Gromov–Witten theory in genus zero under ordinary flops of

Due to the important role played by σ -term in the analysis of second-order cone, before developing the sufficient conditions for the existence of local saddle points, we shall

In order to reveal the real purpose of this important work, the present paper discusses both Dau-shr's historical background, the fierce competition between Buddhist and the

Advantages of linear: easier feature engineering We expect that linear classification can be widely used in situations ranging from small-model to big-data classification. Chih-Jen

Tekalp, “Frontal-View Face Detection and Facial Feature Extraction Using Color, Shape and Symmetry Based Cost Functions,” Pattern Recognition Letters, vol.. Fujibayashi,

Random Forest Algorithm Out-Of-Bag Estimate Feature Selection.. Random Forest