非對稱低空間頻率訊息有助於臉孔區辨

(1)

國立臺灣大學理學院心理學研究所碩士論文

Graduate Institute of Psychology College of Science

National Taiwan University Master Thesis

非對稱低空間頻率訊息有助於臉孔區辨

Asymmetric Low Spatial Frequency Improves Face Discrimination

陳巾眉 Chin-Mei Chen

指導教授：陳建中博士

Advisor: Chien-Chung Chen, Ph.D.

中華民國 98 年 6 月

June, 2009

(2)

i

誌謝

誌謝誌謝

誌謝

(3)

ii

摘要摘要摘要摘要

本實驗的目的在於理解人類視覺系統如何從陰影的訊息中建構臉部的立體訊息。我們利用電腦軟體改變投射在 3D 立體臉部模型的光影變化，然後記錄觀察者對於臉部辨識作業如何隨著照明方向改變而有所不同。我們使用對稱性算則分離臉部影像的照明與臉部的表面特質。我們一共區分了對稱與不對稱的低空間頻率和高空間頻率的四種成分。實驗刺激是這些不同成份的組成。結果顯示非對稱低空間頻率成分，亦即陰影訊息會顯著改變臉部辨識。然而，對稱的低空間頻率成分卻對臉部辨識沒有影響。此外；非對稱低空間頻率也會改變臉部影像的深度知覺。總結以上結果可以確定的是非對稱低空間頻率訊息決定了臉部的特性以及深度知覺。而且這個效果隨著照明光源角度的改變而有所不同。所以，我們的結果支持視覺系統中以立體維度來處理臉部辨識的訊息。

關鍵詞：陰影造型法、照度、對稱性、臉孔知覺、三度空間物體。

(4)

iii

Asymmetric Low Spatial Frequency Information Improves Face Discrimination

Chin-Mei Chen Abstract

To study how the visual system compute 3D shape of faces from shading information, we manipulated the illumination conditions on 3D scanned face models and observed how the face discrimination changes with lighting directions . We used a symmetry algorithm to dissociate surface albedo and illumination component of face images by separating symmetric and asymmetric components of both low and high spatial frequency information of face images. Stimuli were hybrid faces with different combination of symmetric and spatial content. Results show that asymmetric low spatial frequency (shading) information had significant influence on face discrimination, while the symmetric ones have little, if any, effect on face discrimination. The asymmetric low spatial frequency also changed the perceived depth of hybrid faces. Together, these results suggest that (1) the asymmetric low spatial frequency information dramatically affects both perceived face identity and perceived depth; and (2) this effect increased as the lighting direction shifts to the side. Thus, our results provide an evidence for

3D-based face processing.

Key words: Shape from shading, illumination, symmetry, face perception, 3D object.

(5)

iv

LIST OF FIGURES

Figure 1 The symmetric algorithm helps to dissociate the contribution of the surface albedo from illumination component in image. ... 28 Figure 2 Examples of stimuli... 29 Figure 3 The effect of asymLSF on face discrimination. ... 31 Figure 4 Comparisons of the effect of asymLSF and symLSF on face discrimination. .. 32 Figure 5 Results of depth judgment. ... 33 Figure 6 Face discrimination performance is highly correlated with perceived depth. .. 34 Figure 7 All seven observers’ asymLSF condition data. ... 36 Figure 8 All seven observers’ symLSF condition data (compared with asymLSF data). 36

(7)

1

Chapter 1 Introduction

We perceive a 3D object when light reflected from its surface arrive our retinal.

The retinal image, however, is two-dimensional. Hence, the question is how the visual

system reconstructs a 3D representation of an object from 2D retinal images. It is well

known that a human observer can utilize depth cues available in an image to facilitate

this reconstruction. One of the useful cues is shading, or gradual variation of the

luminance on a surface (Horn, 1970). The luminance of a point on a surface depends on

not only the intensity and the incident angle of the light reaching the surface but also the

slant of the surface at that point, which determines the proportion of the light reflected

from the surface reaches the eyes of an observer. Hence, under the same illumination,

the shading provides information of the slant and in turn determined the relative depth

of each point on the surface (Horn, 1986; Horn & Brooks, 1989). Ramachandran (1988a,

1988b) further showed that the human visual system uses two assumptions in extraction

of shape from shading information: One is that there is a single light source illuminating

the whole scene, and the other is that the light is shining from above. With these two

constraints, the visual system is able to solve shape from shading problem as the

top-bottom luminance gradient from bright to dark suggests a convex surface while the

(8)

2

gradient from dark to bright, a concave surface. Such effect is best demonstrated in the

crater illusion in which inverting an image can alter the perception of surface from

convex to concave and vice versa (Gibson, 1950).

The Ramachandranian shape from shading has its limit. As shown in the hollow

face illusion (Gregory, 1970), a hollow face is always perceived as a convex regardless

whether the lighting is from above or from bottom. One interpretation for this hollow

face illusion is that the faces are familiar objects with known shape, and thus does not

require shading information to resolve its 3D shape. Hence, the face perception should

be insensitive to shading. This interpretation, however, contradicts with a number of

studies showing that face recognition is impaired when the luminance distribution of a

face image is inverted as in photographic negative (Gilad, Meng, & Sinha, 2009;

Johnston, Hill, & Carman, 1992; Kemp, Pike, White, & Musselman, 1996; Liu &

Chaudhuri, 1997). In addition it is showed that it is more difficult to recognize

bottom-lit faces than top-lit faces (Hill & Bruce, 1996; Johnston et al., 1992; Liu, Collin,

Burton, & Chaudhuri, 1999; Liu, Collin, Rainville, & Chaudhuri, 2000). That is, a

change of illumination condition has little effect on shape from shading on a face but

does have an effect on face recognition. However, to best of our knowledge, there is no

reliable measurement on whether observers perceive the same depth on a hollow face or

(9)

3

a bottom-lit face as they do on a normal top-lit face. Perhaps only part of information on

those unnaturally lit face can be recovered by the visual system, and the information is

enough to perceive a hollow face as convex but not enough for an observer to recognize

a bottom-lit face as good as a top-lit one.

We thus investigated the effect of shading on face perception by observing how

face discrimination ability changes with lighting directions. To avoid nuisance

“naturalness” factor that may result from Ramachandranian lighting-from-top constraint,

we shifted the position of light source laterally from the front to the side of a face.

Hence, all light source positions are ecologically equally plausible. The illumination and

face information can be separated by an algorithm based on the assumptions that the

face is symmetric (Grammer & Thornhill, 1994; Rhodes, 1988; Rhodes, Geddes, Jeffery,

Dziurawiec, & Clark, 2002; Rhodes, Peters, Lee, Morrone, & Burr, 2005) and is

illuminated by a single light source. Facial characteristics may be partitioned into two

types. One is the information from the inherent coloration and reflectance properties of

the facial surface, known as the surface albedo. And the other is information from the

illumination of the 3D face shape, termed the illumination component. Notice that the

luminance component cannot have a luminance less than zero, we can recast them in

terms of their contrast variation around the mean (scaled as 1) over the domain

(10)

4

where is the contrast of the inherent coloration of the face, or albedo. And

is the illumination component. Since both and in eq. 1 are constrained to

be less than one, their product in the interaction term must be much less than 1 and can

therefore be considered negligible in most situations. Thus,

Each term may be specified in terms of its symmetric (S) and asymmetric (A)

components:

where and for the illumination

component. And for the albedo component that is assumed to be

in symmetric view, and therefore has no asymmetric component .

All that is needed for complete segregation of the surface albedo from the

illumination component is to distinguish the residual symmetric illumination component

LS from the entire albedo component CS. We hypothesize that a) CS contains

predominantly high-frequency energy whereas LS contains predominantly

(11)

5

low-frequency energy. And b) the remaining frequency bands in each case are of little

relevance to face reconstruction. Hence, we can separate the two symmetric components

by a spatial frequency filter. Figure 1 demonstrates our method. Panel A is the

asymmetric low spatial frequency component that represents the asymmetric

illumination information, , of the original face image ( , panel C) while

Panel B is the symmetric high spatial frequency that represents the albedo component

( ). The hybrid face in Panel E is the sum of the LA and CS and is almost

perceptually identical to the original face. That is, the combination of the asymmetric

high frequency and the symmetric low spatial frequency ( ) components (Panel

D) plays little role in face perception. Hence, subtracting them from the original face

image affects little of our face perception.

Therefore, in the current study, we investigated how the change of the asymmetric

low spatial frequency by illumination conditions affects face discrimination and depth

judgment on hybrid faces. The hybrid picture paradigm (Schyns & Oliva, 1997, 1999)

was designed to study the relative contribution of specific spatial frequencies. In our

case, the hybrid face would be a combination of CS and LA or CS and LS. If the

illumination condition can affect the retrieval of 3D shape information from a face, we

should expect the change of illumination condition can affect both face discrimination

(12)

6

and depth judgment on the hybrid faces. In addition, such illumination effect should

exert their influence through the asymmetric low spatial frequency component of the

face images and not the symmetric ones.

(13)

7

Chapter 2 Method

2.1 Apparatus

The stimuli were presented on a ViewSonic VA902 17″ LCD monitor controlled by

a HP D325MT computer with an ATI Radeon 9800PRO graphics card. The spatial

resolution was 1280(H) × 1024(V). The viewing distance was 127.5cm. At this

distance, a pixel subtended about 0.01° (H) × 0.01°(V). The temporal refresh rate of

the monitor was 60Hz (non-interlaced). The gamma function of the monitor was

calibrated with a LightMouse photometer (Tyler & McBride, 1997), and this

information was used to compute linear 8-bit color look–up table. The accuracy of the

look-up tables was verified by an international Light RPS-380 spectroradiometer. The

experimental control software was written in MATLAB (Mathworks Inc., 1993) with

the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997). The display had mean

luminance at 15cd/m² and chromaticity at (0.33, 0.33) in CIE 1931-xycoordinates.

2.2 Stimuli

The face images were constructed from 3D laser-scanned faces developed by Lin

and his collegues (Lin & Ouhyoung, 2005; Lin, Yeh, & Ouhyoung, 2002). The 3D face

(14)

8

models of one female face and one male face with a neutral expression were used. The

front view face images were rendered from the textured 3D models using 3ds Max 2009

(Autodesk, Inc., 2009) . The Blinn–Phong shading model was employed to construct

illumination conditions. We first place four diffused light sources around the model to

produce an even ambient light to keep the shadowed regions from entirely dark. The

point light source was placed at 0°, 15°, 30°, and 60° relative to the tip of the nose and

on the horizontal meridian of faces. That is, the lighting direction of 0^o means the light

source is right in front of the face. The relative intensities between the point light source

and the ambient light were 1.5 and 0.2 respectively. In addition, the 3D face models

under ambient illumination were used for constructing symmetric high spatial frequency

only stimuli. All images were grayscale images with 256 gray levels.

As shown in eq. 3a and 3b, face images could be decomposed into three parts:

asymmetric and symmetric illumination components, and symmetric albedo. The

decomposition was achieved by the following steps. First, the face images were Fourier

transformed to the frequency domain. The real and imaginary parts of the transformed

images were taken as symmetric and asymmetric components respectively. Both

symmetric and asymmetric components were multiplied by a Gaussian low pass

and a high pass function, where the cut-off frequency was

(15)

9

12 cycles per face width, to extract the low and high spatial frequency information

respectively. With an inverse Fourier transform, we separated the four components of a

face images: Symmetric low spatial frequency (symLSF, corresponding to LS in eq.3a),

symmetric high spatial frequency (symHSF, corresponding to CS in eq. 3b), asymmetric

low spatial frequency (asymLSF, corresponding to LA in eq. 3a), and asymmetric high

spatial frequency (asymHSF). The asymHSF information should have little influence on

3D face perception. Therefore, this part was not used in our experiments. The symHSF

images, looks very flat, convey detailed edge information of facial features and in turn

the precise shape of the eyes, nose, mouth eyelashes, etc. In contrast, the LSF face

images represent the coarse-scale facial configure and illumination information. All

components of all images were normalized to have equal RMS contrast before further

processing.

The hybrid face stimuli we used were a combination of a symHSF only image,

constructed from face images created under ambient illumination condition, and a

symLSF or an asymLSF component of face images created under point lighting

conditions. We used FantaMorph 4.0 (Abrosoft, 2009) to generate morphed images

between the male and female symHSF faces. There were seven morphed symHSF

images with different proportion of femininity between 0 (male) to 1 (female) (0, 0.2,

(16)

10

0.4, 0.5, 0.6, 0.8, and 1.0). Figure 2A shows examples of morphed symHSF images.

The asymLSF components were created from faces rendered in four illumination

conditions (0°, 15°, 30°, 60°). The rendered faces and their asymLSF components

under four illumination conditions are illustrated in Figure 2B. To ensure that symLSF

component, which is invariant to asymmetric illumination, has little effect on face

discrimination, we also used the symLSF component of the face under the 60°

illumination as a control.

Finally, the hybrid faces (Figure 2C) contained the symHSF component at a certain

morphing level and asymLSF at one illumination condition or the symLSF component.

The LSF components were from either male or female faces. Totally, there were 56 (4

illumination conditions × 2 genders × 7 morphing level) asymLSF and symHSF

hybrid face images used in the experiment. In addition, for controls, we also used 14

symLSF and symHSF hybrid images (2 genders × 7 morphing level) and 7 symHSF

alone images at different morphing levels. Hence, there were 77 face images used in this

study. The constructed hybrid images were then windowed with a Gaussian filter to

remove irrelevant external cues such as hair, ears, and contour of faces.

(17)

11

2.3 Procedure

2.3.1 Face discrimination

We used the method of constant stimuli to measure the proportion of the female

judgment for each test images. In each trial, the observers were presented with one of

the 77 face images for duration of 500 ms. After the presentation; the observers were

shown a blank screen of mean luminance. The observers were asked to make response

at this time. The next trial began one second after the observer made a response. The

task of the observers was to press keys to determine where the presented image was a

male or a female face. The order of presentation of the images was randomized.

Observers were not given feedback about their performance.

A psychometric function consisted of the proportion of the female judgment on

images of the same illumination condition from the face of the same gender at different

morphing levels. There were 40 trials per face image. Hence, with seven morphing

levels, there were 280 trials for a psychometric function. Given 8 asymLSF, 2 symLSF

and one symHSF only conditions, there were totally 11 psychometric functions.

(18)

12

2.3.2 Depth judgment

In this experiment, the observers were to judge the relative depth of the face

images. In each trial, observers were presented with one of the face images together

with a reference sphere for 1s. The face and the sphere were presented side by side on

the screen with 3' gap between them. The location (left of right of the fixation) of the

face or the sphere was randomly determined. The observer was to press one of the keys

to rate the relative depth of the faces using a 5-ponit Likert scale, (1–flat, 5–steep).

The observers were asked to use the reference sphere as an anchor of depth 3.

The face images used in depth judgment were images of 0 or 1.0 morphing levels

of those used in face discrimination experiment. The reference sphere were created with

3ds Max 2009 (Autodesk, Inc., 2009). The diameter of the sphere was the same as the

width of the face images and had the 0^o lighting direction.

2.4 Data analysis

For each LSF condition, the fraction of female responses was then plotted against

the morphing level of the symHSF component. The psychometric function was fitted

with an accumulated Gaussian function

(19)

13

where determines the mean, and in turn the point of subjective equality (PSE), and

determines the slope of psychometric function, and ( ) and ( ) defines the lower and

upper bound of the psychometric function respectively. The differences between PSEs

for different conditions were tested with a two-tailed paired t test.

2.5 Participants

Seven observers with normal or corrected-to-normal vision participated in this

study. All of them were naïve to the purpose of this study and were not familiar with the

people whose faces were used as stimuli. Informed consent was obtained from each

observer before the experiment. The observers were fiscally compensated for their time.

(20)

14

Chapter 3 Results

3.1 asymLSF information changed perceived facial information of hybrid faces.

Panel (A)-(F) of Figure 3 show that the fraction of female judgment as a function

of symHSF morphing level for hybrid faces with asymLSF from different lighting

directions in three representative observers. The left column shows the psychometric

functions measured with female asymLSF components, while the right, male asymLSF

components. The psychometric function for symHSF only condition (black squares

and curves) are plotted in the panels for the corresponding observers as a comparison.

The red, green, blue, magenta psychometric curves and symbols denote the

psychometric function for hybrid faces with asymLSF from 0ô, 15ô, 30ô, and 60ô

lighting directions respectively.

Adding the asymLSF component from a female face to the symHSF shifted the

psychometric curves to the left. The amount of change in general increased with the

lighting directions. The observers saw a female face even at very low level of femininity.

On the contrary, adding the asymLSF component from a male face shifted the

psychometric functions dramatically to the right. This rightward shift was especially

(21)

15

pronounced with the 30° and 60° illumination conditions in which the observers

always saw a male face regardless the female morphing level. Indeed, the male

asymmetric low spatial frequency information had a stronger effect than the female one.

Two studies (Bruce, Healey, Burton, Doyle, Coombes, & Linney 1991; Bruce, et al.,

1993) also showed the similar male advantage effect in face recognition and

discrimination tasks.

To quantify the effect of asymLSF on face discrimination and summarize the

results from all seven observers, we determined PSE (the proportion of femininity of

hybrid faces corresponding to 50% female responses) for each psychometric function of

each observer. Figure 3G shows the mean PSE relative to the symHSF only condition. A

positive value means a rightward shift of the psychometric curve, or more male

responses, relative to symHSF only condition. While a negative value, a leftward shift

or more female responses. The p-value shown in the Figure 3G was a two-tailed paired t

test of comparing PSE of each illumination conditions with the symHSF only condition.

The error bars show 1 standard error (SE) across observers. All asymLSF effects were

significant except female asymLSF from 0^o lighting direction. The male asymLSF from

the 30^o and 60^o conditions were so effective that their PSEs were beyond the measurable

range.

(22)

16

3.2 symLSF information did not change perceived facial information of hybrid images.

We also investigated whether the symmetric component of LSF influences face

discrimination as asymLSF. Panel (A)-(C) of Figure 4 show the results of three

representative observers in judging the gender of hybrid faces consisted of symLSF and

symHSF (cyan symbols and curves). The psychometric functions of asymLSF

conditions of the same 60^o illumination condition (magenta symbols and curves) and the

symHSF only condition (black symbols and curves) were replotted here as a

comparison. The star symbols denote the male and circles female LSF components.

The symLSF, be contrast with, produced little, if any, effect on the face discrimination

than the asymLSF component. Figure 4D shows a summary of the PSEs from the seven

observers. Relative to the symHSF only condition, the addition of symLSF information

produced no significant effect for female condition

and for male condition). On the other hand, the difference between

asymLSF and symLSF condition was significant for

female condition, and the difference between male conditions was obviously beyond

measurable range).

(23)

17

3.3 Depth judgment

Figure 5 shows the depth rating for asymLSF + symHSF (open circles) faces from

four illumination conditions. The depth rating for symLSF + symHSF (closed triangles)

and symHSF only (closed squares) faces is plotted for a comparison. The symHSF

components were from either 1.0 (female, Panel A) or 0.0 (male, Panel B). The

perceived depth for the asymLSF + symHSF faces increased with the direction of

lighting. This increment can be described by a linear regression line with slope 0.025 for

female faces and 0.031 for male faces. The R² for the linear regression was .983 and .91

for female and male asymLSF + symHSF hybrid faces respectively. The perceived

depth of the symHSF only face was about the same as the 0^o asymLSF + symHSF

for female condition and for male

condition) This result is consistent with the PSE shift with different illumination

conditions. That is, as the illumination direction shift to the side, the asymLSF has a

greater effect not only on face perception, but also on perceived depth.

(24)

18

Chapter 4 Discussion

Our results showed that asymLSF information did have a profound effect on face

perception as it greatly changed the perceived gender of the hybrid faces in our

observers. That is, the asymLSF component from a female face make the observers

more likely to perceive the hybrid face as a female regardless the source of other

components. While the asymLSF component from a male face had an opposite effect.

But the symLSF component had no such effect. In addition, the effect of the asymLSF

component increased with the lighting directions. The asymLSF component also

affected the perceived depth of hybrid faces. Again, this effect was more pronounced as

the lighting direction shift sideward.

4.1 Illumination effects in face recognition

The local interaction between surface orientation and source of illumination brings

about shading gradients on objects. Consequently, shading pattern would be affected by

changes in illumination conditions. Since shading provides informative implication

regarding 3D shape (Horn, 1970), a different lighting direction could result in changes

in face representation via shading information, as our results suggest. There were

(25)

19

studies showing that lighting direction is important for face recognition. In addition to

aforementioned studies investigating the differential effect of top-lit and bottom-lit faces

(Hill & Bruce, 1996; Johnston et al., 1992; Liu et al., 1999; Liu et al., 2000), Braje,

Kersten, Tarr, and Troje (1998) showed left or right illumination variation impaired face

recognition memory. These studies suggested that the lighting directions and

illumination information play an important role in face perception. On the other hand,

the hollow face illusion (Gregory, 1970) shows that 3D perception on a face is not

sensitive to shading information.

As discussed above, such controversial may imply that the visual system may

discount the change of illumination and the resulting depth percept to an extent that the

face can still be perceived as convex but not strong enough to recover all the depth

information. Our result is consistent with the explanation. We showed that the

manipulation of lighting direction changes the shading information on faces, and in turn

changes both the depth perception of faces and performance of face discrimination.

Figure 6 shows that the face discrimination performance is highly correlated with

perceived depth (for female condition, Pearson correlation coefficient, , and

for male condition ). Thus, if the illumination condition did not provide proper

shading information to recover enough depth information, it is difficult for an observer

(26)

20

to perceive the identity of the faces.

4.2 Spatial frequency effect

There is considerable evidence indicating the importance of coarse-scale facial

information in face recognition. The pioneering work ‘Lincoln illusion’ demonstrated by

Harmon and Julesz (1973) showed that applying a low-pass filter to remove the high

frequency noise introduced by the sampling and quantizing improved recognition of a

pixelized portrait. Moreover, they suggested that the low spatial frequency is more

critical for face recognition than high spatial frequency. Subsequent studies found the

critical spatial frequency range for face recognition (For a review, see Ruiz-Soler &

Beltran, 2006). Harmon and Julesz (1973) suggested that a spatial frequency as low as

2.5 cycles per face width was sufficient for face identification. Several studies showed

that face recognition is optimal at medium spatial frequencies (between 8 to 16 cycles

per face width) (Costen, Parker, & Craw, 1996; Gold, Bennett, & Sekuler, 1999;

Näsänen, 1999; Parker & Costen, 1999). The cut-off frequency of our low-pass filter

fell in this range. These results indicated that removing of low to medium spatial

frequencies had the most detrimental effect on face recognition than high spatial

frequencies.

(27)

21

Unlike aforementioned reports, we did not simply investigate the influence of

specific range of spatial frequencies on face perception. The novelty of this study,

specifically, is to identify that the asymmetric low spatial frequency is more critical in

face perception than the symmetric ones. Since the natural shading is most often to be

asymmetric, due to shape from shading, the more critical facial information should

reside in asymmetric than symmetric components. This notion is confirmed with our

result that (1) the asymmetric low spatial frequency information dramatically affects

both perceived face identity and perceived depth; and (2) this effect increased as the

lighting direction shift to the side.

(28)

22

Chapter 5 Conclusions

In this study, with hybrid image paradigm and face images constructed from

different illumination conditions, we demonstrated the role of asymmetric low spatial

frequency information in face discrimination. The asymmetric low spatial frequency

information has a strong influence on face perception as illustrated by its ability to

change the perceived gender of the hybrid faces. In addition, such effect increased as the

lighting direction shifts sideward. In contrast, the symmetric low spatial frequency

information had little, if any, effect on face discrimination. In addition, the asymmetric

low spatial frequency information also affected the perceived depth of a face. Again,

this effect was more pronounced as the lighting direction shifts sideward. Hence, these

results are consistent with the notion that the sideward shift of lighting directions

provides more shading information that increases the perceived depth of

shape-from-shading and in turn improves the facial discriminability in the observers.

(29)

23

References

Abrosoft. (2009). FantaMorph 4.0. Beijing: Abrosoft, Inc.

Autodesk, Inc. (2009). 3ds Max 2009. San Rafael, CA: Autodesk, Inc.

Brainard, D. H. (1997). The psychophysics toolbox. Spatial vision, 10(4), 433-436.

Braje, W. L., Kersten, D., Tarr, M. J., & Troje, N. F. (1998). Illumination effects in face

recognition. Psychobiology, 26(4), 371-380.

Bruce, V., Healey, P., Burton, A. M., Doley, T., Coombes, A., & Linney, A. (1991).

Recognising facial surfaces. Perception, 20(6), 755-769.

Bruce, V., Burton, A. M., Hanna, E., Healey, P., Mason, O., Coombes, A., Fright, R., &

Linney, A. (1993). Sex discrimination: How do we tell the difference between

male and female faces? Perception, 22(2), 131 – 152.

Costen, N. P., Parker, D. M., & Craw, I. (1996). Effects of high-pass and low-pass

spatial filtering on face identification. Perception & Psychophysics, 58(4),

602-612.

Gibson, J. J. (1950). The perception of the visual world. Boston: Houghton Mifflin.

Gilad, S., Meng, M., & Sinha, P. (2009). Role of ordinal contrast relationships in face

encoding. Proceedings of the National Academy of Sciences, 106(13),

5353-5358.

(30)

24

Gold, J., Bennett, P. J., & Sekuler, A. B. (1999). Identification of band-pass filtered

letters and faces by human and ideal observers. Vision Research, 39(21),

3537-3560.

Grammer, K., & Thornhill, R. (1994). Human (Homo sapiens) facial attractiveness and

sexual selection: The role of symmetry and averageness. Journal of

Comparative Psychology, 108(3), 233-242.

Gregory, R. L. (1970). The Intelligent Eye. London: Weidenfeld & Nicolson.

Harmon, L. D., & Julesz, B. (1973). Masking in visual recognition: Effects of

two-dimensional filtered noise. Science, 180(4091), 1194-1197.

Hill, H., & Bruce, V. (1996). Effects of lighting on the perception of facial surfaces.

Journal of Experimental Psychology: Human Perception and Performance,

22(4), 986-1004.

Horn, B. K. P. (1970). Shape from shading: A method for obtaining the shape of a

smooth opaque object from one view. PhD thesis, MIT, 1970.

Horn, B. K. P. (1986). Robot vision. Cambridge, MA: The MIT press.

Horn, B. K. P., & Brooks, M. J. (1989). Shape from shading. Cambridge, MA: The MIT

press.

Johnston, A., Hill, H., & Carman, N. (1992). Recognising faces: Effects of lighting

(31)

25

direction, inversion, and brightness reversal. Perception, 21(3), 365–375.

Kemp, R., Pike, G., White, P., & Musselman, A. (1996). Perception and recognition of

normal and negative faces: The role of shape from shading and pigmentation

cues. Perception, 25(1), 37 – 52.

Lin, I. C., Yeh, J. S., & Ouhyoung, M. (2002). Extracting 3D facial animation

parameters from multiview video clips. Computer Graphics and Applications,

IEEE, 22(6), 72-80.

Lin, I. C., & Ouhyoung, M. (2005). Mirror MoCap: Automatic and efficient capture of

dense 3D facial motion parameters from video. The Visual Computer, 21(6),

355-372.

Liu, C. H., Collin, C. A., Burton, A. M., & Chaudhuri, A. (1999). Lighting direction

affects recognition of untextured faces in photographic positive and negative.

Vision Research, 39(24), 4003-4009.

Liu, C. H., Collin, C. A., Rainville, S. J., & Chaudhuri, A. (2000). The effects of spatial

frequency overlap on face recognition. Journal of Experimental Psychology.

Human Perception and Performance, 26(3), 956-79.

Liu, C. H., & Chaudhuri, A. (1997). Face recognition with multi-tone and two-tone

photographic negatives. Perception, 26(10), 1289-1296.

(32)

26

Mathworks Inc. (1993). MATLAB. Natick, MA: The MathWorks Inc.

Näsänen, R. (1999). Spatial frequency bandwidth used in the recognition of facial

images. Vision Research, 39(23), 3824-3833.

Parker, D. M., & Costen, N. P. (1999). One extreme or the other or perhaps the golden

mean? Issues of spatial resolution in face processing. Current Psychology, 18(1),

118-127.

Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming

numbers into movies. Spatial vision, 10(4), 437-442.

Ramachandran, V. S. (1988a). Perceiving shape from shading. Scientific American,

259(2), 76-83.

Ramachandran, V. S. (1988b). Perception of shape from shading. Nature, 331(6152),

163-166.

Rhodes, G. (1988). Looking at faces: First-order and second-order features as

determinants of facial appearance. Perception, 17(1), 43 – 63.

Rhodes, G., Geddes, K., Jeffery, L., Dziurawiec, S., & Clark, A. (2002). Are average and

symmetric faces attractive to infants? Discrimination and looking preferences.

Perception, 31(3), 315-322.

Rhodes, G., Peters, M., Lee, K., Morrone, M. C., & Burr, D. (2005). Higher-level

(33)

27

mechanisms detect facial symmetry. Proceedings of the Royal Society B:

Biological Sciences, 272(1570), 1379.

Ruiz-Soler, M., & Beltran, F. (2006). Face perception: An integrative review of the role

of spatial frequencies. Psychological Research, 70(4), 273-292.

Schyns, P. G., & Oliva, A. (1997). Flexible, diagnosticity-driven, rather than fixed,

perceptually determined scale selection in scene and face recognition.

Perception, 26(8), 1027 – 1038.

Schyns, P. G., & Oliva, A. (1999). Dr. Angry and Mr. Smile: When categorization

flexibly modifies the perception of faces in rapid visual presentations. Cognition,

69(3), 243-265.

Tyler, C. W., & McBride, B. (1997). The Morphonome image psychophysics software

and a calibrator for Macintosh systems. Spatial vision, 10(4), 479-484.

(34)

28

List of Figures

C.

A. B.

D.

E.

Figure 1. The symmetric algorithm helps to dissociate the contribution of the surface albedo from illumination component in image.

A, LA, is asymmetric component of illumination of face image. Through the process of shape-from-shaping, the LA provided 3D information about a face on a 2D image.

B, CS, is surface albedo. The face containing only CS looks very flat. In addition, it is next to impossible to identify the identity of the owner of the face with only the CS information. C, Original image. D, Residual information. E, Put the LA and the CS together, it is easy to see that the hybrid image and the original are pictures of the same person even though a lot of information (D) was thrown away.

(35)

29

A B

C

Original face

symHSF face

symHSF face 0

1.0

0.8 0.6 0.5 0.4 0.2

Morphing : 7 levels

0∘ 15∘ 30∘ 60∘

Low-pass filtered, asyLSF faces Original faces

Low-pass filtered, symLSF face

0.5 femininity symHSF face

0°

female asyLSF

15°

female asyLSF

30°

female asyLSF

60°

female asyLSF

Hybrid images

Task:

What is the gender of this person ?

Figure 2. Examples of stimuli.

A, We generated a series of intermediate facial information by morphing the symmetric components of high special frequency (symHSF) between the two faces. There were seven morphed images with proportion of femininity from 0 (male) to 1 (female). B, Examples of female asymLSF and symLSF images. There were four lighting directions: 0°, 15°, 30°, and 60°. The original female face was low-pass filtered to preserve coarse-scale shading information. C, The hybrid face stimuli was a combination of the female (or male, not shown) asymLSF of one lighting direction and on one of the morphed face. Here are examples with the 0.5 femininity symHSF face. The symHSF face looks very flat and it is hard to judge the gender. When combined with a female asymLSF face, say, the one with 60° lighting direction, the resulting hybrid face was frequently categorized as female by most observers.

(36)

30

A B

C D

E F

0 0.2 0.4 0.6 0.8 1.0

0 0.2 0.4 0.6 0.8 1

Female asymLSF information

Fraction of Female Responses observer CIT

symHSF 0 ° 15 ° 30 ° 60 °

Proportion of femininity

0 0.2 0.4 0.6 0.8 1.0

0 0.2 0.4 0.6 0.8 1

Male asymLSF information

Fraction of FemaleResponses Observer CIT symHSF 0 ° 15 ° 30 ° 60 °

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Fraction of Female Responses Observer CAE

symHSF 0 ° 15 ° 30 ° 60 °

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Fraction of Female Responses Observer CAE symHSF 0 ° 15 ° 30 ° 60 °

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Fraction of Female Responses Observer CAI

symHSF 0 ° 15 ° 30 ° 60 °

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Fraction of Female Responses Observer CAI symHSF 0 ° 15 ° 30 ° 60 °

(37)

31

G

0° 15° 30° 60°

-1.0 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1.0

PSE shift from symHSF condition (proportion of femininity)

Male asymLSF + symHSF symHSF only

Female asymLSF + symHSF Beyond measurable range

**

p=0.0013

***

p<0.001

p=0.208

*

p=0.019 **

p=0.0012 *

p=0.014

Lighting directions

Figure 3. The effect of asymLSF on face discrimination.

A, B, C, D, E, and F, Psychometric functions from three naïve observers. The fraction of femininity responses is plotted as a function of the proportion of femininity of symHSF.

The black psychometric curve is from symHSF only condition as a comparison. The red, green, blue, magenta psychometric curves are results of asymmetric facial information.

Under the female asymLSF condition, when lighting direction was shifted to more lateral, the observers saw female faces more frequently, and the psychometric curves shifted gradually to left. On the contrary, under the male asymLSF condition, the psychometric curve shifted dramatically to right, and psychometric curve was saturated under 30° and 60° illumination conditions, especially in B and F. These two observers tend to saw male faces in every level of femininity. G, Averaged PSE shift from symHSF only condition by asymmetric low spatial frequency facial information. The error bars represent one standard error. The p-value shown in each lighting direction was calculated from a two-tailed paired t test. The averaged PSE shift in male 30^o and 60^o illumination condition was beyond measurable range, so we used an arrow to plot.

(38)

32

A

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

LSF information

Proportion of femininity Fraction of female response Observer CIT

symHSF F_asymLSF + symHSF M_asymLSF + symHSF F_symLSF + symHSF M_symLSF + symHSF

B

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

LSF information

Fraction of female response

Observer CAE

symHSF F_asymLSF + Morphs M_asymLSF + Morphs F_symLSF + Morphs M_symLSF + Morphs

C

0 20 40 60 80 100

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

LSF information

Fraction of female response

Observer CAI

symHSF F_asymLSF + Morphs M_asymLSF + Morphs F_symLSF + Morphs M_symLSF + Morphs

D

symHSF F_60 ° symLSF

F_60 ° asymLSF

M_60 ° symLSF

M_60 ° asymLSF -0.4

-0.2 0 0.2 0.4 0.6 0.8 1.0

LSF information

N=7

**

p=0.0014 p=0.9996

p=0.5

Beyond measurable

Figure 4. Comparisons of the effect of asymLSF and symLSF on face discrimination.

A, B, C, Psychometric function from three naïve observers. The fraction of female responses is plotted as a function of the proportion of femininity. The black psychometric curve is from symHSF only condition in corresponding observers. The cyan psychometric curves are results of symLSF + symHSF in the female and male conditions. The magenta curves are results of asymLSF + symHSF faces. Results show that the symLSF had very little influence on face discrimination, because the psychometric curves were not significantly shifted. D, The average PSE shift relative to the symHSF only condition. The p-values denote the result of the paired t test. The averaged PSE shift in male 60^o illumination condition was beyond measurable range, so we used an arrow to plot.

(39)

33

Figure 5. Results of depth judgment.

A, female condition. B, male condition. The depth rating value of the hybrid faces:

asymLSF (under four lighting directions) combined with symHSF, symLSF combined with symHSF (under 60^o), and female and male symHSF. The depth value of the reference sphere was set to 3. Using least squares analysis, the linear regression lines of asymLSF condition of both genders show a positive increase in proportion to the lighting directions.

The slope parameters were significantly larger than zero (t(2) = 10.67, p= .0086 for the female and t(2)= 4.49, p=.046 for the male condition)

A

0ô 15ô 30ô 60ô

0 1 2 3 4 5

Lighting directions of LSFs

Y= 2.892+0.025X R²=0.983

Ambient illumination

Female asymLSF + symHSF Female symLSF + symHSF Female symHSF

Depth rating (abstract value)

N=7

B

Y= 2.436+0.031X R²=0.91

0ô 15ô 30ô 60ô

0 1 2 3 4 5

Ambient illumination

Depth rating (abstract value)

Male asymLSF + symHSF Male symLSF + symHSF Male symHSF

Lighting directions of LSFs

(40)

34

Figure 6. Face discrimination performance is highly correlated with perceived depth.

A, female condition, Pearson correlation coefficient of depth rating value and averaged PSE shift, , and B, male condition . The averaged PSE shift in male 30^o and 60^o illumination condition was beyond measurable range, so we used arrows to plot.

A

1 2 3 4 5

-1.0 -0.8 -0.6 -0.4 -0.2

0 Female asymLSF

Female symLSF

Depth rating PSE shift from symHSFcondition (proportion of femininity)

Y= 0.37-0.167X R²=0.903

60^o

0^o

15^o

30^o

60^o

B

Y= -160.37+89.9X R²=0.863

0^o 15^o

60^o

Male asymLSF Male symLSF 60^o

30^o

PSE Beyond measurable range

1 2 3 4 5

0 0.2 0.4 0.6 0.8 1.0

Depth rating

(41)

35

Supplementary data

A

0 0.2 0.4 0.6 0.8 1.0

0 0.2 0.4 0.6 0.8 1

Fraction of Female Responses observer CIT

symHSF 0 ° 15 ° 30 ° 60 °

B

0 0.2 0.4 0.6 0.8 1.0

0 0.2 0.4 0.6 0.8 1

Fraction of FemaleResponses Observer CIT symHSF 0 ° 15 ° 30 ° 60 °

C

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Fraction of Female Responses Observer CAE

symHSF 0 ° 15 ° 30 ° 60 °

D

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Fraction of Female Responses Observer CAE symHSF 0 ° 15 ° 30 ° 60 °

E

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Fraction of Female Responses Observer CAI

symHSF 0 ° 15 ° 30 ° 60 °

F

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Fraction of Female Responses Observer CAI symHSF 0 ° 15 ° 30 ° 60 °

(42)

36

G

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Proportion of femininity Fraction of Female Responses Observer CPY

symHSF 0 ° 15 ° 30 ° 60 °

H

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Proportion of femininity Fraction of Female Responses Observer CPY

symHSF 0 ° 15 ° 30 ° 60 °

I

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Proportion of femininity Fraction of Female Responses Observer CBN

symHSF 0 ° 15 ° 30 ° 60 °

J

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Proportion of femininity Fraction of Female Responses Observer CBN

symHSF 0 ° 15 ° 30 ° 60 °

K

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Proportion of femininity Fraction of Female Responses Observer LNL

symHSF 0 ° 15 ° 30 ° 60 °

L

0 0.2 0.4 0.6 0.8 1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Proportion of femininity Fraction of Female Responses Observer LNL

symHSF 0 ° 15 ° 30 ° 60 °

Figure 7. All seven observers’ asymLSF condition data.

The fraction of femininity responses is plotted as a function of the proportion of femininity.

非對稱低空間頻率訊息有助於臉孔區辨

國立臺灣大學理學院心理學研究所 碩士論文

Graduate Institute of Psychology College of Science

National Taiwan University Master Thesis

非對稱低空間頻率訊息有助於臉孔區辨

Asymmetric Low Spatial Frequency Improves Face Discrimination

陳巾眉 Chin-Mei Chen

指導教授：陳建中 博士

Advisor: Chien-Chung Chen, Ph.D.

中華民國 98 年 6 月

June, 2009

誌謝

誌謝 誌謝

誌謝

摘要 摘要 摘要 摘要

Asymmetric Low Spatial Frequency Information Improves Face Discrimination

Chin-Mei Chen Abstract

CONTENTS

LIST OF FIGURES

Chapter 1 Introduction

Chapter 2 Method

2.1 Apparatus

2.2 Stimuli

2.3 Procedure

2.3.1 Face discrimination

2.3.2 Depth judgment

2.4 Data analysis

2.5 Participants

Chapter 3 Results

3.1 asymLSF information changed perceived facial information of hybrid faces.

3.2 symLSF information did not change perceived facial information of hybrid images.

3.3 Depth judgment

Chapter 4 Discussion

4.1 Illumination effects in face recognition

4.2 Spatial frequency effect

Chapter 5 Conclusions

References

List of Figures

國立臺灣大學理學院心理學研究所碩士論文

指導教授：陳建中博士

誌謝誌謝

摘要摘要摘要摘要