以物理為基礎的彩妝顯像方法

(1)

國

立

交

通

大

學

多媒體工程研究所

碩

士

論

文

以物理為基礎的彩妝顯像方法

A Physically-Based Cosmetic Rendering

研究生：黃程國

指導教授：莊榮宏教授

林文杰教授

(2)

研究生：黃程國 Student：Cheng-Guo Huang

指導教授：莊榮宏 Advisor：Jung-Hong Chuang

林文杰 Wen-Chieh Lin

國立交通大學

多媒體工程研究所

碩士論文

A Thesis

Submitted to Institute of MultimediaEngineering College of Computer Science

National Chiao Tung University in partial Fulfillment of the Requirements

for the Degree of Master

in

Computer Science

October 2011

Hsinchu, Taiwan, Republic of China

中華民國一百年十月

(3)

i

以物理為基礎的彩妝顯像方法

研究生 : 黃程國指導教授 : 莊榮宏博士

林文杰博士

國立交通大學

資訊學院多媒體工程研究所

摘要

在3D人臉動畫領域中，如何模擬逼真的彩妝效果是一項重要的議題。在傳統的化妝顯像方法中，通常是使用影像處理的方式，將彩妝效果轉換至目標圖片上，但是這種方法具有天生的缺陷，即彩妝形狀的變形和需要相似的觀察角度拍攝模型。此外，影像處理的方法需要在相似的打光效果拍攝影像，例如:如果拍攝的打光條件不同，通常在結果上會造成不自然的效果。因此，我們提出一個整合型的方法，結合螢幕空間的顯像方式和庫貝卡-蒙克理論顯像彩妝效果。在實驗過程中，我們量測眾多的化妝品，像液態粉底液、腮紅和唇膏，然後再經由庫貝卡 -蒙克模型計算出相關的參數資料。顯像步驟分為兩個階段，皮膚顯像和彩妝顯像。在皮膚顯像的過程中，考慮光通過彩妝品後的能量，並將其能量套入直接光照的結果，然後再進行原始的螢幕空間的顯像方法計算皮膚表層能量。最後，藉由多層次理論，將皮膚和彩妝效果合併，以獲得最後的顯像結果。

(4)

Student:

Cheng-Guo Huang

Advisor:

Dr. Jung-Hong Chuang

Dr. Wen-Chieh Lin

Institute of Multimedia Engineering

College of Computer Science

National Chiao Tung University

ABSTRACT

Simulating realistic makeup effect is one of important research issues in the 3D facial animation and cosmetic industry. In tradition, people usually use image processing such as warping to transfer one’s makeup to another subject. But there are many disadvantages that consist of shape distortion and restriction on the similar viewing position. Besides, these methods disregard the lighting condition. Hence, we propose an integrated approach, which combines the screen-space rendering technique with the Kubelka-Munk theory, to render the 3D makeup effect. We measure many cosmetics, such as foundations, blushes and lipsticks, and we compute the parameters of Kubelka-Munk model from these data. In the rendering stage, we consider that light penetrates through the cosmetic. Therefore, we use Kubelka-Munk model to compute the total transmittance, and then we apply this value to render the appearance of human skin. Finally, we use the multi-layer theory to combine each layer together.

(5)

I would like to thank my advisors, Professor Jung-Hong Chuang, and Wen-Chieh Lin for their guidance, inspirations, and encouragement. I also want to thank my senior colleague Tsung-Shian Huang, for supporting me to measure data and help me to solve some rendering problems. Thanks to my colleagues during these two years: Jen-Chieh Liao, Jen-Hao Liao, Yi-Shan Li, Bo-Yin Yao, Mu-Shiue Li and Shao-Ti Lee for their assistances and discussions. Especially thanks to Chuan-Chang Wang (NEXT MEDIAr) which gives us many scanning data of human head (digital Coco and Orange), and thanks to Huey-Min Lai and Kuo-Jui Ho (Industrial Technology Research Instituter) which support us to measuring data. I am very thanks for my colleague Fu-His Huang who helps me to design cosmetic maps. Lastly I would like to thank my family for their love and support.

(6)

List of Figures

2.1 divide two example images to obtain cosmetic map[1] . . . 4

2.2 the workflow of digital face makeup by example [2] . . . 5

2.3 the synthesis cosmetic map[3] . . . 6

2.4 the results of computer-suggested makeup[3] . . . 7

2.5 the fitting result of the foundation [4] . . . 7

2.6 the rendering result of [4] . . . 8

2.7 the schema of the two dipole source [5] . . . 9

2.8 the render result of Donner and Jensen [6] . . . 10

2.9 the approximating Dipole profiles with sum-of-Gaussian [7] . . . 11

2.10 the simple diagram of [8] . . . 11

3.1 Framework of our simulation system . . . 14

3.2 The screen-space human skin rendering pipeline . . . 16

3.3 The workflow of texture-space and screen-space . . . 17

3.4 the result of softness shadow boundary . . . 18

3.5 The influence of different global subsurface scattering value . . . 19

3.6 blurred operation . . . 20

3.7 Fitting result by six Gaussian weights and four Gaussian weights . . . 21

3.8 the simple view of Schuster’s assumption . . . 23 3.9 The chromaticity coordinate of liquid-foundation from four different observation. 24

(9)

3.12 the pass of light between foundation and skin . . . 28

3.13 the flow chart about the skin pass . . . 29

3.14 the cosmetic map . . . 30

3.15 the total transmittance after the light goes through the foundation . . . 31

3.16 the irradiance map and applying the transmittance (w/o SSSS) . . . 31

3.17 the irradiance map and applying the transmittance (with SSSS) . . . 32

3.18 the work flow about the cosmetic pass . . . 33

3.19 rendering result . . . 34

3.20 enlarging of skin surface . . . 35

3.21 the rendering result of specular effect . . . 36

3.22 the architecture of simulation system . . . 38

4.1 Our capture device and environment . . . 40

4.2 Radiant energy of liquid foundation . . . 41

4.3 Radiant energy of CIE standard illuminant D55 . . . 41

4.4 Radiant energy of CIE standard illuminant E . . . 42

4.5 the reflectance of barium sulfate and barium sulfate sample . . . 43

4.6 Reflectance of liquid foundation . . . 44

4.7 Liquid foundation sample. Left: background. Mid: thickness layer. Right: thin layer . . . 44

4.8 The sample of our simple container . . . 45

4.9 The reflectance of Ri, R and Rg . . . 45

4.10 The coefficient S and K for each wavelength . . . 46

4.11 The original reflectance and the fitting result . . . 46

(10)

0.57, 0.5) . . . 47

4.13 CIERGB color of fitting curve and measured curve. Left: Original data (R0), color (0.62, 0.48, 0.38). Right: Fitting result, color (0.60, 0.47, 0.38) . . . 47

4.14 Selecting two colors and obtaining S and K . . . 48

4.15 capture coefficient S and K by camera . . . 48

4.16 liquid foundation sample . . . 49

4.17 diffuse color captured by our simple device . . . 49

5.1 original appearance of skin rendering . . . 51

5.2 the measuring data of sample F1 . . . 53

5.3 the fitting data of sample F1 . . . 53

5.4 the comparison between R and R0 of sample F1 . . . 54

5.5 the rendering result of sample F1 . . . 55

(11)

5.22 the measuring data of sample B1 . . . 68

5.23 the fitting data of sample B1 . . . 69

5.24 the comparison between R and R0 of sample B1 . . . 69

5.25 the rendering result of sample B1 . . . 70

5.26 the measuring data of sample B2 . . . 71

5.27 the fitting data of sample B2 . . . 71

5.28 the comparison between R and R0 of sample B2 . . . 72

5.29 the rendering result of sample B2 . . . 73

5.30 the measuring data of sample L1 . . . 74

5.31 the fitting data of sample L1 . . . 75

5.32 the comparison between and R0 of sample L1 . . . 75

5.33 the rendering result of sample L1 . . . 76

5.36 the comparison between R and R0 of sample L2 . . . 77

(12)

5.58 the comparing result of with SSS and without SSS . . . 89

5.59 combination with different cosmetics and apply on Digital Coco . . . 90

5.60 combination with different cosmetics and apply on a man head . . . 91

5.61 combination with different cosmetics and apply on Digital Orange . . . 92

5.62 comparison rendering result with reference image . . . 93

5.63 comparison rendering result with reference image. (do not contain eyelashes and eyebrows) . . . 93

A.1 Obtaining reflectance by remove light effect . . . 97

A.2 CIE 2-degree color matching functions . . . 98

A.3 converting diagram . . . 99

(13)

List of Tables

3.1 blend weight . . . 20

5.1 The brand of liquid foundation . . . 52

5.2 The brand of each cream blush . . . 68

5.3 The brand of each lipstick . . . 74

(14)

1

Introduction

Rendering the realistic image of human skin with makeup is an important issue in the 3D facial animation and cosmetic industry. Facial makeup is a most important routine for some female or some men, and the makeup is a multi-layered process. For example, we usually do the skin care first and smear some cosmetics (such as foundation, blush, lipstick and eye-shadow) on our face. By smearing the cosmetics on our face, we can change our facial appearance in different style.

There are many researches to render the makeup effect and can be divide into two main categories cosmetic transfer and makeup simulation. Cosmetic transfer uses the example image to transfer the makeup effect, and makeup simulation uses some properties to simulate the makeup effect. Both of them have some individual advantages. By using the transfer technique, we can directly change the makeup effect and this technique can obtain additional business value by employing to salon. Makeup simulation can be used in cosmetic industry or be used to simulate different makeup style. We can even render the makeup effect under different situation. For example, in nightclub, in the party or under the sunlight.

In our work, we expect to simulate the realistic image of human skin with the makeup effect.

(15)

We use the technique of screen-space to render the human skin and combine it with the Kubelka-Munk theory to simulate the makeup effect. The cosmetics include many special effects, such as pearl effect and glitter effect. However, these effects are difficult to simulate, because these effects are view-dependent. Therefore, our simulation does not process these complex effects. We concentrate on the cosmetics without the pearl effect and glitter effect, and we support the color changing with the assumption that the cosmetics are homogenous material.

1.1 Contribution

• We propose a complete algorithm to combines the screen-space skin rendering method and the Kubelka-Munk theory to obtain the makeup result.

• The makeup simulation system performs in real-time, and user can modify the style of cosmetic in interactive frame rate.

1.2 Organization of the thesis

The thesis is organized as follows: Chapter 2 gives some related work about the cosmetic ren-dering and the diffuse approximation of translucent renren-dering. Chapter 4 explains how to obtain the parameters from the measuring data, and shows other approximate method to get parame-ters. Chapter 3 illustrates the framework and explains our approach in detail. Chapter 5 shows the results of our simulation system. Finally, we summarize our approach and discuss the future work in Chapter 6.

(16)

2

Related Work

In this chapter, we review the related work in the cosmetic rendering and the diffuse approxi-mation.

2.1 The cosmetic rendering

Cosmetic rendering can be divided into two main categories: cosmetic transfer and cosmetic simulation.

Cosmetic transfer

Cosmetic transfer means to transfer ones makeup information to another subject. Such methods usually require one or more examples to obtain cosmetic information (or the cosmetic map). The cosmetic map represents the change caused by makeup effect. There are some researches in this field, such as Tong et al. [1], Dong Guo and Terence Sim [2] and Scherbaum et al. [3].

Tong et al.[1] use a pair of 2D example images of the same face ”before” and ”after” makeup to find the cosmetic map, and apply the cosmetic map to the subject. In their work, their first

(17)

step is to remove the non-skin feature. For example: eyebrows, eyelashes and some inherent skin features (such as freckles, moles and blemishes), and then warp two example images to the subject image. After the warping, they divide the ”after” image by the ”before” image to obtain the cosmetic map. In this map, each value of this map is the Bidirectional reflectance distribution function (BRDF) value in this view (Figure 2.1). Then the subject is multiplied by the cosmetic map to obtain the makeup result.

A pair of example image

A* divided by A

Cosmetic map

(18)

Dong Guo and Terence Sim [2] propose another convenient approach to transfer cosmetic effect. In their work, they use one example image and one subject image as input. There are four main steps in this approach: face alignment and warping, layer decomposition, synthesis and combination. This method use the image warping to transfer the makeup effect, so it needs a fully alignment before transferring. First, they warp the example image to the subject. Second, they analyze the example image and the subject image to extract the face structure, the skin detail and the color. After decomposition is finished, the information from each layer of example is transferred to corresponding layer of the subject in different way: skin detail is transferred by additive way; color is transferred by alpha blending; shading effect and highlight in the face structure will be transferred by gradient editing. Finally, each layer are composed together and the final result is obtained. The workflow of their approach is illustrated in the figure 2.2.

Subject Face Structure Skin Detail Color Results

Example I Is Id a∗ Ic E Es Ed Ec R Rs Rd Rc + α ∇ W b∗ a∗ b∗ a∗ b∗

Figure 2. The workflow of our approach. W denotes warping. ∇ denotes gradient editing. + denotes weighted addition. α denotes alpha blending. In this figure, the value of skin detail layer, i.e. Idand Ed, is exaggerated 4 times for better visualization. Our approach consists

of four main steps: face alignment, layer decomposition, makeup transferring, and layer composition.

shading effects in face structure layer are transferred in the way of gradient editing (Section 3.5). Finally, three resul-tant layers are composed together.

3.1. Face alignment

For face alignment, we adopt the Thin Plate Spline (TPS) [1] to warp the example image E to subject image I. The control points required by TPS are obtained using an ex-tended Active Shape Model (ASM) [8]. Due to the diversity of face appearance under various possible makeup, ASM may not get the accurate position of points. Our system may still require user to refine the position of control points. Since the control points have been roughly located already, the refinement does not require much effort. It usually takes less than 1 minute to refine the control points for a face. An example of refined control points is shown in Fig. 3(a). There are 83 points in total on a face.

As shown in Fig. 3(b), these control points have defined different face components, viz eyebrows, eyes, nose, nos-trils, lips, mouth cavity (the space between lips), and other facial skin (the rest of the face). These components are fur-ther divided into three classes to be treated in different ways during makeup. These three classes (C1 ∼ C3) are

illus-trated in different colors in Fig. 3(b). C1(the skin region,

the entire face excluding C2and C3) follows the workflow

illustrated in Fig. 2. Since the texture of C2 (lips) varies

greatly from person to person and the region of C2is easy

to deform, we use a special method to transfer the makeup style in this region (discussed in Section 3.6). C3(eyes and

mouth cavity) is kept untouched all the time during entire makeup process.

3.2. Layer decomposition

The subject image I and example image E (after warp-ing) are first decomposed into color and lightness layers. We then further decompose the lightness layer into face structure and skin detail layers.

In the first step, I and E are decomposed into color and lightness layers by converting them to CIELAB colorspace. The L∗_{channel is considered as lightness layer and a}∗_{, b}∗

channel the color layer. We choose the CIELAB colorspace because it performs better than other color spaces in sep-arating lightness from color [17], and it is approximately perceptually uniform [7].

Second, the lightness layer is decomposed to large-scale and detail layers. The large-scale layer is considered as the face structure layer and the detail layer as skin detail layer. Large-scale/detail layer decomposition has been addressed in many works, such as [3] and [19]. The main idea is first to perform an edge-preserving smoothing on the lightness layer to obtain the large scale layer, and then to subtract (or divide) the large scale layer from the lightness layer to ob-tain the detail layer. In this approach, we adapt a weighted-least-squares (WLS) operator recently proposed by Farb-man et al. [4]. An alternative method is bilateral filtering [14], which was used in many previous works. We choose this WLS operator because of its better performance com-pared to the bilateral filter, especially when the blur level Figure 2.2: the workflow of digital face makeup by example. [2]

Both two methods have a fatal problem by 2D image warping which causes the shape of makeup to be deformed, and they need to capture the image at the same or similar viewing position. Dong Guo and Terence Sim [2] use one example image to transfer the cosmetic effect, but it loses much information. For example, they transfer the color by blending operation, and

(19)

the color of makeup effect cannot be transferred correctly. (Because each appearance of human skin has different properties, such as skin color.)

Scherbaum et al. [3] build a database by the device of structure light and use the database to re-synthesize the makeup effect. They capture human faces that include makeup effect and without makeup effect, and then extract the information from the pair of images, such as posi-tion, normal, diffuse map, scattering profile, specular and the cosmetic map, etc. In the run-time, user inputs a 2D image to query the appropriate 3D face and the 3D face is morphed to fit the 2D image. After fetch the suitable face, they query the database to find to best-match facial appearance and cosmetic map by PCA. Then, the system re-synthesizes the cosmetic and ap-plies it to diffuse map. They provide a better way to transfer the makeup effect, but there are many problems about it. First, the style of makeup is fixed. Second, the system needs give more constrains to re-synthesizes the credible cosmetic map. Final, the system needs many examples to give the appearance correctly. Figure 2.3 shows the synthesis cosmetic map and figure 2.4 gives some results.

(20)

Figure 2.4: the results of computer-suggested makeup. [3]

Cosmetic simulation

Moriuchi et al. [4] use a physical model and a statistical approach to simulate the foundation. They measure the foundation from different view angle and capture it reflectance between 400 nm and 700 nm. Then, they use the Cook-Torrance model to find appropriate parameters. Using Cook-Torrance model to describe the translucent material is not enough, because this model is used to describe the lambert material. Therefore, they employ the statistical approach to analyze the reflectance data by PCA. Figure 2.5 shows the fitting result and Figure 2.6 shows the rendering result.

(21)

Figure 2.6: the rendering result of [4]

Doi et al. [9] use the Kubelka-Munk theory to render the foundation on the skin. They measure the reflectance of many skins and liquid foundations, and then use the Kebelka-Munk theory to render result. Even they use a good way to simulate the makeup effect, they do not consider an important issue that measuring the reflectance of skin cannot obtain the translucent appearance. We extend this approach to 3D space, and provide a complete algorithm to render it.

2.2 Translucent rendering

Human skin is a kind of translucent materials. There some rendering technique to render translucent material, but the diffuse approximation is a better way to render translucent ma-terials.

Jensen et al. [5] introduce to use diffuse approximation to solve the radiant transfer equa-tion. By using the diffuse approximation, they translate the computation of complex interior-behavior to the function of distance-related. By this way, they use dipole to compute the diffuse reflectance. Figure 2.7 shows a pair of dipole source. They assume the thickness of material

(22)

is infinity and the scattering coefficient σs is much larger than the absorption coefficient σa.

Hence, the render result is more translucent. _{To appear in the SIGGRAPH conference proceedings}

z_r z_v

r

Figure 3: An incoming ray is transformed into a dipole source for the diffusion approximation.

2.2 Single Scattering Term

Hanrahan and Krueger [10] have derived a BRDF model for subsur-face reflection that analytically computes the total first-order scat-tering from a flat, uniformly lit, homogeneous slab. In this section, we show how their BRDF can be extended to a BSSRDF in order to account for local variations in lighting over the surface.

The total outgoing radiance, L(1)o , due to single scattering is

computed by integrating the incident radiance along the refracted outgoing ray (see Figure 4):

L(1)o (xo, ~ωo) = σs(xo) Z 2π F p(~ωi0· ~ω0o) Z ∞ 0 e−σtcsLi(xi, ~ωi) ds d~ωi (6) = Z A Z 2π S(1)(xi, ~ωi; xo, ~ωo) Li(xi, ~ωi) (~n· ~ωi) dωidA(xi).

Here F = Ft(η, ~ωo)Ft(η, ~ωi) is the product of the two Fresnel

transmission terms, andω~0iand ~ω0oare the refracted incoming and

outgoing directions. The combined extinction coefficient σtc is

given by σtc = σt(xo) + Gσt(xi), where G is a geometry

fac-tor; for a flat surfaceG = |~ni·~ωo0|

|~ni·~ω0_i|. The single scattering BSSRDF,

S(1), is defined implicitly by the second line of this equation. Note that there is a change of variables between the first line, which in-tegrates only over the configurations where the two refracted rays intersect, and the second line, which integrates over all incoming and outgoing rays. This implies that the distributionS(1)contains a delta function.

2.3 The BSSRDF Model

The complete BSSRDF model is a sum of the diffusion approxima-tion and the single scattering term:

S(xi, ~ωi; xo, ~ωo) = Sd(xi, ~ωi; xo, ~ωo) + S(1)(xi, ~ωi; xo, ~ωo)

Here Sd is evaluated using Equation 5 and S(1) is evaluated

us-ing Equation 6. The parameters for the BSSRDF are: σa,σ0s, η,

and possibly a phase function (without a phase function the scat-tering can be modeled as isotropic). This model accounts for light transport between different locations on the surface, and it simu-lates both the directional component (due to single scattering) as well as the diffuse component (due to multiple scattering).

Finally, note the distances involved in both the single scattering term and the diffusion approximations. The average exit point is approximately one mean free path from the entry point. However, these two mean free paths have quite different length scales. In the single scattering case, the mean free path equals1/σt; in the

diffusion case, the mean free path equals 1/σtr. For translucent

materials whereσa σ0s and consequentlyσtr σt, the single

scattering term decreases much faster than the diffusion term as the distance toxoincreases.

x

s

i xo

Figure 4: Single scattering occurs only when the refracted incoming and outgoing rays intersect, and is computed as an integral over path length s along the refracted outgoing ray.

2.4 BRDF Approximation

We can approximate the BSSRDF with a BRDF by assuming that the incident illumination is uniform. This assumption makes it pos-sible to integrate the BSSRDF over the surface. By integrating the diffusion term we find the total diffuse reflectanceRdof the

mate-rial as: Rd = 2π Z ∞ 0 Rd(r) r dr = α0 2 1 + e−43A √ 3(1−α0) e−√3(1−α0). Notice how the diffuse reflectance only depends on the reduced albedo and the internal reflection parameterA.

The integration of the single scattering term results in the model presented in [10]. For a semi-infinite medium this gives:

fr(1)(x, ~ωi, ~ωo) = αF p(~ω 0 i· ~ωo0) |~n · ~ω0 i| + |~n · ~ω0o| .

The complete BRDF model is the sum of the diffuse reflectance scaled by the Fresnel term and the single scattering approximation:

fr(x, ~ωi, ~ωo) = fr(1)(x, ~ωi, ~ωo) + F

Rd

π .

This model has the same parameters as the BSSRDF. It is similar to the BRDF model presented in [10], but with the important differ-ence that the amount of diffusely reflected light is computed from the intrinsic material parameters. The BRDF approximation is use-ful for opaque materials, which have a very short mean free path.

3 Measuring the BSSRDF

To verify our BSSRDF model, and to determine appropriate pa-rameters for rendering different kinds of materials, we used the diffusion theory of Section 2 to make measurements of subsurface scattering in several media. Our measurement approach applies to translucent materials for whichσa σs, implying that far enough

away from the point of illumination, we may neglect single scatter-ing and use the diffusion term to relate measurements to material parameters.

When multiple scattering dominates, Equation 4 predicts the ra-diant exitance per unit incident flux that will be observed due to a narrow incident beam, as a function of distance from the point of illumination. To make the corresponding measurement, we illumi-nate the surface of a sample with a tightly focused beam of white light and take a photograph using a 3-CCD video camera to observe the radiant exitance across the entire surface. We keep our obser-vations at constant angles so that the Fresnel term remains constant for all the measurements. Figure 5(a) illustrates our measurement setup.

4 Figure 2.7: the schema of the two dipole source. [5]

Donner and Jensen [6] extend the dipole approximation. They use the multi-pole to approx-imate the thin layer’s material, such as skin and leaves. After computing the reflectance of each layer, they use multi-layered model to combine each layer. Figure 2.8 shows the render result of Donner and Jensen [6].

(23)

Figure 2.8: the render result of Donner and Jensen [6].

Both of the above methods take long time to render an image. For example, Donner and Jensen [6] use one to five minutes to render an image. Jensen and Buhler [10] provide a hier-archical structure to reduce the computation time. They apply hierhier-archical structure to dipole method [5], and the rendering time takes 7-8 second per image. Therefore, using the hierarchi-cal structure still cannot reach real-time performance.

d’Eon et al. [8] provide a texture-space algorithm to render the translucent materials in real-time frame rate. The multi-pole method [6] provide a good way to simulate translucent materials, but this process is double-integration for gathering energy. Hence, it needs much time to compute. d’Eon et al. [8] use sum-of-Gaussian to approximate the diffuse profile. After the diffusion profile is represented by sum-of-Gaussian, the complex integration of original equation can be approximated by multiple Gaussian-filters. In the run time, they render the irradiance map in texture-space, and then they use Gaussian-filter to blur this map many times. Finally, they combine each blurred result by the weights of sum-of-Gaussian. Figure 2.9 shows the approximating Dipole profiles with sum-of-Gaussian and Figure 2.10 shows a diagram of

(24)

this technique.

Figure 2.9: the approximating Dipole profiles with sum-of-Gaussian.[7].

Figure 2.10: the simple diagram of [8].

Jimenez et al. [11] extend the texture-space technique to the screen-space. They render the shading result in screen-space and store it in render target. Then, they considered screen-space information to blur the result many times. In each blurring pass, they use the alpha blending to accumulate the blurring result and the blending weights are according to the sum-of-Gaussian. We apply this technique to our simulation system.

(25)

d’Eon and Irving [12] propose a quantized-diffusion model for rendering translucent ma-terial. In their work, they decompose the dipole function to sum-of-Gaussian, and extend the multi-layered structure [6] to support the decomposed sum-of-Gaussian. The model of d’Eon and Irving [12] can support many materials, such as low absorption materials and high absorp-tion materials. They give an impressive result, but the rendering time of a frame need 20-30 minutes to render a frame.

(26)

3

Approach

The face make-up is a multilayered structure. It sometimes consisting of the foundation layer, the blush layer, the eye shadow layer and the skin layer. We combine a rendering method to translucent materials with traditional Kubelka-Munk theory to simulate the makeup appearance. In this chapter, we explain it in detail.

3.1 Framework

Our makeup simulation system can be divided into two main passes - skin rendering pass and makeup pass. In our simulation, we assume the cosmetics do not have the pearl effect and the sparkle effect. Figure 3.1 shows an overview of our simulation method.

In the skin pass, we use screen-space human skin rendering technique which is introduced by Jimenez et al [11] to simulate the appearance of human skin. In their work, they generate irradiance map in screen-space. Then, they blur the irradiance map many times and use alpha blending to acculturate each blurring result. Finally, they add specular part to the blurring result and obtains the final result. In order to use this technique to our makeup simulation, we

(27)

Start

Generating depth/stencil map

Rendering direct lighting result to RenderTarget

Blurring pass

Combining each blurred maps and adding specular

Depth/Stencil map Direct lighting Blurred maps output output input output input Skin rendering Result output

Computing cosmetic reflectance and transmittance Cosmetic

map

Combining each cosmetic layer

and skin layer with K-M model input

Computing specular of cosmetic

End Rendering

Skin rendering pass

Cosmetic rendering pass

Reflectance map

Transmittance map

Input

(28)

considered whether the surface is covered by the cosmetics. If the skin surface is covered by the cosmetics, we compute the total energy of light penetration through the cosmetics and apply this energy to generate the irradiance map in screen-space. In subsection 3.4, we explain this part in detail.

In the cosmetic pass, we use Kubelka-Munk theory to compute the reflectance and the trans-mittance of each cosmetic layer, and then we use the multi-layered theory to compute final reflectance.

There are some advantages by using our approach to simulate the makeup appearance:

1. The makeup simulation system performs in real-time, and the thickness of cosmetics can be changed in interactive frame rate.

2. User can modify cosmetic map and render different makeup appearance immediately.

3. Dislike the traditional image warping method, our approach can render in different views and can be used in different environment.

In the following subsections: First, we introduce how to render the appearance of skin by a screen-space technique. Next, we introduce the Kubelka-Munk theory that is used to compute layered reflectance and transmittance. Our system combines each layer by its multi-layered model in our simulation. Finally, we describe how to merge these two techniques to render our result.

3.2 The method of human skin rendering

Human skin is a kind of translucent material. It contains many biological layers, such as epider-mis, dermis and vein, etc. There are many methods to simulate human skin. The suitable way to render it is the diffusion approximate method (Jensen et al. [5], Donner and Jensen [6], d’Eon et al. [8], Jimenez et al. [11] and d’Eon and Irving [12]) and we follow the method proposed by Jimenez et al. [11] to simulate the appearance of human skin.

(29)

3.2.1 Human skin rendering

The screen-space approach translates the texture-space diffusion approximation to screen-space. Jimenez applies the convolution directly to final rendering rather than renders an irradiance map and convolves it many times with the diffusion profile. Figure 3.2 shows the original screen-space human skin rendering pipeline, and figure 3.3 gives a overview of the texture-space diffusion and the screen-texture-space rendering pipeline. The texture-texture-space diffusion has main cost in the blurring pass, because the texture-space diffusion blurs the irradiance map with full quad and it needs much memory to store the blurring result. Hence, the frame rate of texture-space diffusion is slower than the technique of screen-texture-space. For this reason, we choose the screen-space method to render the appearance of human skin.

(30)

Figure 3.3: The workflow of space and screen-space.Top: workflow about the texture-space diffusion. Bottom: screen-texture-space diffusion.[11]

The screen-space human skin rendering is three main passes. In the first pass, it generates shadow map from light source and generates the depth map and the stencil (matte) map from viewing position. (The depth map is used to decide the stretch factor for blurred pass and the stencil map is used to restrict the blurred area.) Second, it uses shadow map and color map to generate the irradiance map, which takes shadow and lambert term into account, in screen-space. Finally, it uses depth map and stencil map to blur the irradiance map many times, and uses alpha blending to accumulate each blurring results. In general, it also puts specular term to the irradiance map, then the softness and smoothness highlight can be obtained after the blurring pass is finished. By adding shadow to the irradiance map, we can achieve softness shadow boundary and make rendering result more reliable. Figure 3.4 shows softness shadow boundary and original shadow boundary. In this case, we use PCF to compute shadow which is introduced by Reeves et al. [13].

(31)

Figure 3.4: the result of softness shadow boundary.

In the traditional texture-space diffusion method, blurring pass needs a stretch map to obtain the stretch factor and uses it to decide the kernel width:

Kernel width = stretch factor * Gaussian width

In order to find a correct method to compute the stretch factors for screen-space approach, Jimenez observed several phenomena:

• The blurry kernels’ widths are related to the distance between camera and object. For example, if the camera is far from the object, it needs narrower kernel widths than close one.

• If the pixel of object surface is at steep angles from the view of camera, it needs narrower kernels than flatten one.

According to the above considerations, Jimenez derives two equations to obtain approximate stretch factors:

Sx =

α

d (x, y) + β*abs(∇x∗ d (x, y))

(32)

Sy =

α

d (x, y) + β*abs(∇y ∗ d (x, y))

(3.2) where d(x, y) is the depth value captured from the viewing position which is stored in the depth map, α indicates the global subsurface scattering level in the image, β controls the subsurface scattering which varies with the depth gradient, and ∇ means the object surface flatness. We can control α value to change the appearance of subsurface scattering. Figure 3.5 demonstrates different global subsurface scattering values.

Figure 3.5: The influence of different global subsurface scattering value.β is fixed by 800 and varying global subsurface scattering values are 40, 28 and 12, respectively.

After determining the blurry kernels’ widths, it blurs each pixel with 7-taps and multiplies each taps to specific weight. These weights are 0.006, 0.061, 0.242, 0.382, 0.242, 0.061, and

(33)

0.006 [7]. We fetch the texel and its 3 neighbors and multiplies these weight, respectively. Blurring operation can be separated to horizontal blurring and vertical blurring, and each blurred result can be directly accumulated by alpha blending. This process can be finished by multiple render targets (MRT). Figure 3.6 shows blurred operation.

RT1 Horizontal Blur Vertical Blur

RT2

RTF

Figure 3.6: blurred operation.

In the above figure, RT1 means render target 1, RT2 means render target2 and RTF means final render target. RT1 is applied horizontal Guassian blurring and store blurred result in RT2. Then, RT2 is applied vertical Guassian blurring and outputs the result into both RT1 and RTF. For RTF, it uses alpha blending to accumulate each blurred values and the blend weights are listed in table 3.1.

Table 3.1: blend weight

Skin

W

i

W

0 i

Variance

R

G

B

R

G

B

0.0064

0.2405

0.4474

0.6157

1.0

1.0 0.0516

0.1158

0.3661

0.3439

0.3251

0.45 0.3583

0.2719

0.1836

0.1864

0.0

0.34 0.1864

0.0 2.0062

0.46

0.0 0.0402

0.46

0.0 0.0402

Jimenez et al. use least square to fit original six Gaussian weights d’Eon et al. [7] into four Gaussian weights Wi. For producing normalized result, Jimenez et al. [14] normalizes these

(34)

value by equation 3.3, and these normalized weights are used in blending operation. Figure 3.7 plots six Gaussian weights and four Gaussian weights. Equation 3.3 generates a normalized weight at each step i of the previous sum, e.g. if we want to compute R channel’s normalized weight in step 2, we use equation 3.3 and 0.1158/ (0.2405+0.115) = 0.3251. W0

i is a blend

weight and we can use equation 3.4 to compute the blend result. After blurred pass finishes, we can obtain the appearance of skin. In subsection 3.4, we describe how to use screen-space method to our simulation system.

W_i0 = _P_iWi

j=1Wi

(3.3)

Figure 3.7: Fitting result by six Gaussian weights and four Gaussian weights.[14]

Colorout = Colorsrc∗ W

0

i + Colordst∗ (1 − W

0

(35)

3.3 The makeup effect of each layer and layered composition

Makeup is a multi-layered structure. It combines many color layers and colorless layers, e.g. Color layers include foundation, blush, eye-shadow and lipstick; colorless layers include toning water and Moisturizing gel. We focus on color layers and these color layers are simulated by the Kubelka-Munk theory which is famous for paints industry.

3.3.1 Kubelka-Munk theory

The Kubelka-Munk model can be derived from radiation transfer equation. The Kubelka-Munk theory uses the scattering coefficient per unit length and the absorption coefficient per unit length to describe the properties of the whole medium; radiation transfer equation use some descriptive coefficients to represent the individual behavior of each particle, such as the absorp-tion coefficient, the scattering coefficient and the scattering phase funcabsorp-tion. There are much research show the relationship between the coefficients of Kubelka-Munk and the coefficients of radiation transfer equation, such as Thennadil [15].

Schuster is the first attempt to use isotropic scattering to find the simplified equation of radiation transfer equation. In this process, Schuster assumes each layers are parallel and each scattering is isotropic, and simplifies radiation field into two opposite radiation fluxes I and J in the x and -x direction, respectively. In order to eliminate the boundary effect, Schuster assumes yz-plane is greater than thickness of layer. Figure 3.8 gives a simple view of these assumptions.

(36)

I X 0 dx d J

Figure 3.8: The simple view of Schuster’s assumption

Kubelka-Munk extends the equation of Schuster and adds some assumptions to simplify complex in-scattering transmission and remission: each scattering is isotropic scattering; the particles are random distribution in the medium; the layer is subject only to diffuse irradiation. By the above conditions, Kubelka and Munk pointed out simplified the radiation transfer equa-tion can be used to calculate the reflectance and transmittance of each layer. Gustav Kortum [16] gives more detail about the Kubelka-Munk theory in his book.

Because Kubelka-Munk assume each scattering is isotropic, we need to measure weather the reflectance of cosmetic from different observations are same or similar spectra. We select a liquid-foundation and measure same place from different observation (In this case, we rotate our sample around 0, π/2, π and 2π. Fig 3.9 gives the chromaticity coordinate and we can observe that the chromaticity from different observation places have similar coordinate and fig 3.10 shows measuring reflectance. Each reflectance has similar shape and the radiant energy are very closer. From the above observation, if the cosmetics do not have view-dependent effect, we almost can assume that the liquid-foundation, blush and lipstick are isotropic.

(37)

Figure 3.9: The chromaticity coordinate of liquid-foundation from four different observation

(38)

Reflectance and transmittance

The Kubelka-Munk model mainly uses the scattering coefficient S, the absorption coefficient K and the certain thickness to compute reflectance and transmittance. Equation 3.5 shows the reflectance equation. R = 1− Rg(a− bcoth(bSX)) a_{− R}g + bcoth(bSX) (3.5) where a = 1 + K_S b = (a2_{− 1)}12 X = thickness of layer Rg= reflectance of the backgraound

K = Absorption coefficient per unit length S = Scattering coefficient per unit length

The scattering coefficient S and the absorption coefficient K are wavelength-dependent and defined in per unit length rather than free mean path. When the material is sufficiently thick, we can ignore the reflectance of the background and equation 3.5 can be simplified as:

R∞= 1 + K S − " K S 2 + 2 K S #1 2 (3.6) where K

S is called constant ratio and defined in equation 3.7.

K S = (1− R∞) 2 2R∞ (3.7)

Hence, we can measure the reflectance of thickness layer and use these spectrums to simu-late the reflectance of thick layer directly.

In our simulation system, each cosmetic layer has a specified thickness value, and we can use equation 3.8 and 3.9 to compute its reflectance and transmittance, respectively.

(39)

R0 =

sinh bSX

a sinh bSX + b cosh bSX (3.8)

T = b

a sinh bSX + b cosh bSX (3.9)

In the equation 3.8, R0 means the reflectance which is not affected by background. We

usually use R0 and T to compute the reflectance and the transmittance of each layer rather than

using R and T.

Multilayered model

Another part of the Kubelka-Munk theory is multi-layered model. By using multi-layered model, we can combine each layer and obtain the final reflectance. Figure 3.11 gives the path of the diffusion radiation.

Layer 1

Layer 2

R1 T1 T1T2 T1R2 T1R2TI T1R2RI T1R2RIT2 T1R2R2RITI

Figure 3.11: Reflectance and transmittance of two non-homogeneous layers.

The above figure can be translated to two geometrical series:

T1,2 = T1T2 1 + RIR2+ R2IR 2 2 +· · · = T1T2 1_{− R}IR2 (3.10)

(40)

R1,2 = R1+ T1TIT2 1 + RIR2+ R2IR 2 2+· · · = R1+ T1TIT2 1_{− R}IR2 (3.11) and T1 = TI, R1,2 = R1+ T1T1T2 1_{− R}IR2 (3.12) In many applications, we assume each layer is homogeneous material. Hence, we can focus on color change and the equation 3.10 and equation 3.11 can be rewritten as:

T1,2 = T1T2 1_{− R}1R2 (3.13) R1,2 = R1+ T1T1T2 1_{− R}1R2 (3.14) In the multi-layered structure, we can use the above equation to solve more than two layered structure by recursive solving.

3.4 The integration of two techniques

In our system, we combine two different techniques to simulate the makeup effect on the human face. In this subsection, we describe rendering system in detail.

3.4.1 The skin pass

Before we explain what process in the skin pass, we need to figure out the interaction between human skin and makeup layers. Suppose we only apply the foundation to our face. In this situation, we have two layers structure, foundation and skin, and its schema is shown in Figure 3.12.

(41)

Foundation

Skin

Incident light

Penetration of light

Figure 3.12: the pass of light between foundation and skin.

From the above figure, we can see the light through the foundation and reach the skin sur-face. Hence, we need to consider this situation rather than apply the intensity of light to skin directly. In our system, we compute the total transmittance before the light hits surface and apply this value to the color of skin. In order to facilitate the process, we use the cosmetic map to mark the area of each makeup layer. Therefore, we can fetch the corresponding texel to check whether this area is covered by cosmetic on surface. Figure 3.13 shows the flowchart about the skin pass and figure 3.14 shows the cosmetic map.

(42)

Cosmetics cover on surface

Apply total transmittance to diffuse term Output the direct illumination

Yes

No

Blurred pass

(43)

Figure 3.14: the cosmetic map. The red color is used to mark the foundation layer.

In the skin pass, we use two equations to compute the direct illumination map and list as follow:

if the skin covers with the cosmetic:

outputcolor = lightcolor∗ shadow ∗ (diffuse + specular) ∗ transmittance

else

outputcolor = lightcolor∗ shadow ∗ (diffuse + specular)

The Kubelka-Munk theory compute the transmittance for each wavelength and we cannot use these value to compute the result. Hence, we first convert the spectra of the total transmit-tance to CIE RGB colors, and apply it to the diffuse color of human skin. Actually, we can use the approximate method to compute the transmittance which is introduced in chapter 4.2. Figure?? shows the total transmittance after the light through the foundation and the value of total transmittance is converted from spectra. Figure 3.15 shows the irradiance map which does not apply screen-space technique and figure 3.17 shows the irradiance map of screen-space after

(44)

applies this transmittance.

Figure 3.15: the total transmittance after the light goes through the foundation.

Figure 3.16: the irradiance map and applying the transmittance. (without screen-space subsur-face rendering)

(45)

After the skin pass, we can obtain the correct direct illumination map, and we blur this map many times to get the correct appearance of skin. Figure 3.17 shows the blurred result.

Figure 3.17: the irradiance map and applying the transmittance. (with SSSS)

3.4.2 The cosmetic pass

In the cosmetic pass, it computes the reflectance of each color and combines them to get the final diffuse result. Figure 3.18 shows the work flow about this pass.

(46)

Compute the reflectance and transmittance to each cosmetic layer

Combine each layer by the multi-layer model

Figure 3.18: the work flow about the cosmetic pass.

We compute the reflectance and the transmittance of each layer by equation 3.13 and equa-tion 3.14, and each result are shown in Figure 3.17.

(47)

(a) the rendering result (b) the reflectance of foundation

Figure 3.19: rendering result.

3.4.3 The computation of specular term

Surface observation

Specular term is the most important term to describe the highlight effect. In our observations, if we apply the cosmetics to the skin, it gives the smooth appearance. Figure 3.20 shows the original skin surface and the surface which applied the foundation. We obtain this information by Skin Visiometer (SV 600).

(48)

(a) the original skin surface (b) the skin surface with foundation

Figure 3.20: enlarging of skin surface.

We can observe that many cutin and fine line on the original surface. After we apply the foundation, the fine lines of the surface are filled with the foundation. Hence, we can obtain the smooth appearance. We can observe that the makeup changes the microfect of skin and let surface more smoothness.

Computing the specular term

In our system, we use the Kubelka-Munk theory to compute reflectance, transmittance and compose each layer together. But, the Kubelka-Munk theory does not consider specular term. Dorsey and Hanrahan [17] propose a simple way to solve this problem. For each layer, they use the Kubelka-Munk theory to compute the reflectance and transmittance and add specular effect by a simple model. Therefore, we use a similar way to simulate the specular of each layer. Figure 3.21 shows the result of specular effect and we render this result with foundation. The left result is only compute the specular term on skin; the medium result is compute the specular term on foundation; the right result compute the specular term for each layer.

(49)

Figure 3.21: the rendering result of specular effect. Left: only applying specular effect on skin surface. Medium: only applying specular effect on foundation. Right: computing specular on each layer.

For each layer, we use the BRDF specular model introduced by Kelemen et al. [18] to compute the specular term, and this model needs some parameters to compute the equation 3.15, such as the normalized of light direction, the normalized of view direction and surface roughness. fr,spec−→L ,−→V = P−→ H −→_H ∗ F ( − → H −→L ) 2 1 +−→_L−→V (3.15)

where F (−→H −→L )is the Fresnel term and P−→ H

−→_H

is Beckmann distribution function. For the Fresnel term, we usually use Schlick’s approximation to simulate the non-metal materials. The Beckmann distribution function is defined as:

P−→ H ,Beckmann −→_H = _m2_cos1 4_α ∗ e −tan2 α m2

where m is the roughness and this parameter affects the smoothness of microfect, e.g. the bigger roughness gives broader specular area and smaller value of highlight; the smaller value gives small area and bigger value of highlight. In the specular computation, because we do not measure the roughness and index of refraction accurately, we simply adjust the roughness to

(50)

control the highlight and set a constant value as index of refraction. We believe that if we can obtain these parameters of each layer, it can render more realistic.

3.5 Implementation

In our simulation system, we do not need to compute the reflectance, the transmittance and the combination of each layer in every frames. In the simulation, both the scattering coefficient S and the absorption coefficient K are fixed. Hence, both the reflectance and the transmittance are changed when the thickness is modified. We add an extra pass to compute the reflectance and the transmittance. So, when the thickness is modified, we recomputed the reflectance and the transmittance by this pass. The computing results are stored in the reflectance map and the transmittance map, respectively. In the rendering stage, we can fetch the color from these maps, and apply to each pass directly. Figure 3.22 shows the flowchart with our simulation system. By using this architecture, we can render the final result in real time and change the thickness in interactive rate.

(51)

Generate reflectance, transmittance and combine each layer Is thickness changed?

Yes

Reflectance map

dƌĂŶƐŵŝƚƚĂŶĐĞ map

The skin pass

The cosmetic pass

No

Final result

Input

(52)

4

Measurement

Our rendering system uses the two-constant Kubelka-Munk theory to simulate makeup appear-ance. In order to obtain appropriate scattering coefficient S and absorption coefficient K, we use spectroradiometer to measure reflectance for each wavelength and other approximation to obtain approximate coefficients.

4.1 Measuring data by spectroradiometer

The two-constant Kubelka-Munk theory gives Equation 4.1 and 4.2 that shows how to evaluate scattering coefficient S and absorption coefficient K, respectively. Both scattering coefficient S and absorption coefficient K are wavelength-dependent.

S = 1 bD coth−1 a− R b − coth −1 a− Rg b (4.1) K = S(a_{− 1)} (4.2) where a = 1 2[ 1 R_∞ + R∞]and b = (a 2_{− 1)}1₂ 39

(53)

In the above equation, Ri means the reflectance of the thick layer, R means the reflectance

of two layers (the thin layer painted on background), and Rg means the reflectance of the

back-ground. Solving both scattering coefficient S and absorption coefficient K needs corresponding Ri, R and Rg values to compute coefficient for each wavelength.

Our measurement uses spectroradiometer (MINOLTA CS-2000) to measure liquid founda-tions, cream blushes, cream eye-shadows and lipsticks. We use D55 light source and measure wavelength between 380 nm and 780 nm with the interval of 5nm. Figure 4.1 shows our device and environment.

Figure 4.1: Our capture device and environment.

Our spectroradiometer device gives the radiant energy which is the convolution radiant en-ergy of light with reflectance of material. Radiant enen-ergy is defined by equation 4.3.

(54)

Em(λ) = EL(λ)∗ R (λ)

R (λ) = Em(λ) EL(λ)

(4.3)

Em(λ) is radiant energy which means the amount of the energy of light hit the surface and

reflected to observer. R(λ) is the reflectance of material and EL(λ)is radiant energy of light

source. Figure 4.2 shows the radiant energy of liquid foundation. We graph two useful light source’s radiant energy D55 and E in Figure 4.3 and Figure 4.4, respectively.

0.00E+00 2.00E-03 4.00E-03 6.00E-03 8.00E-03 1.00E-02 1.20E-02 1.40E-02 1.60E-02 1.80E-02 380 430 480 530 580 630 680 730 780

Figure 4.2: Radiant energy of liquid foundation.

(55)

Figure 4.4: Radiant energy of CIE standard illuminant E.

We can use equation 4.3 to convert energy to reflectance. We use white material to measure radiant energy of light source EL by using purity of 99% barium sulfate, and use equation 4.3

to compute the reflectance of material. After this process, we can obtain the reflectance of each material R(λ). Figure 4.1 shows the reflectance of barium sulfate and barium sulfate sample and Figure 4.6 shows the reflectance of liquid foundation.

(56)

0.00E+00 5.00E-03 1.00E-02 1.50E-02 2.00E-02 2.50E-02 380 430 480 530 580 630 680 730 780

(a) Radiant energy of barium sulfate

(b) The sample of 99% barium sulfate

(57)

0.00E+00 2.00E-01 4.00E-01 6.00E-01 8.00E-01 1.00E+00 1.20E+00 380 430 480 530 580 630 680 730 780

Figure 4.6: Reflectance of liquid foundation.

In our experiment, we paint sample with enough thickness to make it opaque, and we mea-sure its reflectance to represent Ri. Then, we paint the sample to black background with certain

thickness to measure R. Finally, we measure the reflectance of background to determine Rg.

Figure 4.7 shows liquid foundation sample. We also make a simple container to load samples and figure 4.8 shows the container. By using this container, we can easily make the flat sample and easily measure the thickness of sample. Figure 4.9 shows the reflectance of Ri,R and Rg.

Figure 4.7: Liquid foundation sample. Left: background. Mid: thickness layer. Right: thin layer.

(58)

(a) container. (b) Loaded with liquid foundation.

Figure 4.8: The sample of our simple container.

0 0.2 0.4 0.6 0.8 1 1.2 380 430 480 530 580 630 680 730 780 Rg Ri R

Figure 4.9: The reflectance of Ri, R and Rg.

We apply Ri, R and Rg to the equation 4.1 and equation 4.2 for obtaining the scattering

coefficient S and the absorption coefficient K, respectively. Figure 4.10 shows S and K for each wavelength.Each wavelength of coefficient S and coefficient K can be used in equation 4.4 to simulate the original reflectance spectral of thin layer R0. Furthermore, we can use different

(59)

the fitting result. 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 380 430 480 530 580 630 680 730 780 S K

Figure 4.10: The coefficient S and K for each wavelength.

R0 = sinh bSX a sinh bSX + b cosh bSX (4.4) 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 380 430 480 530 580 630 680 730 780 R R0

Figure 4.11: The original reflectance and the fitting result.

(60)

fitting curve represents the reflectance of thin layer, and the measuring curve represents the reflectance of two layer, thin layer and background. These differences are not noticeable, if we covert these curve to RGB color space. Figure 4.12 shows the RGB color result of fitting curve and measurement curve. In figure 4.13, we apply D55 to RGB. (Appendix A provides the method for converting spectrum to CIE RGB color space.)

Figure 4.12: CIERGB color of fitting curve and measured curve. Left: Original data (R0) and the color is (0.62, 0.58, 0.51). Right: Fitting result and the color is (0.60, 0.57, 0.5).

Figure 4.13: CIERGB color of fitting curve and measured curve. Left: Original data (R0), color (0.62, 0.48, 0.38). Right: Fitting result, color (0.60, 0.47, 0.38).

4.2 Approximation way to obtain Kubelka-Munk coefficients

Cassidy et al. [19] provide a simple inversion of the KM equation shown in equation 4.5 and equation 4.6. S = 1 bX cosh −1 b2 _{− (a − R} W) (a− 1) b (1_{− R}W) (4.5) K = S(a_{− 1)} (4.6) where a = 1 2 RW + RB−R_R_bW+1 , b = (a2_{− 1)}12 , and 0 < R

B < RW < 1 for all channel. In

(61)

the color of thin layer painted on white background. By using equation 4.5 and 4.6, we can select two colors RB and RW and then we can obtain scattering coefficient S and absorption

coefficient K, respectively. Figure 4.14 shows a diagram about this process.

Rw Rb

Select two colors

Compute coefficient

by above equation _{Output S and K}

Rendering result

Figure 4.14: Selecting two colors and obtaining S and K.

In above case, S is (0.78, 0.39, 0.18) and K is (0.02, 0.10, 0.16).

In order to pick RB and RW from real material, we build a simple environment to obtain

these colors. We use polarized light source and polarized lens to obtain diffuse color. Figure 12 shows our device and figure 13 shows a liquid foundation sample.

Sample Polarized light

(62)

Figure 4.16: liquid foundation sample.

After capturing, we can pick color directly and use equation 4.5 and equation 4.6 to compute coefficient S and coefficient K. Because the light is polarized, we need multiply color by 2 to obtain approximate diffuse color. Figure 4.17 shows the diffuse color captured by this simple method. After estimating coefficient S and coefficient K, we can use these values to describe reflectance and transmittance in our simulation system.

(63)

5

Result

We have implemented the makeup simulation in DirectX 10 and HLSL, and in this section we present our rendering result and our simulation data. We render the makeup effect on two human head, Digital Coco1 _{and a 3D head scan of a man}2_{. Figure 5.1 shows the original appearance}

of skin rendering. All rendering results have been done on an Intel Core i5 750 2.67GHz CPU, 4GB RAM, and an NVIDIA GTX 260 graphics card. The resolution of each image is 1280 x 720. The frame rate of rendering is 170-175fps with makeup effect.

2_{INFINITE-REALITIES r- http://www.ir-ltd.net/infinite-3d-head-scan-released}

(64)

Figure 5.1: original appearance of skin rendering. (Left: Digital Coco. Right: Digital Orange.)

We divide this chapter into two parts - the measuring data of cosmetic and the makeup rendering. First, we show our measuring data and fitting data of each cosmetic, and then we show each rendering result individually. Then, we choose some cosmetics to compose different makeup result. We also provide some cosmetic map to show different makeup Patten.

5.1 The measuring data of cosmetic

We collect several cosmetics and measured their parameters. Our subjects include liquid foun-dations, cream blushes and lipsticks.

(65)

Liquid foundation

We first show the liquid foundation. We measure five subjects. Table 5.1 shows the brand of each liquid foundation.

Table 5.1: The brand of liquid foundation.

Sample F1 Sample F2 Sample F3 Sample F4 Sample F5

Brand

Maybelline

(No 22.

Natural

Beige)

Orbis

(N03)

Integrate

(00)

BOURJOIS

Healthy

(#51)

BOURJOIS

Healthy

(#52)

The following figures show the measuring data of each sample, the fitting data of each sample, the comparing of R and R0 of each sample and the rendering result with different thicknesses as 0.1, 0.5 and 1.0. We can observe the detail of skin if the thickness is smaller.

There are some measuring mistakes which causes the fitting data error, such as Sample F2. In general, the reflectance spectrum increases when the layer becomes thicker, however the reflectance between 380-500 nm does not follow this rule. There are some reasons causes this effect. For example, the sample is not flat or the thickness is not thin enough. In our experience, the problem is usually caused by the first reason.

In our analysis, we found a phenomenon. If the subject is more opaque, the reflectance of R and R0 are similar. Hence, the reflectance of R and R0of each lipstick are similar.

(66)

0 0.2 0.4 0.6 0.8 1 1.2 380 430 480 530 580 630 680 730 780 Rg R Ri

Figure 5.2: the measuring data of sample F1.

0 0.5 1 1.5 2 2.5 380 430 480 530 580 630 680 730 780 S K R0

(67)

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 380 430 480 530 580 630 680 730 780 R R0

(68)

Figure 5.5: the rendering result of sample F1. Left: the thickness is 0.1. Mid: the thickness is 0.5. Right: the thickness is 1.0.(Each model uses different cosmetic map.)

(69)

0 0.1 0.2 0.3 0.4 0.5 0.6 380 430 480 530 580 630 680 730 780 Rg R Ri

0 0.5 1 1.5 2 2.5 380 430 480 530 580 630 680 730 780 S K R0

(70)

0 0.05 0.1 0.15 0.2 0.25 0.3 380 430 480 530 580 630 680 730 780 R R0

(71)

(72)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 380 430 480 530 580 630 680 730 780 Rg R Ri

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 380 430 480 530 580 630 680 730 780 S K R0

(73)

0 0.05 0.1 0.15 0.2 0.25 0.3 380 430 480 530 580 630 680 730 780 R R0

(74)

(75)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 380 430 480 530 580 630 680 730 780 Rg R Ri

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 380 430 480 530 580 630 680 730 780 S K R0

(76)

0 0.05 0.1 0.15 0.2 0.25 0.3 380 430 480 530 580 630 680 730 780 R R0

(77)

(78)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 380 430 480 530 580 630 680 730 780 Rg R Ri

0 0.2 0.4 0.6 0.8 1 1.2 380 430 480 530 580 630 680 730 780 S K R0

(79)

0 0.05 0.1 0.15 0.2 0.25 0.3 380 430 480 530 580 630 680 730 780 R R0

(80)

Cream blush

We measured two cream blushes and table 5.2 shows the brand of each cream blush. Because two blushes are too similar in the same thickness 1.0, we demonstrate two thicknesses of each blush, 1.0 and 2.0.

(81)

Table 5.2: The brand of each cream blush.

Sample B1

Sample B2

Brand

MISSHA (No. 1 – Left)

MISSHA (No. 2 – Right)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 380 430 480 530 580 630 680 730 780 Rg R Ri

(82)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 380 430 480 530 580 630 680 730 780 S K R0

Figure 5.23: the fitting data of sample B1.

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 380 430 480 530 580 630 680 730 780 R R0

以物理為基礎的彩妝顯像方法

國

立

交

通

大

學

多媒體工程研究所

碩

士

論

文

以 物 理 為 基 礎 的 彩 妝 顯 像 方 法

A Physically-Based Cosmetic Rendering

研 究 生：黃程國

指導教授：莊榮宏 教授

林文杰 教授

研 究 生：黃程國 Student：Cheng-Guo Huang

指導教授：莊榮宏 Advisor：Jung-Hong Chuang

林文杰 Wen-Chieh Lin

國 立 交 通 大 學

多 媒 體 工 程 研 究 所

碩 士 論 文

中華民國一百年十月

以物理為基礎的彩妝顯像方法

研究生 : 黃程國 指導教授 : 莊榮宏 博士

林文杰 博士

國立交通大學

資訊學院多媒體工程研究所

摘 要

Student:

Cheng-Guo Huang

Advisor:

Dr. Jung-Hong Chuang

Dr. Wen-Chieh Lin

Institute of Multimedia Engineering

College of Computer Science

National Chiao Tung University

ABSTRACT

Contents

List of Figures

List of Tables

1

Introduction

1.1 Contribution

1.2 Organization of the thesis

2

Related Work

2.1 The cosmetic rendering

Cosmetic transfer

Cosmetic simulation

2.2 Translucent rendering

3

Measuring the BSSRDF

3

Approach

3.1 Framework

Skin rendering pass

Cosmetic rendering pass

3.2 The method of human skin rendering

3.2.1 Human skin rendering

Skin

W

W

Variance

R

G

B

R

G

B

0.0064

0.2405

0.4474

0.6157

1.0

1.0

1.0

0.0516

0.1158

以物理為基礎的彩妝顯像方法

研究生：黃程國

指導教授：莊榮宏教授

林文杰教授

研究生：黃程國 Student：Cheng-Guo Huang

國立交通大學

多媒體工程研究所

碩士論文

研究生 : 黃程國指導教授 : 莊榮宏博士

林文杰博士

摘要