Automatic Skin Color Beautification

(1)

F. Huang and R.-C. Wang (Eds.): ArtsIT 2009, LNICST 30, pp. 157–164, 2010.

Automatic Skin Color Beautification

Chih-Wei Chen, Da-Yuan Huang, and Chiou-Shann Fuh

Department of Computer Science and Information Engineering National Taiwan University, Taipei, Taiwan

{d95013,r97022,fuh}@csie.ntu.edu.tw

Abstract. In this paper, we propose an automatic skin beautification framework based on color-temperature-insensitive skin-color detection. To polish selected skin region, we apply bilateral filter to smooth the facial flaw. Last, we use Poisson image cloning to integrate the beautified parts into the original input.

Experimental results show that the proposed method can be applied in varied light source environment. In addition, this method can naturally beautify the portrait skin.

Keywords: Skin beautification, color-temperature-insensitive, bilateral filter, Poisson image cloning.

1 Introduction

Recently, digital still camera becomes more widespread. To provide more capability for users and satisfy their different requirements, many techniques have been inte- grated into digital still camera.

Skin beautification is the technique to smooth the skin of the photographed portrait. The result of skin beautification is preferable to the original one. Many consumer cameras have embedded the function of skin beautification. Besides, there are many approaches proposed to deal with skin beautification. However, most of these techniques rely on accurate skin color detection.

In this paper, we propose an original framework based on color-temperature- in- sensitive skin-color detection. Then, we use the characteristic of bilateral filter, which can preserve the significant image edge and can smooth the low gradient parts in given image, to polish the selected skin regions. Finally, to satisfy human perception, we apply Poisson seamless cloning algorithm to integrate the polished part into original image.

2 Backgrounds

2.1 Skin Color Detection

There are many skin color detection algorithms proposed to extract the skin parts from given image [1] [2]. Most of these approaches use statistical method to find out

(2)

Fig. 1. The experimental data of color patches under different light sources [3]

the boundary of human skin color in specific color space. However, skin color is sensitive to color temperature. The relation between color and color temperature is shown in Fig. 1.

To lower the effect of color temperature, the better for us to use face detection algorithm as preprocessing step. Then, we can determine the characteristic of skin color from the abovementioned face detection.

2.2 Face Detection

To extract human face from given image, many algorithms have been proposed. Viola et al. proposed an efficient face detection approach [4]. There are three main princi- ples in their algorithm, including the integral image, Ada boost, and cascade classifier. Furthermore, Lienhart et al. proposed Haar-like features for object detection [5].

In this paper, we apply Lienhart’s frontal face classifier [6] and Hameed’s eye classifier [7] to extract human face and human eyes from given image respectively. The objective of human eye detection is to subtract the effect of non-skin tone element from detected human face. The result of human face detection and human eye detection is shown in Fig. 2(b). The white color in Fig. 2(c) is the region of interest, which is a rough skin-color data.

(3)

Automatic Skin Color Beautification 159

(a) (b) (c)

Fig. 2. (a) Input image. (b) Red circle and blue circle mean face region and eyes region respec- tively. (c) White region represents interested skin color.

2.3 Bilateral Filter

The bilateral filter is a nonlinear technique that can smooth the input image while preserving its main edges [8] [9]. With a Gaussian function

the bilateral filter of image I at pixel p is defined by:

(1)

where w is a normalized factor; and are typical Gaussian kernel functions;

and p and q indicate spatial locations of pixels.

2.4 Seamless Cloning

The objective of Poisson seamless cloning is to find out a function, which makes the gradients of target image closest to the gradients of source image. We implement the discrete Poisson solver [10] to obtain finished result of skin beautification. In this paper, the guidance field vector v is

(2)

where means the gradient of source image.

2 2

( ) exp( / ) g x

_σ

= − x σ

( ) ( )

q

( ) 1 ,

s r

p p q q

I

BF I g p q g I I I

w ∈ ^σ ^σ

=

∑

− −

( ) ( )

^,

s r p q

q I

w g_σ p q g_σ I I

∈

=

∑

− −

g_σs g_σ_r

, v= ∇g

∇g

(4)

Fig. 3. System flowchart

We use down-sampling approach to lower the effect of facial flaw (e.g. scar, zit, spot, and so on). Besides, to downsize the input image can improve the efficiency of face detection. After face detection, the interested skin parts have been preserved, the sampling result is shown in Fig. 2(c). To achieve color-temperature-insensitive skin- color preservation, we translate the interested skin parts into YCbCr color space.

Then, we calculate each mean value and standard deviation of Y, Cb, and Cr respectively. The abovementioned procedure is shown in Fig. 4. To prevent the effect of extreme value, we skip 10% of the head and tail in Y, Cb, and Cr histogram.

The skin-color mask of image I at pixel pis defined by

(3)

where mean the YCbCr value of image I at pixel p respectively; represent the mean value of interested region in YCbCr color space respectively; mean the standard deviation value of interested region in YCbCr color space respectively.

1 , if 2 ( ) 2

and 2 ( ) 2

( )

and 2 ( ) 2

0 , otherwise,

Y Y p Y Y

Cb Cb p Cb Cb

p

Cr Cr p Cr Cr

Y I M I Cb I

Cr I

μ σ μ σ

− ≤ ≤ +

⎧⎪ − ≤ ≤ +

= ⎨⎪⎪ − ≤ ≤ +

⎪⎩

( _p), ( _p), and ( _p) Y I Cb I Cr I

, , and

Y Cb Cr

μ μ μ

Y, , Cb and Cr

σ σ σ

(5)

(a)

(b) (c) (d)

Fig. 4. (a) Region of interest. (b) The Y-channel of (a). (c) The Cr-channel of (a). (d) The Cb- channel of (a).

The abovementioned mask is generated in downsized form due to the down- sampled input image. To accomplish the skin-color mask generation, we apply bilin- ear interpolation to adjust the size of the downsized form to equal to the size of input image.

Concurrently, we apply bilateral filter to smooth the original input image. The result of bilateral filter is called base image. The result of bilateral filter is similar to the output of low-pass filter. Moreover, bilateral filter can preserve the image edge simul- taneously.

Finally, we integrate the input image, mask of skin-color, and the base image ob- tained from bilateral filter algorithm into result image. In this paper, we apply Poisson image cloning as image integration algorithm. The advantage of Poisson image cloning is to integrate the given image into harmonious image, even if there are few mis- detections or false-alarms in the skin-color mask.

4 Experimental Result

To prevent the influence of color temperature, we use flexible method to determine the skin-color boundary. The performance of our skin-color detection is stable in various color temperatures. Table 1 shows the results of the proposed method. The size of input image is 349 × 419, and the and the in bilateral filter is set to 60 and 0.1 respectively.

σs σr

(6)

Low Normal

Color Temperature High

According to human perception, we can generate a touching portrait by smoothing the blemish (e.g. facial wrinkle, sweat pore, rash, and so on) of human face. Besides, our approach can be applied to lower the image noise in portrait photography. Table 2 shows the result of skin beautification in normal color temperature.

(7)

Table 2. The results of skin beautification in normal color temperature

Input Image Mask of Skin Color Result

(8)

Taiwan, under Grants NSC 94-2213-E-002-032 and NSC 95-2221-E-002-276-MY3, by EeRise Corporation, EeVision Corporation, Primax Electronics, Altek, Alpha Imaging, Utechzone, TRI, Genesys Logic, Lite-on, and Syntek Semiconductor.

Reference

1. Tomaz, F., Candeias, T., Shahbazkia, H.: Fast and accurate skin segmentation in color images. In: Proceedings of First Canadian Conference on Computer and Robot Vision, 2004, pp. 180–187 (2004)

2. Kukharev, G., Nowosielski, A.: Visitor Identification - Elaborating Real Time Face Rec- ognition System. Short Communication Papers Proceedings of WSCG 2004, pp. 157–164 (2004)

3. Yung-Cheng, L., Wen-Hsin, C., Ye-Quang, C.: Automatic white balance for digital still camera. IEEE Transactions on Consumer Electronics 41, 460–466 (1995)

4. Viola, P., Jones, M.J.: Robust Real-Time Face Detection. Int. J. Comput. Vision 57, 137–

154 (2004)

5. Lienhart, R., Maydt, J.: An extended set of Haar-like features for rapid object detection. In:

Proceedings of 2002 International Conference on Image Processing, vol. 901, pp. I-900–I- 903 (2002)

6. Lienhart, R.: Stump-based 20x20 gentle adaboost frontal face detector, http://www.lienhart.de/

7. Hameed, S.: Stump-based 20x20 frontal eye detector, http://umich.edu/~shameem

8. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Sixth Interna- tional Conference on Computer Vision, 1998, pp. 839–846 (1998)

9. Paris, S., Kornprobst, P., Tumblin, J., Durand, F.: A gentle introduction to bilateral filtering and its applications. In: ACM SIGGRAPH 2008 classes. ACM, Los Angeles (2008) 10. P’erez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22, 313–

318 (2003)