Chap 3 Proposed methods
3.2 Preferred skin color experiment
3.2.1 Experimental settings
Figure 3-3 shows our experimental settings. A 37-inch LCD panel is used for preferred skin color observation. The distance between a user and the panel is about 60 to 80 cm, and the normal office fluorescent light is used for environmental illumination. Besides, we limit the experiment time of every user to be less than one minute to avoid experimental errors cause by asthenopia. Seventeen students in the Electronics Engineering Department of National Chiao Tung University participated in this experiment.
60~80cm 60~80cm
Figure 3-3 Preferred skin color experimental settings
All the test images are captured by a Konica Minolta DiMAGE X50 camera. Figure 3-4 shows the selection of test patterns. This image set is used to test whether the environmental illumination, gender, and human race may have influence over the preferred skin color. In the test patterns, there are two genders, female and male; and three kinds of environmental illumination, outdoors, indoors with flashlight, and indoors without flashlight.
(a)Caucasian female (b)Caucasian female (
(c)Caucasian female (outdoor) indoor, without flash) (indoor, with flash)
(d)Caucasian male (outdoor)
(e)Caucasian male (indoor, without flash)
(f)Caucasian male (indoor, with flash)
(g) Asian female (outdoor) (h)Asian female (indoor, without flash)
(i)Asian female (indoor, with flash)
(j)Asian male (outdoor)
(k)Asian male (indoor, without flash)
(l)Asian male (indoor, with flash)
(g)Black female (outdoor) (h)Black female (indoor, without flash)
Figure 3-4 Test patterns
3 e
The Graphic User Interface Matlab. In Figure 3-5, we show this interface, which include.
(b) a selection icon to select human race, which can be Asian, Black, or Caucasian;
(c) a
d values of hue, saturation and value (intensity);
.2.2 Experimental m thods
program is written in (a) a region to display the test pattern;
n icon to select the test pattern;
(d) three bars to select preferre (e) a “RESET” button; and (f) a “SAVE” button.
(a)
(b) (d)
(c)
(e)
(b) (d)
(a)
(c)
(e)
Figure 3-5 Graphic User Interface
Since the HSV color space better fits the perceptual color of human eyes than most color spaces, users can easily select the preferred skin color image through the
adjustment of hue, saturatio ng background colors, only
the skin color will be adjusted. Hence, the skin color regions of each image had been manu
n and value. To avoid affecti ally segmented in advance, as shown in Figure 3-6.
(a) test image (b) manually segmented skin regions
Figure 3-6 Hand-labeled skin color mask
Further, to avoid these artifacts appearing at the discontinuous regions between skin color regions and neighboring non-skin color regions, a manually segmented mask is smoothed via a 5-by-5 averaging kernel to reduce the discontinuity. Figure 3-7 illustrates the smoothed mask.
25
Figure 3-7 Illustration of the ma
(a) Skin color mask (b) Blurred Mask
sk passed smoothed via a 5-by-5 averaging kernel
Take the outdoor Asian female image as an example. If a user prefers to increase the saturation and intensity values by 5, then the adjusted image looks like the image presented in Figure 3-8 (b).
(a) original image (b) users’ preferred image
Figure 3-8 Example of a user’s preferred skin color image
After all users have selected their preference, the skin color within a small region, as illustrated in Figure 3-9, will be analyzed among different users to find the differences with respect to environmental illuminations, gender, and human races.
Figure 3-9 Selected skin color region (red square)
3.2.3 Experimental results
Based on the proposed experiment, we can get some statistical results about preferred skin color. Figure 3-10 (b) and (c) show the preferred skin color distribution on the HS and SV color planes for the outdoor Asian female image. The red points represent the original skin color in the image. The blue points (0~16) represents preferred skin colors of the 16 users. The green mark represents the mean of all users’
preferences.
In Figure 3-10, it can be seen that more user (11 persons) prefer higher hue
value while fewer use he numbers of users
who prefer higher saturation values and lower saturation values are almost the same.
r intensity values while only 2 persons prefer lower intensity values. Based on this statistical results, we can assume that most peop
s
rs (6 persons) prefer lower hue values. T Most users (15 persons) prefer highe
le prefer 1~2 degrees of increase on the hue value, no changes on the saturation value, and 6~7 percent of increase on the intensity value.
(a) original image (b) hue-saturation distribution of preferred skin color
(c) saturation-intensity distribution of preferred skin color
Figure 3-10 Preferred skin color distribution of the outdoor Asian female image
Figure 3-11 shows the preferred skin color distribution on the HS and SV color planes for the indoor without flashlight Asian female image. In the results, most users (12 persons) prefer lower hue values while others prefer higher hue values. Most users (11 persons) prefer higher saturation values while others prefer lower saturation values.
All users prefer higher intensity values. Hence, it is assumed that, for the indoor without flashlight Asian female image, users prefer 2~3 degrees of decrease on the hue value, 1~2 percents of increase on the saturation value and 9~10 percents of increase on the intensity value.
(a) original image (b) hue-saturation distribution of preferred skin color
(c) saturation-intensity distribution of preferred skin color
Figure 3-11 Preferred skin color distribution of the indoor without flashlight image
Figure 3-12 shows the preferred skin color distribution on the HS and SV color planes for the outdoor Caucasian female image. Most users prefer higher hue, saturation and intensity values. Hence, it is assumed that, for the outdoor Caucasian female image, users prefer 2~3 degrees of increase on the hue value, 1~2 percents of increase on the saturation value and 4~5 percents of increase on the intensity value.
(a) original image (b) hue-saturation distribution of preferred skin color
(c) saturation-intensity distribution of preferred skin color
Figure 3-12 Preferred skin color distribution of the outdoor Caucasian female image
Figure 3-13 shows the preferred skin color distribution on the HS and SV color planes for the indoor with flashlight Caucasian female image. Most users prefer higher hue values while others prefer lower hue values. Almost every user prefers lower saturation values while a few prefer higher saturation values. All users prefer higher intensity values. Hence, it is assumed that, for the indoor with flashlight Caucasian female image, users prefer 1~2 degrees of increase on the hue value, 4~5 percents of decrease on the saturation value, and 6~7 percents increase on the intensity value.
(a) original image (b) hue-saturation distribution of preferred skin color
(c) saturation-intensity distribution of preferred skin color
Figure 3-13 Preferred skin color distribution of the indoor with flashlight Caucasian female image
Figure 3-14 shows the preferred skin color distribution on th HS and SV color planes for the outdoor Black female image. The numbers of users prefer higher hue/saturation values and lower hue/saturation values are almost the same. Most users (12 persons) prefer higher intensity values while a few prefer lower intensity values.
Based on the statistical analysis, the distribution of preferred skin color of the Black female image has a wide range. Hence, it is roughly assumed that, for the outdoor Black female image, users prefer 1~2 degrees of increase on the hue value, no changes on the saturation value, and 2~3 percents of increase on the intensity value.
However, the preferred skin color of this case is difficult to define.
(a) original image (b) hue-saturation distribution of preferred skin color
(c) saturation-intensity distribution of preferred skin color
Figure 3-14 Preferred skin color distribution of the outdoor Black female image.
Figure 3-15 shows the preferred skin color distribution on the HS and SV color planes for the indoor without flashlight Black female image. The numbers of users prefer higher hue values and lower hue values are almost the same. Most users prefer lower saturation values and higher intensity values. Hence, it is assumed that for the indoor without flashlight Black female image, users prefer no changes on the hue value, 2~3 percents of decrease on the saturation value, and 2~3 percents of increase on the intensity value.
(a) original image (b) hue-saturation distribution of preferred skin color
(c) saturation-intensity distribution of preferred skin color
Figure 3-15 Preferred skin color distribution of the indoor without flashlight Black female image
Table 3-1 shows the prefer skin color results of all test patterns. There are three marks in the first column. The first mark represents the human race, which can be Asian, Black, or Caucasian. The second mark represents the gender, which can be female or male. The third mark represents the environmental condition, which can be outdoor (O), indoor without flashlight (I) and indoor with flashlight (I). Take Asian M. IF. As an example, it represents the image of indoors with flashlight Asian male image.
Table 3-1 Preferred skin color results of all test patterns
H↓ H↑ S↓ S↑ V↓ V↑
Observing in Table 3-1 the hue and saturation values of the preferred skin color, there is no unanimous preference under all conditions, such as different human races, different genders, or different env
value of preferred skin color tends
e lor is darker. Hence, almost every user prefer increases the intensity value of conditions. However, if the image is captured under a high ination condition, some users prefer reducing the intensity value of skin color
ost users still prefer raising the intensity value.
ironmental illuminations. Nevertheless, the intensity to increase for every condition.
Furthermore, if the image is captured under a low illumination condition, th skin co
skin color at all kinds of illum
while m
Since there is no common direction of the preferred skin color adjustment among
all users, the mean of preferred skin color is measured, as listed in Table 3-2. In this table, the HSV colo
Table of skin col preferred skin co
an of or al skin Mean of preferred skin color r space is adopted.
3-2 Mean original or and lor
, but is la er than of Cau ian images. The preferred saturation n images maller t that of Caucasian rger th that ages. The preferred ensity o sian im s is sm er than of
, but arger th hat of Black image ence, w an assu at in colo f Asian ages is between that of Caucasian images and
ry xperiment
peri ntal results mentioned above, we reach the following onclusions.
that of Caucasian
in color under all conditions.
Moreover, higher intensity values require smaller adjustments while lower intensity values require larger adjustments. Hence, the adjustment of skin color intensity can be decided based on the intensity value of the original skin color.
other hand, for a specific user, if we want to know his/her skin color preference, we may still adopt this experiment to gather the statistical trends
pr hue f A ag all that of
Black images rg that cas
value of Asia is s han images, but is la an
¾ There is no unanimous preferred hue and saturation values for all users under all conditions. Hence, soothe preferred values are only defined based on the averaged values of all users’ preferences.
¾ The preferred skin color of Asian people is between
people and Black people. Hence, it is assumed that the preferred skin color of Caucasian people and Black people are two extreme cases. Any other preferred skin colors can be acquired through a weighted summation of these two extreme cases.
¾ Users prefer raising the intensity value of sk
As mentioned before, it is difficult to define the preferred skin color that everyone likes. Hence, in this thesis, the mean of all users’ preference is adopted to define the preferred skin color. On the
of his/her skin color preference.
3.3 ty-bas lassifi
Af preferred , the dete o be
class
who live there have darker skin color. Oppositely, people who live far
Intensi ed skin color c cation
ter the skin color is defined cted skin color has t ified. According to the classification results, the skin color can be reproduced. As mentioned before, the skin color is mainly caused by the hemoglobin and melanin. In the skin of different human races, the amount of hemoglobin is almost the same, but the amount of melanin is varying. There are two important factors that affect the amount of melanin: human race and sunlight. Figure 3-16 [17] illustrates the human skin color distribution of the whole world. Since the sunlight near the equator is stronger, people
from the equator have lighter skin color.
Figure 3-16 Human skin color distribution [17]
e, different human races may share similar skin colors if they are exposed to egrees of sunlight exposu
Henc different d 3-17, i under dar different c
re. This phenomenon can be illustrated in Figure wh ch was plotted by ChiCha. In this figure, three columns show the skin color k, middle and light illuminations, respectively. The nine rows represent ombinations of sunlight conditions and human races.
Dark Middle Light Dark Middle Light Dark Middle Light
Figure 3-17 Skin color of different human races under different sunlight conditions [18]
These 27 kinds of skin colors can be further plotted on the HS and SV color planes, as shown in Figure 3-18. We use three letters to represent each skin color.
The first letter represents sunlight condition, including pallor (P), midtone (M) and tanned (T). The second letter represents human race, including Asian (A), Black (B) and Caucasian (C). The third letter represents lightness condition, including dark (D), middle (M) and light (L).
In the HS color plane, it can be observed that the saturation of skin color is reduced when the environmental lightness increases. However, the perceptual results do not have any help on skin color classification. On the other hand, in the SV color plane, some perceptual results can be obtained. For example, when the environmental lightness increases, the skin color intensity is raised and its color looks brighter. Oppositely, when the environmental lightness decreases, the skin color looks darker. However, considering the sunlight conditions and human
races, the skin co nd offers no referable
information.
lor distribution has a very wide range a
(a) H-S space
(b) S-V space
Figure 3-18 The distribution of different skin color on HS and SV color planes.
Acco
ions in the HSV color space.
¾ The lightness of skin color in an image has more influence on users’ adjustment on the intensity of preferred skin color than human races.
¾ The skin color of different human races under different sunlight conditions may be similar. Environmental illumination has great influence on skin color classification.
It is obvious that environmental illumination is a great influencing factor of skin color classification. Furthermore, the users’ adjustment on preferred skin color intensity is based on the lightness of the original skin color. Therefore, the image intensity, instead of human race, is adopted in our approach for skin color classification. As shown in Figure 3 bership function can be utilized to define the weights of each skin color, including dark skin, midtone skin and light skin.
Through this classification, the adjustment of skin color can be easily decided.
rding to the above results, the skin color characteristics of different human cannot be clearly defined. Hence, it’s difficult to classify the detected skin color ly based on color information. Moreover, according to the skin color
cteristics and the results of preferred skin color experiment, there are some lusions.
The skin color of human race in an image has more influence on users’
adjustment on the hue and saturation values of preferred skin color. From the experimental results, users’ preferred skin color of different human races has occupied different reg
Figure 3-19 Fuzzy membership function of skin color classification
Figure 3-20 shows an example of skin color classification. Blue color represents the light skin color while the green colo midtone skin color. Based on this result, the preferred skin color under dif
r represents the
ferent lightness can be defined.
(a) original image (b) result after skin color classification
Figure 3-20 Example of skin color classification
3.4
in color adjustment based on
detec skin color adjustment.
detec
(b). Therefore, artifacts may appear at the discontinuous regions between skin color regions and neighboring non-skin color regions, as indicated within the red square of Figu
Similarity-based skin color adjustment
This section will introduce the similarity-based sk
the results of skin color detection and classification. As mentioned before, the skin tion of neural network is fuzzy logical and it is helpful on
Since there are only two results, skin color and non-skin color, for most skin color tion methods, the detected regions are often incomplete, as shown in Figure 3-21
re 3-21 (c).
(a) original image (b) detected results (c) correction artifacts
Figure 3-21 Skin color detection results and skin color adjustment artifacts
To diminish this kind of artifacts, the distance between the original skin color
and the preferred sk ent. If the
r degree of skin color adjustment can thus be varied based on the skin color probability. With such a neural network, artifacts can be greatly reduced.
in color is considered to decide the degree of adjustm
neural network is adopted for skin color detection, the detected results at colo discontinuous regions will be fuzzy logical, as shown in Figure 3-22 (b), and can offer the information about the probability of being a skin color. Hence, the
(a) original image (b) result of skin color detection
(c) reproduced image
Figure 3-22 Skin color detection results through fuzzy logic neural network
3.5 System Overview
Figure 3-23 shows the preferred skin color reproduction system. First, a neural network is utilized to detect skin color, and the skin color probability distribution of an image is retrieved. Second, based on our proposed fuzzy membership function, the skin color will be classified based on the intensity value. Third, according to the skin color classification results and the user-defined preferred skin color, the degree of adjustment can be decided. At last, based on the skin color probability model and adjustment degree, skin color will be reproduced.
NN-based
Figu eferred skin color reproducti tem
Chap 4 Experimental Results
Through our proposed method explained in Chapter 3, the skin color regions in an image can be correctly detected. The colors in these regions can be reproduced according to user’s preference. The preferred skin color reproduction for different human races will be shown in the following sections.
Figure 4-1, Figure 4-2 and Figure 4-3 present the reproduction results of Caucasians.
As shown in Figure 4-1 (b), the eyes, lips and necklaces can be eliminated from the detection results. From the skin color classification result as shown in Figure 4-1 (c), it is obvious that skin color of most pixels belong to midtone skin color. Figure 4-1 (d) shows the adjusted skin color. Since the saturation of original image is a bit higher than users’ preference, the saturation value will be reduced. At the same time, the intensity of the image is also increased.
(a) original image (b) result of skin color detection
(c) result after skin color classification (d) reproduced image
Figure 4-1 Experimental results of Caucasian male
Based on users’ preferences, the hue and saturation of the original skin color are reduced, while the intensity value is increased, as shown in Figure 4-2 (d). In the results, preferred skin color is reproduced, and the artifacts at the discontinuous regions between the skin color and non-skin colors are also eliminated.
(a) original image (b) result of skin color detection
(c) result after skin color classification (d) reproduced image
Figure 4-2 Experimental results of Caucasian female
In Figure 4-3, the saturation and intensity of the original skin color are lower than users’ preferences. Hence, both saturation and intensity values are increased to achieve preferred skin color.
(a) original image (b) result of skin color detection
(c) result after skin color classification (d) reproduced image
Figure 4-3 Experimental results of multiple Caucasians
Figure 4-4 and Figure 4-5 show the experimental results for Asians. The saturation and intensity are increased.
(a) original image (b) result of skin color detection
(c) result after skin color classification (d) reproduced image
Figure 4-4 Experiment results of Asian male
(a) original image (b) result of skin color detection
(c) result after skin color classification (d) reproduced image
Figure 4-5 Experimental results of Asian female
Figure 4-6 and Figure 4-7 show the experimental results for Blacks. The hue and
Figure 4-6 and Figure 4-7 show the experimental results for Blacks. The hue and