Colored Pencil Drawing

Very few attempts have been made at the study of colored pencil drawings.

Those researches often tended to concentrate on one individual issue, such as the

blending effect of custom colors [7], the physical model simulation of drawing papers [8], or the direction of drawing strokes [2], without completely considering the picture of actual colored pencil drawings. Although Yamamoto et al. [7] introduced a new idea of color customization, expressing a target color by mixing two colors, the final output result was still deficient in color blending. Based on the physical laws, Takagi et al. [8] performed volumetric renderings which included the different effects of papers, drawing, watering, and erasing. Nevertheless, these rendering effects did not fully resemble the actual colored pencil drawings. In 2005, Matsui et al. [2] combined the concepts stated above and put more emphases on the creation of drawing strokes based on boundaries of regions. This thesis indeed presented much better results compared to those past related researches. However, for those images which contain complex information, the creation of drawing strokes becomes difficult to handle.

Beyond that, some commercial software, such as Photoshop [11], also has function which converts images into colored pencil style drawings. Frankly speaking, those who have ever used this software knew that the filter for colored pencil style drawing is incorrect.

Chapter 3

Algorithm of Colored Pencil Filter

3.1 Paper Model

We will now examine more closely the model of drawing papers. As for actual colored pencil drawings, the texture of drawing papers affects the style of colored pencil artworks. To put it more concretely, pigments are shaved off from the colored pencils by a friction on a convex part of the drawing papers. Therefore, the roughness of drawing papers results in different deposit amount of pigments. The rougher the drawing papers are, the weaker possibility the pigments deposit evenly.

In our simulation method, the drawing paper textures (see Figure 2) were directly downloaded from the internet [12]. Only two textures commonly used by artists are selected in our system. The width and height of these texture images are both 64 pixels. We adjust the size of the texture image according to each input photo. The intensity values of these texture images are also calculated and then used as the deposit efficiency of the pigments. The main concept is to determine the thickness of pigments at each pixel based on the intensities of both the input photo and the texture image. We will later discuss in full detail the pigments distribution in Section 3.4.

Figure 2 Texture images.

Now, let us get down to the issue of paper model. As mentioned above, our system provides two default texture images as paper models for users. Let (xpi, ypi) be the coordinate of a pixel i on the paper model p, and (xti, yti) be the coordinate of a pixel i on the texture image t. We compute each channel of red, green, and blue respectively as follows:

In our experiments, by adding an extra parameter Ep = 45, we could represent the actual paper color more closely. The example of the final paper model is shown in Figure 3.

Figure 3 The paper model.

3.2 Outline Drawing

The next issue which we have to consider is how to draw outline sketches.

Actual artists used to draw rough outline sketches in the beginning of a drawing process. Thus, we introduce the Sobel edge detection technique [1] as our simulation method for outline drawing.

Figure 4 The Sobel Operator.

The masks shown in Figure 4 are called Sobel operators. These operators are used to compute the gradient value of a 3×3 window at the point labeled Z5, as follows: Equation 4 computes the difference of the intensity value between the third and the first rows of a 3×3 window. Gx then represents an approximation of the derivative in the x-direction. Similarly, Gy is the difference between the third and the first columns which approximates the derivative in the y-direction. ∇f is the sum of both the absolute value of Gx and the absolute value of Gy. There is a strong possibility that the point labeled Z₅ is a part of our outline sketches when the ∇f value is large enough.

To be precise, a threshold value Te is used to indicate the complexity of our final outline sketches. The default value of Te is half the maximum intensity of the whole image. By our experiment, the maximum of the intensity value would not exceed 441.67275. Therefore, we set our Te value to be 221. In other words, when the calculated value exceeds 221, we take that point as one part of our outline sketches. Let us see the following operation for further explanation. Another parameter U

∇f

c is adopted especially for users. Users can determine the complexity of the final outline sketches by adjusting the value of Uc.

} Furthermore, we offer an extra calculation (see Equation 8) to control the intensity of the final outline sketches. Let I(xi, yi) be the intensity value of any pixel i in the image. Once a point (x, y) has already been detected as a part of the outline sketch, users could again change its intensity value by increasing or decreasing the Ip value. The smaller the Ip value is, the darker the intensity of the final outline sketch appears.

How to transform an original color photo into a colored pencil style image is the main issue of this thesis. Color transformation is one of the key factors that determine whether the final rendering result resembles the actual colored pencil style drawing or not. Compared with the colors of a true color photo, the colors of an actual colored pencil style drawing are limited in hue and the saturation is relatively low.

Therefore, we choose the HSI color model, as shown in Figure 5, which closely matches the experience of human visual perception as our reference model. HSI

stands for hue, saturation, and intensity respectively. Hue is a color attribute that describes different pure colors such as red, yellow, green, and blue, etc. Saturation is the measure of the proportion of one pure color mixing with white color. For example, pink has less saturation than red. Intensity represents the brightness of an image.

Figure 5 The HSI color model.

After we obtain the RGB values of each pixel from an input color photo, we start to transform these values into the HSI color space. The conversion from the RGB color space to the HSI coordinate system refers to the book of Fundamentals of Color Image Processing [3]. First of all, we rotate the RGB coordinate to form a new coordinate system (I, V1, V2) whose axis is the line R=G=B. The following linear

Next, we continue to transform the rectangular coordinates (V1, V2) to the polar coordinates, as shown in Equations 10 and 11. Incidentally, the range of hue

coordinate is from 0 degree to 2π degree.

Finally, the following procedure performs the inverse transformation from the HIS color space to the RGB color space.

H

3.4 Pigment Distribution of Colored Pencils

Now we will discuss the pigment distribution of colored pencils in detail. As we mentioned at the beginning, we do not concern with neither the styles nor the directions of drawing strokes artists may create. Owing to each artist’s unique characteristic of drawing, it is unreasonable and not objective to guess the artist’s possible drawing strokes. In addition, the drawing stroke is not one of the most important features of colored pencil style drawing. Hence, we concentrate on how pigments adhere to the surface of papers.

First, let us consider the complexity of paper fibers. Instead of carefully simulating the microstructure of papers based on physics, we use a combination of four main directions to represent the complexity of paper fibers. How paper models affect the general deposit efficiency of the pigments has been discussed in Section 3.1.

Here we take a close look at the relationship between the pigments and the paper fibers. The deposit orientations of the pigments would vary because of different directions of paper fibers. Thus, we classify these paper fibers into four main directions including (a) horizontal, (b) up-right diagonal, (c) vertical, and (d) up-left diagonal as shown in Figure 6.

Figure 6 The directions of paper fibers.

Secondly, let Ps be a gradient value of a pixel in an image. Equation 15 defines Ps as the ratio of Gy and Gx. Gy and Gx are calculated according to the Sobel edge

detection technique as mentioned in Section 3.2. Equation 16 further describes how we select the deposit orientation of the pigments.

Gx

b up right diagonal if Ps orientation

c vertical if Ps or Ps

d up left diagonal if Ps

⎧ ≤ <

Thirdly, we create two window sizes of pigment masks, 5x5 and 7x7, to sprinkle pigments on each pixel in an image. According to our experiment, these two window

sizes are most suitable for the final rendering results. We then emphasize the intensity values of the pixels included in the dark areas, whereas those intensity values in the white areas are unchanged, as illustrated in Figures 7 and 8.

(a) horizontal (b) up-right diagonal (c) vertical (d) up-left diagonal Figure 7 The 5×5 pigment mask.

(a) horizontal (b) up-right diagonal (c) vertical (d) up-left diagonal Figure 8 The 7×7 pigment mask.

So how do we choose a 5×5 or 7×7 pigment mask for a pixel in an image? Again we use those values that are already calculated according to the Sobel edge detection technique. Let Gs be the sum of both the square value of Gx and the square value of Gy, and let Gs_avg be the average value of the sum of Gs, as shown in Equations 20

and 21. For any pixel (m, n), if the value of Gs(m,n) is larger than the average value Gs_avg, we apply the 7×7 pigment mask. On the contrary, the 5×5 pigment mask

would be selected. Finally, in order to stress the same powdered quality as the actual colored pencil

style drawing, we observe several actual artworks of colored pencil style drawing. We cut a piece of image from the artwork and then enlarge this piece, as shown in Figure 9. After carefully examining the distinct intensity shown in each pixel, we make a general conclusion. As a whole, there is a 4% probability that the color of the pixel in an image is close to white. Therefore, for each one pixel included in a 5×5 square area of an image, we increase its intensity value twice. Beyond that, we randomly select five pixels included in a 5×5 square area and increase their intensity 1.2 or 1.5 times as mush as the original values. For the intensity of other remaining pixels, we adjust their values in a limited range at random.

Figure 9 The enlarged texture example.

Chapter 4

Implementation and Results

All of our experiments were performed using a Pentium 1.50 GHz, 768 MB RAM in real time. We have implemented all the techniques described above as a Window application using C#. A colored pencil style drawing is automatically generated with a set of default values and other information derived from the input images. Users are allowed to change the final results by interactively adjusting the controlling parameters in each step of the transforming procedure.

Figure 10 illustrates the complete procedure of our transformation for an input image. First, users specify an input photo and then choose a proper paper model.

Second, our system creates initial outline sketches like artists’ hand-drawings. In this step, we offer users two parameters, both intensity and complexity of sketches, to modify the temporary rendering results. Next, we simulate the step of ground color painting because artists usually draw a layer of light ground color on papers after outline sketching. Again a parameter, thickness of color, is provided for users to determine the darkness or lightness of output images. Finally, our system automatically sprinkles the pigments of colored pencils and hence renders a colored pencil style drawing.

Eventually, let us take a look at our final rendered results. All input images for test, includes general scenic photos and still objects, are shown in Figure 11.

Although those scenic photos have very complicated information, we still generate

single still objects, we also render very good results, as shown in Figure 12 (d and e).

Figure 13 illustrates the comparison with Adobe Photoshop, the famous commercial image editing software. Obviously, those results transformed by Adobe Photoshop have unnecessary fractures and are incorrect in colors. In contrast, our rendered results indeed appear better quality of colored pencil style drawings.

Furthermore, we compare our technique with the method Matsui et al. [2]

proposed in 2005. As shown in Figure 14, Matsui et al. [2] create good drawing strokes based on boundaries of regions. However, for complicated objects in images, they fail in generating detailed drawing strokes. As a whole, our rendered results represent a better perception of colored pencil style drawing rather than emphasize the dispensable drawing strokes.

(a) (b)

(c) (d)

(e)

Figure 10 The transforming procedure.

(a) input image, (b) paper model, (c) outline sketching, (d) ground coloring, (e) result

(a) (b)

(c) (d) (e)

Figure 11 The input images for test.

(a)(b)(c) scenic photos, (d)(e) still objects

(a)

(c)

(d)

(e)

Figure 12 The rendered results.

(a) (b)

(c) (d)

(e) (f)

Figure 13 The comparison with Adobe PhotoShop.

(a)(b) input images, (c)(d) Photoshop, (e)(f) our proposed method

(a) (b)

(c) (d)

(e) (f)

Figure 14 The comparison with Matsui et al. [2].

(a)(b) input images, (c)(d) Matsui et al. [2], (e)(f) our proposed method

Chapter 5

Conclusions and Future Works

On the basis of simulating the actual artist’s drawing process, we proposed a novel technique for generating a colored pencil style drawing from an input image.

The process of our proposed method consists of paper model choosing, outline sketching, ground color painting, and the final colored pencil pigments sprinkling.

Our proposed algorithm is not complicated and the rendering time is almost in real time. Instead of guessing or imitating the possible drawing strokes of different artists, we choose to present the characteristic of the colored pencil pigment itself through a series of clever image processes. To sum up, our final rendering results show a very good quality of colored pencil style drawing, and are even better than the existing commercial image editing software, Adobe Photoshop.

As for the future work, we aim to add an extra interactive function for users to determine the subject drawing strokes in the final rendering results. Moreover, we also want to extend our method for converting a video sequence into a colored pencil style animation owing to our fast rendering speed.

References

[1] Gonzalez, R.C., and P. Wintz, Digital Image Processing, Addison-Wesley Publishers, Inc., USA, 1992.

[2] Hajime Matsui, Henry Johan, Tomoyuki Nishita, “Creating Colored Pencil Style Images by Drawing Strokes Based on Boundaries of Regions”, Computer Graphics International 2005, 2005.

[3] I. Pitas, Fundamentals of Color Image Processing, John Wiley & Sons, Inc., USA, 2000.

[4] Kyoko Murakami, Reiji Tsuruno, and Etsuo Genda, “Multiple Illuminated Paper Texture for Drawing Strokes”, Computer Graphics International 2005, 2005.

[5] Sousa M. C. and Buchanan J. W., “Observational Models of Graphite Pencil Materials”, Computer Graphics forum Volume 19 (2000), number 1 pp. 27-49.

[6] Shigefumi Yamamoto, Xiaoyang Mao, Kenji Tanii, Atsumi Imamiya, “Enhanced LIC Pencil Filter”, Proceedings of International Conference on Computer Graphics, Imaging and Visualization, pp.251-256, IEEE, July 2004.

[7] Shigefumi Yamamoto, Xiaoyang Mao, Atsumi Imamiya, “Colored Pencil Filter with Custom Colors”, Proceedings of Pacific Graphics 2004, pp. 329-338, October 2004.

[8] Takagi S., Fujishiro I., and Nakajima M., “Volumetric Modeling of Colored Pencil Drawing”, Pacific Graphics ’99 conference proceedings, 1999.

[9] Xiaoyang Mao, Yoshiyasu Nagasaka and Atsumi Imamiya, "Automatic Generation of Pencil Drawing from 2D Images Using Line Integral Convolution", Proceedings of the 7th International Conference on Computer Aided Design and Computer Graphics CAD/GRAPHICS2001, pp.240-248, August 2001.

[10] 張瓊文，「第一次畫色鉛筆就上手入門篇」，易博士文化，(民國 94 年)。

[11] Adobe System. Adobe Photoshop.

[12] http://fweb.midi.co.jp/~buru_nyan/download