CHAPTER 5 RENDERING SYSTEM
5.2 Q UICK D RAW R ENDERING F UNCTION
Figure 10 Synthesized images by basic rendering function with four layers.
5.2 Quick Draw Rendering Function
The quick draw rendering function is used to synthesize quick drawing effect of Impressionist paintings, e.g. figure 2. To synthesize this kind of effect, the following objective should be achieved. First, use as fewer strokes as possible to represent the foreground object.
These strokes should reveal the shape of the object. The color distribution of foreground object is much less noticeable than its shape in synthesizing. Second, neither of shape and color of the background objects is noticeable in synthesizing.
To achieve the first objective, we change the stroke path generation process which is third phase in basic rendering function. Stroke path no longer follows the normal direction of image gradient. The blending direction of “shape direction” and normal of image gradient is used instead. The shape direction is calculated from primitive extendedness which is explained in Section 4.1. Besides, we use the stencil buffer of canvas to avoid overlapped strokes. To achieve the second objective, we blur the color buffer of primitive belonged to the background object. The blurring operation is done by blending each pixel’s color value with the mean color of the whole color buffer. The blurred color buffer is used as the input to the second phase in basic rendering function.
Figure 11 Quick draw of a goose, three layers.
5.3 Pointillist Rendering Function
Pointillist rendering function is used to synthesize pointillist painting style in neo-Impressionism. In Pointillism, small strokes or dots of pigment are used to complete the whole painting. The sizes of colored dots are regular. The color of these dots represents the actual physical composition of the lighting on object surface. For example, a scene of grass in sunlight will be painted in the following steps. First, green pigment is used to present the inherent color of grass. Then, light blue dots are applied to represent the reflection of sunlight
on grass. Three kinds of color dots are used to represent the shadow area of grass: green, red, and light blue. Red is the complementary color of green and Impressionist usually use complementary color to represent object shadow.
Traditional pointillist algorithm use random saturation and intensity color dots to synthesize this effect. However, by the inspection of the above example, the choice of color of each dot is the key concern. The color dot set, for the above example green, red, and light blue for shadow area of grass, represents the composition of light condition on object surface. Thus, random choice is not suitable.
We provide two pointillist rendering functions. The first approach is similar to the traditional algorithm. It uses the ground color to roughly painting the primitive first. Then color dots are placed on canvas. The colors of dots are the quantization of primitive color in their respective locations. The second approach also uses the ground color to roughly painting the primitive. Then, a color set is defined by the user. Color dots are placed on canvas. The colors of dots are chosen from the color set and will be blended with the color of primitive in their respective locations. Currently, user can specify at most three colors in the color set.
(a)
(b)
(c)
Figure 12 Pointillist rendering function. (a) Original image. (b) Synthesized result of first pointillist rendering function. (c) Synthesize result of second pointillist rendering function.
Chapter 6
Mark System
The list of stroke definitions generated by rendering system is fed to mark system which draws these strokes on canvas. The physical implementation of stroke definition heavily depends on the target painting media. For example, it is obvious that the water color mark system should differ from oil painting mark system a lot. Besides, the convenient usage of mark system is also important. In our framework the stroke definitions are automatically implemented by the mark system. Thus, physically-based mark system or interactive mark system in which too many possibilities exist is not suitable for our framework.
Our mark system basically contains three components: brush model including Pigment,
Canvas, Stroke Path, and Stroke Cross Section: Round CS, Flat CS; bristle canvas interaction
BCInteraction and random system Randomable. Bristle location is determined by stroke path
and stroke cross section. Besides, each bristle contains pigment, so does canvas. The interaction between bristle and canvas occurs on every contact along the stroke path. To synthesize oil painting effects, some properties are made random through random system. The following figure shows the architecture of our proposed mark system.Figure 13 Mark system architecture.
6.1 Brush Model
6.1.1 Stroke Cross Section
It is common for Impressionist to use both round and float-section brushes. Thus, we create two stroke cross section: round stroke cross section and flat stroke cross section which both provide the following information: bristle location based on current brush location, orientation and bristle index; the blended stroke cross section with other cross sections which will be discussed in section 6.2; random interval level which will be discussed in section 6.3.
(a) (b)
Figure 14 (a) Round and (b) flat stroke cross sections consist of low sampling points (Round:
16 x 8; Flat: 16 x 16).
As shown in Figure ??, the bristle distribution in round stroke cross section is based on the intersection between straight lines radiate from the ellipse center O and concentric ellipses bounded in the outmost ellipse. For example, P is such an intersection.
Figure 15 Bristle distribution in round stroke cross section.
The bristle location is calculated by the following formula:
y
In the above formula, (x’, y’) is the coordinate of bristle in ellipse coordinate system, i.e. point
O is treated as (0, 0). When the brush is placed at (t_x, t_y) in world coordinate system with
orientation angle gamma, the third and fourth formula will transfer (x’, y’) into world coordinate system by a rotation and a translation. The first and second formula deal with the bristle location in ellipse coordinate system by given bristle index. The bristle index starts from zero and increases counterclockwise from the inside to outside of an ellipse.Figure 16 The correspondence between bristle index and bristle location. Indices of the first eleven bristle are shown in this figure.
Thus, take point P as an example, by the given bristle index, we can calculate the angle between line OP and x-axis and the length of line segment OP by the following formulas:
π
... (2)
As shown in the above formula, there are four parameters controlling the distribution of bristles. MAJOR_RADIUS and MINOR_RADIUS are the major radius and minor radius of outmost ellipse. SAMPLE_ANGLE_SIZE and SAMPLE_RADIUS_SIZE control the count of sampled point in the contact region of brush.
Due to the nature of ellipse, uniform sampling of radius_ratio in Formula set (2) may sample too many points in the inner of ellipse. As a result, the bristle is not uniformly distributed in the contact region. To solve this problem, we define a mapping function from the original radius_ratio to the transformed radius_ratio. It is a power function and both its domain and co-domain are [0, 1].
(a) (b)
Figure 17 Two set of sampling results, each consists of two sampling (16x8, 32x16). (a) Uniform sampling of radius_ratio results un-uniform distribution of bristles; (b) Transfer
radius_ratio by power function y
=x
0.7 results uniform distribution of bristles.The flat stroke cross section operates similar to the round stroke cross section. As shown in Figure ??, the bristles distribute in a trapezoid area centered at O with BOTTOM_LENGTH
|AO|, UPPER_LENGTH_1 |CD|, UPPER_LENGTH_2 |EF| and HEIGHT |BD|.
Figure 18 Bristle distributions in flat stroke cross section. |AB| = |ED|.
The bristle location is calculated by the following formulas:
y
In the above formula, (x’, y’) is the coordinate of bristle in trapezoid coordinate system, i.e.
point O is treated as (0, 0). The third and fourth formula is similar to those in Formula (1).
The first and second formula deal with bristle location in trapezoid coordinate system by
given bristle index. The bristle index starts from zero and increases from bottom to top and right to left.
Figure 19 The correspondence between bristle index and bristle location. Indices of the first seven bristle are shown.
Thus, take point P as example, by the given bristle index we can calculate column_ratio and
row_ratio in formula set 3 from the following formulas:
1
SAMPLE_V_SIZE and SAMPLE_H_SIZE controls count of sampled point vertically and horizontally in the contact region of brush.
6.1.2 Stroke Path
The stroke path determines the location and orientation of a brush by interpolating a set of control points. Besides, stroke path also provides pressure and active bristle percentage
which will be discussed later. The location information is constructed by Cardinal spline since it will interpolate all control points. In fact, the Cardinal spline is simply constructed by Bezier curve. When the control points are given, additional control points will automatically be inserted into original adjacent control points. Then the original control points and the inserted control points are used together to construct the Bezier curve. Cardinal spline provides one parameter to control how new control points will be inserted. Smaller parameter will result smooth curve.
a. b.
Figure 20 Cardinal spline constructed by five control points with different smooth parameters (a) 10 (b) 2 respectively.
By given control points and the stroke cross sections per control point, the tessellation mechanism will generate the bristle information in any location along the stroke path. Bristle information such as location, orientation, and pressure are linearly interpolated between control points. All of these are done by blending ability of stroke cross section. By given two stroke cross sections and the location orientation information (obtained from the stroke path), the blended stroke cross section will be generated. The tessellation scale can be adjusted to improve the efficiency.
Although the bristle amount of a brush is fixed, the amount of bristles which actually contact the canvas depends on the area of contact region. The larger area is, the more bristles contact the canvas. Hence we add supplementary information per control points: the active bristle percentage (abp). The range of ABP is between zero and one. It depends on the area of contact area. The larger area is, the more percentage is. ABP is calculated once randomly before the tessellation begins.
(a) (b)
(c) (d)
Figure 21 Tessellated strokes with different tessellation scale. (a) Round cross section with tessellation scale 8. (b) Round cross section with tessellation scale 1; (c) Flat cross section with tessellation scale 8; (d) Flat cross section with tessellation scale 1.
6.2 Bristle and Canvas Interaction
When the stroke path, stroke cross section per control point, initial bristle properties are done, they are tessellated by a list of bristle and canvas interactions. The interaction occurs on every touching operation of stroke cross section on canvas along stroke path. Bristle and canvas interaction determines how a bristle interacts with canvas, i.e. the change of properties of bristle and canvas. Canvas and bristle both contains one kind of property: pigment. It has color, quantity, drop scale, and height scale. The quantity, drop scale and the pressure controls the pigment drop amount. The height scale is used for impasto effect of oil painting in which pigment on canvas is three-dimensional, i.e. it has height relate to the canvas plane. The height of a pigment is the multiplication of its quantity and its height scale. There two kinds of interaction: wet-on-dry and wet-on-wet. Wet-on-dry simply drops pigment in bristle onto canvas while wet-on-wet blends pigment in bristle with pigment on canvas. The blending uses subtractive color mixture model.
6.3 Oil Painting Effects
To create real oil painting stroke effects. Two random systems are used: pure color intensity variant random and height gap effect random. Pure color intensity variant concerns the effect that when apply one pure pigment color on canvas, the actual color is not uniform.
In fact the intensity of the pigment color varies. Height gap effect concerns the effect that pigment height is not uniform in a stroke. Usually, height of pigment could form several spans. Each span has roughly uniform height while different spans have different heights.
(a) (b)
Figure 22 Real oil painting strokes. (a) Pure color intensity variant random. (b) height gap effect random.
To synthesize pure color intensity variant effect, we randomly change the pigment intensity of initial bristle properties. In RGB color model, changing intensity is done by scaling RGB values identically.
To synthesize height gap effect, we first randomly generate a series span values and then use these span values to change the drop scale of pigment. The drop scale of pigment affects the drop quantity of pigment and thus affects the height of pigment. The generation of span values can be controlled by four parameters: base_gap, min_gap, random_gap and
random_level. Bristles are collected into the same span according to their y-axis coordinate in
local coordinate system. The span size and value are generated from the following formula:)
The Rand(a, b) function in formula 5 will generate a random number between a and b.
(a)
(b)
(c)
Figure 23 Two random systems: (a) Plain stroke. (b) Pure color intensity variant effect. (c) Plain height. (d) Height gap effect.
Chapter 7 Results
In this chapter, five result sets are presented. Result set 1 shows an ordinary oil painting effect and advance layering effect. Result set 2 shows the quick drawing effect of Impressionism. Result set 3 shows Pointillist’s style which belongs to Neo-Impressionism.
The framework is implemented in C++ language on NB with a Pentium 1.5G CPU and 512 MB RAM.
7.1 Ordinary and Advance Layering
Figure 24 Original input image.
Figure 25 Final image composed of five layers. The whole image is treated as one primitive.
First Layer Second Layer
Third Layer Fourth Layer
Figure 26 Final image constructed by five stages. Five primitives: entire image, sky, grass, haystack and sheep.
First stage (two layers): the whole image is treated as a primitive.
Second stage (three layers): sky.
Third stage (two layers): grass.
Fourth stage (two layers): sheep.
Fifth stage (three layers):: haystack.
7.2 Quick Drawing
Figure 27 Original input image.
Figure 28 Final image composed of two stages. Fourteen primitives: entire image and thirteen geese.
Final image of first stage (two layers): whole image.
Final image of second stage (two to three layers): thirteen geese.
One goose: second stage (three layers).
7.3 Pointillist
Figure 29 Original input image.
Figure 30 Final image composed of two stages. Six primitives: sky, tree, haystack, grass, grass shadow, and road.
Final image of first stage.
Figure 31 Final image composed of two stages with another Pointillist style. Six primitives:
sky, tree, haystack, grass, grass shadow, and road.
Image Size Primitive Layers Time (seconds)
Basic I 375 x 500 1 4 191
Basic II 375 x 500 5 2, 3, 2, 1 241
Quick Draw 800 x 600 15 2, {2~3,...} 1 210
Pointillist I 594 x 792 6 {1,...}, {1,...} 2 145
Pointillist II 594 x 792 6 {1,...}, {1,...} 2 144
Table 1 Result statistics.
Chapter 8
Conclusions and Future Works
A painterly rendering framework has been presented in this thesis. The framework consists of three sub-systems: primitive mapping system, rendering system and mark system.
The framework is written in C++. It is complete and flexible. Any developers can extend this framework by costuming its components. Thus, different painterly algorithm can be implemented in this system. Besides, the division of this framework is based on the actual process of painters which means the results generated can be more evaluative in esthetics aspect.
Impressionist paintings are synthesized by this framework to show its effectiveness in this thesis. We’ve shown three different styles of Impressionist paintings: original, quick drawing and Pointillism. Several results are presented in each style. Achieving each style needs extending or modifying each components of this framework. The extending or modifying work is clear and intuitive.
In the future, we’ll try to enhance our framework in the following aspects:
Add the spatial system to this framework. We omit this part in our thesis. However, it is important especially in oriental paints. I’ll try to figure out how the complex projection system is establishing in painting process. Another future work is to improve the mark system.
So far, we just synthesized two effects of oil painting. We’ll do more in the future. Having a
References
[1] BOMFORD D., KIRBY J., LEIGHTON J. and ROY A., “Art in the Making Impressionism”, The National gallery, London in association with Yale University Press, 1990.
[2] CURTIS C. J., ANDERSON S. E., SEIMS J. E., FLEISCHER K. W., SALESIN D. H.,
“Computer-Generated Watercolor”, Proceedings of ACM SIGGRAPH 97, 1997.
[3] DURAND F., “An Invitation to Discuss Computer Depiction”, Proc. 2nd Intern.
Symposium on Non-Photorealistic Animation and Rendering, 2002.
[4] GRABLI S., TURQUIN E., DURAND F., X F., “Programmable Style for NPR Line Drawing”, Rendering Techniques 2004 (Eurographics Symposium on Rendering), 2004.
[5] HAASE C. S. and MEYER G. W., “Modeling Pigmented Materials for Realistic Image Synthesis”, Master's Thesis, University of Oregon, August 1991, 1991.
[6] HAEBERLI P., “Paint by numbers: abstract image representations”, Proceedings of
ACM SIGGRAPH 86, 1986.
[7] HAYS J., ESSA I., “Image and Video Based Painterly Animation”, The 3rd
International Symposium on Non-Photorealistic Animation and Rendering, 2004.
[8] HERTZMANN A., “Painterly Rendering with Curved Brush Strokes of Multiple Sizes”,
Proceedings of ACM SIGGRAPH 98, 1998.
[9] HERTZMANN A., “A Survey of Stroke-Based Rendering”, IEEE Computer Graphics &
Applications, Special Issue on Non-Photorealistic Rendering. July/August, 2003.
[10] LITWINOWICZ P., “Processing Images and Video for An Impressionist Effect”,
Proceedings of ACM SIGGRAPH 97, 1997.
[11] LEE M., KOWALSKI M. A., TRYCHIN S. J., BOURDEV L. D., GOLDSTEIN D., and
HUGHES J. F., "Real-Time Nonphotorealistic Rendering", Computer Graphics:
Proceedings SIGGRAPH, Aug 1997. Annual Conference Series 1997. ACM Siggraph,
1997.[12] MEIER B. J., “Painterly Rendering for Animation”, Proceedings of ACM SIGGRAPH 96, 1996.
[13] SALISBURY M., ANDERSON C., LISCHINSKI D., and SALESIN D. H., "A
Resolution-Independent Representation for Pen-and-Ink Illustrations", Dept. of
Computer Science and Eng. U. of Washington. Technical Report UW-CSE-96-01-02. Jan
1996.[14] SALISBURY M. P., WONG M. T., HUGHES J. F., and H D., "Orientable Textures for Image-Based Pen-and-Ink Illustration", Proceedings of SIGGRAPH 97, in Computer
Graphics Proceedings, Annual Conference Series, 401-406, August 1997.
[15] STRASSMANN S., ”Hairy Brushes”, Proceedings of ACM SIGGRAPH 86, 1986.
[16] STEVE U., “The Renderman Companion”, Addison Wesley, Reading, MA, 1989
[17] WANG B., WANG W., YANG H., SUN J., “Efficient Example-Based Painting and
Synthesis of 2D Directional Texture”, IEEE Transactions on Visualization and
Computer Graphics Volume 10, Number 3, 2004
[18] WEN S. Z., SHIH Z. C., CHIU H. Y., “The Synthesis of Chinese Ink Painting”, National
Computing Symposium’99, page 461-468, 1999.
[19] WILLATS J., “Art and Representation”, Princeton U. Press, 1997.
[20] WINKENBACH G., SALESIN D. H., “Computer-Generated Pen-and-Ink Illustration”,