CHAPTER 1 INTRODUCTION
1.2 OVERVIEW
In recent years, non-photorealistic rendering (NPR) approaches have received renewed interest in computer graphics-related research. Many researches addressed on Western painting, including watercolors [6, 11], impressionistic painting [21], pencil sketches [31, 32] and hatching strokes [10, 13, 20, 15, 28, 30]. These deliver good results for modeling Western painting. However, most of these methods are not fit for Chinese ink painting. Chinese ink paintings typically consist of simple strokes that convey the artists’ “deep feeling” but not just color filling or pattern placing.
Simulating the style of the Chinese ink painting is not at all trivial. The style includes free brush stroke, surface wrinkle and ink diffusion on the paper [8, 22, 25].
The main goal of this thesis is to render terrain’s wrinkle automatically in Chinese landscape painting. First, a 3D terrain model is drawn with silhouette edges.
Then, information on the terrain surface such as shading or orientation is used to generate texture strokes. Finally, the stylized strokes are applied to have a Chinese ink painting look. Besides the visual styles, we can navigate around the scene and show images viewed from different positions at interactive rates. Furthermore, we resolve the frame coherence problem that we can have a smoothly view while navigating. The proposed rendering technique involves many fundamental parts, as shown in Figure
1.1. The main function of each part is described as follow:
3D Terrain Model
Figure 1.1: System architecture 3D Information Extraction
Ink Diffusion on a rice paper
Geometry
(1) 3D information extraction: We use OpenGL to render 3D terrain model with vertices, edges, faces and light sources. The luminance map is specified as ink tone values. The ID reference image will be used in hidden line removal step.
(2) Mesh simplification: To speedup the computation time, we reduce the total vertices of terrain model at a pre-processing step. Users can specify different levels of simplification.
(3) Feature lines extraction: This part include streamlines generation and silhouette edges detection. Streamlines are direction fields on the surface and they are computed at a pre-processing step. Silhouette edges are contour of an object that is calculated frame by frame. All these steps are done in 3D object space.
(4) Hidden lines removal: We use a fast and efficient algorithm to detect visible feature segments. Then we merge these segments into smooth and long paths for the next step.
(5) Projection: All visible feature lines are projected onto a 2D viewing plane. These projected 2D points are used as control points to apply brush strokes.
(6) Brush strokes: A physically-based brush stroke module [37] is used to simulate the Chinese style of drawing. The ink tone is specified by a luminance map.
synthesized images. This makes the final result the same as the traditional Chinese landscape painting.
(8) Ink diffusion: The motion of ink on the rice paper is simulated [16].
The rest of this thesis is organized as follows. In Chapter 2, we review the related works in non-photorealistic rendering. Chapter 3 describes the extraction of feature lines. In order to synthesize effects of Chinese ink painting, we introduce an applicable brush model in Chapter 4. Chapter 5 shows different kinds of rendering styles and techniques to improve the quality of synthesized images. In Chapter 6 we demonstrate results generated by our proposed system. Finally, conclusions and future work are given in Chapter 7.
Chapter 2 Related Works
According to different styles of painting, researches on non-photorealistic rendering [6, 11, 15, 21, 31, 32] focus on different aspects. For instance, color distribution is important while simulating oil painting, but the crucial point of Chinese ink painting is a proper brush model. Since the goal of this thesis is to simulate Chinese ink painting, we classify related researches into the following categories:
silhouette algorithm, line visibility determination, stylized rendering, and brush model.
2.1 Silhouette Algorithm
Silhouettes play an important role in shape recognition because they provide one of the main cues for figure-to-ground distinction. There are so many methods handling the silhouette extracting and visibility culling problems. Saito and Takahashi [29] proposed an image-space algorithm to find silhouettes that use the z-buffer to
detect edges and apply an edge detector such as the Sobel operator. Hertzmann [12]
extended this method by using a normal buffer instead.
A simple object-space algorithm is based on the definition of a silhouette edge.
The algorithm examines all model edges and selects only those that share exactly one front- and one back-facing polygon. Buchanan and Sousa [4] suggested using a data structure called an edge buffer to support this process for speeding up the computation time. To further speed up the brute-force algorithm, Card and Mitchell [5] suggested employing user-programmable vertex processing hardware. In this thesis, we use a simple object-space algorithm with a data structure similar to winged-edge data structure. This reduces the computation time of silhouette edges.
2.2 Line Visibility Determination
In most cases, object-space silhouette algorithms produce the problem of visibility culling. This problem is the classic computer graphics problem of hidden line removal. There are many algorithms for hidden line removal in object space.
Appel [3] presented the visibility algorithm that based on the notion of quantitative invisibility (QI) – the number of front-facing polygons between the point on the edge to be rendered and the viewer. Markosian et al. [23] used a modified version of
Appel’s algorithm to improve computation time. They defined the QI value as the total number of faces between a point and the viewer. Hertzmann and Zorin [13]
applied this approach to their algorithm for sub-polygon silhouettes.
A simple and fast image space algorithm to determine the visibility of silhouette edges is to use the z-buffer. Silhouette edges are rendered and let the z-buffer remove the hidden lines. However, pixel accuracy and partially occluded problem are main disadvantages of z-buffer algorithm. Northup and Markosian [26] used an ID buffer to determine silhouette edge visibility. A unique color identifies each triangle and silhouette edge in this ID buffer. Isenberg et al. [18] use a similar approach. They base on their visibility test on the z-buffer instead of an additional ID buffer. In this thesis, we use ID buffer to resolve the hidden line problem because it can also identify the visible triangle surfaces that help us to determine other type of feature lines.
2.3 Stylized Rendering
There are many researches on 3D non-photorealistic rendering, such as stylized line illustrations, artistic hand-drawn illustrations, pen-and-ink drawing and hatching.
Winkenbach and Salesin [38] provided a method of rendering smooth surface in pen-and-ink style. Buchanan and Sousa [31, 32] focused on physically simulating real
media, including pencil, crayon, blenders and erasers. Freudenberg’s [10] method involves encoding a stroke texture as a halftone pattern. Salisbury et al. [30]
introduced prioritized stroke texture whose tone value is mapped to stroke arrangement. Lake et al. [20] described an interactive hatching system with stroke coherence in the image space. Hertzmann and Zorin [13] generated high quality silhouette edges and described a scheme for placing image space strokes for cross-hatching.
Recently, many researches use principal curvature directions for surface rendering. The most natural geometric candidate is the pair of principal curvature direction fields [9, 34]. Way [36] developed a simple weighted technique by using a reference direction to generate smooth direction fields on a surface. In this thesis, we follow Way’s method to represent surface’s wrinkles because this method is suitable for generating streamlines for texture strokes of terrain’s surface.
2.4 Brush Stroke Model
For Chinese ink painting synthesis, the most important part is to have an expressive brush stroke model. Little research has addressed on simulating brush stroke and the behavior of ink. Strassmann [33] proposed a basic brush model of ink
painting. He defined a 1-D array as the painting brush, and elements in the array are taken as bristles. Horace and Helena [17] proposed a method to simulate the physical process of brush stroke creation by using a 3D cone and move it dynamically. Their system focused on Chinese calligraphy.
Weng [37] introduced a 2D brush model. He defined a circle as the contact region of brush and canvas. Bristles are distributed uniformly in the contact region.
While painting, the center of contact region moves along the stroke path and leads bristles moving and leaving footprints on the canvas. Various painting styles are predefined and the orientation of brush movement is taken into consideration. Strokes painted with Weng’s model looks more natural and smooth. Hence, Weng’s model is the major reference of our system because of its easy implementation and good visual effects.
Zhang [39] simulated ink behavior with Simple Cellular Automation-based simulation. They build three models: ink, brush and paper, and then take complex interaction of these models to synthesize ink strokes. Curtis [6] simulated various artistic effects of watercolor. His model is empirically-based and incorporated some physically-based models for generating watercolor strokes.
Chapter 3 Feature Line Extraction
This chapter describes feature lines we used to represent the Chinese landscape painting. First, 3D terrain model information is extracted. Second, the streamlines that express the wrinkles of terrain surface are constructed in a preprocess step. Third, the silhouette of the model is generated at run time. Finally, these silhouette and streamlines are projected onto a 2D viewing plane frame by frame. These 2D points are used as control points to apply the brush strokes. We will discuss how to apply brush strokes by control points in the next chapter.
3.1 Extracting 3D Information
Digital Elevation Model (DEM) is widely used in science and engineering applications. It is also known as a grid field and is a popular format of terrain files. In this thesis, we use 3DEM Visualization Software [40] to construct all of the terrain models. The grids of DEM heights were converted into adjacent triangles. The inputs
of the 3D terrain polygonal models are rendered using OpenGL includes vertices, edges and faces. The lightness of shading is determined using the luminance map. The gray-scale value in the luminance map specifies the ink tone values. Geometrical information is used to detect the edges of silhouettes, generate streamlines and ridge meshes. All information can be extracted in preprocess without any user’s intervention.
3.2 Streamline Construction
To generate streamlines, we must choose some direction fields on the surface.
The most natural geometric candidate is the pair of principal curvature direction fields [9, 34]. However, there are various weaknesses. For example, some fields are not defined anywhere on a sphere. Also, on flat areas when both radii of curvature are very small, the fields are likely to yield a far more complex pattern. Yutaka Ohtake et al. [27] presented adaptive smoothing tangential direction fields on a polygonal surface. Their method effectively simulated pen-and-ink drawings of 3D objects.
However, Chinese landscape painting seeks to simulate the texture of a terrain’s surface. In this thesis, we follow Way’s method [36] to generate streamlines that represent the surface wrinkles of the terrain.
Given a triangular surface, we first calculate the principal curvature direction at each vertex. Then, we update each vertex’s direction by calculating a weighted sum of the directions repeatedly and simultaneously. This step smoothes the direction field and preserves the coherent of the reference direction. The reference direction is the gravity direction which indicates the flow of terrain. Since DEM is a grid field of height, so given any vertex P (Px, Py, Pz), Py is the height of the vertex. Figure 3.1
shows a triangular mesh F with normal vector Frn. Gr is the direction of gravity (0, -1, 0). Grref is the projection of vector Gr onto the F. The vector t(P) is the initial principal direction estimated at the vertices in the mesh, as follows:
∑
where Q are all the neighboring vertices of P and W is weighting value between PQ.
Gref
Figure 3.1: The reference direction Gr
ref.
The vector t(F) is the streamline direction field combined with t(P1), t(P2), t(P3)
and Gr
ref on triangle F. It is defined by t(F) = αGref + (1-α)(t(P1)+t(P2)+t(P3)).
The meaning of t(F) can be explained as: if a raindrop falls on the surface F, it would flow along the streamline direction t(F). This is quite reasonable and expressible for the terrain. It also fits the features that painters used to describe the mountains.
After the streamline direction field is smoothed by the reference direction, we calculate the direction field t(F) on the face of every triangle mesh. Then, an initial
point inside a triangle is selected. The streamline is traced from the point, according to Sk+1=Sk +ρt(Fk); ρ is a real number such that the line meets the edge of the triangle ;
Sk+1 is located at the edge of the triangle. Streamline tracing ends when the direction
conflicts with the neighboring triangles’ streamline direction, as shown in Figure 3.2, in which the streamline stops at a ravine. Using this method, each triangle mesh can generate one streamline. Figure 3.3 depicts the resulting strokes according to streamlines obtained by this method.
Figure 3.2: Streamlines construction.
In Chinese landscape painting, painters combine various strokes to depict terrain’s surface wrinkle. Changing the distribution of strokes, the stroke length and other factors generates different texture strokes. To further improve the quality of the final result, we add some terms that can control the distribution of streamlines while constructing streamlines. First, the density term d controls the tightness between streamlines. When we start to construct a streamline from a triangle, we first check its neighboring triangles. If more than d neighbors have constructed streamlines, the current triangle will not generate any streamline. Otherwise, it will. Second, the segment term s determines the segments of a streamline. When a streamline pass
through a triangle, we call it a segment. Originally, the streamline grows until it meets the ravine edge. Now we check the total segments of a streamline at each construction step. When the total segment is greater than s, this streamline stops the construction process. Third, the angle term θ controls the slops of a streamline. When a streamline is constructed, we further check its included angle with the ground level (1, 0, 0). If the included angle is smaller than θ , this streamline is disabled. By adding this restriction, we can express the roughness of the terrain’s surface. Figure 3.3 (a) to (d) show different kinds of distribution by changing terms d, s and θ.
(a) Original Streamlines (b) Lower Density
(c) Fewer segments (d) Greater Included Angle Figure 3.3: Streamlines rendering.
3.3 Silhouette Generation
Since the DEM terrain data has been converted to standard triangular meshes, we need to analyze the structure of these models to determine which edges form silhouette outlines. We define a silhouette edge to be an edge that connects a front-facing triangle to a back-facing triangle. Because this condition depends both on
the viewpoint and the state of the model, we should identify these silhouette edges frame by frame.
We use a brute-force approach to find the silhouette edges that simply checks every edge of the mesh. This may waste a lot of time. To speed up the silhouette identification process, we make use of a winged-edge data structure that provides local connectivity information. When checking if an edge is a silhouette edge, we find its two neighboring faces by looking up the terrain data structures. Then we only need to compare these faces’ normals with the viewing direction, we can identify if the edge is a silhouette edge. Figure 3.4 shows the silhouette edges obtained from the terrain model.
(a) Terrain Model (b) Silhouette Edges Figure 3.4: Silhouette generation.
3.4 Stroke Placement
After we have found out streamlines and silhouette edges, the next thing we are going to do is to find out visible segments from these feature lines. Then these visible segments are merged into long paths. Finally, all the end points of the paths are used as control points to apply strokes. The following sections will describe the detailed steps.
3.4.1 Hidden Line Removal
Once we have detected the feature lines in the current frame, the next step is to determine which portions of the silhouette edges and streamlines are visible. We make use of the “ID reference image” [19] to extract visible segments.
We create the ID reference image by first rendering the scene with each silhouette edges and each triangle mesh drawn in a color that uniquely identifies it.
Then read this image from the frame buffer. Lighting and blending are disabled so that the colors are preserved exactly. Figure 3.5 shows the ID reference image of the corresponding terrain model.
(a) Triangle Meshes (b) ID reference image Figure 3.5: ID reference image.
We next iterate over all the pixels in the reference image and build a list L of edges that contribute at least one pixel. We thus remove from consideration silhouette edges that make no contribution to the pixels of the current frame. For every streamline, we determine the visibility by checking the visibility of triangles it passes through.
Then, we scan-convert along each edge in L to determine which portions of it show up in the ID reference image, and hence are visible. We record each such visible portion, or segment. A segment consists of its two image-space endpoints and a pointer to the associate edge. To determine whether an edge e shows up in the ID reference image at a given image-space point x on e, we check the ID reference image at x, and also at nearby points (within two pixels in practice) along the image-space line perpendicular to e and passing through x. If e shows up anywhere along this line,
we consider e to be visible at x.
3.4.2 Artifact Removal
Since meshes do not provide perfect representation of surface and due to numerical instabilities in silhouette edges detection, we often get overlapping silhouette edges and small zig-zags in image-space. These have to be removed in order to achieve good looking results when applying styles to strokes. We resolved some well known artifacts such as triangles with two visible edges, silhouette edge clusters, silhouette stroke zig-zags, sharp angles and short segments.
3.4.3 Linking Segments into Paths
Now we have a collection of segments that approximate the visible silhouettes of the scene. The next step is to link these segments into long paths that will form the skeleton of strokes. To do this, we first search segment’s endpoints for potential neighbors. The search is an i x i (we use i = 3) pixel local search in the ID reference image. Figure 3.6 shows the algorithm we used to compute the suitability of each potential match between segment s and neighbor n. Here we use and G
0
max =45
θ
max = 5 pixels.
Link (s, n)
if endpoints of s and n don’t overlap and G≤Gmax
and θ angle of s’s current neighbor (if any) ≤ and θ angle of n’s current neighbor (if any) ≤ link s and n
Figure 3.6: The algorithm of linking segments.
3.4.4 Stroke Control Points
After long silhouette paths and streamlines are extracted, we then apply stroke along these paths. Our brush model is constructed using the precise and flexible Cardinal spline. Cardinal spline is an interpolated but not approximated curve, which passes through all the specified control points. So we use endpoints of silhouette paths and streamlines as control points. The other advantage of Cardinal spline is its flexibility. The terrain can be painted softer or harder. Cardinal spline enables slackness control by changing parameter t. A small t implies a slacker curve. Figure
3.7 displays the precision and flexibility of the Cardinal spline. Once the control points are determined, then we can apply the Chinese style of strokes. We will discuss how to achieve this in the next chapter.
(a) t = 0.2
(b) t = -0.5 Figure 3.7: Cardinal curve with different t.
3.5 Mesh Simplification
The Chinese landscape painting pays attention to express an artistic concept. So painters usually care about an overall arrangement among objects. The detail of an object is not so important; we only need to depict major characteristics. To further improve the quality and performance of the rendered image, we have some ideas for
The Chinese landscape painting pays attention to express an artistic concept. So painters usually care about an overall arrangement among objects. The detail of an object is not so important; we only need to depict major characteristics. To further improve the quality and performance of the rendered image, we have some ideas for