CHAPTER 2 Related Works
2.3 Curvature-based Drawing from 3-D Polygonal Objects
Objects
When we render a 3D model as a line drawing, we first draw the contours of the model. But the contours are quite limited in the information to convey about the shape.
So we need to give more information such as depth discontinuities.
Chang [3] proposed a system to create curvature-based pen-and-ink drawings from 3D polygonal objects. According to the method, we first extract the depth information. Then calculate three different terms on the depth information. They are mean curvature, Gaussian curvature, and Laplacian convolution. The result is more perceptible and easier to infer the shape of object by viewer, as shown in Figure 2.4.
11
(a) (b)
(c) (d) (e)
Figure 2.4: Comparision of several visual effects. (a) depth map, (b) only contours, (c) result by mean curvature, (d) result by Gaussian curvature, (e) result by Laplacian convolution.
12
Chapter 3 Algorithm
3.1 System Overview
The goal of this thesis is to create Op-Art illusion by deforming or inflating some particular squares. Figure 3.1 describes an overview of our square-based Op Art system architecture. First, we construct a grid of squares by a user given 3D model using the user desired square number and square spacing. Then the rendered image is rasterized to the grid of squares. Each square has a corresponding depth value which is extracted from the depth buffer. In order to find the raised regions of the input 3D model, we use peak-climbing clustering algorithm [17] to cluster the squares by using the depth information. between two connected squares to perturb the in-between points, as shown in Figure 3.1(f). Finally, we obtain the square-based Op Art result by combining all the above
13
steps.
Figure 3.1: Overview of square-based Op Art system architecture. The following steps produce square-based Op Art form the 3D model.
3.2 Grid of Squares Construction
We construct the 3D polygonal object from the vertex, normal and face lists.
After the image is rendered, the depth map is stored in the depth buffer. User specifies the input 3D model and the desired square spacing. Next, we construct a grid of squares, each size is s × s. Then rendered image is rasterized to a square grid so that
14
each square gets one color, as shown in Figure 3.2.
(a) (b)
Figure 3.2: (a) Rendered image of the input 3D model. (b) If the center pixel of square is covered by the rendered image, the square is marked as an object square.
Otherwise, the square is a background square.
3.3 Squares Clustering
In order to reveal the 3D-like effect, we regard the raised regions as the regions to be inflated. Peak-climbing algorithm [17], also called hill-climbing search algorithm, is used to find the raised squares and clusters the object squares by using the depth values. We assign the depth value of each square as the depth of the square center pixel. For each object square, it looks at its eight neighbors to find the neighbor which has the smallest depth value. Then the square connects to its neighbor with smallest depth by an arrow, as shown in Figure 3.3(a).
Peak-climbing is a greedy local search because it grabs a good neighbor without thinking where to go next. But it is good for finding the local protruding squares. The cluster center is a “peak” where no neighbor has a lower depth value. In other words,
15
the square receives arrows from all neighbors. Besides, we assign the object square to a cluster center by tracing the arrow directions until reaching a cluster center. So each object square belongs to one cluster center. The object squares which are assigned to the same center are in the same cluster. Therefore, we can get the cluster centers and its corresponding member squares by using peak-climbing algorithm as shown in Figure 3.3(b).
(a) (b)
Figure 3.3: Clustering by peak-climbing algorithm: (a) The cluster center is marked as 0. There are five cluster centers in this example. (b) The different colors represent the different clusters.
3.4 Object Squares Deformation
Because the cluster centers are the raised regions, it is directly perceived that the closer the square to the cluster center, the more the square needs to be expanded. The goal of the algorithm is to create the shape which is like a raised ball in the cluster centers. We achieve the idea by making use of a simple radial basis function (RBF) to
16
shift the points of the object squares by the distances between the points and the cluster centers.
Like many other rendering styles, it is difficult to select a set of parameters to achieve the viewer’s desired effect. Our algorithm contains the following parameters:
1. W and H are the width and height of the image, respectively;
2. 𝑁𝐻𝑜𝑟𝑖 and 𝑁𝑉𝑒𝑟𝑡𝑖 are the number of the horizontal squares per row and the number of the vertical squares per column, respectively;
3. 𝑑(𝑝, 𝑐𝑖) is the distance between the point 𝑝 and the cluster center 𝑐𝑖; covered by the object squares, respectively;
7. 𝐶𝐻𝑜𝑟𝑖𝑖 and 𝐶𝑉𝑒𝑟𝑡𝑖𝑖 are the number of the columns and the number of the rows covered by the squares belong to the cluster i, respectively;
17
We propose an algorithm which shifts the points of object squares in Algorithm 1.
On the one hand, the user should input 𝑑𝑀𝑖𝑛 and 𝑑𝑀𝑎𝑥 for determining the influence on 𝑑(𝑝, 𝑐𝑖). When 𝑑𝑀𝑖𝑛 is fixed, the larger 𝑑𝑀𝑎𝑥 represents that the displacements of the points will more sensitive to 𝑑(𝑝, 𝑐𝑖). On the other hand, 𝑆𝑀𝑖𝑛and 𝑆𝑀𝑎𝑥 should be given to decide how large the squares will be inflated.
For each cluster center 𝑐𝑖, we determine how large the squares should be inflated by the number of the rows and the columns covered by the cluster. If the number of the rows and the columns covered by the cluster are the largest, the squares in this cluster will be inflated more than those in other clusters. We calculate 𝐶𝐻𝑜𝑟𝑖𝑖 and 𝐶𝑉𝑒𝑟𝑡𝑖𝑖 as the number of columns and the number of rows covered by the cluster 𝑐𝑖, as shown in Figure 3.4(a). Then we calculate 𝑠𝑐𝑎𝑙𝑒𝑥 and 𝑠𝑐𝑎𝑙𝑒𝑦to scale the maximum displacement 𝑆𝑀𝑎𝑥. To avoid the squares do not inflate like a ball, we set 𝑠𝑐𝑎𝑙𝑒𝑥 and 𝑠𝑐𝑎𝑙𝑒𝑦 to the same value when these two value are quite different.
18
We shift the points which belong to the object squares by the cluster centers, as shown in Figure 3.4. After the points shift according to Algorithm 1, the preliminary result has been made, as shown in Figure 3.5. But the result contains some dislike appearances such as the overlapped squares on the boundary between the object squares and the background squares, and the holes between the connected points. We will solve these undesired appearances in the next section.
Figure 3.5 displays the result of shifting the points by our Algorithm 1.The blue squares are the background squares which are motionless as the rasterized square grid.
The green points and red points belong to the object squares. These points are shifted by the cluster centers marked by red. In this example, the parameter are set as follows:
𝑊 = 𝐻 = 600 , 𝑁𝐻𝑜𝑟𝑖 = 𝑁𝑉𝑒𝑟𝑡𝑖 = 20 , 𝑑𝑀𝑖𝑛 = 1 , 𝑑𝑀𝑎𝑥 = 13 , 𝑆𝑀𝑖𝑛 = 0 , 𝑆𝑀𝑎𝑥 =
19
30.
Figure 3.5: The result of shifting the points by our Algorithm 1.
3.5 Squares Alignment
To avoid the object and background squares being overlapped, we need to align them flawlessly. Firstly, we find the up, down, left and right borders of the object
Figure 3.6(a) shows the up borders and the down borders which are marked as U and D, respectively; Figure 3.6(b) illustrates the left borders and the right borders marked as L and R, respectively. There are two squares belong to the up border in column 5, 𝐺5,5 and 𝐺5,16. 𝐺5,16 is the top most up border among them. There are two squares belong to the down border in column 5, 𝐺5,4 and 𝐺5,7. 𝐺5,4 is the bottom down border among them, as shown in Figure 3.6.
20
(a) (b) Figure 3.6: The borders of the object squares.
Then we calculate the displacement after the point shifted on each border. Finally, add the negative displacement to the points of object squares. We obtain a result that the edges of the quadrilateral are connected seamlessly, as shown in Figure 3.7.
Figure 3.7: The result after deal with the boundaries alignment. The object squares and the background squares connect seamlessly.
21
The result is still unnatural because all the background squares are static. After we aligned all the border squares to the background squares, some squares will look strange. For example, the squares on the back of bunny are odd because these squares are not like a part of the bunny. On the contrary, these squares look like a part of background, as 𝐺10,13, 𝐺11,13 and 𝐺12,13 in Figure 3.7. In order to solve this unnatural appearance, our system creates a type of square called fake object square. In fact, the fake object squares are the background squares. But we regard the fake object squares as the object squares and shift the points in the fake object squares as we do in the object squares.
We use Algorithm 2 to find the fake object squares. For Figure 3.8(a), we can find all the fake object squares simply by step 2 to step 5 of Algorithm 2. But for Figure 3.8(b), the fake object squares in red color need to be found by step 6 to step 17 of Algorithm 2. Figure 3.9(a) and 3.9(b) are the results with and without step 6 to 17. The red colored squares in Figure 3.8(b) will be deformed by Algorithm 1 and the result looks more natural, as shown in Figure 3.9(a). On the contrary, these squares in Figure 3.9(b) are treated as background, so the squares around these background squares are almost static since they need to be aligned, as shown in Figure 3.9(b).
22
(a) (b) Figure 3.8: The fake object squares are marked as F.
23
(a) (b)
Figure 3.9: The results with and without step 6 to 17 of Algorithm 2.
We regard the fake object squares as object squares and shift the points using Algorithm 1. The result is more stereoscopic because the inflated effect decreases gradually from the object squares to the fake object squares, as shown in Figure 3.10(b).
(a) (b)
Figure 3.10: The compare of using the fake object squares or not. (a) The result without adding the fake object squares. (b) The result with the fake object squares.
24
3.6 Points Disturbance
The current result is only affected by the cluster centers. The inflated squares can be understood as the raised regions of the input 3D model. But the result does not account for the surface structure of the model. Besides, the squares farther from the cluster centers are almost static, such as the ears and the tail of the bunny, as shown in Figure 3.10(b). Therefore, we propose a method to show the surface structure by perturbing the points by the neighbors of the square, as shown in Algorithm 3. For each column, if the depth value of the square 𝐺𝑖,𝑗 is bigger than the depth value of the square 𝐺𝑖,𝑗−1, we perturb the points in this column to the right smoothly. On the contrary, we perturb the points in this column to the left smoothly. Likewise, we perturb the points in each row by comparing the depth values of two adjacency squares. By perturbing the points can we perceive the surface structure. Besides, we can observe that the squares deform obviously even when they are farther from the cluster centers, as shown in Figure 3.11.
25
26
(a) (b)
Figure 3.11: The result uses Algorithm 3 to perturb the points. (a) Only use Algorithm 3 to perturb the points. (b) Combine the result using Algorithm 1 and Algorithm 3.
3.7 Rendering Approaches to Op Art Results
After observing a great quality of square-based Op Art works drawn by artists, we select two rendering approaches to render our experiment results. The first type refers to the Vasarely illusion. This illusion is the perception of an oblique light cross which is like a ‘X” along the diagonals of concentric squares [2]. But this cross does not really exist. It has been incorporated into many Op Art paintings, as shown in Figure 3.12(a). Several research studies discussed the Vasarely illusion such as Tsofe et al. [20] and Troncoso et al [19]. The artworks of this type render the object squares by a luminance gradient, as shown in Table 3.1. The squares 𝐺𝑖,𝑗 which is a cluster center is tinted by the color of index 1. Other squares 𝐺𝑚,𝑛 are tinted by the color of index ‖𝑚 − 𝑖‖ + ‖𝑛 − 𝑗‖. If the index is greater than the greatest index 8, we use index 8 as the index. We can create more colors for obvious perception of layers by interpolating the colors in Table 3.1. The second type of our rendering approach is
27
inspired by the Vasarely’s Harlequin. We stored the colors in a table. Then apply these colors to our results. For each row in Harlequin, the squares are tinted by four colors, two of them are for the object squares and the others are for the background squares, as shown in Figure 3.12(b).
(a) (b)
Figure 3.12: (a) The Op Art work Arcturus, by Vasarely. (b) A part of Harlequin, by Vasarely.
Table 3.1: The colors of a luminance gradient are rendered in our experiment results.
28
Chapter 4
Experimental Results
We present the implementations and results in this chapter. The input sources are 3D triangle meshes, and the outputs are 2D square-based Op Art images. All the 3D models we used can be found from Google 3D Warehouse. These models are post-processed to show their depth maps, as shown in Figure 4.1. Even a skilled Op artist may spend several days drawing a painting. By using our rendering system, each result would be generated only in a few seconds.
Figure 4.1: Depth maps of our 3D models.
In our system, users need to define some parameters for generating the desired Op Art images. Figure 4.3 compares four experimental results with different parameters 𝑆𝑀𝑎𝑥 and 𝑑𝑀𝑎𝑥. The corresponding values of each result were defined in Table 4.1. Figure 4.3(a) was produced by the greatest value of 𝑆𝑀𝑎𝑥. So the inflated
29
effect of squares around each cluster center is more obvious than those in Figure 4.3(b)-(d). Figure 4.3(b) and Figure 4.3(c) used the same 𝑆𝑀𝑎𝑥. Figure 4.3(c) looks smoother, especially the squares near to the cluster centers because the parameter 𝑑𝑀𝑎𝑥 is smaller.
(a) (b)
(c) (d)
Figure 4.3: Op Art results with different values of parameters.
30
Table 4.1: The parameters of each experiment results. (𝒅𝑴𝒊𝒏 = 𝟏 , 𝑺𝑴𝒊𝒏= 𝟎.)
Figure 4.4 shows the result of the pig model. We first show the depth map of the model and its corresponding clustering results, the different clusters are represented by different colors, as shown in Figure 4.4(b). Then we render the object squares of the pig model by a luminance gradient which creates the Vasarely illusion. The parameter 𝑁𝐻𝑜𝑟𝑖 and 𝑁𝑉𝑒𝑟𝑡𝑖 of Figure 4.4(d) are greater than those in Figure 4.4(c).
When the number of square increases, there is a seeming lightening at the edges, forming the “X” [2]. We perceive that the Vasarely illusion is clearer with larger 𝑁𝐻𝑜𝑟𝑖 and 𝑁𝑉𝑒𝑟𝑡𝑖.
31
(a) (b)
(c) (d)
(e)
Figure 4.4: The rendered result of pig. (a) The depth map. (b) The clustering result.
(c)-(e) The rendered Op Art images.
Figure 4.5 shows the clustering and rendering result of the zebra model. This model has eight clusters, as shown in Figure 4.5(a). We used black and white to tint the zebra because Op Art often uses two highly contrasting colors to render the artworks. Besides, we rotated the squares to make it diagonal, as shown in Figure 4.5(d).
32
(a) (b)
(c) (d)
Figure 4.5: The rendered results of zebra. (a)(c) The clustering results of the rendered Op Art images (b) and (d), respectively.
Figure 4.6 compares the result with and without using the fake object squares.
The background squares close to Venus’s loin are regarded as the fake object squares.
So they are deformed along with the object squares, as shown in Figure 4.6(b).
Besides, we compare the results with perturbing the points or not. In Figure 4.6(b), we perturb the points so that the thighs are more perceptible. On the contrary, the result which only used the cluster centers to shift the points is shown in Figure 4.6(c).
33
(a) (b) (c)
Figure 4.6: The rendered results of Venus. (a) The clustering results. (b)(c) The rendered Op Art images.
Finally, we compare our result with the Op Art painting The Juggler by Vasarely, as shown in Figure 4.7. The inflated effects on the raised regions look similar. Besides, the background squares left to the loin were deformed analogously. More Op Art experiment results created by our system are shown in Figure 4.8 to Figure 4.10.
Figure 4.7: Compare our result with The Juggler by Vasarely. The left is the rendered result of Venus. The right is Vasarely’s The Juggler.
34
Figure 4.8: The rendered bear with perturbing the points or not.
Figure 4.9: The Op Art results rendered by different approaches of the bunny.
35
(a) (b)
Figure 4.10: The rendered Op Art result of Hulk. (a) The clustering result. (b) The experiment result.
The corresponding execution times of each Op Art output image were shown in Table 4.2. We observe that the number of object squares and fake object squares dominates the execution time. The greater the number is, the more the execution time it spends. Besides, our system spends most of the time finding the new intersections of each horizontal line and vertical line. Because the points of a square may alter after square deformation and square alignment, as shown in Figure 4.11. The red points in Figure 4.11(a) transfer to the red points in Figure 4.11(b) after square deformation and square alignment. We find the new intersections by calculating the distance between the points of each row and column in the user specified 𝑆𝑀𝑎𝑥. The two points with minimum distance will be the new intersection. When 𝑆𝑀𝑎𝑥 is larger, the displacement of the point is greater. So the range of each row and column that we
36
should check is wider, as shown in Figure 4.12. After we find all the new intersections, we can obtain the points in a square.
Table 4.2: The execution times of each experiment results.
(a) (b)
Figure 4.11: The points of a square alter after square deformation and square alignment. (a) Before square deformation. (b)
37
Figure 4.12: A new intersection is obtained by finding the smallest distance between the red points. (a) 𝑆𝑀𝑎𝑥 =𝑁𝑊
𝐻𝑜𝑟𝑖× 2.0. (b) 𝑆𝑀𝑎𝑥 =𝑁𝑊
𝐻𝑜𝑟𝑖× 1.0.
38
Chapter 5
Conclusion and Future Works
This thesis explores the square-based Op Art and the methods for achieving the desired artistic effects. The inflated and deformed squares create the swelling illusion which is similar to the artworks by Op artists. The input 3D models provide the depth information, so the users do not need to decide the inflated squares manually. We can cluster the squares by making use of the peak-climbing algorithm. Our system provides some parameters for the user preference. To imitate the Op artworks which have some deformed background squares, we create a type of square called fake object square. Besides, user can choose the rendered approach to tint the Op Art image. To avoid too many static squares existing in our result, we perturb the points of each square by the depth value of the adjacency squares. One limitation of our approach is that if the squares which have the smaller depth value are not the raised region of the input 3D model, it would fail to produce a reasonable result.
There are a few directions for further research. Our system differentiates the foreground and background by the rendered approach which tints the two regions by different colors. If we tint the two regions by the same color, the contour of the model may become unclear or disappear. Ideally, the system should identify areas that require greater deformation such as the finer details in an image. We expect it can help to create the illusory contour on the boundary of the model. Besides, we expect the technique introduced in this paper can be extended to create the animation of Op Art.
39
References
[1] Arad, N., et al. (1994). "Image warping by radial basis functions: Application to facial expressions." CVGIP: Graphical models and image processing 56(2):
161-172.
[2] Bach, M. (2005). "Pyramid Illusion."
http://michaelbach.de/ot/lum-pyramid/index.html.
[3] Chang, G. "Curvature-based Pen and Ink Drawings from 3-D Polygonal
[3] Chang, G. "Curvature-based Pen and Ink Drawings from 3-D Polygonal