Stroke Combination

Now, we know all the candidates for stroke pairs. We need to check if the stroke pair can be combined together or not. The additional conditions about the order of combination and the grouping of the strokes are important.

Research Method 3.7 Stroke Simplification 19

Figure 3.10: T junction.

Figure 3.11: V junction.

Figure 3.12: Example 1 of V junction.

Research Method 3.7 Stroke Simplification 20

Figure 3.13: Example 2 of V junction.

The first additional condition is to prevent the case as shown in Figure 3.11. If the strokes only have two end point-stroke point pairs, we need to check their orientation. We notify the stroke 1’s tangent of overlapped point tl1 and tl2for stroke 2. We can have tl1· tl2 < 0 if stroke 1’s head is paired with stroke 2’s head or stroke 1’s tail is paired with stroke 2’s tail as shown in Figure 3.12. So as tl1· tl2 > 0 for stroke 1’s head pair with stroke 2’s tail or stroke 1’s tail pair with stroke 2’s head as shown in Figure 3.13. In the Figures, the head means the start point of the stroke, denoted as p₁. The tail means the last point of the stroke, denoted as p_n.

The second additional condition is to prevent the case as shown in Figure 3.14. We need to check if the strokes are fully overlap to each other at the overlap area. The point pairs define two sub-strokes of overlap area as the red part of the Figure 3.14. We need to make sure the distance between sub-strokes is smaller than the spatial error. Actually, we need to find a sequence of corresponding relationship between two strokes’ points in the overlap area. However, we just simply check if all of the points can find a corresponding point on another stroke within a user-specified distance δ_s.

The order of the combination should be important too. Because after the combination, the stroke may change and can not be combined with another stroke pair candidate. One case is that it should be related to the error E_sand E_a. Another case is that it should be related to the weight texture T. Due to the difficulty of the error update during the combination of the first case, we choose the second case.

Research Method 3.7 Stroke Simplification 21

Figure 3.14: Stroke pair candidate without full overlap.

According to the drawing principles summarized by Fu et al. [FZLM11], artists mostly draw from rough to fine. They start with a rough sketch and then gradually refine the drawing strokes.

So we prefer to delete the short and minor strokes first. That is, the strokes with small weights.

We calculate the weight sums of the points on a stroke l and average it with the length of the stroke llengthas followed:

For a stroke pair, we will choose the smaller value of the strokes to represent the weight of the stroke pair. And we sort the strokes in increasing order of the weight. Then we can combine the strokes with small weight first.

The grouping of the strokes can be separated into two problems. First, we need to decide whether we need the classification of the contour and hatching and how to combine the contour and hatching. Second, the strokes should be combined by “two to one” or “many to one” or

“many to many”.

At first, we want to use percentage of overlapping to classify the contour and hatching but failed. We observed that the hatchings overlap a lot so we classify the strokes with high per-centage of overlapping as hatching and that with low perper-centage of overlapping as contour. A stroke in the gray area between the two gaps will be classified by the distance and add it into the closest group. There is another condition for the hatching group. The hatching group must have three or more strokes, otherwise it is a contour group.

In this failed method, we combine the hatching group by deleting some of the strokes in the group. Our goal is to decrease the density of the hatching group. But we use a simple method.

For each pair of hatching strokes, we choose one of the strokes to be the new stroke. The parent

Research Method 3.7 Stroke Simplification 22

Figure 3.15: Problem of the order of combination.

pointer will point to the new stroke too. If one of the stroke l₁ has been simplified and another stroke l₂ is not simplified, l₂’s parent pointer will point to the l₁’s parent. Therefore, there will be only under half of the strokes left.

Although we believe that the contour and hatching need to be processed differently, we find out that our classification of the contour and hatching is still very weak. Especially for the short hatchings, we can not calculate the percentage of overlapping precisely. Finally, we give up this classification method.

The next problem is combining the strokes by “two to one” or “many to one” or “many to many”. “Two to one” is the easiest way to combine the stroke. But the order of the combination will influence the result. After some combinations, some of the strokes may not be combined as in Figure 3.15. At first, l₁ and l₂ can be combined together and l₂ and l₃ can be combined together too. After the combination of l₁ and l₂, l₃ can not be combined with the new stroke.

“Many to one” can be considered the best path for all the strokes in the group. But it is hard to guarantee that there should be only one stroke after combination. Orbay and Kara [OK11]

used lots of steps to eliminate this problem. Although they give many successive cases, we are still not sure if they will fail in other cases. “Many to many” also can be considered the path for all the strokes in the group. The problem is to decide the start and the end of the stroke. In the worst case, the number of the strokes may increase.

We choose the “two to one” method to combine the strokes because it is the easy and steady.

Research Method 3.7 Stroke Simplification 23

Figure 3.16: Cases of combination.

According to the order of stroke pair as we mentioned above, we will combine the stroke pairs one by one. For each stroke pair, we need to check their end point-stroke point pairs again. By the point pairs, we will classify them into three cases as in Figure 3.16. In addition, we still need to check the orientation and distance of the overlapping area as we mentioned above.

Case A is that the two strokes only have two point pairs and the end points belong to dif-ferent stroke as shown in Figure 3.16(a). We will get two sequences of the points in the over-lap area. The sequence of points are the end point to the point of recorded index. Then we will interpolate the points of two sequences. For two sequences P = {p1, p₂, p₃, ..., p_n} and Q = {q₁, q₂, q₃, ..., q_m}, we will choose the sequences with more points. For example, if n > m, we choose P as a base and for each p_i ∈ P we find the closest point qp_i = q_j on Q and interpolate them as shown in Figure 3.17. The point positions and radiuses of the new stroke in the overlapping area are calculated by

p_newi = n− i n p_i+ i

nq_pi, (3.32)

where 0 ≤ i ≤ n − 1. It is also a simplified method because we did not derive the sequence of point pair for the P and Q. The points outside of overlapping area are still at the position as

Research Method 3.7 Stroke Simplification 24

Figure 3.17: Interpolation of Case A.

they are on the old strokes. If the radius r < 1, we set r = 1 to prevent the stroke from too thin for drawing. We need to subdivide the stroke and calculate the tangent for the new stroke for the next stroke combination.

The radius should be related to the old positions before stroke translation and every point should record the farthest combined point during the combination. And the radius should vary smoothly between points. However, we did not finish this part. We just interpolate the radius.

In Case B (Figure 3.16(b)), we only choose the stroke which is not fully overlapped to be the new stroke. In Case C (Figure 3.16(c)), we are free to choose any one of the strokes as the new stroke. After the combination of each stroke pair, we will change the parent pointer to point the new stroke. We will update the average weight ¯w_l for the stroke pairs after each combination and then sort them in increasing order and choose the first pair in the list for pair combination.

We repeat this process until all of the stroke pairs are processed. When all stroke pair candidates are processed, we complete the simplification.

Research Method 3.8 Discussion 25

Figure 3.18: (a) l is not an ε-line. (b) l is an ε-line [BTS05].

3.8 Discussion

There are several previous work discussing line drawing simplification in the context of Non-Photorealistic Rendering (NPR). We will discuss the differences with their work in this section.

Barla et al. proposed the concept of ε-line [BTS05]. They cluster the original curves into consistent groups. Each group can be replaced by an ε-line. An ε-line is a line that does not fold onto itself at the scale ε (see Figure 3.18). Each point of the ε-line has no point along the normal at a distance less than ε that also belongs to the same line.

After they split the initial lines into ε-lines, they will find the pair of ε-lines. They have observed that only five kinds of combination for the paths can be combined as a new line (see Figure 3.5). The pair of ε-lines will be clustered as ε-groups.

An ε-group is a group of ε-lines that can be converted to an ε-line at the scale of ε, as shown in Figure 3.19. They will calculate the error for each pair of ε-lines. It is defined by the spatial error and other attributes of the lines such as color. The constraint will be set in order to forbid clustering if the line attributes are too different. They build a graph, each node represents an ε-line and each edge represents the error between two ε-lines. Then, they use a greedy algorithm to choose the edge of the smallest error at each clustering step. The algorithm stops when no more clusters can be created. In practice, they use two standards, namely pre-defined strategies to choose the new line. The average line interpolates all the original line in the cluster for the

Research Method 3.8 Discussion 26

Figure 3.19: ε-group [BTS05].

contour. The most significant line is one of the original lines chosen according to an application-defined priority measure (base on length, nature...) for the hatching.

Shesh and Chen presented an efficient and dynamic line simplification [SC08]. They define a parameter δ as the maximum distance between two strokes and a parameter ρ as the percentage of overlapping between two strokes. They also define a value to measure how “near”, how “local parallel” and how similar in the color of stroke pairs. They use an s-spanner [GGN06] to find the point pairing. They build the spanner by 2D points on the screen. Given the distance δ, querying the s-spanner will return all pairs of points that are within a distance δ. Then they will built two tables C(p, S) and Q(S, T ) from all the pairs returned by the spanner. C(p, S) maintains the

“closest” points in every stroke S to a given point p. Q(S, T ) maintains the geometric likelihood that stroke S and T will be paired together. They pair stroke S with a stroke T if ρ percent of the points of S are paired to some points in T with comparable corresponding local gradient.

However, if multiple strokes have sufficient overlapping with a single stroke S, only the first stroke (in the order of the time they were created) will be paired with the stroke S during that frame. After they pair the strokes, they will parameterize [Gos00] and sort [KDMF03] the points in S and T into a common domain. And use Rational Gaussian Curve [Gos95] for interpolation.

However, they encountered a problem when some of the lines are drawn too long and winding.

So they segment such strokes by monitoring abrupt curvature changes if need.

Orbay and Kara proposed a new method not only simplifies the sketches but also beautifies them [OK11]. They used a trainable stroke clustering which uses the neural network that first takes the input of the feature extracted from the stroke and then decide whether the two strokes should be grouped together. Their feature contains three parts: remoteness (shortest distance

Research Method 3.8 Discussion 27

Figure 3.20: (a) original strokes (b) bifurcation points [OK11].

between two strokes), misalignment (tangent vectors at the points of minimum distance) and discontinuity (the likelihood of one stroke naturally continuing another one). After the neural network finished the bottom-up stroke fusion, they will do the top-down stroke group fission.

They split the curves by calculating the minimum spanning tree (MST) [Kru56, Pri57] of the point set. Only three significant branches will be considered if the point has more than three branches. The bifurcation points (see Figure 3.20) are identified as those having a _BL^BLmin

max > ε, where BLminand BLmaxare the minimum and maximum cumulative branch lengths emanating from the point, and ε is the threshold 0.05 in their implementation. For n bifurcation points, this results in 1 + 2n distinct point groups. The next step is merging the selective branches. For each point group, they identify the stroke that contributes the most number of points to that group (or multiple if tie) as illustrated in Figure 3.21(a). They use these strokes as the natural

“skeleton” and use the tangents of these strokes’ end point to decide the natural combination among the branches (see Figure 3.21(b)). In the end, for n number of bifurcation points, 1 + n final stroke clusters are generated (see Figure 3.21(c)). At this point, each cluster is just a set of unorganized coordinate points. They reorder the point by Laplacian Eigenmaps [BN03] as shown in Figure 3.22. Last, they beautify the reordered points by curve fitting and smoothing.

They use a technique similar to moving least squares [Lee00]. Finally, the parameterization will be calculated (see Figure 3.23). The user may select to fit a cubic B-spline to the entire point set when desired. They employ a modified error function that incorporates the stylus pressure.

Research Method 3.8 Discussion 28

Figure 3.21: (a) natural skeleton (b) natural combination (c) final stroke clusters [OK11].

Figure 3.22: Laplacian eigenmaps [OK11].

Figure 3.23: Locally quadratic curve is fit to a fixed number of points. (a) The original point x_p, y_p is projected parallel to the local coordinate frame. (b) Curve parameterization is computed after the point placements [OK11].

Research Method 3.8 Discussion 29

The only thing in common is that every method uses the feature of the strokes to find the stroke pairs. The spatial error, angular error and the continuity are mostly used. Some of the methods use other attributes such as color. The detail calculations are different. For stroke combination, every method uses different ways of combining the strokes.

However, the ε-line can not fold onto itself. The method of Barla et al.[BTS05] can not deal with the folding or self-intersecting curves. Shesh and Chen’s concept of overlapping [SC08]

can be weak because the contour curves may not overlap that much. For the method of Orbay and Kara [OK11], they focus on the contour of sketches and did not mention the hatching. They use the neutral network, which need the training and their “many to many” combination method relies on the stability of the guarantee that there should be only one stroke after the combination.

C H A P T E R 4

Results

In this chapter, we present the experiment results of the stroke simplification. We present some experimental results with different parameters. Then we show the final results and com-pare the results of our algorithm to the results of the algorithm proposed by Barla et al. [BTS05]

and Shesh and Chen [SC08]. We also list the computation time of the stroke translation, the stroke pairing and combination, and the total time.

All experiments are performed on Intel Core CPU i5-3450 3.1GHz× 4 cores with 8G ram, using NVIDIA Geforce GTX 560 graphics hardware. The operation system is Windows 7 x64.

All results are rendered in 512× 512 resolution.

4.1 Stroke Translation

There are two steps for the stroke translation. First, draw all of the strokes to the texture and do the low-pass filter using Gaussian kernel. Users can define the size of the filter. The size should cover the thickest part of the overwriting strokes. The results of using the filter size δ_f = 5, 10, and 15 are shown in Figure 4.1. In Figure 4.1(a), the center of the feature strokes is not lighter than the other pixels. Use δ_f = 10 or 15 will be better.