A Skeleton-Based Approach for Consistent Segmentation Transfer
4. CONSISTENT SEGMENTATION TRANSFER
4.4 Boundary Refinement
We have matched the sample points of the source and target boundaries. The final step here is to refine the target boundaries. Most of the mesh segmentations compute boundaries close to the local geometric features, such as, curvature or dihedral angle. Some techniques may compute boundaries that have short and smooth shapes. In our case, we want to refine the target boundaries that may follow particular patterns such as jagged or wavy shapes according to the corresponding source boundaries. We use snake [19] to refine based on the following energy function:
Z"[\ ] ∈; Z^[ _C` ) Za[J` b`,
SAI-KEUNG WONG, JAU-AN YANG, TAN-CHI HO AND JUNG-HONG CHUANG
where Za[J and Z^[ _C` are the shape term and the feature term for a sequence of sample points t of , respectively. Z^[ _C` controls the target boundary for getting close to the geometric features and Za[J` maintains the shape of as similar as the shape of . The snake scheme [19] is adopted for searching the samples of the snake (snaxel) that locate on either the vertices or edges of the target mesh. We compute the corresponding point s of for each point t of based on equation 4. We then compute the offset of each point which is the distance between each boundary point and the fitting plane. The offset of a point is normalized by the length of the bounding box diagonal of the mesh. Assume µ(s) and µ(t) are the offsets of s and t, respectively. Then Za[J`= |µ(s) − µ(t)|.
To avoid the local minimum, a searching region centered at the point t is used for finding the closest vertex whose dihedral angle [18] is similar to s. The size of the region is defined as the longest geodesic distance of the mesh multiplied by a small constant (e.g. 0.02). If the searching region is too large, the target boundary may be drifted away. The feature energy is computed as Z^[ _C` = geo(t, c(t)), where geo(·, ·) is the geodesic distance (on the mesh surface) between two points, c(t) is the closest vertex in the searching region of t such that |dihedral(s) − dihedral(c(t))| < λ. If there does not exist such c(t), Efeature(t)
= 0. In our experiments, λ is set to 0.1.
5 Results and experiments
We present the results of segmentation transfer and the timing information in this section.
Results. Fig. 3 shows the two different types of segmentations for the four-legged animals and the corresponding segment pairs are drawn the same colors. For the top example of Fig. 3, the boundaries cross the legs and form the short shapes. For the bottom example, the boundaries cross the legs in the tilted manners and form longer shapes. We can see that the corresponding boundaries of target models have similar characteristics. Fig. 9 shows more segmentation transfer results. If the target model does not have parts corresponding to the source model (the ears of the dog and human in Fig. 9), the corresponding boundaries are not transferred.
Fig. 9 The segmentation transfer results. The source segmentations (dog, ant, bear, bird, chair and armadillo) are on the left hand side of the arrows. The corresponding segment pairs are drawn in the same colors. The parameter values: θ = 1.5, K = 0.05, λ = 0.1. For sheep, λ = 1.0.
SAI-KEUNG WONG, JAU-AN YANG, TAN-CHI HO AND JUNG-HONG CHUANG
Comparisons. We compare with the method by [10] and Fig. 10 shows the results. The results produced by [10] are poor for the models with large difference in shapes or poses. Our method produces better results since our method adopts skeleton correspondence before segmentation transfer is performed.
Fig. 10 Comparison with [10] for segmentation transfer.
In comparing with the method by Kalogerakis et al. [14], our method is more capable of controlling the positions of boundaries. Notice the differences to the rightmost airplanes in Fig. 11 and the giraffes in Fig. 12. The seat and back of the rightmost chair in Fig. 13 form a single segment since there is no boundary on some pillars. The method of [14] is region-based so that it can produce segments for all the pillars of the chairs. Finally, we compare with the other segmentation algorithms based on the Princeton Benchmark [8]. The results between ”Human” and our method ”Seg Trans.” are similar to each other. Our method can consistently transfer the source segmentation to the target mesh. The results also show that our method is comparable to the other methods, including Randomized Cuts [9], Shape Diameter Function [26], Normalized Cuts [9], Core Extraction [15], RandomWalks [18], Fitting Primitives [1] and K-Means [27], as indicated in Fig 14.
Fig. 11 The comparison with [14] on airplane models. The wings of the rightmost airplanes have difference segmentation types. A target segment boundary is not smooth in the second left airplane in our approach due to the poor alignment of the fitting plane.
Fig. 12 The comparison with [14]. Our result is better for the giraffe.
SAI-KEUNG WONG, JAU-AN YANG, TAN-CHI HO AND JUNG-HONG CHUANG
Fig. 13 The comparison with [14]. The seat and back chair form a single segment for the right chair.
Fig. 14 Princeton Segmentation Benchmark: Comparison of segmentation algorithms with four evaluation metrics. Our method is denoted as ”Seg Trans.” and the given segmentation is denoted as ”Human”. The four metrics: CD: Cut Discrepancy; Consistency error (Global Consistency Error and Local Consistency Error); Hamming distance and RandIndex.
Timing information. All the results were performed on Intel Core 2 Duo CPU E8400 3.0GHz with 4GB memory, using a single thread implementation. We precomputed AGD function, MSP function, skeleton extraction and dihedral angle function. The total preprocessing time ranges from one minute to 30 minutes. The computation time depends on the complexity of the objects.
Table 2 The computation time of the segmentation transfer results in Fig. 10.
Model Number of
faces
Segment transfer time (seconds)
Number of boundaries
Avg. boundary transfer time (seconds)
Dog(source) 18976 - 15 -
Deer 7402 1.31 15 0.09
Human 11258 1.46 11 0.13
Armadillo 20000 2.15 15 0.14
Dragon 16000 1.51 13 0.12
Triceratops 15764 1.48 15 0.10
Elephant 30000 3.75 15 0.25
Asian dragon 28198 3.18 15 0.21
Armadillo(source) 50382 - 17 -
Armadillo(tg 1) 50212 6.21 17 0.37
Armadillo(tg 2) 49226 6.08 17 0.35
SAI-KEUNG WONG, JAU-AN YANG, TAN-CHI HO AND JUNG-HONG CHUANG
Table 2 lists the computation times of our segmentation transfer results presented in Fig. 10, including the total computation time, the number of segment boundaries, and the average time for performing a boundary transfer. On average it took around four seconds to perform segmentation transfer. Our method requires the manual operations to change the parameters if the segmentation results are not good. Usually, it took three to four attempts to obtain high quality results.
Discussions and limitations. The quality of the segmentation transfer results depends on the quality of the skeleton correspondence. If the skeletons do not capture the shape of the meshes faithfully, our method may not work properly. If the fitting planes do not fit well the source boundary, the arc ratios are not reliable for determining the target boundary positions. We cannot handle segment boundaries which enclose multiple arcs. The target boundaries may be distorted for non-cylindrical covering regions. Further studies are needed for building the local coordinate systems for computing the covering regions of the target segment boundaries.
6. CONCLUSION
We have presented a segmentation transfer approach for mapping consistently a segmentation of a source mesh to a target mesh. The segmentation transfer approach is based on the skeleton correspondence between the meshes. The characteristics of boundaries including the relative orientation between the boundaries and skeletal arcs, the shape of the corresponding parts and local surface features, are considered.
We have shown that our method generates consistent segmentations for a variety of similar meshes of different shapes and poses. Our approach relies on the skeleton-mesh correspondence and skeleton correspondence to perform segmentation transfer. Our method does not handle the segment boundaries that do not enclose a skeletal arc.
In the future, we would like to develop techniques to transfer any kind of segment boundaries by combining consistent parameterization with skeletons. We believe that the skeleton-based approach can enhance the quality of the segment boundaries for the techniques based on consistent parameterization and other segmentation techniques.
7. REFERENCES
1. M. Attene, B. Falcidieno, and M. Spagnuolo, “Hierarchical mesh segmentation based on fitting primitives,”
The Visual Computer, 22(3):181–193, 2006.
2. M. Attene, S. Katz, M. Mortara, G. Patane, M. Spagnuolo, and A. Tal, “Mesh segmentation - a comparative Study,” in IEEE Int’l Conf. on Shape Modeling and Applications, page 7, 2006.
3. O.-C. Au, C. Tai, D. Cohen-Or, Y. Zheng, and H. Fu, “Electors voting for fast automatic shape correspondence,” Comp. Graph. Forum, 29(2):645–654, 2010.
4. O. K.-C. Au, C.-L. Tai, H.-K. Chu, D. Cohen-Or, and T.-Y. Lee, “Skeleton extraction by mesh contraction,”
ACM TOG, 27(3), 2008.
5. P. Besl and N. McKay, “A method for registration of 3-D shapes,” IEEE Trans. on PAMI, 14(2):239–256, 1992.
6. S. Biasotti, S. Marini, M. Spagnuolo, and B. Falcidieno, “Sub-part correspondence by structural descriptors of 3D shapes,” Comp. Aided Design, 38(9):1002–1019, 2006.
7. Y. Boykov and V. Kolmogorov, “An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision,” IEEE Trans. on PAMI, 26(9):1124–1137, 2004.
8. X. Chen, A. Golovinskiy, and T. Funkhouser, “A benchmark for 3D mesh segmentation,” ACM TOG, 28(3):1–12, 2009.
9. A. Golovinskiy and T. Funkhouser, “Randomized cuts for 3d mesh analysis,” ACM TOG, 7(5), 2008.
10. A. Golovinskiy and T. Funkhouser, “Consistent segmentation of 3D models,” Computers and Graphics, 33(3):262–269, 2009.
11. R. Gonzalez and R. Woods, “Digital image processing”, Addison-Wesley Publishing Company, 1992.
12. M. Hilaga, Y. Shinagawa, T. Kohmura, and T. L. Kunii, “Topology matching for fully automatic similarity estimation of 3D shapes,” in SIGGRAPH, pages 203–212, 2001.
SAI-KEUNG WONG, JAU-AN YANG, TAN-CHI HO AND JUNG-HONG CHUANG
13. T. C. Ho, “Slice-driven shape analysis and geometry processing of 3d models,” Ph.D. thesis, The National Chiao Tung University, Taiwan, R.O.C., 2011.
14. E. Kalogerakis, A. Hertzmann, and K. Singh, “Learning 3D mesh segmentation and labeling,” ACM TOG, 29(4):1–12, 2010.
15. S. Katz, G. Leifman, and A. Tal, “Mesh segmentation using feature point and core extraction,” The Visual Computer, 21(8-10):649–658, 2005.
16. S. Katz and A. Tal, “Hierarchical mesh decomposition using fuzzy clustering and cuts,” ACM TOG, 22(3):954–961, 2003.
17. V. Kraevoy and Sheffer, “Cross-parameterization and compatible remeshing of 3d models,” ACM TOG, 23(3):861–869, 2004.
18. Y.-K. Lai, S.-M. Hu, R. R. Martin, and P. L. Rosin, “Fast mesh segmentation using random walks,” in ACM Symp. on Solid and Phys. Modeling, pages 183–191, 2008.
19. Y. Lee, S. Lee, A. Shamir, D. Cohen-Or, and H.-P. Seidel, “Mesh scissoring with minima rule and part salience,” Computer Aided Geometric Design, 22(444–465), 2005.
20. R. Liu and H. Zhang, “Mesh segmentation via spectral embedding and contour analysis,” Computer Graphics Forum, 26(3):385–394, 2007.
21. M. M¨uller, B. Heidelberger, M. Teschner, and M. Gross, “Meshless deformations based on shape matching,”
ACM TOG, 24(3):471–478, 2005.
22. E. Praun, W. Sweldens, and P. Schr¨oder, “Consistent mesh parameterizations,” in SIGGRAPH, pages 179–
184, 2001.
23. J. Schreiner, A. Asirvatham, E. Praun, and H. Hoppe, “Intersurface mapping,” ACM TOG, 23(3):870–877, 2004.
24. A. Shamir. A survey on mesh segmentation techniques,” Computer Graphics Forum, 27(6):1539–1556, 2008.
25. L. Shapira, S. Shalom, A. Shamir, R. H. Zhang, and D. Cohen-Or, “Contextual part analogies in 3d objects,”
Int’l Journal of Comp. Vision, 89(2–3), 2010.
26. L. Shapira, A. Shamir, and D. Cohen-Or, “Consistent mesh partitioning and skeletonisation using the shape diameter function,” The Visual Computer, 24(4):249–259, 2008.
27. S. Shlafman, A. Tal, and S. Katz, “Metamorphosis of polyhedral surfaces using decomposition,” Computer Graphics Forum, 21(3):219–228, 2002.
28. P. Simari, D. Nowrouzezahrai, E. Kalogerakis, and K. Singh, “Multi-objective shape segmentation and labeling,” Computer Graphics Forum, 28(5), 2009.
29. H. Sundar, D. Silver, N. Gagvani, and S. Dickinson, “Skeleton based shape matching and retrieval,” in Shape Modeling International, page 130, 2003.
30. T. Tung and F. Schmitt, “Augmented reeb graphs for content-based retrieval of 3D mesh models,” in Shape Modeling International, pages 157–166, 2004.
31. O. van Kaick, A. Tagliasacchi, O. Sidi, H. Zhang, D. Cohen-Or, L. Wolf, and G. Hamarneh, “Prior knowledge for part correspondence,” Computer Graphics Forum, 30(2), 2011.
32. O. van Kaick, H. Zhang, G. Hamarneh, and D. Cohen-Or, “A survey on shape correspondence,” in Eurographics State-of-the-art Report, 2010.
33. Y. Wang and J. Zheng, “Tubular triangular mesh parameterization and applications,” Computer Animation and Virtual Worlds, 21(2):91–102, 2010.
34. Y. Xu, B. Wang, W. Liu, and X. Bai, “Skeleton graph matching based on critical points using path similarity,”
in Asian Conference on Computer Vision, pages 456–465, 2009.
35. H. Zhang, A. Sheffer, D. Cohen-Or, Q. Zhou, O. van Kaick, and A. Tagliasacchi, “Deformation-driven shape correspondence,” in Symposium on Geometry Processing, pages 1431–1439, 2008.
1