CHAPTER 2 BACKGROUND
2.1 Skeleton extraction
Nowadays, the applications of skeleton extraction are wide-ranging, as we can see such applications in medical treatment [2], animation [3], shape recognition and analysis, etc.
Some studies focused on the detection of (visual) objects in certain environments [4] [5] [6], and some others focused on the reconstruction of (visual) objects [7] [8] [9]. These studies are based on the fact that the full shape of an object is known. However, sometimes there may be noise in data or the data may not be complete. For example, the information may be lost during the process of data transmission. Even though some studies considered noisy data, many of them put emphasis on reconstructing the surface of an object [10] [11] [12]
rather than extracting the skeleton of an object. As an example, Jankowski and Stanberry proposed a method to identify the skeleton in noisy data [13], but the purpose of their method is to try to find the centerline curve in noisy data. In this thesis, we focus on generating the skeleton of an object represented by a set of data points some of which contain noisy values, and the generated skeleton can be further used to reconstruct the surface of an object.
In this chapter, we are going to give background information and discuss related studies.
We will start from a discussion about skeleton and skeleton extraction, then a discussion about point cloud, and finally a discussion about noise and noise handling.
2.1 Skeleton extraction
The study of the extraction of skeleton can be dated back to 1960s. The definition of
11
‧
skeleton can be found in a number of papers. Below are some examples: According to Bucksch and Lindenbergh [14], “A skeleton is defined as a graph containing geometric information with its vertices and edges.” For a (solid) shape, a skeleton is a “compact and effective representation” that contains information about the geometry and topology of the shape [15]. Described in the paper written by Barnachon et al. [16], skeleton can be used to
“make an efficient and compact representation of a shape.”
The types of skeleton can be roughly divided into two categories, and Rossi and Torsello described these two types as follows [17]: “The curve skeleton provides a minimal yet efficient representation for shape analysis and recognition. The medial surfaces, on the other hand, carry enough information to accurately reconstruct the original shape from the skeleton.”
Skeleton in this thesis refers to the curve skeleton. Au et al. defined curve skeletons as follows [18]: “Curve-skeletons are 1D structures that represent a simplified version of the geometry and topology of a 3D object.” Moreover, according to Cornea, Silver, and Min [19], “The curve-skeleton captures the essential topology of the underlying object in an easy to understand and very compact form.” Curve skeletons have been used in various applications. For instance, according to Cornea, Silver, and Min [19], “Curve-skeletons are thinned 1D representations of 3D objects useful for many visualization tasks including virtual navigation, reduced-model formulation, visualization improvement, animation, etc.”
Additionally, Au et al. stated that “the extraction of curve-skeletons from 3D models is a fundamental problem in computer graphics and visualization” [18]. Moreover, Sobiecki et al. described that [20], “Curve skeletons are among the most well-known, and widest used, descriptors for 3D shapes.”
The applications of skeleton are very broad. As an example, Menier, Boyer, and Raffin used skeleton in body pose recovery [21]. Lee et al. used the extracted skeleton in 3D
12
‧
pipeline reconstruction [22]. Skeleton extraction has also been used in many important applications in various fields of science and engineering. For example, as described by Wang et al. [23], “Extracting curve skeletons for vascular structures is vital for many medical applications.” Skeleton is a widely-used to represent or describe shapes for 2D or 3D shape recognition [17]. It can be used in human action recognition [24] or motion analysis [25].
Furthermore, according to Tagliasacchi, Zhang, and Cohen-Or [26], “The skeleton of a shape, especially an articulated shape such as a human or animal, provides an intuitive and effective abstraction which facilitates shape understanding and manipulation.”
In this thesis, we mainly deal with the skeleton of an object or a 3D model in the form of a point cloud. The skeleton extraction for objects or 3D models represented as point clouds has attracted much research attention for decades, and each of the approaches proposed by researchers has its own characteristics (strengths and weaknesses), such as the robustness to noise [22].
There are two methods to generate the skeleton of a model: Volumetric methods and geometric methods. Volumetric methods require a voxelization of the model while geometric methods work on meshes or point clouds [1]. The algorithm that we use in this thesis to generate the skeleton of a model is the Laplacian-based Contraction [1], which is an algorithm working on point cloud data without the reconstruction of the surface. The
contraction method was first proposed by Au et al. [27]. It was used on triangle meshes then, and it was later extended to be used on point cloud data in the paper [1]. In the algorithm, the input model, or the input point cloud, is contracted and topologically thinned to reduce the skeleton to a line. Because the contraction process depends on a local analysis, a
topology-preserving procedure is used in order to avoid the incorrect topology structure [28].
The procedure of the algorithm is as follows: Given a point cloud P = {𝑝𝑝𝑖𝑖}, it first
13
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
contracts P to a zero-volume point set C; next, it builds a skeleton graph G using sub-samples of C; it then contracts edges of G iteratively in order to build a skeleton T, which is a curve skeleton with cycles; finally, in order to restrict T to the interior of P as much as possible, it moves every vertex of T to the center of its local neighborhood in P [1]. An example demonstrating the process of the algorithm is shown in Figure 5.
Figure 5. An example illustrating how the algorithm works [1].
The algorithm can be further divided into two parts: geometric contraction and topological thinning.
The geometric contraction was first proposed in [27], it uses an implicit Laplacian
smoothing process to choose anchor points for maintaining the global shape of the input model [29][30], so that the contracted point set C captures the topological characteristics of the original model with minimal value. As stated in [27], the contraction is computed by solving Eq. (1) iteratively for the vertex positions:
� 𝑊𝑊𝐿𝐿 𝐿𝐿
𝑊𝑊𝐻𝐻 � 𝑃𝑃′ = � 0𝑊𝑊𝐻𝐻𝑃𝑃� (1)
In Eq. (1), L is a n*n Laplacian matrix with cotangent weights; 𝑃𝑃′ is a contracted point cloud;
14
‧
W𝐿𝐿 and W𝐻𝐻 are the diagonal weighting matrices that balance the contraction and attraction constraints, and the i-th diagonal element of W𝐿𝐿 (W𝐻𝐻) is denoted by W𝐿𝐿,𝑖𝑖 (W𝐻𝐻,𝑖𝑖) [1]. The solution to Eq. (1) is equivalent to minimizing the quadratic energy:
‖𝑊𝑊𝐿𝐿𝐿𝐿𝑃𝑃′‖ + ∑ 𝑊𝑊𝑖𝑖 𝐻𝐻,𝑖𝑖2 ‖𝑝𝑝𝑖𝑖′− 𝑝𝑝𝑖𝑖‖2 (2)
In Eq. (2), the first term removes geometry details using implicit Laplacian smoothing and the second term preserves shape geometry during contractions [1]. Because solving the above equation will not collapse the model into a 1-dimentional shape, it requires a number of iterations for the process to converge into a thin shape. The iterative contraction process is as follow, where the superscript t denotes the iteration number:
1. Solve �𝑊𝑊𝐿𝐿𝑡𝑡𝐿𝐿𝑡𝑡
𝑊𝑊𝐻𝐻𝑡𝑡 � 𝑃𝑃𝑡𝑡+1= � 0𝑊𝑊𝐻𝐻𝑡𝑡𝑃𝑃𝑡𝑡� for 𝑃𝑃𝑡𝑡+1, 2. Update 𝑊𝑊𝐿𝐿𝑡𝑡+1= 𝑆𝑆𝐿𝐿𝑊𝑊𝐿𝐿𝑡𝑡 and 𝑊𝑊𝐻𝐻,𝑖𝑖𝑡𝑡+1= 𝑊𝑊𝐻𝐻,𝑖𝑖0 𝑆𝑆𝑆𝑆𝑖𝑖0
𝑖𝑖𝑡𝑡 where 𝑆𝑆𝑖𝑖𝑡𝑡 and 𝑆𝑆𝑖𝑖0 are the current and original neighborhood extent of point i, respectively,
3. Compute the new Laplacian operator 𝐿𝐿𝑡𝑡+1 with the current point cloud 𝑃𝑃𝑡𝑡+1 [1].
The first two images in Figure 5 show an example of the contraction process. As shown in the second image in Figure 5, after several iterations, we can obtain a set of point clouds that keep the original geometric structure but contain fewer points.
The result of the above contraction process is a point cloud set C, and there are some methods to convert a point cloud to a curve skeleton, such as [31] and [32]. In the algorithm, the conversion process is similar to the one discussed in the paper [33]: First, it imposes an
15
‧
initial connectivity to build a skeletal structure from C, and then it applies an edge-contraction operation, collapses unnecessary edges until no triangles exist when it builds a curve skeleton [1]. The corresponding example is shown in the last two images in Figure 5. In this thesis, we use the same method to generate the skeleton of the 3D models (with or without being processed by our preprocessing method) in our database.