Chapter 5 Object Reconstruction
5.2 Point Cloud Processing
The real environment (Fig. 5.2(a)) is complex and the depth sensor with some noise makes the point cloud have too much rough information (Fig. 5.2(b)). Many point cloud processing methods can be used to deal with the point cloud set. Then, the useful information is extracted to reconstruct the object.
(a) (b)
Fig. 5.2 3D Imaging (a) Real environment, (b) point cloud from the depth sensor.
5.2.1 Pass Through Filter
The raw data of point cloud contains too many unnecessary points, as illustrated in Fig. 5.3. The necessary points are kept in the red rectangular. The point cloud is a data structure that contains all of the point information like position in space or color from the depth sensor. The pass through filter sets the range to keep the points in it. Those points out of the range will be removed from the point cloud. The range is ad-hoc variable in each experiment. In Fig. 5.3, we set setting the range
2 x 2 2 y 2 0.6 z 1.2
, and the output of point cloud only keeps thepoints in the red rectangular (Fig. 5.4).
Fig. 5.3 Raw data of point cloud set
Fig. 5.4 The output of pass through filter
5.2.2 Down-Sampling
The number of points in a given point cloud is not always optimal. Too many points provide no additional information because of noise or inaccuracy of the capture.
On the other hand, too few points prevent the use of some surface reconstruction methods. The process of artificially decreasing a number of points in the point cloud is called down-sampling. Adding or removing random points is not of course an option, so there are many methods to achieve a desirable outcome. One down-sampling method uses volume pixels (voxels) to create a grid of the given data points. Once the grid has been created, the down-sampling algorithm approximates a point in the middle of each
fewer points close to regular distances. Therefore the cloud will have less noise, more consistency and will be easier to process for many surface construction methods (Fig.
5.5(b)).
There are 78335 points in Fig. 5.5(a). After down-sampling, then remain 9853 points in the point cloud. A rough geometry of the objects shown in Fig. 5.5(a) can be quickly reconstructed based on down-sampling algorithm, as shown in Fig. 5.5(b).
(a) (b)
Fig. 5.5 Down-sampling algorithm (a) original point cloud, (b) after down-sampling.
5.2.3 Random Sample Consensus
Random Sample Consensus (RANSAC) method can be used to search model hypotheses like a plane, and will generate the plane coefficients, which can help separate the plane from the objects on the plane. The algorithm is given as follows 1. Randomly select three non-collinear unique points
p p pi, j, k
from point cloud P.2. Compute the model coefficients from the three points (ax by cz d 0).
3. Compute the distances from all pP to the plane model ( , , ,a b c d ).
4. Count the number of points p*P whose distance d to the plane model falls between 0 d dt , where d represents a user specified threshold. t
The last step represents one of the ways of scoring a specific model. Every set of points p is stored, and the above steps are repeated for k iterations. The detail *
about selecting k is given in [46]. After the algorithm is terminated, the set with the largest number of points (inliers) is selected as the support for the best planar model found. The RANSAC algorithm is applied to the same scenario as Fig. 5.5(a). The result is shown in Fig. 5.6(b), and the plane parameters are
a b c d
0.0231 0.9734 0.2278 0.2359
. In robot grasping planning we only care about the object on the table. Using the plane parameters can simply keep the points on the plane. The next issue is how many objects on the table, so next section will discuss the method of segmentation.(a) (b)
Fig. 5.6 RANSAC algorithm (a) Original point cloud, (b) The plane search by RANSAC.
5.2.4 Euclidean Segmentation
After separating the objects from table, next problem is how to detect the number of the objects. The simple method uses the Euclidean distance to cluster the object, and the method is called Euclidean Segmentation. The algorithm is given below
1. Create a KD-tree representation for the input point cloud dataset P.
2. Set up an empty list of clusters C , and a queue of the points that need to be checked Q .
3. Then for every point piP, perform the following steps:
Add to the current queue
For every point piQ do:
- Search for the set Pik of point neighbors of p in a sphere with radius i
rdth.
- For every neighbor pikPik, check if the point has already been processed, and if not, add it to Q .
When the list of all points in Q has been processed, add Q to the list of clusters C , and reset Q to an empty list.
4. The algorithm terminates when all points piP have been processed and are now part of the list of point clusters C .
KD-tree is the data structure which can help to speed up (O
log( )n
) the search the neighbor of each point in the point cloud. It is one of the multidimensional binary search trees [10]. The output of the algorithm is shown in Fig. 5.7(b), where different color represent different objects. The defection of the algorithm is when two different objects are too close and the Euclidean distance is smaller than a threshold such that they are considered as the same object. This algorithm is a simple method for segmentation. There are higher level algorithms that consider about the color of a point to cluster the point cloud set.(a) (b)
Fig. 5.7 Segmentation (a) Original point cloud, (b) after segmentation.