Chapter 7 Human Modeling and Display of Human Activities
7.2 Human Modeling from Single KINECT Device
7.2.1 Review of Distance-weighted Correlation (DWC)
The measure of distance-weighted correlation (DWC) was proposed by Fan and Tsai [12] originally for automatic Chinese seal identification. The measure is defined as the minimum distance between two groups S and T of pixels of seal imprint images after the two seal imprint images are overlapped. For each pixel s in group S, we will find out a pixel t in group T which has the minimum distance to pixel s. If the pixel t is in a limited circular region C with a pre-selected radius K, then a weight
69
where the coefficient 1/2 is included to treat S and T symmetrically and NS and NT are the total numbers of pixels in S and T, respectively. It can be verified that 0CK 1, k-dimensional space. It is a useful data structure for searching nearest points with high dimensions on the tree. The K-d tree is also a kind of binary tree in which every node is a k-dimensional point. We use a group of k-dimensional points to construct the K-d tree. Every non-leaf node on the K-d tree can be considered as a splitting plane, which is perpendicular to the axis of one of the k-dimensions, to divide the spatial domain into two parts. Points to the left of this splitting plane are represented by the left subtree of that node and points to the right are represented by the right subtree. When we want to search the closest point in a group of points, it will take less time to search a K-d tree of the group rather than to search the whole group directly. The time complexity of searching becomes O(N1-1/K) instead O(N) where the K is the number of dimensions and is usually greater than one. So when we run algorithms using the DWC, we can convert one of the input groups of points into the K-d tree form. It will reduce the time for the step of searching points of the minimum DWC.
7.2.3 Modeling by Speeded-up DWC Using K-d Tree
We will use sets of constructor sequences described in Section 7.1 to build up several models. For each set of constructor sequences, we convert all depth images in the constructor sequences into the human’s depth images by the method described in
70
Section 6.2.2. In this way, we can get sets of sequences of 3D human images. We then take one of the sets of the sequences of the 3D human images as a human modeling example. Because the sequence of 3D human images is recorded in accordance with the time sequence, each human’s 3D image in the sequence is located at a different position with a small distance from each other. We want to find some transformations which can be used to merge every two consecutive 3D human images in the sequence.
And then, we extend these transformations to merge all 3D human images to construct a human model. We will use the DWC measure and the K-d tree structure to assist finding these transformations. An algorithm for finding such transformations between 3D human images by speeding up the DWC computation using the K-d tree is shown as follows.
Algorithm 7.1: Finding transformations between 3D human images.
Input: a sequence of 3D human images I0, I1, …, IN, where N is the total number of sequences of 3D human images; a prepared set of transformations Tis; the threshold value K used in the computation of the DWC measure; an initial maximum value M; a counter with its value C set to be 0 initially; an initial transformation T0. IKDT in the K-d tree structure form.
Step 3. Apply a transformation Ti, which is not used yet in the prepared
71
transformation set, to the 3D human image IC+1 and get a 3D image IT.
Step 4. Take the two 3D images IKDT and IT and the threshold value K as input data for computing the DWC between them.
Step 5. Start the DWC computation to get a DWC value RV.
Step 6. If the value RV is greater than the maximum value M, then update the maximum value M with the value RV and the desired transformation RTC
with the transformation Ti.
Step 7. If the transformations Tis are not exhausted yet, then repeat Steps 2 through 6; else, go to the next step.
Step 8. Increment the value C of the counter by 1.
Step 9. If the value C is smaller than N, then repeat Steps 1 through 8; else, exit.
7.2.4 Modeling Algorithm
After executing the algorithm for finding transformations described above in Section 7.2.3, we get a set of transformations, by which we can start to merge all 3D human images in the sequence. First, we use the first 3D human image as a pivot. And then, merge the other 3D human images into the first one. Finally, we get a human model. The merging algorithm is as follows.
Algorithm 7.2: merging 3D human images.
Input: the sequence of 3D human images I0, I1, …, IN, where N is the total number of the sequences of 3D human images; the transformations RT0, RT1, …, RTN1 obtained from Algorithm 7.1, where RTj is the transformation which, when applied to 3D human image Ij+1, will merge 3D human image Ij+1 into 3D human image Ij where j = 0, 1, …, N-1; a counter with its value C set to be 0 initially.
72
Output: the merging result MR.
Steps:
Step 1. Use the first 3D human image I0 as a pivot.
Step 2. Put 3D human image I0 into the merging result MR.
Step 3. Merge the transformations RT0, RT1, …, RTC to get a merged transformation MRTC.
Step 4. Apply the transformation MRTC to 3D human image IC+1 to get an integrated 3D human image TIC.
Step 5. Put the resulting 3D human image TIC into the merging result MR.
Step 6. Increment the value C of the counter by 1.
Step 7. If the value C is smaller than N, then repeat Steps 3 through 6; else, go to the next step.
Step 8. Take the final merging result MR as the desired human model.