Foundational Correspondences

The foundational correspondences are a set of one-to-one corresponding points ob-tained from the stereo contours. The correspondences represent the corresponding relationship between the two contours and are used in the CECT algorithm as refer-ence points for locating further correspondrefer-ences in the region enclosed by the edge contour. To obtain the foundational correspondences, the edge contours of the hu-man back are extracted from the stereo images and the contour in the left image is partitioned into a pre-determined number of sampling points. The corresponding sampling points on the contour extracted from the right image are then obtained by applying the uniqueness constraint and the epipolar constraint. Briefly, the unique-ness constraint ensures that all the correspondences lie on both stereo contours, while the epipolar constraint restricts the search range for the corresponding sample point in the right image to the epipolar line.

Extraction of Edge Contours

To extract the edge contour, it is first necessary to segment the region of the human back from the captured image. In the CECT algorithm, this is achieved using the detection method proposed in [65,66]. Specifically, the skin region of the image is detected using a skin classifier based on the spatial distribution of the skin color characteristics in the YCbCr color space. The segmented results obtained using the skin classifier are noisy and contain many small redundant segments. Therefore, a maximum connected component algorithm is applied to extract the main segments from the image. Morphological closing and opening operations are then performed to fill the cavities caused by spots in the skin and artefact noise. Finally, the edge contour of the skin segment is extracted using the Canny Edge Detector [59]. Fig. 3.2 presents a schematic illustration of the edge contour extraction process.

(a) (b) (c)

Figure 3.2: (a) Captured image. (b) Image following skin-color segmentation. (c) Extracted edge contour of back.

(a) (b)

Figure 3.3: Schematic illustration of correspondence detection in CECT algorithm. Note that the blue line is the epipolar line corresponding to point c

^𝐿

^𝑅

^𝐿

Obtaining Foundational Correspondences

In establishing the foundational correspondences, two constraints are applied, namely the uniqueness constraint and the epipolar constraint. The uniqueness con-straint states that for an existing world point, its corresponding projected image point must be found in both the left image and the right image. The CECT algo-rithm operates on contours (silhouettes) of the human back observed by two cameras with specific viewpoints. In the proposed vision-based massage machine, the view-points of the two cameras are very similar, and thus the silhouettes observed by these two cameras are effectively the same. In other words, the image contours captured by the two cameras are projected from the same silhouette, and thus the uniqueness constraint is tenable. That is, the correspondence of any point on the left contour should lie exactly on the right contour, and vice versa.

The epipolar constraint limits the search space for the correspondences between the two stereo images to one dimension. In stereovision, the epipolar geometry defines the correspondence relations between the left image and the right image and is represented algebraically by the fundamental matrix F [35]. According to the epipolar constraint, given a point x^𝐿 in the left image, its correspondence x^𝑅 in the right image must lie on the epipolar line l^𝑅, where the epipolar line is defined as

l^𝑅 = Fx^𝐿. (3.1)

Fig. 3.3 presents an illustrative example of the correspondence matching process performed by the CECT algorithm. Suppose that c^𝐿 is a point located on the contour in the left image. In accordance with the uniqueness constraint and the epipolar constraint, its corresponding point, c^𝑅, must lie on both the edge contour in the right image and the epipolar line. In other words, the corresponding point is located at the intersection of the epipolar line and the edge contour.

As discussed above, the foundational correspondences used in the CECT al-gorithm are a set of one-to-one point correspondences along the pair of contours extracted from the images captured by the stereo camera. In computing the foun-dational correspondences, the edge contour extracted from the left image is parti-tioned into 𝑁 sampling points which collectively form the set C^𝐿= {c^𝐿₁, . . . c^𝐿_𝑁}. As shown in Fig. 3.4(a), the sampling points are equally distributed along the contour.

Moreover, points with odd indices are located on the left side of the contour, while points with even indices are located on the right. Thus, c^𝐿₁ and c^𝐿₂ represent the two extreme points of the shoulder, while c^𝐿_{𝑁 −1}and c^𝐿_𝑁 represent the two extreme points of the waist. Similarly, C^𝑅 = {c^𝑅₁ . . . c^𝑅_𝑁} represents the set of the corresponding sampling points of the edge contour in the right image, which are obtained using the method described above. Take c^𝐿_𝑖 as an example. Its correspondence c^𝑅_𝑖 is found at the intersection of the epipolar line l^𝑅_𝑖 = Fc^𝐿_𝑖 and the edge contour in the right image. Having determined all of the points in C^𝐿and C^𝑅, the matching pairs within the two sets constitute the foundational correspondences set, defined as

Ω ={︀ℱ𝒞𝑖 =(︀c^𝐿_𝑖, c^𝑅_𝑖 )︀ | c^𝐿_𝑖 ∈ C^𝐿 and c^𝑅_𝑖 ∈ C^𝑅, 𝑖 ∈ {1 . . . 𝑁 }}︀

(3.2) where ℱ 𝒞_𝑖 is the 𝑖-th foundational correspondence pair.

(a) (b)

Figure 3.4: (a) Sampling points along contour extracted from left image. (b) Corresponding points obtained along contour extracted from right image.