Point-Based Methods

2.3.1 Region-Based Methods

Region-based methods usually represent the moving objects in an image as regions in one camera view and then compare the features of these regions with other regions in anther camera view to construct the correspondence relations. Color information is the popular feature used to establish the correspondence such as color histogram [21] or GMM color model [22]. Although this method is really simple, it is not reliable due to the variation of illumination. Also, the color information of objects in different camera views has an important impact on the correspondence result. For example, if two persons in different camera views dressed in the clothes with same color or two sides of clothes are different colors, it is easy to make mistakes in objects correspondence.

2.3.2 Point-Based Methods

Point-based methods construct the correspondence by comparing the features between different camera views based on some constraints of cameras. According to different geometric constraints, we can classify the methods into correspondence in 2-D domain and correspondence in 3-D domain.

The concept of correspondence in 3-D domain is shown in Figure 2-16. The left figure is an example of indoor surveillance system and the right figure shows the top view of this indoor environment where the cameras mounted on the ceilings in a circle. Each camera monitors one view and then transmits the 2-D information of that view to 3-D domain. By fusing all 2-D information coming from cameras of different views, we can roughly estimate the positions of objects in 3-D domain. The correspondence relations are able to be constructed by back-projecting the 3-D estimations into each 2-D image plane. However, this method needs the camera calibration before the system starts to work.

Figure 2-16 An example of indoor surveillance system[23]

Utsumi [23] finds the COG (center of gravity) of the detected objects in each 2-D image plane. The points inside a detected region are called COG if they have the longest distances to their closest boundaries As shown in Figure 2-17, the left is the original residual after detection and the right figure illustrates the distance map of the residual and the black points are so-called COG. How to find the correspondence relations by COGs? First of all, we project COGs from all camera views to 3-D domain and use the Gaussian distributions to estimate the positions of objects in 3-D domain. Through the back-projection of 3-D Gaussian distributions to all 2-D image planes, we can compare the detected COGs with the projected COGs in each camera view to find the possible correspondence relations.

Figure 2-17 An example of Utsumi’s method [23]

The left one is the original residual. And the right one is the distance map where the black points are so-called COG.

Figure 2-18 The illustration of correspondence in 3-D domain by Utsumi’s method[23]

The objects are modeled as Gaussian distributions with N

(

)

Black, Jamesa [24] uses the constraints of epipole plane to correspond objects in different camera views. As shown in Figure 2-19 it is an illustration of epipole plane where m1 and m2 are obtained by back-projecting the 3-D object located at M on image plane 1 and image plane 2 respectively. The back-projection lines, Mm1 and Mm2, will go through Es1 and Es2 which are camera focuses. The epipole plane constraint tells us that Es1, Es2 and M are in the same 2-D plane called epipole plane and m2 will lay on ep11 which is the intersection line of epipole plane and image plane 2.

Based on the epipole constraint, the center of each detected object in a 2-D image can project a line called epipole line on a 2-D image of another camera view and this epipole line will go through the center of the corresponding object belonging to the same 3-D object in that camera view. However, there may be some errors of 2-D tracking results. So the center of the corresponding object could not lay on the epipole line exactly. By combing this constraint and the 2-D tracking results, the task of multi-objects correspondence can be achieved.

Figure 2-19 The illustration of epipole constraint [24]

Es1,Es2 are focuses of camera1 and camera2, M is the position of object in 3-D domain ,m1,m2are the back-projection of M to image plane 1 and image plane 2 respectively, and ep11 is the intersection line of epipole plane and image plane 2.

Figure 2-20 Black, Jamesa’s correspondence method based on the epipole constraint [24]

There are epipole lines with different colors in four images above. T he color of the epipole lines of detected object in image1 is red and green, blue, and yellow are the colors of the epipole lines of image2, image3,and image4 respectively. The green rectangles in image and image4 are ground plane regions, and the yellow rectangles in image2 and image3 are occlusion plane regions.

The correspondence method in 2-D domain doesn’t need the calibrated cameras. It only needs some clues of 2-D images in different camera views to find the correspondence relations. S.

Khan [25] finds the overlap regions of current image and other images of different camera views.

Only when the moving object appears in such overlap regions, it has the correspondence relation to the objects detected in other camera views. As shown in Figure 2-20, the range of each camera view is shown in the left figure and the grey point represents the object. We can find that the object is visible in camera2’s view but invisible in camera3’s view. The right figure is camera1’s view and the region of other camera’s views projecting on it.

Figure 2-21 S. Khan’s correspondence in 2-D method [25]

The left figure illustrates the ranges of three camera views The right figures is the camera1’s view. There are some lines indicated the regions of the overlap views from other camera’s views. We can easily find that the black point in the bottom of object is visible in camera1’s view and camera2’s view but invisible in camera3’s view.

J. Black [26] is to find the homography matrix between two 2-D images of different camera views. With this matrix, we can transform any point of current camera view to the point of another camera view. This method is much easier than the method using the epipple constraint.

As shown in Figure 2-22, the blue lines are epipole lines and red points are the correspondence points using homography matrix transformation.

Figure 2-22 An example of J. Black’s method [26]

Viewpoint correspondence using: epipole line analysis and homography alignment

The tasks of multi-objects correspondence in both 2-D domain and 3-D domain are hard due to that the results are easier degraded due to the influence of noise or the inaccuracy of 2-D tracking results.