In order to derive a more consistent corresponding set, two spatial constraints are used to remove possibly incorrect correspondence. The initial correspondences step provides a set of corresponding points. The 3-D position in the camera coordinate system can be calculated by combining 2-D position and depth information. The 3-D feature points in the previous and current camera coordinate system are denoted by Pit−1 and Pit, with corresponding index i and time index t− 1 and t. The motion transformation between two camera coordinate systems is denoted by a rotation matrix R and a translation vector T . The mathematical relation is expressed in equation (3.1). The geometrical bias due to noise in point position is symbolised by bgi.
Pit−1 = RPit+ T + bgi with i = 1...n (3.1) It is expected that there are a large number of outliers in the initial correspondences.
The noise term bgiwould become a critical issue for these outliers, the calculation of R and T would derive a bad estimated motion. In most papers, statistics method ( e.g. RANSAC ) is a common way to calculated R and T in the presence of these outliers. However, there are many outliers would be involved in the calculation of transformation. Therefore, we exploit the spatial constraints to efficiently remove most of these outliers, before R and T are calculated.
3.3.1 Relative Distance Constraint
The first constraint is the relative distance between two 3-D points should remain unchanged under a rigid transformation. Therefore, the distance between any two points dpikt−1 =||Pit−1− Pkt−1|| should be the same for the corresponding points dpikt =||Pit− Pkt||.
Hence, we set a small d represents the difference between two relative distance. All of the feature correspondences should satisfy inequality ( 3.2 ).
Hirschmuller et al. [19] have showen that how models the image based errors and
feature correspondences should satisfy inequality ( 3.2).
||dpikt−1− dpikt|| ≤ 3d with i, k = 1...n (3.2)
(a) (b)
Figure 3.8: The relative distance between Pi and Pk should be the same at time t and t− 1 ( i.e. the length of orange line keeps unchanged ). Similarly, the gray line also keeps unchanged.
3.3.2 Rotation Angle Constraint
The second constraint is derived by setting an upper bound θ on the rotation angle of the camera between consecutive images. The rotation angle is limited by a high frame rate (e.g. 30 fps). That is, camera wouldn’t suffer a large rotation movement within 1/30 sec (i.e. between two consecutive images). Mathematically, all rotations in three dimensions can be represented by a rotation axis and a rotation angle. If the vector Pit−1− Pkt−1
is orthogonal to the rotation axis, then the angle between Pit−1− Pkt−1 and Pit− Pkt
matches exactly the rotation angle. If Pit−1− Pkt−1 is parallel to the rotation axis, then the angle between Pit−1− Pkt−1 and Pit− Pktis always 0. In all other cases, the angle is in between 0 and the rotation angle. The rotation axis is unknown, but the angle is limited by θ as described above. Therefore, all pairs of correct correspondences must
satisfy inequality (3.3).
(Pit−1− Pkt−1)T(Pit− Pkt)
dpikt−1dpikt ≥ cosθ with i, k = 1...n (3.3)
(a) (b)
Figure 3.9: A vector ⃗a = Pit−1− Pkt−1 is formed by two 3-D points Pit−1 and Pkt−1 at time t− 1. This vector becomes vector ⃗b = Pit− Pktby suffering a transformation at time t (left). The rotation angle constraint is setting a limitation of the angle θ between vector ⃗a and ⃗b (right).
3.3.3 Consistent Subset of Correspondences
The initial correspondence of matching pairs can be validated by check both con-straints. If there is a pair (i, k) of corresponding matching pairs (Pit−1, Pit) and (Pkt−1, Pkt) satisfies both constraints, then both correspondences may be correct. If one of the constraints doesn’t satisfy, then at least one of the correspondences in the pair is abso-lutely wrong. Although we cannot determine that which one would be the wrong cor-respondence. However, these information can be used to construct a consistent table of corresponding points. Each entry in the table represent whether the corresponding match-ing pairs satisfy both constraints. If some pairs satisfy both constraints , then the entry will equal to one. Otherwise, entry in the table will be filled in zero. Figure 3.10 visualizes an example of consistent table. After we construct a consistent table, our goal is to find a
Figure 3.10: An example of the consistent table.
Theoretically, we want to find out the largest subset would involve testing all the combinations of n points with 2n possibilities, but it is obvious that this task is not be a practical issue. Actually, in real usage, it is not necessary to find the largest subset, but only a large enough subset. The first thing is selecting the point i with the largest number of consistencies (i.e. the column i with the largest number of 1-elements) and putting into the consistent subset. Then, selecting another point into subset insures that the consisten-cies are maintained for all points in the subset. Repeating this procedure until there are no other points can be added into subset. In the Figure 3.11, a 5× 5 consistent table example is given to demonstrate how to find the consistent subset by using the scheme mentioned above.
Although we perform an outlier detection procedure, it may remain incorrect cor-respondences. However, the remaining outliers also satisfies the distance and rotation constraints ( 3.2 ) and ( 3.3 ). We can expect that the influence of remaining outliers would be small. Therefore, these errors can be well handled during calculation of R and T . The results of original matching and matching after removing outliers are shown in Figure 3.12.
(a) A 5× 5 A consistent table which the en-tries have been filled with 0 or 1.
(b) First, we select the column with most 1-elements ( second column in this example ).
(c) Adding the second corresponding pair into subset.
(d) Finding next corresponding pair ( e.g.
third pair ) is consistent with second pair.
(e) Checking the consistencies between third pair and all pairs in subset.
(f) If the consistencies satisfied, then adding the third pair into subset.
(g) Repeating the same steps, then the forth pair is also added into subset.
(h) Next, finding corresponding pair ( e.g.
fifth pair ) is consistent with second pair.
(i) Checking the consistencies between fifth pair and all pairs in subset.
(j) Since the consistencies don’t satisfy, the fifth pair is not added into subset.
Figure 3.11: A 5× 5 example to show how to finding the consistent subset.
(a) original matching result
(b) matching result after outlier detection
Figure 3.12: feature matching results