A geometry-based error estimation for cross-ratios
夽
J.-S. Liu, J.-H. Chuang*
Department of Computer and Information Science, National Chiao Tung University, 1001 Ta Hsueh Rd, Hsinchu, Taiwan 30050, ROC
Received 17 March 2000; accepted 19 October 2000
Abstract
For choosing speci"c cross-ratios as 2D projective coordinates in various computer vision applications, a reasonable error analysis model is usually required. This investigation adopts the assumption of normal distribution for positioning errors of point features in an image to formulate the error variances of cross-ratios. Based on a geometry-based error analysis, a straightforward way of identifying the cross-ratios with minimum error variances is proposed. Simulation results show that the proposed approach, as well as a further simpli"ed alternative, yield much better estimations of minimum error variances in terms of accuracy, cost, and stability compared with some other methods, e.g., the one based on the rule given by Georis et al. (IEEE Trans. Pattern Anal. Mach. Intell. 20 (4) (1998) 366). Some causes of the performance di!erences in the estimations are explained using a special con"guration of point features. 2001 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.
Keywords: Error analysis; Cross-ratio; Computer vision; 3D reconstruction
1. Introduction
Recently, more and more computer vision researchers have been paying attention to error analysis so as to ful"ll the accuracy requirements arising from various applications such as outer-space exploration, industrial robots, and so on. In fact, one of the main purposes of computer vision is to construct a reliable system that can carry out its tasks with satisfactory e$ciency and pre-cision in a realistic environment. Early works regarding these requirements are mainly concerned with the analy-sis of error propagation, which are well known in the photogrammetry literature [2}4], and thereby provide relevant information of quality estimation for di!erent steps of a vision algorithm [5]. In particular, such an analysis is often required for 3D shape reconstruction methodologies. There are basically two classes of methods to reconstruct 3D shapes from 2D images. The
夽This work was supported by the National Science Council,
Republic of China, under grant NSC88-2213-E009-062.
* Corresponding author: Tel.: 3-573-1979; fax:
#886-3-572-1490.
E-mail address: jhchuang@cis.nctu.edu.tw (J.-H. Chuang).
"rst class involves strategies relying on camera calib-ration [6}8] and the second consists of methods based on projective geometry, which usually utilize reference points as prior knowledge [9}13]. Due to the simplicity, some of the projective geometry-based approaches have also been used in other applications [14}17].
Consider the projective geometry-based approaches for 3D reconstruction. In Ref. [9], it is shown that refer-ence points in a sequrefer-ence of images can be used easily to derive 3D information of objects in a scene. It is also found in Ref. [10] that, given more locations of epipoles, in addition to only four corresponding reference points, a projective invariant structure can be established to reconstruct a 3D scene without any prior knowledge of camera geometry or internal calibration. Subsequently, a relatively a$ne structure is proposed with one of its applications being the basis for algorithms performing 3D reconstruction from multiple views [11]. Similarly, geometric constructive solutions to 3D vision problems, e.g., positioning a point in the 3D space using two stereo images, are reported in Refs. [12,13].
For the projective geometry-based 3D reconstruction relying on reference points, the quality of the reconstruc-tion strongly depends on that of the image data. In addition to other possible measurement uncertainties,
0031-3203/01/$20.00 2001 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved. PII: S 0 0 3 1 - 3 2 0 3 ( 0 0 ) 0 0 1 7 4 - 6
Detailed description of the criterion is given in Section 3.
Note that the de"nition of cross-ratio is not unique. Di!er-ent cross-ratios can be obtained by reordering the four points in Eq. (1). The de"nitions will give a total of six di!erent cross-ratio values and any one of the values can be used to derive the rest of them [21].
2D coordinates of feature points in an image plane will always have quantization errors due to the limited image resolution. Hence, the projective coordinates, i.e., pairs of cross-ratios with respect to the reference points, will also have error in their values. Such errors must be carefully analyzed and controlled so as not to seriously in#uence the "nal reconstruction results.
Several researchers consider the error analysis for cross-ratios obtained for point feature with respect to four reference points [18,19]. The analyses are based on the assumption of an independent, identical, Gaussian distribution of errors in the locations of the four image points in an image plane. A given image of four collinear points is "rst classi"ed by making comparisons between the measured cross-ratio and those stored in the model database. Subsequently, the performance of the classi"ca-tion is described quantitatively by the probabilities of rejection, false alarm and misclassi"cation. Recently, a complete sensitivity analysis of the 3D reconstruction method based on projective geometry has been presented in Ref. [1] in which the error estimation for the projective coordinates, i.e., cross-ratios, is considered. As one of the main results, it is suggested by the authors that, instead of expensively calculating the error variances of all 24 cross-ratios associated with a 2D feature point, one could choose the cross-ratios which minimize!903. How-ever, such a geometrically phrased criterion is only based on observations on limited number of examples and lacks obvious mathematical support. In this paper, a new approach of estimating error variances of cross-ratios is proposed. It is shown that with a more clear geometric interpretation of the mathematical formulation of the error variances, the proposed approach will perform the error estimation more satisfactorily in terms of accuracy, cost and stability.
The rest of the paper is organized as follows. In Section 2, we give an overview of the projective geometry-based 3D reconstruction. Subsequently, the er-ror generated in the output of the "rst stage of the reconstruction, i.e., the projective coordinates, due to quantization errors in locations of image points is for-mulated in Section 3. In Section 4, the geometry-based mathematical reasoning of the above error is carried out and, accordingly, a new error estimation approach is established. Furthermore, by simplifying the estimation process, a low-cost alternative is also introduced. In Section 5, simulations are performed using a typical con"guration of a set of four reference points as well as a special con"guration wherein three of the four reference points are nearly collinear. Experiments using a real image are also carried out for the latter. Finally, we draw conclusions in Section 6.
2. Projective geometry-based 3D reconstruction
In this section, we brie#y review some mathematics involved in the projective geometry-based 3D recon-struction approach. Let J , K , ¸ and M be four col-linear points, as shown in Fig. 1; their cross-ratio is de"ned as
k"[J, K;¸, M]"J¸ ) KMK¸ ) JM , (1)
where J¸ stands for the directed distance from J to ¸, and so on. In fact, the cross-ratio is the basic invariant in projective geometry: all other projective invariants can be derived from it [20].
Let l denote the line containing the four points with the line equation
r"b#d, (2)
where r is the position vector of any point on l, b and d are the base and directional vectors of the line, respec-tively, and is a parameter taking real values. If (, ), * and + are the -parameters associated with
J, K, M and ¸, respectively, the cross-ratio de"ned in
Eq. (1) can also be expressed as
k"[J, K;¸, M]"(!*)!*)!+(!+ . (3)
An immediate application of the invariant property of cross-ratio is to locate a point on a line. For example, assuming that cross-ratio k is given, so are the locations of J, K, ¸ on l, it is easy to see from Eq. (3) that the location of M can be determined as
+"k(()!*)!)((!*)k()!*)!((!*) . (4) Therefore, k can be regarded as the 1D projective
coordi-nate of M with respect toJ, K, ¸.
On the other hand, the same idea can be extended to locate a 2D point P, also shown in Fig. 1, in a projective plane P. This can be done by using two cross-ratios as follows. Assume that points A, B, C and D are given, and so is an arbitrarily chosen line l. The location of M, and thus line l, can be obtained using Eq. (4) if k is given. Similarly, l can be determined if another cross-ratio, say
k, is given for the intersections of another arbitrarily
chosen l (not shown) and the four lines (including
Fig. 1. De"nition of cross-ratio and its application in "nding 1D (2D) location of point M (P).
Fig. 2. Reconstruction of a 3D point using projective coordi-nates.
obtained as the intersection of l and l. Therefore, (k,k) can be regarded as the 2D projective coordinates of P with respect to A,B, C,D.One main application of the projective coordinates is to reconstruct the world coordinates of a 3D point from its projective images. To see the reconstruction proced-ure, consider the example shown in Fig. 2. Assume that two reference planes R and R and the 3D locations of four reference points on each of them are given (only
A, B, C and D on R are shown). Let p and p be two
images of the feature point P and the two viewing lines intersect R and R at P, P and P,P, respective-ly. Since the projective coordinates of P can be obtained from a, b, c, d and p in the left image plane, the location of P on R and thus its 3D location can be determined. Similarly, the 3D location of P, can also be calculated. In the same way, from the right image plane, the 3D locations of P and P can also be obtained. Finally, the 3D location of the feature point P in the world coordi-nates system can be determined as the intersection of the two viewing lines, PP and PP.
Thus, for a 3D feature point P, the projective ge-ometry-based 3D reconstruction approach using two ref-erence planes can be summarized with the following procedure:
The line intersection (LI) procedure:
Stage 1. Calculate the 2D projective coordinates of P in
the left and the right image, respectively, for each refer-ence plane.
Stage 2. Calculate the 3D locations of the two images
of P on each reference plane using the 2D projective coordinates computed in Stage 1.
Stage 3. Reconstruct P as the intersection of two
view-ing lines.
In the above reconstruction procedures, quantiza-tion errors in locating point features in the image planes will result in errors in the location of recon-structed object points. However, there will be di!erent error ampli"cation e!ects on di!erent projective co-ordinates used in Stage 1 of procedure ¸I. In the next section, an error analysis based on a normal distribution assumption for the above quantization error will be pre-sented.
3. Error analysis
We now present an analysis of the errors in the projec-tive coordinates due to quantization errors in locating point features in 2D images. Such coordinates are useful in projective geometry-based computer vision, e.g., in the reconstruction procedure reviewed in the previous sec-tion. Consider the left image shown in Fig. 2. Let j, k, l and m (not shown) be the projective projections of
J, K, ¸ and M (see also Fig. 1), respectively. Moreover,
let bH#dH denote the line equation of lH (the image of
l) for the 2D coordinates system used in that image,
where bH"(bH
V, bHW) and dH"(dHV, dHW). The intersection of lH
and ab can be obtained as
H"aVbW!aWbV#bHWbV!bHWaV!bHVbW#bHVaWdH V(bW!aW)!dHW(bV!aV)
, (5) where (aV,aW) and (bV,bW) are the coordinates of points
a and b, respectively. With similar computations,I, J
and K can also be obtained. By using H, I, J and K in place of (, ), * and +, respectively, in Eq. (3), we have
k"(aVbW!aWbV#bVdW!bWdV#dVaV!dWaV)(aVcW!aWcV#cVdW!cWdV#dVaV!dWaV)
;(aVcW!aWcV#cVpW!cWpV#pVaW!pWaV) (aVbW!aWbV#bVpW!bWpV#pVaW!pWaV). (6)
In particular, we have 䉭(A, B, C)"䉭(B, C, A)" 䉭(C, A, B)"!䉭(A, C, B)"!䉭(C, B, A)"!䉭(B, A, C). Table 1
Cross-ratios associated with a point p and four reference pointsa, b, c, d
i Cross-ratio kG Dep. i Cross-ratio kG Dep.
1 [ab, ac, ad, ap]"䉭(a, b, d)䉭(a, c, p)
䉭(a, c, d)䉭(a, b, p) t 13 [cb, ca, cd, cp]"
䉭(c,b,d)䉭(c,a,p)
䉭(c,a,d)䉭(c,b,p) tt 2 [ab, ad, ac, ap]"䉭(a, b, c)䉭(a, d, p)
䉭(a, d, c)䉭(a, b, p) 1!t 14 [cb, cd, ca, cp]"
䉭(c, b, a)䉭(c, d, p)
䉭(c, d, a)䉭(c, b, p) t!tt 3 [ac, ab, ad, ap]"䉭(a, c, d)䉭(a, b, p)
䉭(a, b, d)䉭(a, c, p) 1
t 15 [ca, cb, cd, cp]"
䉭(c, a, d)䉭(c, b, p)
䉭(c, b, d)䉭(c, a, p) tt 4 [ac, ad, ab, ap]"䉭(a, c, b)䉭(a, d, p)
䉭(a, d, b)䉭(a, c, p) t!1t 16 [ca, cd, cb, cp]"
䉭(c, a, b)䉭(c, d, p)
䉭(c, d, b)䉭(c, a, p) t!tt 5 [ad, ac, ab, ap]"䉭(a, d, b)䉭(a, c, p)
䉭(a, c, b)䉭(a, d, p) t!1t 17 [cd, ca, cb, cp]"
䉭(c, d, b)䉭(c, a, p)
䉭(c, a, b)䉭(c, d, p) t!tt 6 [ad, ab, ac, ap]"䉭(a, d, c)䉭(a, b, p)
䉭(a, b, c)䉭(a, d, p) 1 1!t 18 [cd, cb, ca, cp]" 䉭(c, d, a)䉭(c, b, p) 䉭(c, b, a)䉭(c, d, p) t!tt 7 [ba, bc, bd, bp]"䉭(b, a, d)䉭(b, c, p) 䉭(b, c, d)䉭(b, a, p) t 19 [db, dc, da, dp]" 䉭(d, b, a)䉭(d, c, p) 䉭(d, c, a)䉭(d, b, p) t!tt!1 8 [ba, bd, bc, bp]"䉭(b, a, c)䉭(b, d, p) 䉭(b, d, c)䉭(b, a, p) 1!t 20 [db, da, dc, dp]" 䉭(d, b, c)䉭(d, a, p) 䉭(d, a, c)䉭(d, b, p) t!1t!1 9 [bc, ba, bd, bp]"䉭(b, c, d)䉭(b, a, p) 䉭(b, a, d)䉭(b, c, p) 1 t 21 [dc, db, da, dp]" 䉭(d, c, a)䉭(d, b, p) 䉭(d, b, a)䉭(d, c, p) t!tt!1 10 [bc, bd, ba, bp]"䉭(b, c, a)䉭(b, d, p) 䉭(b, d, a)䉭(b, c, p) t!1t 22 [dc, da, db, dp]" 䉭(d, c, b)䉭(d, a, p) 䉭(d, a, b)䉭(d, c, p) t!tt!1 11 [bd, bc, ba, bp]"䉭(b, d, a)䉭(b, c, p) 䉭(b, c, a)䉭(b, d, p) t!1t 23 [da, dc, db, dp]" 䉭(d, a, b)䉭(d, c, p) 䉭(d, c, b)䉭(d, a, p) t!tt!1 12 [bd, ba, bc, bp]"䉭(b, d, c)䉭(b, a, p) 䉭(b, a, c)䉭(b, d, p) 1 1!t 24 [da, db, dc, dp]" 䉭(d, a, c)䉭(d, b, p) 䉭(d, b, c)䉭(d, a, p) t!1t!1
In this table, Dep. indicates the dependency between the twenty four de"nitions of cross-ratio; for example, if k is repre-sented by t, then k can be obtained by 1!t.
To give a geometric interpretation to the above equa-tion, de"ne the triangle function
䉭(A, B, C)O (AVBW!AWBV#BVCW!BWCV #
CVAW!CWAV), (7)
whose magnitude,䉭(A, B, C), gives the area of triangle 䉭ABC. With this de"nition, we can rewrite Eq. (6) as
k"䉭(a, b, d) 䉭(a, c, p)䉭(a, c, d) 䉭(a, b, p). (8)
Since six di!erent cross-ratios can be obtained for each of the four reference points being the origin of the pencil of the reference lines going through (i) the other three refer-ence points and (ii) the measured point p, a total of 24
di!erent cross-ratios kG 1)i)24 can be used for p, as listed in Table 1. In theory, if the projective coordinates, as well as other relevant quantities, are calculated pre-cisely, 3D object points can be reconstructed perfectly with procedure ¸I. However, quantization errors are intrinsic to locations of point features extracted from an image, which will result in errors in the computation of projective coordinates as well as in the reconstruction. Often, the normal distribution is utilized to model such quantization errors, which is usually assumed to have a zero mean and the following covariance matrix:
VV VW VW WW.k aV"
(䉭(a, b, d)䉭(a, c, p)) (䉭(a, b, p)䉭(a, c, d))!(䉭(a, b, p)䉭(a, c, d))(䉭(a, b, d)䉭(a, c, p)) (䉭(a, b, p)䉭(a, c, d))
"[(bW!dW)䉭(a,c, p)#(cW!pW)䉭(a,b, d)]䉭(a,b, p)䉭(a,c,d) (䉭(a, b, p)䉭(a, c, d))
![(bW!pW)䉭(a, c,d)#(cW!dW)䉭(a, b,p)]䉭(a,b,d)䉭(a,c,p)
(䉭(a, b, p)䉭(a, c, d)) . (10)
Accordingly, the variance of cross-ratio k, for example, can be calculated as I I"
kaV #k bV #k cV #k dV #k pV VV #k aW #k bW #k cW #k dW #k pW WW #2k aV kaW#kbV kbW#kcV kcW #k dV kdW#kpV kpWVW OeI VVV#eIWWW#eIVWVW, (9)where eIV, eIW, and eIVWare de"ned as the error
ampli"-cation factors forVV, WW, and VW, respectively. Thus, given the repetition/symmetry of Table 1, and the prob-ability that in any practical application all measured points will have equal error, di!erent cross ratios (kG's) will statistically have di!erent error variances (IJ
IG)
ac-cording to Eq. (9). Speci"cally, I
J IG is equal to the
weighted sum of VV, WW and VW with the weighting factors equal to non-identical eIGV, eIGW, and eIGVW,
respec-tively. For simplicity, for locating of a point feature in an image, it is assumed that the correlation between the inaccuracies in x and y directions can be ignored, i.e., VW0. Consequently, only the two ampli"cation factors,
eIV and eIW, will need to be considered in Eq. (9).
Note that, even though the above error analysis is formulated for k, a similar analysis for other projective coordinates can also be performed. In general, the ampli-"cation factors may vary widely for di!erent cross-ratios being used as projective coordinates, e.g., in Stage 1 of procedure ¸I. In Ref. [1], instead of calculating all the error variances associated with the 24 cross-ratios and then choosing the ones with the smallest variance, it is suggested that cross-ratios which minimize !903
should be used. According to their de"nition, for each of the 24 cross-ratios listed in Table 1,
"Loop,
where o, o3a, b, c, d, oOo, and 䉭(o, o, p) is the second triangle function in the denominator. For example, L
CAP"82.83 in Fig. 1 and will minimize !903.
Therefore, either k or k shown in Table 1 should be used. However, the authors simply use an example to suggest that a cross-ratio thus obtained is`more likelya to be robust, without giving any obvious mathematical support.
4. Proposed algorithms for error estimation
As noted previously, the calculation for all the ampli"-cation factors is time-consuming and a quick estimation of their relative magnitude is usually desired in time-limited situations such as in real-time computer vision, etc. In this section, we will suggest di!erent ways of estimating the minimum error ampli"cation factors based on a geometry-based analysis of Eq. (9). It is shown in Section 5 that the estimation approaches pre-sented in this paper can generate better estimates than the afore mentioned!903 criterion with less compu-tation.
4.1. Dexnition and a geometry-based error analysis
To estimate the relative magnitude of error ampli"ca-tion factors for di!erent cross-ratios, let us "rst consider the factor eIV in Eq. (9). For the "rst partial derivative
given in Eq. (9), we have
Similar expressions can be obtained for other partial derivatives. Subsequently, the ampli"cation factor eIVin
Eq. (9) can be evaluated as
eIV"
1
[䉭(a, b, p)䉭(a, c, d)]
;[[(bW!dW)䉭(a,c,p)#(cW!pW)䉭(a,b,d)] ;䉭(a, b, p)䉭(a, c, d)
The notations 䉭GN, 䉭GN ,䉭GN and 䉭GN , which are identical to
kG?, kG@, kGA and kGB de"ned in Ref. [1], are introduced here to
make easy the mathematical reasoning for proposed approaches to estimating minimum ampli"cation factors of error variances for the cross-ratios listed in Table 1.
Due to its complex computation, similar rules based on the numerator part are yet to be developed.
Table 2
De"nitions of䉭GN, 䉭GN ,䉭GN, and 䉭GN for all kG's
i 䉭GN 䉭GN 䉭GN 䉭GN i 䉭GN 䉭GN 䉭GN 䉭GN
1 䉭(a, b, p) 䉭(a, c, d) 䉭(a, c, p) 䉭(a, b, d) 13 䉭(c, b, p) 䉭(c, a, d) 䉭(c, a, p) 䉭(c, b, d)
2 䉭(a, d, c) 䉭(a, d, p) 䉭(a, b, c) 14 䉭(c, d, a) 䉭(c, d, p) 䉭(c, b, a)
3 䉭(a, c, p) 䉭(a, b, d) 䉭(a, b, p) 䉭(a, c, d) 15 䉭(c, a, p) 䉭(c, b, d) 䉭(c, b, p) 䉭(c, a, d)
4 䉭(a, d, b) 䉭(a, d, p) 䉭(a, c, b) 16 䉭(c, d, b) 䉭(c, d, p) 䉭(c, a, b)
5 䉭(a, d, p) 䉭(a, c, b) 䉭(a, c, p) 䉭(a, d, b) 17 䉭(c, d, p) 䉭(c, a, b) 䉭(c, a, p) 䉭(c, d, b)
6 䉭(a, b, c) 䉭(a, b, p) 䉭(a, d, c) 18 䉭(c, b, a) 䉭(c, b, p) 䉭(c, d, a)
7 䉭(b, a, p) 䉭(b, c, d) 䉭(b, c, p) 䉭(b, a, d) 19 䉭(d, b, p) 䉭(d, c, a) 䉭(d, c, p) 䉭(d, b, a) 8 䉭(b, d, c) 䉭(b, d, p) 䉭(b, a, c) 20 䉭(d, a, c) 䉭(d, a, p) 䉭(d, b, c) 9 䉭(b, c, p) 䉭(b, a, d) 䉭(b, a, p) 䉭(b, c, d) 21 䉭(d, c, p) 䉭(d, b, a) 䉭(d, b, p) 䉭(d, c, a) 10 䉭(b, d, a) 䉭(b, d, p) 䉭(b, c, a) 22 䉭(d, a, b) 䉭(d, a, p) 䉭(d, c, b) 11 䉭(b, d, p) 䉭(b, c, a) 䉭(b, c, p) 䉭(b, d, a) 23 䉭(d, a, p) 䉭(d, c, b) 䉭(d, c, p) 䉭(d, a, b) 12 䉭(b, a, c) 䉭(b, a, p) 䉭(b, d, c) 24 䉭(d, b, c) 䉭(d, b, p) 䉭(d, a, c) ! [(bW!pW)䉭(a,c, d)#(cW!dW)䉭(a, b,p)] ;䉭(a, b, d)䉭(a, c, p)] # [(dW!aW)䉭(a,c, p)䉭(a,b,p)䉭(a,c,d) !
(pW!aW)䉭(a, c,d)䉭(a,b, d)䉭(a,c, p)] #
[(pW!aW)䉭(a,b,d)䉭(a, b, p)䉭(a,c,d) !
(dW!aW)䉭(a, b, p)䉭(a,b, d)䉭(a,c,p)] # [(aW!bW)䉭(a,c, p)䉭(a,b,p)䉭(a,c,d) ! (aW!cW)䉭(a,b,p)䉭(a,b, d)䉭(a,c, p)] # [(aW!cW)䉭(a, b, d)䉭(a,b,p)䉭(a,c,d) !
(aW!bW)䉭(a, c,d)䉭(a,b, d)䉭(a,c, p)]. (11) As can be seen in Eq. (8), the denominator part of the cross-ratio k corresponds to the product of signed areas of two triangles. Let䉭N (䉭N ) denote the signed area with (without) p as a vertex of the triangle, i.e., 䉭N"䉭(a,b,p) and 䉭N "䉭(a, c, d). Similarly, in the numerator, let 䉭N"䉭(a,c,p) and 䉭N " 䉭(a, b, d). (See Table 2 for the de"nitions for all k
G's.)
Accordingly, Eq. (11) can be expressed in a simpler form as eIV" 1 䉭N 䉭N ;[[(bW!dW)䉭N#(cW!pW)䉭N ]䉭N䉭N ! [(bW!pW)䉭N # (cW!dW)䉭N]䉭N 䉭N] # [(dW!aW)䉭N!(pW!aW)䉭N ] (䉭N 䉭N) # [(pW!aW)䉭N ! (dW!aW)䉭Nt] (䉭N䉭N ) # [(aW!bW)䉭N ! (aW!cW)䉭N ] (䉭N䉭N) # [(aW!bW)䉭N!(aW!cW)䉭N] (䉭N 䉭N ). (12) Expressions of similar form can also be derived for the ampli"cation factor eIW, as well as for the ampli"cation
factors for other kG's, 1)i)24.With the 1/(
䉭N䉭N ) term in Eq. (12), it is very
much likely that the kG which has the maximum value of 䉭GN 䉭GN will have the minimum eIVand eIW. Such an
observation motivates approaches proposed in the fol-lowing subsections for a quick identi"cation of minimum error ampli"cation factors, and the corresponding cross-ratios.
4.2. Maximum denominator method
According to the geometry-based error analysis pre-sented in the previous subsection, we now propose the
"rst algorithm, namely the maximum denominator (MD) method. The MD method identi"es, for each point fea-ture, the cross-ratio with the maximum denominator magnitude
kG, i"arg
maxWGW䉭GN 䉭GN
(13) as the one with the smallest error ampli"cation factors
eIGV and eIGW. (Assume VV"WW, the minimization of
eIGV#eIGW is considered.) In fact, it is not necessary to
check all 24 cross-ratios since, as one can see from the third and the last column (dependency) of Table 1, they can be divided into pairs of cross-ratios with each pair having identical behavior in error ampli"cation; hence, only 12 cross-ratios are to be examined. Thus, for each point feature under consideration, the cross-ratio with minimum error ampli"cation factors is identi"ed by the following procedure (assuming that only odd i's are taken into account).
The MD procedure:
Step 1. Identify the cross-ratio
kG, i"arg
maxWGW䉭G\N 䉭G\N
(14) as the one with smallest error ampli"cation factors
eIGV and eIGW.
For some situations which are not uncommon, on the other hand, we can consider䉭GN's and 䉭GN 's separately for more e$cient computation, as will be discussed next.
4.3. Two-step method䉭GN then 䉭GN
To accelerate the estimation of error ampli"cation factors based on Eq. (14), the second method, namely the two-step (TS) method, is developed based on the follow-ing observations. Consider the 12䉭GN 䉭GN 's in Eq. (14) calculated for each feature point. If the four reference points are selected in advance, the䉭GN 's will be "xed. Furthermore, the䉭GN 's will not be very di!erent from one another if locations of the reference points are chosen properly, e.g., the points are distributed fairly symmetric-ally with respect to their centroid. On the other hand, the 䉭GN's may vary signi"cantly because, in principle, a fea-ture point p can appear in any location in an image. The above observations suggest that the identi"cation of the maximum denominator may be achieved approximately by evaluating䉭GN's "rst and then 䉭GN 's.
According to the properties of the triangle function, de"ned in Eq. (7), only six di!erent䉭GN's, each represent-ing two cross-ratios, need to be considered. For example, we have䉭N"䉭N for k and k from Table 2. Ac-cordingly, the TS method "rst calculates six di!erent values of䉭GN and identi"es two of the 12 cross-ratios having the maximum䉭GN. Subsequently, one of the two cross-ratios which has larger䉭GN 䉭GN is identi"ed as
the one with minimum error ampli"cation factors. In summary, we have the following procedure to identify the cross-ratio with minimum error ampli"cation factors.
The TS procedure:
Step 1. Calculate the six䉭GN's, i.e., for i"1,3,5,9,11
and 17.
Step 2. For the maximum 䉭GN obtained in Step 1,
identify the cross-ratio
kG, i"arg ( maxG$H
䉭GN 䉭HN
(䉭GN , 䉭HN )) (15) as the one with smallest error ampli"cation factors
eIGVand eIGW.
Consider the time complexity of the three methods. The!903 criterion suggested in Ref. [1] requires the computation of 12's for comparison. Simpli"cation of the estimation is possible by considering the value of cos "
oo ) opoo op
. (16)
Thus, a total of 68 multiplication and 12 division opera-tions is required. On the other hand, the MD method requires the evaluation of 12 cross-ratio denominators, which involve the calculation of ()"10 cross-products, or 20 multiplication operations equivalently, to "nd 䉭N's and 䉭N 's, and additional 12 multiplication opera-tions to "nd the 12 䉭N 䉭N 's. Finally, the TS method checks 6 cross products in the "rst step, and then 2 more in the second, which require a total of 16 multiplication operations.
5. Simulation results
This section reports the simulation results obtained with estimation methods, MD and TS, proposed in the previous section, as well as the method suggested in Ref. [1], denoted here as orthogonal method (OR). Two simulations are provided to examine their performances under di!erent conditions. Fig. 3 shows a 500;500 im-age of reference points used in the two simulations. While a, b, c,d, representing a typical geometry of reference points, are used in the "rst simulation, point c is re-placed with c in the second simulation in which three of the reference points are nearly collinear.
5.1. Simulation 1 * a typical situation
In this simulation, locations of the four reference points, a"(109, 112), b"(96, 285), c"(365,390) and
d"(312, 227), are assumed to be identical to that shown
in "g. Fig. 6 of Ref. [1]. The error variance is considered for the projective coordinates of feature points pG's located on a regular grid of size 21;21"441. For example, p"(0,0), p"(0, 25), and so on.
Fig. 3. The 500;500 image of reference points used in the two experiments.
Fig. 4. The minimumIG
IG estimated by the three estimation methods for the 441 feature points.
For simplicity, assume thatVV"WW"1 and VW0. The error variances of kG, similar to that formulated in Eq. (9) for k, can be expressed as
IG IG"eIGV#eIGW. (17)
Accordingly, the 12 error variances,I
G\IG\, 1)i)12,
can be calculated and the minimum error variance, de-noted asK, can be identi"ed for each feature point.
Without carrying out the complex calculation of all 12 variances, the three estimation methods (MD, TS and OR) are applied, respectively, to choose a kG which is supposed to have the minimum IG
IG for each feature
point. Fig. 4 shows the minimumI
GIG estimated by the
above three methods, denoted asK +", K 21 and K -0, respectively, for the 441 feature points. One can see
clearly that while MD and TS methods both yield low error variances with hardly noticeable di!erences, the minimum error variances estimated by the OR method are signi"cantly higher. For example, a peak can be observed for the OR method for p"(325,225), which is located very close to d. Since Lc dp is the closest to 903, k with a very small 䉭GN"䉭dcp is chosen accord-ing to the!903 criterion, resulting in a large error variance. Similar explanations can be given for peaks associated with p, p, etc., obtained from the OR method.
Such a performance di!erence could be explained part-ly as follows. For any 2D feature point p, the error variances associated with the 12 kG's depend on the coor-dinates of four reference points and that of p itself, i.e., there are ten variables (aV, aW, . .., dV,dW and pV, pW) which need to be considered. Without an obvious mathematical support, the !903 criterion adopted in the OR method can hardly capture the complex e!ects due to so many variables. On the other hand, the denominator part of cross-ratio adopted in the proposed methods exhibits itself as a reasonable and e$cient indication of the rela-tive magnitude of error variances expressed in Eq. (9).
From another perspective, the above results can also be demonstrated directly on the image plane. As the basis for performance comparison, Fig. 5(a) shows the 21;21 minimum error varianceK obtained through direct cal-culation, e.g., using Eq. (9) for k. The gray value is made to vary logarithmically with the darkest (brightest) gray level denoting the value of 10\ (10\ ). Simi-larly, Fig. 5((b)}(d)), show the di!erences, K -0!K, K +"!K and K 21!K, respectively, with the darkest (brightest) gray level denoting the di!erence of 10\ (10\). It is readily observable from these di
!er-Fig. 5. Gray-level images representing the minimum error vari-ances and the di!erences between these minimum and those obtained with OR, MD and TS methods, respectively. (a)K, (b) K -0!K, (c) K +"!K, (d) K 21!K.
Fig. 6. The total number of the 21;21 locations for which the cross-ratio with the smallest (black), the second smallest (gray) and the third smallest (white) error variances will be selected.
In theory, the cosine function given in Eq. (16) will yield the same magnitude for both k and k. Hence, the selection bet-ween the two is simply a consequence of numerical di!erences.
ences that (i) the deviations fromK are much higher for the OR method than the other two, and (ii) the MD and TS methods have very similar behaviors in estimating K in terms of the magnitude of estimation error as well as the distribution of such error in di!erent image loca-tions. While (i) is consistent with the results shown in Fig. 4, (ii) is due to the fact that, in principle, the TS method is a simpli"cation of the MD method with minor modi"cations.
In general, for each feature point, besides the kG with the smallest error variance (K), those with slightly higher error variances may also be used as projective coordi-nates to generate satisfactory reconstruction results. Fig. 6 shows for each of the three estimation approaches, the total number of the 21;21 locations for which the cross-ratio with the smallest (black), the second smallest (gray) and the third smallest (white) error variances will be selected. While the MD method results in good esti-mates for most locations, the OR results correspond to the least, as expected.
To investigate the behavior of the OR method further, a special con"guration of reference points will be con-sidered in the following example. Such an example also partly addresses the issue of the selection between the MD and TS methods under di!erent conditions.
5.2. Simulation 2 * a special case
In Fig. 3, consider the set of reference points which is slightly di!erent from those used in Simulation 1 in that
c is replaced with c.The situation corresponds to a
special con"guration of the reference points, i.e., a, b and
c, are near-collinear in the projective image. Figs. 7 and
8 show simulation results similar to Figs. 4 and 5, respec-tively, obtained in Simulation 1. The error variances in Fig. 7 are shown in logarithmic scale to accommodate dramatically increased dynamic range of the error vari-ances obtained with the OR method. Notice that ex-tremely large error variance values occur for a subset of the "rst 170 feature points. On the other hand, the dy-namic ranges for the MD and TS methods seem to be similar to that shown in Fig. 4. In Fig. 8(a), the minimum error variances are shown to have a di!erent distribution of overall increased values due to the change of the con"guration of the reference points. For the results shown in Figs. 8(b)}(d), observations similar to that for Figs. 5(b)}(d) can be obtained except for (i) the extremely large values resulting from the OR method (as mentioned earlier), and (ii) a bigger di!erence between the MD and TS methods compared with the results shown in Fig. 5. For a more in-depth analysis of (i), Fig. 9 extracts the error ampli"cation zone (K -0*1) from Fig. 8(b) and shows the cross-ratio, explicitly by its index, chosen by the OR method at each grid location in the zone. Clearly, the zone can be divided into three parts. For example, for the darkest (black) area shown in Fig. 9, which includes a region near the line passing through point b and per-pendicular to bd, Ldbp is the closest to 903 compared to the other 11's; hence, according to the OR estimation,
cross-Fig. 7. The minimumIG
IG similar to Fig. 4, estimated by the three estimating methods for the 441 feature points, for simulation 2.
Fig. 8. Gray-level images, similar to those shown in Fig. 5, obtained for simulation 2. (a)K, (b) K -0!K, (c) K +"!K, (d)K 21!K.
Fig. 9 . The cross-ratio chosen by the OR method, at each grid location in the error ampli"cation zone.
ratios correspond to the worst choices. This is because their denominators also contain a 䉭GN (䉭(b, c, a) or 䉭(b, a, c), see Table 2) with extremely small magnitudes due to the near collinearity, resulting in unacceptably large error variances. Similar observations can be made for other feature points shown in Fig. 9 which are located near the line passing through point a (c) and perpen-dicular to ad (cd) for which cross-ratio k or k (k or
k) will be selected. In contrast, since both 䉭N and 䉭N
are considered, an extremely largeK does not appear in the MD or the TS results. For example, for the TS method, we have max䉭N"䉭(b,d,p)"䉭(d,b,p) for p; however, since 䉭(b,c,a)0, the TS method will choose䉭N"䉭(d,b,p). Therefore, either k or k will be selected.
The reason for (ii) is that, while the MD method takes into account 12 䉭N 䉭N 's, the TS method considers only 6䉭N's and then proceeds with the estimation only for the max䉭N. It is easy to see that such a strategy
Fig. 10. Simulation results, similar to that shown in Fig. 6, obtained for simulation 2.
Fig. 12 . Errors in values of cross-ratios corresponding toK +",K 21,K -0, respectively, calculated for the images of the 441 grid points shown in Fig. 11.
Fig. 11. A 642;1024 image of the 21;21 grid points and the four reference points marked with squares.
will reduce the overall searching space and, somehow in the special case considered in Simulation 2, results in a larger performance di!erence between the two methods. Fig. 10 shows the results similar to that shown in Fig. 6. If one only considersK +" and K 21 (black areas), an improvement of the MD results as well as a degeneration of the TS results can be seen clearly. Even so, on the grounds that both K +" and K 21 do not exceed 10\ at all times, as shown in Fig. 7, both methods are considered as generating satisfactory results. Nevertheless, deciding whether the MD or the TS method should be adopted under di!erent conditions is not trivial in general and requires further investigations.
5.3. Experiments using real images
In our work, experiments using real images are carried out to verify the simulations performed in the previous subsections. For brevity, only the experiments for Simu-lation 2 are presented in this subsection. Fig. 11 shows a 642;1024 image of the 21;21 grid points and the four reference points marked with squares. Locations of all these points with image pixel precision are obtained with ordinary point detection algorithm.
For each grid point, cross-ratios which are identi"ed in the previous subsection to have K +", K 21, K -0, re-spectively, are calculated. Fig. 12 shows errors in these cross-ratio values compared with their theoretical values for all the grid points. It is readily observable that the errors are fairly consistent with the statistical results shown in Fig. 7. Similar results, which are not included here for brevity, are also obtained for images of the same scene taken from other viewpoints.
6. Summary
This paper concerns error analysis for perspective projection-based 3D shape reconstruction. Based on a normal distribution assumption, formulation of error variances for di!erent cross-ratios with respect to a set of reference points utilized in the reconstruction is estab-lished. According to a geometry-based mathematical rea-soning, the proposed error estimation approach suggests that the cross-ratios with the maximum denominator magnitudes, i.e., the maximum products of areas of two corresponding triangles, will lead to the minimum ampli-"cations of error variance. Subsequently, the MD pro-cedure for error estimation is developed. Compared with the OR method suggested in Ref. [1], the proposed approach generates better results for an ordinary con"g-uration of four reference points as well as for a special one. In the latter case, three reference points are nearly collinear. A brief explanation is also given for the signi"-cantly worse results obtained with the estimation method presented in Ref. [1] for such a special case.
As an alternative to the MD method which already requires less computation than the OR method, the more e$cient TS method is also introduced. The latter is a straightforward simpli"cation of the former which con-siders the area of one triangle, instead of the product of two triangles, at a time and e!ectively halves the compu-tation costs. However, as for the trade-o! between the two methods, it is not trivial to decide whether the MD or the TS method should be adopted under di!erent conditions, and this requires further investigations. References
[1] N. Georis, M. Petrou, J. Kittler, Error guided design of a 3D vision system, IEEE Trans. Pattern Anal. Mach. Intell. 20 (4) (1998) 366}379.
[2] C. Slama (Ed.), The manual of photogrammetry, The American Society of Photogrammetry, Falls Church, VA 22046, 1980.
[3] E. Mikhail, Observations and Least Squares, IEP * A Dun-Donnelley Publisher, New York, 1976. [4] K.R. Koch, Parameter estimation and hypothesis testing
in Linear Models, Springer, Berlin, 1987.
[5] R.M. Haralick, Propagating covariance in computer vi-sion, Proceedings of the 12th IAPR, Jerusalem, Israel, 9}13 October, IEEE CS Press, Los Alamitos, CA, 1994, pp. 493}498.
[6] R.Y. Tsai, A versatile camera calibration technique for high-accuracy 3D machine vision metrology using o!-the-shelf TV cameras and lenses, IEEE J. Robot. Automat. 3 (4) (1987) 323}344.
[7] A. Cumani, A. Guiducci, A new camera calibration method for high accuracy non-contact metrology, Pattern Recognition Lett. 14 (1993) 415}419.
[8] Z.Q. Hong, J.Y. Yang, An algorithm for camera calib-ration using a three-dimensional reference point, Pattern Recognition 26 (1993) 1655}1660.
[9] J.J. Koenderink, A.J. van Doorn, A$ne structure from motion, Tech. Rep., Utrecht University, Utrecht, The Netherlands, October 1989.
[10] A. Shashua, Projective structure from uncalibrated images: structure from motion and recognition, IEEE Trans. Pat-tern Anal. Mach. Intell. 16 (8) (1994) 778}790.
[11] A. Shashua, N. Navab, Relative a$ne structure: theory and application to 3D reconstruction from perspective views, IEEE Trans. Pattern Anal. Mach. Intell. 18 (9) (1996) 873}883.
[12] R. Mohr, L. Morin, Relative positioning from geometric invariants, Proceedings of the Conference of Computer Vision and Pattern Recognition, 1991 pp. 134}144. [13] R. Mohr, E. Arbogast, It can be done without camera
calibration, Pattern Recognition Lett. 12 (1991) 39}43. [14] S. Carlsson, Projectively invariant decomposition and
recognition of planar shapes, Proceedings of the Inter-national Conference of Computer Vision, 1993, pp. 471}475.
[15] H. Chabbi, M. Berger, Recovering Planar Surfaces by Stereo Vision Based on Projective Geometry, Tech. Rep. 93-R-054, CRIN/CNRS-INRIA, Lorraine, 1993. [16] R.T. Collins, R.J. Beveridge, Matching perspective
views of coplanar structures using projective unwarp-ing and similarity matchunwarp-ing, Proceedunwarp-ings of the Confer-ence of Computer Vision and Pattern Recognition, New York, IEEE CS Press, Los Alamitos, CA, June 1993, pp. 240}245.
[17] O.D. Faugeras, What can be seen in three dimension with an uncalibrated stereo rig?, Proceedings of the Second European Conference of Computer Vision, Santa Mar-gherita Ligure, Italy, May 1992, pp. 563}578.
[18] S.J. Maybank, Probabilistic analysis of the application of the cross ratio to model based vision: misclassifcation, Int. J. Computer Vision 14 (1995) 199}210.
[19] S.J. Maybank, Probabilistic analysis of the application of the cross ratio to model based vision, Int. J. Computer Vision 16 (1995) 5}33.
[20] N. E"mov, Advanced Geometry, MIT, Moscow, 1978. [21] R. Mohr, Projective geometry and computer vision,
Handbook Pattern Recognition Comput. Vision, (1993) 269}293.
About the Author*JAIN-SHING LIU was born in Taipei, Taiwan, in 1970. Currently, he is working toward the Ph.D. degree in Department of Computer and Information Science, National Chiao Tung University, Hsinchu, Taiwan. His current research interests include 3-D modeling, computer vision and image processing.
About the Author*JEN-HUI CHUANG (S'86}M'91) received the B.S. degree in electrical engineering from National Taiwan University, Taipei, Taiwan, ROC, in 1980, the M.S. degree in electrical and computer engineering from the University of California at
Santa Barbara in 1983, and the Ph.D. degree in electrical and computer engineering from the University of Illinois at Urbana-Champaign in 1991.
Between 1983 and 1985, he was a Design and Development Engineer with the LSI Logic Corporation, Milpitas, CA. Between 1989 and 1991, he was a Research Assistant with the Robot Vision Laboratory, Beckman Institute for Advanced Science and Technology, University of Illinois, Champaign, IL. Since August 1991, he has been on the faculty of the Department of Computer and Information Science, National Chiao Tung University, Hsinchu, Taiwan. His research interests include 3-D modeling, computer vision, speech and image processing and VLSI systems.
