Optimal Shape Error Analysis of the Matching Images for Free-form Surface

* Corresponding author. Tel.: 2362-0032; fax:

#886-2-2364-1186.

E-mail address: fan@ccms.ntu.edu.tw(K.-C. Fan).

Optimal shape error analysis of the matching image for

a free-form surface

Kuang-Chao Fan

*, Tung-Hsien Tsai

Department of Mechanical Engineering, National Taiwan University, 1, Roosevelt Rd., Sec. 4, Taipei 10617, Taiwan, ROC

Department of Mechanical Engineering, Shu-Teh Junior College of Technology and Commerce, 11, Dah-ching St., Sec. 2, Taichung, Taiwan, ROC Received 1 April 2000; received in revised form 24 August 2000; accepted 15 September 2000

Abstract

When using an optical non-contact scanning system to measure an object that has a large surface, large curvature, or a full 3603 pro"le, one can acquire only one set of sectional measurement points each time. For reconstructing the entire object, every set of sectional measurement points acquired at di!erent positions must match. Therefore, the optimal shape error analysis for the matching image of two or more sets of sectional measurement points is desired. This paper presents a measurement system that combines two CCD cameras, one line laser and a three-axis motion stage. It forms an optical non-contact scanning system in association with the mathematical method of direct shape error analysis for the use in reverse engineering. This analysis and measurement system can be used for the pro"le measurements of free-form objects. It analyzes the matching image of a free-form surface with high e$ciency and accuracy. The validity and applicability of this system are demonstrated by two practical examples.  2001 Elsevier Science Ltd. All rights reserved.

Keywords: Reverse engineering; Shape error; Surface "tting; Image matching; Optimization

1. Introduction

Optical non-contact scanning measurement systems have been widely used in reverse engineering for rapid product prototyping in recent years. Because it is #exible and quick, it can be used for pro"le measurement of objects with complex surfaces. However, when measuring an object that has a large surface, large curvature, or a full 3603 pro"le, one can acquire only one set of sec-tional measurement points in each measurement. For reconstructing the entire object, every set of sectional measurement points acquired at di!erent positions must match. Therefore, the optimal shape error analysis for the matching images of the same area for two sets of sectional measurement surfaces is desired.

Some approaches for registering and building three-dimensional (3-D) models from multiple range images

have been developed in recent years [1}7]. However, the goal of the newapproach is to develop an algorithm that has high e$ciency and accuracy. In general, there is no existing absolute reference surface when matching every set of sectional measurement surfaces. Therefore, this study has determined the "rst set of sectional measure-ment surfaces to be a matching reference surface. In this article, an analysis and measurement system is proposed. It combines an optical non-contact scanning measure-ment system using two CCD cameras with a direct shape error analysis approach. This system can be used for the pro"le measurement of a large object and can analyze the matching image of a free-form surface with high e$ciency and accuracy.

2. Triangulation measurement

The principle of triangulation measurement with one camera is shown in Fig. 1. In the "gure, P(x, y, z) is a point in the world coordinate (X, >, Z) and P
(u, v) is its focused point in the image plane (;, <). According to the geomet-rical optics and similar triangles, the coordinates of point

0736-5845/01/$ - see front matter  2001 Elsevier Science Ltd. All rights reserved. PII: S 0 7 3 6 - 5 8 4 5 ( 0 0 ) 0 0 0 2 9 - 6

Fig. 1. The principle of triangulation measurement.

Fig. 2. Schematic diagram of the proposed optical scanning measure-ment systems. P(x, y, z) can be calculated by f u" z x, (1) b#x z "cot, (2) where f is the focal length,  is the angle between the

X-axis and light direction, and b is the distance between

the light source and the lens's optical center O. From Eqs. (1) and (2), we can get

x" bu

f cot!u. (3)

Similarly, in the Y-direction we have

f v" z y and x y" u v, which yield y" bv f cot!u, (4) z" !bf f cot!u. (5)

Therefore, the coordinates of point P(x, y, z) are



Triangulation measurement is one of the popular measurement techniques. It has advantages in fast computation and easy operation. However, it possesses disadvantages in complicated calibration of system para-meters and tedious system set-up. In practice, inaccuracy in system parameters and system set-up will induce large measurement errors.

3. The optical non-contact scanning measurement system In principle, the one CCD camera system can grab the projected line image on the surface and "t a free-form line at a time. Having scanned the whole image, the entire free-form surface pro"le of the object can be con-structed. In practice, the one CCD camera cannot always successfully capture the line image at each position be-cause the inspected surface may have steep slopes and protruded pro"les. In order to solve this problem, this study developed a measurement system using two CCD cameras to collect image information. The cameras are located on both sides of the laser diode so that more image information is obtained and compensated for the loss of data that resulted from using only one CCD camera. Fig. 2 shows the schematic diagram of the con-structed scanning measurement system. The optical de-tector is composed of two CCD cameras and a line laser diode, and is mounted on a linear stage. As the laser diode projects a laser stripe onto the object, the two CCD cameras detect the deformed laser line image simulta-neously. Between the two image data collected the clearer one will be stored. By stepping the optical detector to prescribed positions, a series of deformed line images can be collected. Consequently, the entire image information for an object is acquired using this optical scanning system.

3.1. System set-up for parameters calibration

Fig. 3 shows the experimental set-up for the CCD system. A laser strip is projected onto the standard template, and the CCD cameras detect the line image. The template was made using a laser writer with 1
m

Fig. 3. Experimental set-up for parameters calibration.

accuracy of line spacing. Since the distance between each horizontal line on the standard template is known, the intersection point between each horizontal line and verti-cal laser line stripe indicates the (>, Z) coordinate of the space plane. The standard template is successively moved step-by-step along the Z-axis direction. The CCD cameras can grab the image of those points with respect to several space planes, respectively. The two-dimensional (2-D)-measurement information in the image plane can be transformed into its space position using the least-squares mapping algorithm described as follows.

3.2. Coordinate mapping principle

Let (>I, ZI) be a coordinate of the standard template in the space position, and (;I, <I) be the corresponding coordinate in the image plane. Then, a mapping algo-rithm can be established using the least-squares poly-nomial function [8]. >(;, <)", H ,\H  GC7GH ;G<H, (6) Z(;, <)", H ,\H_ GC8GH ;G<H, ₍₇₎

where C7GH and C8GH are the coe$cients of the polynomial

mapping functions.

The related error functions E7 and E8 of >(;,<) and

Z(;, <), respectively, can be obtained by

E7",

I(>I!>), E8" ,



I(ZI!Z).

(8)

Therefore, the coe$cients C7GH and C8GH can be

deter-mined from the minimum error E7

C7GH

"0, E8 C8GH

"0. (9) From Eqs. (6) and (7), one can obtain the related space coordinate for each measurement data on the image plane. The corresponding space coordinate (>, Z) for each pixel position (;, <) of the laser beam image in the CCD plane can be determined. Hence, the entire pro"le can be formed by the integration of the scanned data.

4. The direct shape error analysis method

The construction of the surface measurement points can be determined by several approaches [9}12]. This paper adopted the bicubic uniform B-spline interpolation approach to describe the "rst set measurement points, and to reconstruct multiple surface patches. These mul-tiple bicubic B-spline patches must pass through all measurement points of the "rst set and match their boundaries with C and C continuity. Therefore the accuracy in this surface reconstruction step is guaran-teed. The shape error of the matching image of the free-form surface in this paper is de"ned as the maximum value of the nearest distances from the second set measurement points to the reference (the "rst set) surface. Obviously, the objective function of shape error analysis is the sum of the squared nearest distances. In general, an iterative method algorithm for "nding the nearest dis-tance is widely used [16]. This method is, however, time consuming and tedious for "nding the convergence solu-tion. Zhang [13] proposed iterative algorithm for "nding the closest point for registration of free-form curves and surfaces. Bergevin et al. [14] employed least-squares es-timation to minimize the nearest distance. Eggert et al. [15] adopted k}d tree algorithm for searching the closest point. Ho [19] employed Powell's iterative searching method to "nd the corresponding point in the reference surface having the shortest distance from each of the second set measurement point. This algorithm is called the inverse method. In this study, an improved algorithm, which is called the direct method, was proposed.

For "nding the nearest distance between the second set measurement points and the reference surface, the direct method solves the intersection between the reference sur-face normal vector and the second set measurement point directly. Fig. 4 shows the direct method concept for "nding the nearest distance. Based on this concept, a detailed algorithm of the direct method is described as follows:

Let qGH"[xKGH, yKGH, zKGH]2 be the measurement points of the second set, and pGH"[xPGH, yPGH, zPGH]2 be the para-metric representation point of the ijth B-spline patch in

Fig. 4. The direct method concept for "nding the nearest distance.

Fig. 6. The initial mapping of two-sectional measurement data using coordinate translations and rotations.

Fig. 5. Two sets of sectional measurement data for a car model with a complex surface taken from di!erent measurement angles.

the reference surface. Then the nearest distance between the second set measurement point and the reference surface can be determined by solving the following two non-linear equations:

F(u,v)"(xKGH!xPGH)xPGH_u #(yKGH!yPGH)yPGH

u # (zKGH!zPGH)zPGHu "0, (10) F(u,v)"(xKGH!xPGH)xPGH_v # (yKGH!yPGH)yPGH_v # (zKGH!zPGH)zPGH_v "0. (11) Using the Newton}Raphson Method [17], the solution to the above equations can be determined by solving the following matrix equations. The process is repeated until the deviation converges to a required value.

F(uI, vI)#J(uI, vI)ZI"0, (12) where

F(uI, vI)"







vI>!vI



and the Jacobian matrix

J(uI, vI)"





From the direct method description, one can realize that the computing time for "nding the nearest distance is faster than the iterative method. The highlight of this paper is to use the direct method to "nd the nearest distance between the second set measurement point and the reference surface, which reduces the computation time. From the applied examples, the computing time for

this approach was reduced by about 82}92% when com-pared with the inverse method.

5. Initial localization and optimization

In general, there is no existing absolute reference point between the two sets of measurement data when measur-ing an object. This study determined the initial geometric matching location of the two sets of measurement data using human}computer interaction. Fig. 5 shows the two sets of sectional measurement data of a car model with a complex surface taken from di!erent measurement angles. The initial two sets of sectional measurement data were located using coordinate translations and rotations as shown in Fig. 6.

When the initial sets were located, the initial shape error and initial parameters of a rigid-body transforma-tion of the two sets of sectransforma-tional measurement data were obtained. The accuracy of the shape error would certain-ly not satisfy our requirements. Thus, according to the direct method, the least-squares method and optimiza-tion algorithm, an optimal shape error analysis approach

Fig. 7. The DSEAM #owchart for the matching free-form surface image.

Fig. 9. The area for optimal shape error analysis of the matching image for a large free-form surface.

Fig. 8. The two sets of measurement points of a large free-form surface.

is proposed. It can determine the optimal results of the matching image. The least-squares method and optim-ization algorithm are described as follows:

Let (x, y, z,,,) be the parameters of a rigid-body transformation and d,d, d,2,dL be the nearest dis-tances between the second set measurement points qG (i"1!n) and the reference surface, respectively. The objective function is de"ned as the sum of the squared nearest distances.

F(x, y, z,,,)"L

G[dG(x,y,z,,,)],

(13) where (x, y, z) are the linear translations of a rigid-body transformation in the three coordinate directions and (,,) are the angular rotations of a rigid-body trans-formation with respect to the three coordinate axes. The shape error analysis of a measured 3-D surface involves searching for the six variables (x, y, z,,,) so that the objective function is minimized.

When applying the shape error analysis of a measured line, the design variables would be reduced to three, namely (x, y,). In this study, the DFPM algorithm (Davidon}Fletcher}Powell Method) was adopted [18]. Based on the construction of a B-spline surface and the DFPM algorithm, this work developed a computer pro-gram to analyze the shape error of measured data with respect to the reference surface. It is called the direct shape error analysis method (DSEAM). The #owchart for the DSEAM is shown in Fig. 7. The slave data is moved to the blending area of the master data by the initial variables (x, y, z,,,). The DFPM algo-rithm then searches the optimum variables so that the objective function is minimized.

6. Application examples

6.1. Example 1: a large free-form surface

Because the optical detector was mounted on the lin-ear stage, if the dimension of the object is larger than the linear stage range, we must divide the object into several parts and measure it progressively. Fig. 8 shows the two sets of measurement points for a large free-form surface, which was divided by two sections between which there was an unknown overlapping area. The initial two sets of measurement points were partially matched by approxi-mation using human}computer interaction. We then se-lected the blending area for optimal shape error analysis of the matching image, as shown in Fig. 9. In this case, there were 793 points in the matching image area.

The optimal shape error results and the optimal para-meters of a rigid body transformation using the DSEAM are shown in Table 1. The optimal shape error is 0.3334 mm, which is reduced by 7
m when compared to the initial shape error. Fig. 10 shows an enlarged error diagram of the line segments at optimal condition. The selected zone is again enlarged in Fig. 11. Fig. 12 shows the complete shape of this large free-form surface after shape error analysis. The computation time was reduced by about 92% when compared with the inverse method.

Table 1

The results of optimal shape error analysis of a large free-form surface

DSEAM

Initial set Optimal blending set

x !_{0.050 mm x} !_{0.0482 mm} y 35.999 mm y 35.9712 mm z 0.010 mm z 0.0095 mm  0 rad  !0.000662 rad  0 rad  !0.004135 rad  0 rad  0.001124 rad

Initial shape error 0.3405 mm

Optimal shape error 0.3334 mm

Computing time 19.430 s

Fig. 10. The enlarged diagram of optimal shape error of selected line segments.

Fig. 11. The local enlargement of the line segments of optimal shape error.

Fig. 12. The complete measured shape of the large free-form surface.

Fig. 13. Two sets of measurement points for a car rear-view mirror case.

Fig. 14. The area of optimal shape error analysis for a car rear-view mirror case.

6.2. Example 2: a car rear-view mirror case

For an object having large curved surface, the sectional measurement points will be obtained from di!erent measurement angles. Fig. 13 shows the two sets of measurement points of a car rear-viewmirror case, which were taken from di!erent measurement angles. The

initial mapping of the second set data to the "rst set data was located using human}computer interaction. The area for optimal shape error analysis of the matching image was then selected, as shown in Fig. 14. In this case, there were 2880 points in the area of the matching image. Fig. 14, however, shows only a reduced data set.

Table 2

The results of optimal shape error analysis of a car rear-viewmirror case

DSEAM

Initial set Optimal blending set

x !_{0.528 mm x} !_{0.5303 mm} y 50.000 mm y 49.8268 mm z 0.200 mm z 0.0479 mm  0.139712 rad  0.133798 rad  0 rad  0.053395 rad  0 rad  0.005709 rad

Initial shape error 0.6849 mm

Optimal shape error 0.4664 mm

Computing time 273.925 s

Fig. 15. The line segments of the optimal shape error for a car rear-view mirror case.

Fig. 16. The local enlargement of the line segments for the optimal shape error of a car rear-viewmirror case.

Fig. 17. The complete measured shape of the car rear-viewmirror case.

The optimal shape error results and the optimal para-meters of a rigid body transformation using the DSEAM are shown in Table 2. The optimal shape error is 0.4664 mm, which is reduced by 219
m when compared with the initial shape error. Fig. 15 shows the enlarge-ment of the line segenlarge-ments of the optimal shape error. The local enlargement of the selected line segments of the

optimal shape error is shown in Fig. 16. Fig. 17 shows the complete shape of this object. The computation time was reduced by about 82% when compared with the inverse method.

7. Conclusions

Optical non-contact scanning measurement systems have been widely used in reverse engineering for rapid product prototyping in recent years. Because such systems are #exible and quick, they can be used for the pro"le measurement of objects with complex surfaces.

This paper presented a measurement system, which combines two CCD cameras, one line laser, and a three-axis motion stage to form the optical non-contact scanning system associated with the DSEAM. This measurement system can collect more image information and compensate for the loss of data that resulted from using only one CCD camera. This system can execute the associated analysis for the matching image of a free-form surface with high e$ciency and accuracy. The validity and applicability of this measurement system were dem-onstrated by two practical examples.

References

[1] Shoo KC, Menq CH. Localization of 3-D object having complex sculptured surfaces using tactile sensing and surface description. ASME, J Engng Ind 1991;113:85}92.

[2] Yau HT, Menq CH. Path planning for automated dimension inspection using coordinate measuring machines. Proceedings of the 1991 IEEE International Conference on Robotics and Auto-mation, Sacramento, California, April 1991. p. 1934}9.

[3] Chen Y, Medion G. Object modelling by registration of multiple range images. Image Vision Comput 1992;10(3):145}55. [4] Blais G, Levine MD. Registering multiviewrange data to create

3D computer objects. IEEE Trans 1995;17(8):820}4.

[5] Bergevin R, Soucy M, Gagnon H, Lanrendeau D. Towards a gen-eral multi-viewregistration technique. IEEE Trans Pattern Anal Mach Intell 1996;18(5):540}7.

[6] Masuda T, Yokoya N. A robust method for registration and segmentation of multiple range images. Comput Vision Image Understanding 1995;61(3):295}307.

[7] Higuchi K, Hebert M, Ikeuch K. Building 3-D models from unregistered range images. Graphical Models Image Process 1995;57(4):315}33.

[8] Tai WC, Chang M. Non-contact pro"lometric measurement of large-form parts. Opt Engng 1996;35(9):2730}5.

[9] Beeker E. Smoothing of shapes designed with free-form surface. Comput-Aided Des 1986;18(4):224}32.

[10] Armit AP. Curve and surface design using multipach and multiobject design systems. Comput-Aided Des 1993;25(4): 251}61.

[11] Chen YD, Tang XJ, Ni J, Wu SM. Automatic digitization of free-form curve by coordinate measuring machine. Engi-neered surface, PED-vol. 62, NewYork: ASME, 1992. p. 47}59.

[12] Rogers DF, Adams JA. Mathematical elements for computer graphics. NewYork: McGraw-Hill, 1990.

[13] Zhang Z. Iterative point matching for registration of free-form curves and surfaces. Int J Comput Vision 1994;13(2):119}52. [14] Bergevin R, Lanrendeau D, Poussart D. Registering range views

of multipart objects. Comput Vision Image Understanding 1995;6(1):1}16.

[15] Eggert DW, Fitzgibbon AW, Fisher RB. Simultaneous registra-tion of multiple range views for use in reverse engineering of CAD models. Comput Vision Image Understanding 1998;69(3):253}72. [16] Wismer DA, Chattergy R. Introduction to nonlinear

optimiza-tion. Amsterdam: North-Holland, 1978.

[17] Patel VA. Numerical analysis. London: Saunders College, 1994. [18] Press WH. Numerical recipes in C. Cambridge: Cambridge

University Press, 1994. p. 317}28.

[19] Ho CC. 3D Surface matching techniques in image measurements. Master thesis, National Taiwan University, 1997.