Projection moiré profilometry with
high-dynamic range image
Sen-Yih Chou
Chia-Hung Cho
Ming-Jhe Du
Sien Chi
Projection moiré profilometry with high-dynamic
range image
Sen-Yih Chou
National Chiao Tung University
Department of Photonics and Institute of Electro-Optical Engineering
Hsinchu, Taiwan and
Industrial Technology Research Institute Center for Measurement Standards Building 12, 321 Sec. 2, Kuang Fu Road Hsinchu, Taiwan
E-mail: [email protected] Chia-Hung Cho
Ming-Jhe Du
Industrial Technology Research Institute Center for Measurement Standards Building 12, 321 Sec. 2, Kuang Fu Road Hsinchu, Taiwan
Sien Chi
National Chiao Tung University
Department of Photonics and Institute of Electro-Optical Engineering
Hsinchu, Taiwan and
Yuan Ze University
Department of Photonic Engineering Chungli, Taiwan
Abstract. As a type of optical measuring apparatus, the charge-coupled diode (CCD) camera provides the capability of increasing the speed of measurement by inspecting an area with only one shot. However, the CCD camera’s high-variation range of reflectivity presents an exceptional challenge for the optical measurement established on the surface. We pre-sent a method that could enable one to acquire an image with a high-dynamic range in one shot without any reduction in spatial resolution. Because of the sufficient signal-to-noise ratio, the method presented could perform the robustness of the phase-retrieving algorithm, and the surface topography could be measured more accurately.© 2012 Society of Photo-Optical Instrumentation Engineers (SPIE). [DOI: 10.1117/1.OE.51.2.021112]
Subject terms: high-dynamic range image; projection moiré; profilometry. Paper 110467SS received May 1, 2011; revised manuscript received Aug. 4, 2011; accepted for publication Aug. 29, 2011; published online Mar. 2, 2012.
1 Introduction
Accurately measuring the three-dimensional (3-D) shapes of objects is important for the industry to speed up product development and ensure manufacturing quality. In general, the techniques of 3-D shape measurement can be classified into two categories: contact surface measurement and non-contact surface measurement.
The contact surface techniques can provide high accuracy for the measurement of any“hard” objects that are insensi-tive to the optical properties of the surface. However, there are risks of the contact surface techniques damaging the sur-face of the object inspected. Moreover, as a point-by-point measuring technique, the speed of the contact surface tech-nique is usually very slow. In contrast, the noncontact surface methods do not damage the surface of the object inspected. Although different optical methods are extensively adopted between these two types of techniques, it is still an excep-tional challenge for an object tested in an optical inspection system with a wide range of variation of the surface reflec-tivity. In addition, for increasing the speed of the measure-ment, the image detectors with a low-dynamic range (LDR) are utilized, typically providing only 8 bits of brightness data for each pixel. Hence, the image captured by the imaging system ends up being too dark in some areas and possibly
being saturated in others. Since the optical signal of the mea-suring region cannot be properly retrieved, these inspection methods would result in the loss of its accuracy.
An overview of 3-D shape measurement using various optical methods was provided by Chen et al.1 The merits of structured light method, also categorized as active trian-gulation, are (1) easy implementation, (2) fast full-field mea-surement, and (3) phase shifting with the fringe density and the direction change implemented without moving parts if computer-controlled liquid crystal on silicon (LCoS)/digital light processing (DLP) is used.2–4However, the optical
prop-erties of the object surface affect the accuracy; thus, a variety of optical 3-D shape measurement methods have been pro-posed for shiny surfaces.5,6
Nevertheless, for an object with a very high dynamic range (HDR) of its surface reflectivity, all these proposed methods are potentially problematic. Zhang et al. proposed a HDR technique to measure this type of object.7 They reported that multiple shots of the fringe images with differ-ent exposures were taken for each measuremdiffer-ent. The final fringe images, used for phase retrieval, were produced pixel by pixel by choosing the brightest but most unsaturated corresponding pixel from one shot. A phase-shifting algo-rithm was employed to compute the phase, which was then further converted to 3-D coordinates. Therefore, they found that multiple shots could overcome the very HDR of surface reflectivity; but this method is also very time consuming. 0091-3286/2012/$25.00 © 2012 SPIE
On the other hand, Nayar suggested that using an optical mask adjacent to a conventional image detector array can achieve an HDR image detector.8 On the mask, there was a pattern with spatially varying transmittance, thereby giving adjacent pixels on the detector different exposures to the scene. The captured image was mapped to an HDR image by using an efficient image reconstruction algorithm; how-ever, this method required downgrading the spatial resolution to gain an HDR image.
In comparison with the previous studies, this paper pre-sents a technique for an LDR imaging device, such as a CCD camera, to acquire an HDR image in one shot. Thereby, it is possible to measure a very wide range of the surface reflectivity without any reduction in the spatial resolution. The availability of the extra bits of the data at each image pixel enhances the robustness of the phase-retrieving algorithms so that an accurate surface topography of a measured object can be obtained. DLP is used as the light modulation for the control of the distribution of the light intensity when a sample is in higher reflectivity regions and under lower light illumination. The dull regions are illuminated with a higher light intensity to produce a raw image whose surface brightness levels for all pixels are ranged within the dynamic range of a CCD camera. Thereafter, the single raw image is processed by a compen-sation operation according to an intensity gain ratio of the light intensity before and after being modulated by DLP. As a result, an HDR image can be obtained from the LDR-imaging CCD camera.
Since this system only requires its imaging device to cap-ture one image for processing, the advantages are that it is not time consuming, it produces a small amount of errors during multiple sampling, and also has high spatial resolu-tion. This proposed technique is not limited to 3-D shape measurement systems; it is applicable to any optical mea-surement techniques with variant spatial brightness.
2 Measurement Method
This work developed a projection moiré system for inspect-ing the variation range of surface reflectivity with high-speed measurement and preserving the spatial resolution. This system is based on a digital fringe projection and is associated with a three-step, phase-shifting algorithm. It retrieves the phase value of the fringe images and converts it to a 3-D shape. The basic configuration of the moiré sys-tem using digital fringe projection is shown in Fig.1. A lamp was used for providing a uniform intensity distribution onto the DLP chip, and then the modulated light was projected onto the object through a telecentric lens. The DLP chip con-trolled by a computer generated the fringe images, which were projected onto the object under measuring. These fringe images were distorted and reflected by the object and then captured by a CCD camera.
The DLP chip not only generated the fringe images for the phase-shifting method, but also adjusted the light intensity distribution so that it was within the dynamic range of the CCD camera. Then, a frame grabber, installed in the computer, acquired the digital fringe images through a camera-link interface. The computer processed the fringe images obtained to retrieve the phase by using both the phase-shifting algorithm and the phase-unwrapping algo-rithm with further conversion to 3-D coordinates.9According
to the three-step, phase-shifting algorithm, the original projected fringe image intensities are presented as
I1ðx; yÞ ¼ αðx; yÞ þ βðx; yÞ · cos
ϕðx; yÞ −2π3
; I2ðx; yÞ ¼ αðx; yÞ þ βðx; yÞ · cos½ϕðx; yÞ; I3ðx; yÞ ¼ αðx; yÞ þ βðx; yÞ · cos
ϕðx; yÞ þ2π3
;
(1)
whereα is the DC component or average intensity, β is the amplitude of the intensity modulation, andφ is the phase of the spatial modulation. The fringe image was reflected by the surface of the object being tested and was then captured by the camera. The fringe images actually captured by the cam-era were I10ðx;yÞ ¼ rðx;yÞ αðx;yÞþβðx;yÞ · cos ϕðx;yÞ−2π3 ; I20ðx;yÞ ¼ rðx;yÞfαðx;yÞþβðx;yÞ · cos½ϕðx;yÞg; I30ðx;yÞ ¼ rðx;yÞ αðx;yÞþβðx;yÞ · cos ϕðx;yÞþ2π3 ; (2) where rðx; yÞ is related to the reflectivity of the object and the camera sensitivity and ϕðx; yÞ is the phase of the fringe images after being modulated by the object. The phase can be retrieved through Eq. (2) from Eq. (3)
ϕðx; yÞ ¼ tan−1 ffiffiffi 3 p ðI0 1− I30Þ 2I0 2− ðI10þ I30Þ : (3)
The height of the object’s surface is proportional to the phase difference, Δϕ ¼ ϕ − φ. The traditional method is on the basis of the constant average intensity (α) and the con-stant amplitude (β) of Eq. (1) for all field of view inside the measuring configuration. From Eq. (2), if a shiny region is within the field of view, the region presents large reflectivity and dominates both the average intensity and the average amplitude. Consequently, with a possible shiny region, it is necessary for traditional methods to pick up small average
Fig. 1 The system configuration. Chou et al.: Projection moiré profilometry with high-dynamic range image
values of the intensity and the amplitude to prevent the camera from being saturated. However, in general case, the values are too small for some dull regions to retrieve the phase from Eq. (3).
From the point of view of mathematics, the intensity con-trast could not affect the precision of the phase-retrieval pro-cess, but the system is discrete and the small contrast will invoke large digitized noise during the phase-retrieval pro-cess according to Eq. (3). In this paper, we present a novel projection moiré system that could avoid contrast loss for inspecting the high-variation range of surface reflec-tivity. The DLP chip not only adjusts the light intensity for the whole region, but also adjusts the intensity pixel by pixel. The system could optimize the average intensity and the average amplitude of each pixel. This function could ensure that the whole region has enough contrast for performing the phase-retrieval process.
In this study, we used an Optoma EP728 DLP projector to create the signals of the fringe image with a resolution of 1024 × 768 and with a fringe pitch of 3.3 mm (150 pixels). The camera that we used was a SONY XCL5005 with a reso-lution of2400 × 2014 and 12 bits/pixel. The frame grabber was DALSA X64 Xcelera-CL PX4 with a camera-link inter-face. The field of view in this system was53 × 45 mm. The distance between camera and sample was 40 cm.
3 Results and Discussion
The block diagram of the intensity control unit is shown in Fig. 2. Firstly, we controlled the DLP to form uniform dis-tribution of the light intensity and projected the light on the sample. Since there were shiny and dull regions on the sur-face of this sample, the image captured by the CCD camera ended up with darkness in some areas and possibly saturated in others. The calibration module received the raw image data from the CCD camera, and an image-processing algo-rithm indicated the boundary of the regions with different values of the surface reflectivity within the field of view, which resulted in calibration factors for each region. The estimated factors were fed back to the intensity configuration module for the adjustment of the intensity of the fringe images and for the guarantee of the intensity of all regions being within the dynamic region of the CCD camera.
For the industrial manufacturing process, a similar inspec-tion condiinspec-tion would ensure that the boundary regions and calibration factors could be loaded from the database. For the phase-shifting algorithm, the modulated fringe images with the revision of the average intensity and the amplitude for different region from calibration factors were sent to the
DLP projector and projected onto the sample. Therefore, several sets of raw images with the surface brightness levels of all pixels could be produced within the dynamic range of the CCD camera. The raw images were reconstructed as HDR images according to the calibration factors and were sent to the phase-retrieval algorithm. Because of the images with a larger signal-to-noise ratio from the HDR, a higher quality of 3-D data could be obtained.
To demonstrate our method, we measured a slide mounted on a base plane with high reflectivity. Two grooves were formed with low reflectivity on the top of the slide by sand blasting. We compared the measurement results between the traditional fringe projection moiré and the pre-sented the fringe projection moiré system. The system included a DLP module, a uniform lighting module, and a CCD camera. Firstly, we aligned the system carefully and measured the relation between the pixel of the DLP and the pixel of CCD camera. We were then able to create a map-ping table for looking up the intensity of the image that was captured by the CCD camera according to the corresponding pixels on the DLP chip. The traditional fringe projection moiré system and the DLP module provided uniform sine fringe over the whole field of the measurement. For solving the profile of the slide, the DLP module projected three sine-fringe images with a phase shift of 2π∕3 to the slide; the three-step, phase-shifting algorithm was then used to solve the profile, as shown in Fig.3.
The cross-sectional plot of the marked region of the slide is shown on the left-hand side of Fig.3, the horizontal axis stands for the pixels of the CCD camera, and the vertical axis is the height of the slide. The bottom-left image is one of the fringe image captured by the CCD camera, indicating that the low reflectivity on the top of the slide made the image lose its contrast, since the contrast of the fringe was insufficient to perform the phase-retrieving algorithm. There were many spark noises in the darkest region.
In order to overcome the insufficient contrast, the novel projection moiré system invoked the calibration module, cal-culating the light intensity distribution over the field of view by modifying the edge-detection algorithm and by defining the calibration factors for each region. By referring to the mapping table and using a mixed-pixels algorithm, we used the DLP module to adjust the average intensity and the average amplitude of the intensity modulation based on the factors and the fringes projected to the slide.
The images captured by the CCD camera were then fed in the calibration module for the reconstruction of the HDR images, as shown at the bottom-left side of Fig. 4. After
the process of the phase-retrieving algorithm, we obtained the 3-D profile of the slide. Figure 4 shows the cross-sectional view of the marked region of the slide. Compared with Fig. 3, the contrast is adequate for performing the phase-retrieval algorithm leading to the HDR image obtained from the intensity control unit. Hence, the spark noise disap-peared. This result demonstrated that the proposed method can successfully measure objects with large dynamic surface reflectivity.
4 Conclusion
We present a 3-D profile system that is able to acquire an HDR image in one shot and preserve spatial resolution. An algorithm is proposed for calculating the calibration fac-tors according to the different reflectivity for each region, and the DLP module is designed for adjusting the illumina-tion. A CCD camera is used for the capture of the image, and then an algorithm is performed for the reconstruction of the HDR image by a compensation operation according to the calibration factors. This technique can improve the image contrast without reducing spatial resolution and overcome the wide range of the variation of the surface reflectivity. Because an HDR image is obtained with one shot, it is economic without induced errors of multiple exposures. The enhanced fringe contrast by the HDR image provides the robustness of the phase-retrieval algorithms so that an accurate surface topography of a measured object can be
obtained. The proposed method could be applicable to any optical measurement techniques for which the reflectiv-ity of the surface of the measuring objects varies abruptly. References
1. F. Chen, G. M. Brown, and M. Song,“Overview of three-dimensional shape measurement using optical methods,” Opt. Eng. 39, 10–22 (2000).
2. M. Sjodahl and P. Synnergren, “Measurement of shape by using projected random patterns and temporal digital speckle photography,” Appl. Opt.38(10), 1990–1997 (1999).
3. E. Muller, “Fast three dimensional form measurement system,” Opt. Eng.34(9), 2754–2756 (1995).
4. G. Sansoni et al.,“Three dimensional imaging based on gray-code light projection: characterization of the measuring algorithm and develop-ment of a measuring system for industrial application,” Appl. Opt. 36(19), 4463–4472 (1997).
5. Q. Hu et al.,“Shiny parts measurement using color separation,”Proc. SPIE6000, 6000D (2005).
6. R. Kokku and G. Brooksby,“Improving 3D surface measurement accu-racy on metallic surfaces,”Proc. SPIE5856, 618–624 (2005). 7. S. Zhang and S.-T. Yau,“High dynamic range scanning technique,”
Opt. Eng.48, 033604 (2009).
8. S. K. Nayar and T. Mitsunaga,“High dynamic range imaging: spatially varying pixel exposures,” in IEEE Conf. Computer Vision and Pattern Recognition 1, 472–479 (2000).
9. S. Zhang, X. Li, and S. T. Yau, “Multilevel quality-guided phase unwrapping algorithm for real-time three-dimensional shape reconstruc-tion,”Appl. Opt.46(1), 50–57 (2007).
Sen-Yih Chou received BS and MS degrees from the Department of Physics at National Sun Yat-sen University in Taiwan in 1994 and 1996, respectively. He is currently in the photonics PhD program at National Chiao Tung University, Taiwan. From 1998 to 2000, he worked as a research and devel-opment engineer in the Materials and Elec-tro-optics Research Division at the Chung-Shan Institute of Science and Technology. He currently is a research and development engineer at the Center for Measurement Standards at the Industrial Technology Research Institute, Taiwan.
Chia-Hung Cho received a BS degree from the mechanical and electro-mechanical engi-neering at the Tamkang University in 2005 and an MS degree from the Institute of Auto-mation Technology at the National Taipei University of Technology in 2007. Since 2008, he has worked as a research and development engineer at the Center for Mea-surement Standards at the Industrial Tech-nology Research Institute. His current research interests are automatic optical inspection, dynamic three-dimensional surface profilometry, digital image processing, and imaging colorimeter technology.
Ming-Jhe Du received a BS degree in elec-trical engineering from National Cheng Kung University in Taiwan in 2003 and an MS degree in electrical engineering from Nation Taiwan University in Taiwan in 2006. Since 2006, he has worked as a research and development engineer at the Industrial Tech-nology Research Institute in Taiwan.
Sien Chi: Biography and photograph not available. Fig. 3 The captured image and retrieved profile of the traditional
fringe projection.
Fig. 4 The captured image and retrieved profile of the regional adjust-ing fradjust-inge projection.
