Facial Model Reconstruction for Plastic Surgery Simulation
全文
(2) points can be the nature face features or artificial markers. Face feature points such as the canthi, the corners of the mouth and the nose tips are common features used in these systems. Methods that do not rely on corresponding points typically employ surface-based approaches, such as ICP (iterative closet point) algorithm [20] and its extensions [24-25]. Most of them are more complicated and time-consuming than the point-based method. In the proposed system, the color structured lighting and robust matching algorithm are designed to obtain accurate 3D facial data. Image processing techniques are employed to detect the spot features for correspondence establishment automatically. The spatial coded patterns are adopted that is insensitive to patient motion. An adjustment algorithm is also designed to generate patterns with optimal color, shape and density. For recovering a complete facial model, we image patient’s face from four different positions so that any face points can be viewed by at least two cameras. Three partial models are generated from any two consecutive cameras by triangulation. The system then registers these models and integrates them with the distance-weighted blending technique. Three slide projectors are used so that the patterns can cover the whole face. The point-based registration algorithm is then used to provide fast and simple registration. This paper is organized as follows: Section 2 describes the procedure for pattern generation. The proposed corner detection algorithm is addressed in Section 3. The algorithm for registering and integrating incomplete models are presented in Section 4. Experimental results are given in Section 5, followed by conclusions. 2. PATTERN DESIGN There are several coded patterns reported in the recent literature [12]. According to the temporal dependence, the patterns can be divided into two classes: temporal and spatial codes. To capture 3D model of living or moving object, we choose the spatial code design. There are two essential requirements for these patterns: 1) code-uniqueness, and 2) fault-tolerance. In our development, the code is a matrix M of size H ×W. Each element mij of matrix M is assigned with one of the possible L letters. Considering N neighboring elements P1, P2, P3,… PN of arbitrary mij, the position of mij can be computed (or decoded) from the code letters of mij and its N neighboring elements. We denote the index code word of mij by Fij=(f(P1), f(P2), f(P3),… ..f(PN), f(mij)), where f(Pi) is the code letter of position Pi for i=1,2 ...N. We choose (3,3) de Bruijn sequences [17] for pattern encoding. The encoded matrix is established according to the method proposed by Y.-C. Hsieh[13], which ensures that each mij has three possible letters, i.e. L=3, and is uniquely indexed by its 4-neighboring system and itself. That is, mij is indexed by Fij=(f(P1), f(P2), f(P3), f(P4), f(mij))) and f(P1),f(P2),f(P3),f(P4) are the letters of mij’s 4-neighbors. Since Fij appears exactly only once in the matrix, the Hamming distance between each distinct Fij pair will be equal to or exceed 1. If misidentified code word or loss of pattern elements should happen, this special prop-. erty provides preliminary error detection and correction ability. When the coded matrix is determined, we have to assign the color and shape for each mij. In [4,6-7], red, green and blue colors (hue=0, 85 and 170 for hue [0,254]) are first chosen to represent the three code letters. However, we have tested several different color sets and found that h=42.5, 127.5 and 212.5 are better than RGB colors. Hence, the code colors are assigned by (42.5,255,128), (127.5,255,128) and (212.5,255,128) on HSI color model. The element shape can be circles [4], slits [5,9] or other geometric appearances [6-7]. We choose squares as the element shape, since squares are effective in increasing mesh density. Fig. 1 shows the object illuminated by our patterns.. Figure 1. Object after pattern projection 3. SQUARE CORNER DETECTION To increase the resolution of the recovered model, we will detect corner points as features for establishing more correspondences. Possible solutions to locating corner points include general corner detection algorithms, polygonal approximations, or dominant points detection on planar curves [18]. However, the acquired squares are corrupted and distorted (Fig.2 (a)) that makes corner detection a very difficult task. Therefore, we develop a new algorithm to estimate corners for the corrupted or distorted squares. First, it computes the number of pixels on each line segments in both vertical and horizontal directions, which yields two force field maps Fh(i,j) and Fv(i,j), respectively. The values of Fh(i,j) and Fv(i,j) are in fact their corresponding segment lengths. For example, if (i,j) is contained in a horizontal line segment of length 3 and a vertical line segment of length 5, then Fh(i,j)=3 and Fv(i,j)=5. It is assumed that the length of line segments represents the possibility of actual square location, and the pixels near the intersections of horizontal and vertical lines may be the actual square corner positions. Thus, we define the total force by Ftotal(i,j) =Fh(i,j)+Fv(i,j). The total force value near the segment end points should become the maximum. Dividing the bounding box into four squares evenly, we then choose the pixels with the greatest force in the square to be the corner points. In case of multiple maximum, the optimal position is the interpolation among all candidates. The detected corners are shown on Fig.2 (b). 2.
(3) (a) (a). (b) Figure 2. (a) A corrupted and distorted square. (b) Gray pixels are the detected corners. (b) 4. REGISTRATION AND INTEGRATION. Figure 3. 3D positions of the point sets before registration. (a) Front view; (b) Top view. There are three 3D models reconstructed from any two consecutive cameras. We register these models by evaluating their mutually rotation and translation. Registration algorithms usually can be divided into two categories: feature-based and surface-based. In our experiment, the face grids provide very robust and convenient landmarks for feature-based registration. Utilizing the consistency of grid color code, we can establish the correspondences between these models. Then, we can make use of the feature-based registration to estimate the rotation and translation parameters by minimizing the total registration error D given in the following equation:. D=. 1 n ∑ pi − ( Rqi + T ) n i =1. 2. (a). where pi and qi are the corresponding feature point pair of the two models, and n is the total number of correspondences. We fix pi and transform qi by choosing appropriate rotation R and translation T so as to best match these two point sets. We estimate the two unknown parameters R and T by the least-square optimization. Figures 3 and 4 show the relative projection positions of these point sets before and after registration, respectively, from the front and top views. Although these models were already aligned, some points may not exactly match due to the calibration or quantization errors. To compensate for this drawback, we calculate the 3D position by inverse-distance weighted averaging for points with two or more correspondences. The distance measure is defined as the length between a vertex and its nearest border.. (b) Figure 4. 3D positions of the point sets after registration. (a) Front view; (b) Top view. 3.
(4) 5.EXPERIMENT RESULTS In our experiments, the encoded color pattern was generated on a 35-mm slide. This slide was projected onto the object surface by a slide projector. Two parallel digital cameras imaged the scene simultaneously and the 3D information was measured by the stereo triangulation. The digital cameras are AGFA e-Photo 1680 with spatial resolution 640x480. We tested our method by a plastic head model. The captured images were first transformed into the HSI color model [14], which yielded the corresponding hue, intensity and saturation images. The intensity image was used to detect the positions of squares by Canny edge detector [15]. The hue image was applied to identify the color patterns. Based on these color codes, we could determine their code configurations for each grid and then resolve the grid correspondence accordingly. In this process, the grid centroids and corners were used as the actual corresponding points. After triangulation, we first obtain a set of scattered 3D points. A 3D surface model can then be built by Delaunay triangulation [16] for generating triangles on the given 3D points. Fig.5 (a) and Fig.5 (b) are the results before and after increasing resolution. The model with corner detection is five times the number of vertices and triangles as that before the corner detection.. The partially reconstructed results are shown on Fig 6. Three partial models were combined to a facial model. The reconstructed model can be visualized from arbitrary viewpoints. Fig. 7 shows the texture-mapped 3D model from different views. And Fig. 8 is a textured-mapped image sequence of a real case from different viewpoints.. (a). (b). (a). (c) Figure 6. Three partial face model from different camera sets. (b) Figure 5. Model wireframe (a)before and (b)after corner detection.. 4.
(5) (a). (b). (c). Figure 7. The complete facial model obtained by integration. (a) wireframe ; (b) texture-mapped model ture-mapped model from another viewpoint.. ; (c) Tex-. Fig. 8. An image sequence of a real subject with texture-mapping.. 5.
(6) 6. CONCLUSIONS. Light for Rapid Active Ranging,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 1, pp. 14-28, 1987.. In order to recover 3D information from smooth facial surfaces, we propose a new system to reconstruct 3D surface from stereo images with color encoded pattern projection. The 3D object surface is efficiently computed from the color encoded stereo patterns. The de Bruijn’s sequence-encoding scheme is adopted to generate the color codes. It considerably reduces the correspondence computation and provides the fault-tolerant capability. The designed corner detection algorithm is capable of estimating corners from corrupted or distorted squares. As a result, the number of vertexes and triangles are multiplied, and a dense facial model can be reconstructed with fewer squares. The proposed method is effective for instant face imaging thus the system is hardly affected by patient motion. Moreover, we use four cameras simultaneously to resolve the view range limitation problem of standard stereovision. After registration and integration, a complete facial model was successfully reconstructed. Thus, the proposed method is simple, inexpensive, non-radioactive and convenient for facial surgical planning. It provides a tempting alternative for physicians and patients in facial surgical applications.. [10] C.-S. Chen, Y.-P. Hung, C.-C. Chiang, and J.-L. Wu, “Range Data Acquisition using Color Structured Lighting and Stereo Vision,” Image and Vision Computing, vol. 15, pp. 445-456, 1997.. 7.REFERENCES. [16] M. de Berg, M. van Kreveld, M. Overmars and O. Schwarzkopf, Computational Geometry, Springer, Berlin Heidelberg, 1997.. [1] R. Klette, K. Schluns and A. Koschan, Computer Vision : Three-dimensional Data from Images, Springer, Singapore, 1998. [2] J. L. Moigne and A. M. Waxman, “Structured Light Patterns for robot mobility,” IEEE J. Robotics and Automation, vol. 4, no. 5, pp. 541-548, 1988. [3] D. Caspi, N. Kiryati, and J. Shamir, “Range Imaging with adaptive color structured light,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 5, pp. 470-480, May 1998. [4] R. A. Morano, C. Ozturk, R. Conn, S. Dubin, S. Zietz, and J. Nissanov, “Structured Light using Pseudorandom Codes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 3, pp. 322-325, Mar. 1998. [5] J. Salvi, J. Batlle, and E. Mouaddib, “A Robust-coded Pattern Projection for Dynamic 3D scene measurement,” Pattern Recognition Letters, vol. 19, pp. 1055-1065, 1998. [6] P. M. Griffin, L. S. Narasimhan and S. R. Yee, ”Generation of Uniquely Encoded Light Patterns for range data acquisition,” Pattern Recognition, vol. 25, no. 6, pp. 609-616, 1992. [7] S. R. Yee and P. M. Griffin, ” Three-dimensional imaging system,” Optical Engineering, vol. 33, pp. 2070-2075, 1994.. [11] A. Koschan, V. Rodehorst, and K. Spiller, “Color Stereo Vision using Hierarchical Block Matching and Active Color Illumination,” Proc. Int. Conference on Pattern Recognition, pp. 835-839, 1996. [12] J. Batlle, E. Mouddib and J. Salvi, “Recent Progress in Coded Structured Light as A Technique to Solve the Correspondence Problem: A Survey,” Pattern Recognition, vol. 31, no. 7, pp. 963-982, 1998. [13] Y.-C. Hsieh,” A Note on the Structured Light of Three-dimensional Imaging System,” Pattern Recognition Letters, vol. 19, pp. 315-318, 1998. [14] R. C. Gonzales and R. E. Woods, Digital Image Processing, Addison-Wesley, Reading, MA, 1992. [15] J. Canny, “ A Computational approach to edge detection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, pp. 679-698, 1986.. [17] G. Chartrand, O. R. Oellermann, Applied and algorithmic graph theory, McGraw-Hill, Singapore, 1993. [18] E. R. Davis, Machine Vision, Academic Press, 1997. [19]R. M. Bolle and D. B. Copper, “On optimally combining pieces of information, with application to estimating 3-D complex-object position from range data,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 5, pp. 619-638, 1986. [20]P. J. Besl and N. D. McKay, ”A method for registration of 3D shapes,” IEEE Tran. Pattern Analysis and Machine Intelligence, vol. 12, no. 2, pp. 239-256, 1992. [21]C. Dorai, G. Wang and A. K. Jain, “Registration and integration of multiple object views for 3D model construction,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp. 83-89, Jan. 1998. [22]J. Tarel, “A Coarse to fine 3D registration method based on robust fuzzy clustering,” Computer vision and image understanding, vol. 73, no. 1, pp. 14-28, Jan. 1999. [23]L. G. Brown, “A Survey of Image Registration Techniques,” ACM Comput. Surv., vol. 24, no. 4, pp.325-375, Dec. 1992.. [8] R. J. Valkenburg and A. M. McIvor, “Accurate 3D Measurement using a Structured Light System,” Image and Vision Computing, vol. 16, pp. 99-110, 1998.. [24]J. Feldmar, “Extension of the ICP Algorithm to Nonrigid Intensity-Based Registration of 3D volumes,” Computer Vision and Image Understanding, vol. 66, no. 2, pp. 193-206, May 1997.. [9] K. L. Boyer and A. C. Kak, “Color-encoded Structured. [25]Z. Zhang, “Iterative Point Matching for Registration of 6.
(7) Free-form Curves and Surfaces,” Int. J. Comput. Vis., vol. 13, no. 2, pp.119-152, 1994.. 7.
(8)
數據
相關文件
We are not aware of any existing methods for identifying constant parameters or covariates in the parametric component of a semiparametric model, although there exists an
He(She) has a round face and two small eyes.. Are there
Two distinct real roots are computed by the Müller’s Method with different initial points... Thank you for
For a polytomous item measuring the first-order latent trait, the item response function can be the generalized partial credit model (Muraki, 1992), the partial credit model
At least one can show that such operators has real eigenvalues for W 0 . Æ OK. we did it... For the Virasoro
• Similar to Façade, use a generic face model and view-dependent texture mapping..
¾ Relocation, which modifies the object program so that it can be loaded at an address different from the location originally specified.. ¾ Linking, which combines two or
- - A module (about 20 lessons) co- designed by English and Science teachers with EDB support.. - a water project (published