In this thesis, we proposed two new reflectance models for 3D surface reconstruction. First, a novel 3D image reconstruction model was proposed. This method considers the components of both diffusion and specular reflection in the reflectance model. We used two neural networks with symmetric structure to estimate these two reflection models separately and combined them with an adaptive ratio for each point on the object surface. The proposed network estimates the point-wise adaptive combination ratio of the diffusion and specular intensities such that the different reflecting properties of each point on the object surface can help to achieve better performance of surface reconstruction. The proposed symmetric neural network structure with adaptive learning procedure does not need any special parameter setting and the smoothing conditions. It is also easier to achieve the convergence condition and to make the system stable. The critical parameters, such as the light source and the viewing direction and so on, are also obtained from the learning process of the neural network. The obtained normal vectors of the surface can then be applied to 3D surface reconstruction by enforcing integrability approach.
Secondly, we further proposed another new nonlinear reflection model consisting of the diffusion and specular components. We do not need to separate the two components in the proposed nonlinear reflection model. Using the unsupervised non-linear ICA network for solving photometric stereo problems does not need any desired outputs and the smoothing conditions. It is easier to achieve the convergence
condition and make the system stable.
For 3D surface reconstruction, several conclusions are listed below. (a) When we estimate the surface shape, the success of the reflectance model depends on two major components, including the diffusion and specular components. (b) In our methods, we do not know the locations of light sources for solving the photometric stereo problems.
(c) The proposed symmetric neural network structure and the unsupervised post-nonlinear ICA network do not need any special parameter setting and the smoothing conditions.
On the other hand, we proposed a new approach in the surrounding illumination estimation of an image for color reconstruction. The proposed algorithm estimated the illuminant based on chromaticity histogram of the image. And a neural network back-propagation (BP) learning algorithm is used to estimate the spectral power distribution of the illuminant according to the center values of the chromaticity histogram. The proposed algorithm also eliminated the interference of the dominant colors and illumination estimation through low-pass filtering of the chromaticity histogram. The illumination estimation based on the chromaticity histogram can avoid unrealistic assumptions on the color images and provide the highly efficient and robust estimation. Compared with the methods based on neural networks which we proposed before, the size of their architectures is huge. Those methods need many connecting parameters and are not easy to implement on hardware. The size of our architecture is much smaller; hence our methods can estimate the relative parameters of outside sources effectively and rapidly. Thus, we could reconstruct the color of 3D objects by using these parameters. The comparisons in performance have demonstrated the superiority of the proposed algorithm both in estimation accuracy and robustness for color constancy.
Bibliography
[1] K. P. Horn, “Shape from Shading: A Smooth Opaque Object from One View,”
Ph. D. dissertation, Massachusetts Inst. Technol., Cambridge, MA, 1970.
[2] K. P. Horn and M. J. Brooks, Shape from shading, Cambridge, MA: MIT Press, 1989.
[3] K. P. Horn and R. W. Sjoberg, “Calculating the reflectance map,” Appl. Opt, Vol. 18, No. 11, pp. 37-75, 1990.
[4] R. Zhang, P. S. Tsai, J. E, Cryer, and M. Shah, “Analysis of shape from shading techniques,” in Proc. IEEE Computer Vision and Pattern Recognition, pp. 377-384, 1994.
[5] M. J. Brooks and K. P. Horn, “Shape and source from shading,” in Proc.
International Joint Conference on Artificial Intelligence, pp. 932-936, 1985.
[6] Q. Zheng and R. Chellappa, “Estimation of illuminant direction, albedo, and shape from shading,” IEEE Trans. Pattern Anal. And Machine Intelligence. Vol.
13, No. 7, pp. 680-702, 1991.
[7] K. M. Lee and C. C. J. Kuo, “Shape from shading with a linear triangular element surface model,” IEEE Trans. Pattern Anal. And Machine Intelligence.
Vol. 15, No. 8, pp. 815-522, 1993.
[8] M. Bichsel and A. Pentland, “A simple algorithm for shape from shading,” in Proc. IEEE Computer Vision and Pattern Recognition, pp. 459-464, 1992.
[9] C. H. Lee and A. Rosenfeld, “Improved methods of estimating shape form shading using the light source coordinate system,” Artif. Intell., Vol. 26, pp.
125-143, 1985.
[10] A. Pentland, “Shape information from shading: a theory about human perception,” in Proc. IEEE Second International Conf. on Computer Vision, pp.
404-413, 1988.
[11] R. J. Woodham, “Photometric Method for Determining Surface Orientation from Multiple Images,” Journal of Optical Engineering, Vol. 19, No. 1, 1980.
[12] K. Barnard, V. Cardei, and B. Funt, “A comparison of computational color
constancy algorithms - Part I: Methodology and experiments with synthesized data”, IEEE Trans. On Image Processing, Vol. 11, No. 9, pp. 972-984, Sept.
2002.
[13] F. H. Cheng, W. H. Hsu, and T. W. Chen, “Recovering colors in an Image with chromatic illuminant,” IEEE Trans. on Image Processing, vol. 7, No. 11, pp.
1524-1533, Nov. 1998.
[14] R. Gershon, A. D. Jepson, and J. K. Tsotsos, “From [R, G, B] to surface reflectance: Computing color constant descriptors in images,” Perception, pp.
755-758, 1988.
[15] H. Helson, “Fundamental problems in color vision. I. The principle governing changes in hue, saturation, and lightness of nonselective samples in chromatic illumination,” J. Exper. Psychol., vol. 23, pp. 439-476, 1938.
[16] E. H. Land, “Recent advances in retinex theory,” Vision Research, vol. 26, pp.
7-21, 1986.
[17] M. D. Zmura and P. Lennie, “Mechanisms of color constancy,” J. Opt. Soc.
Amer. A, vol. 3, pp. 1662-1672, 1986.
[18] E. H. Land and J. J. McCann, “Lightness and retinex theory,” J. Opt. Soc. Amer.
A , vol. 61, pp. 1-11, 1971.
[19] E. H. Land, “The retinex theory of color vision,” Scientific American, pp.
108-129, 1977.
[20] J. J. McCann, S. P. McKee, and T. H. Taylor, “Quantitative studies in retinex theory,” Vision Research, vol. 16, pp. 445-458, 1976.
[21] S. Usui, S. Nakauchi, and Y. Miyamoto, “A decorrelating neural network for color constancy,” Proceedings of the Annual International Conference of the IEEE, Engineering in Medicine and Biology Society, vol. 14, pp. 1030-1031, 1992.
[22] B. V. Funt, V. C. Cardei, and J. J. Barnard, “Method of estimating chromaticity of illumination using neural networks,” United States Patent, Patent Number:
5907629, May 1999.
[23] G. D. Finlayson, P. M. Hubel, and Hordley, “Color by correlation,”
Proceedings of IS&T/SID Fifth Color Imaging Conference: Color Science, systems, and Applications, pp. 6-11, Scottsdale, AZ, 1997.
[24] P. M. Hubel, J. Holm, and G. D. Finlayson, “Illuminant estimation and color
Wiley, 1999.
[25] P. M. Hubel and G. D. Finlayson, “White point determination using correlation matrix memory,” United States Patent, Patent Number: 6038339, Mar. 2000.
[26] K. M. Lee and C. C. J. Kuo, “Shape Reconstruction from Photometric Stereo,”
Journal of Optical Society of America: A, Vol. 10, No. 5, 1993.
[27] C. Cho and H. Minanitani, “A New Photometric Method Using 3 Point Light Sources,” IEICE Trans. Inf. & Syst. V.E76-D, No. 8, pp. 898-904, August 1993.
[28] Y. Iwahori, R. Woodham, and A. Bagheri, “Principal Components Analysis and Neural Network Implementation of Photometric Stereo,” Proceedings of the Workshop on Physics-Based Modeling in Computer Vision, pp. 117-125, June 1995.
[29] G. Kay and T. Caelli, “Estimating the Parameters of an Illumination Model Using Photometric Stereo,” Graphical Models and Image Processing, Vol. 57, No. 5, pp. 365-388, 1995.
[30] G. McGunnigle, “The Classification of Textured Surfaces under Varying Illuminant Direction,” Ph.D. Thesis, Department of Computing and Electrical Engineering, Heriot-Watt University, Edinburgh, 1998.
[31] S. K. Nayar, K. Ikeuchi, and T. Kanade, “Determining Shape and Reflectance of Hybrid Surfaces by Photometric Sampling,” IEEE Transactions on Robotics and Automation, Vol. 6, No. 4, pp. 418-431, 1990.
[32] E. Angelopoulou and J. P. Williams, “Photometric Surface Analysis in a Triluminal Environment,” Proceedings of IEEE International Conference on Computer Vision, 1999.
[33] P. N. Belhumeur, D. J. Kriegman, and A. L. Yuille, “The Bas-Belief Ambiguity,” CVPR, pp. 1060-1066, 1997.
[34] S. Georghiades, P. N. Belhumeur, and D. J. Kriegman, “From Few to Many:
Illumination Cone Models for Face Recognition under Variable Lighting and Pose,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 23, No.
6, pp. 643-660, June 2001.
[35] S. Georghiades, “Incorporating the Torrance and Sparrow Model of Reflectance in Uncalibrated Photometric Stereo,” Proceedings of the Ninth IEEE International Conference on Computer Vision, 2003.
[36] K. Ikeuchi, “Determining Surface Orientations of Specular Surfaces by Using
the Photometric Stereo Method,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 3, pp. 661-669, 1981.
[37] E. N. Coleman and R. Jain, “Obtaining 3Dimensional Shape of Textured and Specular Surfaces Using Four-Source Photometry,” Processing of Computer Vision, Graphics, and Image, Vol. 18, pp. 309-328, 1982.
[38] H. D. Tagare and R. J. P. deFigueiredo, “Simulataeous Estimation of Shape and Reflectance Maps from Photometric Stereo,” IEEE International Conference on Computer Vision, pp. 340-343, 1990.
[39] F. Solomon and K. Ikeuchi, “Extracting the Shape and Roughness of Specular Lobe Objects Using Four Light Photometric Stereo,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 466-471, 1992.
[40] H. Rushmeier, G. Taubin, and A. Gueziec, “Applying Shape from Lighting Variation to Bump Map Capture,” In Eurographics Rendering Techniques’97, pp. 35-44, St. Etienne, France, June 1997.
[41] O. Drbohlav and A. Leonardis, “Detecting Shadows and Specularities by Moving Light,” Proceeding of Computer Vision Winter Workshop, Ljubljana, Slovenia, pp. 60-74, 1998.
[42] O. Drbohlav and R. Sara, “Specularities Reduce Ambiguity of Uncalibrated Photometric Stereo,” Proceedings of the 7th European Conference on Computer Vision, Copenhagen, Denmark, 2002.
[43] K. Hayakawa, “Photometric Stereo under a Light Source with Arbitrary Motion,” Journal of the Optical Society of America: A, Vol. 11, No. 11, 1994.
[44] H. Wang, Y. Wang, and H. Wei, “Face Representation and Reconstruction under Different Illumination Conditions,” Proceedings of the 7th International Conference on Information Visualization, 2003.
[45] G. Healey and T. O. Binford, “Local shape from specularity,” Computer Vision, Graphics, Image Processing, Vol. 42, pp. 62-86, 1988.
[46] E. Torrance and E. M. Sparrow, “Theory for off-specular reflection from roughened surfaces,” J. Opt. Amer., Vol. 57, pp. 1105-1114, 1967.
[47] B. T. Phong, “Illumination for computer generated pictures,” Communications of the ACM, Vol.18, No.6, pp. 311-317, Juni 1975.
[48] G. Q. Wei and G. Hirzinger, “Learning shape from shading by a multilayer network,” IEEE Trans. on Neural Networks, Vol. 17, pp. 985-995, 1996.
neural-based reflectance model,” IEEE Trans. on Neural Networks, Vol. 10, pp.1536-1541, 1999.
[50] S. Y. Cho and T. W. S. Chow, “Neural computation approach for developing a 3D shape reconstruction model,” IEEE Trans. on Neural Networks, Vol. 12, No.
5, September 2001.
[51] R. T. Frankot and R. Chellappa, “A method for enforcing integrability in shape from shading algorithms”, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 10, No. 4, pp. 439-451, July 1988.
[52] P. S. Tsai and M. shah, “Shape from shading with variable albedo,” Optical Engineering, Vol. 37, No. 4, April 1998.
[53] C. H. Lee and A. Rosenfeld, “Albedo estimation for scene segmentation,”
Pattern Recognition Lett. 1. 155-160, 1983.
[54] K. P. Horn, “Obtaining shape from shading from shading information,” In The Psychology of Machine Vision, P. H. Winston, Ed, New York: McGraw-Hill, 1975, pp. 115-155.
[55] Online available = http://cvc.yale.edu/projects/yalefacesB/yalefacesB.html.
[56] S. Georghiades, P. N. Belhumeur, and D. J. Kriegman, “From few to many:
Illumination cone models for face recognition under variable lighting and pose,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 23, No.
6, pp. 643-660, June 2001.
[57] S. Y. Cho and T. W. S. Chow, “Learning parametric specular reflectance model by radial basis function network,” IEEE Trans. on Neural Networks, Vol. 11, No. 6, pp. 1498-1503, November 2000.
[58] University of Notre Dame Biometrics Database Distribution, Online available
= http://www.nd.edu/~cvrl/UNDBiometricsDatabase.html.
[59] A. Taleb and C. Jutten, “Nonlinear Source Separation: the Pose-Nonlinear Mixtures,” in Proc. ESANN, Bruges, Belgium, pp. 279-284, Apr. 1997.
[60] A. Taleb and C. Jutten, “Source Separation in Post-Nonlinear Mixtures,” IEEE Trans. on Signal Processing, Vol. 47, No. 10, October 1999.
[61] J. Bell and T. J. Sejnowski, “An Information-Maximization Approach to Blind Separation and Blind Deconvolution,” Neural Computation, Vol. 7, pp.
1129-1159, 1995.
[62] R. T. Frankot and R. Chellappa, “A Method for Enforcing Integrability in Shape From Shading Algorithms,” IEEE Trans. on Pattern Analysis and
Machine Intelligence, Vol. 10, No. 4, pp. 439-451, July 1988.
[63] S. I. Amari, “Natural Gradient Works Efficiently in Learning,” Neural Computation, Vol. 10, pp. 251-276, 1998.
[64] A. Hyvärinen, J. Karhunen, and E. Oja, Independent component analysis, John Wiley & Sons, Inc., 2001.
[65] L. T. Maloney and B. A. Wandell, “Color constancy: A method for recovering surface spectral reflectance,” J. Opt. Soc. Amer. A, vol. 3, pp. 29-33, 1986.
[66] G. Buchsbaum, “A spatial processor model for object color perception,” J.
Franklin Inst., vol. 310, pp. 1-26, 1980.
[67] P. Sallstrom, “Color and physics: Some remarks concerning the physical aspects of human color vision,” Perception, pp.73-09, Stockholm Inst. Phys., Univ. Stockholm, Sweden, 1973.
[68] M. H. Brill, “A device performing illuminant-invariant assessment of chromatic relations,” J. Theor. Biol., vol. 71, pp. 473-478, 1978.
[69] B. A. Wandell, “Color constancy and Natural Image,” Phycica. Scripta., vol.
39, pp. 187-192, 1989.
[70] J. Cohen, “Dependency of the spectral reflectance curves of munsell color chip,” Psychon. Scripta., vol 1, pp. 369-370, Aug. 1964.
[71] D. B. Judd, D. L. MacAdam, and G. Wyszecki, “Spectral distribution of typical daylight as a function of correlated color temperature,” J. Opt. Soc. Amer., vol.
54, pp. 1031-1040, Aug. 1964.
[72] B. A. Wandell, “Color rendering of color camera data,” Color Research and Application, vol. 11, pp. 30-33, 1986.
[73] L. T. Maloney, “A Computational Approach to Color Constancy,” Ph.D.
dissertation, Appl. Psychol. Lab., Stanford Univ., Stanford, CA., 1985.
[74] J. Ho, B. V. Funt, and M. S. Drew, “Separating a color signal into illuminant and surface reflectance components: Theory and Application,” IEEE Trans. on Pattern Recognition and Machine Intelligence, vol. 12, pp. 966-977, 1990.
[75] AgfaIT8.7/2 Color chart, http://www.agfa.com.
[76] R. W. G. Hunt, “Measuring colour,” Ellis Horwood series in applied science and industrial technology, Halsted Press, New York, NY, 1989.
[77] H. R. Kang, “Color technology for electronic imaging devices,” SPIE- The international Society for Optical Engineering, Washington, 1997.