Most outdoor vision systems such as surveillance, tracking, object recognition, remote-sensing, and autonomous vehicles require images with clear visibility for robust features detection. Without clear visibility (for example: bad weather and underwater), these systems need more reliability and stability to maintain the detection quality.
Despite their advantages, the performances of existing detection systems are limited in underwater environment and bad weather conditions like haze, rain, and snow. Note that the disturbance weather and environment effects removal is a difficult and challenging problem. The key points of the difficulty are illustrated as follows. First, images are degraded exponentially with the distance between the objective and observer. Second, recovering the scene structure from a single image is an ill-posed problem. Third, the contrast and color fidelity of images are drastically degraded due to atmospheric and water particles. To make outdoor and underwater vision systems more flexible in all environmental conditions, the new image and video processing algorithms are proposed to eliminated or reduced the effect of haze, rain, snow, underwater with high image quality and color naturalness.
In this thesis, we investigate the dehazing, rain removing, snow removing, and underwater enhancing processing in harsh environment. First, we introduce the physical properties and visual effects and then reference the three existing typical single image dehazing methods: contrast-based [1], independent component analysis [2], and dark channel prior-based analysis [3]. A robust, flexible, and effective dehazing method is proposed during daytime and nighttime to further improve the dehazing quality. For presentation completeness, three existing typical rain and snow removal methods in single image, including guidance image based [4], image decomposition analysis [5],
adaptive nonlocal means filter [6], and frequency-based analysis [7] are also introduced in the literature. For rain and snow removing investigation, we design a simple but effective removal method by dividing the rain or snow removal scheme into two parts, the first part is detection of rain or snow and the second part is inpainting. Here, a proper threshold is important. Furthermore, we extend the column into both side columns to produce a block-matrix for rain removal so that it may get more benefits for single image consideration when streak or flake is not obvious. The fuzzy random impulse reduction method is also used for noise removal. Specifically, to obtain vision quality of a resulting image, color transfer is utilized to protect the final snow-free image’s color from high dynamic. These steps may be used in many applications for outdoor vision systems. Besides, it shows the worthy and important referencing for image and video processing categories by these steps. Three existing typical underwater enhanced methods: histogram-based equalization [8], wavelength-based compensation [9], and fusion based [10] are also introduced in the literature. To further improve the underwater image and video qualities, we design a simple but effective underwater enhanced method by combining the color correction, contrast stretching, and histogram equalization.
In the future, real-time processing is necessary and mandatory for all system applications. For this sake, our algorithms must be modified with more effective, more adaptive, and less complexity for authentic real-time realization. The preliminary strategy is to combine with the cloud computing. In our opinion, the haze, rain, and snow removal and underwater enhancement techniques will be more practical by the real-time processing.
REFERENCE
[1] R. Tan., “Visibility in Bad Weather from a Single Image,” in IEEE Conference on Computer Vision and Pattern Recognition, June 2008, pp. 1–8.
[2] R. Fattal, “Single Image Dehazing,” ACM Transactions on Graphics, vol. 27, pp.
72:1-72:9, August 2008.
[3] K. He, J. Sun, and X. Tang, “Single Image Haze Removal Using Dark Channel Prior,” in IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 1956-1963.
[4] J. Xu, W. Zhao, P. Liu, and X. Tang, “An Improved Guidance Image Based Method to Remove Rain and Snow in a Single Image,” CCSE Computer and Information Science Journal 5(3), 2012, pp. 49-55.
[5] L.-W. Kang, C.-W. Lin, and Y.-H. Fu, “Automatic single-frame-based rain streak removal via image decomposition,” IEEE Trans. Image Processing, vol. 21, no. 4, pp. 1742-1755, Apr. 2012.
[6] J.-H. Kim, C. Lee, J.-Y. Sim, and C.-S. Kim, “Single-image deraining using an adaptive nonlocal means filter,” 2013 20th IEEE International Conference on Image Processing (ICIP), vol., no., pp.914-917, 15-18 Sept. 2013.
[7] P. Barnum, T. Kanade, and S. Narasimhan, “Spatio-Temporal Frequency Analysis for Removing Rain and Snow from Videos,” in Conjunction with International Conference on Computer Vision, 2007, pp. 1-8.
[8] K. Iqbal, R. Abdul Salam, A. Osman, and A. Zawawi Talib, “Underwater image enhancement using an integrated color model,” Int. J. Comput. Sci., vol. 34, no. 2, pp. 2–12, 2007.
[9] J.-Y. Chiang and Y.-C. Chen, “Underwater Image Enhancement by Wavelength Compensation and Dehazing,” IEEE Transactions on Image Processing, vol.21, no.4, pp.1756-1769, April 2012.
[10] C. Ancuti, C.O. Ancuti, T. Haber, and P. Bekaert, P., “Enhancing underwater images and videos by fusion,” Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on , vol., no., pp.81,88, 16-21 June 2012.
[11] S. G. Narasimhan and S. K. Nayar, “Vision and the atmosphere,” International Journal of Computer Vision, vol. 48, pp. 233-254, July-August 2002.
[12] L. Kratz, and K. Nishino, “Factorizing scene albedo and depth from a single foggy image,” IEEE Twelfth International Conference on Computer Vision ICCV’09, pp. 1701-1708.
[13] J. P. Tarel and N. Hautière, “Fast visibility restoration from a single color or gray level image,” IEEE International Conference on Computer Vision (ICCV’09), pp.
2201–2208. Kyoto, Japan 2009.
[14] N. Carlevaris-Bianco, A. Mohan, and R. M. Eustice, “Initial results in underwater single image dehazing,” in Proc. IEEE OCEANS, 2010, pp. 1–8.
[15] Y. Y. Schechner, S. G. Narasimhan, and S. K. Nayar, “Instant Dehazing of Images Using Polarization,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition , 2001, pp. 1:325–332.
[16] S. Shwartz, E. Namer, and Y. Schechner, “ Blind haze separation,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2006, pp. 1984 - 1991.
[17] F. Cozman and E. Krotkov, “Depth from scattering,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1997, pp. 801–806.
Conference on Computer Vision, 1999, pp. 820–827.
[19] S. G. Narasimhan and S. K. Nayar, “Contrast restoration of weather degraded images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.
25, pp. 713 - 724, June 2003.
[20] N. Hautiere, J. -P Tarel, and D. Aubert, “Towards Fog-Free In-Vehicle Vision Systems through Contrast Restoration,” Computer Vision and Pattern Recognition, 2007. CVPR '07. IEEE Conference on , vol., no., pp.1,8, 17-22 June 2007.
[21] S. Narasimhan and S. Nayar, “Interactive deweathering of an image using physical models,” in IEEE Workshop on Color and Photometric Method in Computer Vision, 2003, pp. 1- 8.
[22] J. Kopf, B. Neubert, B. Chen, M. Cohen, D. Cohen-Or, O. Deussen, M.
Uyttendaele, and D. Lischinski, “Deep photo: Model-based photograph enhancement and viewing,” ACM Transactions on Graphics, Vol. 27, pp.
116:1-116:10, December 2008.
[23] N. Hautiere, J. Tarel, and D. Aubert, “Toward fog-free invechicle vision systems through contrast restoration,” in IEEE Conference on Computer Vision and Pattern Recognition, 2007, pp. 1- 8.
[24] C. Ancuti, C. Hermans, and P. Bekaert, “A fast semi-inverse approach to detect and remove the haze from a single image,” in: Proc. ACCV, 2011, pp. 501–514.
[25] J. Yu and Q. Liao, “Fast single image fog removal using edge-preserving smoothing,” in: Proc. IEEE ICASSP, 2011, pp. 1245–1248.
[26] P. Carr and R. Hartley, “Improved single image dehazing using geometry,” in:
Proc. DICTA, 2009, pp. 103–110.
[27] L. Schaul, C. Fredembach, and S. Süsstrunk, “Color image dehazing using the nearinfrared,” in: Proc. IEEE ICIP, 2009, pp. 1629–1632.
[28] X. Dong, X. Hu, S. Peng, and D. Wang, “Single color image dehazing using sparse priors,” in: Proc. IEEE ICIP, 2010, pp. 3593–3596.
[29] A. Levin, D. Lischinski, and Y. Weiss, “A closed form solution to natural image matting,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2006, pp. 61 - 68.
[30] W B. Thompson, P. Shirley, and J A. Ferwerda, “A spatial post-processing algorithm for images of night scenes,” Journal of Graphics Tools, vol. 7, pp. 1-11, November 2002.
[31] K. Gibson, T. Nguyen, “On the Effectiveness of the Dark Channel Prior for Single Image Dehazing by Approximating with Minimum Volume Ellipsoids,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2011, pp. 1253-1256.
[32] S.-C. Pei, T.-Y. Lee, “Nighttime haze removal using color transfer pre-processing and Dark Channel Prior,” Image Processing (ICIP), 2012 19th IEEE International Conference on , vol., no., pp.957,960, Sept. 30 2012-Oct. 3 2012.
[33] E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, “Color Transfer between Images,” IEEE Trans. on Computer Graphics and Applications, vol.21, pp. 34–41, Sep/Oct 2001.
[34] R. Schettini, F. Gasparini, S. Corchs and F. Marini, “Contrast image correction method,” Journal of Electronic Imaging, vol.19, no. 2, pp. 023005-1 – 023005-11, April 2010.
[35] B. Stoppee and Stoppees, Guide to Photography and Light, Mar. 2010.
the gap between color images and the human observation of scenes, ,” IEEE Transactions on Image Processing, vol. 6, no. 7, pp. 965 –976, July 1997.
[37] Gaurav Sharma, Digital Color Imaging Handbook, CRC Press, 2003.
[38] C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,”
Computer Vision, 1998. Sixth International Conference on , vol., no., pp.839,846, 4-7 Jan 1998.
[39] S.-C. Pei, T.-Y. Lee, “Effective image haze removal using dark channel prior and post-processing,” Circuits and Systems (ISCAS), 2012 IEEE International Symposium on , vol., no., pp.2777,2780, 20-23 May 2012.
[40] R. He, Z. Wang, Y. Fan, D.D. Feng, “Multiple scattering model based single image dehazing,” Industrial Electronics and Applications (ICIEA), 2013 8th IEEE Conference on , vol., no., pp.733,737, 19-21 June 2013
[41] C.O. Ancuti and C. Ancuti, “Single Image Dehazing by Multi-Scale Fusion,”
Image Processing, IEEE Transactions on , vol.22, no.8, pp.3271,3282, Aug.
2013.
[42] B. Li, S. Wang, and Y. Geng, “Image enhancement based on Retinex and lightness decomposition,” in IEEE International Conference on Image Processing, 2011, pp.
3417–3420.
[43] X. Dong, G. Wang, Y. Pang, W. Li, J. Wen, W. Meng, and Y. Lu, “Fast efficient algorithm for enhancement of low lighting video,” in Proceedings of International Conference on Multimedia and Expo. IEEE, 2011, pp. 1–6.
[44] J.-H. Kim, W.-D. Jang, J.-Y. Sim, and C.-S. Kim, “Optimized contrast enhancement for real-time image and video dehazing,” Journal of Visual Communication and Image Representation, Volume 24, Issue 3, April 2013, Pages 410-425.
[45] K. Nishino, L. Kratz, and S. Lombardi, “Bayesian Defogging,” Int. J. Comput.
Vision 98, 3 (July 2012), 263-278.
[46] J. Zhang, L. Li, G. Yang, Y. Nishino,and J. Sun, “Local albedo-insensitive single image dehazing,” Vis. Comput., 2010, 26, (6– 8), pp. 761–768.
[47] K. Garg, and S. Nayar, “Detection and Removal of Rain from Videos,” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004, pp. I-528 - I-535.
[48] K. Garg, and S. Nayar, “When does a camera see rain?,” Computer Vision, 2005.
ICCV 2005. Tenth IEEE International Conference on , vol.2, no., pp.1067,1074 Vol. 2, 17-21 Oct. 2005.
[49] K. Garg and S. K. Nayar, “Vision and rain,” Int. J. Comput. Vis., vol. 75, no. 1, pp.
3–27, Oct. 2007.
[50] X. Zhang, H. Li, Y. Qi, W. K. Leow, and T. K. Ng, “Rain Removal in Video by Combining Temporal and Chromatic Properties,” in IEEE International Conference on Multimedia and Expo, 2006, pp. 461 - 464.
[51] J. Bossu, N. Hauti`ere, and J. P. Tarel, “Rain or snow detection in image sequences through use of a histogram of orientation of streaks,” Int. J. Comput.
Vis., vol. 93, no. 3, pp. 348–367, July 2011.
[52] H. Hase, K. Miyake, and M. Yoneda, “Real-time snowfall noise elimination,” in IEEE International Conference on Image Processing, 1999, pp. 406 - 409.
[53] S. Starik and M. Werman, “Simulation of rain in videos,” in Int’l. Workshop on Texture Analysis and Synthesis, 2003.
[54] J. Xu, W. Zhao, P. Liu, and X. Tang, “Removing rain and snow in a single image using guided filter,” Computer Science and Automation Engineering (CSAE),
[55] K. He, J. Sun and X. Tang, “Guided Image Filtering,” in European conference on Computer vision, 2010, pp. 1-14.
[56] N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,”
in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., San Diego, CA, Jun. 2005, vol. 1, pp. 886–893.
[57] S. Mallat and Z. Zhang, “Matching pursuits with time-frequency dictionaries,”
IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3397–3415, Dec. 1993.
[58] H. Takeda, S. Farsiu, and P. Milanfar, “Kernel regression for image processing and reconstruction,” IEEE Trans. Image Process., vol. 16, no. 2, pp. 349–366, Feb.
2007.
[59] L. Vincent, “Morphological grayscale reconstruction in image analysis:
Applications and efficient algorithms,” IEEE Trans. Image Process., vol. 2, no. 2, pp. 176–201, 1993.
[60] N. Otsu, “A threshold selection method from gray-level histograms,” Automatica, vol. 11, nos. 285–296, pp. 23–27, 1975.
[61] D. Eigen, D. Krishnan, and R. Fergus, “Restoring an Image Taken through a Window Covered with Dirt or Rain,” 2013 IEEE International Conference on Computer Vision (ICCV), vol., no., pp.633,640, 1-8 Dec. 2013.
[62] A. Cord and D. Aubert, “Towards rain detection through use of in-vehicle multipurpose cameras,” 2011 IEEE Intelligent Vehicles Symposium (IV), vol., no., pp.833,838, 5-9 June 2011.
[63] A. Cord and N. Gimonet, “Detecting Unfocused Raindrops: In-Vehicle Multipurpose Cameras,” IEEE Robotics & Automation Magazine, vol.21, no.1, pp.49-56, March 2014.
[64] S. You, R. T. Tan, R. Kawakami, and K. Ikeuchi, “Adherent Raindrop Detection
and Removal in Video,” 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol., no., pp.1035,1042, 23-28 June 2013.
[65] O. Le Meur, M. Ebdelli, and C. Guillemot, “Hierarchical Super-Resolution-Based Inpainting,” IEEE Transactions on Image Processing, vol.22, no.10, pp.3779-3790, Oct. 2013.
[66] Z. Xu and J. Sun, “Image Inpainting by Patch Propagation Using Patch Sparsity,”
IEEE Transactions on Image Processing, vol.19, no.5, pp.1153,1165, May 2010 [67] S. Schulte, M. Nachtegael, V. D. Witte, D. V. der Weken and E. E. Kerre, “A fuzzy
impulse noise detection and reduction method,” IEEE Transactions on Image Processing, vol. 15, pp. 1153-1162, 2006.
[68] E. E. Kerre, Fuzzy Sets and Approximate Reasoning, Xian Jiaotong University Press, 1998.
[69] P. Sand and S. Teller, “Particle video: long-range motion estimation using point trajectories,” In CVPR, 2006.
[70] S.-C. Pei, Y.-T. Tsai and C.-Y. Lee, “Removing Rain and Snow in a Single Image Using Saturation and Visibility Features,” 2014 IEEE International Conference on Multimedia & Expo Workshop, Chengdu, China, July 14-18, 2014.
[71] Y. Schechner and Y. Averbuch, “Regularized image recovery in scattering media,”
IEEE Trans Pattern Anal Mach Intell., 2007.
[72] Zuiderveld, Karel, “Contrast Limited Adaptive Histograph Equalization,” Graphic Gems IV. San Diego: Academic Press Professional, 1994. 474–485.
[73] M. S. Hitam, W. N. J. H. W. Yussof, E. A. Awalludin, and achok, “Mixture contrast limited adaptive histogram equalization for underwater image enhancement,” 2013 International Conference on Computer Applications
[74] S. Q. Duntley, “Light in the sea,” J. Opt. Soc. Amer., vol. 53, no. 2, pp. 214–233, 1963.
[75] Q. Liu, M. Y. Chen, and D. H. Zhou, “Fast haze removal from a single image,”
2013 25th Chinese Control and Decision Conference (CCDC), vol., no., pp.3780-3785, 25-27 May 2013.
[76] T. O. Aydin, R. Mantiuk, K. Myszkowski, and H.-S. Seidel, “Dynamic range independent image quality assessment,” In SIGGRAPH, 2008.
[77] [Online]. Available: http://www.youtube.com/user/bubblevision.
[78] L. Chao and M. Wang, “Removal of water scattering,” 2010 2nd International Conference on Computer Engineering and Technology (ICCET), vol.2, no., pp.V2-35,V2-39, 16-18 April 2010.