Characteristic Image Decomposition
2.3 Proposed Applications
In this dissertation, a computational framework is developed for characteristic image decomposition from a single image using environmental supporting information to classify the edges in the image. The classified edges are separated into desired characteristic components, and each component is integrated to form the target image.
Three applications of this computational framework (interference reflections, dichromatic reflections, and intrinsic images) are developed in this dissertation. The methodologies of interference reflection images, highlight reflection images, and intrinsic images extraction are described in Chapters 3, 4 and 5, respectively.
Characteristic image decomposition from a single image
Chapter 3
Interference Reflection Separation
Characteristic image decomposition from a single image
The interference image is defined as the superpositioning of a reflection image and an object image. A technique for separating reflection and object components of a single interference image is presented. The proposed method does not try to separate reflection and object components directly from the interference image, but to classify edges of the interference image into either reflection or object related. Our method utilizes total variation (TV) method, blur measure, and region segmentation as evidence with a fuzzy integral technique to classify the edge pixels. Based on the results of edge pixel classification, the reflection and object components of the input image are reconstructed.
Compared to previous published research, the proposed method is fast and requires no manual operations. The experimental results have demonstrated that the proposed method can perform separation of a single interference image effectively with small misadjustments and rapid convergence.
3.1 Introduction
Transparent covers made of glass or plastic are commonly used to protect demonstration objects in galleries, museums, jewel shops, and department stores.
However, transparent covers easily reflect the scenes located in front of the covers while viewing exhibition objects behind the covers. Figure 3.1(a) shows an image taken for a kitchen in a restaurant through a glass shield. The reflection from the glass interferes with the view of the kitchen behind the shield. We call such an image an interference image. Figure 3.1(b) shows an image taken by a camera with a polarizing filter manually adjusted to reduce as many reflections as possible from the glass shield. The
Chapter 3 Interference reflection separation
3-3 image obtained by further rotating the polarizing filter 90° is displayed in Figure 3.1(c).
This image is hardly visible in the original image (see Figure 3.1(a)). The proposed technique, which in a sense acts as a pseudo-polarizer, decomposes an interference image into its object and reflection images.
In this paper, a technique for eliminating the effect of reflections from an interference image is presented. Let Ii denote the input interference image and Io denote the image after removing the effect of reflections from Ii. We refer to Io as the object image, which contains the exhibition objects of our interest. Let Ir = Ii - Io, which is referred to as the reflection image. In this paper, Io and Ir are collectively called the component images of Ii. Although we are interested in object image Io, reflection image Ir preserves the information about the reflected scene, which can be of use in some applications.
(a) (b) (c)
Figure 3.1. (a) An interference image ; (b) an image taken by a camera with a polarizing filter manually adjusted to reduce as much reflection as possible; (c) the image obtained by rotating the polarizing filter 90° in (b).
Characteristic image decomposition from a single image
Many methods [AGR 05, BRO 03, FAR 99, IRA 92, LEV 04, LEV 04-1, LEV 07, SAR 04, SAR 05, SHE 99, SZE 00, TSI 03] have been proposed for extracting component images from interference images. They can be broadly categorized into three classes: using multiple images (acquired by rotating at different angles a polarizing filter accompanying a camera,) video sequences, stereo images and flash/no-flash image pairs.
Extracting component images from a single interference image assuming that interference decomposition should result in component images having fewer edges and corners than the original image. A simpler problem to extracting component images from a single interference image, in which the user provides labels of component images for a number of critical gradients in the interference image. The literal review of interference images extractions have been discussed in Sec. 1.2.1.
In this study, we propose a technique to automatically label boundary pixels. To this end, a number of evidences are derived from the input interference image, based on which boundary pixels are classified as being object- or reflection-related. Since no iteration is involved in the reconstruction of component images, our method greatly reduced the processing time and achieved reasonable results.
The rest of this paper is organized as follows. The fundamental idea and the flowchart of the proposed interference image decomposition method are addressed in Sec. 3.2. The details of techniques, including total variation decomposition, blur measure, region segmentation, fuzzy integral, classification refinement, and component image reconstruction are addressed in Sec. 3.3. The experimental results of the proposed method are demonstrated in Sec. 3.4. Concluding remarks and future work are finally given in Sec. 3.5.
Chapter 3 Interference reflection separation
3-5 3.2 Interference Image Decomposition
Figure 3.2 shows a block diagram for the proposed approach, which consists of four major steps: (1) boundary detection, (2) evidence extraction, (3) boundary classification, and (4) component image reconstruction. Let I = (Ir, Ig, Ib) represent the interference input image, where Ir, Ig and Ib are the red, green and blue components of the input image. The first three steps of the proposed approach are applied separately to each of the three color component images. The results are then integrated in the last step to form the object and reflection images of the input image.
Figure 3.2. Bock diagram for the proposed approach to decomposing an interference image into its reflection and object images.
Characteristic image decomposition from a single image
Consider any color component image, say Ii, i = r, g, b. In the first step of boundary detection, Ii is convolved with a horizontal derivative filter fh and a vertical derivative filter fv. Let Eih and Eiv denote the resultant derivative magnitude images, respectively. In this study, Prewitt filters are used as the derivative filters. Thereafter, four evidence maps, including a cartoon map C, a texture map X, a blur map B, and a region map S are derived from image I. The first two maps are derived using a total variation method to be addressed in Sec. 3.3.1. The derivations of the blur map and the region map are discussed in Sec. 3.3.2 and Sec. 3.3.3, respectively. In the third step of boundary classification, edges are classified as being object- or reflection-related based on the first third evidence maps using a fuzzy integral approach. Moreover, the classification results is further refined based on the information of the region map. The reflection and object images are finally reconstructed from the associated edges in the last step. The flowchart of detail processes to decompose reflection and object images from the interference image is illustrated in Fig. 3.3. In the following, we address in depth the individual steps.
Chapter 3 Interference reflection separation
Figure 3.3. Flowchart for separating reflection and object images from a single image.