1.1 Demosaicking
The natural images contain three different spectral channels, i.e. red (R), green (G) and blue (B). However, each individual optical sensor is able to capture only one single color of the three color bands. That’s means we need three sensors to get the full color in one pixel. In order to reduce the cost and size, most digital still cameras (DSC) today using a single-chip CCD or CMOS sensor array whose surface is covered with a color filter array (CFA). The CFA is located between the lens and the sensors. The following figure describes this system.
Fig. 1.1 Optical system in digital still camera
According to this arrangement, there is only one color be sampled of each pixel. To get a full-color image, other colors must be estimated or interpolated from the neighboring samples. This color filter array interpolation is known as demosaicking.
Bayer pattern [1] is the most popular type of color filter array. It has four possible arrangements which can be described as Fig. 1.2.
Fig. 1.2 Four possible arrangements of Bayer CFA
As we see, G samples are obtained by a checkerboard lattice and R and B samples are obtained by a rectangular lattice. The G pixels are sampled at a twice rate than others as the human eye is more sensitive to G compared to R and B.
However, we choose the first arrangement of Fig. 1.2 for the whole discussion.
1.2 Narrow-band Image
For the last few years, endoscope has been spread widely for diagnosing abnormal pathological changes. Except for its less invasive, for the most important point, it is capable of identifying early-stage lesions of cancers. Early detection and treatment of cancers has become a demanding goal in the diagnosis field.
Physician detects those abnormal regions based on the color, shape, and the surface pattern of patient’s mucosa. Throughout the judgment, if the lesions is observed and identified as a tumor, treatment is decided among surgery or other therapy. In order to afford complete cure by endoscope, lesions must be found at the early stage.
Therefore, for the sake of better identifying of abnormal findings and more accurate diagnostic performance, several investigators and researches have been done for the same object, the improvement of the endoscope system and the enhancement of its observation.
Narrow-band Image (NBI) [28], [29] can achieve the goal we mentioned above.
This kind of endoscope enables physicians to detect tumors in the deep layer of mucosa. Under the narrow-band light, due to the absorption of hemoglobin, the fine vessels of mucosa can be displayed clearer than under the white light. K. Gono in [27]
had proved that center wavelengths of 415nm and 540nm enable to emphasize images of vessels. According to the experiments of backside mucosa of human tongue, as shown in Fig. 1.3, the 415nm image can display the structure of fine vessels at the
Fig. 1.3 Images of backside mucosa of human tongue under the light of different wavelengths.1
superficial layer, and the 540nm image can present the vessel structure in the relatively deep layer of mucosa if the living tissue is observed under narrow band light 415 and 540nm. Besides, Gono also proved that under the same center wavelength 415nm, broad-bandwidth (100nm) of 415nm light is not enough to show the fine vessels of mucosa. However, narrow-bandwidth (60nm) of 415nm light can increase the contrast of micro vessels in superficial layer.
Nowadays, NBI has been popularly used in endoscope throughout the world.
Olympus developed the first endoscope system and now its latest platform “EVIS LUCERA SPECTRUM” imaging platform (SPECTRUM) offers not only NBI, but also provides autofluorescence image (AFI) and infrared image (IRI). All of them are HDTV image quality. In order to display NBI in color image, SPECTRUM is designed so that 415nm image is assigned to B and G plane and 540 nm image is assigned to R plane. This assignment of the final image let the fine vessels are displayed in brownish-red, and thicker vessels in the deep layer are displayed in cyan color. Physicians observed the abnormal findings when making overall judgment, they can switch to NBI mode to investigate any pathological changes through the high-contrast image of the vessels.
1 This figure refers to [27]
1.3 Motivation
The object of our experiment is to design the gastrointestinal capsule endoscope with NBI image. The sensor of existing NBI system has two layers of optical filters [27]. The first layer is mode selector for choose the vision mode of white light image (WLI) or NBI. The second layer is used to sense RGB color. However, it is not practical to place two layers of sensors to capture RGB and NB color individually.
Similar to the reasons of most commercial digital cameras today, demosaicking is a way to solve this difficulty. The following figure describes the existing system diagram. We use two different light sources: white light and NB light. And the CFA
Fig. 1.4 Sensor structure of existing capsule endoscope system
we used is similar to Bayer pattern but withdrawing one G sample, as shown in Fig.
1.4. When the capsule endoscope captures the image, these two sources are emitted continually. While the source is white light, the CFA is open for sense mosaic RGB color. Otherwise, the sensor is record the N information under gray-level resolution at the location of withdrawing G.
Now we desire to develop a demosaicking algorithm with satisfied results so that we can make use of it on the endoscope system. Demosaicking problem has been studied and researched broadly in recent years. Lots of demosaicking algorithms are also presented continually. The simplest way is the bilinear interpolation. Although it is easy to implement, the reconstructed image can not maintain the edge information and high-frequency component. Later, many characteristics or correlations have been utilized in order to get better results. Some may aim to suppress artifacts around edge, and some may exploit the strong inter-plane frequency correlation to improve the high-frequency component. Among several ways, recursive methods can achieve a better result [2]. The iteration had been widely used in demosaicking since the closed-loop estimation can approximate the original color values and hence enable to reduce these artifacts. However, too much iteration may cause the zipper effect and tend to de-saturate the color of image [20]. Besides, in our research, we found that some output images can be closed to the original one after many times of iterations
but some can not. If we can diagnose the image is suitable to execute the iteration beforehand, we can decide to enter the image to the recursive step or not. Thus, the discussion of how to adaptively use the iteration as we proceeding demosaicking is also an important part in this dissertation.
1.4 Outline
This thesis is structured as below: