Feature-Pair Distributions

For each of the I, RG, and BY channels, we compute the feature-pair distribution as proposed in [12]. As mention in Section 2.4, Jen et al. proposed the concept of intensity-pair distribution for the enhancement of image contrast. Since this distribution possesses both local information and global information, it may offer useful information for us to detect visual saliency regions in the image. In a feature-pair distribution, the global information tells us what kinds of features are common in the image, while the local information tells us which portions of the image may exhibit large contrast. Hence, by properly using the pair-distributions of the I, RG, and BY channels, we can efficiently detect those image portions with unusual appearance or with stronger contrast.

To establish the feature-pair distribution for the I channel, we check at each pixel the intensity pairs between that pixel and its 8-connection neighbors. Figure 2-15 shows an illustration of a pixel and its 8-connection neighbors. If we denote the I values of these nine pixels as A to I, respectively, then the eight intensity pairs {(E, A), (E, B), (E, C), (E, D), (E, F), (E, G), (E, H), and (E, I)} are formed and accumulated in the feature-pair distribution. Clearly, we can expect that the intensity pairs over smooth regions will lie around the 45-degree line; whereas these intensity pairs across edges will lie somewhere away from the 45-degree line.

Figure 3-3 shows an example of the intensity-pair distribution. For the airplane image shown in Figure 3-3, since the sky and grass are the major backgrounds of the image, the intensity pairs over these two regions form two major clusters in the intensity-pair distribution. Here, we intentionally colorize these two clusters to indicate their correspondence. On the other hand, the aircrafts map to a smaller cluster in the lower-left corner of the distribution. Moreover, the intensity pairs over the sky-grass boundary and the aircraft-sky boundary form four clusters (represented in red color) far away from the 45-degree line. Based on this intensity-pair distribution, we can easily deduce that the boundary between the aircraft and the sky exhibits a stronger contrast than the sky-grass boundary. With the facts that (1) the aircraft is

“less common” than the sky and the grass; and (2) the aircraft has a stronger contrast with respect to its background, we may deduce that these two aircrafts may catch the attention of most observers.

Figure 3-3 A matching example of modified intensity-pair distribution

Here rises a question: how large should the input image be? In Figure 3-4, we show four intensity-pair distributions with their input image being scale 0 to scale 3.

When the scale is increased by 1, the image’s height and width are reduced by 2, respectively. The choice of scale is image dependent. However, in Scale 0 or Scale 1, the image usually contains quite a large number of scattered data and requires longer processing time. Hence, in our approach, we typically work on Scale 2 and Scale 3, as shown in Figure 3-4(c) and (d).

(a) scale 0 (b) scale 1

(c) scale 2 (d) scale 3

Figure 3-4 An example of intensity-pair distribution with different scale input

Based on the same concept, we can form the RG-pair distribution for the RG channel, and the BY-pair distribution for the BY channel. These three feature-pair

distributions may offer us plentiful clues about the global statistics and the local variations of the image contents.

(a) Input image (b) Intensity-pair distribution

(c) RG color-pair distribution (d) BY color-pair distribution Figure 3-5 An example of the feature-pair distributions

3.2.1. C ^LUSTERING

To identify the most common properties in the image, we need to identify the major clusters in the feature-pair distributions. From the feature-pair distributions obtained at the previous section, there are apparent clusters which we can tell easily.

The existing clustering algorithms seem to be a good tool for us to segment each cluster out. Figure 3-6 is an example of the intensity-pair distribution processed by the mean-shift clustering algorithm. The resulting clusters are reasonably good.

Unfortunately, these existing clustering algorithms are usually computationally expensive and time-consuming. These disadvantages disobey our major requirement that the system should not possess complicated computations and should be fast enough for real-time processing and analysis.

(a) (b)

Figure 3-6 An example of intensity-pair distribution after mean-shift clustering (a)intensity-pair distribution (b) mean-shift clustering algorithm passing through (a)

3.2.2. 3-D H ^ISTOGRAM R EPRESENTATION

To simplify the computations, we choose another approach that operates over the feature-pair distributions directly. In Figure 3-7, we show the 3-D histogram representation of the feature-pair distribution. This 3-D histogram is formed by dividing the x-y plane into a few uniform cells and count for each cell the total number of feature pairs within that cell. Clearly, we can expect that, in general, most clusters occur around the diagonal line in the 3-D histogram since most regions in a natural image are smooth. Moreover, the background elements would yield the largest cluster since the background usually occupies the largest area in the image. On the contrary, foreground objects usually correspond to smaller clusters. Besides, those clusters away from the diagonal line correspond to the boundary regions or the texture regions in the image.

Figure 3-7 The 3-D histogram representation of feature-pair distribution.

In this 3-D histogram, we denote the cell at the intersection of the ith column and jth row as C(i,j). We further define a cell to be a “diagonal” cell when |i-j| ≤ D_th, where Dth is a pre-selected threshold. On the contrary, a cell is defined as “off-diagonal” if

|i-j| > Dth.