ELSEVIER Pattern Recognition Letters 17 (1996) 481-490
Pattern Recognition Le~ers
Reduction of color space dimensionality by moment-preserving thresholding and its application for edge detection
in color images
1Chen-Kuei Yang a,b, Wen-Hsiang Tsai a,*
a Department of Computer and Information Science, National Chiao Tung University, HsinChu, Taiwan 300, ROC b Department o f Information Management, Ming Chuan College, Taipei, Taiwan 111, ROC
Received 17 May 1995; revised 22 November 1995
Abstract
A method for reduction of color space dimensionality by moment-preserving thresholding and its application in edge detection for color images is proposed. An input color image is partitioned into n x n non-overlapping blocks. A moment-preserving thresholding technique is then applied individually to each color plane of each image block. Two sets of (R, G, B) tristimulus values are obtained from the thresholding results to form two representative color vectors for each block. The difference vector between these two representative color vectors is used as an axis onto which all the data in the block are projected to reduce the color space to one dimension. A single-spectral image block is so obtained. Due to the use of analytic formulas in the thresholding step, the proposed dimensionality reduction method is found faster than the KL expansion or vector median approaches which are also applicable for dimensionality reduction. An (n + 1)X ( n + 1) circular window is selected to sample the resulting single-spectral image, which in turn includes the n × n square block.
Some mass moments of the window data are computed and used for edge detection in the circular window. Due to the use of the larger detection window which results in smaller overlapping detection areas, the computation time for the edge detection step is reduced, compared with other similar approaches using overlapping detection windows. Experimental results show that the proposed approach is effective in reduction of color space dimensionality and edge detection in color images.
Keywords: Dimension reduction; Color edges; Moment-preserving
1. I n t r o d u c t i o n
In color image processing, the color o f a pixel is usually given as three values corresponding to the tristimuli o f red ( R ) , green ( G ) , and blue ( B ) . Many
* Corresponding author. Email: [email protected] J This work was supported partially by National Science Coun- cil, Republic of China under Grant NSC83-0408-E009-010.
different coordinate systems have been e m p l o y e d for the specification o f color as discussed in (Pratt, 1991). Typically, the tristimulus values of (R, G, B) are h i g h l y correlated with one another (Pratt, 1971).
In the d e v e l o p m e n t of efficient quantization, coding, and processing techniques for color images, it is often desirable to w o r k with components that are uncorrelated. A lot o f literature (Pratt, 1991) has been d e v e l o p e d to transform the color coordinate system for different purposes o f color image process- 0167-8655/96/$12.00 © 1996 Elsevier Science B.V. All rights reserved
SSDI 0 1 6 7 - 8 6 5 5 ( 9 5 ) 0 0 1 12-3
482 C.-K. Yang, W.-H. Tsai / Pattern Recognition Letters 17 (1996)481-490
ing. But they still use a set of tristimulus values to specify a color. For color image processing, such approaches usually apply grey-scale image process- ing techniques individually to each spectral band according to the characteristics of the spectral band, and then merge the processing results in some ways.
The computation is slow because color model trans- formation is performed and the three spectral bands are individually treated. In this study, a method for reduction of color space dimensionality is proposed.
Only one spectral band is preserved. So, most exist- ing grey-scale image techniques can be applied to color images without using the merge operation.
Edge detection is one of the basic techniques for image segmentation and compression, and various techniques for edge detection have been proposed (Pratt, 1991; Gonzalez and Wintz, 1987). Surveys of literature on edge detection can be found in (Davis, 1975; Fu and Mui, 1981; Peli and Malah, 1982; Di Zenzo, 1983). Edges are defined as a discontinuity in some image attributes, usually the brightness for grey-scale images. For color images, the situation is different. Several definitions of color edges have been proposed (Pratt, 1991). A color edge can be said to exist if and only if the luminance field contains an edge. This definition ignores discontinu- ities in hue and saturation that occur in regions of constant luminance. Another way to define a color edge is to check if an edge exists in any of its constituent tristimulus components. A third defini- tion is based on forming the sum of gradients of the tristimulus values or some linear or nonlinear color components. A color edge is said to exist if the gradient exceeds a threshold.
Most existing edge detection techniques for grey- scale images can be applied to color images with some merge operations on the detected results of the three spectral bands (Pratt, 1991). However, this needs a lot of computation time because they process pixels one by one and treat three spectral bands separately. Nevatia (1977) extended the Hueckel edge operator (Hueckel, 1971) to color images. He com- puted the edges in three chosen color components separately and merged them by a certain procedure.
Di Zenzo (1986) extended the gradient-based edge detection technique to multispectral images and com- bined the three color component results by taking the RMS (root mean square), or the sum, or the maxi-
mum of their absolute values. Shiozaki (1986) devel- oped an entropy operator using entropy of brightness in a local region of a picture, applied it to each color component, and merge the three values of the de- tected entropy for color edge detection. Machuca and Phillips (1983) treated color images as vector fields and transformed the color space from the RGB model into the YIQ model and detected edges in the hue plane. Here, additional computation time is required for color model transformation. Trahanias and Venetsanopoulos (1993) also treated color images as vector fields and a class of color edge detectors is defined using the magnitudes of linear combinations of the sorted vector samples. The operation of the essential vector order statistics on detecting local minima and maxima in the vector ordering is re- quired. They combined them (the detected local min- ima and maxima) in a suitable way in order to produce a positive response for an edge pixel.
In this study, a method for reduction of color space dimensionality and its application to color edge detection are proposed. The proposed dimen- sionality reduction method is based on the moment- preserving technique originally proposed for block truncation coding (Mitchell et al., 1978). The method has two major steps as described in the following.
First, an input image is partitioned into n X n non- overlapping square blocks. Then, a moment-preserv- ing thresholding technique (Tsai, 1985) is applied to each color plane of each square block. The color values of each color plane are thresholded to two levels and a representative value is obtained for each level. Accordingly, two representative color vectors (R 1, Gj, Bj) and (R2, G 2, B 2) are obtained for each block. The difference vector between these two color vectors ( R j - R 2, G l - G 2 , B~ - B 2) is computed to define an axis. All the data in the block are projected onto this axis. The three-spectral color image is so dimensionally reduced into a one-spectral image.
Second, for each n X n block, the mass moment-pre- serving edge detection technique (Tabatabai and Mitchell, 1984) is used to locate an edge to subpixel accuracy in a given circular window within an (n +
1) x (n + 1) block, which in turn includes an n X n block. Alternatively, most existing edge detection techniques for grey-scale images can also be applied here if preferred. Besides, the information of the two representative color vectors (R~, G~, B~) and
C.-K. Yang, W.-H. Tsai / Pattern Recognition Letters 17 (1996) 481-490 483
(R 2, G 2, B 2) and the edge location can be utilized further for color image compression.
In what follows, a brief review of moment-pre- serving thresholding (Tsai, 1985) and edge detection by mass moment preserving (Tabatabai and Mitchell, 1984) is first given in Section 2. In Section 3, the proposed approach to reduction of color space di- mensionality and its application in color edge detec- tion is described. Several experimental results to show the feasibility of the proposed method are presented in Section 4. Finally, some conclusions are made in Section 5.
2. Review of moment-preserving thresholding and subpixel edge detection
2.1. Moment-preserving thresholding
The thresholding operation is a necessary step in many image analysis applications. Surveys of litera- ture on thresholding techniques can be found in (Rosenfeld and Kak, 1982; Weszka, 1978). The mo- ment-preserving principle has been employed to per- form many image processing works (Mitchell et al., 1978; Tsai, 1985; Tabatabai and Mitchell, 1984;
Delp and Mitchell, 1991; Lyvers et al., 1989; Yang et al., 1994). A survey on moment-preserving tech- niques and applications can be found in (Chen and Tsai, 1989). Now, the moment-preserving threshold- ing algorithm developed by Tsai (1985) is reviewed below.
Given a grey-scale image f with N pixels whose grey-value at pixel (x, y) is f ( x , y), the ith grey- moment of f is
m i = ~ E E f ~ ( x , y ) , 1 i = 0 , 1 , 2 . . . (1)
x y
Let O be an operator applied to the input image f , and F be the output image. If the ith moment of F is set equal to that of f , then O is said to preserve the ith moment of the input data in the output data.
By definition, we see that if an operator preserves more moments, then more information of the input image will be retained in the output.
Suppose that it is desired to threshold f into two levels hj and h 2 with Pl and P2 being the fractions
of pixels with h I and h 2 , respectively. The algorithm by Tsai (1985) selects a threshold value in such a way that if all the pixels with below-threshold grey- value in f are replaced by grey-value h i and all the pixels with above-threshold grey-value in f are re- placed by grey-value h 2 , then the first three mo- ments of image f are preserved in the resulting bi-level image F. The concept can be expressed as follows:
Plhl + p E h 2 = m l , Plh~ + p 2 h2 = m E ,
Plh~ + P2h~ = m3, Pl +P2 = 1,
(2)
where the left term in each equality of (2) is a moment of F and the corresponding right term is a moment of f. To find the desired threshold value t, the equalities of (2) are first solved to obtained P l, P2, hi, and h 2. Then, t is chosen as the grey-value closest to the Pl-tile of the histogram of f. When t, h l, and h 2 have been computed, f can be bi-levelly thresholded into F as follows:
h I if f ( x , y) < t ,
F ( x , y ) = h2 i f f ( x , y ) > / t . (3) In more general cases, it is desired to threshold a given image f into more than two pixel classes. To threshold f into M pixel classes, the first 2 M - 1 moments are preserved to get the solution (Tsai, 1985). The threshold values can be computed deter- ministically and so the threshold selection is made automatically. This is very convenient for practical applications.
2.2. Subpixel edge detection
The subpixel edge detector proposed by Tabatabai and Mitchell (1984) is based on the mass moment- preserving principle and is reviewed here. Edge loca- tions are described by line equations. The edge operator accepts as input a set of grid squares con- sisting of 69 pixels, arranged so as to best approxi- mate the area of a unit circle (see Fig. l(a)). The edge operator generates an ideal edge element de- fined over a unit circle with two grey-values, h I and h 2. The edge element includes a border line that separates the two intensity levels as shown in Fig. l(b) and Fig. 2. Based on the moment-preserv- ing principle, the moments of output image F are
484 C.-K. Yang, W.-H. Tsai / Pattern Recognition Letters 17 (1996) 481-490
( a ) 7 9 9 8 7 9 8 6 7 7 5 4 9 8 8 7 7 7 6 4 3 9 8 8 7 7 7 6 4 3 9 8 8 7 7 7 6 4 3 8 8 7 7 9 5 4 2 3 7 8 6 8 8 5 3 2 2 7 7 6 5 4 5 4
7 6 6 3 5
Fig. 1. Tabatabai-Mitchell edge detection on a unit circle. (a) Empirically obtained edge element. (b) Ideal edge element after edge detection.
then derived in terms of the desired parameters as well as some geometric relations and other informa- tion. By enforcing the moments in F to equal the original moments in f , a set of equalities are ob- tained. Finally, the equalities are solved to get the values of the desired parameters, from which the output image can be reproduced. Note that such an output image preserves the moment information of the input image.
A more exact definition of the output disk is obtained if the ideal edge element is described by the function U ( x , y, a , p, hl, h 2) as follows:
U( x, y, a , p, h l, h 2 ) = hj if x c o s a + y s i n a ~ < p , U( x, y, a , p, hi, h 2 ) = h 2 if x c o s a + y s i n a > p .
(4)
To find the value of a , let (2, Y) be the coordinates of the center of gravity of the grey-values inside the circle. It can be found that the direction of the edge
Fig. 2. Edge line equation as a function of a and p.
1 2 3 4 5 6 7 8 9101112 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 5051 52 53 54 55 56 57
58 59 60 61 62 63 64 65 66 67 68 69
Fig. 3. Indexing associated with each square grid.
is perpendicular to the direction of the vector from the origin to (~, 7). So, angle a can be calculated as
sin a - ~ , (5)
2
cos + y2 (6)
Let the grids of the unit circle be indexed as shown in Fig. 3. Then ( ~, Y) used in Eqs. (5) and (6) above can be estimated as follows:
( ~ ) / ( 6j_~ 1 )
2 = x j l j w i l iw j , (7)
k j = t
where
lj = the intensity associated with the jth grid, wj = the weight associated with the jth grid, ( x j, yj) = the coordinates of the center of the
jth grid,
W 2 = W 4 = W22 = W40 = W68 ~--- W66 = W48
= w30 = 0.013782918,
W 3 = W31 = W39 = 14,'67 = 0.015573185, w 6 = w58 = w64 = Wl2 = 0.013068037,
and the remaining weighting coefficients are as- signed a value of "0.015719006".
Angle a thus can be computed in terms of (2, Y) above according to Eqs. (5) and (6).
To find p, h I, and h2, first define the first three sample moments of the empirically obtained data in the unit circle as
69
m i =
E wjlj,
i = 1 , 2 , 3 . ( 9 ) j = lC.-K. Yang, W.-H. Tsai / Pattern Recognition Letters 17 (1996) 481-490 485 By preserving the first three sample moments of the
input in the output of the operator, the values of P l, P2, hl, and h E can be obtained from Eqs. (2).
Given a circle of radius unity and an arbitrary angle 0 ~</3 ~< 7r/2, the area A shown in Fig. 2 is given by
A = / 3 - ½sin 2/3. (10)
If we let p = min(pl, p2 ), which denotes exactly the fraction of area A with respect to the total circle area ~r, then we get
/ 3 - ½sin 2/3 = 7rp. (11)
Eq. (11) is a transcendental equation, and one can use some numerical approximations to obtain /3.
Once /3 is obtained, then
p = c o s / 3 . ( 1 2 )
With a and p being both computed, the edge line equation is determined, which can be used to locate the edge to subpixel accuracy. The criteria for accep- tance of a pattern as an edge was based on
Ih I - h 2 l >~2o" (13)
where o-2 is the variance of the observed data. If condition (13) is satisfied, then an edge pattern is determined to be present in the circular window.
3. Proposed approach to reduction of color space dimensionality and its application in color edge detection
3.1. Proposed reduction of color space dimensional- ity
With regard to the color edge definitions which are described previously, in many image blocks the
Jl
CountR ~ P A
Fig. 4. Data distribution of a two-spectral image;
the shaded regions are the original data, and PA represents an axis on whichan edge can be found more easily.
three color components are weakly correlated and the edges in different color bands may be different from one another. Taking the two-spectral image shown in Fig. 4 (with R and G planes only) as an illustrative example, we can see that the edge is not obvious on either of the two color planes but can be found more easily on a combination of them. There exist some ways to find out the color edge in such practical cases. The idea is to find first an axis onto which all color component values are projected to produce a new image with a single spectral band, and then find the edge on the new image. By this way of reducing the 3-dimensional color space to one dimension, edge detection methods for grey-scale images become applicable to color images. Three of the color space dimensionality reduction approaches investigated in this study are the Karhunen-Lo~ve (KL) expansion (Fukunaga, 1990), the vector median (Astola et al., 1990), and the proposed method.
From the theory of feature extraction, the KL expansion can be employed to perform feature space dimensionality reduction (Bigun, 1993), and the axis found is called the principal axis on which the features have the largest variance values. For each image block of the RGB color space, there exists a covariance matrix ~, defined as follows:
I E(R2) - E2(R) E ( R G ) - E(R)E(G)
=/E(RG) - e(R)e(O) E(G2) - E2(G) LE(RB) - E(R)E(B) E(GB)- E(G)E(B)
E(RB)- E(R)E(B) ]
E(GB)- E(G)E(B) ]. (14)
E( B2) - E2( B)
has three non-negative eigenvalues because it is a covariance matrix (Noble and Daniel, 1988). The greater the eigenvalue is, the more information is contained in the corresponding eigenvector. The eigenvector associated with the largest eigenvalue is the principal axis.
Projection on a vector median is another approach to reducing the color space dimensionality. The me- dian operation in signal processing is a nonlinear filtering operation where a window moves over a signal, and at each point the median value of the data within the window is taken as the output. The vector median has properties similar to those of the stan- dard median in the scalar case. According to Astola
486 C.-K. Yang, W.-H. Tsai / Pattern Recognition Letters 17 (1996) 481-490
et al. (1990), a vector median can be found as follows.
(1) For each vector x i, compute the distance to all the other vectors using either the L~ (absolute dis- tance) or L 2 (Euclidean distance) norm and add them together, resulting in
S i = ~
II
x i - x jI],
i = 1 . . . n, (15) j = lwhere n is the number of pixels in each block.
(2) Find the value Xmi n among all x i such that Si is the minimum of all S~.
Once the vector median of a block is determined, all the data of a block are projected on the vector median of the block.
In the above two approaches, a lot of computation time is required when the eigenvalues and eigenvec- tors are computed or a vector median is selected.
Due to this reason, another way based on the mo- ment-preserving principle to find the desired axis is proposed in this study.
First, the bi-level thresholding method proposed by Tsai (1985), reviewed previously, is applied indi- vidually to each color plane of each color image block. T w o sets of representative color vectors (R 1, G~, B I) and (R 2, G 2, B 2) are obtained. More specifically, R~ and R 2, for example, correspond to h~ and h 2 in (3), respectively. Because the two color vectors (R l, G I, B 1) and (R 2, G 2, B 2) can be re- garded as the centroids of the two clusters in the RGB feature space on the two sides of the color edge, the difference vector ( R 1 - R 2 , G ~ - G 2, B ~ - 9 2) is meaningful. This direction is approxi- mately perpendicular to the color edge, and so can be used as the desired axis for data projection men- tioned previously. Since the computation involved in finding (R E , G 1 , B I) and ( R 2, G 2 , B 2) uses analytic formulas, the computation time is reduced consider- ably, compared with that for computing the principal axis using the K L expansion or selecting a vector median. All the three methods are implemented and their results are compared with one another in Sec- tion 4.
3.2. Application in color edge detection
After finding the axis specified by the difference v e c t o r ( R 1 - R 2, G~ - G 2, B t - B2), we can project
all the data in the image block onto the direction of this axis. The projected data behave just like a grey-scale image. Then, the Tabatabai-Mitchell (1984) edge detector is applied to the projected data to perform edge detection.
The criterion for acceptance of a pattern as a color edge is described as follows:
[R 1 - R 2 1 + I G I - G2 I + I B I - B 2 1
>~ T( ~ + ~ + O-b), (16)
where ~- is a preset threshold value and ~ , O-g, and tr b are the standard deviations of the three color planes of all image blocks, respectively. According to (Tabatabai and Mitchell, 1984), the lower bound value for ~- is 2 and in this study 3.5 is selected experimentally.
3.3. Algorithm
An algorithm is given below to summarize the proposed method.
A l g o r i t h m (Reduction of color space dimensionality and its application in color edge detection)
Step 1. Read in a color image f.
Step 2. Partition f into n × n non-overlapping square blocks.
Step 3. For each block fj, perform the following stages.
Stage 3.1. Dimensionality reduction
(a) Apply the bi-level thresholding operation of Tsai (1985) to each color plane o f fj individually to compute two representative color vectors (R~, G~, B I) and (R2, G2, B2).
(b) Compute the absolute difference vector of the two color vectors d = I x~ - x 2 I, where x t and x 2 are two representative color vectors ( R 1, G~, B 1) and ( R 2, G 2, B2), respectively.
(c) Normalize the difference vector, resulting in d = d/l[ d II where IId [I represents a ~/r 2 + g2 + b 2 norm, making the found color coordinates consistent from block to block.
(d) Project all the pixels of fj onto d by comput- ing (ri, gi, bi) X a~as the result.
(e) If I R I - R 2 [ + I G I - G 2 1 + I B I - B 2 1 >1 z ( ~ + cr + O'b), then continue to perform edge de- tection; otherwise skip the next stage and process the next block.
C.-K. Yang, W.-H. Tsai / Pattern Recognition Letters 1 7 (1996) 481-490 487
- - / ~ -"-- ~. -.00000 -.01497 -.01902 .00000 .01902 .01497 .00000 ) , _ ~ , ) . . . ~ , ~ ) , . _ , , -.02235 -.04664 -.02332 .00000 .02332 .04664 .02235 //~!'i ! ~ ~ ~7:~ \ -.05734 -.04664 -,02332 .00000 .02332 .04664 .05734
)-()--()---
/ ~ ~ .0,9oo . 0 ~ .02332 00000 02332 04,,4 0 , ~ 0~ ii~-' ~ ~,:~-'~ -.05734 -.04664 -.02332 .00000 .02332 .04664 .05734 ," -.02335 -.04664 -.02332 .00000 .02332 .04664 .05734
\ !~'~i ~ i "Z: ~ '.~ ] -.O0000 -.01497 -.01902 .OO000 .01902 .0,497 .OO000
\ ... i~',: .-:-: ~
