Hierarchical image representation by mathematical morphology subband decomposition

(1)

~_N~_~ February 1995

Pattern Recognition

Letters

El .SEV II-~R Pattern Recognition Letters 16 ( 1995 ) 183-192

Hierarchical image representation by mathematical morphology

subband decomposition

Soo-Chang Pei a,., Fei-Chin Chen b

a Department of Electrical Engineering, National Taiwan University, Taipei, Taiwan, ROC b Department of Electrical Engineering, China Junior College of Technology, Taipei, Taiwan, ROC

Received 6 January 1994; revised 30 August 1994

AbsUa~

An efficient subband image decomposition method by mathematical morphology has been proposed recently. This method decomposes the input signal spectrum into 4 subbands by using two separable structuring elements. Based on this method, a hierarchical image representation referred to as subband pyramid is described. This subband pyramid scheme preserves the number of pixels as in the original image, also the data structure itself is very compact. More importantly, morphological filtering manipulates the specific shape in the image, it only alters certain geometric details without affecting the remaining image structure. When the resolution is reduced, no overall blurring is introduced in the image as the conventional linear filtering method does. This natural decomposition of multiple scale shapes might be a useful hierarchical representation for the images.

1. Introduction

A well known multiresolution image representation is the pyramid. Burt and Adelson used them for image coding and hierarchical representation (Burt, 1981; Butt and Adelson, 1983) which allows analyz- ing the image at a certain resolution level, thus avoid- ing unnecessary fine details. The most pronounced advantage of such a code is its reversibility: the original image can be perfectly reconstructed from its La- placian pyramid representation. Also, some research works have been recently reported on this pyramid image decomposition using mathematical morphology (Toet, 1989). The problem with this Laplacian pyramid approach is that too many image samples are used to describe the original image. The total number o f pixels in the Laplacian pyramid is greater

* Corresponding author.

than that in the original image by a factor of about 4/3 (Kronander, 1987).

However, the subband pyramid (Kronander, 1987; Woods and O'Neil, 1986; Gharavi and Tabatabi, 1988) can overcome the above problem. The total number of pixels in the subband pyramid is equal to that in the original image, and the data structure itself is very compact. An efficient subband image decomposition using mathematical morphology has been proposed recently by the authors (Pei and Chen, 1991 ). Four-subband decomposition has been widely used for image compression by means of the charac- teristic of each band. However, the implementation efficiency is decreased due to the multiplications of the linear quadrature mirror filtering. Thus, the morphological low-pass filters are used here instead of the linear quadrature mirror filters. The major advan- tages of the morphological approach over the conventional linear filtering approach are its direct geometric interpretations, simplicity and efficiency in the 0167-8655/95/$09.50 © 1995 Elsevier Science B.V. All riots reserved

(2)

184 S.-C. Pei, F.-C. Chen /Pattern Recognition Letters 16 (1995) 183-192

hardware implementation. However, no analysis and theory have been done so far to understand this subband morphological decomposition for image representation. This paper will describe and explain this hierarchical representation in details from a progressive morphological decomposition of simple shapes at each scales. The representation may consist of an ordered sequence of multiresolution images with progressive structure contents. This can provide a better basis and understanding for constructing morphological multiresolution representation for image analysis and computer vision. Moreover, global in-

formation can be used to impose constraints on local operations. Therefore, a hierarchical subband decomposition algorithm can be more efficient than operations performed on a single level. It will be very useful for signal processing and data compression.

The outline of this paper is organized as follows: Section 2 reviews mathematical morphology operations on binary images and gray-level images briefly and shows the method of the subband decomposition for images using morphological filters. Section 3 dis- cusses the hierarchical representation of images by subband decomposition using mathematical mor-

O 0

0 0 0 0 0

0 0 0 0

0 ~ 0

O 0

0 0 0 0 0 0

O 0

0 0 0 0 0

0 0 0 0

0

X B X o B X o B d e n o t e s t h e o r i g i n o f t h e s t r u c t u r i n g e l e m e n t B. (a)

O

X B X o B X o B (b)

Fig. 1. Example of binary image morphology. (a) Erosion and dilation o f X b y B. (b) Opening and closing o f X b y B. (The dark solid curves correspond to the boundaries of the transformed sets. )

(3)

S.-C PeL F.-C Chen / Pattern Recognition Letters 16 (1995) 183-192 185

phology. Section 4 gives some image examples to show the effectiveness of this approach. The conclusions are made in Section 5 finally.

2. Subband decomposition using morphological filters The theory of mathematical morphology is based on set algebra (Serra, 1982 ). The basic operations on binary images can be extended to gray-level images (Serra, 1982; Haralick et al., 1987). Mathematical morphology provides an effective approach to ma- nipulate the shapes in the image. There are four basic operations: erosion, dilation, opening and closing.

2. I. Binary image morphology

Assume X (an original image) and B (a symmet- rical structuring element) are subsets of a two-dimensional Euclidean space, let (X)b denote the translation of X by the vecto r b,

(X)b={y l y=x+b, xeX} . (1) Then the two most fundamental morphological operations, dilation and erosion, can be defined respectively, as follows:

X ~ B = U (X)b

beB

={y l y = x +b, xeX, beB} , (2) X G B = n (X)_ b

b~B

={Yl beBimplies ( y + b ) e X } . (3) Fig. 1 (a) shows the example of erosion and dilation of X by B. Dilation and erosion cause the expanding

or shrinking of areas when the structuring element has a disklike shape.

An opening is defined as an erosion followed by a dilation by the same structuring element and is shown as

XoB= ( X O B ) ~ B . (4)

On the other hand, we define a closing as a dilation followed by an erosion as

X e B = ( X~B)E3B. (5)

Opening on an image with a structuring element B can be pictured by moving B inside all the shapes in an image and marking only those places where B fits. Similarly, closings on an image with a structuring element B can be pictured by moving B around the out- side of an image with the result that the concave corners are rounded and the convex corners remain square. An example of opening and closing of X by B is shown in Fig. 1 (b).

2.2. Gray-scale morphology

Morphological concepts can be extended to gray- scale images (Serra, 1982; Haralick et al., 1987).

Let the image X(x) be represented as a function of coordinates x. The analytical definitions of the gray- level morphology operations are as follows:

X ~ B = m a x [ X ( x - b ) +B(b) ] , (6) beB'

X @ B = m i n

[X(x+b)-B(b)

].

(7)

beB'

where the B ( b ) ' s are weights that are a function of b, and B' is the region of support of the structuring element B.

The opening and closing operations for gray-level

(a) (b) (c)

Fig. 2. Example of gray-level image morphology. (a) Disk shaped structuring element. (b) Gray-level image landscape. (c) Opening by a disk shaped structuring element.

(4)

186 S.-C. Pei, F.-C. Chen / Pattern Recognition Letters 16 (1995) 183-192 x ( m , n ) S t ' r u c t u r i n g e l e m e n t o & • o : o p e n i n g • : c l o s i n g

+<

_>

"222~(ix5) S t r u c t u r i n g e l e m e n t o & •

)

YOo(m,n) Y l o ( m , n ) .... g ( ix5 ) S t r u c t u r i n g e l e m e n t 0 & • Y o l ( m , n ) + Y l l ( m , n ) Fig. 3. System diagram of 4-band splitting using mathematical morphological analysis filter banks.

images are defined similarly as in binary case, i.e.

X o B = ( X e B ) ~ B , (8)

X e B = ( X ~ B ) e B , (9)

In gray-level morphology the disk shaped structuring element shown in Fig. 2 (a) is three-dimensional. The result of a three-dimensional opening is to move the top areas under the top surface of the landscape in Fig. 2(b) defined by the image, and keep those areas where the disk shaped structuring element fits. Strongly peaked areas with widths less than the di- ameter of the disk shaped structuring element will flatten out as illustrated in Fig. 2 (c).

2.3. Four-subband decomposition

The input image is split into four rectangular bands as in Fig. 4. The overall analysis/synthesis system in four-subband coder is shown in Fig. 5. Now the morphological low-pass filters are used here instead of the linear quadrature mirror filters (Pei and Jaw, 1987).

t l Io 11 01 O0 01 t0 11

I

-'R.

F i g . 4. Ideal 4-band splitting in 2-D subband coder.

Sequential alternating application of the morphological operations of opening and closing removes details of the image that are small relative to the struc-

(5)

S.-C. Pei, F.-C. Chen / Pattern Recognition Letters 16 (1995) 183-192 . . . . I I x ( m , n ) - C ' l I I I I

i

l

I

[

)~o(m,n)l

- c a u

I I

I t _ _

I

~

- ~ ( m , n )

Fig. 5. System diagram of 2-D subband coder with 4 channel analysis/synthesis fdter banks.

187

lY°l¢mTn'l lyl ( 'nq

Fig. 6. Hierarchical representation of an image by four-subband decomposition. turing element. We call these alternating sequential

filters as morphological low-pass filters. The high-pass filter can be constructed with the original image X and the complement of the low-pass filter, - H o (X), as follows:

Ho(X)

= closing[opening(X) ] ,

n~ (X) =X-Ho(X)

(10)

(11)

where

Ho(X)

and H~ (X) denote low-pass and high-

pass filters, respectively. In analogy to the linear filters one can imagine that the length and direction of the structuring elements will determine the passband bandwidth and spectrum orientations of the filters. The larger the structuring elements is, the narrower the filter passband becomes. Then 2-D filter banks in a four-subband coder are designed by a cascade op-

eration of the above I-D horizontal/vertical mot- phological filters, i.e.:

Hoo( X) = H[[ Hh( X) ] ,

Hol (X)

=H'{[Hh(X) ],

Hio(X)

=H'~[Hh(X) ],

H~ (X) =H'~[Hh(X) ],

(12)

(13)

(14)

(15)

where H/h and H7 for i= 0, 1 are the horizontal and vertical low-high-pass filters, and Hoo(X), Ho~ (X),

Hm(X), HI~ (X)

denote LL, LH, HL and HH filters of X, respectively. These 2D filters have the same al- gebraic properties of idempotence, increasing and duality as openings and closings.

From Eq. (10), the 1-D morphological low-pass filter is

(6)

188

Fig. 7. Hierarchical representation of "Picture A'" by four-subband decomposition using mathematical morphology. (a) "Pic- ture A", original image x( m, n ). (b) yoo( m, n ), ylo( m, n ), Yoi ( m, n), yll (m, n). (c) y~o(m, n),ylo(m, n), y~l(rn, n),ylt(m, n).

S.oC. Pei, F.-C. Chen / Pattern Recognition Letters 16 (1995) 183-192 Ho (X) = closing [ opening (X) ]

= ( X o B ) o B (16)

where X is the original image and B is the structuring element. To prove the idempotence of H o ( X ) , we need to use the equation

X ~ 2 B = ( X @ 2 B ) o 2 B

= ( ( X ~ 2 B ) ~ 2 B ) ~ 2 B (17) which is Proposition 37 in (Haralick et al., 1987), and assume 2B to be represented as B O B . The Prop- osition 37 is proven on the binary morphology in (Haralick et al., 1987 ), but it is also true on the gray- scale morphology. Proof. H o ( H o ( X ) ) = ( ( ( X o B ) ,,B) o B ) ,,B = (( ( (( ( ( X e B ) ~B) e B ) e B ) G B ) ~ B ) ~ B ) O B = ( ( ( ( X G B ) @2B) G2B) ~ 2 B ) G B = ( ( X O B ) ~ 2 B ) e B = ( ( ( X e B ) ~ B ) e B ) e B = ( X o B ) . B = H o ( X ) .

The 1-D morphological high-pass filter, H ~ ( X ) = X - H o ( X ) , constructed with the original image X and the complement of the low-pass filter, is also idempotent. Proof. H, ( H , ( X ) ) = ( X - H o ( X ) ) - ( H o ( X - H o ( X ) ) ) = ( X - H o ( X ) ) - ( H o ( X ) - H o ( H o ( X ) )) = ( X - H o ( X ) ) - ( H o ( X ) - H o ( X ) ) = X - H o ( X ) = H , ( X ) .

Therefore, the 2-D fdters, Hoo( X ) , Ho~ ( X ) , Hlo( X ) and H ~ I ( X ) designed by a cascade operation of the

1-D low/high-pass filters are also idempotent. With mathematical morphology filtering, the complexity of filters design and implementation is greatly reduced. Also it can provide a unique geometric representa-

(7)

S.-C. Pei, F.-C. Chen /Pattern Recognition Letters 16 (I 995) 183-192 189 5

]'

4 (a D e t a i l e d s i z e of P i c t u r e A. 5 (b) E n l a r g e d s c a l e of the o b l i q u e b a r in P i c t u r e A. 0 0 19 0 0 0 0 ~ 0 0 (c) 5xl v e r t i c a l ) & ix5 ( h o r i z o n t a l ) s t r u c t u r i n g e l e m e n t s .

Fig. 8 Detailed size of"Picture A" and structuring elements.

tion of an image by summarizing its shape and con- veying its size, orientation and connectivity. The system diagrams of four-subband splitting using morphological filters are shown in Fig. 3.

3. Hierarchical subband pyramid

The hierarchical representation of images by subband decomposition is referred to as a subband pyramid. Image analysis frequently requires a hierarchical representation that consists of multiresolution images with progressive structure contents. Compar-

(8)

190 S.-C Pei, F.-C Chen / Pattern Recognition Letters 16 (1995) 183-192

ing the morphological filtering with the linear filtering approach, the conventional low-pass filter will re- move spatial frequencies which are higher than the Nyquist frequency of the sample spacing. This results in an overall blurring of the reduced image. How- ever, mathematical morphology directly operates on certain shapes without affecting the remaining image structure. Therefore, morphological filters are a better choice for constructing pyramids than conventa- tional low-pass filters. Based on the above four-subband image decomposition by using mathematical morphology, a four-subband pyramid is obtained by the following steps.

First, the original image

X(m, n)

is decomposed into four subbands, the lowest band

Yoo(m, n)

= Hoo(X), the vertical band Yl0 (m, n) = HIO(X), the horizontal band Yot (m, n) = Hot (X), and the diagonal band y~ ~ ( m, n ) = HI 1 (X), by the 2-D filter banks described in Eqs. ( 12 )- ( 15 ) and the procedures described in Fig. 3. The three high band images do not contain much details; further decomposition is no more necessary except for the lowest band. Then, the lowest band Yoo (m, n ) is decomposed into four subbands,

y~o(m, n), y~o(m, n), yll (m, n)

and Y lI (m, n), by the same procedures again. The hierarchical 4-subband decomposition is shown in Fig. 6.

Fig. 9. Hierarchical representation of "Lena" image by four-subband decomposition using mathematical morphology. (a) "Lena", original image x(m, n). (b) yoo(m, n), Ylo(m, n), Yol (m, n ) , y l l ( m , n ) . (c)y~o(m,n),y~o(m, n), yol (m, i n ) , y H ( m , n). l

4. Experimental results and discussions

The "Picture A" in Fig. 7(a) is used in our experiments. The detailed size of Picture A is shown in Fig. 8. It consists of a diagonal and horizontal lines and several rectangles with different thickness. These rectangles are specially designed with some specific widths, and will be suitable for different level decom- positions. In the computer simulations, we have used a ( 1 × 5) horizontal structuring element and a (5 × 1 ) vertical structuring element at each level in the four- subband image decomposition. The experimental results for the first level decomposition are shown in Fig. 7(b) as the horizontal/vertical resolution has been reduced by 2 by down-sampling operations. The central lowest band analysis image

Yoo(m, n)

con- tains the most information with some details being removed by morphological filtering. The vertical band and horizontal band analysis images

Yto(m, n)

and Yol

(/'n, n)

have extracted the thin or narrow vertical

(9)

S.-C. Pei, F.-C. Chen / Pattern Recognition Letters 16 (1995) 183-192

191 I

Input

Textures[

Images

1

Thresholding I

1

Morphological

I

Subband

Decomposition

I

Morphological

Subband

Decomposition

Feature

I

Extraction

I

Classification

I

Fig. 10. System diagram of texture classification based on hierarchical image representation by morphological subband decomposition.

and horizontal sharp edges of the bottom-right rectangle very successfully. Also, the diagonal band analysis image Yl i (m, n) shows the central diagonal edge and some sharp comers of the PiCture A. We notice that there is some residual structure of the central diagonal edge existing in the vertical band image Y~o (m, n). This is due to the zig-zag nonperfectness inherent in digital straight lines. The central lowest band

Yoo (m, n) is further decomposed into four subbands

y~o(rn, n),ylo(m,n),y~l(m,n)

and

yI,(m,n)

as shown in Fig. 7(c). The bottom-left rectangle with thicker or wider width has been decomposed again into two horizontal, vertical edges and sharp comers in each high band image. Now only the three black blocks still remain in the lowest band image. Notice that no overall blurring is introduced at each stage in

(10)

192 S.-C Pei, F.-C Chen / Pattern Recognition Letters 16 (1995) 183-192 morphological filtering; the subband images remain

sharp as the original ones at each level.

The " L e n a " image in Fig. 9 (a) is decomposed by the same procedures as processed on "Picture A". The results are shown in Figs. 9 ( b ) and 9(c). The fine smallest details are restricted to the top level and p r o - gressively coarser details are restricted down to the bottom levels. From these results, it is evident that the hierarchical image representation (the subband pyramid) is very useful for signal analysis.

An application example of the hierarchical image representation is shown in Fig. 10, a system diagram o f texture classification based on hierarchical image representation by using morphological subband decomposition is introduced. Recently, a texture classification based on a one-level morphological subband decomposition scheme, which was firstly proposed by the authors (Pei and Chen, 1991 ), is presented by K i m et al. (1993). The classification accuracy rate for 10 natural textures selected from Brodatz's (1966) texture album is very high in their experiments. We modify their approach by the hierarchical image representation scheme as shown in Fig. 10. Because of the hierarchical representation of texture images, miltiresolution features of texture images will be extracted, and the classification accuracy rate will be improved for future research works and experiments.

Moreover, the above experiments and analysis are very helpful for us to understand the four-subband decomposition procedures by mathematical morphology. This can provide a better basis and understanding for constructing a morphological multiresolution representation for image analysis and computer vision.

5. Conclusions

In this paper, we have presented a method of the efficient four-subband image decomposition using

mathematical morphology. Based on this method a hierarchical representation of images is described. These efficient hierarchical representation schemes preserve the number ofpixels and remain sharp as in the original image, also the morphological operations involved are very simple, fast and well suitable for VLSI implementation. Some image examples are illustrated to show the effectiveness of this approach.

References

Brodatz, P. (1966). Textures: A Photographic Album for Artists and Designers. Dover, New York.

Butt, P.J. (1981). Fast filter transforms for image processing. Computer Graphics and Image Processing 16, 20-51. Butt, P.J. and E. Adelson (1983). The Laplacian pyramid as a

compact image code. IEEE Trans. Commun. 31,532-540. Gharavi, H. and A. Tabatabi (1988). Subband coding of

monochrome and color images. IEEE Trans. Circuits Syst. 35, 207-214.

Haralick, R.M, S.R. Sternberg and X. Zhuang (1987). Image analysis using mathematical morphology. IEEE Trans. Pattern Anal. MachineIntell. 9 (4), 532-550.

Kim, G.S., K.H. Do, ICH. Kwon, J.C. Shim and Y.H. Ha (1993). Texture classification using morphological pattern spectrum

based on morphological subband decomposition. 1993 Joint Technical Conf. on Circuits~Systems, Computers, and Communications, Nara, Japan, Aug. 1993, 212-216. Kronander, T. (1987). Sampling of bandpass pyramids. IEEE

Trans. Commun. 35, 125-127.

Pei, S.C. and F.C. Chen (1991). Subband decomposition of monochrome and color images by mathematical morphology. Opt. Engineering, July, 921-933.

Pei, S.C. and S.B. Jaw (1987). Design of 2D quadrature mirror FIR filters for image subband coding. IEEE Trans. Circuits Syst. 34, 438-441.

Serra, J. ( 1982 ). Image Analysis and Mathematical Morphology. Academic Press, London.

Toet, A. (1989). A morphological pyramid image decomposition. Pattern Recognition Lett. 9, 255-261.

Woods, J.W. and S.D. O'Neil (1986). Subband coding of images. IEEE Trans. Acoust. Speech Signal Process. 34, 1278-1288.