Content-based color trademark retrieval system using hit statistic

(1)

c

World Scientific Publishing Company

CONTENT-BASED COLOR TRADEMARK RETRIEVAL SYSTEM USING HIT STATISTIC

CHUAN-CHENG WANG∗and LING-HWEI CHEN†

Department of Computer and Information Science, National Chiao Tung University, 1001 Ta Hsueh Rd., Hsinchu, Taiwan 30050, R.O.C.

∗_{[email protected]} †_{[email protected]}

This paper proposes a content-based color trademark retrieval system. First, the edges of each color trademark are detected. Then, the smallest circle that covers the trademark is derived. Based on the smallest circle and edges, the feature called hit statistic on a dartboard of the color trademark is extracted. Using this feature, this paper constructs an efficient and simple color trademark retrieval system, which is robust to rotation, translation, scaling and some geometric distortions. Some experiments are conducted to compare the proposed system with the existing one using Zernike moment, and the results show that the proposed system is superior to that using Zernike moment. Keywords: Content-based color trademark retrieval; edge; smallest circle; hit statistic on a dartboard.

1. Introduction

In order to design a unique, unambiguous and representative color trademark for an enterprise, the designer should create design a color trademark that is not similar to others. To treat this problem, many systems of content-based color trademark re-trieval have been proposed. QBIC1,8_{proposed by IBM is an image retrieval system.} In QBIC, users can choose any one of the features: color histogram, color layout, texture or shape to do image retrieval. The color histogram is invariant to rotation, translation and scaling. However, if two images have similar color histograms, they may look very different. This is because the locations of color regions in an image are ignored. This problem can be solved by color layout, which concerns more on the rough color positions, and the detailed context of an image is not considered. Texture features derived from the second-order statistics are suitable only for im-ages full of textures, such as cloud, sand, grass and cloth, etc. However, in the real world, only a few images have texture spreading over the entire image. Thus for trademarks, if users only take the texture features to do retrieval, a good result cannot be expected. Shape features consist of shape area, circularity, eccentricity, major axis orientation and invariant moments. For some special type of images such as trademarks and Chinese antiques, the shape features are very important.

603

Int. J. Patt. Recogn. Artif. Intell. 2002.16:603-619. Downloaded from www.worldscientific.com

(2)

Kim5,6_{used the Zernike moment magnitudes (ZMMs), which are rotation and} scale invariant and robust to noise and slight shape deformation, to do retrieval. But for some geometric deformation, such as the sphere transformation, the retrieval result is poor. Some methods4,9 _{used the histogram of the edge directions of the} shape boundary in a trademark as the feature. However, the histogram does not contain the location information.

There are two types of shape descriptors: contour-based shape descriptors and region-based shape descriptors. Contour-based shape descriptors, such as Fourier descriptors,10 _{are not appropriate for describing shapes consisting of several} dis-joint regions. Region-based shape descriptors, such as moments, are more reliable for shapes that have complex boundaries. The drawback of regular moments7 _is that there is redundant information in the moments since the bases are not orthog-onal and high-order moments are sensitive to noise. To treat the above-mentioned drawbacks, a new content-based trademark retrieval system is proposed here. The proposed system is insensitive to translation, rotation, scaling and geometric distortion.

There are a lot of color trademarks to be designed as representative symbols of enterprises in the world. As shown in Fig. 1, the color trademarks can be clas-sified into three classes, color trademarks with word only, color trademarks with totem only and color trademarks with word and totem. The paper focuses on color trademarks with totems only.

The RGB color model3_{that separates the brightness from the chromaticity} com-ponent is taken, based on this model, the proposed system uses the inner product of the RGB vectors of two neighboring pixels to detect edge pixels of a trademark. After locating edge pixels, the smallest circle that covers the trademark is derived. Based on the smallest circle and edges located, the feature called hit statistic on a dartboard is extracted. The feature is robust to noisy and distorted patterns. With a proper normalization method, scale invariance is also achieved. Rotation problem is solved in the retrieval step. All queried and retrieved trademarks can contain complex shapes with several disjoint regions. In order to show the effectiveness of the proposed system, the ZMMs based method is also implemented to compare with the proposed method.

(a) (b) (c)

Fig. 1. Three classes of color trademarks. (a) Color trademark with word only. (b) Color trade-mark with totem only. (c) Color tradetrade-mark with word and totem.

(3)

In the remainder of this paper, we introduce the feature used for retrieval in Sec. 2. The retrieval method will be proposed in Sec. 3. Section 4 will present the experimental results. Finally, conclusions will be given in Sec. 5.

2. Feature Extraction

In order to detect the edge pixels, an algorithm of edge detection is developed. Before describing the edge detector, we will first introduce the color model3 _used in the paper.

2.1. Color model

In essence, a color model is a specification of a 3-D coordinate system and a subspace within that system where each color is represented by a single point. In this paper, we take the RGB color model, which is based on the Euclidean vector space as shown in Fig. 2.

In the model, each color (see Fig. 3) appears in its primary spectral components of red, green, blue3 _{and is represented by (r, g, b). And each color (r, g, b) is} considered as a vector formed by connecting (0, 0, 0) to (r, g, b). The magnitude of a RGB vector decides the bright level of a color. Except the magnitude of a RGB vector, there is an included angle between two different color vectors, it decides how different these two colors are.

Considering several RGB vectors with the same direction but the magnitudes, as shown in Fig. 4, these vectors will have the same chromaticity.3 _{Thus we} de-fine a color tone to be a set of color vectors with the same direction but with different magnitudes. Each color tone can be represented by an unit vector, and the included angle between two different color tones can be used to measure the difference between two colors.

Fig. 2. The RGB color model.

(4)

Fig. 3. Each color appears in its primary spectral components of red, green and blue.

Fig. 4. All colors with the same color tone will be a set of RGB vectors with the same direction but different magnitudes.

Fig. 5. The included angle between two unit vectors of different color tones.

Figure 5 illustrates the included angle between two different color tone vectors.

2.2. Edges of a color trademark

In a color trademark, there should be one or more symbols separated from the background. Since the color tones of adjacent symbols are different from the color tone of the background and different amongst themselves, this difference can be

(5)

used to detect symbol edges. One example shown in Fig. 6 is given to illustrate this idea. Figure 6(a) shows a color apple trademark, by comparing the color tone of each pixel with its neighbors, the boundary of the apple trademark shown in Fig. 6(b) can be obtained.

Based on the above illustration, the edge detector is now described. On the scan sequence from left to right, top to bottom, consider the four pixels shown in Fig. 7. Each pixel compares with the other three pixels. If the color difference is over a threshold, then the darker pixel (the color vector with smaller magnitude) is considered as an edge pixel. Moreover, since the included angle between one zero vector and one nonzero vector cannot be derived, the zero vector (the black pixel)

(a) (b)

Fig. 6. The edges of the color apple trademark obtained by considering the color tone difference. (a) The apple color trademark. (b) The edges of (a).

Pixel A Vector A Pixel B Vector B Pixel D Vector D Pixel C Vector C

Fig. 7. Four pixels used in the edge detector.

(a) (b)

Fig. 8. The result of edge detection. (a) A color trademark. (b) Edges detected.

(6)

is considered as an edge pixel. Note that in the paper, a black pixel is defined as a pixel with magnitude under a threshold. Figure 8 shows the result of edge detection for a trademark. The details of the edge detection algorithm is described as follows:

1. get| ~A|, | ~B|, | ~C|, | ~D| (See Fig. 7)

2. if ((| ~A| or | ~B| or | ~C| or | ~D|) < black threshold) if ((| ~A| or | ~B| or | ~C| or | ~D| > black threshold) {

if (| ~A < black threshold ) pixel A is an edge; if (| ~B < black threshold ) pixel B is an edge; if (| ~C < black threshold) pixel C is an edge; if (| ~D < black threshold ) pixel D is an edge; }

3. if ((| ~X| and |~Y |) > black threshold) if

cos−1 ~_{| ~}X_X·~_||~Y_Y > angle threshold

if (| ~X < ~Y|)X is an edge pixel; else Y is an edge pixel ;

where (X, Y ) = (A, B), (A, C), (A, D), (B, C), (B, D), (C, D)

2.3. The smallest circle finding

To obtain the smallest circle covering a color trademark and its center (see Fig. 9), first, we will get the four boundaries of the color trademark: the left vertical bound-ary, the right vertical boundbound-ary, the top horizontal boundary and the bottom hori-zontal boundary. Second, the middle point of these boundaries is considered as the initial rough center and the longest distance of the middle point to the four bound-aries is considered as the initial rough radius. Third, we will adjust the center of the current circle and its radius to get the desired circle and center. The adjusting direction for the center depends on the pixels’ positions outside the current circle. Counting the pixels outside the current circle in four directions: up, down, left, right respectively, then the circle center is moved to the direction in which most of the pixels outside the current circle appear. If all the edge pixels are inside the

Fig. 9. The smallest circle covering Fig. 8(a) presented by a red circle and its center marked by “+”.

(7)

circle, then the radius of the circle will be reduced. If changing the center ten times continuously cannot reduce the radius, the center and the radius of the smallest circle that can cover all the edge pixels are derived. By using this algorithm, the smallest circle covering the color trademark can be derived. The detailed algorithm is described as follows:

1. get the left(L), right(R), top(T ) and bottom(B) boundary of a color trademark ; 2. set initial center (Cx, Cy) to be L+R₂ ,T +B₂

; set initial radius (r) to be

√

(L_−R)2_+(T_−B)2

2 ;

set a over circle time counter (OCT ) to be zero; 3. while (1)

{set three flag, overcircle (OC), Horizontal (H), Vertical (V ) to be zero; for all edge pixels (X, Y )

if ((X− Cx)2+ (Y − Cy)2≥ r) {if ((X − Cx) > 0)H+ = 1; if ((X− Cx) < 0)H− = 1; if ((Y − Cy) > 0)V + = 1; if ((Y − Cy) < 0)V− = 1; OC = 1;}

if (OCT > 10) get the center and radius then break the while loop; if (OC = 1) {OCT + +; if (H > 0)Cx+ +; if (H < 0)Cx− −; if (V > 0)Cy+ +; if (V < 0)Cy− −;} else {OCT = 0; r− −;}}

2.4. Hit statistic on a dartboard

Based on the edge pixels and the smallest circle that covers the trademark, the feature can be derived and described as follows. Figure 10 shows a dartboard, which consists of four concentric circles cut by 32 straight lines. The outmost circle of the dartboard is the smallest circle covering the trademark and the center of the dartboard is the center of the smallest circle. The dartboard is partitioned into 256 regions. If at least one edge pixel appears in a specified region, then the specified region is marked as hit and given value one. Otherwise marked as missed and given value zero. All values of these regions are grouped into a feature vector, which

(8)

! " # $ % & ' ( ) * Fig. 10. A dartboard.

consists of 256 values and called hit statistic on a dartboard. Its sequence is from inside to outside, 0◦to 360◦. This feature is scale invariant and robust to noise and slight shape deformation, it also contains the location information. The rotation problem is solved in retrieval phase.

3. The Retrieval Methods

The proposed retrieval method and the Zernike-moments based method used for comparison are described in the following subsections.

3.1. The proposed retrieval method based on hit statistic on dartboard

The proposed method evaluates the difference between two hit statistics of the query and a queried image. To solve the rotation problem, we can rotate the dartboard of the query image and calculate the difference for each rotation. All differences are checked and the smallest one is taken. The smaller difference means that two images are more similar. The mirror problem is also solved by creating a mirror hit statistic to do retrieval. The detailed algorithm is described as follows:

create a mirror feature HSMS, Rquery = HS query 63_{−S, R} {difference =P3 R=0 P63 S=0HS query (S+I)%64, R∧ HS queried S, R ; difference M =P3_R=0P63_S=0HSM_{(S+I)%64, R}query ∧ HSqueriedS, R ; differenceI = min (difference, difference M);

}, I = 0 . . . 63.

difference = min(differenceI), I = 0 . . . 63;

(9)

Here HS is the hit statistic, HSM is the mirror hit statistic, and operator “∧” counts the number of bits of difference of two hit statistics. S and R are indexes of partitioned regions.

3.2. The existing retrieval method using Zernike moments

Zernike moments5,6 are defined inside a unit circle. First, the radial polynomial vector R(ρ) is defined as:

Rnm(ρ) = n−|m| 2 X s=0 (−1)s (n− s)! s!(n+₂|m|− s)!(n−|m|₂ − s)!ρ n−2s R(ρ) ={Rnm(ρ)|n = 0, 1, 2, . . ., ∞, |m| ≤ n , and n − |m| is even} . Then the two-dimensional Zernike moment of an image I(ρ, θ) in polar coordi-nate is defined as:

Anm= n + 1 π X ρ X θ [Vnm(ρ, θ)]∗I(ρ, θ) , ρ≤ 1 . Here, Vnm(ρ, θ) = Rnm(ρ) exp(−jmθ).

The magnitude of Zernike moment (ZMM) is defined as

z ={znm|n = 0, 1, 2, . . . , ∞, |m| ≤ n , and n − |m| is even} where znm=kAnmk.

Here, we take the edge detected images as I(ρ, θ), ρ = 1 25 ∼

25

25, θ = 0◦, 5◦, 10◦, . . . , 355◦to evaluate ZMM, the smaller difference of the ZMMs of two images means that these two images are more similar.

4. Experimental Results

In our experiment, a database consisting of 88 groups of color trademarks are used. Based on the database, the proposed retrieval method is tested and compared with the method using Zernike moment. Eighty-eight different color trademarks are retrieved from Internet (see Fig. 11). For each color trademark, 19 similar trademarks are produced according to the following rules.

(1) Flip horizontal [see Fig. 12(b)].

(2) Scaling: Down sizing 75% [see Fig. 12(c)].

(3) Clockwise rotation 45◦, 90◦, 135◦, 180◦, 225◦, 270◦and 315◦ [see Figs. 12(d)– 12(j)].

(4) Distortion of pinch:−40%, −20%, 20% and 40% [see Figs. 12(k)–12(n)]. (5) Distortion of ripple: 50% and 100% [see Figs. 12(o) and 12(p)].

(6) Distortion of twirl:−30◦ and 30◦[see Figs. 12(q) and 12(r)]. (7) Add noise uniformly: 30% and 40% [see Figs. 12(s) and 12(t)].

(10)

Fig. 11. Eighty-eight different color trademarks.

Each color trademark and its 19 created similar images construct a group (see Fig. 13). All the color trademarks construct a database with 1760 images. Let X be the number of images belonging to the same group of the query image in the first n retrieved images, the recall rate Rn and precision Pn are defined as

(11)

(a ) (b ) (c ) (d ) (e ) (f ) (g ) (h ) (i ) (j ) (k ) (l ) (m ) (n ) (o ) (p ) (q ) (r) (s ) (t )

Fig. 12. Similar color trademarks creating rules. (a) Original. (b) Flip horizontal. (c) Down sizing 75%. (d)–(j) Rotation 45◦, 90◦, 135◦, 180◦, 225◦, 270◦and 315◦. (k)–(n) Pinch−40%, −20%, 20% and 40%. (o)–(p) Ripple 50% and 100%. (q)–(r) Twirl−30◦and 30◦. (s)–(t) Add noise 30% and 40%. follows: Rn = X 20× 100% , Pn= X n × 100% .

To evaluate the performance of the proposed color trademark retrieval system, each color trademark will be used as a query image and the recall rates (R20, R30 and R40 and precision (P05, P10, P15 and P20) will be derived. Using the hit statistic on a dartboard as feature, the average recall and precision curves are shown in Fig. 14.

Figure 15 illustrates the recall rates R20 using the original color trademark of every group as a query image.

(12)

Fig. 13. A group of color trademarks formed by a normal trademark and its 19 created versions. ! " " ! # $ (a) ! " # $ (b)

Fig. 14. The performance curve of hit statistic in a dartboard. (a) The recall curve. (b) The precision curve.

(13)

! "#$ % &' () *+ ,

Fig. 15. The recall rates R20of using hit statistic and using the original color trademark of every group as a query image.

Fig. 16. The top twenty retrieved images in the worst case of the retrieval result shown in Fig. 15.

The recall rate of the worst case based on hit statistic is 50%. Figure 16 shows the top twenty retrieved images in the worst case (the query image 01520.bmp) of the retrieval result.

Using Zernike moments as the feature to do retrieval, the average recall and precision rates are shown in Fig. 17.

(14)

! " (a) !" !# $ % (b)

Fig. 17. The performance curve using Zernike moment. (a) The recall curve. (b) The precision curve. ! " # $ % & '( ) * +, -. / / +. 0, 1

Fig. 18. The recall rates R20 of using Zernike moments and using the original color trademark of every group as a query image.

Figure 18 illustrates the recall rates R20 of using Zernike moments and using the original color trademark of every group as a query image. The recall rate of the worst case is 35%. Figure 19 shows the top twenty retrieved images in the worst case (the query image 01040.bmp) of the retrieval result.

(15)

Fig. 19. The top twenty retrieved images in the worst case of the retrieval result shown in Fig. 18. N

umber of Retrieval Images

! " (a)

Fig. 20. The performance comparison of using hit statistics and Zernike moments. (a) The com-parison of recall. (b) The comcom-parison of precision.

(16)

! " # $ % & ' ( ) ' (b) Fig. 20 (Continued ).

Figure 20 shows the recall and the precision comparisons of using hit statistic and Zernike moment. From this figure, we can see that the proposed method is superior to the method using Zernike moment.

5. Conclusion

This paper has presented a new feature called hit statistic on a dartboard to do trademark retrieval. The feature is robust to noise and some geometric distortion and have rotation and scaling invariant characteristics. An additional advantage of using this feature is that only some simple mathematical operations are needed and good recall rate can be reached. The experimental result also shows that the proposed method using hit statistic of dartboard is superior to that using Zernike moments.

References

1. M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, D. Steele and P. Yanker, “Query by image and video content: the QBIC system,” IEEE Comput. 28, 9 (1995) 23–32.

2. R. C. Gonzalez and R. Woods, Digital Image Processing, 1992, 226 pp.

3. T. Horprasert, D. Harwood and L. S. Davis, “A robust background subtraction and shadow detection,” Proc. ACCV’2000, Taipei, Taiwan, 2000.

4. A. K. Jain and A. Vailaya, “Image retrieval using color and shape,” Patt. Recogn. 29 (1996) 1233–1244.

5. W. Y. Kim and Y. S. Kim, “A region-based shape descriptor using Zernike moments,” Sign. Process. Imag. Commun. 16 (2000) 95–102.

(17)

6. Y. S. Kim and W. Y. Kim, “Content-based trademark retrieval system using a visually salient feature,” Imag. Vis. Comput. 16 (1998) 931–939.

7. C. H. The and R. T. Chin, “On image analysis by the methods of moments,” IEEE Trans. Patt. Anal. Mach. Intell. 10, 4 (1998) 496–513.

8. QBIC, the IBM QBIC project, Demo: http//wwwqbic.almaden.ibm.com/

9. A. Vailaya, A. K. Zhong and J. Yu, “A hierarchical system for efficient image retrieval,” Proc. IEEE Int. Conf. Pattern Recognition, Vol. 3, 1996, pp. 356–360. 10. C. T. Zhan et al., “Fourier descriptors for plane closed curves,” IEEE Trans. Comput.

21 (1972) 269–281.

Chuan-Cheng Wang received the B.S. degree in electrical engineering from National Sun Yat-Sen University, Kaoh-siung, Taiwan in 1992 and the M.S. degree in computer science from National Chiao Tung University, Hsinchu, Taiwan in 2001.

Ling-Hwei Chen re-ceived the B.S. degree in mathematics and the M.S. degree in applied mathematics from the National Tsing Hua University, Hsinchu, Taiwan in 1975 and 1977, respectively, and the Ph.D. in computer engineering from National Chiao Tung Uni-versity, Hsinchu, Taiwan in 1987.

From August 1977 to April 1979, she worked as a research assistant in the Chung-Shan Institute of Science and Technology, Taoyan, Taiwan, after which she worked as a research associate in the Electronic Research and Service Organization, Indus-try Technology Research Institute, Hsinchu, Taiwan. From March 1981 to August 1983, she worked as an engineer in the Institute of Information Industry, Taipei, Taiwan and is now a Professor at the Department of Computer and Information Science at the National Chiao Tung University.

Her current research interests include image processing, pattern recognition, docu-ment processing, image compression and image cryptography.

(18)

network. Pattern Recognition Letters 26:15, 2315-2335. [CrossRef]