Image Retrieval Based on Fractal Signatures

全文

(1)Image Retrieval Based on Fractal Signatures John Y. Chiang Z. Z. Tsai Department of Computer Science and Engineering National Sun Yat-Sen University Kaohsiung, Taiwan 80424. E-mail:[email protected]. E-mail: [email protected]. Abstract. orthonormal basis, iterative function system. 1.. The objective of the present work is to propose a. Introduction. novel method to extract a stable feature set. The retrieval of digital image is an active area of. representative of image content. Each image is. research in computer science due to the inefficiency of. represented by a linear combination of fractal. query processing utilizing traditional textual language.. orthonormal basis vectors. The mapping coefficients of. Most image retrieval paradigms fall between automated. an image projected onto each orthonormal basis. pixel-oriented. information. constitute the feature vector. The set of orthonormal. human-assisted. database. basis vectors are generated by utilizing fractal iterative. approaches differ in application domain, visual features. function through target and domain blocks mapping.. extracted, features discrimination criteria employed,. The distance measure remains consistent, i.e., isometric. and query mechanisms supported. Feature vector. embedded, between any image pairs before and after. characterizing image properties is generally composed. the projection onto orthonormal axes. Not only similar. of color, texture, shape and/or location information.. images generate points close to each other in the feature. Distance measure, e.g., n-dimensional Euclidean. space, but also dissimilar ones produce feature points. distance, is utilized to compute the similarity between. far apart. The above statements are logically equivalent. different feature vectors. Query specification tools are. to that distant feature points are guaranteed to map to. provided to allow user-constructed sketches and weight. images with dissimilar contents, while close feature. assignments among different feature components, etc.. points correspond to similar images. Therefore, utilizing. As an example, the QBIC system allows the color,. coefficients derived from the proposed linear. texture, or shape of an image or part of an image be. combination of fractal orthonormal basis as key to. compared with feature vectors from database images. search image database will retrieve similar images,. using Euclidean similarity measure [2]. The retrieval of. while at the same time exclude dissimilar ones. The. similar images from database corresponds to determine. coefficients associated with each image can be later. neighboring points in the proximity of the feature point. used to reconstruct the original. The content-based. of a query image.. models schemes. and [1].. fully These. query is performed in the compressed domain. This. The mapping of an image to the corresponding. approach is efficient for content-based query. Scaling,. feature vector is a process of dimensionality reduction.. rotational, translation, mirroring and horizontal/vertical. By finding a lower-dimensional representation of the. flipping variations of a query image are also supported.. image, an effective feature vector is expected to contain. Keywords:content-based image retrieval, fractal. vital characteristics of the original. The pitfall associated. -1-.

(2) spaces is highly desirable.. with the traditional approach is that even though similar images generally derive feature points close to each. Image retrieval by content allows a user to search. other. However, there is no guarantee that dissimilar. image database by specifying the content of an. images will map to distant feature points. For example,. exemplary image as the basis for retrieval [3]. In. the comparison of color feature usually employs certain. traditional content based indexing, content indices. measure of histogram. Images with resembling. (colors, shapes or textures) for each image in the. histogram distribution will be regarded as similar under. database are first extracted and appended to the image. this scheme. However, even with analogous histogram. data as overheads. The corresponding feature vector of. distribution, the color within a dissimilar image or. a query image is computed and compared to the stored. sub-image might be spatially distributed in a totally. feature vectors. Images most similar to the query are. different manner. Using color feature as a measure of. returned to the user. Given that images are usually. similarity between images is not powerful enough to. coded in compressed format in a database, it would be. exclude the false-positive cases. Moreover, a query. more efficient if the compressed data can also be used. image might be rotational, scaling, shifted, or. directly as indices for content-based query. In our. noise-corrupted variations of database images. A. proposed scheme, each image is decomposed into a. traditional retrieval algorithm might not be robust to. linear combination of fractal orthonormal basis. The. include similar database images of these variations,. coefficient of each term serves both as a feature. causing the occurrence of false dismissal.. component in the corresponding feature vector and. The corresponding feature vectors f 1 , f 2 , f 3 , f q. compressed data. The content-based query followed is. of images i1 , i2 , i3 , iq , respectively, are shown in. performed in the compressed domain. Contents of the. Figure 1. The derived feature points in the feature. image are embedded in the compressed data, which can. domain might not preserve the same spatial distance. be easily and efficiently used as indices for. relationship as their counterparts in the image domain.. content-based image retrieval. The extracted feature. When an image iq is used for querying a database,. vector, composed of linear coefficients, will be proved. i1 ,i3 will be included in the search result due to the. in the following section to preserve distance metric. proximity of points f1 , f 3 with f q in the feature. between the corresponding image points in the image. space. However, image i2 will be excluded since. domain.. point f 2 is considered as too distant from f q . A. In what follows, fractal orthonormal basis. dissimilar image, e.g., image i3 , mistakenly classified. approach will be introduced first. The procedure of. as similar one is called false-positive, while a similar. generating a set of fractal orthonormal basis for an. image, e.g., image i2 , incorrectly excluded from the. ensemble of database images will be outlined. Next, the. final search result is referred as false-negative. Being. conservation of Euclidean distance measure before and. unable to provide stable distance measure, most. after the mapping onto orthonormal basis will be. systems try to minimize false-negative results at the. proved. Image pairs with long feature distance in the. expense of an increased number of false positives. A. feature domain are guaranteed to be dissimilar ones,. compact, perceptually relevant representation of an. while feature points close to each other correspond to. image content that preserves the distance relationship in. similar images. The last section shows the effectiveness. terms of similarity metric in both image and feature. of this novel approach using a butterfly image database. -2-.

(3) as an example. Due to the preservation of distance. vectors v~1 , v~2 and v~3 , determined a priori according. relationship in both the image and feature domains,. to Vines’ approach, are orthonormalized later to form. consistent search results are obtained.. the first three of the required M r orthonormal basis. 2.. vectors, where v~1 = {1,1,Λ ,1}T , the DC value, v~ ={0,1,2,3,4,5,6,7,0,1,2,3,4,5,6,7,Λ ,0,1,2,3,4,5,6,7}T , the tilt. Fractal Orthonormal Basis Approach. 2. Barnsley suggested that storing images as. along. the. x-axis,. and. collections of transformations could lead to image. v~3 = {0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,Λ ,7,7,7,7,7,7,7,7} , the tilt. compression [4]. Jaquin was among the first to publish. along the y-axis.. T. The remaining basis vectors will be chosen to span. a fractal image compression scheme by regular. ( M r − 3). -dimensional. subspace. S0. partitioning of the image [5]. The accurate coding of a. the. range block is dependent upon there being a self-similar. perpendicular to the subspace spanned by a priori. domain block in the codebook. Because this piecewise. vectors v~1 , v~2 and v~3 . At the k-th iteration, the i-th. self-similar model is an approximation of real-world. projected range vector is denoted as s ik that resides in. data, there is no guarantee that perfect mapping can be. a corresponding subspace S k . The optimal basis. found. Observing that the iterative function system (IFS). vector direction is determined by taking the sik vector. coding technique seems to have a limit in the accuracy. with the largest correlation to all of the other s ik. that an image can be coded [6], Vines proposed a. vectors, i.e., the vector sik maximizes the following. scheme by finding a set of basis vectors to best. equation is selected:. represent the image in the sense of achieving the higher. NR. ∑ (s. fidelity with good compression [7,8]. Vines’ method. j =1, j ≠ i. k i. ⋅ s kj ) ,. was intended to improve the decoded signal-to-noise. where ( sik ⋅ s kj ) is the absolute value of the inner. ratio of fractal compression, no application to image. product of sik and s kj .. database retrieval was ever suggested.. Once each basis vector direction is determined, the. According to Vines’ approach, a set of. remaining sik vectors are projected onto the subspace. orthonormal basis vectors is created by Gram-Schmidt. orthogonal to s lk by the following projection operator. procedure and the range blocks are coded by projecting. PS k = I − slk ( slk slk ) −1 slk . T. the block elements onto this basis. The principle in. T. determining the orthonormal set is to create a basis that. The chosen basis vector direction is saved as t k and. allows each range block to be accurately represented. the process is repeated until the necessary M r − 3. with a minimum number of the basis vectors. These. vectors are obtained. In this manner, the set of M r − 3. fractal orthonormal bases are derived from the domain. orthogonal vectors, {ti }iM=1r − 3 , that best represents the. vectors. With these vectors, the range blocks can be. subspace S 0 is determined. A search is then. encoded with a simple projection operation, and the. performed through the domain vectors to find the best. map parameters will be the corresponding weights for. set of domain vectors for these direction vectors. The. this orthonormal basis.. domain vector with the largest component in the. For a range block of size LR × LR , let M r = L R 2. direction of the direction vector is selected. Because it is. NR i =1. be the length of the range vectors. Let RI = {~ ri }. possible that one domain vector has the largest. be the set of all range vectors in an image I. Three basis. component on more than one direction vector, each. I. -3-.

(4) two images I, J is determined by comparing a distance. domain vector is only allowed to be used once. The three fixed vectors v~1 , v~2 , v~3 and the. metric d IJ between WI and WJ . Next, we will. M r − 3 domain vectors form a set of M r vectors. show that the distance metric employing Euclidean. that span the space of the range vectors. If the selected. measure is isometric embedded in both image and. M r − 3 domain vectors are denoted sequentially as {v~ }M r , then the set of fractal basis vectors is equal to. feature domains, i.e., the proximity of two image points. [v~1 , v~2 , v~3 , v~4 ,Λ , v~M r ] . These basis vectors are. feature points WI , WJ , and vice versa.. further processed using the Gram-Schmidt procedure to. Proposition:. obtain the corresponding fractal orthonormal basis. The Euclidean distance between images I and J in the. I, J in the image space indicates that of corresponding. i i=4. ~ , q~ ,Λ , q~ ] . The coding of a given matrix Q = [ q 1 2 Mr I ~I range vector ~ r of image I with a set of weight w i. image domain and that of the corresponding feature vectors WI and WJ , derived by projecting range. i. ~ I = QT ~ ~ I . The is equivalent to w ri I or ~ ri I = Qw i i. blocks of I and J onto a set of orthonormal basis vectors,. previous two equations define the basic encoding and. are equivalent.. decoding process. An image I with range vector set. Proof:. RI = {~ ri }. The Euclidean distance d I J between images I and J. I. NR i =1. ,. the. set. of. WI = {wijI , i = 1,Λ , N R , j = 1,Λ M r }. weights can. be. can be formulated as. dI J = I − J ,. derived according to the fractal orthonormal basis matrix Q . The set of weights WI serves both as a. or expressed in terms of range. feature vector and compression coefficients of image I.. blocks. NR ~I ~J dI J = ∑ ~ ri I − ~ ri J , where ri ∈ RI , ri ∈ RJ . i=1. From the perspective of image database retrieval, the weight matrix WI represents the signature of image I. Each range block ~ ri I and ~ ri J of image I and J can. and a distance metric d IJ is employed to measure the. be further represented as a linear combination of M r orthonormal basis vectors Q = [ q~ , q~ ,Λ , q~ ] , with. similarity of images I and J based on feature points. 1. WI and WJ in the M r -dimensional space. The decompression process to reconstruct the original. NR. dI J = ∑ i =1. image from the image coding/decoding perspective. According to the above paradigm, an image I is into. non-overlapping. range. Mr. coefficients ~I ∈W w ~J respectively. w ij I , ij ∈WJ , i = 1Λ NR , j = 1Λ Mr ,. weight matrix WI is also utilized in the later. partitioned. 2. Mr. ∑ (w. I ij. j =1. NR. Mr. i =1. j =1. − wijJ ) q~ j .. = ∑ ( ∑ ( wijI − wijJ ) 2 q~ j2 )1 / 2 ,. blocks. ri I is decomposed RI = {~ ri I }iN=R1 . Each range block ~ into a linear combination of orthonormal basis vectors. Since all basis vectors are orthonormal, i.e., q~j2 = 1, q~i ⋅ q~j = 0, ∀i, j ∈{1Λ M r },i ≠ j.. by employing the same fractal orthonormal basis matrix. All cross-product terms are zeros.. Q . The set of coefficients for all range blocks, WI , is. NR. Mr. d I J = ∑ ( ∑ ( wijI − wijJ ) 2 )1 / 2. the signature for image I used in the retrieval of image. p =1 q =1. NR. database. Since the original image can be reconstructed. =∑. by employing the feature set WI with high S/N ratio,. i =1. Mr. ∑ (w j =1. I ij. − wijJ ) = d W I W J ,. WI therefore is a good representation of image I with. The above proposition states that the Euclidean distance. little information loss. The similarity measure between. measure remains the same after the projection of points. -4-.

(5) in image space into a set of orthonormal basis vectors in. truncation errors incurred in ignoring less significant. the feature domain. The image space and the feature. terms. The directions of axes for the aforementioned. space are “isometric” to each other. From this, we can. fractal orthonormal basis vectors are chosen with the. conclude that the closeness of two image points in the. largest correlation to the other range vectors in the. image space, i.e., d I J ≤ ε , implies the proximity of. ensemble of database images. The linear coefficients by. the corresponding feature points in the feature domain,. projecting an image onto the proposed orthonormal. dW I W J ≤ ε . The above statement is logically equivalent. space and the frequency components by transforming. to “if feature vectors WI and WJ are distant to each. the same image by Fourier transform are compared.. other, then image I is also distant to image J.” Since. The projected coefficients and frequency components. similar images are mapped to close feature points and. are first ranked according to their magnitudes,. only points close to the feature point of a query image. respectively. The accumulated ranked power spectrum. will be included in the retrieval results, images. starting with the largest coefficients or frequency. corresponding to distant features points will be. components are tabulated and normalized, as shown in. excluded. This property suggests that false-negative. Figure 2. Much fewer fractal orthonormal coefficients. cases are unlikely to occur. Therefore, employing the. are needed to constitute the same amount of energy in. proposed paradigm will not falsely ignore any similar. comparing with those derived by using Fourier kernels.. 3.. images based on the Euclidean metric in the feature. Experimental Results. space. Similar objects will be included in the final. In order to demonstrate the power of the proposed. retrieval set. Another facet of the above proposition. fractal orthonormal basis approach, a database. reveals that the proximity of feature points in feature. consisting of butterfly images is constructed. A total of. domain indicates the closeness of image points in. 1013 butterfly images with natural or uniform. image space. This statement is equivalent to that. backgrounds. dissimilar image points imply feature points far apart.. http://www.thais.it/entomologia/,. Therefore, the search in near proximity of the feature. http://turing.csie.ntu.edu.tw/ncnudlm/index.html,. point of a query image will not return dissimilar images.. http://www.ogphoto.com/index.html,. This property makes sure that no false-positive will. http://yuri.owes.tnc.edu.tw/gallery/butterfly,. occur. Utilizing the coefficients of the linear. http://www.mesc.usgs.gov/resources/education/butterfl. combination of an orthonormal basis set as feature. y,. vectors will retrieve consistent database retrieval result. http://mamba.bio.uci.edu/~pjbryant/biodiv/bflyplnt.htm.. excluding both false-positive and false-negative cases.. All images acquired are trimmed down to 320 × 240. are. downloaded. from. websites. and. Even though any orthornormal basis set can be. pixels with 24 bits of depth per pixel. Each image is. utilized to construct the feature space, a compact,. partitioned into non-overlapping range blocks with size. efficient representation of an image that leads to. 8 × 8 . The R, G, B color components are processed. concentrations of energy in as few coefficients as. independently to determine the fractal orthonormal. possible is preferred. Image energy concentrated in as. basis in each color plane. The fractal orthonormal basis. low-dimensional subspace as possible is highly. matrixes. desirable due to lower computation complexity. each. Figure 3 shows the fractal orthonormal basis. required in feature comparison process and fewer. matrixes QR , QG , QB derived by following the. -5-. QR , QG ,. and QB derived are 64-dimensional.

(6) procedures outlined in Section II for a training set of. corresponding to q~1 in the similarity computation to. 100 butterfly images in the database. The 64 fractal. counter the influence of changing light intensity. basis vectors of each color plane are composed of. between images.. uniform, edge or texture regions. The coefficient corresponding to the vector q~1 , the orthonormalized. image for retrieval. Since each image is coded with a. version of the first a priori vector v~1 = {1,1,Λ ,1}T , is. range block size of 8 × 8 , a sub-region with integer. considered as the brightness level of a specific color. multiples of 8 × 8 pixels can be expressed as a. component within an image.. partitioning of non-overlapping range blocks, as shown. A user can also specify a sub-region of a query. A total of 64 coefficients for each color component. in Figure 5 (a). However, if the specified sub-region is. are derived by projecting a range vector into an. not integer multiples of 8 × 8 , as illustrated in Figure 5. orthogonal space with 64 dimensions. A color range. (b), then a mask with the largest integral multiples of. vector can therefore be losslessly reconstructed by. 8 × 8 that can be fit into the sub-region is applied from. employing 192 linear coefficients. Since the energy is. the top-left corner with an increase of one pixel. highly concentrated in relatively few numbers of axes,. horizontally or vertically toward the bottom right corner.. most coefficients are negligible in the later similarity. On each iterative step, only the range vectors under. comparison stage. Only three most significant. current mask are formulated as a superposition of. coefficients per color component are employed in later. orthonormal basis vectors. The coefficients derived are. Euclidean distance computation, the remaining less. used as signature in the later matching process.. significant coefficients are considered with zero values.. The proposed fractal orthonormal basis approach. Since all projection coefficients of database images are. is also scale- and rotational-invariant. Users can specify. calculated only once and stored as compression. the range of scales and rotation angles of the query. coefficients, the computation of similarity measure. image. Variations of the query image multiplied or. involves only the derivation of feature coefficients for. rotated by different scaling factors and rotation angles. the query image, subtraction of matching coefficients. are coded. If the size of the query region after scaling. and summation of all squared differences. Therefore,. and rotation operations is not integral multiple of 8 × 8 ,. the retrieval process is very efficient. Figure 5. then the aforementioned mask will be applied. The. demonstrates the retrieval result by using a typical. coefficients corresponding to each scale factor and. butterfly image (scientific name: abpiercani) as query.. rotation angle of the range vectors under the current. The images retrieved are arranged according to the. mask are compared with those of all database images.. degree of similarity from left to right, top-to bottom.. Figure 6 illustrates the retrieval result by specifying a. The scientific name of the butterfly is listed on top of. sub-region with scaling factors ranging from 0.8 to 1.2. the image. After providing a query image, a user can. and a 30 degree increment of rotation angle. In. choose a subset of R, G, or B color components as. comparison with Figure 4, since a brown dark pattern. matching indices for feature discrimination. Only the. on the wing of the butterfly is specified for searching,. coefficients corresponding to the selected color planes. images with possible slight scaling (0.8 ~ 1.2) and. will be included in the calculation of Euclidean distance.. orientation difference (every 30 degree) of the marking. The brightness factor can also be selectively turned on. are included in the retrieval set.. or off by including or excluding the coefficient. -6-.

(7) 4.. Conclusions. References. A feature extracting method that preserves. 1. Gupta, A. and Jain,R. Visual Information Retrieval,. distance measure in the image and feature spaces is. Commun. ACM 40, 5 (May 1997), 70-79.. provided. The fractal orthonormal basis set introduced. 2. Flickner, M., Shawney, H., Niblack, W., Ashley, J.,. can better summarize image contents with fewer. Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D.,. orthonormal axes than those of Fourier kernels. Lower. Perkovic, D., Steel, D., and Yonker, P. Query by. computation requirements and truncation errors are. image and video content: The QBIC system, IEEE. obtained in comparison with other orthonormal. Comput. 28, 9 (Sept 1995), 23-31.. decomposition techniques. In retrieving similar images. 3. Rui, Y., Huang, T. S., and Chang, S-. F. Image. from database, only few coefficients are required to be. retrieval: Current techniques, promising directions. evaluated in the computation of Euclidean distance.. and open issues, Journal of Visual Communication. The retrieval efficiency in terms of computation. and Image Representation (March 1999). 4. Barnsley, M. F. Fractals Everywhere, Academic. complexity and speed is very high.. Press, San Diego (1988).. A database consisting of butterfly images collected from existing websites is constructed to demonstrate the. 5. Jacquin, A. A fractal Theory of Iterated Markov. power of this approach. The feature discrimination. Operators with Applications to Digital Image Coding,. procedure by calculating the Euclidean distance. PhD thesis, Georgia Institute of Technology (August. between the corresponding linear coefficients can. 1989).. retrieve shift-, rotation-, and scale-variations of the. 6. Cheung, K.-M. and Shahshahani M. A comparison. query image, as specified by the user through the query. of the fractal and JPEG algorithms, TDA Progress. interface. Contents of the image extracted are. Report 42-107 (Nov 1991), 21-26.. embedded in the compressed data that can be easily and. 7. Fisher, Y. Ed. Fractal Image Compression, Theory. efficiently used as indices for content-based image. and Application, Springer-Verlag, New York (1995),. retrieval. Logic predicates, e.g., AND, OR, NOT, or. 199-214.. spatial constraints might be further imposed on plural. 8. Vines, G. and Hayes, M. H. Nonlinear Address Maps. number of sub-regions of a query image to proceed. in a One-Dimensional Fractal Model, IEEE Trans.. more complicated image retrieval applications.. Signal Processing 41, 4 (April 1993), 1721-1724.. through. most. Image-feature. feature pair. extraction. process.. and. {i3 , f 3 }. {i2 , f 2 }. illustrates the case of false-negative and false-positive, respectively.. Figure 1. The distance relationship between image points and corresponding feature points is not preserved. -7-.

(8) Figure 2. Normalized accumulated ranked power spectrum. G, B color components and brightness level of the. of the proposed fractal orthonormal basis (FOB). query image are all selected. The rectangular area in. approach and Fourier transform, starting with. the upper right-hand corner provides an enlarged. component with the largest magnitude. Much. viewing window for the image retrieved.. fewer fractal orthonormal coefficients are needed to constitute the same amount of energy in comparing with those derived by using Fourier kernels. (a). (b). Figure 5. An enlarged view of the selected sub-region of a query image with size that is (a) integral multiples of 8 × 8 , (b) not integral multiples of 8 × 8 . For the case of (b), a mask with the largest integral multiples of 8 × 8 that can be fit into the. (a). (b). sub-region is applied from top-left toward the lower-right corner with an increment of one pixel is applied. On each iteration, the coefficients derived are used as features of the sub-region to compare with those of database images.. (c) Figure 3. The 64 8 × 8 fractal orthonormal basis vectors of (a) R, (b) G, and (c) B color components, respectively, derived from an ensemble of 100 butterfly database images. The size of each vector is enlarged by two for ease of observation... Figure 6. Retrieval results based on a sub-region of a query image with scaling factors 0.8 through 1.2 and rotation angles every 30 degrees.. Figure 4. An image retrieval example. The features from R,. -8-.

(9)