Shape-Direction-Adaptive Lifting-Based Discrete Wavelet Transform for Arbitrarily Shaped Segments in Image Compression

全文

(1)IEICE TRANS. INF. & SYST., VOL.E91-D,. NO.10 OCTOBER. 2008 2467. PAPER. Shape-Direction-Adaptive. Lifting-Based. Transform. Shaped. in Image. for Arbitrarily. Wavelet. Segments. Compression Sheng-Fuu. LINa),. SUMMARY In thispaper,a newlifting-basedshape-direction-adapti discretewavelettransform(SDA-DWT)whichcan be used for arbitraril shapedsegmentsis proposed.The SDA-DWTcontainsthreemajor techniques:the lifting-basedDWT,the adaptivedirectionaltechnique,and the conceptof object-basedcompressionin MPEG-4. WithSDA-DWT,the numberof transformedcoefficientsis equal to the numberof pixelsin the arbitrarilyshapedsegmentimage,and the spatial correlationacross subbandsis well preserved. SDA-DWTalso can locally adapt its filtering directionsaccordingto the texture orientationsto improveenergycompactionfor imagescontainingnon-horizontalor non-verticaledgetextures. SDA-DWTcan be appliedto any applicationthat is waveletbasedand the liftingtechniqueprovidesmuch flexibilityfor hardwareimplementation Experimentalresults show that, for still objectimageswith rich orientation textures,SDA-DWToutperformsSA-DWTup to 5.88dB in PSNR under 2.15-bpp(bit/object pixel)condition,and reducesthe bit-budget up to 28.5%for losslesscompression.SDA-DWTalso outperformsDADWT up to 5.44dB in PSNRunder 3.28-bppcondition,and reducesthe bit-budgetup to 14.0%. key words: compression,textures, set-partitioningembeddedblock codex(SPECK),object-basedvideocoding,shape-direction-adaptive DWT (SDA-DWT) 1.. Discrete. Introduction. The conventional separable 2-D DWT can be implemented by consecutively applying the 1-D DWT in horizontal and vertical directions, or vice versa. Hence, only these two directions of the high-pass filters have vanishing moments. For images containing large amount of edges which are not vertical or horizontal, the conventional DWT cannot provide efficient representations, since there are many large amplitude coefficients in the high frequency subbands of the transformed images. In order to solve this problem, we need to design a DWT whose directions are not fixed to vertical or horizontal. On the contrary, the new separable 2-D DWT will be capable of choosing the best directions for executing two 1-D DWTs. Such DWTs that can choose the optimal transform directions are usually called direction-adaptive DWTs. Recently, Ding et al. proposed the adaptive directional lifting-based wavelet transform (ADL-DWT) [1] for Manuscript received February 22, 2008. Manuscript revised May 19, 2008. The authors are with the Department of Electrical and Control Engineering of National Chiao Tung University, Hsinchu, Taiwan. The author is also with the Department of Electrical Engineering of Chung Hua University, Hsinchu, Taiwan. a) E-mail: [email protected] b) E-mail: [email protected] DOI: 10.1093/ietisy/e91-d.10.2467 Copyright (c). 2008. The. Institute. of Electronics,. Nonmember. and Chien-Kun. SU,b),. Student. Member. image compression, and they claimed that ADL-DWT can improve the compression performance of a texture-rich image up to 2.0dB. In [1], they used the lifting scheme [3], [4] and the sinc-interpolation [5] to design ADL-DWT. About the same time, Chang et al. also proposed a directionadaptive discrete wavelet transform (DA-DWT) [6] for image compression which was based on the lifting scheme, too. DA-DWT does not involve the sub-pixel interpolation of ADL-DWT, but they use the existing input samples for prediction and update. DA-DWT can attain a gain up to 2.5dB in PSNR over the traditional DWT for typical testing images. From [1] and [6], we clearly know that DWT with direction-adaptive capability can improve the performance of the conventional DWT to a new level, but the extra cost is complicated computation and the side information processing, which contains the direction information, to be stored, coded, and transmitted. Besides the complexity and side information, ADL-DWT and DA-DWT are designed for compressing a rectangular image, and they cannot be used to object-based or arbitrary-shape image compression directly. Both ADL-DWT and DA-DWT are based on the lifting-based DWT, since the lifting-based DWT is convenient for direction-adaptive functionality realization and hardware implementation. The convolution-based DWT or FIR (finite impulse response) bank structure DWT proposed by Mallat [7] is the traditional method to implement DWT. The convolution-based DWT suffers from the problems that it is complex and difficult for hardware implementation. Therefore, Daubechies and Sweldom proposed the lifting-based DWT [3] which is less complicated than the convolution-based DWT. The main concept of the lifting-based DWT is to break-up the low-pass and high-pass wavelet filters into a sequence of lower-triangular and uppertriangular matrices, and implement the filter by banded matrix multiplications. Because of low complexity, ease for hardware implementation, and the capability of lossless reconstruction, the lifting-based DWT was recommended by JPEG2000. An important feature of MPEG-4 is the functionality that the compressed forms of visual objects are available, and this feature provides great flexibility for manipulating visual objects in multimedia applications. For this functionality of MPEG-4, many coding techniques for coding arbitrarily shaped visual object were developed, and the shapeadaptiVediscrete cosine transform (SA-DCT) [8] is the most Information. and Communication. Engineers.

(2) IEICE TRANS. INF. & SYST., VOL.E91-D,. NO.10 OCTOBER. 2008. 2468. popular one to code the texture of the intra frame of a visual object in video coding. Since SA-DCT divides the object to be compressed into many 8-by-8 blocks, the non-vertical and non-horizontal boundaries of objects can not be represented perfectly, i.e. some positions of the 8-by-8 boundary block are not in the object. In order to overcome the problems of SA-DCT, Li et al. proposed the shape-adaptive discrete wavelet transform (SA-DWT) [9] for coding the texture of an arbitrarily shaped visual object. SA-DWT uses DWT to replace DCT, and it can handle arbitrarily shaped 2-D objects and gets better performance by using the more complicated algorithm. Lu and Pearlman combined the shape-adaptive DWT and the set-partitioning embedded block coder (SPECK) [10] to propose the object-based SPECK algorithm [11] for coding the arbitrarily shaped objects of the intra frames in MPEG-4. In this paper, we propose a shape-direction-adaptive lifting-based DWT (SDA-DWT). By using SDA-DWT, we can perform directional adaptive DWT on an arbitrarily shaped and partitioned visual object while the shape mask and partition of the object image are given. SDA-DWT can be directly applied to still image compression and objectbased visual compression in MPEG-4 with high efficiency. Experimental results show that SDA-DWT outperforms SADWT (a conventional-separable-2D-DWT based method) by 5.76dB in PSNR for the test image under 1-bpp condition, and reduces 28.5% bit-budget of the coded bit-stream. For our test image, SDA-DWT also outperforms DA-DWT (a lifting-based method) by 4.88dB under 1-bpp condition, and reduces the bit-budget of the coded bit-stream 14.0%. The remainder of this paper proceeds as follows. In Sect. 2, the proposed SDA-DWT is described in detail. The liftingbased DWT, the adaptive directional DWT, and the shapeadaptive DWT are also included in this section. Experimental results are given in Sect. 3, followed by the conclusions in Sect. 4. 2. Shape-Direction-Adaptive form (SDA-DWT). Discrete Wavelet Trans-. In this section, we present a new discrete wavelet transform, which is both shape-adaptive and direction-adaptive and named shape-direction-adaptive discrete wavelet transform (SDA-DWT). Besides the direction-adaptive capability like ADL-DWT or DA-DWT, SDA-DWT is capable of handling arbitrarily shaped segments. Some related topics such as the lifting structure of DWT, the direction-adaptive DWT, and the shape-adaptive DWT are introduced in this section, too. 2.1 Lifting-Based Structure Wavelet transform is well known as a multi-resolution analysis that provides many advantages: joint space-spatial frequency localization, clustered wavelet coefficients of significance with strong correlations between subbands, and exact reconstruction, which are truly beneficial to image compres-. Fig. 1 The block diagramsof the 1-level1-Dconvolution-based DWT and IDWT. sion. Discrete wavelet transform (DWT) decomposes a signal: Sl(n) at resolution l into two components: (1). (2) where. Sl(n). olution. l+1,. two. is. its. approximation. Dl(n). successive. is. the. at. detail. resolutions:. l and. g(n)=<ƒÕ,ƒÓ-1,-n>, <•E,•E> is is. a valid. (mother). an wavelet, ƒÓ. ƒÓ1,-n(x)=2-1/2ƒÓ(x/2-n). actly. reconstructed. following. inverse. DWT. Sl+1(n). l+1,. next. coarser. the. product scaling. original and. res-. between. the. h(n)=<ƒÓ,ƒÓ-1. inner is. The from. the. information. ,-n>,. operator, ƒÕ function,. signal. Dl+1(n). can by. using. and be. exthe. (IDWT):. (3) where h(n)=h(-n) and g(n)=g(-n). The DWT whose transform is based on Eqs. (1) and (2) and inverse transform on Eq. (3), is called the convolution-based DWT. Figure 1 shows the block diagrams of the convolution-based one-level DWT and IDWT, where s and d are equivalent to Sl+1 and Dl+1, respectively, and x is equivalent to Sl. The convolution-based DWT was widely used for researching and implementing DWT for a long time, so most researchers are familiar to it. The disadvantages of convolution-based DWT are its complexity, large storage space requirement, and difficulty of hardware implementation. Daubechies and Swelden had proposed a new approach, called lifting-based DWT, for implementing DWT. The lifting-based scheme is to decompose a discrete wavelet transform into a finite sequence of simple filtering steps, which are called lifting steps. Using the language of algebraists, the decomposition of lifting-based DWT corresponds to a factorization of the polyphase matrix of the wavelet into elementary matrices. The lifting-based approach can provide advantages such as in-place implementation of the fast DWT, capability of integer-to-integer transform, ease for hardware implementation, less storage space requirement, and flexibility for some adaptations on DWT. For the lifting structure, each finite impulse response (FIR) wavelet filter is factored into several pairs of lifting steps. One pair of lifting steps includes a prediction step.

(3) LIN and SU: SDA-DWT FOR ARBITRARILY. SHAPED SEGMENTS. IN IMAGE COMPRESSION 2469. Fig. 2. The block. of lifting. steps.. followed the. by. 5/3. only. one. shows. of. step. lifting-based. (even. predicted. by. pends. on. and. each. samples. its the. have. been. odd. are. are. in the. the. high. DWT. IDWT. preceding. can. be. one. pair. implemented. by. of the. 5/3. low. de-. update they. Finally,. frequency. and. complete. steps.. wavelet. is. since step.. The. A direction. 5/3-wavelet. step,. the. The. reversing. DWT.. Fig. 4. one-dimensional. generated. in. the. of lifting. of a lifting-based. prediction. value. subband. The structure. residual. update. prediction. lifting-based. a realization. the. inputs,. to produce. Fig. 3 DWT.. two. sample. the. that,. original. frequency. with. corresponding are. Note the. down-sampled. and. lifting-based. not. by. the. 2. one. (This. by. followed by. samples.. samples. changed. outputs. tions. is updated. Fig.. into. odd. samples. the. with in. one-dimension. each. sample. in. discuss. classified. replaced. odd. only realized. containing. the. are. Then,. step,. odd-neighbor. subband. based. the. even. For. and. one pair. diagram. even-neighbor. subtracting. the. we. DWT. step.. type.),. containing. be. block. samples first.. specific. prediction. step,. can. lifting-based. odd). the. from. paper,. update. After of. this. The. input. wavelet. from. value.. one. DWT. wavelet. steps.. the. some. the. obtained. the. and. In. 5/3. of the. DWT,. categories. the. lifting. structure. prediction. step.. and. pair. the. of the lifting-based. update. wavelet,. one. the. diagram. a. the. steps. following. example. with. angle ƒÆ in 1-D •ehorizontal•f. in. equa-. lifting-based. selection. DWT.. lifting-. x'(2k)=yext(2k)-[yext(2k-1). DWT: +yext(2k+1)+2]/4,. (10). y(2k+1)=xext(2k+1) -[xext(2k)+xext(2k+2)]/2,. (4). x'(2k+1)=yext(2k+1)+[x'(2k) +x'(2k+2)]/2,. y(2k)=xext(2k)+[y(2k-1)+y(2k+1)+2]/4,. (5). d(k)=y(2k+1),. (6). s(k)=y(2k).. (7). where. yext. is. reconstruction. the. based. the. the. 5/3. Equations to. wavelet. prediction (6). generate. high the. the. frequency 1-D. version. and. (7). are. low. data,. x.. in. and. this the. two. (4). d(k),. output. of x is. corresponding Fig.. 3.. in Eqs.. The. even,. the. are. (4). and IDWT. (5). can of. the. x)/2. be 5/3. The. also. the. we. the. operations. is. 2.2. by. version. the. applying. input Eqs.. y. Then,. using. odd. Eq.. of. of. x. (8). the. (10).. samples. Lifting. conventional. and. even. Finally,. y. The (9). input. and. x'. liftingfor. up-. samples. using. Eq.. (11). x.. DWT. as shown. tical. (9). to. directions. axes.. For. the. and. based tion. or. structure. update. in. the. 1-D. vice. update. 2-D. parallel. to. steps. are. (DA-DWT). step. the. or. of. be. means. implemented and. if. the. is. adaptive 4 shows. a direction-adaptive. we. ver-. use. the. prediction. horizontal. whose. adaptive. Figure. can. in horizontal. DWT,. DWT. (ADL-DWT). lifting. That. implement are. DWT DWT. versa.. lifting-based. DWT DWT. 2-D. applying. directions,. diction (8). separable. consecutively. adaptive. y(2k)=s(k).. by the. Directional. lifting. (8)-(11).. y(2k+1)=d(k),. from. of. s to produce. reconstructed reconstruct. The. have. represented. wavelet. begins. d and. extended. version. x is. Assuming of. symmetric. used and. extended and. are to. where. symmetric. x={1,2,3,4,5},. 0•…k•…(length. to Eqs. lifting-based. (5). s(k). respectively,. xext={•c,4,3,2,1,2,3,4,5,4,3,2,•c}. length. and. respectively. relations. subband. xext means that. Eqs. function,. down-sampling. output. and. Assume. paper,. update. frequency. subband. input of. used. function. the. IDWT. sampling For. is. (11). or. directions called. and vertical of. pre-. a direction-. directional a direction DWT,. liftingselecwhere.

(4) IEICE. 2470. age. into. many. directional some. from. the. square merging. some. direction. this. compression, in a block. value. for. a given. for. an odd. sample. in. the. other. band. be. is. equivalent. the. the. is. the. . Applying any. tions. sub-pixel. value were. for. used. prediction. the. same. tions. in. shows. as. an. In. Fig.. odd. neighbors. [1].. In. those. in. nine. 6, each along. a rectangular of fixed-size. in of. they. but. these. In. the. has and. their. line. with ƒÆ=45•‹.. [1]. and. DA-DWT. an. nine. we. use. the. dividing blocks.. [6]. are. have. strong On sub-. direc-. Figure. adaptive. DWT,. adaptive. DWT. DWT. of of. neighbors. by. proposed. its. two. odd. to com-. the whole image into After dividing an im-. in the. low. the. ing. and intra divided. ing.. The. gion. that. can object. be. motion we. focus. frame into shape the. (Fig.. 0 for. which each. is. this. efficiency major. intra. parts frame. will. not. frame. coding. is used. to. the. simplest 1 for. coding, cod-. this. paper.. coding. can. texture. represent shape. each the. an. discussed. in. object. in in. frame be. video. outside. pro-. in multime-. which. value. object. functionality. two. and. stan-. a visual. object. the. object. MPEG-4. inter. and. the. manipulating. The. 7 (b)). has. direction-. meaningful. coding. pixel. direction-. applications,. object-based. occupied,. figure. the •ecolumn•f. and. coding.. intra the. shape. mask. value. the. subsampling. compression. One. of. object. object. a binary and. on. coding. the. degrees.. urgent.. are. prediction,. is. on. in a one-level. of. and. the. frame. di-. DWT. the •erow•f. a visual. There. inter. the. of making. improves. like. directional. after •erow•f. more. form,. coding.. is the. involving. The be. video other. is. as. sub-. subsampling. DWT. semantically. coding.. a. Segments. manipulating. and. bit-rate. step. indexing,. compressed. of. just. adaptive. multimedia. and. is. is performed. 90. a functionality. for. applications. the. in. sum. is. a •erow•f. block. of. more. such. for. same. accessing, at. the. realization. Shaped. growth. absolute. DWT. i.e.. last. segment. can. direction. method. directional. Arbitrarily. becoming. flexibility. here,. DWT.. the. clockwise. information. the. DWT. is rotated. fast. supports. and. the. searching,. are. dard. if. for. object-based. 5. and. The. very. not. block,. frequency. selection. directional. is the. vides. complex. a 5/3-wavelet. is updated. image. subband.. direction. DWT.. dia. were. the •erow•f. DWT. available. direc-. directions. involve. corresponding of. can. nine. directions. not. angle ƒÆ. angle ƒÆ. performance.. step. the. used. paper,. better. samples. image by small square. the. by. only. nine. does. prediction even. also. this. method. and. directions. sample. [6],. [1]. this. axis. (radians),. PSNR. lossy. high. directional. direction. the •ecolumn•f. in. smallest. DWT,. direction-adaptive. needs. although. and ƒÎ/2. update,. because. ADL-DWT press a lot. 0. computation the. of. in and. [6],. sub-pixel. technique,. between. lifting. frequency. choosing. subsampling. direction-adaptive. level. be. to im-. subband.. low. left. the. 1-D. the. After. segment. visual. horizontal. or. direction. frequency. the. the •ecolumn•f. Because. the. has. horizontal. following. 2.3. intersects. energy. conventional. DWT.. whole. line. bits. lossy. highest. should. in. by. high. of and. the. complete,. step. in the. in. adaptive. direction. the. frequency. optimal. DWT. step. stage,. way. the the. determined. last. rection. with ƒÆ=45•‹.. to that. coefficients. sampling. the. of. For. lossless. low. best dilifting. directional. has. of. rectan-. amount. of the that. di-. instead. a larger. least. compaction. directional. The. stage. form. selection. in the. Hence,. approximately. the. update. same. [1] . Different splits a larger. block.. one. direction. energy. is little.. which. The. the. Either. effect. hand,. subband. 6. best. the. are. 1-D •ehorizontal•f. DWT.. Fig.. the. bit-budget.. compaction. with. image compression, the update of the directional. direction. be. . Then,. blocks. to. for. direction. in [6]. rectangular). best. should. lifting. used. 2008. direction. determined. rectangular. spends. or. the. DWT. the. that. (square. optimal is. block. blocks. and. OCTOBER. information. method. square. NO.10. same. side. small. is the. energy directions. the. DWT. compression,. prediction. [1],. small. the. of. several. block. For lossless (ƒÆ) of prediction. age. Nine. bits. gular rection. compress. 5. in. into. the block. rectangular. the. method. block. blocks,. with. a large. saving. VOL.E91-D,. of each. blocks. to form for. square. DWT. connected. grouped. INF. & SYST.,. small. lifting. rection. Fig.. TRANS.. pixel. object.. codthe. re-. mask in the Thus,.

(5) LIN and SU: SDA-DWT FOR ARBITRARILY. SHAPED SEGMENTS. IN IMAGE COMPRESSION 2471. Fig.. 7. with. background,. Test. objectl (b). and shape. its. shape. mask. mask: of object. (a). 256•~256. object 1. image. 1.. Fig. 9 pixels. Fig. 8 pixels. An arbitrarily in the prediction. shaped. segment. and the relation. step of the I-D row direction. An arbitrarily in the update. shaped. segment. and the relation. step of the 1-D horizontal. of its even and odd. lifting-based. DWT.. of its even and odd. lifting-based. DWT.. using the shape mask, the object in an image can be easily segmented. The most popular technique for object texture coding is the shape-adaptive DCT (SA-DCT) [8], which uses 8-by-8 blocks to approximate the shape of the object to be coded. Since an object usually can not be represented by 8-by-8 blocks perfectly, a lot of boundary blocks do not totally reside in the object and make this method inefficient. S. Li et al. proposed a shape-adaptive discrete wavelet transform (SA-DWT) for arbitrarily shaped visual object coding [9], and they used SA-DWT for the texture coding of the intra frame part in object based video coding. Lu et al. also proposed an object texture coding technique [11] that combined SA-DWT and SPECK algorithm [10]. The experimental results in [11] showed that SA-DWT with extensions of zerotree entropy coding (ZTE) outperforms SA-DCT up to 0.97dB in Y-plane PSNR, 1.29dB in U-plane PSNR, and 0.89dB in V-plane PSNR, for the Akiyo sequence (CIF) at 1.0bpp. The works of [9] and [11] used the convolution-based DWT, and both of them involved complicated computation. Using the lifting-based DWT and global even-odd relation we can realize SA-DWT easily. Figure 8 shows that an arbitrarily shaped segment contains 10 pixels in a 6-by-8-pixel image, and it also shows the relation of even and odd pixels in the prediction step. In Fig. 8, the two arrows, pointing to each odd pixel, are used to indicate that the odd pixel's two nearest neighbors in the same row are used to predict. Fig. 10 The subsamplingresult of the arbitrarilyshaped segmentin Figs.8 and 9 after 1-Dhorizontallifting-basedDWT.. the odd pixel. For the 5/3 wavelet DWT, the prediction value of each odd pixel in the lifting-based horizontal 1D DWT is the mean value of its right and left neighbors. After the prediction step, the residual that each odd pixel subtracts its prediction value is stored in the position of the odd pixel. If the even neighbor does not in the segment, the symmetric extension is used to generate a new even pixel value for prediction. For the single point in a row (e.g. the pixel at row 1 and column 3), its two neighbors for prediction are set to zero. Figure 9 shows the update relation of the arbitrarily shaped segment when the lifting-based horizontal 1-D 5/3 DWT is performed on the segment. Each even pixel in the segment is updated by using Eq. (5) in the update step, and the corresponding pixels (coefficients) are its left and right odd neighbors in the 1-D lifting-based horizontal 5/3-wavelet DWT. The processing methods of the symmetric extension and single points are the same as those in the prediction step. The last step of the 1-D lifting-based DWT is a subsampling step by which the transformed 1-D data are classified into high and low frequency subbands, and the result is shown in Fig. 10. Then, the transformed image in Fig. 10 is transformed by the vertical lifting-based 5/3-wavelet transform. The algorithm of 1-D lifting-based vertical 5/3-wavelet DWT is just like the algorithm of the.

(6) IEICE TRANS. INF. & SYST., VOL .E91-D,. 2472. Fig.. 11. The. DWT. first. (ƒÆ=45•‹). prediction. performed. step on. an. of. the. 2-D. arbitrarily. The first update. (ƒÆ=45•‹). performed. 1-D. lifting-based. 2.4. The. segment.. Proposed. results. DWT so. [1],. for [6]. they. still. were can. rectly. proposed they do paper,. the. in not we. adaptive. and. [11] offer. to. arbitrarily. DWT. or without age. this. transformed. of. paper,. partition, the. are. and. object. and and. the. very. and. DA-. The. segment the. and. has. both. and inputs. image. outputs. images,. shaped. we of. containing. corresponding. the. are. [1]. di-. method. shaped object, but functionality. In this. which. the. the. they. [9]. abilities, DWT.. are. experimen-. rectangular. can process arbitrarily the direction-adaptive a new. the. that. SA-DWT. direction-adaptive. in. be. show. an. shape-direction-adaptive proposed. and. ADL-DWT. processing. hand,. proposed and. [11]. for. process other. the. corresponding. shape-. call. it the. SDA-DWT, the. shape are. the. object. mask. with. transformed. im-. shape. mask. after. described. by. using. SDA-DWT. The Figs.. 11-16,. Figs.. 8 and. Compared SDA-DWT. proposed and 9. and. to step. SDA-DWT. can. the. same. the. 5/3-wavelet. SA-DWT is shown. of Fig. 12 in SDA-DWT.. Fig.14 The secondpredictionstepof the2-D SDA-DWTon an arbitrarily shapedsegment.. functionalities DWT,. coding.. designed not. On. [9],. result. the 45-degree direction is selected, so each odd pixel in the segment is predicted by two nearest even neighbors on the 45-degree line passing through this pixel. Then, each odd of. image. subsampling. DWT.. direction-adaptive. [6],. The horizontal. DWT. segment.. 5/3-wavelet. improvements in. efficient. shaped. SDA-DWT. and. important. tal. of the 2-D shape-direction-adaptive. horizontal. Shape-adaptive two. step. on an arbitrarily. 2008. shape-direction-adaptive. shaped. Fig. 13. Fig. 12. NO.10 OCTOBER. be. arbitrarily. step. shaped are. in. in Fig.. used. Fig. 11.. 8, In. the Fig.. segment for. in. illustration.. corresponding 11,. assume. that. pixel is replaced by the residual obtained from subtracting the pixel value by the prediction value. If the prediction is good enough, the residual will be a small value. In the prediction step, the symmetric extension method is used for generating those even samples not in the segments, and the symmetric relation is about the line, passing through the odd pixel to be predicted, of 45 degree. According to Eq. (5), the update step in Fig. 12 is corresponding to the Fig. 9 of SA-DWT, and every even sample in the segment is updated by its two nearest odd neighbors (They already have been replaced by the residual values in the previous prediction step.) on the 45-degree line. After performing a pair of lifting steps (i.e. a prediction and an update steps), the transformed image is subsampled, and the result is shown in Fig. 13. The subsampling process is the same as the conventional horizontal subsampling method, and the subsampled coefficients are classified into the low-frequency subband and the high-frequency subband. When the horizontal subsampling step is complete, the second part (corresponding to the vertical conventional 1D DWT) of SDA-DWT begins from a prediction step (the second prediction step in SDA-DWT). Each odd sample in.

(7) LIN and SU: SDA-DWT FOR ARBITRARILY. SHAPED SEGMENTS. IN IMAGE COMPRESSION 2473. columns of the segment is predicted by its left and right even neighbors on the 45-degree line compared to the vertical line (Fig. 14). Then, the second update step of SDA-DWT is performed on the even samples in columns of the segment (Fig. 15). Finally, a conventional subsampling along the vertical direction is performed on the coefficients in Fig. 15, and the image is transformed and divided into four subbands LL, LH, HL, and HH (Fig. 16). The symmetrical extension is used to generate the even samples and odd samples, not in the segment, for prediction and update, respectively. From Fig. 11 through Fig. 16, the one-level SDA-DWT is performed, and the LL subband can be used to be further transformed for multi-level SDA-DWT. The new shape mask is generated by subsampling the input shape mask along the horizontal and vertical directions, respectively. Both ADL-DWT and DA-DWT partition an image into many small blocks. For texture features which are smaller than the smallest block size in the two methods, ADL-DWT and DA-DWT can not well exploit the correlation of the texture features in the small block. Moreover, even for large. texture features, since the block locations are fixed, this makes the partition usually not optimal. Thus, some energy will be left in the high-frequency subband. The proposed method can handle any shaped segments at any location, so it can exploit the correlation of such textures and get better energy clustering to attain better compression efficiency.. Fig. 15 shaped. The second. update. step of the 2-D SDA-DWT. on an arbitrarily. segment.. Fig. 16. The vertical. subsampling. result. of Fig. 15 in SDA-DWT.. SA-DWT and the object-based SPECK are for object image compression, and they have the ability to process any shaped visual object. They do not have the texture-featuresize and fixed-block-location problems, but they use conventional (i.e. horizontal and vertical) directions for prediction and update. Lack of the capability of directional adaptability makes them unable to well exploit the spatial correlation of non-vertical and non-horizontal line textures. On the contrary, SDA-DWT can adapt the filter direction, according to the texture features in each of the partition segments of the interested object, for well exploiting the correlation and obtaining better compression efficiency. In SDA-DWT, for handling the finite length data of 1D wavelet transform, the symmetric extension of input data is used. The symmetric extension is effective and easy to implement, but, for the boundary between two partition segments, applying symmetric extension causes blocking effect for low bit-rate conditions. For such a problem, using the actual data at the extension points can alleviate the blocking effect. Periodic extension is another solution for finite length 1-D DWT computation, but it usually suffers from causing abrupt change at boundaries and needs more registers to implement. 3. Experimental Results In this section, three test object images (Figs. 7, 18, 20) are used for simulation to evaluate the performance of SDADWT, SA-DWT and DA-DWT. The original sizes of test images 1(Fig. 7) and 2 (Fig. 18) are 256-by-256 pixels, and the third test image (Fig. 20) is 128-by-128 pixels. Although the video frame size in MPEG-4 is 360-by-288, we choose square images in order to reduce the bits used for coding the paths in SPECK coding. For comparison, all methods (i.e. SA-DWT, DA-DWT, and SDA-DWT) use the same 5/3 wavelet, and both SA-DWT and SDA-DWT use symmetric extension for transform calculation while DA-DWT uses symmetric extension for transform calculation only on the boundary between the object image and background. For the partition boundaries in the object image, DA-DWT uses the practical values at the extension points. Here, we ignore the bits for side information (i.e. the partition of DADWT and the shape masks of SA-DWT and SDA-DWT) for simplification and focusing on the main problem. The decomposition-level decision in wavelet transform is important and difficult. For a suitable design of decomposition levels, energy clustering effect will make compression efficient. However, excessively many levels can not improve the overall compression efficiency, since the LL subband becomes a very small region that may degrade the overall compression efficiency. The suitable number of wavelet decomposition levels mainly depends on the image size, image content, and the coder/decoder used. In most cases, for a 512-by-512-pixel image, we select 3, 4, or 5 levels empirically. In this paper, 4 decomposition levels were used because the test images are small size. In the followings, PSNR (peak-signal-to-noise ratio) values and the lengths.

(8) IEICE TRANS. INF. & SYST., VOL.E91-D , NO.10 OCTOBER. 2474. 2008. Table 1 The bit numbers of the bit stream of each test object image after SPECK coding. (SDA1 and SDA2 represent SDA-DWT without object partition and with object partition, respectively. SA means SA-DWT and DA is DA-DWT). Table 2 The PSNR results for lossy compression of object image 1. (Object 1 contains 30,535 pixels). of. bit. streams. mance. after. The. by-256-pixel image. image. (object. is based. the. the. and. occupies. object. 30,353. DWT. and. image. the. same,. i.e.. the. pixels. we. orientations. do. whole. pixel). our. calculation. interested. textures,. by. not. partition. visual. object. the. the. object. image.. SDAobject-1. in object. object. is. object. and. compressing. of lines. a 256-. 128-by-128-pixel. in a 256-by-256-pixel evaluated. perfor-. on. image.. 7),. line. two. based. an. object. (Fig.. as. is. (bit/object. many. are. the. used. 2) or. in an. image with. SA-DWT. Since. bpp. number. covered. are. calculation 1 and. the. pixel. first. a suitcase. coding. PSNR. (objects. 3),. on. For is. SPECK. measures.. Fig. 17 The reconstruction images of object 1 under 1-bpp condition: (a) the result of SDA-DWT, (b) the result of SA-DWT. (Object 1 contains 30,353pixels). into. a large. 1 are. small. almost. Fig.. 18. age. in. contains. 4-level. formed. object. rithm,. and. sion. file. on. the. object. test. image and. image. bits). of. that. by. SDA-DWT. is. number. of. the. After. of. of. ues. of. odd. pixels.. ages (i.e.. of. 30,353bits),. of For. and values. much and. are. makes. shown of. the. the. overall. choose. the +45•‹. trans-. the. actual. valin. wavelet. the. trans-. compression. reconstruction under. in Fig. SDA-DWT. beFor. clustered. makes. object-1. SA-DWT,. SDA-. performance. object.. is. size. SA-DWT. to. energy. 17. 1-bpp. for is. imcondition. comparison.. obvious. better. The than. SA-DWT. the. test. SA-DWT and. image. are file. sizes.. of. simulated The. object and gray-level. 2 (Fig.. 18 (a)),. compared object-2. partition: with. (a) object-2 partition.. im-. (Object. 2. 45,012pixels). 3. The. ject. 2 contains. PSNR. results. out. object. 45,012pixels.. for. lossy. partition. and. SDA1. compression. with. object. and. SDA2. of object. partition,. represent. image. SDA-DWT. 2.. (Obwith-. respectively). us bit. 2.15-bpp. 1-D •ehorizontal•f close. mask. (in. The. that. 1, if we. very. with. shape. the. 77.8%. the. the. of. it tells is.. of. on. mask. per-. sizes. under. that. object. Two and. quality. case,. that. which. SDA-DWT. this. of the. Thus,. efficiently.. reconstruction that. step will. successful, very. 5.88dB. textures. values. and. 2 shows. to. than. on. prediction. subband,. very. scheme. line textures. predicted. low-frequency form. the. in the the. In. file. is about. Table. up. shape. object-2. replaced. the. SA-DWT file. better. directional of. than. file.. always. are. 1 shows method,. compression. condition.. the. direction. efficient. algo-. is. compression. each. SA-DWT. is. characteristic. form,. for. its. a compres-. procedures. Table. (b). trans-. SPECK. SDA-DWT. another. SA-DWT.. more. the. 2 and. frame,. per-. the. represent. same. that. compression. SDA-DWT. cause. have. object,. using can. The. image. outperforms. (256•~256bits). visual. by. 1 except. SDA-DWT. SA-DWT. DWT. coded. image.. we. object. the. bit-stream. using. each. on. is. resulted. SA-DWT,. object. of. image. the. of. formed by. SDA-DWT. object. segments,. segment.. Table. forming. Test 256•~256. SDA-DWT. by. their is. PSNR. segmented. from the famous test image Barbara, and Fig. 18 (b) shows the shape mask of the visual object. Two cases are simulated for evaluating SDA-DWT. First, the whole object 2 without partition is used for simulation, and second, object 2 is partitioned into two parts (Fig. 18 (b), the white region and the gray part) for simulation. The partition shown in Fig. 18 (b) is an example for arbitrarily shaped partition which is not the optimal one. Table 1 shows that, for compression-file size, SDA-DWT with object-image partition is the most efficient case among these cases, SDA-DWT without object partition is second place, and SA-DWT is third place. SDADWT with object partition reduces 0.95% bit budget of SADWT's, and SDA-DWT without object partition reduces 0.24% bit-budget. On the other hand, the PSNR values in Table 3 show that SDA-DWT with partition has the best performance. The results show that for a texture rich (especially, non-horizontal or non-vertical edges) image, the performance of lossy compression can be enhanced by suitably partitioning the object image. The proposed method offers much flexibility for partition, since it can handle segments with any shape. The reconstruction object images of.

(9) LIN and SU: SDA-DWT. FOR ARBITRARILY. SHAPED SEGMENTS. IN IMAGE COMPRESSION 2475. Fig. 19 The object-2 reconstruction images under 1.46-bpp condition: (a) the result of SDA-DWT with object partition according to Fig. 18 (b), (b) the result of SA-DWT.. Fig. 20 Test object 3 and its shape mask with partition: (a) object-3 image in 128•~128 frame, (b) object-2 mask with partition. (Object 3 contains. Fig. 21 The reconstruction object images, under 1-bpp condition: (a) the result of SDA-DWT,(b) the result of SA-DWT.. Fig. 22 The reconstruction object image and the partition and direction in DA-DWT: (a) the reconstruction result under 1-bpp condition. (b) mask partition and block directions of partition used in DA-DWT.. 10,000pixels). Table 4 The PSNR results for lossy compression of object image 3. (Object 3 contains 10,000pixels.) SDA-DWT. with. bpp. condition,. the. experiments. Claire,. are. the. Akiyo. or. the. the. three. are. evaluated.. third. methods. images. in. USC. the. area,. object.. Hence,. 5 segments. (Fig.. 20. 3 image. is. objects.. DA-DWT image,. an. squared. 128-by-128-pixel. 3.. DA-DWT 22. can. not. less. compression,. and. DA-DWT. seen. 5.66%. is the Tables. SDA-DWT. pixel) (the. discover. in large. SA-DWT. /object. to. scale.. most. bit. 1 and up. to. also. is. for. image. can. and. 67,726bits). outperforms. the. be. an. for DA-DWT. that,. For. up. 4.. small. loss-. of the. bits,. PSNR outper-. 1-bpp. (bit up. to. compression. to. Conclusions. which. bit-budget. lossless. PSNR under 3.28-bpp condition, and reduces the bit-budget up to 14.0%. The reconstruction results under 1-bpp condition are shown in Figs. 21 and 22 (a). From the experiments of object 3, we understand that SA-DWT can not well exploit the correlation of the directional textures, so it has poor performance for this test object image. For DA-DWT, since its resolution is not high enough (the smallest partition block is 16-by-16), can not represent non-rectangular segment boundaries perfectly, and wastes coding bits on the object background; DA-DWT has the poorest performance for the special object-image.. object. for. under. the. as. many. SDA-DWT. PSNR. reduces. a. viewed. amount. one.. that. shaped. direction. least. consuming. in. into object-. any. 1 shows the. in. textures. processing. into. texture. Table. the. pixels. containing image. 4 show. 4.31dB. condition, base. handle. uses. from. partitioned. can. object. SDA-DWT. comparison, forms. (b)). is. all. SDA-DWT). Although. image the. (a)),. different. designed. object-3. partitions. (Fig. be. but. five. image. originally. 20. 10,000. SDA-DWT.. SDA-DWT is. rectangular. blocks. are. object. for. rectangular,. contains. perforsame.. (synthesized. there. the (b)). the. and. non-. textures. the. (Fig.. DA-DWT, image. and. of. almost. image. Lena,. lack. partition,. object. database). from. directions. are. 1.46-. performed. images. the. object. object-3. image. a 128-by-128-pixel on. or. (SA-DWT, The. also. segmented object. SA-DWT. gray-level. under. We. images. suitable and. SA-DWT,. 19.. these edges. SDA-DWT. For. Fig.. object Since. without. of. and. in. non-vertical. random,. mance. partition. shown. on. and. horizontal. object are. 5.44dB. in. In this paper we propose SDA-DWT, which can be used for arbitrarily shaped image segments, and whose direction of prediction and update are adaptive. From the experimental results, SDA-DWT has superior performance than SA-DWT or DA-DWT does for visual objects with non-horizontal or non-vertical edge textures. SDA-DWT can be applied to any wavelet-based application, although, in this paper, we only give the examples of the intra frame compression of.

(10) IEICE TRANS. INF. & SYST., VOL.E91-D,. NO. 10 OCTOBER. 2008. 2476. the. object-based. The. extra. the. increased. the. side. the. object. video costs. of. compression. in. SDA-DWT. complexity. and. information. of. MPEG-4. compared the. the. standard.. to. storing. and. directions. in. Sheng-Fuu. SA-DWT. are. processing. each. of. segment. Lin. of. respectively, image.. compress that. the. the. work.. For. partitioned. partition In. convenience, still-object. of. order. to. segmentation. we. the. object. achieve. method. is. image image. the. focus. how. while. has. optimal. on. result,. done. the second. this. gineering. from the University. necessary.. trol Engineering Acknowledgment. for. would. their. paper.. like. comments This. Science. to. thank. that. work of. anonymous. significantly. was. Council. the. Taiwan,. improve. supported. by. Grant:. NSC. under. Fuzzy. reviewers. helped. partially. the. Taiwan.. His research. System. Association,. at National. interests. and Chinese. Chiao Tung Univer-. include. fuzzy. Automatic. theory,. 96-2221-E-. Chien-Kun. References. University University. Control. Society.. Su. the B.S.. was born degree. from. in 1962. National. F. Wu,. X.. lifting-based. wavelet. age. vol.16,. Process.,. Wu,. S. Li,. transform no.2,. and. for. H.. image. pp. 416-427,. Li, •gAdaptive. directional. coding,•h. Trans.. Feb.. IEEE. Im-. 2007.. tional. Chiao. wan.. He is also an instructor. Engineering [2]. JPEG sion. 2000. Image. Coding. System,. ISO/IEC. CD15444-1:. 1999. (Ver-. 1.0).. [3]. I. Daubechies lifting. and. steps,•h. pp. 247-269, [4]. and. [6]. [8]. tronic. Image. 2003,. C.-L.. Chang. and. image. for no.5,. and. video,•h. B.. Feb.. S.. W.. W.A.. and. Z.. Lu. 2001),. vol.42,. its applications. Annu.. in. Symp.. Elec-. discrete Trans.. wavelet. Image. Process.,. 2007.. of Signal. Processing,. 2nd. ed.,. Academic. 1999.. Makai, •gShape-adaptive Circuits. DCT. Syst.. Li, •gShape-adaptive visual. object. pp. 725-743,. 2000.. A.. N.. Islam,. image. and. discrete. Process.,. and 14th. IEEE. May. Tour. no.5,. object-based. lifting-based. Signal. Video. for. generic. Technol.,. coding. vol.5,. no.1,. 1995.. coding. Trans.. pp. 1219-1235, [11]. on. IS & T/SPIE. Pearlman,. IEEE. survey J. VLSI. compression,•h. Trans.. shaped. complexity coder,•h. into no.3,. 2002.. USA,. IEEE. pp. 59-62,. vol.10,. vol.4,. Girod, •gDirection-adaptive. image. York,. arbitrarily. [10]. Jan. B.. A Wavelet. New. Li. transforms. sins-interpolation. processing,•h. pp. 1289-1302,. S. Mallat,. of. [9]. signal. and. T. Sikora. wavelet Applications,. 2006.. signal. Press,. and. Chakrabarti, •gA. L. Yaroslavsky, •gFast. vol.16,. Analysis. architectures,•h. Feb.. transform. [7]. C.. transform. pp. 321-339, [5]. Swendens, •gFactoring. Fourier. 1998.. T. Acharya wavelet. W.. J.. Nov. W.A. SPECK. pp. 413-416,. discrete coding,•h. Nagaraj,. with. Circuits. wavelet. IEEE. A.. Said, •gEfficient, embedded. Video. Technol.,. for. Circuits. a set-partitioning Syst.. and. transforms. Trans.. vol.14,. Syst.,. lowblock no.11,. 2004.. Pearlman, •gWavelet algorithm,•h April. 2001.. coding Picture. Coding. of. video Symposium. object. by (PCS. Tung. image. University. Department. sity in Hsinchu, include. He reTaiwan. in 1989 and M.S. degree from the of Southern California in 1992. Cur-. rently, he is a Ph.D. student of the cal and Control Engineering Department Ding,. auto-. and image recogSociety, Chinese. this. ceived. W.. Cham-. National. 009-238.. [1]. in com-. of Illinois,. matic target recognition, scheduling, image processing, nition. Professor Lin is a member of the IEEE Control. authors. degree. paign, in 1988. Since 1988, he has been on the faculty of the Department of Electrical and Con-. texture-. sity, Hsinchu,. The. M.S.. puter science from the University of Maryland in 1985, and the Ph.D. degree in electrical en-. in. a good. the Re-. to. assuming. been. was born in Taiwan,. public of China, in 1954. He received the B.S. and M.S. degrees in mathematics from National Taiwan Normal University in 1976 and 1979,. Taiwan. processing. Electriat Na-. in Hsinchu,. Tai-. of the Electrical. of Chung. Hua Univer-. His research. interests. and computer. vision..

(11)