New Joint Demosaicing and Arbitrary-Ratio Resizing Algorithm for Color Filter Array Based on DCT Approach

Contributed Paper

Original manuscript received December 16, 2009

Kuo-Liang Chung, Senior Member, IEEE, Wei-Jen Yang, Wen-Ming Yan, and Chiou-Shann Fuh, Member, IEEE

Abstract — This paper presents a new joint demosaicing and arbitrary-ratio resizing algorithm for mosaic images. By using the adaptive heterogeneity projection masks and Sobel- and luminance estimation-based (SL-based) masks, more accurate edge information is extracted first. Next, the fully populated green color plane is constructed by using the edge- sensing approach and color difference idea. Instead of interpolating the R and B color planes directly, the green-red color difference plane and green-blue color difference plane are therefore interpolated in order to reduce the estimation error. Next, based on the discrete cosine transform (DCT) technique, the above three constructed planes are resized to the arbitrary sized ones. Finally, the resized red and blue color planes are constructed by using the three resized planes, and then the arbitrary sized full color image is obtained. To the best of our knowledge, this is the first time that such a joint demosaicing and arbitrary-ratio resizing algorithm for mosaic images is presented. Based on twenty-four popular testing mosaic images, the proposed resizing algorithm has better image quality performance when compared with three native algorithms which are the combinations of three well- know demosaicing methods and one existing resizing method.

Thus, the proposed algorithm can be used in consumer electronic products, such as digital cameras and digital camcorders, to provide the quality-efficient arbitrary-ratio resizing effect. ¹

Index Terms — Arbitrary-ratio resizing algorithm, Color difference, Color filter array, DCT, Demosaicing algorithm, Digital cameras.

I. INTRODUCTION

In order to economize the hardware cost, instead of using three CCD/CMOS sensors, most digital cameras use a single sensor array to capture the color information based on Bayer color filter array (CFA) structure [3] which is depicted in Fig. 1.

1 This work is supported by the National Science Council of R. O. C. under contract NSC 98-2221-E-011-102-MY3.

Kuo-Liang Chung and Wei-Jen Yang are with the Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, No. 43, Section 4, Keelung Road, Taipei, Taiwan 10672, R.

O. C. (e-mail: {k.l.chung, a9115001}@mail.ntust.edu.tw).

Wen-Ming Yan and Chiou-Shann Fuh are with the Department of Computer Science and Information Engineering, National Taiwan University, No. 1, Section 4, Roosevelt Road, Taipei, Taiwan 10617, R. O. C. (email:

{ganboon, fuh}@csie.ntu.edu.tw).

In Bayer CFA structure, each pixel in the captured image has only one of the three primary colors and this kind of

image is called the mosaic image. Because the green (G) color plane is the most important factor to determine the luminance of the color image, half of the pixels in Bayer CFA structure are assigned to G color plane; the red (R) and blue (B) color planes share the remaining parts evenly.

Since each pixel in the mosaic image has only one color component, the two missing color components for one color pixel should be recovered as best as possible and such a recovering process is called the demosaicing process which has been studied extensively [1], [4], [5], [7], [9], [10], [12], [13], [15], [17], [18], [19], [22], [24], [25], [26], [27]. Besides the demosaicing issue, how to resize mosaic images is another important research issue. The terms “resize” and “zoom” are used exchangeably. Several resizing algorithms for mosaic images have been developed [2], [6], [8], [16], [20], [21], [28]

Unfortunately, all of them only focus on the quad-zooming process. The motivation of this research is to develop a new joint demosaicing and arbitrary-ratio resizing algorithm for mosaic images.

In this paper, a new joint demosaicing and arbitrary-ratio resizing algorithm for mosaic images is presented. First, more accurate edge information is extracted from the mosaic image by using the adaptive heterogeneity projection masks and SL- based masks [7]. Next, the fully populated G color plane is constructed by using the edge-sensing approach and color difference idea. Instead of interpolating the R and B color planes directly, the green-red color difference plane and green-blue color difference plane are therefore interpolated in order to reduce the estimation error. Then, based on the composite length DCT technique [23], the three constructed planes can be resized to obtain arbitrary-ratio sized ones.

Finally, the resized red and blue color planes are gotten by using the two resized color difference planes and the resized green color plane, and then the arbitrary-ratio resized full color image is followed. The flowchart of the proposed

New Joint Demosaicing and Arbitrary-Ratio Resizing Algorithm for Color Filter Array Based on DCT Approach

B B

R

B B

G G

R R

G G

R G G

R

G B G B

G

R G

G G G B

B G B G

G R G R G R R

R G G G R

R R

R G G G R

i-3 i-2 i-1 i i+1 i+2 i+3

j-3 j-2 j-1 j j+1 j+2 j+3

Fig. 1. The Bayer CFA structure.

resizing algorithm is illustrated in Fig. 2. This is the first time that such a joint demosaicing and arbitrary-ratio resizing algorithm for mosaic images is presented. Based on twenty-four popular testing mosaic images, the proposed algorithm has better image quality performance in terms of two objective color image quality measures, the color peak signal-to-noise ratio (CPSNR) and the S-CIELAB 'E_ab , and one subjective color image quality measure, the color artifacts, when compared with three native algorithms which are the combinations of three well-know demosaicing methods [5], [19], [24] and one existing resizing method [23].

The remainder of this paper is organized as follows. In Section II, the adaptive heterogeneity projection masks and SL-based masks, which are used to extract more accurate edge information from mosaic images, are introduced. In Section III, based on the extracted edge information and the composite length DCT, the proposed arbitrary-ratio resizing algorithm for mosaic images is presented. In Section IV, some experimental results are demonstrated to show the quality advantage of the proposed algorithm. Finally, some concluding remarks are addressed in Section V.

II. EXTRACTING MORE ACCURATEEDGEINFORMATION FROM MOSAICIMAGES

In this section, how to use the adaptive heterogeneity projection masks and the SL-based masks [7] to extract more accurate edge information from mosaic images is introduced.

For exposition, the R, G, and B color pixels at position

( , ) i j

in the mosaic image I_mo are denoted by I_mo^r ( , ),i j ( , ),

g

Imo i j and

I

_mo^b

( , ), i j

respectively.

A. Adaptive heterogeneity projection

Based on the concept of adaptive heterogeneity projection [7], three possible heterogeneity projection masks with different sizes adopted in this paper are shown in Table I where the terms N and

M

_hp

( ) N

denote the mask size and the corresponding heterogeneity projection mask, respectively. Given a mosaic image

I

_mo

,

the horizontal heterogeneity projection map HP_{H map} and the vertical heterogeneity projection map

HP

_{V map} can be generated by

( )

( )

HP I M NT

H map mo hp

HP I M N

V map mo hp

_ … _



_ … _



(1)

where the symbol “

…

” denotes the 1-D convolution operator;

_ ˜ _ denotes the absolute value operator; the operator “T ” denotes the transpose operator. In order to extract more accurate horizontal and vertical edge information and reduce the computation time, the two proper mask sizes N_H( , )i j and N i j_V( , ) for each pixel at position

( , ) i j

should be determined. For simplicity, we only describe the determination of N_H( , ).i j

The horizontal spectral-spatial correlation (SSC) [24] map is utilized to determine the proper horizontal mask size for each pixel. For the horizontal SSC map, the horizontal SSC value S_H( , )i j at position

( , ) i j

can be calculated by using Eq. (2).

After generating the horizontal SSC map, the proper horizontal mask sizes N_H( , )i j can be determined thereafter.

The depiction of the determination for N_H( , )i j is illustrated in Fig. 3 and the procedure of the determination for N_H( , )i j consists of the following three steps:

x Step 1: Initially, we set the left boundary x_l j 2, the right boundary

x

_r

 j 2,

the mask size

Mosaic image

Determine direction of variation by adaptive heterogeneity projection

Extract gradient information by using SL- based masks

Edge-sensing interpolation estimation for

mosaic G plane Stage 1: Interpolating the mosaic G plane

Edge-sensing interpolation estimation for mosaic G-B and G-R color difference planes

Resizing fully populated G-B and G-R color difference planes Resizing fully

populated G plane Resizd G plane

Resizd B and R planes

Resized full color image

+

Stage 2: Interpolating the mosaic G-R and G-B

color difference plane

Stage 3: Resizing the three planes

Fig. 2. The flowchart of the proposed arbitrary-ratio resizing algorithm.

TABLEI.

FOUR POSSIBLE HETEROGENEITY PROJECTION MASKS.

N Mhp^{( )}N

5 >^{1 2 0 2 1}@

7 >¹ ^{4 5 0} ^{5 4} ¹@

9 >^{1 6 14 14 0 14 14 6 1}@

if the current pixel is G and iodd.

if the current pixel is G and ieven. (2) if the current pixel is .R

( , ) ( , 1) ( , ) ( , 1) ( , )

( , ) ( , 1) ( , ) ( , 1)

g r

mo mo

g b

mo mo

H r g

mo mo

b g

mo mo

I i j I i j I i j I i j S i j

I i j I i j I i j I i j

­  

°°  

®°

 

°°

 

°¯ if the current pixel is .B

( ) 5,

N

H

i j 

and the maximum mask size

N

_max

9.

x Step 2: If the condition Max('  'S_l S_r)T_h where

( ) ( 1) ;

l H l H l

S S i x S i x

' _     _ 'S_r _S_H(i x _r)S_H(i x _r _1) ;

h

8

T

holds, the mask size

N

_H

( i j  )

is output as the proper horizontal mask size. Otherwise, go to Step 3.

x Step 3: Update

N

_H

( i j  ), x

_l

,

and x_r by performing

( ) ( ) 2,

H H

N i j  N i j   x

_l

x

_l

 1,

and

r r

1,

x x 

respectively. If

N

_H

( i j  ) N

_max

, ( )

H max

N i j  N

is output as the proper mask size and stop the procedure. Otherwise, go to Step 2.

After generating the two heterogeneity projection maps

H map

HP _ and

HP

_{V map}

,

the horizontal and vertical heterogeneity projection values at position

( , ) i j

are denoted by HP i j_H( , ) and

HP i j

_V

( , ),

respectively. Further, the tuned horizontal and vertical heterogeneity projection values

can be computed by ⁴

( , ) 4 ( , )

H k k H

HP i jc

¦

_G HP i jk ^and

4

( , ) 4 ( , ),

V k k V

HP i jc

¦

_G HP ik j respectively, where

G

_k 2 if

k 0; G

_k

1,

otherwise.

B. SL-based masks for mosaic images

In order to make the Sobel operator workable on mosaic images to extract more accurate gradient information, the luminance estimation technique [1] is embedded into the Sobel operator [11]. The detailed derivations of embedding the luminance estimation technique into the Sobel operator are described in [7]. The four normalized SL-based masks are illustrated in Fig. 4.

By running the above four SL-based masks on the 5 5u mosaic subimage centered at position

( , ), i j

the horizontal gradient response

' I

_dm^H

( , ), i j

the vertical gradient response

( , ),

V

I

dm

i j

'

the S 4-diagonal gradient response

' I

_dm^S4

( , ), i j

and the S 4 -diagonal gradient response

' I

_dm^S4

( , ) i j

can be obtained easily.

III. THE PROPOSEDNEWJOINT DEMOSAICING AND

ARBITRARY-RATIORESIZING ALGORITHM FOR MOSAIC

IMAGES

Our proposed algorithm consists of the following three stages: (1) interpolating the mosaic G plane to construct the fully populated G plane by using the edge-sensing interpolation estimation; (2) interpolating the mosaic G-R and G-B color difference planes to construct the fully populated G-R and G-B color difference planes, respectively; (3) resizing the three constructed planes mentioned above to obtain the arbitrary-ratio sized ones, and then based on the three resized planes, recovering the resized R and B planes to obtain arbitrary-ratio resized full color image.

A. Stage 1: Interpolating the mosaic G plane

In this subsection, the interpolation for the mosaic G plane ( , )

g

Imo i j to construct the fully populated G plane I_dm^g ( , )i j by using the edge-sensing approach and color difference idea is presented. For exposition, let us take the central pixel at position ( , )i j in Fig. 1 as the representative to explain how to estimate the G color value I_dm^g ( , )i j from its four neighboring pixels with movement

^

( , ) ( , ) ( 1, ), ( , 1) .

`

g x y x y i j i j

: r r First, according to the

tuned horizontal heterogeneity projection value HP i j_Hc( , ) and the tuned vertical heterogeneity projection value

( , )

HP i jVc of the current pixel, three cases, namely the horizontal variation as shown in Fig. 5(a), the vertical variation as shown in Fig. 5(b), and the other variations as shown in Fig. 5(c), are considered in the interpolation estimation phase for I_mo^g ( , )i j . The arrows in Fig. 5 denote the relevant data dependence.

Further, in order to estimate I_dm^g ( , )i j more accurately, four proper weights in terms of the gradient magnitude are assigned to the corresponding four pixels in the interpolation estimation phase. Given a pixel at position ( , )i j , based on the horizontal and vertical gradient magnitudes, its horizontal and vertical weights can be determined by

1

( , , ) 1 [1 1 ^H( , )]

g k k dm

w H x y 

¦

_G 'I x y k ^and

1

( , , ) 1 [1 1 ^V ( , )],

g k k dm

w V x y 

¦

_G 'I x k y respectively, where

k

3

G

if

k 1; G

_k 1, otherwise. Considering the neighboring pixel located at position (i1, ),j if the vertical gradient magnitude is large, i.e. there is a horizontal edge passing through it, based on the color difference assumption [15], [24], it reveals that the G component of this pixel makes less contribution to estimate that of the current pixel;

otherwise, it reveals that the G component of this pixel makes more contribution to estimate that of the current pixel.

According to the above analysis, the vertical weight

Fig. 3. The depiction of the determination for the proper horizontal mask size N_H( , )i j

(a) (b) (c) (d)

Fig. 4. The four normalized SL-based masks. (a) The horizontal SL- based mask. (b) The vertical SL-based mask. (c) The S 4-diagonal SL- based mask. (d) The S 4-diagonal SL-based mask.

( , 1, ) 1

1 ( , ) 3 ( 1, ) ( 2, ).

g V V V

dm dm dm

w V i j

I i j I i j I i j

  '  '   ' 

is selected for the pixel at position (i1, ).j By the same argument, the weights of the other three neighbors are selected by

w V i

_g

( ,  1, ), j w H i j

_g

( , ,  1),

and

w H i j

_g

( , ,  1),

respectively. Consequently, the value of I_dm^g ( , )i j can be estimated by

( , , ) ( , ) ( , , )

( , ) ( , )

( , , ) ( , , )

w d x y D x y

d x y g g gb

g b

I i j I i j

dm mo d x y gwg d x y

[ [

¦ 

 ¦ 

if ( , ) ( , ) 1

if ( , ) ( , ) 2

Ohterwise

1 2

V H

H V

HP i j HP i j HP i j HP i j g

[ D

[ [ D

[ [

­ c  c

°° c  c

®°

°¯ ‰

(3) where [1

^

( ,V ir1, )j

`

^and [2

^

( , ,H i jr1) ;

`

^for

1 1 1 1

( , ,d x y)[,

1, 1 1 1 1 1

{ 1}

( ) ( , ) 1 ( , );

2

g b

gb mo mo

k

D x y I x y I x k y

 r



¦

 ^for

2 2 2 2

(d x y, , )[ ,

2, 2 2 2 2 2

{ 1}

( ) ( , ) 1 ( , );

2

g b

gb mo mo

k

D x y I x y I x y k

 r



¦

 ^the

parameter

D

is set to

D

0.55 empirically.

Finally, a new proposed refinement approach, which combines the concept of the local color ratios [17] and the extracted more accurate edge information, is addressed to refine the fully populated G plane. For the current pixel at position ( , ),i j its G value I_dm^g ( , )i j can be refined by the following rule:

( , , ) ( , , )

( , , ) ( , , )

( , , ) ( , )

( , ) ( ( , ) )

( , , )

g

g

d x y g gb

d x y

g b

dm mo

d x y g d x y

w d x y R x y

I i j I i j

w d x y

[

[

E E G

G

c

c



  

¦

¦

' if ' ( , ) ' ( , ) 1

' ' if ' ( , ) ' ( , ) 2

' ' Ohterwise

1 2

V H

H V

HP i j HP i j HP i j HP i j g

[ D

[ [ D

[ [

­ 

°°

® 

° ‰

°¯

(4) where [1c

^

( ,V i k j k , ) | {0, 2} ;r

`

^ `

2 ( , ,H i j k) |k {0, 2} ;

[c   r ( , ) ^{( , )} ;

( , )

g dm

gb b

mo

I x y R x y

I x y E E





the parameter

E

is set to

E

256 empirically.

B. Stage 2: Interpolating the mosaic G-R and G-B color difference planes

Instead of interpolating the R and B color planes directly, we interpolate the G-R and G-B color difference planes because the color difference plane is much smoother than the original color plane and it would alleviate the amplification of the estimation error in the later resizing stage. Since the interpolation for G-R color difference plane is the same as that for G-B color difference plane, we only present it for G-R color difference plane consisting of three steps.

In Step 1, according to the mosaic image I_mo^g (see Fig. 1) and the fully populated G plane

I

_dm^g , the mosaic G-R plane can be obtained by D_gr( ,i j_{r r}) I_dm^g ( ,i j_{r r})I_mo^r ( ,i j_{r r}) ^where

( , ) {(i j_{r r}  ir2m1,jr2n1)}. After performing Step 1, Fig. 6 illustrates the pattern of the obtained mosaic G-R color difference plane for the positions depicted in in gray cells. The G-R color difference plane interpolation estimation for the other positions consists of two steps: Step 2: interpolating the G-R color difference values of the pixels at positions

{( i r 2 , m j r 2 )} n

in Fig. 6; Step 3: interpolating the G-R color difference values of the pixels at positions

{( i r 2 , m j r 2 n  1)}

and

{( i r 2 m  1, j r 2 )}. n

For simplicity, the central pixel at position ( , )i j in Fig. 6 is taken as the representative to explain the G-R color difference plane interpolation performed in Step 2. The G-R color difference value

D

_gr

( , ) i j

can be estimated from its four neighboring pixels, which have been interpolated in last

subsection, with movement

^

( , ) ( , ) ( 1, 1) .

`

r x y x y i j

: r r In order to estimate

( , )

D

gr

i j

more accurately, the gradient magnitudes of four diagonal variations are considered to determine the proper four weights. Given a pixel at position ( , )x y , its

4 -diagonal S

 and S 4 -diagonal weights can be determine

by ^{1 4}

( 4, , ) 1 [1 1 ( , )]

gr k k dm

w S x y 

¦

_G'I^S x k y k  ^and

1 4

( 4, , ) 1 [1 1 ( , )]

gr k k dm

w S x y 

¦

_G'I^S x k y k  respectively, where

k

3

G

if

k 1; G

_k 1, otherwise. Thus, the four weights of the four diagonal neighbors of the current pixel at position ( , )i j are denoted by w_gr(S 4,i1,j1),

( 4, 1, 1),

wgr S i j w_gr(S 4,i1,j1), and G

G G

G B

G G

G G B

(a) (b) (c)

Fig. 5. Data dependence of our proposed interpolation estimation for

( , )

g

Imo i j . (a) Horizontal variation (vertical edge). (b) Vertical variation (horizontal edge). (c) The other variations.

Fig. 6. The pattern of the mosaic G-R color difference plane.

( 4 , 1, 1),

wgr S i j respectively. Based on the four weights, the G-R color difference value at position ( , )i j can be estimated by

( , , )

( , , )

( , , ) ( , ) ( , )

( , , )

gr

gr

gr gr

d x y gr

d x y gr

w d x y D x y D i j

w d x y

[

[





¦

¦

(5) where [gr ^⁽ S^{4, 1,}i j^{1),( 4, 1,}S i j^{1),( 4, 1,}S i j^1),(S^{4, 1,}i j^{1) .}` After performing Step 2, the current pattern of the G-R color difference plane is illustrated in Fig. 7(a). For easy exposition, the central pixel at position ( ,i jc c) in Fig. 7(b), which is obtained by shifting Fig. 7(a) one pixel down, is taken as the representative to explain the G-R color difference plane interpolation in Step 2.

Referring to Fig. 7(b), it is not hard to find that the pattern of the G- R color difference plane at present is the same as that of the G plane in the mosaic image as shown in Fig. 1. Therefore, the interpolation estimation approach described in last subsection can be directly used to estimate the G-R color difference value at position ( ', ').i j Consequently, the G-R color difference value of the current pixel can be estimated by

( , , ) '

( , , ) '

( , , ) ( , ) ( , )

( , , )

gr

gr

gr gr

d x y gr

d x y gr

w d x y D x y D i j

w d x y

[

[





¦

¦

if ( , ) ( , ) 1

if ( , ) ( , ) 2

Ohterwise

1 2

V H

H V

HP i j HP i j HP i j HP i j g

[ D

[ [ D

[ [

­ c  c

°°

c ® c  c

° ‰

°¯

(6) where [1 ^( ,V ir1, )j` and [2 ^( , ,H i jr1) ;`

1

( , , ) 1 [1 1 ^H( , )]

gr k k dm

w H x y 

¦

_G 'I x y k ^and

1

( , , ) 1 [1 1 ^V( , )]

gr k k dm

w V x y 

¦

_G'I x k y ^where

G

_k

3

if

k 1;

k 1,

G

otherwise; the parameter

D

is set to

D

0.55 empirically.

After constructing the fully populated G plane, G-R color difference plane, and G-B color difference plane, in next subsection, the three constructed planes will be resized to the arbitrary-ratio sized ones by using the DCT approach, and then the arbitrary-ratio resized full color image is followed.

C. Stage 3: Resizing the fully populated G plane, G-R color difference plane, and G-B color difference plane

Based on the composite length DCT [23], this subsection presents the arbitrary-ratio resizing stage for constructing the fully populated G plane

I

_dm^g

,

G-R color difference plane

D

_gr, and G- B color difference plane

D

_gb. Since the resizing stage for

I

_dm^g is the same as that for

D

_gr and

D

_gb

,

we only present it for

I

_dm^g

.

Let

DCT B (

^{( )a}

)

and

IDCT B (

^{( )a}

)

be the DCT and inverse DCT on the a au block

B

^{( )a}

.

For the fully populated G plane

g

,

I

dm with size

M u N ,

we first divide it into a set of the image blocks, each with size

8 8, u

and the obtained image block set is denoted by ⁽⁸⁾

{ . | 0 1, 0 1}

8 8

g

dm m n

I

M N

B m n

) d d  d d  where

(8 )

.

B

m n denotes the ( ) 8

mN n -th block in g

.

Idm

)

Then, the DCT is performed on each 8 8u image block to obtain the set of transformed blocks, ⁽⁸⁾

ˆ .

ˆ { | 0 1, 0 1}.

8 8

g

dm m n

I

M N

B m n

) d d  d d  If

we want to resize the MuN G plane

I

_dm^g to the one with size qpMuqpN the resizing ratio is said to ^q _.

p According to the resizing ratio ^q _,

p first

p

² blocks in ˆ

g

Idm

) are collected to be an active unit which are surrounded by dashed lines in Fig. 8. In order to achieve resizing ratio ^q _,

p the

p

² blocks (a) (b)

Fig. 7. Two patterns of the G-R color difference plane. (a) The pattern of the G-R color difference plane after performing Step 1. (b) The pattern shifting (a) one pixel down.

…

…

… p blocks

…

…

…

… 8

8

…

…

…

(8)

ˆ0,0

B Bˆ⁽⁸⁾0,1 ˆ⁽⁸⁾0,1

Bp (8) 0,8

ˆN

B

(8) 1,0

Bˆ Bˆ1,1⁽⁸⁾ ˆ1,⁽⁸⁾1

Bp

(8) 1,1

ˆ

p p

B 

(8) 8,1

ˆ

M p

B _

…

(8) 8,1

ˆ BM (8)

8,0

ˆ

BM ˆ⁽⁸⁾_{8, 8}

M N

B

(8) 1,8

ˆN

B

(8) 1,8

ˆp N

B (8)

ˆ 1,1

Bp (8)

ˆ 1,0

Bp

g

Idm

Fig. 8. An example of the active unit in ˆ g .

Idm

)

ˆAct

< ˆ

cAct

< Iup

(8)pqz

p·(8+z)

p·(8+z)

… … … … …

…

…

…

…

(8 ) p qz ( (8))

0,0

ˆp q z

R ^Rˆ_0,1⁽p q^(8z⁾⁾Rˆ⁽_0,2p q^(8z⁾⁾

( (8)) 1,0 ˆp q z R ^Rˆ_1,1^{(p q(8z))}Rˆ_1,2⁽p q^(8z⁾⁾

( (8))

3,0

ˆ^{p q z}

R ^Rˆ⁽3,1^{p q(8z))}Rˆ⁽3,2p q^(8z⁾⁾

( (8)) 1,0 ˆp q z Rq ^ ˆ⁽p q1,1⁽⁸z⁾⁾

Rq_ ^ ˆ⁽p q_1,2⁽⁸z⁾⁾

Rq_ ^

( (8 )) 0,1

ˆp q z

Rq_^

( (8))

1,1

ˆp q z

Rq^

( (8)) 3,1

ˆ^{p q}_q ^z R^

( (8))

1,1

ˆp q z qq

R^

…

…

…

…

…

8+z

8+z

p·(8+z)

p·(8+z) ……

(8 ) 0,0

Bc^z 0,1^{(8 )}

Bc^z 0,⁽⁸1⁾ z

Bcp^_

(8 ) 1,0

Bc^z B_1,1c^(8z) 1,⁽⁸1⁾ z

Bcp^_

(8 ) 1, 1

z p p

Bc_^_

(8 ) 1,1

z

Bpc_^

(8 ) 1,0

z

Bpc_^

p blocks

pblocks …

…

…

…

…

…

…

8 8

(8) 0,0

Bˆ ˆ⁽⁸⁾_0,1 Bp_ (8)

1,0

Bˆ Bˆ_1,1⁽⁸⁾ ˆ_1,⁽⁸⁾₁ Bp_

(8) 1, 1

ˆp p

B_{ }

(8) 1,1

ˆ Bp_ (8)

1,0

ˆ Bp_

(8)

ˆ0,1

B

B

8 1,0

ˆ

8 0…0

0…0 0…

0 0…0

0…0

0…0

0…0 0…

0 0…0

0…0

0…0

0…0 0…

0 0…0

0…0

0…0

0…0 0…0

0…0

0…0

0…0

0…0 0…0

0…0

0…0

0…0

0…0 0…0

0…0

0…0

0…0

0…0 0…0

0…0

0…0

0…0

0…0 0…0

0…0

0…0

0…0

0…0 0…0

0…0

0…0

…

…

…

…

…

…

…

8+z

8+z

p·(8+z)

p·(8+z)

(8)

ˆ0,0

B

(8)

ˆ1,0

B Bˆ1,1⁽⁸⁾ (8) 0,1

Bˆ ˆ_0,⁽⁸⁾₁ Bp_

(8) 1, 1

ˆ Bp_

(8)

ˆ1,1

Bp_ (8)

ˆ 1,0

Bp_ ˆ⁽⁸⁾1, 1

p p

B_{ }

q blocks

qblocks

…

…

…

…

… …

8 8

…

…

…

(8)

ˆ0,0

R Rˆ⁽⁸⁾0,1 Rˆ0,2⁽⁸⁾ (8) 1,0

Rˆ Rˆ1,1⁽⁸⁾ Rˆ1,2⁽⁸⁾

(8)

ˆ3,0

R Rˆ3,1⁽⁸⁾ Rˆ3,2⁽⁸⁾

(8) 1,0

ˆ Rq_ ˆ⁽⁸⁾_1,1

Rq_ ˆ⁽⁸⁾_1,2 Rq_

(8) 0,1

ˆq

R _

(8) 1, 1

ˆq

R _

(8) 3,1

ˆq

R _

(8) 1, 1

ˆ

q q

R_{ }

ˆRe

ˆ <

cRe

<

Step 1 Step 2

Step 3

Step 4

Fig. 9. The depiction of the q p-fold resizing procedure.

In each active unit should be increased or decreased to

q

²

blocks.

For the active unit

\ ˆ

_Act

{ B ˆ

_{m n}^{(8 )}_.

| 0 d m n , d p  1},

the q

p-fold resizing procedure consists of the following four steps and the resizing procedure is depicted in Fig. 9.

x Step 1: Each 8 8u DCT coefficient block

(8 )

ˆ

.

B

m n in the active unit is expanded to a

(8  z ) (8 u  z )

block

(8 )

ˆ

, ^z

B

m n

c

^ by the following zero padding rule:

(8) (8 )

. ,

ˆ ( , ) if 0 x,y<8 ˆ ( , )

otherwise 0

z m n

m n

B x y Bc ^ x y ®­° d

°¯

(7)

, {0,1, , 1};

x y z

     m n ,  {0,1,  , p  1}

where

z

denotes the smallest nonnegative integer satisfying the condition:

p (8  z ) Cq ,

^Ct^8. For example, assume 4

3 q

p then the smallest

z

is 4 due to

3(8 4)  u 9 4.

Consequently, we have the set of zero

padded DCT coefficient blocks

(8 )

ˆ ,

ˆ_Act {B_{m n}^z | 0 m n, p 1}.

\c c ^ d d 

x Step 2: For each

ˆ

⁽⁸_, ^z)

B

m n

c

^ in

\ ˆ

_Act

c ,

the

(8  z ) (8 u  z )

IDCT is performed on it to obtain the upsized image:

(8 ) (8 )

, ^z

( ˆ

, ^z

)

m n m n

B c

^

IDCT B c

^ (8) After performing the

(8  z ) (8 u  z )

-sample IDCT’s on all the zero padded DCT coefficient blocks in

\ ˆ

_Act

c ,

the upsized subimage

I

_up

( 

_{0dm n p,d 1}

B

_{m n}

c

⁽⁸_,^z)

)

is constructed.

x Step 3: Then, the upsized subimage

I

_up is divided into

q

2 blocks, each block with size q (8 z) q (8 z),

p  u p 

and the set of the resampled image blocks is denoted by

( (8 ))

Re { , | 0 , 1}.

q z p

Rm n m n p

\



d d  Next, the DCT is performed on each

( (8 ))

,

q z

p

Rm n



to construct the resampled DCT coefficient block

( (8 ))

ˆ ,

q z

p

Rm n



by

((8 )) ((8 ))

, ,

ˆ ( ),

q z q z

p p

m n m n

R DCT R

 

0dm n, d q 1 (9)

We thus have the set of resampled DCT coefficient blocks

((8 ))

Re ˆ ,

ˆ { | 0 , 1}.

q z

p

Rm n m n p

\



d d 

x Step 4: Finally, the high-frequency DCT coefficients of each block in

\ ˆ

_Re are truncated by the following rule:

( (8 )) (8)

, ,

ˆ ( , ) ˆ ( , )

q z p

m n m n

R x y R x y



, {0,1, , 7};

 x y    m n ,  {0,1,  , q  1}

(10) where

ˆ

⁽⁸⁾_,

R

m n denotes the left-upper 8 8u subblock of

( (8 ))

ˆ , ;

q z p

Rm n

 ( (8 ))

ˆ , ( , )

q z

p

Rm n x y



and

(8 )

ˆ

_{m n},

( , )

R x y

denote the DCT

coefficients of the pixels at position

( , ) x y

in

( (8 ))

ˆ ,

q z

p

Rm n



and

R ˆ

_{m n}⁽⁸⁾_,

,

respectively. Consequently, the resized active unit ⁽⁸⁾

Re ˆ ,

ˆ {R_{m n}| 0 m n, q 1}

\^{c c} d d  is constructed.

After performing the above resizing procedure on all the active units in

I

_dm^g

,

a set of 8 8u DCT coefficient blocks

( 8 )

ˆ .

ˆ { | 0 1, 0 1}.

8 8

g

dm m n

Z

q M q N

p p

B m n

) d d  d d  can be obtained.

Consequently, the ^q _M ^q _N

p u p sized G plane

Z

_dm^g can be obtained by performing the IDCT on each 8 8u DCT coefficient blocks in ˆ g .

Zdm

) By the same argument, the qpMuqpN sized G-R and G-B color difference planes, which are denoted by

ZD

_gr and

ZD

_gr

,

respectively, can be

Fig. 10. The twenty-four testing images from Kodak PhotoCD.

TABLE II.

AVERAGECPSNR COMPARISON FOR THE FOUR CONCERNED ALGORITHMS.

Resizing ratio q p

2 8 5 4 3 8 7 Average

A1 28.4051 30.2171 32.1575 34.1260 31.2264 A2 28.4107 30.1089 31.8667 33.6307 31.0043 A3 28.6069 30.5344 32.5719 34.6149 31.5820 Ours 29.1714 31.3134 33.6423 35.8527 32.4950

TABLE III.

AVERAGES-CIELAB'E_ab COMPARISON FOR THE FOUR CONCERNED ALGORITHMS.

Resizing ratio q p

2 8 5 4 3 8 7 Average

A1 3.52927 3.05053 2.62441 2.17526 2.84487 A2 3.50429 3.09378 2.74100 2.35818 2.92431 A3 3.41551 2.92146 2.50336 2.07863 2.72974 Ours 3.17182 2.70866 2.32772 2.00323 2.55286