植基於計算智能之主動式佇列管理系統

行政院國家科學委員會專題研究計畫 成果報告

植基於計算智能之主動式佇列管理系統

計畫類別： 個別型計畫

計畫編號： NSC94-2213-E-151-013-

執行期間： 94 年 08 月 01 日至 95 年 07 月 31 日

執行單位： 國立高雄應用科技大學電子工程系

計畫主持人： 謝欽旭

共同主持人： 潘正祥，廖斌毅

報告類型： 精簡報告

報告附件： 出席國際會議研究心得報告及發表論文

處理方式： 本計畫涉及專利或其他智慧財產權，1 年後可公開查詢

中 華 民 國 95 年 10 月 31 日

Quantization

Chin-Shiuh Shieh1, 2,*_{, Chao-Chin Chang}2_{, Wei Kuang Lai}1_{, and Shu-Chuan Chu}3 1 _{Department of Computer Science and Engineering, National Sun Yat-sen University}

Kaohsiung 804, Taiwan, ROC csshieh@cc.kuas.edu.tw

2 _{Department of Electronic Engineering, National Kaohsiung University of Applied Sciences}

Kaohsiung 807, Taiwan, ROC

3 _{Department of Information Management, Cheng Shiu University}

Kaohsiung County 833, Taiwan, ROC scchu@csu.edu.tw

Received 20 March 2006; Revised 18 May 2006 ; Accepted 10 June 2006

Abstract. Digital watermarking had been well recognized as an effective measure for the copyright protection of multimedia data. Numerous schemes in the domain of vector quantization had been proposed to achieve the desired goal. A new scheme, also in the domain of vector quantization, is developed and presented in this article. The proposed approach use genetic algorithms to reassign the indices of code words. As a result, em-bedded information will be diffused more evenly across the image to be protected, and therefore possible se-curity leakage can be avoided. Experimental results reveal that the proposed scheme is free from the potential limitations in previous approaches, while maintaining the robustness against various kinds of attacks. Keywords: digital watermarking, genetic algorithm, index assignment, information leakage

1 Introduction

Data secrecy had become an important issue as communication networks getting commoditization and widely spread, especially with the blooming of the Internet. Among others technologies, watermarking technology has received considerable attention in recent years for their theoretical and practical significance. Aimed at copyright protection, arbitration, and authentication, watermarking is the process of embedding extra information into a media clip. There have been a vast number of established methods [1]-[3]. However, it is still far from trivial to make the embedded watermark robust. Various criteria are addressed in judging a watermarking technique, such as perceptibility, security, embedding rate, whether original clip is required for extraction, robustness to common signal processing or intentional attacks, and so on.

According to the hiding domain, digital watermarking can be roughly classified into spatial-domain based methods [4] and transform-domain based methods. Transform-domain approaches have been intensively studied, such as discrete cosine transform (DCT) [5], discrete Fourier transform (DFT) [6], discrete wavelet transform (DWT) [7], and the Chirp-Z transform [8].

To reduce the space requirement for storage and the bandwidth requirement for communication, wide variety of compression techniques had been developed [9]. In applications regarding image, audio, and video, human sensory system is sophisticated enough to filter out limited data loose during the process of encoding and decod-ing. In this kind of applications, vector quantization (VQ) [10]-[11] had received considerable attention for its high compression rate and its essential role in various compression applications. VQ and its descendents try to maintain high compression rate while retaining essential information carried in media clips. As an extension to scalar quantization, vector quantization works on vectors of raw data. A vector can be fixed number of consecu-tive samples of audio data or a small block of image/video data, for example, the gray-level values of a 4 ×4

pixel image block forms a 16-dimentional vector. Fig. 1 gives an illustration of the operation of vector quantiza-tion compression.

電腦學刊 第十七卷第二期（Journal of Computers, Vol.17, No.2, July 2006）

20

In the sender (compression) end, the codeword search process looks for a “nearest” codeword, C , from the i

codebook for the given input vector X . Euclidean distance was used in the search process to measure the dis-t

tance between two vectors as indicated in Equation (1). 1 ,..., 0 ), , ( min arg = − = D X C j N i j t j . (1) where

∑

= − = K k k k V V V V D 1 2 2 1 2 1, ) ( )

( is the Euclidean distance between two K-dimensional vectors V and 1 V , and 2 k

V is the k-th component of vector V.

The index of selected codeword is then transmitted to the receiver end. With the same codebook, the decom-pression process can easily reconstruct vector '

t

X by simple table look-up. Of course, there will be distortion introduced by the compression-decompression process, since Xt' is only an approximated version of the original

t

X . If we work on 8-bit gray-level image, using a block size of 4 ×4 pixel and a codebook of 256 code words,

then the compression ratio is up to 16 256 log 8 4 4 2 = × × .

Fig. 1. A block diagram for vector quantization

The codebook plays an essential role in vector quantization. The codebook size, i.e. the number of code words in a codebook, is a tradeoff between compression quality and compression rate. The code words in the codebook decide the resultant compression distortion. A dedicated procedure is required for the generation of appropriate codebook. One may regards the problem of codebook generation as a problem of finding N most representative vectors, Ci,i=0,...,N−1, from M given training vectors, Xj,j=0,...,M−1. The located Ci’s serve as code

words used to partition the M given data into N mutual exclusive clusters, Si,i=0,...,N−1. A given vector Xj

is considered to belong to cluster Si if Ci is the nearest codeword to Xj. Among other alternatives, LBG

algo-rithm [12] is widely used in various applications. The following pseudo code illustrates the operation of LBG algorithm:

Pseudo Code for LBG Algorithm

Randomly pick up N vectors from M training vectors as initial code words DO

Conduct clustering using the current code words Calculate new centers C_i for each cluster S_i

UNTIL Improvement falls bellow threshold ε

Vector quantization had been distinguished for its high compression rate in lossy data compression applica-tions. To be of practical significance, a digital watermarking technique should take into account the effect of vector quantization compression. Recently, several researchers pay much attention to explore the hiding scheme based on vector quantization [13]-[15]. In Section 2, we will review these representative works, and point out their relative merits and potential limitations. We will discuss the details of using genetic algorithms for index assignment in Section 3. Such an approach will reduce the possibility of information leakage. An improved scheme based on the results of Sections 3 is presented in Section 4. Experimental results are given in Section 5, followed by some conclusions in Section 6.

Codeword Search Code words C0,…,CN-1 Code words C0,…,CN-1 Table Look Up Xt _X’t Index i

2 Related Works

Huang et al. [13] had developed a robust digital watermarking algorithm in the domain of vector quantization, from which various variants were developed. To facilitate our discussion, a brief review of their work is given here. In [13], the imageXto be protected is an 8-bit gray-level image of sizeM×N, and the watermark W is a binary image of size MW ×NW . The X is discomposed into non-overlapping blocks of size

w w N N M M _{× .}

(

m n

)

x , is used to denote the block located at

(

m,n

)

. For a given code book C , the set of indices Y can be obtained by regular search.

( )

{

VQ

(

x

(

m n

)

)

}

{

y

(

m n

)

}

VQ = , = ,

= X

Y . (2)

Withy

(

m,n

)

, _σ2

(

m,n

)

is defined to be the variance of index values of those 3×3 blocks centered around

(

m,n

)

. The polarity set P=

{

p

(

m,n

)

}

can then be evaluated according to

(

)

(

)

   ≥ = otherwise , 0 , if , 1 , 2 T n m n m p σ . (3)

where threshold T is set to half of the code book size.

Two secret keys, namely Key1 and Key2 were used in the algorithm. Key1 served to be the random seed for a

random sequence, which is used to disturb W and result in W . During the embedding phase, p Key was ob-2

tained by applying an exclusive-OR operation on W and P . For watermark extraction, polarity set Pp ′can be

evaluated according to the reconstructed X′. Wp′ can then be obtained with the help of Key . Finally, W′ can 2

be arrived at by an inverse disturbing based on Key1.

The algorithm in [13] had successfully achieved the desired robustness. However, there are some potential problems with it. Lu et al. [14] had pointed out two possible negative effects associated with the algorithm in [13]. Firstly, it is possible to extract the watermark from a non-watermarked image. Secondly, the code book C must be kept secret. Otherwise, third parties can use it to embed their own watermarks. Moreover, Charalampidis [15] also discovered an abnormal phenomenon with the algorithm in [13]. With all keys available, the watermark

f

W extracted from a non-watermarked image X might closely resemble the true watermark W . This phe-n

nomenon will certainly cause a major issue in copyright arbitration.

Two countermeasures were employed in Charalampidis [15] as remedies to the above problems. One of them is redefining the threshold values as follows:

(

)

{

mn

}

medium Tm , 2 σ = . (4) m

T also serves to be the third secret key Key3. The other countermeasure is to reassign indices to the code

words in order to smooth out their statistic characteristics. The new indices are assigned according to the follow-ing pattern:

( )

( )

(

)

    < + = otherwise , 1 2 / min max if , 0 , , , , k i i k i i k i k i c c c I . (5)

where

I

_i,k

,

k

=

1

,

L

,

k

is the k-th bit in i-th index value; ci,k is the k-th element of i-th code word.

After that, a procedure was carried out to reduce the number of bits required to encode the new indices. In idea case, it should take log2

( )

L bits for a code book of size L.

Variance av

(

m,n

)

2

σ was then defined, based on the new indices, according to:

(

)

( )

( )

            −     =

∑ ∑ ∑

∑ ∑

+ − = + − = = + − = + − = 2 1 1 1 1 1 1 1 1 1 2 2 , 9 1 , 9 1 1 , m m i n n j k K k m m i n n j k av y i j yi j K n m σ . (6)

Following the same embedding and extraction procedures, with the newly defined Tm and av

(

m,n

)

2

σ , the work of Charalampidis [6] did resolve the problem it intended. However, according to our study, the code length

re-電腦學刊 第十七卷第二期（Journal of Computers, Vol.17, No.2, July 2006）

22

the development of our scheme on index assignment using genetic algorithms. The proposed scheme is capable of retaining the virtue of vector quantization without scarifying the robustness against common attacks.

3 Index Assignment by Genetic Algorithms

Inspired by Darwin’s theory of evolution, J. Holland [16] introduced the genetic algorithm [17] as a powerful computational model for optimization. Genetic algorithms work on a population of potential solutions, in the form of chromosomes, and try to locate a best solution through the process of artificial evolution, which consists of repeated artificial genetic operations, namely evaluation, selection, crossover, and mutation. Genetic algo-rithms maintain a population of candidate solutions and conduct stochastic search via information selection and exchange. With genetic algorithms, near-optimal solutions can be obtained within justified computation cost. A pseudo code for the operation of genetic algorithms is given bellows:

Pseudo Code for Genetic Algorithms Randomize initialize population P(0) WHILE terminate condition is not met Evaluate P(t) using object function

Select P(t+1) from P(t) based on fitness value

Conduct genetic operators, crossover and mutation, on P(t+1)

Although the operation of genetic algorithms is quite simple, it does have some important characteristics pro-viding robustness: 1. Search from a population of points rather than a single point. 2. Use object function directly, not their derivative. 3. Use probabilistic transition rule, not deterministic one, to guide the search toward promis-ing region. In effect, genetic algorithms maintain a population of candidate solutions and conduct stochastic search via information selection and exchange. It is well recognized that, with genetic algorithms, near-optimal solutions can be obtained within justified computation cost. However, it is difficult for genetic algorithm to pin point the global optimum. In practice, hybrid approach is recommended by incorporating gradient-based or local greedy optimization techniques. In such integration, genetic algorithms act as course-grain optimizers and gradi-ent-based method as fine-grain optimizers.

Genetic algorithm is used in our scheme to find out a suitable index assignment such that nearby indices are assigned to similar code words. To encourage the formation of chromosomes fulfilling these criteria, the object function is defined as follows:

(

)

∑ ∑

− = + − = = 1 0 , L i s i s i j j i c c D f . (7)

where D

(

ci,cj

)

is the Euclidian distance between code words ci and cj; s is the range of comparison.

The first design issue in applying genetic algorithms is to select an adequate coding scheme to represent poten-tial solutions in the form of chromosomes. For our problem, each chromosome is a permutation of indices repre-senting a possible index assignment. In the initial population, all chromosomes are randomly set. During the course of evolution, each chromosome is subject to fitness evaluation according to Equation (7). Fitness values are then used in the process of roulette wheel selection to generate next population. The crossover operator plays an essential role in combing good genes. A modified version of the crossover operator, as illustrated in Fig. 2, is developed to ensure the legality of newly generated chromosomes. Mutation operator acts as a background op-erator against the premature phenomenon. In our work, mutation is done by swapping the indices of two ran-domly selected code words in a chromosome.

Fig. 2. Modified crossover operator

ck ck+1 i j i j i

4 Watermark Embedding and Extraction

As the genetic algorithm terminates, we will arrive at a new index assignment I′=

{

i0′,i1′,L,iL′−1

}

, with which a

new code book C′=

{

c′0,c′2,L,cL′−1

}

can be constructed. Therefore, I′ can be regarded as a third secret key 3

Key . The new code book C′ will be used for internal operation during the processes of watermark embedding and extraction. With the help of the newly generated code book, statistical characteristics of the image to be pro-tected will be diffused evenly across the entire image. As a result, the risk of possible information leakage can be effectively suppressed. Fig. 3 illustrates the process diagram for watermark embedding.

Fig. 3. Process diagram of watermark embedding

In our scheme, the vector quantization is conducted using the new code book C′. We denote the index set for the image to be protected as YIA =

{

yIA

(

m,n

)

}

. The mean index value is defined, based on YIA, as follows:

(

)

(

( )

)

      = =

∑

= K k n m y k m NI cIA K n m y 1 , 1 , m Y . (8)

where NIk

( )

c is the index of the k-th nearest code word to c, and K is a parameter controlling the size of neighborhood.

We can then evaluate the variance of mean index value

{

m

(

m,n

)

}

2 σ = m Σ according to:

(

)

( )

( )

_           −     =

∑ ∑

∑ ∑

+ − = + − = + − = + − = 2 1 1 1 1 1 1 1 1 2 2 , 9 1 , 9 1 , m m i n n j m m m i n n j m m mn y i j y i j σ . (9)

A new thresholdTmean, also serves as a fourth keyKey , is defined to be: 4

(

)

              + × =

∑ ∑

− = − = 5 . 0 , 1 0 1 0 2 N M W M m W N n m mean W N W M n m T M N σ . (10)

Finally, we arrive at the following polarity set Pm =

{

pm

(

m,n

)

}

:

(

)

(

)

   ≥ = otherwise 0 , if 1 , 2 mean m m T n m n m p σ . (11)

For watermark embedding, Key2 an be obtained by applying an exclusive-OR operation on Wp and Pm, as

follows: P W Key = ⊕ . (12) VQ NMI Polarization IIA VQ-1 IA C C’ Key3 C Key3 X YIA Ym Y X’ Pm Key2 Wp

電腦學刊 第十七卷第二期（Journal of Computers, Vol.17, No.2, July 2006） 24

(

)

( )

{

ym,n =iY′IAm,n

}

= Y . (13)

The process diagram for watermark extraction is given in Fig. 4. For watermark extraction from a given im-age

X ′′

,

Y′

_m can be obtained from

Y′

_IAwith the help of Key3. We then have P′m by incorporatingKey4.

p

W′can be evaluated by applying an exclusive-OR operation on P′m and Key2.

2 m

p P Key

W′ = ′ ⊕ . (14)

Finally,

W′

can be arrived at by an inverse disturbing based on

Key

₁.

Fig. 4. Process diagram of watermark extraction

5 Experimental Results

A series of experiments was made to verify the feasibility and effectiveness of our approach. Key experiment setting is given below:

Test image: 512 ∗512 8-bit, gray-level Lena
Watermark: 128 ×128 binary Rose

Generation limit of GA: 10,000 generations
Population size of GA: 20 chromosomes
Crossover rate of GA: 70%

Mutation rate of GA: 0.1%
Range of comparison s: 4
Size of neighborhood K: 6

We also implemented the algorithm by Huang et al. [13] for comparative study. For possible information leak-age, experiments were made to extract watermark from non-watermarked images, including Baboon, Goldhill, and Peppers. The result is given in Fig. 3. It is clear that the potential problem with Huang et al. [13] ceases to exist with our scheme. Fig. 4 reports the watermarks extracted using our scheme under various kinds of attacks. Table 1 summarizes the bit correct rate for the two approaches under consideration. We can see that there is comparable performance between them.

6 Conclusions

A new scheme, in the domain of vector quantization, is developed and presented in this article. The proposed approach use genetic algorithms to reassign the indices of code words. As a result, embedded information will be diffused more evenly across the image to be protected, and then possible security leakage can be avoided. Ex-perimental results reveal that the proposed scheme is free from the potential limitations in previous approaches, while maintaining the robustness against various kinds of attacks.

VQ NMI Polarization IA C C’ Key3 X” YIA’ Ym’ Pm’ Key2 Wp’

(a) (b) (c)

(d) (e) (f)

Fig. 5. Watermarks extracted from non-watermarked images. (a)-(c): Huang et al. [4]. (d)-(f): our scheme.

(a) (b) (c)

(d) (e) (f)

Fig. 6. Watermarks extracted under various kinds of attacks Table 1. Comparison on bit correct rate

Huang et al. [4] Ours (a) No Attack 1.0 1.0 (b) JPEG 90% 0.9998 0.9993 (c) JPEG 60% 0.9913 0.9559 (d) Crop 25% 0.7600 0.8437 (e) LPF 0.9984 0.9294 (f) MPF 0.9982 0.9581

電腦學刊 第十七卷第二期（Journal of Computers, Vol.17, No.2, July 2006）

26

7 Acknowledgement

This work is supported by the National Science Council, Taiwan, R.O.C., under the grant No. NSC 94-2213-E-151 -013.

References

[1] R. Barnett, “Digital watermarking applications, techniques, and challenges,” IEE Electronics and Communication Engi-neering Journal, Vol. 11 1999, pp. 173-183.

[2] S. Katzenbeisser and F. Petitcolas, Information Hiding – Techniques for Stenography and Digital Watermarking, Artech House, 2000.

[3] J.-S. Pan, H.-C. Huang, and L.C. Jain, (Eds.), Intelligent Watermarking Techniques, World Scientific Publishing Com-pany, Singapore, 2004.

[4] R.G. Van Schyndel, A.Z. Tirkel, N. Mee, and C.F. Osborne, “A digital watermark,” Proceedings of the IEEE Interna-tional Conference on Image Processing, Vol. 2, 1994, pp. 86-90.

[5] I. Cox, J. Kilian, F.T. Leighton, and T. Shamoon, “Secure spread spectrum watermarking for multimedia,” IEEE Trans-actions on Image Processing, Vol. 6, 1997, pp. 1673-1687.

[6] J.J.K. O’Ruanaidh, W.J. Dowling, and F.M. Boland, “Phase watermarking of digital images,” Proceedings of the IEEE International Conference on Image Processing, Vol. 3, 1996, pp. 239-242.

[7] J.-S. Pan, S.-Y. Wu, C.-S. Shieh, and Y. Shi, “Genetic zero-tree selection for robust watermarking system,” Proceedings of the 7th World Multiconference on Systemics, Cybernetics and Informatics, 2003, pp. 177-179.

[8] S. Pereira and T. Pun, “An iterative template matching al-gorithm using the Chirp-Z transform for digital image water-marking,” Pattern Recognition, Vol. 33, 2000, pp. 173-175.

[9] K. Sayood, Introduction to Data Compression, 2nd Ed., Morgan Kaufmann, 2000. [10] R.M. Gray, “Vector quantization,” IEEE ASSP Magazine, 1984, pp. 4-29.

[11] A. Gersho and R.M. Gray, Vector Quantization and Signal Compression, Kluwer Academic Publishers, 1992.

[12] Y. Linde, A. Buzo, and R.M. Gray, R.M., “An algorithm for vector quantizer design,” IEEE Transactions on Communication, Vol. 28, 1980, pp. 84-95.

[13] H.-C. Huang, F.-H. Wang, and J.-S. Pan, “Efficient and robust watermarking algorithm with vector quantisation,” Elec-tronic Letters, Vol. 37, No. 13, 2001, pp. 826-828,.

[14] Z.M. Lu, D.G. Xu, and S.H. Sun, “Multipurpose image watermarking algorithm based on multistage vector quantiza-tion”, IEEE Transactions on Image Processing, Vol. 14, No. 6, 2005, pp. 822-831.

[15] D. Charalampidis, “Improved robust VQ-based watermarking,” Electronics Letters, Vol. 41, No. 23, 2005, pp. 1272-1273.

[16] J. Holland, Adaptation In Natural and Artificial Systems, University of Michigan Press, 1975.