網路多媒體數位浮水印及其應用之研究

國

立

交

通

大

學

資訊工程學系

博

士

論

文

網路多媒體數位浮水印及其應用之研究

A Study on Digital Watermarking and

Its Application on Network Multimedia

研 究 生：陳岳宏

指導教授：傅心家 教授

網路多媒體數位浮水印及其應用之研究

A Study on Digital Watermarking and

Its Application on Network Multimedia

研 究 生：陳岳宏 Student：Yueh-Hong Chen

指導教授：傅心家 教授 Advisor：Prof. Hsin-Chia Fu

國 立 交 通 大 學

資訊工程學系

博 士 論 文

Submitted to Department of Computer Science College of Computer Science

National Chiao Tung University in Partial Fulfillment of the Requirements

for the Degree of Doctor of Philosophy

in

Computer Science July 2008

Hsinchu, Taiwan, Republic of China

誌謝

網路多媒體數位浮水印及其應用之研究

學生：陳岳宏

指導教授：傅心家教授

國立交通大學資訊工程學系博士班

摘 要

在本論文中，我們提出了一種用於影像的浮水印技術以及一種用於視訊的指

印技術，並將之應用於一個以網頁為基礎的多媒體新聞資料庫系統。由於影像的

漸進傳輸在許多應用中均相當重要且常用，特別是在網際網路上傳輸影像時。另

一方面，複製及傳輸數位影像變得容易，使得利用數位浮水印技術來嵌入版權資

訊的需求應運而生。因此，在本論文中，我們提出一個用於漸進影像傳輸的浮水

印方法。此方法可以在影像仍在傳輸的過程中，即可偵測部份的浮水印資訊。隨

著傳輸的進行，所取出的浮水印，其錯誤的位元數也會逐漸減少。此方法不僅能

在影像傳輸的過程中，漸進地偵測浮水印，也會針對不同的係數大小，適當的選

取修改值來嵌入浮水印。因此能有效降低嵌入浮水印後影像品質上的損失。

我們也提出了一個新的視訊加密與指印嵌入方法。此方法包含兩部份：(1) 伺

服器端視訊加密，及 (2) 客戶端指印嵌入。首先，伺服器將一段視訊加密並以群

播方式傳送給所有客戶端。接著，利用(v, k, 1)-BIBD 來將解密金鑰分解成 v =

O(

)把解密子金鑰，並群播給 n 個客戶端。在收到各自的解密金鑰後，每個客

戶端便可組合他們各自的解密子金鑰，得到一把嵌入指印後的解密金鑰。當使用

這把金鑰來解開加密的視訊後，便會在視訊中嵌入一個對應於該客戶端的指印。

通常嵌入指印會使視訊中產生若干雜訊。由實驗結果發現，當指印由 15 個以下

的浮水印組成，或浮水印強度小於 0.5 時，畫面的訊噪比(PSNR)即高於 35db。故

本技術極適合用於網路環境中的視訊著作權保護。

最後，我們整合數種多媒體資訊擷取技術，建立了一個多媒體新聞網站。其

中所使用到的技術包括針對新聞標題、主播與過場，進行語音、影像與視訊分析，

以建立一個有組織、有意義的新聞視訊摘要。讓使用者可以利用非循序、快速而

具彈性的方式來瀏覽大量的新聞資料。藉由聲音分析，我們可以將新聞節目分成

新聞片段與廣告片段。利用 2003 年七月至 2004 年 8 月間的 400 小時電視新聞做

為實驗資料發現，本方法具有 96%的正確率。我們並使用 OCR 技術來辨視銀幕

上的新聞標題或子標題，做為每段新聞故事的文字資訊。此外，所抽取出的新聞

標題也可以用在網路上的相關新聞的連結與導覽。藉由整合人臉與場景分析與辨

識技術，OCR 的結果可用來提供使用者針對特定新聞的多種查詢。

我們也將所發展的浮水印與指印技術實際用於所建立的多媒體新聞資料庫

中。在關鍵畫面的抽取過程中，浮水印系統會為每個畫面嵌入浮水印以保護內容

擁有者的著作權。此外，換鏡偵測的結果可以提供給指印系統，做為更新客戶端

指印的時間點，以增強所嵌入指印的強固性。在多媒體的應用中，演算法的效能

是一項重要的課題。由測試結果發現，我們所提出的浮水印與指印方法，均極適

合於網路上的各種多媒體應用。

A Study on Digital Watermarking and Its Application on Network Multimedia

student：

Advisors：Prof. Hsin-Chia Fu

Department of Computer Science

National Chiao Tung University

ABSTRACT

In this dissertation, we propose a progressive image watermarking scheme and a

video fingerprinting scheme for copyright protection of multimedia applications on

the Internet. The ease of transmission and copying of images creates the need to use

digital watermarking to embed the copyright information seamlessly into the media.

On the other hand, progressive transmission of images is very useful in many

applications, especially in image transmission over the Internet. Since the progressive

image transmission has been widely used, in this dissertation, we first propose a

progressive image watermarking scheme. In this scheme, the watermark is embedded

in such a way that we can retrieve part of it even when the watermarked image is still

being transmitted. As transmission progresses, the retrieved watermark has a

decreasing bit error rate. Our proposed method can not only detect the watermarked

image progressively, but also intelligently select watermark embedding locations and

is robust to various attacks.

We also propose a new video scrambling and fingerprinting approach for digital

media right protection. The proposed method contains two parts: (1) video scrambling

at server side, and (2) fingerprint embedding at client side. First, a content server

scrambles and multicasts video contents to end users. Then, by applying a (v, k,

1)-BIBD scheme, the server partitions a descrambling key into v=O(

)

descrambling subkeys, and multicasts to n users. On receiving descrambling subkeys

from the content server, each user combines descrambling subkeys into a

descrambling key embedded with a fingerprint. By using he's descrambling key, a

scrambled video becomes a fingerprinted video designated to the user.According to

the experiment results, when the fingerprint consisting of less than 15 watermarks or

watermark strength

α

s less than 0.5 the PSNR of video frames can be 35 or higher.

Thus, this approach is suitable for multimedia applications over The Internet.

Finally, an integrated information mining techniques for multimedia TV-news

archive is addressed. The utilized techniques from the fields of acoustic, image, and

video analysis, for information on news story title, newsman and scene identification.

The goal is to construct a compact yet meaningful abstraction of broadcast news video,

allowing users to browse through large amounts of data in a non-linear fashion with

flexibility and efficiency. By using acoustic analysis, a news program can be

partitioned into news and commercial clips, with 90% accuracy on a data set of 400

hours TV-news recorded off the air from July 2003 to August of 2004. By applying

speaker identification and/or image detection techniques, each news stories can be

segmented with an accuracy of 96%. On screen captions or subtitles are recognized by

OCR techniques to produce the text title of each news stories. The extracted title

words can be used to link or to navigate more related news contents on the WWW. In

cooperation with facial and scene analysis and recognition techniques, OCR results

can provide users with multimodality query on specific news stories. Some

experimental results are presented and discussed for the system reliability and

performance evaluation and comparison.

The proposed web based TV news archive was also used as a test bed of the

proposed image watermarking and video fingerprinting methods. Watermarks can be

embedded automatically when key frames of a news story are extracted, and

information obtained from video analysis process can be used to increase the

robustness of video fingerprinting. In multimedia applications, computational

efficiency is one of the important issues. Our testing shows that the proposed

watermarking and fingerprinting methods are efficient to real world applications.

Table of Contents

Abstract i

Table of Contents iii

List of Figures vii

List of Tables x 1 Introduction 1 1.1 Motivations . . . 1 1.2 Research goals . . . 3 1.3 Dissertation organization . . . 3 2 Background 4 2.1 A Brief introduction of digital watermarking . . . 4

2.2 Progressive transmission of images on Internet . . . 5

2.2.1 Transmission-sequence based methods . . . 6

2.2.2 Successive approximation methods . . . 6

2.3 Fingerprinting methods for multicast video . . . 7

2.3.1 Independent Gaussian fingerprints and c-secure code . . . 7

2.3.2 Performance metric for multicast fingerprinting schemes . . . 7

3 Adaptive Watermarking Using Relationships between Wavelet

3.1 Introduction . . . 9

3.2 Related works . . . 9

3.3 Extreme-value based watermarking . . . 11

3.3.1 Hiding binary string into relationships of wavelet coefficients . . . . 11

3.3.2 An overview of EVBW . . . 13

3.3.3 Perceptually adaptive embedding . . . 16

3.4 Experimental results . . . 22

3.5 Summary . . . 23

4 Progressive Watermarking for Images on the Internet 24 4.1 Introduction . . . 24

4.2 Related works . . . 25

4.3 Proposed watermarking for progressive transmission . . . 26

4.3.1 Watermark embedding . . . 27

4.3.2 Watermark detection . . . 29

4.4 Experimental results . . . 32

4.4.1 Experimental setting . . . 33

4.4.2 Robustness of the proposed progressive watermarking scheme . . . 33

4.4.3 Earlier detection of the proposed progressive watermarking scheme 41 4.5 Summary . . . 43

5 Fingerprinting for Multicast video 44 5.1 Introduction . . . 44

5.1.1 Research goal . . . 45

5.1.2 Chapter organization . . . 45

5.2 Related works . . . 45

5.2.1 Transmitter-side fingerprint embedding . . . 46

5.2.2 Receiver-side fingerprint embedding . . . 46

5.2.4 Joint fingerprinting and decryption (JFD) . . . 47

5.3 Joint independent fingerprinting and decryption . . . 47

5.3.1 Notations and background assumption . . . 47

5.3.2 Video scrambling and descrambling . . . 49

5.4 Design and multicast a set of O(√U ) descrambling keys for U clients . . . 50

5.4.1 Balanced incomplete block designs (BIBD) . . . 51

5.4.2 Multicasting descrambling subkeys with BIBD . . . 52

5.5 Practical issues . . . 54

5.6 Experimental results . . . 56

5.7 Summary . . . 59

6 Application: Web-Based Multimedia News Archive 62 6.1 Introduction . . . 62

6.2 Related works . . . 63

6.3 A fully automated web-based TV-news system . . . 65

6.3.1 An overview of the TV-news system . . . 65

6.3.2 Multimedia data acquisition . . . 66

6.3.3 Key frame extraction . . . 66

6.3.4 Web-based user interface . . . 67

6.4 News information tree generation . . . 69

6.4.1 News information tree . . . 69

6.4.2 Analysis units . . . 69

6.4.3 Semantic labels of units . . . 70

6.5 Data mining on the news information tree . . . 71

6.5.1 Mine the news preference of a TV station . . . 72

6.5.2 The evolution of a series of news stories . . . 73

6.5.3 The mining on TV commercial . . . 73

6.7 Summary . . . 75

7 Conclusions and future works 85

7.1 Conclusions . . . 85 7.2 Future works . . . 86

List of Figures

3.1 _{Curves of (x − 112)}2_{+ (x + 5)}2 _{(curve 1), (x − 112)}2_{+ (x − 107)}2_{+ (x + 5)}2

(curve 2) and (x − 112)2_{+ (x − 107)}2_{+ (x + 1)}2_{+ (x + 5)}2 _{(curve 3) . . . 15}

3.2 Results of applying watermark detection process to 58600 watermarked and non-watermarked images . . . 17 3.3 Results of applying watermark detection process to six watermarked and

non-watermarked test images . . . 18 3.4 Results of watermark detection after JPEG compression . . . 19 3.5 Results of watermark detection after line removing attack . . . 20 3.6 Results of watermark detection after Gaussian filtering and sharpening . . 21 4.1 A flow diagram of the proposed binary valued watermark embedding

ap-proach. . . 27 4.2 An example of the proposed watermark embedding approach. . . 28 4.3 A flowchart of the proposed progressive watermark detection. . . 30 4.4 Examples of progressive watermark detection when partial watermarked

image data are received . . . 32 4.5 The result images of applying four common image processing operations

on watermarked Lena image. . . 34 4.6 The result images of applying four common image processing operations

on watermarked Baboon image. . . 35 4.7 The result images of applying four common image processing operations

4.8 The result images of applying four common image processing operations on watermarked Fishing boat image. . . 37 4.9 The result images of applying four common image processing operations

on watermarked Sailboat image. . . 38 4.10 The result images of applying four common image processing operations

on watermarked Peppers image. . . 39 4.11 The plot of percentage of correct watermark bits versus JPEG quality

fac-tor. Six watermarked images were compressed by JPEG. . . 40 4.12 The plot of percentage of correct watermark bits versus PSNR. Six

water-marked images were compressed by JPEG2000. . . 40 4.13 The plot of percentage of correct watermark bits versus received image

data amount. JPEG hybrid mode and JPEG2000 were used to compress the watermarked image Lena. . . 42 4.14 The plot of percentage of incorrect watermark bits versus received image

data amount. JPEG hybrid mode and JPEG2000 were used to compress the watermarked image Lena. . . 43 5.1 Overview of partitioning and multicasting a descrambling key from a

con-tent server to clients, and embedding a fingerprint to the descrambled video. 50 5.2 Methods to prevent malicious users from using key collusion attacks. . . . 56 5.3 The experimental results show that the relationship of the image visual

quality (PSNR) vs. the numbers of embedded watermarks. . . 58 6.1 A sample of evening news index from the Vanderbilt TV news archive. . . 64 6.2 Flow chart of automatic news content generation. . . 65 6.3 An example of spatio-temporal slice. The locations indicated by arrow

symbols are just the shot-change locations. . . 67 6.4 User interface of the proposed web-based multimedia news archive. . . 68 6.5 The data structure of a news information tree. . . 77

6.6 The flow diagram of the proposed news story analysis and information extraction processes. . . 78 6.7 The flow diagram of a BIC-based audio segmentation method. . . 78 6.8 The general structure of a news story. On-site scene story contains three

major news contents: locations, interview and tables or quoted words. . . . 79 6.9 On-site scene segmentation flow. . . 79 6.10 Information flow of the generation of a news information tree. . . 80 6.11 An example of locality scene frame. The locality scene is used to show

where and what the news occurred. Thus, the location information and event description can be retrieved from the close-captions of a locality scene. 81 6.12 An example of interview scene frame. The interview scene is used to present

the news persons’ point of view. Thus, the interviewee’s name and their opinion can be extracted from the screen characters or closed captions. . . 81 6.13 An example of data chart scene frame. The data chart scene is used to

present information in a organized manner. Additional information is also available from the on screen characters. . . 82 6.14 Three sets of (representative) keywords are used to associate the appearing

frequency of social, political and sport news in a news program. . . 82 6.15 The life-cycle of a specific news events along with a period of time. . . 83 6.16 The overall architecture and information processing flow of the proposed

List of Tables

4.1 Experimental results of watermark detection after Gaussian filtering and sharpening. . . 41 5.1 Average similarity values of four testing images. . . 60

Chapter 1

Introduction

1.1

Motivations

The growth of high speed computer networks and that of Internet, in particular, has explored means of new business, scientific, entertainment, and social opportunities in the form of electronic publishing and advertising, real-time information delivery, product ordering, transaction processing, digital repositories and libraries, web newspapers and magazines, network video and audio, personal communication, lots more. The new op-portunities can be broadly grouped under the label ”electronic commerce” . The cost effectiveness of selling software, high quality art work in the form of digital images and video sequences by transmission over World Wide Web (www) is greatly enhanced con-sequent to the improvement of technology. Sending hard copies by post may soon be a thing of past.

Though the commercial exploitation of the www is steadily being more appreciated, apprehension on the security aspect of the trade has only funneled the exploitation to be restricted to the transmission of demo and free versions of software and art. Ironically, the cause for the growth is also of the apprehension-use of digital formatted data.

Digital media offer several distinct advantages over analog media: the quality of digital audio, images and video signals are higher than that of their analog counterparts. Edit-ing is easy because one can access the exact discrete locations that should be changed. Copying is simple with no loss of fidelity. A copy of a digital media is identical to the

original.

The ease by which a digital information can be duplicated and distributed has led to the need for effective copyright protection tools. Various software products have been recently introduced in attempt to address these growing concerns. It should be possible to hide data (information) within digital audio, images and video files. The information is hidden in the sense that it is perceptually and statistically undetectable.

One way to protect multimedia data against illegal recording and retransmission is to embed a signal, called digital signature or copyright label or watermark [1, 2], that completely characterizes the person who applies it and, therefore, marks it as being his intellectual property.

However, most of digital watermarking methods prevent illegal use or spreading digital contents by passively checking whether the user owns lawful ownership. Thus, if a method that can detect the source of illegal distribution actively, so as to stop the transmission of unauthorized digital media will be a much better digital right protection approach. In other words, the system can actively poll other networks (e.g., underground media distribution centers) to determine if they own the rights to circulate the content. If a watermark is detected, the system will notify the content owner, and thus creating a natural deterrent to piracy.

When an embedded watermark is associated with a particular user, it can be considered as a fingerprint. Once a fingerprinted media has been illegally distributed, the original user could be easily traced from redistributed version. Unfortunately, fingerprint embedding process often makes a signal media copy into many different versions, which have to be transmitted via unicast method. Generally, it is more efficient to transmit a single media via multicast method to massive users. It becomes quite important in the field of video-based applications to transmit video data embedded with fingerprints efficiently to all the users. Thus, how to allow most of video content to be transmitted via multicast and only the very little fingerprint related contents are transmitted via unicast have been an important issue to network multimedia applications.

1.2

Research goals

In this dissertation, we first proposed a robust watermarking scheme for digital images. Then, we study the properties of a watermarking scheme that can be used in the active wa-termark detection system discussed above, and propose a progressive image wawa-termarking method. To decrease the bandwidth required for delivering fingerprinted video, we also propose a new video scrambling and fingerprinting approach for digital media copyright protection. Finally, as an application of the proposed watermarking and fingerprinting methods, a WWW-based multimedia TV-news archive, which integrates text, image and video data is addressed.

1.3

Dissertation organization

In the rest of this dissertation, survey of various image and video watermarking systems are presented in Chapter 2. In Chapter 3, we present an adaptive watermarking scheme for images. The progressive watermarking scheme for image data is discussed in Chapter 4. In Chapter 5, the problem of embedding a fingerprint for each multicast client is addressed. Then, two methods are proposed respectively to reduce the bandwidth required by multicast server when video clips with fingerprints are transmitted to clients. In Chapter 6, we present a web-based news archive system in which the proposed image and video watermarking schemes were used. Finally, conclusions and suggestions appear in Chapter 7.

Chapter 2

Background

2.1

A Brief introduction of digital watermarking

The watermark can be regarded as an additive signal W , which is a binary string or a sequence of independent Gaussian random numbers. Although a watermark consisting of Gaussian random numbers is more secure, binary watermark can be used to represent a user ID or logo. To achieve a perceptually indistinguishable watermarked and original signal, a watermarking approach usually keep the power of the watermark signal very low. Commonly used embedding techniques can be classified into additive, multiplicative, quantization-based, and relationship-based schemes. In additive schemes, a very weak W is added into original signal x, as shown in Eq. 2.1

Y = X + αW, (2.1)

where X is the original signal, Y is the watermarked signal and α is a constant, referred to as watermark strength. In multiplicative schemes, samples of the original data are multiplied by an independent signal (1 + αW ). Precisely, multiplicative schemes can be described by Eq. 2.2:

Y = X × (1 + αW ). (2.2) In quantization based watermarking schemes, X is modified such that the quantization indices imply a watermark for a certain quantization step q. For example, a binary

watermark W can be embed into the signal X with following Eq. 2.3:    ⌊Y q + 0.5⌋ mod 2 = 0 if W = 0 ⌊Yq + 0.5⌋ mod 2 = 1 if W = 1 . (2.3)

In this example, signal X is modified into Y such that its quantization index is an even number to imply a binary value ’0’, and vice versa.

The basic idea of relationship-based watermarking is to use two pixel values or trans-form domain coefficients in a image to represent each bit of a binary watermark. For example, if the value of the first coefficient is larger than the second, then an ’1’ is en-coded; otherwise, a ’0’ is encoded [3, 4]. Some watermarking approaches use more than two coefficients for each watermark bit to increase the robustness. Typically, these ap-proaches adequately modify the selected coefficients such that a pre-specified one among them becomes smallest or largest, according to the binary value to be embedded [5, 6, 7].

2.2

Progressive transmission of images on Internet

Generally, the regular scanning pattern is used in image transmission on Internet. In other words, image pixels are displayed from left to right and top to bottom. Often an user can recognize the image and decide whether or not to download the whole image only when a substantial portion of data has been transmitted. Considering that signif-icant amounts of data are needed to represent large digital images, image transmission across low-bandwidth channels can be exceedingly slow. Thus, a better solution is to improve the transmission method so that the image is transmitted progressively. Pro-gressive image transmission makes effective use of communication bandwidth. Instead of transferring an image at full resolution sequentially, progressive transmission first trans-mits an approximate version of the entire image so that its structural information is sent early in the transmission. The quality of this image is progressively improved over a number of transmission passes. The advantage is that it allows the user to quickly recog-nize an image. Two classes of progressive transmission methods are commonly used and briefly introduced in this section: successive approximation methods and transmission

sequence-based methods.

2.2.1 Transmission-sequence based methods

A progressive transmission method using the transmission-sequence (TS) based approach is composed of a classifier that separates the image data into different transmission groups, a mechanism for ordering the groups, and a method for specifying the ordering. Generally, an increase in the number of groups improves the buildup quality but involves a higher overhead cost. The approach is adopted by progressive JPEG compression, and is referred to as spectral selection mode in progressive JPEG. In spectral selection mode, JPEG encoder takes advantage of the spectral (spatial frequency spectrum) characteristics of the DCT coefficients: the higher AC components provide only detail information. So, the order of coefficients to be transmitted may be as follows:

Group 1: Encode DC and first few AC components, e.g., AC1, AC2. Group 2: Encode a few more AC components, e.g., AC3, AC4, AC5.

...

Group k: Encode the last few ACs, e.g., AC61, AC62, AC63.

2.2.2 Successive approximation methods

In successive approximation methods, progressive transmission is achieved by refining the precisions of the coded data at each transmission stage. In the initial stage, a low-precision form of the data is sent. In subsequent stages, the precision is gradually increased until the full precision of the coded data at the final stage. This approach is adopted by both JPEG and JPEG2000 standards. Taking JPEG as an example, instead of gradually encoding spectral bands, all DCT coefficients are encoded simultaneously, but with their most significant bits (MSBs) first.

Scan 1: Encode the first few MSBs, e.g., bits 7, 6, 5, and 4. Scan 2: Encode a few more less-significant bits, e.g., bit 3.

...

Scan m : Encode the least significant bit (LSB), bit 0.

Successive approximation used in JPEG2000 is similar to that in JPEG, but a more efficient bit-plain encoding method is used.

2.3

Fingerprinting methods for multicast video

2.3.1 Independent Gaussian fingerprints and c-secure code

Fingerprinting is an useful technique because the original user could be easily traced from redistributed version. However, fingerprints could be identified and removed by compar-ing certain amount of media versions, which are embedded with various fcompar-ingerprints. This kind of attack is often referred to as collusion attack. Several studies [8, 9, 10] have been proposed to addressing collusion-resistant ability among fingerprint approaches. In [8], Su et al., showed that fingerprints constructed from independent Gaussian watermarks re-quired shorter fingerprint sequence than fingerprints constructed from c-secure code [9, 10] did. Moreover, [8] indicated that in the circumstance of a simple linear collusion attack consisting of adding noise to the average of several fingerprinted copies, no other wa-termarking schemes could offer better collusion resistance than independent watermarks. However, embedding independent watermarks into a video as fingerprints often makes a single video into a large amount of different copies, even though each copy is only slightly different from each other. Thus, video contents embedded with independent watermark based fingerprints can hardly be transmitted by multicast methods.

2.3.2 Performance metric for multicast fingerprinting schemes

Multicast transmission described in this paper is similar to that in [11]. First, we as-sume that there is only a public channel between a media server and all clients. Data transmitted with the public channel can be received by all clients simultaneously. In other words, sending data directly with the public channel is broadcast transmission. If

the server needs to send secret data to a specific client, the data should be encrypted with the client’s secret key before transmission. All clients’ secret keys are delivered with via a secure channel. Encrypting and transmitting data to a specific client is referred to as unicast transmission. We use the term multicasting for the transmission of data using both the unicast and broadcast methods. Qualitatively, the transmission of media content is efficient if it incorporates both the broadcast and unicast methods such that the broadcast channel is used a few times, while the unicast channel is seldom employed. Quantitatively, the efficiency of a distribution method is measured relative to the purely naive broadcasting scenario and can be defined by the ratio given in Eq. 2.4 [11],

ηD :=

mD

m0

, (2.4)

where mD is a value proportional to the bandwidth used by a fingerprinting scheme,

and m0 is a value proportional to the bandwidth used in the unicast channel case. In

particular, m0 is defined to be the number of times the public channel is used when the

fingerprinted content is sent to each user respectively, and mD is the number of times the

public channel is used by the fingerprinting scheme. We expect η to be between 0 and 1. In addition, for a fingerprinting method 1 that is more efficient than a fingerprinting method 2, we have η1 < η2.

Chapter 3

Adaptive Watermarking Using

Relationships between Wavelet

Coefficients

3.1

Introduction

In this Chapter, we propose an image watermarking approach for the purpose of proving ownership. This approach hides watermarks in relationships between wavelet coefficients and afterward detects the watermarks blindly.

The rest of the chapter is organized as follows. A brief review of those research efforts is described in Section 3.2. Section 3.3 introduces the assumption in this chapter and presents the extreme value based watermarking approach. The experimental results are shown in Section 3.4. Finally, Section 3.5 summarizes our approach and provides a brief concluding remarks.

3.2

Related works

As described in Section 2.1, the relationship-based watermarking is to use two pixel values or transform domain coefficients in a image to represent each bit of a binary watermark. If the first value is larger than the second, then an ’1’ is encoded; otherwise, a ’0’ is encoded. Based on this idea, several watermarking methods have been proposed to use relationships

between pixels or transform domain coefficients. In [3], Koch and Zhao proposed to group selected coefficients of an 8×8 DCT block in the image into ordered pairs. Each bit of the watermark is then encoded using one of the coefficient pairs. However, there was no experimental result showing the robustness and imperceptibility of the watermark in the paper. Hsu and Wu [4] proposed an approach using middle frequency coefficients chosen from one or more 8×8 DCT blocks to embed watermarks. Quantization operation is taken into account in this approach so that watermarks can survive the JPEG lossy compression. For watermark extraction, the original image and the watermark used during the embedding step are required. They are, however, unavailable in some applications such as copy control.

Some watermarking approaches use more than two coefficients for each watermark bit to increase the robustness. Typically, these approaches adequately modify the selected coefficients such that a pre-specified one among them becomes smallest or largest, accord-ing to the binary value to be embedded. In [5], three coefficients selected from an 8×8 DCT block are altered to meet the situations in which the third coefficient is largest or smallest, in accordance with the watermark bit to be embedded. However, the extractor proposed in [5] ignores the three-coefficient sets in which the third coefficient is between other two. As a result, a failure of detecting one single bit may make the whole watermark undetectable. A similar approach in [6] selects six coefficients from a DCT block and then exchanges the first coefficient with the largest or smallest coefficient among them. Never-theless, a significant degradation in image quality may be caused if the difference between two coefficients to be exchanged is large. A closely related approach was proposed in [7]. This approach divides the pixels of an image into two groups. The sum of the pixels in one group is subtracted from the sum of the pixels in the other group to obtain a detect statistic, which is then compared against a certain threshold to determine whether the watermark is present. However, [7] did not use a human vision system (HVS) model to increase imperceptibility of watermarks.

min-imized in the sense of an appropriate distortion metric. Thus, an extreme value based watermarking (EVBW), using the relationship between wavelet coefficients, is proposed. When embedding watermarks, the EVBW will adaptively minimize the distortion, mea-sured with PSNR or just noticeable distortion (JND). By minimizing the images dis-tortion, the strength of watermarks can be enlarged to increase the robustness of the watermarks, while keeping the quality of watermarked images visually accepted. The experimental results shown in section 3.4 illustrate the performance of the proposed ap-proach.

3.3

Extreme-value based watermarking

It is assumed in this chapter that a watermark is a binary string consisting of two symbols, -1 and 1. All bits of the watermark are embedded into an image with the same manner, separately. The manner to represent a binary string using relationships of wavelet co-efficients is first introduced in this section. An overview of EVBW is then proposed in Subsection 3.3.2. The embedding algorithm of EVBW is discussed in detail in Subsection 3.3.3.

3.3.1 Hiding binary string into relationships of wavelet coefficients

To embed a watermark, an image is firstly transformed into Haar wavelet domain. For each bit of the watermark, a number of coefficients in pre-specified subband (e.g., LH2, HL2 or HH2) are then randomly chosen and modified. Finally, inverse wavelet transform is applied to obtain the watermarked image.

When one bit of the watermark is to be embedded, an user-specified number of co-efficients are chosen randomly. These coco-efficients are then modified such that the first coefficient, in the order of being chosen, becomes the largest one if an ’1’ is to be embed-ded. If a ’-1’ is to be embedded, the coefficients should be modified such that the first coefficient becomes the smallest one. Suppose ci, i = 1, · · · , n, are the chosen coefficients,

to be embedded, and L is the length of the watermark W . Precisely, after modification step, the relationship behind the coefficients is as equation (3.1)

   c′ 1 ≥ max(c ′ 2, c ′ 3, · · · , c ′ n) + δ if wl = 1 c′₁ < min(c′₂, c′_{3, · · · , c}′n) + δ if wl = −1 (3.1) where c′

i, i = 1, · · · , n are the modified coefficients, wl is a particular bit of the watermark

code, and δ, δ ≥ 0, is the strength parameter specifying the difference between the first coefficient, c′₁, and the largest (smallest) one among remaining coefficients. Intuitively, the larger the value of δ, the more robust the watermark. However, the perceptual fidelity of the watermarked image will decrease when a larger δ is adopted. Thus, for different applications, the value of δ should be specified by the user.

To clarify the description, a simple example of implying a watermark bit with coeffi-cients is given. Suppose an ’1’ is to be embedded and five coefficoeffi-cients, -5, 112, -1, 107 as well as 13, are chosen randomly. A straightforward manner is to increase the value of the first coefficient, -5, to a value equal to or larger than 112, and other coefficients are left unchanged. By this manner, the first coefficient, -5, should be increase to 112 + δ to embed an ’1’.

The simplest watermark extracting method is to pick up the same coefficients and determine if the first coefficient is the largest (smallest) one. However, the watermarked image may be distorted due to some image-processing operations, and the first coefficient is possibly no longer the largest (smallest) one. Hence, the purposed extracting method is to compare the first coefficient with the largest and smallest ones among remaining coefficients. If the value of the first coefficient is closer to the largest one among remaining coefficients, an ’1’ will be extracted; otherwise, a ’-1’ will be extracted. This method can be described as equation (3.2): wl′ =    1 if c” 1 ≥ c” max+c”min 2 −1 otherwise (3.2) c” max = max(c”2, c”3, · · · , c”n) c” min = min(c”2, c”3, · · · , c”n) (3.3)

where c”

i, i = 1, · · · , n are the coefficients obtained from an image to be judge, and w

′

l,

1 ≤ l ≤ L, is the extracted binary value. The percentage of matching bits between the extracted binary string W′

, W′

= {w′

l|w

′

l ∈ {1, −1}, 1 ≤ l ≤ L}, and the watermark W is

then calculated. Finally, the percentage of matching bits between W′

and W is compared with a certain threshold to determine if the watermark exists or not.

3.3.2 An overview of EVBW

The main idea of EVBW is to modify more than one coefficient at the same time. To embed an ’1’, if the first coefficient c1 is increased to x + δ, all coefficients larger than x

should be decreased to x to fit the rule shown in equation (3.1). Therefore, it is possible to find the optimal value of x such that the watermarked image have the best visual quality according to an appropriate quality metric.

One of the image quality metrics that are widely used is peak signal-to-noise ratio (PSNR). Based on the definition of PSNR, if the mean square error (MSE) of the modified coefficients is minimized, the PSNR value is maximized simultaneously. Suppose a bit ’1’ is to be embedded into n coefficients. If c1 is increased to x + δ and all coefficients larger

than x are decreased to x, the square error (SE) value can be calculated as equation (3.4): SE(x) = ((x + δ) − c1)2+

X

ci>x

(ci− x)2 (3.4)

Then the minimum of SE(x) can be obtained by finding out the value of x where the first derivative of SE(x) is equal to 0. The first derivative of SE(x) is shown in equation (3.5), and the optimal value of x is shown in equaion (3.6).

d dxSE(x) = 2 × (x + δ − c1) + 2 × X ci∈M(x) (x − ci) (3.5) x = P ci>x ci ! + c1− δ k + 1 , (3.6)

where M (x) is a set that consists of the coefficients larger than x except c1, i.e., M (x) =

{ci|ci > x, 2 ≤ i ≤ n} and k is the number of elements in M(x). In equation (3.6), it

x should be larger than the (k + 1)-th largest coefficient but smaller than k-th largest coefficient. Therefore, the algorithm to find the optimal value x is as follows:

Obtain d1, d2, · · · , dn by sorting c1, c2, · · · , cn

such thatd1 ≥ d2 ≥ · · · ≥ dn

Suppose c1 is the (k + 1)-th largest value

If (k + 1) = 1 xopt= d2, Stop End If Fori = 1 to k x =  Pi j=1dj  +dk+1−δ i+1 Ifdi+1 < x ≤ di xopt = x, Stop End If End For xopt = c1, Stop

After the algorithm finishes, the optimal value of x can be found. c1 can then be modified

to x + δ, and all coefficients larger than x be modified to x to embed a bit ’1’. A similar algorithm to find the optimal value to embed a ’0’ is as follows:

Obtain d1, d2, · · · , dn by sorting c1, c2, · · · , cn

such thatd1 ≤ d2 ≤ · · · ≤ dn

Suppose c1 is the (k + 1)-th smallest value

If (k + 1) = 1 xopt= d2, Stop Fori = 1 to k x =  Pi j=1dj  +dk+1+δ i+1 Ifdi+1 > x ≥ di xopt = x, Stop

Figure 3.1: Curves of (x − 112)2_{+ (x + 5)}2 _{(curve 1), (x − 112)}2_{+ (x − 107)}2_{+ (x + 5)}2 _{(curve 2) and}

(x − 112)2_{+ (x − 107)}2_{+ (x + 1)}2_{+ (x + 5)}2 _{(curve 3)} End If

End For xopt = c1, Stop

Finally, c1 is decreased to x − δ, and all coefficients smaller than x be increased to x to

embed a bit ’0’.

Continuing the example in previous subsection, if δ = 0, the SE(x) value is:

SE(x) =                            (x − 112)2_{+ (x + 5)}2 if 107 ≤ x < 112 (x − 112)2_{+ (x − 107)}2_{+ (x + 5)}2 if −1 ≤ x < 107 (x − 112)2_{+ (x − 107)}2_{+ (x + 1)}2_{+ (x + 5)}2 if −5 ≤ x < −1

By applying the proposed algorithm, the optimal value of x, about 71.3, can be ob-tained. The curve of SE(x) is shown in figure 3.1. It is clear that SE(x) is the minimum when x = 71.3.

3.3.3 Perceptually adaptive embedding

Several researchers have indicated that PSNR may not be a ideal measurement of image quality [2]. Thus, the algorithm proposed in previous subsection is extended so that the watermark can be embedded based on JND.

A level of distortion that can be perceived in 50% experimental trials is often referred to as just noticeable difference, or JND [2]. Suppose a certain JND evaluation method in wavelet domain is used, and a JND value is always larger than zero. The perceptual distortion could be measure by weighting with the JND value the square error between original and modified coefficients. If a bit ’1’ is to be embedded into n coefficients, the weighted square error caused by watermark embedding process is:

∆(x) = (x + δ) − c1 J1 !2 + X c∈M(x) c − x Ji 2 , (3.7)

where Ji, i = 1, · · · , n is the JND value of ci. The minimum of ∆(x) can then be obtained

by finding out the value of x where the first derivative of ∆(x) is equal to zero. The first derivative of ∆(x) is shown in equation (3.8), and the optimal value of x is shown in equation (3.9). d dx∆(x) = 2 × (x + δ − c1) J2 1 + 2 × X ci∈M(x) x − ci J2 i ! , (3.8) x = P ci>x ci J2 i ! + c1 J2 1 − δ J2 1 1 J2 1 + P ci∈M(x) 1 J2 i ! . (3.9)

According to equation (3.9), an algorithm similar to that described in previous subsection can be used to find the optimal value of x for embedding watermarks.

Intuitively, the extracting algorithm described in Section 3.3.1 can be used to ex-tract watermarks embedded with PSNR-maximized or perceptually adaptive embedding algorithm. In the next section, several experimental results are presented to show the robustness and fidelity of the proposed algorithms.

(a) PSNR-maximized embedding

(b) perceptively adaptive embedding

Figure 3.2: Results of applying watermark detection process to 58600 watermarked and non-watermarked images

(a) PSNR-maximized embedding

(b) perceptively adaptive embedding

Figure 3.3: Results of applying watermark detection process to six watermarked and non-watermarked test images

(a) PSNR-maximized embedding

(b) perceptively adaptive embedding

(a) PSNR-maximized embedding

(b) perceptively adaptive embedding

(a) PSNR-maximized embedding

(b) perceptively adaptive embedding

3.4

Experimental results

In this section, results of experiments using Strmark [12] are proposed to demonstrate the robustness of the proposed algorithms. An 1000-bit watermark was generated randomly and used throughout the experiments. To embed one bit of the watermark, twelve coeffi-cients were chosen from LH2, HL2 or HH2 subband. In other words, n was equal to 12 in our experiments. The strength parameter δ was assigned to 0. The method proposed in [13] was used to evaluate JND values in perceptually adaptive embedding algorithm.

To determine the threshold for watermark detection process, 58600 images chosen from Corel Gallery 1000000 were watermarked using PSNR-maximized embedding and perceptually adaptive embedding algorithm. The results are shown in Fig. 3.2. Then, the threshold was chosen such that watermarked and non-watermarked images could be well separated. According to the experimental result in Fig. 3.2, the threshold was assigned to 60% in all following experiments.

To evaluate the robustness of the proposed approach, six popular testing images: Lena, Baboon, F16, Fishing Boat, Pentagon and Peppers were watermarked with the proposed watermarking approaches. Then, four image processing operations, JPEG compression, Gaussian filtering, sharpening and line removing were applied on the watermarked images. The result are shown in Fig. 3.3 - Fig. 3.6. As shown in Fig. 3.3, six watermarked test images can be distinguished from their original versions easily. Although the perceptively adaptive embedding method can not successfully embed all watermark bits due to the adopted JND model, the proposed watermarking schemes are still very effective. The experimental results of watermark detection after JPEG compression are shown in Fig. 3.4. It is obvious that the watermark was still detectable until JPEG quality was lower than 15%. As shown in Fig. 3.5 and Fig. 3.6, similar experimental results can be obtained after line removing, Gaussian filtering or sharpening. These experimental results show that the proposed watermarking approaches are robust on minimizing the perceptual distortion.

3.5

Summary

In this chapter, we propose a strategy that hide watermarks in the relationship between wavelet coefficients. The proposed strategy would minimize the perceptual distortion of embedded images, measured by PSNR or JND. Experimental results illustrate the robustness of the proposed algorithm after common image processing operations such as JPEG compression, Gaussian filtering and sharpening.

Since the appropriate quality of images is different from application to application, it should be able to be pre-specified by the user. The proposed method can also be used to embed watermarks under the constraint of pre-specified image quality after slightly modification.

Chapter 4

Progressive Watermarking for

Images on the Internet

4.1

Introduction

The rapid developments in computer and communication technologies have made more and more images and multimedia delivery over Internet. Progressive transmission methods are often used to allow a user to preview an image in advance. According to [14], there are four kinds of progressive image transmission methods, that are (1) Transmission Sequence-Based (TS-based) Method, (2) Successive approximation method, (3) Multistage residual method, and (4) Hierarchical method.

When there is more and more illegal spreading of digital media via Internet, stopping and/or preventing Internet piracy turns into an important affair. Among various media protection methods, digital watermarking [2] has long been an important and attractive digital protection technologies. However, most of digital watermarking methods prevent illegal use or spreading digital contents by checking whether the user owns lawful owner-ship. Thus, if a method that can detect the source of illegal distribution, so as to stop the transmission of unauthorized digital media will be a much better digital right protection approach. In this chapter, we propose a new wavelet based progressive watermarking system, which addresses the following scenarios:

de-veloped on wavelet transform, the major concepts can be applied to other frequency based transform domain, such as DCT or block DCT.

2. Progressive Detection: The progressive detection characteristics of the proposed watermarking scheme can be nicely fit into some commonly used compression stan-dards offering progressive encoding/decoding capability, such as JPEG and JPEG2000. 3. Optimized imperceptibility: The proposed watermarking method modifies

sev-eral coefficients of an image at once to embed a binary value (e.g., a ’0’ or ’1’). Thus, optimization methods can be used to select a proper value for each coefficients so that the watermarked can image achieve the best visual quality.

In order to verify the functionality and to evaluate the performance of the proposed watermarking system, two types of experiments were exercised: (1) robustness against image attacks and (2) early watermark detection along with progressive image trans-mission. Experiment results show that the proposed watermarking scheme has excellent robustness and imperceptibility as most of existing watermark systems, also successful early detection for various compression methods.

The rest of the chapter is organized as follows. Some related researches are discussed in Section 4.2. Section 4.3 describes the proposed watermarking embedding and detection methods. Section 4.4 presents experimental results of robustness against various attacks and earlier detection with respect to commonly used compression methods. Finally, Sec-tion 4.5 summarizes our methods and provides a brief concluding remarks.

4.2

Related works

In general, most of watermarking systems require robustness against attacks and im-perceptibility of hiding watermarks. An network based progressive watermark system requires the following additional properties:

1. Progressive detection: when partial image contents are received, watermarking detection can start to check whether or not a suspected watermark exits.

2. Early decision: watermark detecter can make an early decision of whether a sus-pected watermark exists or not, without waiting to see the whole image contents. These two properties provide a progressive watermarking system a great saving of processing resources as well as network bandwidth. Among various existed watermarking embedding methods, three different types are classified in [15]. These embedding methods are called as (1) additive, (2) multiplicative, and (3) quantizing watermarking. In general, these watermarking methods are applied when whole image contents are available, instead of partial image contents. Recently, new methods are proposed to detect watermarks in partial image contents. Aiming to spectral selection mode of JPEG compression, Chen et al., [16] proposed to embed additive watermarks to 8 × 8 DCT coefficients, and to have the coefficients in middle bands to be compressed and transmitted at an earlier time than that in high frequency bands. Therefore, the embedded watermarks can be detected in a progressive manner. However, this method does not provide early decision function, which is an important feature for saving computing resource and network bandwidth in progressive watermarking detection methods. In [17], Ashoka Jayawardena et al., proposed to transform watermarks into binary wavelet domain, so that the watermarks can be embedded in JEPG2000 compressed images, and then the watermarked image can be transmitted in multiresolution channels. Thus, when more wavelet coefficients of the watermark are transmitted and received, the more watermark data can be detected in a progressive manner. However, this method lacks of considering the decrease of image quality due to embedded watermarks.

4.3

Proposed watermarking for progressive transmission

By observing the transmission procedures of TS-based and successive approximation pro-gressive transmission [18], we noticed that:

1. For the successive approximation transmission, the approximation value associated with larger wavelet coefficients often transmitted and received in the early part of a

Figure 4.1: A flow diagram of the proposed binary valued watermark embedding approach.

compressed data stream;

2. For the TS-based approximation transmission, when a watermark was embedded in earlier transmitted data stream, the earlier the watermark will be detected.

As described in Chapter 3 and in [19], we proposed a binary value watermark embed-ding method, which inserts each bit of a watermark accorembed-ding to the numerical relationship between wavelet coefficients of an image. Thus, embedding watermarks according to the numerical relationship of image data or coefficients seems to propose a new direction of watermark detection before the image data were all received. In the following, we will propose a new progressive watermarking embedding and detection approach.

4.3.1 Watermark embedding

In this section, a binary valued watermarking embedding approach for copyright protec-tion is introduced. As shown in Fig. 4.1, W = [w1, w2, ..., wi, ..., wL]T is a binary valued

watermark where wi ∈ {0, 1} for i = 1, ..., L and L is the length of the watermark W .

First, a random seed S is selected to generate N + 1 of random numbers for each bit wi

in W . These random numbers are used as indices to address N + 1 middle band wavelet coefficients Ci = [ci,0, ci,1, ..., ci,N]T of an image. The first coefficient ci,0 is called mark

proposed method of embedding each bit wi of W into an image is to modify the selected

wavelet coefficients Ci into Ci′ = [c ′ i,0, c

′ i,1, ..., c

′

i,N]T according to Eq. 4.1, where c ′ max = max(c′ i,1, ..., c ′ i,N) and c ′ min = min(c ′ i,1, ..., c ′ i,N).    ifwi = 1 c′i,0 ≥ c′ max+c′min 2 ; ifwi = 0 c′i,0 < c′ max+c′min 2 . (4.1)

(a) The numerical relationship of original wavelet coefficients Ci

(b) A feasible watermark embedding method for wi= 1

(c) Another feasible watermark embedding method for wi= 1

Figure 4.2: An example of the proposed watermark embedding approach.

In the following, C′

example of numerical relationship of original wavelet coefficients of Ci, and Figs 4.2(b)

and 4.2(c) show two different watermark embedding results for wi = 1. As shown in

Fig. 4.2, for N = 5, Ci = [ci,0, ci,1, ..., ci,N]T = [−1, 5, −3, −8, 9, 3]T, where ci,min = −8,

ci,max = 9. Suppose a watermark bit wi = 1 is to be embedded into Ci, then Ci can be

modified as either C′

i = [1, 5, −3, −8, 9, 3]T, or C ′

i = [0, 5, −3, −9, 8, 3]T. Since there are

many choices of C′

i to satisfy Eq. 4.1, thus we propose to use optimization methods [20]

to select a C′

i, such that a watermarked image with best visual quality [2] can be achieved.

4.3.2 Watermark detection

In this section, we proposed a new progressive watermark detection method for various progressive image transmission schemes, such as: (1) successive approximation, (2) TS-Based method, (3) hybrid method of (1) and (2).

As shown in Fig. 4.3, the detector first checks to see whether or not the received image is transmitted in a progressive manner. If it is not, then the whole image will be received and then passed to non-progressive watermark detection procedure. If the image is transmitted progressively, then partial image will be passed to progressive watermark detection procedure.

Non-progressive watermark detection:

First, the same random seed S (as stated in Section 4.3.1) are used to generate N +1 indices to address N +1 middle band wavelet coefficients, C′′

i = [c ′′

i,0, c′′i,1, ..., c′′i,N]T from the received

image to calculate c′′ max = max  c′′ i,1, c ′′ i,2, · · · , c ′′ i,N  and c′′ min = min  c′′ i,1, c ′′ i,2, · · · , c ′′ i,N  . Then, a possible watermark w′

i can be derived from Eq. 4.2:

w′ i =    1 if c′′ 0 ≥ c′′ max+c′′min 2 0 if c′′ 0 < c′′ max+c′′min 2 . (4.2) Finally, check if w′

i is equal to wi for 1 ≤ i ≤ L. If the number of matched watermark

bits are larger or equal to a predetermined threshold Tw, then the received image can be

Progressive watermark detection:

This method uses partially detected watermark coefficients to estimate embedded water-marks. As shown in Section 4.3.1, the proposed watermark embedding method use the numerical relationship between ci,0 and 1/(2 × (ci,min + ci,max)) to decide how to

mod-ify the value of ci,0. Detecting the watermarks can be performed in a similar manner.

However, detecting progressively transmitted watermarks wi needs to find the range

re-lationship from partially received data of c′′

i,0 and 1/(2 × (c ′′

i,min+ c ′′

i,max)). Suppose the

first m-th most significant bits of watermarked wavelet coefficients are received, and the same random seed S is used as the indices to select N + 1 watermarked coefficients, C′′ i = [c ′′ i,0, c ′′ i,1, · · · , c ′′ i,j, · · · , c ′′

i,N]T. Let the range of each coefficient c ′′ i,j be (l ′′ j, h ′′ j), if the

following Eq. 4.3 holds, then the watermark wi can be estimated:

w′ i =    1 if l′′ 0 ≥ h′′ max+h′′min 2 0 if h′′ 0 < l′′ max+l′′min 2 , (4.3) where l′′ max = max  l′′

i,1, l′′i,2, · · · , l′′i,N

 , h′′ max= max  h′′ i,1, h ′′ i,2, · · · , h ′′ i,N  , l′′ min = min  l′′ i,1, l ′′ i,2, · · · , l ′′ i,N  , and h′′ min = min  h′′

i,1, h′′i,2, · · · , h′′i,N



. As shown in Fig.4.4(a), the ranges of c′′

i,0 and 1/(2×(c′′i,min+c ′′

i,max)) are not overlapped and

the range of c′′

i,0 is to the right of the range of 1/(2 × (c ′′

i,min+ c ′′

i,max)), thus the value of wi

(=1) can be estimated. However, in Fig. 7(b), the ranges of c′′

i,0 and 1/(2×(c ′′ i,min+c

′′ i,max))

are overlapped to each other, and thus more lower bits of watermarked data are needed to separate the ranges of c′′

i,0 and 1/(2 × (c ′′

i,min+ c ′′ i,max)).

When the number of matched watermark bits is equal or larger than the predetermined threshold Tw, then the received image can be determined to contain the target watermark,

ever through the whole image has not been completely received. On the other hand, if the number of mismatched watermark bits is larger than L − Tw, it can be inferred that

(a) A detectable situation

(b) A not yet detectable situation

Figure 4.4: Examples of progressive watermark detection when partial watermarked image data are received

4.4

Experimental results

In order to evaluate the performance of the proposed watermarking system, two types of experiments were exercised: (1) robustness against image processing operations and (2) early watermark detection under the circumstance of progressive image transmission. In the first type of experiments, we use some commonly used image processing operations such as lossy compressions, Gaussian blurring and sharpening [12] to attack watermarked images, and then check whether or not the watermark can be detected. These experiments can briefly demonstrate the robustness of the proposed watermarking method. In the experiments for early watermark detection, watermarked images were firstly compressed

by common compression standards such as JPEG and JPEG2000. Then, the experiments evaluate: (1) the percentage of watermark bits detected during progressive transmission, and (2) the amount of image data required to confirm the presence of a watermark. We also conduct the experiments on images without watermark and evaluate the amount of image data required to infer the absence of a watermark (i.e., earlier rejection).

4.4.1 Experimental setting

In the experiments, we randomly generate a binary string containing 1000 bits as a wa-termark (i.e., L = 1024). All wavelet coefficients used to embed wawa-termark bits are randomly selected from LH2, HL2 and HH2, and six coefficients are used for embedding each watermark bit (i.e., N = 5). While the watermark is to be embedded into 8 × 8 DCT coefficients of images, six coefficients are randomly chosen from AC6 to AC15 of all 8 × 8 DCT blocks. The watermark strength δ are always set as 1. Six images: Lena, Baboon, F16, Fishing Boat, Sailboat and Peppers are used as test image through all the experiments.

4.4.2 Robustness of the proposed progressive watermarking scheme

To evaluate the robustness of the proposed approach, the six test images were water-marked with the proposed watermarking approach. Then, four image processing oper-ations, JPEG compression, JPEG2000 compression, Gaussian filtering, and sharpening are applied on the watermarked images. The result images are shown in Fig. 4.5 to Fig. 4.10. The experimental results of applying four image processing operations are depicted in Fig. 4.11, Fig. 4.12 and Table 4.1.

As shown in Fig. 4.11, Fig. 4.12 and Table 4.1, even visual quality of the test image is largely decreased by the image processing operations, the watermark embedded in the image can still be detected by the proposed method. Thus, if someone tries to remove the watermark embedded in an image, the image processing operations will cause huge decrease to visual quality of the image such that the image will be of no commercial value.

(a) Result image after JPEG compression. The PSNR com-paring to watermarked Lena image is 33.91. The percentage of correct watermark bits is 60.54%.

(b) Result image after JPEG2000 compression. The PSNR comparing to watermarked Lena image is 37.47. The per-centage of correct watermark bits is 62.69%.

(c) Result image after Gaussian filtering. The PSNR com-paring to watermarked Lena image is 31.47. The percentage of correct watermark bits is 75.39%.

(d) Result image after sharpening. The PSNR comparing to watermarked Lena image is 31.74. The percentage of correct watermark bits is 99.9%.

Figure 4.5: The result images of applying four common image processing operations on watermarked Lena image.

(a) Result image after JPEG compression. The PSNR com-paring to watermarked Baboon image is 23.88. The percent-age of correct watermark bits is 63.28%.

(b) Result image after JPEG2000 compression. The PSNR comparing to watermarked Baboon image is 32. The per-centage of correct watermark bits is 94.53%.

(c) Result image after Gaussian filtering. The PSNR com-paring to watermarked Baboon image is 21.37. The per-centage of correct watermark bits is 60.64%.

(d) Result image after sharpening. The PSNR comparing to watermarked Baboon image is 17.53. The percentage of correct watermark bits is 94.62%.

Figure 4.6: The result images of applying four common image processing operations on watermarked Baboon image.

(a) Result image after JPEG compression. The PSNR com-paring to watermarked F16 image is 30.39. The percentage of correct watermark bits is 60.25%.

(b) Result image after JPEG2000 compression. The PSNR comparing to watermarked F16 image is 31.56. The per-centage of correct watermark bits is 61.03%.

(c) Result image after Gaussian filtering. The PSNR com-paring to watermarked F16 image is 26.6. The percentage of correct watermark bits is 68.84%.

(d) Result image after sharpening. The PSNR comparing to watermarked F16 image is 25.42. The percentage of correct watermark bits is 99.21%.

Figure 4.7: The result images of applying four common image processing operations on watermarked F16 image.

(a) Result image after JPEG compression. The PSNR com-paring to watermarked Fishing Boat image is 28.67. The percentage of correct watermark bits is 61.03%.

(b) Result image after JPEG2000 compression. The PSNR comparing to watermarked Fishing Boat image is 31.76. The percentage of correct watermark bits is 67.38%.

(c) Result image after Gaussian filtering. The PSNR com-paring to watermarked Fishing Boat image is 25.59. The percentage of correct watermark bits is 66.99%.

(d) Result image after sharpening. The PSNR comparing to watermarked Fishing Boat image is 23.56. The percentage of correct watermark bits is 98.82%.

Figure 4.8: The result images of applying four common image processing operations on watermarked Fishing boat image.

(a) Result image after JPEG compression. The PSNR com-paring to watermarked Sailboat image is 28.48. The per-centage of correct watermark bits is 60.54%.

(b) Result image after JPEG2000 compression. The PSNR comparing to watermarked Sailboat image is 31.98. The percentage of correct watermark bits is 78.41%.

(c) Result image after Gaussian filtering. The PSNR com-paring to watermarked Sailboat image is 25.16. The per-centage of correct watermark bits is 64.35%.

(d) Result image after sharpening. The PSNR comparing to watermarked Sailboat image is 23.15. The percentage of correct watermark bits is 99.31%.

Figure 4.9: The result images of applying four common image processing operations on watermarked Sailboat image.

(a) Result image after JPEG compression. The PSNR com-paring to watermarked Peppers image is 31.7. The percent-age of correct watermark bits is 60.54%.

(b) Result image after JPEG2000 compression. The PSNR comparing to watermarked Peppers image is 33.97. The percentage of correct watermark bits is 57.12%.

(c) Result image after Gaussian filtering. The PSNR com-paring to watermarked Peppers image is 28.5. The percent-age of correct watermark bits is 71.28%.

(d) Result image after sharpening. The PSNR comparing to watermarked Peppers image is 26.34. The percentage of correct watermark bits is 99.31%.

Figure 4.10: The result images of applying four common image processing operations on watermarked Peppers image.

12 13 14 15 16 17 18 19 20 54 56 58 60 62 64 66 68 70 72

JPEG quality factor

% of correct watermark bits Baboon

F16 Fishboat Lena Peppers Sailboat

Figure 4.11: The plot of percentage of correct watermark bits versus JPEG quality factor. Six water-marked images were compressed by JPEG.

30 32 34 36 38 40 42 44 46 48 50 55 60 65 70 75 80 85 90 95 100 PSNR of images compressed by J2K

% of correct watermark bits Baboon

F16 Fishboat Lena Peppers Sailboat

Figure 4.12: The plot of percentage of correct watermark bits versus PSNR. Six watermarked images were compressed by JPEG2000.

Table 4.1: Experimental results of watermark detection after Gaussian filtering and sharpening. Baboon F16 FishBoat Lena Peppers Sailboat 3 × 3 Gaussian filtering PSNR 23.36 30.84 28.92 36.52 31.86 28.89 % of correct bits 76.6 83.7 83.5 92.1 91.5 82.4 5 × 5 Gaussian filtering PSNR 21.37 26.60 25.59 31.47 28.50 25.16 % of correct bits 60.6 68.8 66.9 75.3 71.2 64.3 Sharpening PSNR 17.53 25.42 23.56 31.74 26.34 23.15 % of correct bits 94.6 99.2 98.8 99.9 99.3 99.3

These experimental results show that the proposed watermarking method is robust and can be used on the applications of copyright protection.

4.4.3 Earlier detection of the proposed progressive watermarking scheme In this experiment, we embed watermarks into 8 × 8 DCT coefficients of the six test images, and then compress these images by common JPEG progressive modes (i.e., hybrid method). Then, the percentage of detectable watermark bits versus the percentage of received image data is evaluated. On the other hand, we embed watermarks into DWT coefficients of the six test images and compress these images by JPEG200. The percentage of detectable watermark bits versus the percentage of received image data is shown in Fig. 4.13.

As shown in Fig. 4.13, if Tw is equal to 0.6, it can be confirmed that the image is

contained the watermark when about 15% to 35% image data is received. Moreover, the result shows that earlier detection of JPEG2000 images are significantly superior to JPEG image. This result may change when watermark strength δ or the subband chosen for watermark embedding is different. The larger the δ is, the earlier the embedded watermark can be detected. However, a large δ will decrease visual quality of watermarked images significantly. Embedding watermarks into low-frequency wavelet subband can also reduce the amount of image data required for progressive watermark detection; however, the watermark payload (i.e., coefficients that can be used to embed watermark) will also