A Novel Image Recovery Algorithm for Visible Watermarked Images

(1)

Correspondence

A Novel Image Recovery Algorithm for Visible Watermarked Images Soo-Chang Pei and Yi-Chong Zeng

Abstract—A novel image recovery algorithm for removing visible

water-marks is presented. Independent component analysis (ICA) is utilized to separate source images from watermarked and reference images. Three in-dependent component analysis approaches are examined in the proposed algorithm, which includes joint approximate diagonalization of eigenma-trices, second-order blind identification, and FastICA. Moreover, five dif-ferent visible watermarking methods to embed uniform and linear-gra-dient watermarks are implemented. The experimental results show that visible watermarks are successfully removed, and that the proposed algo-rithm is independent of both the adopted ICA approach and the visible watermarking method. In the final experiment, several public domain im-ages sourced from various websites are tested. The results of this study demonstrate that the proposed algorithm can blindly and successfully re-move the visible watermarks without knowing the watermarking methods in advance.

Index Terms—Image recovery, independent component analysis (ICA),

visible watermark removal.

I. INTRODUCTION

Various image recovery schemes have been developed recently and used in digital photograph restoration [1], [2], ancient painting restora-tion [3], and visible watermark removal [4]. Image recovery attempts to fill the selected area with the appropriate textures, where the undesired object was initially located. There are two kinds of the undesired ob-ject, which are: 1) the undesired solid object occluding the background and 2) the undesired transparent object merging with the background.

Reviewing previous studies on solid objects removal, Bertalmio et al. developed the image-inpainting approach [1] for filling a selected area with the surrounding pixels. Sun et al. suggested a structure propaga-tion method for image complepropaga-tion, and adopted patch-based texture synthesis to restore selected areas [2]. A texture synthesis scheme is also utilized in the virtual restoration of ancient Chinese paintings [3] by adding some auxiliaries to improve the method. In the transparent object removal, Huang and Wu employed the image-inpainting ap-proach to remove visible watermarks [4]. However, the iterative process of image-inpainting is costly and time-consuming.

This work presents a fast, simple, and efficient image recovery al-gorithm for removing visible watermarks. The rest of this paper is or-ganized as follows. Section II briefly introduces the related methods. Section III then describes the visible watermark removal algorithm.

Manuscript received November 11, 2005; revised May 3, 2006. This work was supported by the National Science Council of Taiwan, R.O.C., under Con-tract NSC94-2213-E-002-072 and ConCon-tract NSC93-2752-E-002-006-PAE. The assoicate editor coordinating the review of this manuscript and approving it for publication was Prof. Mohan S. Kankanhalli.

S.-C. Pei is with the Department of Electrical Engineering, National Taiwan University, Taipei 10617, Taiwan, R.O.C. (e-mail: pei@cc.ee.ntu.edu.tw).

Y.-C. Zeng is with the Graduate Institute of Communication Engi-neering, National Taiwan University, Taipei 10617, Taiwan, R.O.C. (e-mail: d89942010@ntu.edu.tw).

Color versions of Figs. 1 and 6–8 are available online at http://ieeexplore. ieee.org.

Digital Object Identifier 10.1109/TIFS.2006.885031

Section IV presents the experimental results. Conclusions are drawn in Section V.

II. RELATEDMETHODS A. Visible Watermarking Methods

A visible watermark represents the owner of the product. Many on-line images, digital documents, and video are embedded with visible watermarks. Braudaway et al. embedded visible watermarks to protect public images [5]. They formulate the nonlinear equation to accom-plish the luminance alteration on the pixel domain, and then the wa-termark is placed onto the image. In addition, various parameters are adopted in the nonlinear equation in order to make the watermark dif-ficult to remove. Meng and Chang added visible watermarks to video sequence in the discrete cosine transform (DCT) domain [6]. To ex-tend the Braudaway’s method, they developed the simple stochastic ap-proximation model on the DCT domain and modified the original non-linear equation. The feature is to directly implement the adaptive wa-termarking technique to the compressed video steams. Mohanty et al. proposed a watermarking method in the DCT domain [7] by com-bining a visible and invisible watermarks as a dual watermark to be embedded into an image [8]. Hu and Kwong implemented an adap-tive visible watermarking in the wavelet domain [9]. Both the host image and watermark are first decomposed to a four-level multireso-lution wavelet-based structure. In each subband, the scaling factors of the wavelet coefficients are calculated, and then the coefficients of the watermark are imposed on those of the host image with the scaling fac-tors. Chen developed a visible watermarking mechanism in the pixel domain based on the statistical approach [10]. The mechanism is dif-ferent from Braudaway’s method in the watermarked ratio, which is determined by the standard deviation of the block. Regardless of ex-ploiting the visible watermarking technique, the watermarked image can generally be formulated as

yw(i; j) = k1(i; j)x(i; j) + k2(i; j)w(i; j) (1) wherex(i; j); yw(i; j) and w(i; j) are the (i; j)th pixels of the host image, the watermarked image, and the visible watermark, respectively, and k1(i; j) and k2(i; j) are the two weighting factors. In most of the visible watermarking methods, all weighting factors are usually variable.

B. Independent Component Analysis (ICA)

ICA is also called blind signal separation (BSS). It is a statistical and computational scheme that yields the latent correlation of a set of random variables or signals. The simplest independent component analysis (ICA) model assumes thatm sources (s1; s2; . . . ; sm) gen-eratem mixtures (t1; t2; . . . ; tm)

tk= ak;1s1+ ak;2s2+ 1 1 1 + ak;msm (2) whereak;ldenotes the mixing value of thelth independent source in thekth mixture, and 1 k; l m. The ICA estimates the separating valuebk;l, such that the approximate sources^skcan be separated from the mixtures, and^s_k= b_k;1t₁+ b_k;2t₂+ 1 1 1 + b_k;mt_m.

Cardoso and Souloumiac presented a blind identification algorithm by the joint approximate diagonalizaition of eigenmatrices (JADE) to estimate the separating matrix [11]. FastICA, proposed by Hyvarinen

(2)

544 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 1, NO. 4, DECEMBER 2006

Fig. 1. Segment the watermark area using the seed-growing algorithm. (a) The watermarked image. (b) The marked points for seed growing are added on the watermarked image as indicated. (c) The segmented-watermarked area.

and Oja [12], is an ICA approach with fast convergence. Belouchrani et al. exploited stationary second-order statistics based on a joint diag-onalization of a set of covariance matrices to solve the ICA problem, which is called second-order blind identification (SOBI) [13].

To analyze (1), the host image to be separated from the watermarked image corresponds to solve the ICA problem of two independent sources in (2). However, only a mixture (the watermarked imagey_w) is available, another mixed image (the reference imageyf) must be generated for the ICA. Section III further discusses those details.

III. VISIBLEWATERMARKREMOVALALGORITHM(VWR) The proposed VWR has three phases: watermarked area segmenta-tion, reference image generasegmenta-tion, and image recovery. These phases are described below.

A. Watermarked Area Segmentation

Before the image recovery algorithm is implemented, the visible wa-termark must be segmented. First, the user sets red and blue marked points manually to represent the watermarked and the nonwatermarked pixels, respectively. For instance, a watermarked image is shown in Fig. 1(a) and the marked points for seed growing are added on the wa-termarked image as shown in Fig. 1(b). Subsequently, the seed-growing algorithm is exploited to turn all pixels into the marked points. Let p(i; j) be the (i; j)th marked point and pN(i; j; n) be the nth neighbor unmarked point ofp(i; j). The intensity difference between p(i; j) and pN(i; j; n) is defined as

di(I(p(i; j)); I(pN(i; j; n))) = jI(p(i; j)) 0 I(pN(i; j; n))j (3) whereI(p) denotes the intensity of p. If the intensity distance is smaller than the threshold , then pN(i; j; n) will be designated as a red marked point whenp(i; j) is a red marked point, or p_N(i; j; n) will be desig-nated as the blue marked point sincep(i; j) is a blue marked point. However, the seed-growing algorithm may be stagnated; the solution is to increase the threshold with the increment 1 (i.e., +1 ). Fig. 1(c) represents the segmented-watermarked area.

B. Reference Image Generation

Based on the description in Section II-B, the simplest ICA model requires at least two mixtures to estimate the two independent sources. However, the first mixture is the watermarked imageywderived from (1), but the second mixture is none and unknown in our case. Accord-ingly, the reference imagey_fmust be generated as the second mixture; it is proposed as a mix of watermarked imageywand estimated visible watermarkw^

yf(i; j) = k30(i; j)yw(i; j) + k04(i; j) ^w(i; j) (4)

wherek0₃(i; j) and k0₄(i; j) are the two weighting factors k₃0(i; j) + k0

4(i; j) = 1. Furthermore, k30(i; j) is defined as

k0

3(i; j) = 0;_{1; elsewhere.}if the(i; j)th pixel locates at the watermarked area (5) This work considers two kinds of visible watermarks: one is the uni-form watermark and the other is the linear-gradient watermark. There-fore, the reference image is generated using two different estimation methods. Asy_w is embedded with the uniform watermark, the esti-mated visible watermark is designed to be a constant image, that is

^

w(i; j) = 1. For a K 2 L linear-gradient watermark, the estimated visible watermark is designed as

^

w(i; j) = r 2 f(i; j; ) = r 2 i 0 1_{K 0 1}cos + j 0 1_{L 0 1}sin (6) wherer is a factor that adjusts the intensity slope, is the angle of inclination between the direction of the gradient and the horizontal axis, 0 < 360; 1 i K, and 1 j L. The estimated visible watermark is an 8-bit gray-level image and the maximum ofjf(i; j; )j in (6) isp2, so r satisfies 0 < r 255=p2.

If the visible watermark is successfully removed, the recovered pixels will be close to the neighboring nonwatermarked pixels. Hence, the error between the pixel values of recovered pixels and those of the neighboring nonwatermarked pixels is measured in order to estimate the proper slope r and the angle for the linear-gradient visible watermark. The difference function is formulated as

DF = 6i6j

(i;j)2cjyr(i; j) 0 P(i; j)j

2 ₍₇₎

wherey_r(i; j) is the pixel value of the (i; j)th recovered pixel, and P(i; j) is an average of all nonwatermarked pixel values in a 5 2 5 windowP, centered at position (i; j). Let ccc be the set of coordinate pairs in the interior areaMint

Mint= Mw0 (Mw9 S1) (8)

whereS₁ is a 32 3 unitary block and the symbol “9” is the mor-phology erosion operation. For example, Figs. 2(a) displays a water-marked image andM_w and M_int are the watermarked and interior areas, which are shown in Fig. 2(b) and (c), respectively. Fig. 3 presents the difference calculation according to (7). Whileyr(i; j) is close to P(i; j), the difference DF is low, whereas the boundary between the watermarked and nonwatermarked areas is indistinguishable. In con-trast, the high-difference DF indicates that the boundary between the watermarked and nonwatermarked areas is noticeable.

Initially, 432 reference images are generated with 12 slopes and 36 angles(r = 612_k=1152k and = 635_k=0102k), and these images are applied to VWR . To calculate the differences of 432 recovered images by (7), the first proper parametersr1and1correspond to the recovered image with minimum difference. Subsequently, 35 reference images with five slopes and seven angles (r = r1 + 62k=023 2 k and = 1+63k=0332k) are generated for VWR. To calculate the differences of 35 recovered images by (7), the second proper parametersr₂ and 2 correspond to the recovered image with the minimum difference. Finally, 25 reference images with 5 slopes and 5 angles (r = r₂ + 62

k=02k and = 2 + 62k=0212 k) are generated for VWR. To calculate the differences of 25 recovered images by (7), the final proper parametersrnal and nal correspond to the recovered image with the minimum difference as well. Then, not onlyr_naland_nalof the estimated linear-gradient watermark are found, but the image is also

(3)

Fig. 2. (a) Watermarked image. (b) Watermarked areaM . (c) Interior area M . (d) Exterior area M .

Fig. 3. Diagram presents the difference calculation according to (7). A 52 5 windowP(i; j) consists of the nonwatermarked and recovered pixels.

recovered. A total of 492 reference images is generated in the above iterative process.

C. Image Recovery

Since the watermarked image is a mixture of the host image and the visible watermark, the reference image is generated to be a mix of the watermarked image and estimated visible watermark as in (4). The reference image can also be expressed as another mixture form of the host image and the visible watermark

yf(i; j) = k03(i; j)yw(i; j) + k04(i; j) ^w(i; j) = k0

3(i; j)[k1(i; j)x(i; j) + k2(i; j)w(i; j)] + k04(i; j) ^w(i; j) = k3(i; j)x(i; j) + k4(i; j)w(i; j) (9)

Fig. 4. Results of uniform watermark removal. The left-hand column presents the watermarked images shown in (a), (c), (e), (g), and (i) by the following five embedding algorithms, which include Braudaway’s, Meng’s, Mohanty’s, Hu’s, and Chen’s methods. The right-hand column shows that the recovered images by our proposed VWR algorithm correspond to the left-hand column images. The image recovery process takes an average of 1.8 s, and the PSNRs of those recovered images are 48.84, 47.24, 34.38, 43.95, and 37.44 dB for (b), (d), (f), (h), and (j), respectively.

wherek₃0(i; j) = k₃0(i; j)k1(i; j) and k4(i; j) = k03(i; j)k2(i; j) + k0

4(i; j) ^w(i; j)=w(i; j). When the watermarked image ywand the ref-erence imagey_f are obtained, the ICA is utilized to estimate a 22 2

(4)

Fig. 5. Proposed algorithm recovers the linear-gradient watermarked image. (a) Watermarked image withPSNR = 30:97 dB. (b) Linear-gradient watermark withr = 12 and = 220 . (b) Recovered image with PSNR = 47:22 dB in 783.30 s (r = 8 and = 233 ).

TABLE I

MEANS()ANDVARIANCES( )OF THEORIGINAL, WATERMARKED,

ANDRECOVEREDAREASASSOCIATEWITH THEUNIFORM ANDLINEAR-GRADIENTWATERMARKS

separating matrixB, such that the two source images X1andX2can be separated X1 X2 = B yw yf andB = b1 b2 b3 b4 : (10)

The issue in which the two separated source images should be applied to recover the watermarked image is of concern. By the heuristic exper-iment with 50 images, the solution is determined by two coefficients of B; b1andb3. Ifjb1j jb3j, then X1is the appropriate source image; on the contrary,X2is the appropriate source image ifjb3j > jb1j. The appropriate source image is called the nonwatermarked source image Xnw.

The mean_nwand variance_nw2 of the watermarked area inX_nw are calculated as nw = i jXnw(i; j)Mw(i; j) i jMw(i; j) and 2 nw = i j (Xnw(i; j) 0 nw)2Mw(i; j) i jMw(i; j) : (11) TABLE II

SEPARATINGMATRICES ANDPSNROF THERECOVEREDIMAGES, VWRISINTEGRATEDWITHTHREEICA APPROACHES TORECOVER

FIVEVISIBLEWATERMARKEDIMAGES

Mextis the exterior area, which surrounds the watermarked area shown in Fig. 2(d)

Mext= (Mw8 S2) 0 Mw (12)

whereS₂is an 112 11 unitary block and the symbol “8” is the mor-phology dilation operation. Letextand2_extbe the mean and variance of the exterior area in the watermarked imagey_w

ext= i jyw(i; j)Mext(i; j) i jMext(i; j) and

2

ext= i j(yw(i; j) 0 out) 2_M

ext(i; j)

i jMext(i; j) : (13)

Although the exterior area surrounds the watermarked area, both areas should have similar means and variances before the visible wa-termark is added to the image. Consequently, the mean and standard deviation of the watermarked area inXnw are adjusted, and the re-covered image is defined in (14), shown at the bottom of the page. If the quality of the recovered image is unsatisfactory, then the recovered image replaces the watermarked image and the above VWR algorithm is iteratively processed again.

yr(i; j) = ext

nw(Xnw(i; j) 0 nw) + ext; if pixel locates at the watermarked area yw(i; j); elsewhere

(5)

Fig. 6. Recovering the public domain image. (a) 7682 512 watermarked image is embedded with a small watermark [17]. (b) Recovered image with a process time of 4.77 s.

IV. EXPERIMENTALRESULTS

The experiments focus on three issues to show that VWR is an ef-fective image recovery algorithm for removing the visible watermark. These three issues include different visible watermarking methods, var-ious ICA approaches, and the blind removal of visible watermarks. The threshold parameters for segmenting the watermarked area are = 8 and1 = 4.

A. Visible Watermarking Methods

The 5122 512 gray-level image is tested in the experiment. First, five test images are embedded with five 5122 512 uniform visible watermarks using the different watermarking methods of Braudaway et al. [5], Meng [6], Mohanty et al. [8], Hu [9], and Chen [10]. JADE [11] of the ICA approach is used in our proposed VWR algorithm. The left-hand column of Fig. 4 shows these five watermarked images,

and the right-hand column of Fig. 4 shows the recovered images. The image recovery process takes an average of 1.8 s, and the peak signal-to-noise ratios (PSNRs) of those recovered images are 48.84, 47.24, 34.38, 43.95, and 37.44 dB for Fig. 4(b), (d), (f), (h), and (j), respectively. In the second experiment, a linear-gradient watermark [shown in Fig. 5(b)] with the sloper = 12 and angle = 220is embedded in the test image by Braudaway’s method. The PSNRs of the watermarked and recovered images are 30.97 and 47.22 dB, and are displayed in Fig. 5(a) and (c), respectively. The process takes 783.30 s and the parameters are estimated asrnal = 8 and nal = 233. Table I lists the means and variances of the original, watermarked, and recovered areas associated with the uniform and linear-gradient watermarks. The mean and the variance of the recovered area are close to those of the original area, and it indicates that the recovered area is similar to the original area.

(6)

Fig. 7. Recovering the public domain image. (a) 10002 688 watermarked image is embedded with a large watermark [18]. (b) Recovered image with a process time of 10.75 s.

(7)

Fig. 8. Recovering the public domain image. (a) 6402 353 watermarked image is embedded with a color watermark [19]. (b) Recovered image with process time in 4.92 s.

B. ICA Approaches

Three ICA approaches, which include JADE, FastICA, and SOBI, are examined in our experiments. TheMATLABsource codes of these approaches are downloaded from [14]–[16]. VWR is integrated with each ICA approach to recover five watermarked images [as shown in Fig. 4(a), (c), (e), (g), and (i)], and 15 recovered images are obtained. Table II lists the separating matrices and PSNR values of these 15 re-covered images. The high PSNR value demonstrates that VWR is inde-pendent of the adopted ICA approaches, and successfully removes the visible watermarks embedded with the currently existing visible water-marking methods.

C. Blind Removal of Visible Watermarks

Numerous public domain images are available at various websites, and some of them are visible watermarks embedded with unknown wa-termarking methods [17]–[21]. In the final experiment, the embedded watermarks are assumed to be uniform, and these visible watermarks are removed successfully and blindly from the public images without knowing the watermarking methods in advance. Fig. 6(a) presents an

image embedded with the small watermark [17], the 7682 512 re-covered image with a process time of 4.77 s is shown in Fig. 6(b). Fig. 7(a) displays an image embedded with the large watermark [18], the 10002 688 recovered image with process time 10.75 s is shown in Fig. 7(b). The final image is shown in Fig. 8(a), which is embedded with the color watermark [19], the 6402 353 recovered image with process time 4.92 s is shown in Fig. 8(b).

V. CONCLUSION

This paper presents a novel image recovery algorithm for removing the visible watermarks. The proposed algorithm not only removes the specified visible uniform and linear-gradient watermarks, but also suc-cessfully and blindly recovers the public domain images. Furthermore, three independent component analysis approaches and five visible wa-termarking methods are examined in the visible watermark removal algorithm, and the experimental results demonstrate that the proposed algorithm is independent of both the adopted ICA approach and the visible watermarking method. All programs are executed inMATLAB software using a 1.5-GHz Pentium-M processor.

(8)

REFERENCES

[1] M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image in-painting,” SIGGRAPH, pp. 417–424, Aug. 2000.

[2] J. Sun, L. Yuan, J. Jia, and H.-Y. Shum, “Image completion with struc-ture propagation,” SIGGRAPH, pp. 861–868, Aug. 2005.

[3] S.-C. Pei, Y.-C. Zeng, and C.-H. Chang, “Virtual restoration of ancient Chinese paintings using color contrast enhancement and lacuna texture synthesis,” IEEE Trans. Image Process., vol. 13, no. 3, pp. 416–429, Mar. 2004.

[4] C.-H. Huang and J.-L. Wu, “Attacking visible watermarking schemes,”

IEEE Trans. Multimedia, vol. 6, no. 1, pp. 16–30, Feb. 2004.

[5] G. Braudaway, K. A. Magerlein, and F. Mintzer, “Protecting publicly available images with a visible image watermark,” in Proc. SPIE, Int.

Conf. Electronic Imaging, Feb. 1–2, 1996, vol. 2659, pp. 126–133.

[6] J. Meng and S.-F. Chang, “Embedding visible video watermarks in the compressed domain,” in Proc. ICIP, Oct. 4–7, 1998, vol. 1, pp. 474–477.

[7] S. P. Mohanty, K. R. Ramakrishnan, and M. S. Kankanhalli, “A dual watermarking technique for image,” in Proc. 7th ACM Int. Multimedia

Conf., Oct./Nov. 1999, vol. 2, pp. 49–51.

[8] S. P. Mohanty, K. R. Ramakrishnan, and M. S. Kankanhalli, “A DCT domain visible watermarking technique for image,” in Proc. IEEE Int.

Conf. Multimedia and Expo., Jul./Aug. 2000, vol. 2, pp. 1029–1032.

[9] Y. Hu and S. Kwong, “Wavelet domain adaptive visible watermarking,”

Electron. Lett., vol. 37, no. 20, pp. 1219–1220, Sep. 2001.

[10] P.-M. Chen, “A visible watermarking mechanism using a statistic ap-proach,” in Proc. 5th Int. Conf. Signal Processing, 2000, vol. 2, pp. 910–913.

[11] J. F. Cardoso and A. Souloumiac, “Blind beamforming for non Gaussian signals,” Proc. Inst. Elect. Eng. F, vol. l40, no. 6, pp. 362–370, Dec. 1993.

[12] A. Hyvarinen and E. Oja, “Independent component analysis: Algo-rithms and applications,” Neural Netw., vol. 13, pp. 411–430, May/Jun. 2000.

[13] A. Belouchrani, K. Abed-Meraim, J. F. Cardoso, and E. Moulines, “A blind source separation technique using second order statistics,” IEEE

Trans. Signal Process., vol. 45, no. 2, pp. 434–444, Feb. 1997.

[14] JADE Program. [Online]. Available: http://sig.enst.fr/~cardoso/stuff. html.

[15] FastICA Program. [Online]. Available: http://www.cis.hut.fi/projects/ ica/fastica/.

[16] SOBI Program. [Online]. Available: http://www.tsi.enst.fr/icacentral/ icalistArchive/2004/0727.html.

[17] [Online]. Available: http://www.fvs.co.uk/fvshome/watermhelp.asp. [18] [Online]. Available:

http://www.dlib.org/dlib/december97/ibm/12lot-spiech.html.

[19] [Online]. Available: http://www.biblepicturegallery.com/.

[20] [Online]. Available: http://www.blinkred.com/gallery/static/home. asp.