Thesis Organization

In the remainder of this thesis, the related works about privacy protection in video surveillance, visible watermarking in images, data hiding in images and image steganography are reviewed in Chapter 2. Methods for protecting selected private regions and private motion activities in surveillance videos are described in Chapter 3 and 4, respectively. In Chapter 5, the application of the proposed method to image steganography will be introduced. Finally, conclusions and some suggestions for future works are included in Chapter 6.

12

Chapter 2

Review of Related Works

2.1 Review of Techniques for Privacy Protection in Video Surveillance

In modern times, video surveillance systems have been widely deployed in many circumstances such as office buildings, public transportation stations, residential areas, etc. They not only monitor people in the environment but might also expose some privacy-sensitive information unintentionally. For this reason, privacy protection has become indispensible in video surveillance. Many different approaches have been introduced [1-10]. In this section, we review the existing works on privacy protection in video surveillance.

Senior et al. [1] presented an object-based privacy-preserving video console.

Depending on the end-user access control authorization, the system presents responsively a modified video in which concerned objects are masked out.

Accordingly, privacy-sensitive information will not be exposed but eliminated. Elaine et al. [2] postulated that face recognition techniques, which identify people in a video surveillance scene automatically, increase the violation of privacy, and proposed a privacy-enabling algorithm that guarantees that face recognition software cannot recognize de-identified faces reliably, even though many facial characteristics are preserved. These systems obliterate relevant information such as object tracks or suspicious activities from videos. Following this principle, a plenty of works proposed various methods to modify videos for privacy protection. For example, in Venkatesh

13

et al. [3] a method was presented to remove objects and inpaint the eliminated regions with a background image portion.

The above-mentioned works aim at privacy information modification, but the feasibility of recovering the original video is also important. Dufaux et al. [4] utilized the scrambling of sensitive information by a private key to securely preserve the original video, as shown in Figure 2.1(a). Boult et al. [5] presented a technique to protect privacy information by invertible cryptographic obscuration using a private key, as shown in Figure 2.1(b). With these methods, the scene remains visible but the privacy-sensitive information is unidentifiable. If retrieving the sensitive contents is demanded, the modified region may be extracted to undo the private key to recover original information. However, scrambling or cryptographic obscuration still reveals certain privacy information of the individual, such as shapes, routes, motions, etc.

(a) (b)

Figure 2.1 Examples of privacy protection in video surveillance (a) Dufaux’s video scrambling [4].

(b) Boult’s invertible cryptographic obscuration [5].

In Hung and Tsai [6] and Paruchuri et al. [7], additional data hiding techniques were introduced for privacy protection in video surveillance. The preservation and modification of privacy information are separated by utilizing data hiding techniques.

As shown by Figures 2.2(a) and 2.2(b), these schemes cover privacy-sensitive

14

information with background image parts instead of scrambling them so that the privacy information becomes completely invisible. With regard to privacy data preservation, the major challenge they faced is how to embed the huge amounts of privacy information into the modified video. The maximum protectable region of these methods depends on the embedding capacity of the data hiding technique and the compression rate of the privacy information.

(a) (b)

Figure 2.2 An example of privacy protection by using Hung and Tsai’s method [6]. (a) A privacy-sensitive frame in a video. (b) A privacy-protected frame produced from (a).

2.2 Review of Techniques for Visible Watermarking in Images

From copyright protection to authentication, digital watermarking has been used in a great number of applications [11-16]. In general, digital watermarking techniques can be categorized into two major types: visible and invisible. In invisible watermarking, watermarks are embedded as digital data into host images, and these watermarks is indistinguishable to human visual perception. In visible watermarking, the embedded watermarks are generally clearly visible in images or videos. Typically,

15

visible watermarks are company logos or personal digital signatures which are used to declare the ownership of the digital images.

Both visible and invisible watermarking techniques introduce distortion into the host image during the embedding process. However, in some applications such as medical imaging, remote sensing, and military image analysis, any permanent distortion introduced by watermarking is not acceptable. For this requirement, several reversible watermarking schemes have been proposed which allow users to remove the embedded watermark and recover the original host image losslessly. Most of the existing lossless watermarking algorithms focus on invisible watermarking, whereas there are relatively few results of lossless visible watermarking in the literature.

In [11, 12], some methods were proposed to embed visible watermarks into images with reversible data embedding techniques, which are based on losslessly data compression. The low embedding capacities of these techniques hinder the possibility of embedding large-sized visible watermarks into host images.

In [13, 14, 15], some schemes using deterministic and reversible mapping functions of the pixel values or DCT coefficients of the watermark region were proposed. One advantage of these approaches is that watermarks of arbitrary sizes can be embedded into any host image. However, the types of embedded watermarks are restricted to binary visible ones.

In Liu and Tsai [16], a new approach to lossless visible watermarking was proposed. Two applications of this approach were presented, with opaque monochrome watermarks and non-uniformly translucent full-color ones being embedded respectively into color images, as shown in Figure 2.3. This approach is based on the use of appropriate compound mappings that allow the mapped values to be controllable. The proposed mappings have been proved to be reversible for use in lossless recovery of the original image. In this study, we utilize one of the proposed

16

compound mapping functions, called two-fold one-to-one mapping function, to produce a modified privacy-protected image from a privacy-sensitive image and a background image. With the reversibility, the covered privacy-sensitive image can be retrieved losslessly if necessary.

(a) (b)

Figure 2.3 Watermarked images by using Liu and Tsai’s method [16]. (a) A watermarked image with a monochrome logo. (b) A watermarked image with a non-uniformly translucent full-color watermark.

2.3 Review of Techniques for Data Hiding via Images

Information hiding has recently played an important role in many applications of information and network security. This technique refers to a process of inserting information bits into cover media without introducing perceptible artifacts in the resulting stego-media. For example, Wu and Tsai [17] presented a data hiding method for gray-valued cover images. They utilized the difference of two pixel values to embed secret data. According to the characteristic of human visual perception, pixels in edged areas can tolerate larger changes of pixel values before the changes are

17

detected visually than those in the smooth areas. Based on this principle, they embedded more secret data in the edged areas in cover images than in the smooth areas to keep the changes in the modified image unnoticeable.

With regard to information hiding in digital images, the most well-known method is to modify the least-significant-bits (LSBs) of the pixels of the cover image.

A number of data hiding techniques in images are based on the concept of LSB modification. A drawback of traditional LSB modification techniques is that the cover image data are changed and cannot be recovered. Several researches were conducted to implement reversible LSB modification techniques.

In [18], Tian proposed a reversible data embedding method by using a difference expansion technique which is a simple reversible integer transformation. This method calculates the differences of neighboring pixel values, and selects some difference values for the difference expansion. Thereafter, information is embedded into the expanded differences. Since the modified values are generated from the differences between manipulated pixel pairs, the original pixel values can be recovered easily. In [19], Alattar extended Tian utilized vectors instead of pixels for difference expansion to increase the hiding ability and the computation efficiency of the previous method.

In [20], Celik et al. presented a lossless generalized-LSB data embedding method, called a G-LSB method, which is also based on LSB modifications. The cover image is quantized and by subtracting the result from the cover image, a residue image is obtained. Then, the residues are compressed by a lossless compression algorithm. The compressed residues and the embedded data are concatenated and then embedded into the quantized cover image by the G-LSB scheme. Lossless recovery of the original image is achieved by extracting the compressed residues and reconstructing the original residues along with the quantized cover image.

All above-mentioned methods incorporate a lossless data compression stage.

18

There are some schemes that do not rely on additional data compression, for instance, the circular histogram interpretation schemes proposed in [21]. But this method has the drawback of low data embedding. In [22], Coltuc and Chassery presented a high-capacity data embedding scheme without using any additional data compression operation. This scheme is based on a reversible contrast mapping (RCM) scheme, which is a simple integer transform defined on pairs of pixels. It partitions the cover image into pairs of pixels and divides the pairs into three groups, and then conducts respective embedding process on each pair group. This RCM scheme provides almost similar embedding bit-rates when compared to the difference expansion approach, while it has a considerably lower mathematical complexity. In this study, we embed a recovery sequence into a camouflage image by this scheme. In order to retrieve the privacy-sensitive image from the camouflage image and the background image, the method designed for embedding the recovery sequence must be reversible and has a high data embedding capacity, as is done in this study.

2.4 Review of Techniques for Image Steganography

The word steganography is originally derived from Greek words which means

‘‘Covered Writing.’’ It has been used in various forms for thousands of years. For example, in 1945 the Morse code was concealed in a drawing, as shown in Figure 2.4.

The hidden information is encoded onto the stretch of grass alongside the river. The long grass denoted a line and the short grass denoted a point. The decoded message reads: “Compliments of CPSA MA to our chief Col Harold R” [27].

19

Figure 2.4 Concealment of Morse code (1945) in a picture. The hidden information is encoded onto the grass lengths alongside the river.

Digital technology gives this ancient art new ways to hide images and information. Digital image steganography and its derivatives are growing in use. This technique differs from cryptography. The former is to hide information so that we cannot see the secret information while the latter is to scramble information so that we cannot read what we see. Unlike data hiding and digital watermarking, the main goal of steganography is to create complete imperceptibility.

Image steganography is a process that disguises a secret image as a cover image.

It can be used for delivering secret images such as those of confidential documents, military maps, etc. The simplest idea is to embed a secret image into a cover image with a data hiding technique. Nevertheless, the information of an image usually generates a large quantity of data to be embedded. Several researches aimed at dealing with this issue.

In [23], Wu and Tsai presented a method that utilizes the property of grey-value similarity among adjacent pixels to embed a secret image into a cover image. A stego image is produced by replacing the difference image of the cover image with the difference image of the secret image and inversing the difference image of the stego image. The process preserves the secret image with no loss and produces the stego

20

image with low degradation.

In [24], Wang and Chen presented an image steganography method that utilizes a two-way block-matching procedure to search for the highest similarity block for each block of the secret image. The indexes of the secret blocks are obtained in a block-matching procedure and recorded in the least significant bits of the cover image.

Recently, Lai and Tsai [25] created a new type of mosaic image, called secret-fragment-visible mosaic image, which is composed of the fragments of a secret image, as shown in Figure 2.5. In the method, firstly a target image similar to the secret image is chosen from a database. Then, the secret image is divided into many tile images which then are rearranged to fit the target image to create a so-called secret-fragment-visible mosaic image. Finally, a recovery sequence of the rearrangement information is embedded into the mosaic image by a lossless LSB-modification scheme. With the recovery sequence, the secret image can be retrieved quickly and easily.

(a) (b)

Figure 2.5 An example. (a) A secret image. (b) A result secret-fragment-visible mosaic image created with (a) as a source image.

21

Chapter 3

Protection of Selected Private Regions in Surveillance Videos Using a Reversible

Prediction-based Mapping

3.1 Introduction

With the public increasingly concerned about personal privacy protection issues, it is desired to develop privacy protection methods for use in video surveillance systems. We propose a method for privacy protection of selected private regions in image frames of surveillance videos to deal with this issue in this study. The method will be described in detail in this chapter.

In Section 3.1.1, the related problem definitions are given. In Section 3.1.2, the basic idea of the proposed method is described. The principle behind the proposed method is based on the concept of reversible prediction-based mapping, which we describe in Section 3.2. A modified version of the mapping proposed in this study is also presented. Detailed algorithms for private region concealment and recovery based on the principle are presented in Section 3.3. Experimental results showing the feasibility of the method are given in Section 3.4. Finally, a brief summary and some discussions are given in the last section of this chapter.

22

3.1.1 Problem definition

As security surveillance becomes increasingly extensive, there is a growing concern that surveillance systems pose threats to privacy. Since privacy is highly subjective, varying across cultures and individuals, the method of privacy protection should be adapted as much as possible to suit individual requirements. With regard to this demand, in the proposed method we allow an authorized user to specify a private region R in a surveillance video in advance. The image content in R is defined as a privacy-sensitive image part and not to be revealed to unauthorized people. The goal of the proposed method is to disguise the privacy-sensitive image part as a pre-selected background image part to conceal privacy-sensitive information in the image frames in the surveillance video. In addition, it is hoped that the protected image frames can be restored to include the original privacy-sensitive image part with a secret key as input.

Using traditional data hiding techniques to hide the privacy-sensitive image part may achieve this goal, but such techniques usually are time-consuming and demand large spaces for data embedding. Therefore, we design alternatively a general method for concealing the privacy-sensitive image part imperceptibly and recovering the original content of this image part from the resulting protected image losslessly. In addition, even if a person knows the algorithms implementing the method, he/she still cannot retrieve the privacy-sensitive image part without the secret key. The security of the protected privacy in the surveillance video is thus ensured.

3.1.2 Major idea of proposed method

Camouflage is a method of hiding a secret. It allows a secret, which is embedded

23

visibly in an object, to remain unnoticed. The major idea of the proposed method is inspired by this concept of camouflage, and aims to produce a protected image by disguising a privacy-sensitive image part as a pre-selected background image part in a surveillance video.

Specifically, the proposed method produces a camouflage image by using a prediction-based mapping, which is a deterministic one-to-one (reversible) compound mapping function proposed by Liu and Tsai [16]. The prediction-based mapping function, according to [16], has a good property for our applications here, i.e., for each pixel value Vs in a given image, the similarity between a computed value Vm

obtained from Vs and a target value Vt assigned to Vm is dependent on the similarity between the source value Vs and a prediction value Vp of Vs. With this useful property, it is found in this study that we can integrate a new prediction technique into the prediction-based mapping to make the resulting camouflage image closely resemble the background image in appearance. Another beneficial property of the prediction-based mapping function is its reversibility, which allows us to retrieve losslessly the privacy-sensitive image part from the camouflage image using the pre-selected background image part.

The proposed new prediction technique can estimate more effectively pixel values for use in the prediction-based mapping. Specifically, it uses the property of pixel-value similarity among adjacent pixels and employs a simple edge detection technique coming from the JPEG-LS standard [28]. The detailed algorithms about the proposed methods and the complete processes of concealing and recovering privacy-sensitive image parts will be presented in the following sections.

24

3.2 Principle of Proposed Reversible Prediction-based Mapping

In this study, we introduce a new method for protecting selected private regions in surveillance videos by using a prediction-based mapping function, as mentioned previously. In Section 3.2.1, the adopted reversible prediction-based mapping function is described. In Section 3.2.2, the detail algorithms implementing the function are described. And in Section 3.2.3, a wrap-around problem found in the values of the mapping result is described and a solution is presented.

3.2.1 Adopted reversible prediction-based mapping

The proposed method for privacy protection in surveillance videos is based on the use of the reversible prediction-based mapping proposed by Liu and Tsai [16], which is a deterministic one-to-one compound mapping of values. The details about this compound mapping function are described in the following.

A. A general method for converting a source value to a target value

As proposed in Liu and Tsai [16], a general scheme for converting the value p of a source pixel P, called the source value, in an image to another value c which is usually close to a target value b using a compound one-to-one mapping function f proceeds in the following way. First, a forward one-to-one mapping using a function Fc(p) = p − c is conducted, where c is a parameter of F to be determined. As is well known, the values of the adjacent pixels of P in the image are usually close to p. So, if we compute the average a of them, a will usually be close to p and can be regarded as a prediction of p. Therefore, we can take a as the parameter c of the function Fc above so that Fc(p) = Fa(p) = p − c = p − a which will be called the prediction residue value

25

r of p subsequently. Because a is usually close to p, the prediction residue value r is usually close to zero.

Next, a second one-to-one mapping using another function Fb−1(r) = r + b is performed, which adds r to the target value b, where Fb−1 is the inverse function of F with parameter b. After this step, we take the resulting value, denoted as q, as the output. The overall function of the 2-step mappings results in a compound one-to-one mapping function f such that f(p) = Fb−1(r) = Fb−1(Fa(p)) = r + b = (p − a) + b = q. As stated earlier, r = p − a is usually close to zero so the value q = f(p) will be close to the target value b, creating an effect of steganography. Therefore, we will call q a stego-value. Also, it is obviously that the smaller the prediction residue value r is, the closer the stego-value q is to the target value b.

As an example, suppose that the value p of a source pixel is 130, and the target

As an example, suppose that the value p of a source pixel is 130, and the target