利用3D KINECT影像做資訊隱藏並應用於影像驗證、秘密傳輸及版權保護

國

立

交

通

大

學

多媒體工程研究所

碩 士 論 文

利用 3D KINECT 影像做資訊隱藏並應用於影像驗

證、秘密傳輸及版權保護

Data Hiding via 3D KINECT Images and Its Applications for

Authentication, Covert Communication, and Copyright

Protection

研 究 生：廖為帥

指導教授：蔡文祥 教授

利用 3D KINECT 影像做資訊隱藏並應用於影像驗證、秘密

傳輸及版權保護

Data Hiding via 3D KINECT Images and Its Applications for

Authentication, Covert Communication, and Copyright Protection

研 究 生：廖為帥

Student: Wei-Shuai Liao

指導教授：蔡文祥

Advisor: Prof. Wen-Hsiang Tsai

國 立 交 通 大 學 多 媒 體 工 程 研 究 所

碩 士 論 文

A Thesis

Submitted to Institute of Multimedia Engineering College of Computer Science

National Chiao Tung University in partial Fulfillment of the Requirements

for the Degree of Master

in

Computer Science

June 2013

Hsinchu, Taiwan, Republic of China

i

Data Hiding via 3D KINECT Images and Its

Applications for Authentication, Covert

Communication, and Copyright Protection

Student : Wei-Shuai Liao

Advisor: Wen-Hsiang Tsai

Institute of Multimedia Engineering

College of Computer Science

National Chiao Tung University

ABSTRACT

With the growing popularity of the Internet and KINECT devices, related applications of KINECT devices are increasing in various fields and the images acquired by KINECT devices are more commonly seen in people’s daily life. Protection of the 3D image data which are acquired by KINECT devices becomes an important issue. In this study, we propose three data hiding methods for authentication and copyright protection of KINECT images (including the depth and color images), as well as for covert communication via them.

For authentication, a data hiding method for authenticating KINECT images by embedding authentication signals into the depth and color images acquired with the KINECT device is proposed. The proposed method utilizes the features of KINECT images and the ranges of the pixel values in them to achieve the goal of authenticating the color and depth images as a whole.

ii

For cover communication, a data hiding method for this purpose via KINECT images is proposed, which utilizes depth holes in the depth image acquired by the KINECT device as well as a new interpolation technique to hide secret messages in the depth image.

For protection of the copyright of KINECT images, a method of 3D visible watermarking is proposed for embedding 3D visible watermarks into 3D images, which results from combining depth and color images acquired by the KINECT device. In addition, embedding the 3D visible watermark will cover some regions of the original 3D image. This problem is solved by transforming the original data of the regions into a recovery data sequence, which is then embedded into the watermarked 3D image by two schemes, one being a difference expansion scheme and the other a reversible contrast mapping scheme. In order to prevent malicious users from extracting the recovery information, the recovery data sequence and the embedding locations are both randomized by a secret key.

Good experimental results are also presented to show the feasibility of the proposed methods for the related applications.

iii

利用 3D KINECT 影像做資訊隱藏並應用於影像驗

證、秘密傳輸及版權保護

研 究 生：廖為帥

指導教授：蔡文祥 博士

國 立 交 通 大 學 多 媒 體 工 程 研 究 所

摘要

隨著網路及 KINECT 裝置的普及，在各個領域中與 KINECT 相關的應用也隨著 增加，而在人們日常生活中藉由 KINECT 裝置所擷取到的影像也更為常見。因此 保護這些經由 KINECT 裝置所獲取的 3D 影像資料已成為一個重要的議題。此研究 針對 KINECT 影像(包含深度和彩色影像)提出三個資訊隱藏的方法，來達到影像 驗證、祕密傳輸及版權保護的目的。 在影像驗證的方面，本研究提出了一個資訊隱藏的方法，該方法是基於 KINECT 影像的特性和其像素值範圍，藉由嵌入驗證訊號到影像中的方式，來達 到同時驗證 KINECT 深度及彩色影像的目的。 在祕密傳輸方面，本研究使用 KINECT 影像提出了一個資訊隱藏的方法，該 方法利用 KINECT 深度影像中的 「深洞」(“depth holes”)並使用一新內插技術， 將祕密訊息藏入深度影像中並保持影像品質不變。

在 KINECT 影像的版權保護方面，本研究提出了一個直入 3D 可視浮水印的 方法，該方法嵌入一個 3D 可視浮水印到一個由 KINECT 深度和彩色影像合成的 3D 影像之中，嵌入 3D 可視浮水印時會覆蓋掉原始 3D 影像中的某些區域；為此， 本研究將這些區域的原始資料轉換成一種回復資訊，並使用差值擴張(difference expansion)和反轉式對比應對(reversible contrast mapping)兩方法將該資訊嵌入到 具有浮水印的 3D 影像中。為了防止有心人士取得回復資訊，本研究使用一把祕 密鑰匙，將回復資訊的順序與所嵌入的位置之順序打亂。最後，上述方法經由實

iv

v

ACKNOWLEDGEMENTS

The author is in hearty appreciation of the continuous guidance, discussions, and support from his advisor, Dr. Wen-Hsiang Tsai, not only in the development of this thesis, but also in every aspect of his personal growth.

Appreciation is also given to the colleagues of the Computer Vision Laboratory in the Institute of Computer Science and Engineering at National Chiao Tung University for their suggestions and help during his thesis study.

Finally, the author also extends his profound thanks to his dear mom and dad for their lasting love, care, and encouragement.

vi

CONTENTS

ABSTRACT (in English) ... i

ABSTRACT (in Chinese) ... iii

ACKNOWLEDGEMENTS ... iv

CONTENTS ... v

LIST OF FIGURES ... viii

LIST OF TABLES ... x

Chapter 1

Introduction ... 1

1.1 Motivation ... 1

1.2 General Review of Related Works ... 2

1.3 Overview of Proposed Methods ... 4

1.4 Contributions ... 6

1.5 Thesis Organization ... 7

Chapter 2

Review of Related Works and KINECT Image

Structures ... 8

2.1 Previous Studies of 3D Image Acquisition Using KINECT Devices .... 8

2.2 Previous Studies of Structures of Depth and Color Images Taken by KINECT Devices ... 10

2.3 Review of Techniques for Image Authentication ... 11

2.4 Review of Techniques for Covert Communication via Images ... 12

2.5 Review of Techniques for Copyright Protection of Images ... 13

Chapter 3

Authentication of KINECT Images by Data Hiding

Technique ... 15

3.1 Introduction ... 15

3.1.1 Problem Definition ... 16

3.1.2 Proposed Ideas ... 16

3.2 Generation and Embedding of Authentication Signals ... 18

3.2.1 Authentication Signal Generation ... 19

3.2.2 Embedding of Authentication Signals ... 21

3.3 Authentication of KINECT Images ... 23

3.3.1 Extraction of Authentication Signals ... 24

3.3.2 Detection of Tampering in Protected Depth and Color Images 26 3.4 Repairing and Recovery of Depth and Color Images ... 28

vii

3.4.2 Recovery of Protected Depth and Color Images ... 31

3.5 Experimental Results ... 33

3.6 Discussions and Summary ... 36

Chapter 4

Covert Communication via KINECT Images by

Interpolation at Depth Holes ... 37

4.1 Introduction ... 37

4.1.1 Problem Definition ... 38

4.1.2 Proposed Idea... 38

4.2 Interpolation at Depth Holes for Embedding of Secret Messages ... 39

4.2.1 Transformations of Secret Messages ... 39

4.2.2 Interpolation at Depth Holes ... 41

4.3 Extraction of Secret Messages ... 44

4.3.1 Extraction of embedded data from depth image ... 44

4.3.2 Transformation of the extracted data ... 46

4.4 Experimental Results ... 47

4.5 Discussions and Summary ... 49

Chapter 5

Copyright Protection of KINECT Images by 3D Visible

Watermarking ... 51

5.1 Introduction ... 51

5.1.1 Problem Definition ... 52

5.1.2 Proposed Idea... 52

5.2 Transformation of KINECT images into 3D images ... 53

5.3 Embedding and Eliminating 3D Visible Watermarks in KINECT Images by Difference Expansion ... 55

5.3.1 Embedding of 3D Visible Watermarks... 56

5.3.2 Extraction of Information for Recovery ... 60

5.3.3 Eliminating 3D Watermarks and Recovery of KINECT Images ... 63

5.3.4 Experimental results ... 65

5.4 Embedding and Eliminating 3D Visible Watermarks in KINECT Images by Reversible Contrast Mapping ... 68

5.4.1 Embedding of 3D Visible Watermarks... 69

5.4.2 Extraction of Information for Recovery ... 73

5.4.3 Eliminating 3D Watermarks and Recovery of KINECT Images ... 76

viii

5.5 Experimental Results ... 81

5.6 Discussions and Summary ... 87

Chapter 6

Conclusions and Suggestions for Future Works ... 88

6.1 Conclusions ... 88

6.2 Suggestions for Future Works ... 90

ix

LIST OF FIGURES

Figure 2.1 Hardware of the KINECT device. ... 9 Figure 3.1 Illustration of the authentication signals generation and embedding. ... 18 Figure 3.2 Illustration of proposed process of authentication signal verification and tampered pixel marking in protected depth images. ... 24 Figure 3.3 Illustration of the process of repairing tampered depth pixels. ... 29 Figure 3.4 The original KINECT images. (a) The color image. (b) The depth image. 34 Figure 3.5 The protected KINECT image with authentication signals embedded. (a) The color image with authentication signals embedded. (b) The depth image with authentication signals embedded. ... 34 Figure 3.6 The results of tampering protected KINECT images in which some regions have been modified. (a) The color result. (b) The depth result. ... 34 Figure 3.7 The authentication result of the tampered KINECT images of Fig. 3.6. (a) The color authentication image. (b) The depth authentication image... 35 Figure 3.8 The result of repairing the tampered KINECT images of Fig. 3.6. (a) The resulting color image. (b) The resulting depth image. ... 35 Figure 3.9 The result of recovering the protected KINECT images in which

authentication signals have been embedded. (a) The resulting color image. (b) The resulting depth image. ... 35 Figure 4.1 Illustration of secret-message transformation and hiding in the depth image. ... 40 Figure 4.2 The original depth image acquired by KINECT devices (left) and the depth image with the secret message embedded (right). ... 48 Figure 4.3 The original depth image of CVLAB (left) and the depth image with secret message embedded (right). ... 48 Figure 4.4 The original depth image of the other view in CVLAB (left) and the depth image with secret message embedded (right). ... 48 Figure 4.5 Extracted secret messages. (a) The correct secret message extracted with a right key. (b) The incorrect secret message extracted with an erroneous key. ... 49 Figure 5.1 Illustration of the pinhole camera model. ... 55 Figure 5.2 Illustration of image transformation and 3D watermark embedding. ... 57 Figure 5.3 Illustration of eliminating the 3D watermark and recovering the covered region in the 3D image. ... 64 Figure 5.4 The result of combining the original depth and color image into a 3D

x

image. ... 66 Figure 5.5 The 3D visible watermark. ... 66 Figure 5.6 The result of the watermarked 3D image. ... 67 Figure 5.7 The watermarked 3D image in which the recovery data string is embedded by the difference expansion method. ... 67 Figure 5.8 The result of eliminating the 3D watermark and recovering the covered region. ... 68 Figure 5.9 The result of combining the original depth and color image into a 3D image. ... 78 Figure 5.10 The 3D visible watermark. ... 79 Figure 5.11 The result of the watermarked 3D image. ... 79 Figure 5.12 The watermarked 3D image in which the recovery data string is

embedded by the reversible contrast mapping method. ... 80 Figure 5.13 The result of eliminating the 3D watermark and recovering the covered region. ... 80 Figure 5.14 The original 3D image. ... 82 Figure 5.15 Watermarked 3D image and the position of the watermark is in the left side of the image. ... 83 Figure 5.16 The watermarked 3D image with the recovery information embedding by the reversible contrast mapping method. ... 83 Figure 5.17 The results of the watermarked 3D image with the recovery information is embedded after the extraction process in different viewpoints. (a) the view of the position in front of the 3D image. (b) the view of the lowwer position in front of the 3D image. (c) the view of right position of the 3D image. (d) the view of left position of the 3D image. ... 84 Figure 5.18 The result of eliminating the 3D watermark and recovering the covered region. ... 84 Figure 5.19 The result of eliminating the 3D watermark and recovering the covered region by using the wrong secret key to extract the hidden data. ... 85 Figure 5.20 The watermarked 3D image with the recovery information embedding by the difference expansion method. ... 85 Figure 5.21 The result of eliminating the 3D watermark and recovering the covered region. ... 86 Figure 5.22 The result of eliminating the 3D watermark and recovering the covered region by using the wrong secret key to extract the hidden data. ... 86

xi

LIST OF TABLES

Table 3.1 Bins, bin number, bin codes, and representative values of bins used in the generation of authentication signals for depth image pixels in this study. ... 20 Table 3.2 Bins, bin number, bin codes, and representative values of bins used the generation of authentication signals for color image pixels in this study. ... 20 Table 4.1 Two bits corresponding to the different weights. ... 42

1

Chapter 1

Introduction

1.1 Motivation

With the rapid development of computer network technologies, the Internet becomes popular and used widely in people’s daily life nowadays. Through the Internet, we can not only conduct communication with people but also share information. Too many kinds of information can be found on the Internet, such as chatting contents, photo images, intelligence data, etc. Certain important information is private and should be protected because malicious hackers might want to steal or tamper with such information on the Internet. Therefore, how to protect different types of information and check the correctness of them has become important in people’s life.

In addition, with the fast advance of computer vision technologies, 3D techniques become more and more popular. Many devices are invented to capture 3D information, such as the KINECT sensor provided by Microsoft. Because of the ubiquity of KINECT sensors, even nontechnical people can easily get 3D information in their daily life. Because 3D image data contains more information than 2D ones, we can check in more detail the spatial relation between any two objects in the real world via the use of the 3D data captured by 3D imaging devices. 3D image data usually contains depth information that is very helpful for various purposes in our daily life. For example, in the application of video surveillance, we can get easily from 3D image data information like the height of an intruding man, the distance between two objects in an event, and so

2

on. Because of these characteristics and usefulness of 3D image data, the protection of 3D image data in various applications becomes an important work.

In this study, we propose a method for authentication of KINECT images (including the depth and color images) to verify their correctness in order to prevent the images from being tampered with by malicious persons. Besides, due to the popularity of the Internet, image data are shared and transferred frequently on the Internet; therefore, we also propose a method for covert communication via KINECT images, by which a user can send secret messages to persons he/she want without being intercepted or tampered with. In addition, to protect the copyright of KINECT images, we propose as well a method of 3D visible watermarking. The details of these proposed methods will be described in detail in subsequent sections or chapters.

1.2 General Review of Related Works

The invention of the Microsoft KINECT sensor [1] brings a lot of effects and opportunities not only in multimedia computing, but also in the gaming industry. The Kinect sensor changes the way by which people communicate with computers or games, i.e., changes the human interfacing technique. With the Kinect sensor, people can use just the body language to make the computer “understand” what the user is doing. The key change in this kind of user experience created by the Kinect sensor is the “third-dimension detection” technique that enables the computer to “realize” the user’s movement as well as the environment condition.

Because of the popularity of the KINECT sensor, acquisitions of 3D image data are getting easier and easier. The 3D information captured by the KINECT sensor includes both depth and color images, with the former showing the distance from the KINECT sensor to the target object, and the color image being just like a

3

commonly-seen RGB image. We call either the depth image or the color image a

KINECT image. The more information is included in KINECT images, the more

important the protection of the security of KINECT images is. Therefore, in this study we try to design information hiding techniques for the purpose of protecting KINECT images.

The hardware details of the KINECT sensor and the principle of using it to acquire depth image data will be introduced in Section 2.1. In Section 2.2, we will review previous studies of the structures of the depth and color images taken by the KINECT sensor.

Besides, many techniques for embedding fragile authentication signals for the purpose of image authentication have been proposed in the past. Specifically, the studies of [2-4] authenticate images at the pixel level such that any tampered image part can be identified pixel by pixel, yielding detailed tampering localization results. We will review the details of these studies in the Section 2.3. Furthermore, many data hiding techniques via images [5-12] have been proposed and are widely used in a lot of applications, for instance, covert communication, copyright protection, etc. Some techniques among them can be employed to conduct losslessly reversible data embedding and extraction, like those in [6-10]. As to the application of covert communication, a method of combining a new data hiding technique with a secret sharing scheme [13] is proposed in this study. Before that, we will also give a detailed review of existing related methods in Section 2.4. In Section 2.5, some methods of visible watermarking for images proposed in the past will be reviewed as well.

4

1.3 Overview of Proposed Methods

1.3.1 Terminologies

The definitions of some related terminologies used in this study are described as follows.

1. Secret: a secret (or a secret message) is a piece of information that is

important and should be preserved properly and not revealed to unauthorized people.

2. Secret share: a secret share is generated from a given secret message

according to the Shamir secret sharing scheme [3].

3. Image authentication: image authentication is a process for verifying the

integrity and fidelity of an image by checking the authentication signals embedded in it.

4. Protected image: a protected image is one into which some authentication

signals are embedded for the purpose of image authentication.

5. Embedding process: an embedding process is the process of embedding

some kind of data (e.g., secret messages) into an image.

6. Extraction process: an extraction process is the process of extracting the hidden data from an image.

1.3.2 Brief Descriptions of Proposed Methods

In this study, we propose several methods for data hiding via KINECT images and their applications. These methods are briefly described in the following.

(A) Authentication of KINECT images by data hiding techniques 

5

embedding authentication signals into the depth and color images captured with the KINECT sensor is proposed in this study. Specifically, we try to achieve the goal of authenticating both the depth and color images together by authenticating either of the depth and color images using the data of the other. That is, we use the depth image to generate authentication signals and embedding them into the color image; and reversely use the color image to generate authentication signals and embedding them into the depth image. To enhance the security of the proposed method, we select random positions in the depth or color images to embed authentication signals in order to reduce the possibility for a malicious user to figure out the locations of the embedded authentication signals in the KINECT images.

(B) Covert communication via KINECT images by interpolation at depth holes 

A data hiding method for covert communication via KINECT images by interpolation at depth holes is proposed in this study, where depth holes mean those 3D locations where the KINECT sensor does not provide depth data due to hardware or environment limitations. At first, a secret sharing scheme [3] is used to generate secret shares from a given secret message. Then, the generated secret shares are hidden, by a new technique of interpolation proposed in this study, at the depth holes in the depth image captured with the KINECT camera. Therefore, the proposed method can enhance the quality of the depth image, in addition to hiding the secret message into the KINECT image.

(C) Copyright protection of KINECT images by 3D visible watermarking 

A method of using data hiding techniques to protect the copyright of KINECT images by 3D visible watermarking is proposed in this study. In

6

embedding process, at first the depth and color images are combined into a single

3D image. Then, a given 3D visible watermark is embedded into the generated 3D image simply by data replacement. To recover the replaced original 3D image data, recovery information should be saved. For this, two schemes are proposed. One is to hide the recovery information in the 3D image by a difference expansion scheme and the other is to use the reversible contrast mapping scheme. Afterwards during the stage of watermark removal, the recovery information is extracted first. Then, the embedded 3D visible watermark is eliminated from the 3D image. And finally the replaced original 3D image data is recovered by using the extracted recovery information. In this way, the copyright of KINECT images can be protected.

1.4 Contributions

New methods for data hiding and applications via 3D KINECT images are proposed in this study. The contributions made in this study are listed in the following.

1. Three new applications involving KINECT images (including depth and color images) are proposed.

2. A data hiding technique using KINECT image features based on Lee and Tsai’s image authentication technique [2] is proposed.

3. A method of using the features of the depth image to authenticate the color image and using those of the color image to authenticate the depth image is proposed.

4. A data hiding technique in KINECT depth images by interpolation at depth holes is proposed.

7

5. A method of covert communication using the above technique of interpolation at depth holes is proposed.

6. Two techniques using difference expansion and reversible contrast mapping for hiding data into 3D images constructed from KINECT images are proposed.

7. A method for copyright protection of KINECT images by 3D visible watermarking using the above two data hiding techniques is proposed.

1.5 Thesis Organization

The remainder of this thesis is organized in following descriptions. An introduction to the Kinect sensor as well as the structures of image data captured by the Kinect sensor, and reviews of the applications and methods of image authentication, covert communication via images, and visible watermarking in images will be described in Chapter 2. The proposed method for authentication of KINECT images is described in Chapter 3. The proposed method for covert communication via KINECT images by interpolation at depth holes is described in Chapter 4. In Chapter 5, the proposed method for copyright protection of KINECT images by 3D visible watermarking is described. Finally, conclusions and some suggestions for future works are included in Chapter 6.

8

Chapter 2

Review of Related Works and

KINECT Image Structures

In this chapter, we will give a review of the KINECT sensor and the structures of the depth and color images taken by the KINECT device in Sections 2.1 and 2.2, respectively. And then we will give also a review of the existing data hiding techniques for image authentication, covert communication via images, and copyright protection of images, in Sections 2.3 through 2.5, respectively.

2.1 Previous Studies of 3D Image

Acquisition Using KINECT Devices

The release of the Microsoft Kinect sensor was probably one of the biggest impacts in the research field of computer vision. Kinect sensors have created many opportunities for multimedia computing. In this section, we give a review of the hardware of the KINECT sensor and the principle of the operations which can be conducted using the KINECT sensor.

The Kinect sensor includes a color VGA video camera, a depth sensor, a multi-array microphone, and a tilt motor for sensor operations and adjustments. The horizontal field of view of the KINECT device is 57 degrees, the vertical field of view is 43 degrees, and the physical tilt range is ± 27 degrees. The color camera aids in detecting three color components: red, green and blue, like other cameras. The main difference between commonly-seen cameras and the Kinect sensor is that the Kinect

9

device has an extra depth detection sensor. The depth detection sensor is composed of an infrared projector and a monochrome Complementary Metal-Oxide Semiconductor (CMOS) sensor, which work together to capture the distance information between the depth sensor and the objects in front of the Kinect device. The reason why the Kinect sensor can acquire the depth information comes from the use of the PrimeSense’s light coding-patented technology. The light coding technology works by coding the scene with near-IR light, which is invisible to the human eye, and then uses the CMOS image sensor and the chips to execute sophisticated parallel computational algorithms to decipher the received light-coding infrared patterns to produce a VGA-size depth image of the scene.

The key of the mentioned light coding technique is the use of laser speckles  when the laser is projected on objects with rough surfaces or through the frosted glasses, it will generate random reflecting speckles, integrally called the speckle pattern. The speckle pattern is highly randomized and the pattern image changes with different distances. The speckle pattern images that are captured by the Kinect sensor in any two places of the real space are different. According to speckle patterns, all the places in the real space can be marked. Then, the depth information can be obtained by decoding the laser speckle pattern on the object.

10

2.2 Previous Studies of Structures of

Depth and Color Images Taken by

KINECT Devices

In this section, we will give a review of the functions of the KINECT sensor and the structures of the depth and color images taken by the KINECT sensor. The KINECT sensor brings many effects in not only various fields of research but also our daily life. Because of the popularity of the KINECT sensor, people can use it easily to develop various kinds of applications, for example, detection of the positions of the user’s hands for browsing of websites according to the user’s movements only.

Many developers use the Open Natural Interaction (OpenNI) software development kit (SDK) of the KINECT sensor to develop related applications. The OpenNI framework is an open-source SDK useful for development of 3D sensing middleware libraries and applications. In addition, there are other SDKs like the Kinect-for-Windows SDK, which is also used by many developers for R & D. In this study, we use the OpenNI SDK to acquire data with the KINECT sensor. The acquired data include depth and color information, with the color information being like a commonly-seen color image with resolution 640480; and the depth information being a range image also with resolution 640480. The depth range provided by the KINECT sensor using the Kinect-for-Windows SDK is from 800mm to 4000mm, but that provided by the use of the OpenNI SDK is up to is the maximum of about 8000mm. As the depth distance detected by the KINECT sensor increases, more depth data will be missed in the detected result. For example, the missing depth value start from 611mm, and then 622mm, 631mm, 638mm, etc. When the depth value comes to 7960mm, the values of 7961mm to 8146mm are missing, so the inaccuracy of the detected value is 185mm. According to this phenomenon, we can realize that the larger the depth value

11

captured by the KINECT sensor, the lower accuracy the detected depth values have. If a user wants to interact with the KINECT sensor, the better range of distances between the KINECT sensor and the user is from 1200mm to 3600mm, which is advised by the KINECT development official website.

The acquired data by the KINECT sensor includes depth and color images as mentioned previously, and either of the depth and color images will be called a KINECT image in this study.

2.3 Review of Techniques for Image

Authentication

With the advance of the computer and Internet technologies, security of digital image data is considered as a significant issue today. Thus, many techniques for embedding authentication signals for the purpose of image authentication have been proposed in the past. Specifically, the methods proposed in [2-4] authenticate images at the pixel level such that any pixel of a tampered image part can be identified, and then the result of detailed tampering localization is reported.

The method proposed in Liu et al. [2] generates a binary image that is mapped from the difference image computed from the cover image and its so-called chaotic pattern. And the least-significant-bit (LSB) plane is used to accommodate the binary image as a fragile watermark for use in later image authentication.

In Lee and Tsai [3, 4], a grayscale image authentication method was proposed. The method is based on a bin-mapping scheme which divides the grayscale range into two parts, the five MSBs and the remaining three LSBs. The former is used to generate a 3-bit bin code as the authentication signal for each pixel in the input cover image. Then, the authentication signals are embedded randomly into the other pixels of the image.

12

The authentication signals are utilized not only for detecting and localizing tampered pixels but also for generating representative values for repairing the tampered pixels.

2.4 Review of Techniques for Covert

Communication via Images

Many data hiding techniques have been proposed for various purposes such as authentication and covert communication in the past. To achieve the goal of hiding the data imperceptibly, data hiding utilizing the weaknesses of the human’s vision system have been investigated intensively. A widely known method is least significant bit (LSB) modification which changes the LSBs of the pixel value in an image to embed information. For example, Chan and Cheng [5] presented a data hiding method by simple LSB substitution. On the other hand, data hiding techniques using pairs of image pixels to hide information have also been proposed, like Tian [6] who proposed a reversible data embedding method by using a difference expansion scheme which is based on simple reversible integer transformations. This method calculates the differences of neighboring pixel values, and selects some difference values of the pixel pairs for the difference expansion. Then, the information is embedded into the expanded differences of the pixel pairs. Since the modified values are generated from the differences between manipulated pixel pairs, the original pixel values can be recovered easily.

13

2.5 Review of Techniques for

Copyright Protection of Images

Because of the popularity of the Internet, acquisitions of digital information from the network becomes easier and easier nowadays. Thus, protection of the copyright of digital information on the Internet is more important than ever before. About the topic of copyright protection of images, digital watermarking has been used widely in a lot of applications [7-12]. Digital watermarking means embedding some kinds of information, like ownership information, company logo, etc., into digital images that should be protected.

In general, digital watermarking techniques for images can be categorized into two types: visible and invisible. The first type of technique, visible watermarking, is to embed clearly visible marks into images. The embedded visible watermark is usually irremovable and leaves permanent distortion to the original image. The technique of the second type aims to embed the copyright information imperceptibly into images so that in case of copyright infringement, the hidden information can be retrieved to identify the ownership of the protected host image.

Both visible and invisible watermarking techniques yield the distortion of the host image after the embedding process. A group of techniques, named reversible watermarking [7-10], allow authorized users to remove the embedded watermark and save the original content of images as needed. Besides, some reversible watermarking techniques guarantee lossless image recovery, which means that the recovered image is identical to the original image.The techniques oflossless recovery is important in some applications , for example, images for military uses, historical art imaging, or related applications of medical image analysis. In these applications, any permanent distortion generated by watermarking is not allowed.

14

In [7], Alattar extended Tian’s algorithm [6] to utilize the difference expansion of vectors, instead of pairs of pixels, to increase the hiding ability and the computation efficiency.

In [8], Coltuc and Chassery presented a high-capacity data embedding scheme without using any additional data compression operation. This scheme is based on a reversible contrast mapping (RCM) technique, which is a simple integer transform defined on pairs of pixels. It partitions the cover image into pairs of pixels and divides the pairs into three groups, and then conducts a respective embedding process on each pair group. This RCM scheme provides almost similar embedding bit-rates when compared to the difference expansion approach, while it has a considerably lower mathematical complexity.

15

Chapter 3

Authentication of KINECT Images

by Data Hiding Technique

3.1 Introduction

Because of the growing popularity of the KINECT device, the depth and color images, also called KINECT images in this study, acquired by KINECT devices are more commonly seen in various applications. Like other digital images, the KINECT images also need be verified for their correctness in order to prevent them from being tampered with by malicious persons when they are transmitted or kept in storages. For this reason, we propose a method for authentication of KINECT images in this study. The detail of the method will be described in this chapter.

First, the definition of the problem and the idea of the proposed method are given in Section 3.1. In Section 3.2, we will describe the process of generating authentication signals and embedding them into the KINECT images. Then, in Section 3.3, the process of extracting the embedded authentication signals from the protected KINECT images will be described. In Section 3.4, the recovery of the original depth and color images, and repairing of the possibly tampered versions of them will be described. Experimental results showing the feasibility of the method are given in Section 3.5. Finally, some discussions and a brief summary are given in the last section of this chapter.

16

3.1.1 Problem Definition

As 3D sensing technologies are growing vigorously, related applications of the KINECT sensor are also increasing in various fields. Since the KINECT sensor can be used to acquire the depth information, the data can be used more extensively in applications than the 2D data can, for example, as the 3D information of marble sculptures created by famous artists and the models of historical architectures or objects encountered in daily life. These kinds of information are often very important, so the correctness of them must be guaranteed.

The range of the depth values that are provided by the KINECT device is different from that of the general values of color-image pixels. In addition, the pair of the depth and color images acquired by the KINECT device at each identical instant should be protected together to keep their relation in time unchanged. No matter whether the color or the depth image is tampered with by malicious persons or not, the tampered region should be detected by an authentication method, and better be repaired. These requirements should be considered when designing an authentication method for KINECT images, as is done in this study.

3.1.2 Proposed Ideas

The major idea of the proposed authentication method for KINECT images was inspired by the concept involved in the authentication method proposed by Lee and Tsai [4] as well as by some natures of KINECT image features. The proposed method aims to authenticate, as a whole, every pair of color and depth images taken by a KINECT device at an identical instant. The proposed method utilizes the features of KINECT images and the ranges of the pixel values in them to achieve the goal of authenticating them together as a whole.

17

As an inherent feature of KINECT images, a pixel value in the depth image is represented by a 13-bit binary string, as mentioned in Chapter 2, in order to cover the entire range of the depth values. And the value of a pixel in the color image, including the information of red, green, and blue, is represented by three 8-bit binary strings, respectively.

Furthermore, in the proposed method for KINECT image authentication, as shown in Fig. 3.1 the 13-bit depth value D of each pixel Pd in a depth image is divided

into two parts － the nine MSBs of D and the three LSBs. The former is used to generate an authentication signal for the depth pixel Pd itself. The authentication

signal is then embedded into five LSBs of a color-image pixel Pc randomly, where the

five LSBs of Pc includes two LSBs of the red color value, two LSBs of the green

color value, and one LSB of the blue color value, or we can say equivalently, the mentioned five LSBs of Pc includes the (2, 2, 1) LSBs of the (R, G, B) values of Pc.

On the other hand, the nine MSBs of the color-image pixel Pc are used to generate an

authentication signal for the color-image pixel Pc itself, and the signal is embedded

into the three LSBs of the depth pixel Pd randomly, where the nine MSBs of Pc

includes (3, 3, 3) MSBs of the (R, G, B) values of Pc. In addition, the generated

signals not only can be used to verify the correctness of a pixel P (Pd or Pc) in the

color or depth image, but also can be used to repair part of the value of P when P is authenticated to have been tampered with.

The detailed algorithms about the proposed methods and the related processes of authentication of KINECT images are presented in the following sections.

18

3.2 Generation and Embedding of

Authentication Signals

In this section we will introduce the details of the implemented processes of authentication signal generation and embedding according to the proposed method. An illustration of the processes is illustrated in Figure 3.1. The detailed process of generation of authentication signals is described in Section 3.2.1, and the detailed process of embedding the generated authentication signals into KINECT images is described in Section 3.2.2.

[r

7

r

6

r

5

g

7

g

6

g

5

b

7

b

6

b

5

]

Bin mapping 3-bit authentication signal

[d

12

d

11

d

10

d

9

d

8

d

7

d

6

d

5

d

4

]

Bin mapping 5-bit authentication signal

Save the original bits [g1r1b0g0r0d2d1d0] to

alpha channels Color-image pixel Pc

Depth-image pixel Pd

19

3.2.1 Authentication Signal Generation

In the proposed authentication signal generation process, at first we transform the value of a depth pixel Pd in the depth image into a 13-bit binary string, d12, d11, d10, …, d0. Then, we also transform the (R, G, B) values of each color-image pixel Pc into three

8-bit binary strings, r7, r6, r5, …, r0, g7, g6, g5, …, g0, and b7, b6, b5, …, b0, respectively.

Next, we transform the nine MSBs of Pd into an integer m and the (3, 3, 3) MSBs of the

(R, G, B) values of Pc as a whole are also transformed into an integer n. Then, we apply

a modified version of a bin-mapping scheme mentioned in Lee and Tsai [4] for the purpose of compressing these MSBs information before embedding them. Additionally, we map the depth-value range specified by the nine MSBs into 32 equal-length intervals, called bins. And each bin is indexed by a decimal integer called a bin number, which corresponds to a 5-bit binary number called a bin code. The 32 bins and their corresponding bin numbers and bin codes are shown in Table 1. On the other hands, the color-value range specified by the nine MSBs is also mapped into eight bins, and the eight bins and their corresponding bin numbers and bin codes are shown in Table 2. Finally, the bin code of each pixel is taken to be the authentication signal of the pixel.

The proposed technique above for generating authentication signals using parts of depth pixel values and color pixel values is described as an algorithm, Algorithm 3.2.1, as follows.

20

Table 3.1 Bins, bin number, bin codes, and representative values of bins used in the generation of authentication signals for depth image pixels in this study.

Bin Bin number Bin code Representative value of bin

[0, 31] 0 00000 16 [32, 63] 1 00001 48 [64, 95] 2 00010 80 … … … … [416, 447] 29 11101 432 [448, 479] 30 11110 464 [480, 511] 31 11111 496

Table 3.2 Bins, bin number, bin codes, and representative values of bins used the generation of authentication signals for color image pixels in this study.

Bin Bin number Bin code Representative value of bin

[0, 63] 0 000 32 [64, 127] 1 001 96 [128, 191] 2 010 160 [192, 255] 3 011 224 [256, 319] 4 100 288 [320, 383] 5 101 352 [384, 447] 6 110 416 [448, 511] 7 111 480

Algorithm 3.2.1: authentication signals generation.

Input: a color pixel value of Pc including a red value R, a green value G and a blue

value B and a depth pixel Pd with value D.

Output: the 3-bit color authentication signal s and the 5-bit depth authentication

signal t.

Steps:

Step 1. Transform the depth value of Pd into thirteen bits, d12, d11, d10, …, d0. And

21

r5, …, r0, g7, g6, g5, …, g0, and b7, b6, b5, …, b0.

Step 2. Transform the nine MSBs, d12, d11, d10, …, d5into an integer m; and

concatenate the three binary sub-strings, r7, r6, r5, g7, g6, g5, b7, b6, b5, and

transform the result into an integer n.

Step 3. Map the integer m into a bin indexed by a bin number Bm computed by the

function B_m 



m/16



; and map also the integer n into a bin indexed by a bin number Bn computed by the function Bn 



n/64



.

Step 4. Transform Bm into a 5-bit bin code t = e4e3e2e1e0 for use as the

authentication signal for Pc; and transform also Bn into a 3-bit bin code s = h2h1h0 for use as the authentication signal for Pd.

3.2.2 Embedding of Authentication Signals

Because the color and depth image is saved as the PNG images, the alpha channels are used to hide the original bits for the process of original-image recovery. The original bits of a pair of a depth pixel and a color pixel includes the (2, 2, 1) LSBs of the original color-image pixel and the three LSBs of the original depth-image pixel. And the original bits of the depth and color image will be replaced with the authentication signals in this process. Therefore, in this process, the original bits of the depth and color image are hidden into the alpha channels of the depth and color image, respectively.

To embed the generated authentication signals generated as described above. At first we select a pixel Pd from the depth image I and a pixel Pc from the color image J

both in a raster-scan order. Then, we consider pixel Pd and pixel Pc as the input to

Algorithm 3.2.1 to generate the corresponding authentication signals, a 3-bit signal s for the color-image pixel Pc and a 5-bit signal t for depth-image pixel Pd. Besides, the

22

original bits of Pd and Pc are hidden to the alpha value of the pixel Pc' and Pd',

respectively. In order to achieve the goal that the pair of the depth and color images should be protected together, we embed the authentication signal s, which is generated from the pixel Pc of the color image into a pixel Pd' in the depth image. On the other

hands, we embed the authentication signal t, which is generated from the pixel Pd of the

depth image, into a pixel Pc' in the color image. In addition, both of the pixels, Pc' and Pd' are selected by using a secret key K and a random number generator f. When all the

pixels in the color and the depth image have been processed, we get two protected images, a color and a depth image with authentication signals embedded.

The process described above for embedding the generated authentication signals is described in Algorithm 3.2.2 below.

Algorithm 3.2.2: embedding of authentication signals.

Input: a depth image I, a color image J, a random number generator f, and a secret

key K.

Output: a depth image Is with authentication signals embedded, and a color image Js

with authentication signals embedded.

Steps:

Step 1. In a raster-scan order, select a pixel Pd from the depth image I and a pixel Pc

from the color image J.

Step 2. Perform the process of Algorithm 3.2.1 to generate two authentication signals, a 3-bit signal s for the depth pixel Pd and a 5-bit signal t for the

color pixel Pc.

Step 3. Using the secret key K and the random number generator f to select randomly a pixel Pc' in the color image J and a pixel Pd' in the depth image I.

23

Step 4. Extract the original bits, the (2, 2, 1) LSBs of Pc, g1, r1, b0, g0, r0, and the

three LSBs of Pd, d2, d1, d0; and embed g1, r1, b0, g0 into the four LSBs of

the alpha-channel value of Pd' and embed r0, d2, d1, d0 into the four LSBs of

the alpha-channel value of Pc'.

Step 5. Embed the 5-bit authentication signal t = e4e3e2e1e0 of Pd into Pc' in J by

replacing the (2, 2, 1) LSBs of the (R, G, B) values in the pixel Pc' with t;

and embed the 3-bit authentication signal s = h2h1h0 of Pc into Pd' in I by

replacing the three LSBs of Pd' with s.

Step 6. If there remain unprocessed pixels in I or J, then go to Step 1; otherwise, take the processed I and J together with their alpha channels as the desired embedded images Is and Js in PNG format as output.

3.3 Authentication of KINECT Images

In this section, we introduce the proposed method for authentication of KINECT images. An illustration of the authentication signal extraction and verification process to apply the authentication function to the protected depth image is shown in Figure 3.2. In Section 3.3.1, the process of extracting the embedded authentication signals is described. Then, the process of detecting possible tampered regions in the depth or color image is described in Section 3.3.2.

24 Protected depth image Is

A protected pixel Pd in Is A selected pixel Pc' corresponding to Pd in Js Extract the (2, 2, 1) LSBs from Pc' as extracted authentication signal Gengerate computed authentication signal Matching Regard Pd as authentic Yes Mark Pd's corresponding

pixel on authentication image Ia as tampered No

Final authentication image Ia'

Key

Figure 3.2 Illustration of proposed process of authentication signal verification and tampered pixel marking in protected depth images.

3.3.1 Extraction of Authentication Signals

In the process of authentication signal extraction, firstly we use the nine MSBs of each pixel Pd in the protected depth image to compute an authentication signal t = e4e3e2e1e0, which is called the computed authentication signal. And according the

random number generator f and the secret key K used in Algorithm 3.2.2, we find out the corresponding pixel Pc' in the protected color image. Then, we extract the (2, 2, 1)

LSBs of the (R, G, B) values in the color pixel Pc' to form a binary string t' = g4'r3'b2'g1'r0'. The string t' is called the extracted authentication signal. On the other

25

Pc' and the extracted authentication signal s' in the depth pixel Pd' also can be found out.

Finally, the computed authentication signals s and t, and the extracted authentication signals s' and t' are used in the authentication signal verification process.

The detail of the process described above for the extractions of the embedded authentication signals and the computed authentication signals is presented as algorithm, Algorithm 3.3.1, below.

Algorithm 3.3.1: extraction of authentication signals.

Input: a color pixel value of Pc from a color image Js; a depth pixel value Pd from a

depth image Is; the random number generator f and the secret key K used in

Algorithm 3.2.2.

Output: the computed authentication signals s and t, and the extracted authentication

signals s' and t'.

Steps:

Step 1. Transform the depth value of Pd into a 13-bit string d12, d11, d10, …, d0, and

transform the three RGB color values of Pc into three eight-bit strings r7, r6, r5, …, r0, g7, g6, g5, …, g0, and b7, b6, b5, …, b0, respectively.

Step 2. Transform the nine MSBs, d12, d11, d10, …, d5, of d into an integer m, and

concatenate the three 3-bit binary strings, r7, r6, r5, g7, g6, g5, b7, b6, b5, in

order into a string and transform the result into an integer n.

Step 3. Map the integer m into a bin indexed by a bin number Bm computed by the

function Bm m/ 16; and map also the integer n into a bin indexed by a

bin number Bn by the function Bn n/ 64.

Step 4. Transform Bm into a 5-bit bin code t = e4e3e2e1e0 for use as the computed authentication signal from Pd; and transform Bn into a 3-bit bin code s = h2h1h0 for use as the computed authentication signal from Pc.

26

pixel Pc' corresponding to Pd from the color image Is and a depth pixel Pd'

corresponding to Pc from the depth image Js.

Step 6. Transform the depth value of Pd' into a 13-bit string d12' , d11', d10', …, d0',

and transform the RGB color values of Pc' into three 8-bit strings r7', r6', r5', …, r0', g7', g6', g5', …, g0', and b7', b6', b5', …, b0', respectively.

Step 7. Extract the three LSBs of Pd' to form a binary string s' = d2'd1'd0', and

extract the (2, 2, 1) LSBs of the (R, G, B) values in pixel Pc' to form a

binary string t' = g4'r3'b2'g1'r0', and call s' and t' the extracted authentication signals.

3.3.2 Detection of Tampering in Protected Depth and

Color Images

During the process of detecting possibly tampered regions in the protected KINECT image, an authentication signal is computed from the nine MSBs of every depth pixel Pd. And the authentication signal, which is embedded in the color pixel Pc'

corresponding to Pd selected randomly as described in Algorithm 3.2.2, is extracted.

Then, the two authentication signals are compared with each other. If the two signals are different, the depth pixel Pd is regarded as having been tampered with. On the other

hand, an authentication signal is computed from the (3, 3, 3) MSBs of every color pixel

Pc. Also, the authentication signal embedded in the three LSBs of the corresponding

depth pixel Pd' is retrieved. Then, we also compare the two authentication signals. If the

signals are different, the color pixel Pc is regarded as having been tampered with. In

addition, when mismatching occurs, the depth pixel Pd or the color pixel Pc is regarded

27

blank image Ia or Ja as a black point to create two authentication images Ia' and Ja',

respectively.

The detail of the process above for detection of tampering in the KINECT image is described in Algorithm 3.3.2 below.

Algorithm 3.3.2: tampering detection in protected depth and color images.

Input: a protected depth image Is and a protected color image Js, both with

authentication signals embedded; two corresponding blankimages, Ia and Ja;

and the random number generator f and the secret key K used in Algorithm 3.2.2.

Output: an authentication depth image Ia' and an authentication color image Ja'

corresponding to Is and Js, respectively. Steps:

Step 1. Select a pixel Pd from Is, and a pixel Pc from Js, both in a raster-scan order,

and perform the following steps.

Step 2. Take pixels Pd and Pc as inputs and apply Algorithm 3.3.1 to yield the computed authentication signals, t and s, and the extracted authentication signals, s' and t' for Pd and Pc, respectively.

Step 3. Match the computed authentication signal t = e4e3e2e1e0 and the extracted

authentication signal t' = g4'r3'b2'g1'r0' bit by bit; and if mismatching

happens, regard Pd as having been tampered with and mark its

corresponding pixel as a black point in the blank depth image Ia.

Step 4. Compare the computed authentication signal s = h2h1h0 and the extracted

authentication signal s' = d2'd1'd0'; and if mismatching occurs, regard Pc as

having been tampered with and mark its corresponding pixel as a black point in the blank color image Ja.

28

take the final possibly marked Ia and Ja as authenticationimages Ia' and Ja',

respectively.

3.4 Repairing and Recovery of Depth

and Color Images

In this section, the proposed method of recovering and repairing the depth and color images is described. In Section 3.4.1, the process of repairing the tampered pixels is described. Then, the process of recovering the original images from images in which authentication signals are embedded is described in Section 3.4.2.

3.4.1 Repair of Tampering in Protected Depth and

Color Images

As illustrated by Figure 3.3, at the beginning of the tampered-pixel repairing process, we can find the tampered pixels Pd' and Pc' respectively in Is and Js, which are

the depth image and color image with authentication signals, according to the black points in the authentication images Ia and Ja, respectively. Then, the corresponding

pixels Pc'' and Pd'' located at the positions where the authentication signals of Pd' and Pc' are embedded can be found out by the use of the random number generator f and the

secret key K used in Algorithm 3.2.2. According to the extracted signals from Pc'' and Pd'', we can recover the bits which are used to generate these authentication signals. In

29

the repairing results.

A pixel Pd in Is corresponding to a

tampered pixel da specified in Ia'

A selected pixel Pc'

corresponding to Pd in Js

Extract the bin code from Pc'

Find the representative value R

Pad R in bits with three 0's to get a new value T

Replace Pd with

T

A repaired pixel Pd

Figure 3.3 Illustration of the process of repairing tampered depth pixels.

The above process for repairing the tampered pixels in the color or depth image is described in more detail as an algorithm, Algorithm 3.4.1, as follows.

Algorithm 3.4.1: repairing the tampered pixels of the color and depth images. Input: the depth image Is and the color image Js both with authentication signals

embedded; corresponding authentication images Ia, and Ja; and the random

number generator f and the secret key K used in Algorithm 3.2.2.

Output: a depth image Ir, and a color image Jr, which are repaired, if the input images

have been tampered with.

Steps:

Step 1. Find each black point da in Ia' and each black point pa in Ja', both in the

30

and the pixel Pc' in Js corresponding to pa in position.

Step 2. Use the random number generator f with the secret key K to find the pixel

Pc'' in Js corresponding to Pd', where Pc'' is with a previously-embedded

authentication signal; and use the random number generator f with the secret key K as well to find the pixel Pd'' in Is corresponding to Pc' where Pd'' is

with a previously-embedded authentication signal.

Step 3. Transform the RGB color values of Pc'' into three 8-bit strings r7'', r6'', r5'', …, r0'', g7'', g6'', g5'', …, g0'', and b7'', b6'', b5'', …, b0'', respectively;

extract the (2, 2, 1) LSBs g1''r1''b0''g0''r0'' from them; and transform g1''r1''b0''g0''r0'' into an integer Bn' as the index of the bin; transform as well

the depth value of d'' into a binary string d12'', d11'', d10'', …, d0'', extract the

three LSBs d2'', d1'', d0'' to form a binary string, and transform it into an

integer Bm' as the index of the another bin.

Step 4. Repair the tampered depth pixel Pd' by the following steps.

4.1 Derive the representative value Md of the bin indexed by Bm'.

4.2 Transform Md into a 9-bit binary string d12d11d10d9d8d7d6d5d4.

4.3 Pad three trailing 0’s to d12d11d10d9d8d7d6d5d4 to get a 13-bit string Td

= d12d11d10 d9d8d7d6d5d40000.

4.4 Transform Td into an integer u and replace the depth value of Pd' with u as the repairing result.

Step 5. Repair the tampered color pixel Pc' by the following steps.

5.1 Derive the representative value Mp of the bin indexed by Bn'.

5.2 Transform Mp into a 9-bit binary string r7r6r5g7g6g5b7b6b5.

5.3 Separate the binary string into three parts, r7r6r5,g7g6g5,and b7b6b5.

5.4 Pad five trailing 0’s to each of the three binary strings to obtain respectively three new strings Tr = r7r6r500000, Tg = g7g6g500000, Tb

31

= b7b6b500000.

5.5 Transform these three 8-bit strings into three integers and replace the color values (R, G, B) of Pc' with them, respectively, as the repairing

result.

Step 6. If there remain unprocessed black pixels in Ia or Ja, then go to Step 2; else,

take the final Is and Js as the output repaired images Ir and Jr.

3.4.2 Recovery of Protected Depth and Color Images

At the beginning of the process of recovering the protected depth and color images, we can find the pixel Pd' in the depth image Is with an authentication signal

corresponding to Pc as well as the pixel Pc' in the color image Js with an authentication

signal corresponding to Pd by using the secret key K and the random number generator f. Then, the hidden bits in the alpha-channel values of Pd' and Pc' can be extracted and

divided into two parts, one for the recovery of Pd and the other for the recovery of Pc.

Furthermore, we can use these two parts to replace the (2, 2, 1) LSBs of Pc and the three

LSBs of Pd, respectively. Finally, we can get the original color and depth images after

all the pixels in Js and Is are processed.

The process described above for recovering the original bits in the depth and color images is described in more detail as an algorithm, Algorithm 3.4.2, as follows.

Algorithm 3.4.2: recovering the original color and depth images.

Input: a protected depth image Is and a protected color image Js both with

authentication signals embedded; and the random number generator f and the secret key K used in Algorithm 3.2.2.

32 Steps:

Step 1. In a raster-scan order, select a pixel Pd from the protected depth image Is

and a pixel Pc from the protected color image Js.

Step 2. Use the secret key K and the random number generator f to find the pixel Pd'

corresponding to Pc in the protected color image Js and the pixel Pc'

corresponding to Pd in the protected depth image Is.

Step 3. Extract the four LSBs, v3, v2, v1, v0, from the alpha-channel value of Pc'; and

extract also the four LSBs, u3, u2, u1, u0, from the alpha-channel value of Pd'.

Step 4. Take u3, u2, u1, u0, v3, in order as five original bits of Pc, concatenate them

into a 5-bit string Tc = u3u2u1u0v3, and take the other three bits, v2, v1, v0, in

order as three original bits of Pd.

Step 5. Replace the three LSBs of Pd with u2, u1, u0 and transform the resulting

13-bit binary string of Pd into an integer as the original value of Pd; divide Tc into three parts, Sr = u2v3, Sg = u3u0, Sb = u1; replace the (2, 2, 1) LSBs of

the (R, G, B) values of Pc with Sr, Sg, Sb, respectively; and transform the

three resulting binary strings into three integers as the original R, G, B values of Pc.

Step 6. If there remain the unprocessed pixels in Is or Js, then go to Step 1;

otherwise, take the final Is and Js as the original images Io and Jo,