Chapter 5 Protection of Privacy-sensitive Contents in Surveillance
5.2 Hiding of Privacy-sensitive Contents
5.2.2 Process for Hiding Privacy-sensitive Contents
The proposed process is applied to the prediction frames of an input WebM video.
If a region R specified by an authorized user has privacy-sensitive contents which need to be protected, the motion vectors and frequency coefficients of the currently-processed macroblock within R is all set to zero. Therefore, the video contents of R become the corresponding part of the background image which has appeared in the golden reference frame of the input video. It also places a restriction on this proposed process that the first frame of the input video must be a background frame. The values of the original motion vectors and frequency coefficients of the macroblocks of R are grouped into a report and then hidden into the prediction frames in the input video. Furthermore, we use a secret key to randomize the report for the security protection issue.
86
As mentioned before, macroblocks are encoded in a raster-scan order in the VP8 video codec. So, we have to perform an encoding process to remove the privacy-sensitive contents and get the decoding information which may be used to recover the privacy-sensitive contents if necessary, and then perform another encoding process to hide the decoding information into the video. The following algorithm describes the details of the above-mentioned process.
Algorithm 5.1 Process for removing the privacy-sensitive contents in a user-specified
region.Input: a WebM video V and a region R specified by an authorized user
Output: a WebM video V’ with privacy-sensitive contents in R removed and a report E of privacy-sensitive contents
Steps.
Stage 1 --- initialization.
1. Set the golden reference frame as the reference frame for each prediction frame F in V.
2. Restrict the encoder to use inter prediction modes during the prediction step.
Stage 2 --- removing privacy-sensitive contents in the region R.
3. For each prediction frame F in V, perform the following steps.
3.1 Detect motions in the region R of F by checking the prediction mode of each macroblock within R; set the value of the motion flag to be 1, if there exist prediction modes other than ZEROMV within R; otherwise, set the value of the motion flag to be 0.
3.2 If the value of motion flag is set as 1, then for each macroblock MB within
R of F, perform the following steps.
3.2.1 For each subblock in MB, perform the following steps.
87
(1) Record all the sixteen coefficients of the subblock of Y2 into E and set all coefficients of the Y2 block to be zero.
(2) Record the DC coefficient of each subblock of the chroma color channels (including the U channel and the V channel) into E and set all coefficients of these subblocks to be zero.
(3) Record seven coefficients of each subblock of the luma color channel (the Y channel) into E and set all coefficients of these subblocks to zero.
(4) Record the index of MB and the index of F into E.
3.2.2 Record the motion vector of MB into E and set the motion vector to be zero.
4. Repeat Step 3 until all frames of V have been processed.
Considering the capacities of the proposed hiding data method, we cannot record all the coefficients in a macroblock which will be used to recover privacy-sensitive contents. Therefore, we conduct some tests in order to decide which coefficients of color channels need to be recorded. In Section 2.5.1, we mentioned that there exists a 4×4 subblock called Y2 which records the DC coefficients of a Y block. If we lose Y2 coefficients, we cannot recover the contents in this macroblock. Figure 5.3 shows an example if we lose the Y2 coefficients information. Therefore, we record all Y2 coefficients information in order to make sure we can recover the privacy-sensitive contents. In Step 3.2.1, because the VP8 video codec uses the zig-zag scan order, which is shown in Figure 5.4, to encode subblocks, after our experimental tests we decide to record seven coefficients of each Y subblock according to the zig-zag scan order.
88
Figure 5.3 Comparison between the original image and the image whose Y2 coefficients have been lost.
(Left) the original image. (Right) the image whose Y2 coefficients have been lost.
Figure 5.4 The zig-zag scan order.
Algorithm 5.2 Process for embedding the privacy-sensitive contents into a WebM
video.Input: a WebM video V with privacy-sensitive contents in R removed, a secret key K,
a random number generator f, and a report E of the privacy-sensitive contents.Output: a WebM video V’ with privacy-sensitive contents in R removed
and embedded.Steps.
Stage 1 --- initialization.
1. (Randomizing the message) Transform the report E into a binary string B, use the secret key K as a seed to generate a sequence Qr of random numbers using the
89
random number generator f, and randomize B with Qr to get a randomized binary string B'.
2. Divide B' into a linear array A of 4-bit segments.
3. Use the secret key K as a seed to generate a sequence Q of random numbers using the random number generator f; and take the first eight random numbers from Q, denoted as N1 through N8.
Stage 2 --- embedding message data into the video.
4. Perform the following steps to embed the input report E into each unprocessed macroblock MB in every prediction frame of V, assuming V is large enough to embed the entire message.
4.1 Take eight unprocessed 4-bit elements from A, denoted as A1 through A8, with each element Ai corresponding to a data pattern denoted as Pi (e.g., if
A
i = 10012 = 910, then Pi = data pattern 9).4.2 Combine Pi with Ni by exclusive-OR operations to become a new data pattern DPi:
i i i
DP P N (5.1)
where i = 1, 2, …, 8.
4.3 Save the original quantized coefficients of the chroma color channels in
MB.
4.4 (Embedding the secret data patterns) Check the coefficients of each of the eight corresponding subblocks of the chroma color channels (four in the U channel and the other four in the V channel), denoted as SBi, i = 1, 2, …, 8:
if the original coefficientsof SBi do not match those of data pattern DPi, then modify them to be so by changing those mismatching ones in SBi to be the corresponding ones of data pattern DPi.; else, do nothing.
4.5 (Computing the resulting distortion) Calculate the mean square quantization
90
error (MSQE) between the content MBo of the original macroblock MB and the content MBo
' of the modified macroblock MB' of the chroma color
channels (i =1, 2 for the U channel and the V channel, respectively):2
where Speak means the maximum possible pixel value of the image.
4.7 Calculate the average PSNR value PSNRavg of the chroma color channels (i
= 1, 2): than the pre-selected threshold T; else, use the default value of the ROI map index which is 0 and recover the modified coefficients of the chroma color channels in MB to original values (meaning that macroblock MB is
non-data-embeddable).
5. Repeat Step 4 until the entire secret message in array A has been embedded (i.e., until all elements in A have been processed).
91