Detection of Motion Activities by Use of Speeded Up Robust

In the proposed method for protecting privacy-sensitive motion activities, at the beginning we search motion activities in videos, and then decide a corresponding protected region R for detecting motion event automatically. Also, the image content in R is defined as a privacy-sensitive image part and will be disguised as a pre-selected background image part B which corresponds to the privacy-sensitive image part in position in the first image frame of the video. Then, we apply the previously-proposed concealment process (described in Chapter 3) with the background image part to the privacy-sensitive image part to produce a camouflage image looking close to the background image part B.

Before we start, we briefly introduce the principles of SURFs (speeded up robust features) and how to detect motion activities by use of SURFs. SURFs are robust

61

local features, first presented by Herbert Bay et al. [33] in 2006, that can be used in computer vision tasks like object recognition or 3D reconstruction. It is partly inspired by the SIFT descriptor. The SURFs have been proven to achieve high repeatability and distinctiveness. The method using SURFs uses a Hessian matrix-based measure for the detection of interest points and a distribution of Haar-wavelet responses within the interest point neighborhood as a descriptor. An image is analyzed at several scales, so interest points can be extracted from both global and local image details. Therefore, the SURF extraction and matching scheme are one of the best interest point detectors and descriptors currently available.

(A) Extraction of feature points

First, based on the good performance in computation time and accuracy of the

Next, the SURFs approximate second order derivatives of the Gaussian with box filters. Image convolutions with these box filters can be computed rapidly by using integral images. The determinant of the Hessian matrix is written as:

)2 In order to localize interest points in the image and over scales, a non-maximum

62

suppression in a 3x3x3 neighborhood is applied. Finally, the found maxima of the determinant of the Hessian matrix are then interpolated in the scale and image space.

(B) Descriptor of feature extraction

The SURF descriptor is extracted from an image in two steps: the first step is assigning an orientation based on the information of a circular region around the detected interest points. The orientation is computed using Haar-wavelet responses in both x and y direction. Once the Haar-wavelet responses are calculated and weighted with a Gaussian (σ = 2.5s) centered at the interest points. In a next step the dominant orientation is estimated by summing the horizontal and vertical wavelet responses within a rotating wedge which covers an angle π/3 in the wavelet response space. The resulting maximum is then chosen to describe the orientation of the interest point descriptor.

In the method proposed in this study, at first we segment respective privacy-sensitive image from currently-processed surveillance images automatically.

Next, we extract feature points from these images using the SURF extraction equations (4.1) and (4.2), and then find the descriptor of the feature points in the currently-processed surveillance images. At last, the feature points in the currently-processed surveillance image shown in Figure 4.1, where the size of the circle specifies the scale and the line in the circle is the orientation of the feature point.

63

(a) (b)

(c) (d)

Figure 4.1 Feature extractions from surveillance image. (a) The color-image of the 23th frame (b) Feature points of the color-image of the 23th frame (c) The

depth-image of the 23th frame (d) Feature points of the depth image of the 23th frame.

After extracting feature points from both the privacy-sensitive image and the currently-processed surveillance images, we compare the motion object frame by frame. In this way, we can find the feature points of the object specifically, and won’t be affected by other factors. Therefore, when we obtain the matched feature points, we try to find a bounding box to include them. As shown in Figure 4.2, the red box is the bounding box of the matched feature points and the motion object. Finally, we obtain the region of each motion object, and match the privacy-sensitive image and currently-processed surveillance images together.

64

(a) (b)

(c) (d)

Figure 4.2 A result of the proposed method for detecting moving objects by use of SURFs. (a) The color-image of the20th frame compare with 21th frame. (b) The color-image of the 21th frame compare with 22th frame. (c) The depth-image of the 20th frame compare with 21th frame. (d) The depth -image of the 21th frame compare with 22th frame.

The following algorithm describes the process to match feature points between privacy-sensitive images and currently-processed surveillance images.

Algorithm 4.1: Detecting motion activities, and matching the privacy-sensitive image and currently-processed surveillance images by the use of SURFs.

Input: A surveillance color image sequence {Sc1, Sc2, …, Scn} and a surveillance depth image sequence {S_d1, S_d2, …, S_dn}; and an initial surveillance color image Sc0 and an initial surveillance depth image Sd0, both being rectangular in

65

shape with width w₀ and height h₀ and including an identical object (a human).

Output: A privacy-sensitive color image sequence Pc1, Pc2, …, Pcn and a privacy-sensitive depth image sequence P_d1, P_d2, …, P_dn.

Steps:

Step 1. For i = 0, 1, 2, …, n1, conduct the following the follows steps.

1.1 Extract a feature point set Fi from Pci by the SURF extraction algorithm mentioned previously.

1.2 Extract a feature point set Wi+1 from Sc(i+1) by the SURF extraction algorithm mentioned previously, respectively.

1.3 Match F_i with W_i+1 to obtain as a match set F_i' those feature points in W_i+1 which have corresponding feature points in F_i.

1.4 Find the feature point f_i'in the match set F_i', which has the minimum Euclidean distance Di+1 to the origin oi of Pci and whose corresponding feature point in W_i+1 is w_i+1.

1.5 Use the distance Di+1 between fi'and oi to find the corresponding points c_i+1 and d_i+1 of w_i+1 from S_c(i+1) and S_d(i+1), respectively.

1.6 Find the regions of Pc(i+1) and Pd(i+1) with origins ci+1 and di+1, respectively, both with width w_i and height h_i, in S_c(i+1) and S_d(i+1), respectively.

1.7 Take P_c(i+1) and P_d(i+1) as the output.

4.2.2 Proposed process of motion-activity