Pattern generation

By joining two k-spatterns in the CV-tree, it generates a new (k+1)-spattern and the rplist of the new spattern is the intersection of rplists of two k-spattern. By using the rplist intersected, we can count the support of the newly generated spattern.

If the support is not less than minsup, we can add the rplist of the newly generated spattern to the CV-tree.

To join 2-spatterns P and Q in a video database with frame size of g×g, we have to shift the first pixel of P to match that of Q to avoid generating duplicate spatterns. If any pixel in the shifted P is greater than (g, g), we say that shifted P is not a valid pattern. Thus, the join operation is not allowed.

15

Definition 16. Any two frequent 2-spatterns S1 and S2 are joinable. The spattern of S1

joining to S2 is equal to S1∨S2, which is a 3-spattern, where ∨ is the logical OR operator for a bit-string. The rplist of the joined spattern is equal to the intersection of the rplists of S1 and S2.

For example, in Figure 4, the spattern of the first node (110 000 000) is joinable to (100 100 000) since the first pixels of both spatterns are at the same location. The intersection of the rplists of both spatterns are {(2,1,2,1), (2,1,3,1), (2,2,1,2), (2,1,2,2), (2,1,3,2)}. The joined spattern is equal to (110 000 000)∨(100 100 000)=(110 100 000), as shown in Figure 5(a), which is a 3-spattern with the support equal to 2. If minsup=2, the joined 3-spattern can be added to the CV-tree and becomes a child node of the node of (110 000 000).

Figure 5. Joining two spatterns

However, when the first pixels of both joining 2-spatterns are not at the same location, we fix the location of the latter 2-spattern and shift the former 2-spattern

(d) (c) (b) (a)

16

such that the location of first pixel of the former one is the same as that of the latter one. For example, to join both 2-spatterns (110 000 000) and (010 100 000) as shown in Figure 5(b), we shift the former spattern to (011 000 000) as shown in Figure 5(c) and then join both spatterns as shown in Figure 5(d). Finally, we can obtain a 3-spattern (011 100 000). Next we define a set of joinable nodes called joinable class.

Definition 17. Two frequent k-spatterns S1 and S2 are joinable if both share the first (k-1) pixels, k>3. The joined spattern is equal to S1∨ S2, which is a (k+1)-spattern. The rplist of the joined spattern is equal to the intersection of the rplists of S1 and S2. Definition 18. The joinable class of a frequent k-spattern S is JC(S)={S1,S2, …,Sn}, where S is joinable to Si and Si is a frequent k-spattern, k>2, 1<i<n. Note that the rplist of Si is a sibling node of the rplist of S. That is, the rplists of Si and S share the same parent in the CV-tree.

Definition 19. Two frequent k-vpatterns V1 and V2 are joinable if both share the first (k-1) spatterns, the last spatterns of both vpatterns are joinable, and all the time spans of both vpatterns are the same, k>2. The joined vpattern is obtained by replacing the the last spattern of V1 with the joined spattern of both last spatterns. The rplist of the joined spattern is equal to the intersection of the rplists of V1 and V2.

Definition 20. The joinable class of a frequent k-vpattern V is JC(V)={V1,V2, …,Vn}, where V is joinable to Vi and Vi is a frequent k-spattern, k>2, 1<i<n. Note that the node of Vi is a sibling of the node of V. That is, the nodes of Vi and V share the same parent in the CV-tree.

During the process of pattern generation, we first grow frequent video patterns in the spatial dimension and then grow them in the temporal dimension. To grow the frequent video patterns from a node of the CV-tree in the spatial dimension, we join the vpattern (P) of that node to each vpattern in P’s joinable class. If the joined pattern is frequent, it is added to the CV-tree and becomes the child node of node P. The procedure is repeated in a depth-first search manner until no more frequent vpatterns

17

can be found.

To grow the frequent video patterns from a node of the CV-tree in the temporal dimension, we first mine the frequent 2-spatterns in the projected database of the vpattern of that node so that the distance between the vpattern and each frequent 2-spattern mined is not greater than maxinterval. For each frequent 2-spattern mined, we append it to the vpattern to generate a new video pattern Q and compute the time span between the vpattern and 2-spattern. Next, we grow Q in the spatial dimension as the steps described above.

Let us consider how to mine vpatterns by a CV-tree as shown in Figure 6, where a part of the growing processes both in the spatial and temporal dimension is shown. In Figure 6, we use the example video database in Figure 2 to explain the process of growing patterns in the spatial and temporal dimensions. Assume that minsup=2 and maxinterval=2. After putting all frequent 2-spatterns to the second level of the CV-tree, we generate patterns from the first 2-spattern (110 000 000) The 2-spattern (110 000 000) can be joined to each pattern in its joinable class, namely, (100 100 000), (100 010 000), (100 000 100), (010 100 000), (010 000 100), (001 100 000). By joining (110 000 000) to each pattern in its joinable class, we obtain three frequent 3-spatterns, namely, (110 100 000), (110 000 100) and (011 100 000). We add these 3-spatterns to be the child nodes of node (110 000 000). Next, we grow the patterns from the 3-spattern node (110 100 000) in the same way and obtain two frequent 4-spatterns, (110 100 100) and (011 110 000). By joining both 4-spatterns, we obtain a 5-spattern node (011 110 010). At this point, we find that the joinable class of the 5-spattern is empty. Thus, no new spattern can be generated. Therefore, we finish growing the patterns in the spatial dimension.

Next, we start to grow the patterns in the temporal dimension. The projected database of the 5-spattern (011 110 010) is {(2,1,3,1), (2,1,2,2)}. We mine frequent 2-spatterns in the projected database so that the distance between every 2-spattern and

18

the 5-spattern is not greater than the maxinterval. Nevertheless, we cannot find any frequent 2-spattern in the projected database. Then, we backtrack to node (110 100 100) and grow the patterns from this node in the temporal dimension. The projected database of (110 100 100) is {(2,1,2,1), (2,1,3,1), (2,1,2,2), (2,1,3,2)}. We can mine five frequent 2-spatterns from the projected database, namely (110 000 000), (100 100 000), (100 000 100), (010 100 000), and (010 000 100). The projected database of the first frequent 2-spattern (110 000 000) is {(1,2,3,1), (2,1,3,1), (2,1,3,2)}. We compute the time span between the 4-spattern and the 2-spattern, which is [1,1]. Then we append the 2-spattern (110 000 000) to the 4-spattern to generate a new 2-vpattern, (110 100 100)[1,1](110 000 000). Similarly, we can find other four four 2-vpatterns as shown in Figure 6.

Then, we can grow the pattern from (110 100 100)[1,1](110 000 000) in the spatial dimension. We can obtain two 3-spatterns, (110 100 000) and (110 000 100), and one 4-spattern, (110 100 100). That is, we can obtain three v-patterns, (110 100 100)[1,1](110 100 000), (110 100 100)[1,1](110 000 100), and (110 100 100)[1,1](110 100 100). For each spattern obtained, we need to grow the patterns from it in temporal dimension. The steps described above will be recursively in a depth-first search manner until no more patterns can be found.

19

Figure 6. Growing patterns in the spatial and temporal dimensions

20

Figure 7. The growing representation of the CV-tree

To explain our concept more clearly, we use the example shown in Figure 7 to demonstrate the growing process, where the s-nodes and t-nodes are generated in the DFS manner recursively. In this example, we start from node A of a 2-spattern. Three s-nodes of 3-spatterns are generated from node A by joining the pattern of A to the patterns in its joinable class, where the joinable class of A contains the sibling nodes of A. Next, one s-node of a 4-spattern is generated from node B and no s-node can be grown from node C. Thus, we start to grow t-nodes by mining frequent 2-spatterns from the projected database of C so that the time span between the pattern of C and each frequent 2-spattern mined is not greater than maxinterval. Then, we append four frequent 2-spatterns mined to C and form four frequent 2-vpatterns, each of which consists of one 4-spattern and one 2-spattern.

For the vpattern of node D, we continue to generate two 3-spatterns E and F in the spatial dimension. No s-node and t-node can be generated from node E. Thus, E is

21

a leaf node. No s-node but one t-node G can be grown from node F to form a 3-vpattern. Then, we backtrack to node D and start to grow t-nodes from it. Since we can find a frequent 2-spattern H, a new 3-vpattern can be formed. Since H is a leaf node, we backtrack to node D then node C, and go to node I. The steps described above will be repeated in a depth-first search manner until no more patterns can be generated.