利用局部動作查詢與組合之角色動作合成

(1)

國

立

交

通

大

學

多媒體工程研究所

碩

士

論

文

利用局部動作查詢與組合之角色動作合成

Partial Motion Query and Assembly for Character

Animation

研究生：江國豪

指導教授：林奕成助理教授

(2)

利用局部動作查詢與組合之角色動作合成

Partial Motion Query and Assembly for Character Animation

研究生：江國豪 Student：Gwo-Hao Juang

指導教授：林奕成 Advisor：I-Chen Lin

國立交通大學

多媒體工程研究所

碩士論文

A Thesis

Submitted to Institute of MultimediaEngineering College of Computer Science

National Chiao Tung University in partial Fulfillment of the Requirements

for the Degree of Master

in

Computer Science July 2007

Hsinchu, Taiwan, Republic of China

(3)

利用局部動作查詢與組合之角色動作合成

研究生：江國豪指導教授：林奕成博士國立交通大學多媒體工程研究所

摘要

近年來動作捕捉技術正被廣泛的應用在電影、電玩與多媒體相關產業中。因此，如何運用這些捕捉而來有限的動作資料一直是很重要的課題。在本篇論文中，我們將提出一個簡單又直覺的局部動作查詢與合成系統，藉由此系統有效又方便的創造出看起來自然又真實但原本資料庫中沒有的動作，以擴大動作資料庫的變化性。我們的想法是，若我們找出看起來相似的局部動作，將相似的局部動作更換後的動作該會是很接近自然的。因此，我們的系統提供一個簡單的動作編輯器，讓使用者藉由幾個關鍵的姿勢創造出期望之動作雛型。再由我們的系統經由查詢的方式至資料庫中將相似的動作片段挑出。使用者可從中選出準備接合的基礎動作與替換之局部動作來組合。為了保持原局部動作的行為變化與接合的自然性，我們利用動作統計分佈的方式做調整。本論文所提出之系統將可使使用者快速建立合理之動作片段且可擴增原資料庫之動作變化性關鍵字：動作合成、資料查詢、局部動作、動作編輯

(4)

Partial Motion Query and Assembly for

Character Animation

Student: Gwo-Hao Juang Advisor: Dr. I-Chen Lin

Department of Computer Science National Chiao Tung University

ABSTRACT

Using motion capture data to create novel and realistic motions is an attractive topic in motion synthesis. In this thesis, we propose a simple and intuitive editing tool to assemble the different parts of character motion to create novel and natural-looking motion. Herewith the variety of the database is enriched. In our system, users can assign the partial figure of a query motion which they want to synthesize. Our system then retrieves partial figures from large data sets similar to the assigned ones. Finally, we assemble the different parts of character motion. However, simply assembling partial figures may cause unrealistic or strange motions. To solve this problem and also keep the query motion style, we use the statistic properties between the query motion and pre-assembly motion. When assembling the retrieval partial figures and adjusting the variance according to the query motion, our system can retain the style and naturalness of the synthesized motion.

(5)

Acknowledgements

First of all, I would like to thank my advisor, Dr. I-Chen Lin, for his guidance in the past two years. Also, I appreciate all members of Computer Animation & Interactive Graphics Lab for their help and comments. Finally, I am grateful to my family for their support and encouragement.

(6)

List of Figures and tables

Figure 1: Motion graph………..4

Figure 2: Time alignment restrictions………5

Figure 3: Match graph……...……….5

Figure 4: Geometric feature and Segmentation………..6

Figure 5: The right arm and leg of a walk (Left top), walk& throw (Left bottom), the difference (Right)……….7

Figure 6: The figure is constrained to be tripping, then running, then jumping while running……….7

Figure 7: (a) and (b) are real motion capture data. (c) is transplanted from (a) and (b)………...8

Figure 8: A manipulating task for a long thin vertical object……….9

Figure 9: Mean ratings of motions with horizontal, vertical, and gravity errors...9

Figure 10: System flowchart………11

Figure 11: Key-framing animation generator………...12

Figure 12: Control panel. Part A and B are the key frame and joint index selection button respectively, part D and E are the joint orientation and root position controller. After creating the needed key frames of motion, a user can also create the needed size of motion (C)……….13

Figure 13: A walking motion with 45 frames………...14

Figure 14: Feature vector of a walking motion clip with 45 frames………15

Figure 15: Segmentations of feature vectors………15

Figure 16: The top motion is a retrieved motion with 25 frames. Left to right are frame 1, 4, 23 respectively. The bottom motion is the query motion with 32 frames, left to right are frame 1, 15, 32 respectively………18

Figure 17: Motion segment alignment……….19

Figure 18: Punch motion (top) and kicking motion (bottom) with the similar arms’ trajectories……….20

Figure 19: Skeleton of human body (file format .bvh)………21

Figure 20 An example of punch motion (a) z-axis orientation of the critical joint in query motion. (b) z-axis orientation of the critical joint in retrieval motion. (c) z-axis orientation of the critical joint in assembly motion. It is clear that after the statistic tuning the local distribution of assembly motion is similar to the retrieval motion but the mean and standard deviation is close to the query motion……..23

Figure 21: Key poses of user editing punch motion……….…...26

Figure 22: Key poses of user editing walk motion………..26 Figure 23: The assembly system. (a)the query motion (b) the assembly motion (c)

(9)

the retrieval motions………27 Figure 24: Assembly directly (top motion sequence) and assembly with statistic adjustment (bottom motion sequence)……….26 Figure 25: A punch motion sequence (from left to right) of query motion (top two), retrieved motion (middle two) and assembly motion (bottom two)……...27 Figure 26: Another punch motion sequence (from left to right) of query motion (top two), retrieved motion (middle two) and assembly motion (bottom two)…28 Figure 27: Walking style motion of query motion (top), a retrieved motion (middle) and assembly motion (bottom)………..29 Table 1: Definition of the features………17 Table 2: Motion Classes………...24 Table 3: Availability of punch style motions with 81 retrieved results. P1~P8: 8 punch style motions S1~S4: subject 1 to 4……...…………...25 Table 4 The availability of walk style motions with 168 retrieved results. W1~W16: 16 walk style motions 1 to 16. S1~S4: subject 1 to 4………..……..25

(10)

1. Introduction

1.1 Motivation

In recent years, human body animation has become an important topic from academic research to industries. Two major animation techniques have been developed to create human animations: simulation-based and data-driven. Simulation-based motion synthesis generates human motions by applying kinematics, physics and biomechanical models. These approaches provide a flexible representation of motion. Nevertheless, it is sometimes difficult to construct a natural-looking motion if users do not provide appropriate constraints. Data-driven approaches use the motion capture techniques (MoCap) to acquire real persons’ motions to MoCap data. This technique can easily obtain rich details and styles of human motion. The main advantage is that it can produce complex and realistic motions. In our thesis, we employ this technique to accomplish our system. However, the acquisition of motion capture data is a time-consuming and labor-intensive job. The storage size is also another problem. So, how to efficiently reuse compact size of motion capture data to enrich the variety is what we focus on.

To synthesize novel motion based on mo-cap data, researchers developed varies kinds of editing methods. L. Kovar and M. Gleicher[2] introduced the motion graphs which can automatically generate transitions between pieces of original motion. K. Yamane also synthesized the human manipulation tasks by using a path planner with the domain knowledge inherent in data-driven, constraint-based inverse kinematics. Before motion synthesis, it is important to find out the relationship between motion data. Intuitively, the major relationship we need is the similar poses between different

(11)

motion data. One frequently used approach is to establish a distance metric regardless of root orientation to locate the best match frames of each motion clip. But this method is a time-consuming problem since it has to compute distance of each frame pair. In order to extract the meaningful part of motions and simultaneously improve the efficiency. Muller et al.[1] has brought up an approach that extracted the logically similar motions by introducing various kinds of qualitative features and they used feature vectors to retrieve the input motion. When we have the extracted motion segments, various kinds of synthesis approaches could be applied. Nadia et al. used Combined Partial Motion Clips (CPMPs)[16] to synthesize a user specified motion with other subject’s walking, standing and sitting base motions, such as throwing during walking. Leslie advanced a method that cutting limbs from one motion sequence and attaching them in another motion sequence. Accordingly, we provide an intuitive editing tool allowing user to simply create a novel motion by retrieval and assembling different partial figures of character motion.

1.2 Overview

In this thesis, we present an interactive system to replace and synthesize the partial figure of character motion from large motion data sets and still retain the natural visualization of human motions. We separated our system into two major phases:

1. Partial motion figure editing and retrieval 2. Partial motion assembly.

In the first phase, we aim at the retrieving partial motion of the human body, such as waving hands, running legs or walking leg, etc. In the beginning, the user can create a simple motion by editing the major key frame. Then our system will retrieve

(12)

the candidate motion clips which has the similar partial figure to the editing motion. Like the approach introduced by Muller et al[2], we define geometric relations to locate the partial figure and use a feature vector to indicate each of the features. By using these feature vectors, we can easily extract the partial figures similar to the input motion. In the second phase, we assemble the extracted partial motion with the original input motion. In our system, we are not only synthesizing the similar partial motions but also the similar style motions. We combine other different partial figures but still have the same style by employing the statistic properties between two different motions.

1.3 Contribution

The contributions in the thesis are (1) We support a simple and intuitive partial figure motion retrieval and assembly system. (2) Using statistic property to adjustment the junctures’ orientation in assembly phase instead of complex physics rules. (3) Extensive enrich the variety of our database about punch and walking style motion. For example, if we have retrieved n punch motions from database, we can create at most n*(n-1) assembly punch motions.

1.4 Organization

The remainder of this paper is divided into five chapters. First, we review related work in chapter 2. Second, we introduce our method about motion retrieval and assembly in chapter 3. Then, we show our experimental results in chapter 4. At last, chapter 5 gives the discussion conclusion.

(13)

2. Related Works

2.1 Motion retrieval

The reuse of motion capture data has been an important topic in computer animation. In order to efficiently reuse the captured motion data, motion retrieval became an important part of various motion editing tools. The critical point of motion retrieval is the notion of similarity. There are plenty of approaches proposed for this purpose. These approaches can be separated into two classes: numerical similarity and logical similarity. In numerical similarity, given a distance metric and a query motion sequence, the system searches the database and retrieves motion sequences whose distance to the query are below a threshold. L. Kovar & M. Gleicher[2] (Fig. 1) defined a simple distance metric to calculate the distance between two frames and then used the result to create transitions between pieces of original motion.

(a) Original motion capture data (b) New motion generated by motion graph

Figure 1 Motion Graph

In 2004, L. Kovar & M. Gleicher[4] further provided two criteria to determine numerical similarity: (Fig. 2 & 3)

(14)

2. Frame correspondences should be easy to identify.

Figure 2 Time alignment restrictions

Figure 3 Match graph

Following these two criteria, they used the technique called time alignment to seek out the similar motion sequence. Most of the previous methods in comparing motion frames used 3D positions or joint angles. However, these approaches are computationally expensive, especially when the motion data sets become larger. There is also a potential drawback of this technique, that is, numerically similar motions would not always be perceptually similar. It would cause some unsatisfactory retrieval results. For instance, when querying with a straight punch it may skip hook punches. In logical similarity, instead of using numerical features, the geometric features are chosen to compare the similarity between motion clips. M. Muller et al [1] introduce a class of Boolean features expressing geometric relations between certain body points

(15)

of a pose.(Fig. 4)

(a) Segment of walking motion

(b) Geometric feature of foot

Figure 4 Geometric feature and Segmentation

Besides, they also simplified the notation of the motion clips. In this retrieval method, they give users a flexible retrieval process. Users can concentrate on a special part of motion but not the full motion. This concept inspired our proposed method.

2.2 Motion synthesis

Besides motion retrieval, we concerned about the assembling different partial motions of the human body. N. Al-Ghreimil and J. K. Hahn provided a motion editing method called Combined Partial Motion Clips (CPMCs)[16]. (Fig. 5)

(16)

Figure 5 The right arm and leg of a walk (Left top), walk& throw (Left bottom), the

difference (Right)

The method estimated the detailed motion information for parts of the articulated figure. They first collected the base motion with the same style of partial motion such as reach when sitting, reach when running, etc, and then analyzed the trajectory of there difference and store the main trajectory. At last, they added all the computed and selected trajectories to their corresponding ones in the base motion by a simple vector addition at each frame.

Figure 6 The figure is constrained to be tripping, then running, then jumping while

running

(17)

synthesize human motion and still retains its qualitative properties. First, the user annotates a large collection of motions and then paints a timeline with annotations from a vocabulary which is freely chosen by the user. The system will assemble frames from motion database and show the result of specified actions at specified times. L. Ikemoto and D.A. Forsyth also proposed a method that can cut limbs from one motion sequence and attach them to another to enrich a motion collection. The method uses randomized search based on four rules to generate transplants that are likely to be successful. The results are evaluated by a SVM classifier to tell whether they are natural-looking or not. (Fig. 7)

(a) (b) (c)

Figure 7 (a) and (b) are real motion capture data. (c) is transplanted from (a) and (b)

Because of the transplant generates many frames quite close to the original frames, so that it is easier to find a path for the motion synthesis process. There is still another popular approach to motion synthesis by constructing statistical models. K. Yamane et al.[18] explore an approach that employ a path planner to find a motion satisfied the geometric, kinematics, and posture constraints for animating characters manipulating objects. (Fig 8)

(18)

Figure 8 A manipulating task for a long thin vertical object

(a) (b)

Figure 9 Mean ratings of motions with horizontal, vertical, and gravity errors.

In 2003, Paul S. A. Reitsma and N S. Pollard introduce a study of user sensitivity to errors in animated human motion. (Fig. 9)

(19)

1. Subjects found added acceleration easier to detect than added deceleration 2. Subjects found low gravity easier to detect than high gravity

3. Subjects found errors in horizontal velocities easier to detect than errors in vertical velocities

(20)

3. Partial figure retrieval and

assembly

(21)

In this chapter, we introduce the framework of our system (Fig. 10). Since we want to enrich the database by assembling the different partial figures from compact size of motion capture data, how to obtain the required motion from the database is the first concern. We provide a simple and intuitive animation generator(Fig. 11 & 12) for user to create the skeletal motion with several key frames. Then, we retrieve motion clips which are similar to the designated ones for user to select. Finally, we assembly and replace the partial figure of the selected motion clip.

(22)

Figure 12 Control panel. Part A and B are the key frame and joint index selection

button respectively, part D and E are the joint orientation and root position controller. After creating the needed key frames of motion, a user can also create the needed size of motion (C).

We separate our system into two major phases. According to our purpose of synthesizing partial figure of human motion, we aim at finding the similarity of partial figure instead of the whole human motion The first phase is about motion retrieval. We adopted the concept of relational motion features proposed by M. Muller et al in 2005 First, we represent the poses by a boolean value for each feature from database, and give a query motion with user-specified features. By comparing the geometric features between query motion and those in database, we retrieve the similar partial figure of character motion. The detail of this phase will be illustrated in section 3.1. After we acquire several logically-similar partial motions from database, we propose using these retrieved motions to replace the original one. With the assembly stage, we can dramatically enrich the variety of synthetic motion. How to assemble partial motion will be elaborated in section 3.2.

(23)

3.1 Partial figure retrieval

3.1.1 Definition and Combination of Geometric

features

In this section, we present the detail method about how we extract the logically similar motion between two motions. M. Muller et al used a class of boolean features indicating geometric relations of a pose. For instance, (1) represents that the right foot lies in front of the plane spanned by the left foot, the left hip and the root and (0) represents the contrary. Let’s take a walking motion as an example (Fig. 13 and 14).

(24)

Figure 14 Feature vectors of a walking motion clip with 45 frames

Relation features are useful to identify the correspondence of logically similar motion. Further, by combining the related features, it can be more useful to represent a special motion. Scilicet, a walking pose would be represented by a feature vector

T ) 1 , 0 , 0 , 1

( which means left hand in the front, right hand behind, left foot behind and right foot in the front respectively. However, in our research, we focus only on the partial figure of the character motion but not the whole figure. We also include special features in order to extract more detail representation and the partial style of a pose. These geometric feature sets are described in Table 1. Instead of using a boolean expression to indicate a feature, we use a more diversified expression to represent this kind of feature. For example, we use 1 to indicate a moving forward motion, 0 for moving backward and 2 for immoveable motion.

Now we have transformed the motion data streams into simple feature vectors, and then we merge the consecutive subsequence with the same feature vectors of related features called a segment. For the example in Fig. 14, the lower body of walking motion now can be simply represented by (Fig. 15)

Figure 15 Segmentations of feature vectors

3.1.2 Retrieval

(25)

segment sequence. However, using exacted match would miss plenty of candidate motion segments with logical similarity. In the example of Fig.14, the third feature segment (0,0)T means that the foot almost lies on the same line parallel to the one spanned by right shoulder and left shoulder. It might not appear in other walking motions because of the numerical error. In fact, the major feature of walking motion is that the right foot in front of the left foot and the left foot is in front of the right foot alternately. So, we use the fuzzy comparison introduced by M. Muller et al. Let

) ,..., ,

( 1 2 L

Q V V V

F = be the motion segment sequence of query motion Q with f

l

V ⊆(0,1) for 1 < l < L, and F_D =(w₁,w₂,...,w_M) be the target sequence in database with f

m

w ∈(0,1) for 1 < m < M. Besides, V I_l V_l₊₁ =φ . The search hit means that

L L k k k D Q

F

w

V

w

V

w

V

F

⊂

⇔

∈

₁

,

₊₁

∈

₂

,...,

₊ ₋₁

∈

. The set of all hits in the database can be denoted as

}

|

]

:

1 [

{

)

(

F

_Q

k

M

F

_Q

F

_D

Hit

=

∈

⊂

where k means the started feature vector index of the target sequence.

We now use an efficient method called inverted lists to evaluate Hit(F_Q). For each Vl we have a list L(V_l) ={k∈[1:m]|V_l ⊂ F_D} where k means that the k-th feature vector of FD equals one of the vector in V . So,l Hit(FQ) can be computed by

I

] : 1 [

)

1 )

(

)

(

L l l Q

L

V

l

F

Hit

∈

+

−

=

with

U

l V v l

L

v

V

L

∈

=

(

)

(

_.

Now, we give a simple example for each V_l =

{ }

v withv∈(0,1)f . The walking

motion is ) 0 1 1 0 0 0 0 1 0 0 1 0 0 0 0 1 ( _                                                =

D and query motion is )

1 0 0 0 0 1 (             = Q .

(26)

Then

)

{

1 ,

5 ,

8 }

0

1 (

=









L

, ) {2,4,6} 0 0 ( _ =      L ,

)

{

3 ,

7 }

1

0 (

=









L

.

)

2 )

1

0 (

(

)

1 )

0

0 (

(

)

0

1 (

)

(

−









−

















=

L

F

Hit

_Q

I

=

{

1 ,

5 ,

8 }

I

{

1 ,

5 }

I

{

1 ,

5 }

=

{

1 ,

5 }

This indicated that the extracted motions start at 1st and 5th segment of D, respectively.

There would be a problem about the inverted list, since the number of the lists is dependent on the number of features. In M. Muller et al, they define 31 geometric features. This caused large number (231) of the lists. They settled this problem by dividing the features into three parts: lower body, upper body and the interaction of upper and lower body. It reduced the number of the lists to (211 +212 +28). However, in our system, the problem would not occur, since we only focus on the partial figure of the motion. For examples, the motion of the legs, the arms, etc. We do not concern about the whole human figure. In fact, our inverted list could only be 2 for each 2 process.

Feature Set Features Description F1 Is right foot in front?

1: in front, 0:behind F2 Is left foot in front? 1: in front, 0:behind F3 Does right elbow bend?

1: bend, 0:stretch General

feature

F4 Does left elbow bend? 1: bend, 0:stretch

(27)

In our system, we defined 4 major features in order to retrieve the walking styles and punch styles. F1 and F2 are used to retrieve the lower body of walking motion, F3 and F4 are used to retrieve the arms of punch motion.(Table 1)

3.2 Partial figure assembly

In the above steps, we have retrieved the partial figures similar to users’ request. In this section, we will explicate the detail about the partial motion assembly. Several issues have to be settled.

3.2.1 Segment alignment

Figure 16 The top motion is a retrieved motion with 25 frames. Left to right are

frame 1, 4, 23 respectively. The bottom motion is the query motion with 32 frames, left to right are frame 1, 15, 32 respectively

(28)

The first issue is the velocity problem in different motion clips. We take punch motion as an example. The query motion and retrieved motion do not have the same frame number in each segment. To settle this problem, we take advantage of the property of segmentation in the retrieval step. In Fig. 16, the top motion sequence is a motion with 2, 5, 18 frames in each segment respectively. The bottom ones are 7, 9, 16 frames. So, the top motion looks faster than the bottom one in the first two segments but slower in the last segment.

Since we consider that two motion clips similar when they have the same feature vector segments. The benefit of the method is that it has highly logical similarity in each segment. This makes it easily to settle the velocity problem without using dynamic time warping techniques (DTW). In order to retain the style of the retrieval figure and have the query motion’s velocity. We propose using the Bezier interpolation in joint angles in each segment. (Fig. 17)

(29)

After this step, the partial figure motion would have the same speed to the query motion and still have its own style properties. However, it may cause discontinuous between segments. To make assembly motion smooth and continuous, we do another Bezier interpolation on the whole assembly motion clips.

3.2.2 Different type adjustment

Figure 18 Punch motion(top) and kicking motion(bottom) with the similar arms’

(30)

The second issue is natural-looking adjustment. In our system, we first abstract the similar partial figure from different motion clips. Intuitively, it should be reasonable when we assembly the similar retrieval partial figure to the query motion. However, assembling different parts of motion directly would cause unnatural-looking results when the retrieved motion is not analogue to the query motion. In the case of punch motion, it might retrieve the arm from a kicking motion because they have similar trajectories of arms. (Fig. 18)

(31)

For the sake of making the assembly motion look naturally. We utilize the statistic properties to improve the reliability of the assembly motion. First of all, we must ensure the style of the retrieved partial figure should be retained. From our observation, we find out improper angles of key joints, called critical joints, will degrade the assembly results. In our experiment, we discover that shoulder is the critical joint in punch motion and the chest is the critical one in walking motion (Fig. 19). In fact, the junctures are usually the critical points. When we get the critical joint, we analyze the joint orientation in x, y, z-axes respectively. We exploit the mean and standard deviation of query and retrieved motion to evaluate the anticipated orientation for further adjustment.

z y Qx z y Qx z y Rx z y x R z y x i z y x i

M

R

S

_, _, _, _, , , , , , , , ,

∗

+

−

=

σ

for 1 < i < m

Where

S

i_x_,_y_,_z and

R

i_x_,_y_,_z are anticipated and retrieved critical joint’s x, y, z

orientation respectively. M is the size of anticipating motion.

M

Rx,y,z and

z y x Q

M

,

, are mean of the retrieval and query motion clip.

σ

Rx,y,z and

σ

Qx,y,z

are standard deviation of the query and retrieval motion clip.

This formula ensures that the critical joint of assembly motion retain the local variation of the retrieved motion and have the similar global distribution to the query motion. We show the z-axis orientation in Fig 20. With the hypothesis, we avoid the complex computation about the physical theorem but still have a natural-looking assembly result.

(32)

(a) (b)

(c)

Figure 20 An example of punch motion (a) z-axis orientation of the critical joint in

query motion. (b) z-axis orientation of the critical joint in retrieval motion. (c) z-axis orientation of the critical joint in assembly motion. It is clear that after the statistic tuning the local distribution of assembly motion is similar to the retrieval motion but the mean and standard deviation is close to the query motion.

(33)

4. Experiment and results

In this chapter, we will describe our experiment and the results. At first, we introduce the collection of our motion capture data and the available ratio of our experiment. Finally, show the final results.

4.1 Collection of the motion capture data

There are 226 original motion capture data with 22437 frames (about 680 seconds) in our experiment (Table 2).

Motion Number 226 Total motion size 22437 frames

Motion types Punch 40 Kick 25 Walk 10 Run 10 Jump 18 Shadowbox 40 defense 6 Reach 7 Dodge 7 push 16 Be hit 18 Pull 7 Combination motion 22

Table 2 Motion classes

4.2 Experiment

In order to test the naturalness of our synthetic result, we invite four males subject to measure the results. Given a mark O if the assembly result is considered okay and a mark X on the contrary.

We test the punch and walk style motions for our system. First, user simply edits key frames of a motion. Because our system will have n2 assembly results when we have n retrieved motion clips. In order to shorten the experiment time, we randomly choose 1/10 retrieved motion clips to start our study (Table 3 and Table 4).

(34)

Punch style motions

Retrieved results 81 motion clips The # of retrieved punch motion clips 43

P1 P2 P3 P4 P5 P6 P7 P8 Subject 1 62 54 57 58 63 60 63 62 Subject 2 70 64 47 63 68 57 67 50 Subject 3 73 73 52 80 69 68 71 72 Subject 4 69 66 50 64 62 68 61 69 Average accepted ratio 84% 79% 63% 82% 81% 79% 81% 78% The ratio of availability 78%

Table 3 The availability of punch style motions with 81 retrieved results. P1~P8: 8

punch motions. Subject1~Subject4: subject 1 to 4.

Walk style motions

Retrieved results 168 motion clips The # of retrieved walk motion clips 95

W1 W2 W3 W4 W5 W6 W7 W8 Subject 1 97 116 100 102 111 98 106 110 Subject 2 102 107 97 103 106 96 110 107 Subject 3 113 98 102 100 101 103 107 99 Subject 4 101 112 90 104 98 105 99 95 Average accepted ratio 63% 64% 59% 61% 62% 60% 63% 61% W9 W10 W11 W12 W13 W14 W15 W16 Subject 1 92 102 116 110 102 94 106 105 Subject 2 97 109 115 104 105 97 103 103 Subject 3 103 99 108 102 107 92 100 114 Subject 4 87 106 104 94 96 90 97 101 Average accepted ratio 56% 62% 66% 61% 61% 55% 60% 63% The ratio of availability 61%

Table 4 The availability of walk style motions with 168 retrieved results. W1~W16:

(35)

The result shows that we have assembled 63%~82% motions which considered naturally about punch style motions and 55%~68% about walk style motions.

4.3 results

(a) key pose 1 (b) key pose 2 (c) key pose 3

(d) key pose 4 (e) key pose 5

Figure 21 Key poses of user editing punch motion

(a) key pose 1 (b) key pose 2 (c) key pose 3 (d) key pose 4

(36)

Figure 23 The assembly system. (a)the query motion (b) the assembly motion (c) the

retrieval motions

Figure 24 Assembly directly (top motion sequence) and assembly with statistic

(37)

1 4 12 ₁₄ 16 19 21 31 1 5 8 12 15 18 24 38 1 4 12 14 16 19 21 31

Figure 25 A punch motion sequence (from left to right) of query motion (top two),

(38)

Figure 26 Another punch motion sequence (from left to right) of query motion (top

(39)

1 9 19 25 33 43 55 63 1 8 16 ₂₄ 32 40 48 53 1 8 16 24 32 40 48 53

Figure 27 Walking style motion of query motion (top), a retrieved motion (middle)

(40)

5. Conclusions and Future works

In this thesis, a simple and intuitive motion assembly tool is presented. The system contains three parts — Key pose establishment, retrieval of partial figure of query motion, partial figure assembly of retrieved motion and the query one. First, a user can simply assign several key poses of a motion. Second, our system will semi-automatically retrieve the partial figure similar to user’s selection. Third, the system replaces the partial figure of query motion and assembles the one from retrieval motion by a statistical adjustment.

Our major contributions are (1) A simple and intuitive motion retrieval and assembly system. (2) Using statistic property to adjustment the junctures’ orientation in assembly phase instead of complex physics rules. (3) Extensive enrich the variety of our database about punch and walking style motion.

In our future works, we will add more geometric features to retrieve different partial figure action. With these different partial figures, we can arbitrary combine them to create a novel motion. We would also plan to add physics-based adjustment to make our result more convincingly. For instance, space-time constraint with physics rules. However, there is a limitation of our current system. We can not assemble too complex motion, such as the arms figure of tai chi. In our system, we use the most important feature to retrieve the partial figure. We can also combine more features to make the retrieved results become more similar to the query motion. However, this also restricts the variation of the retrieved results. Therefore, the tradeoff between less combination of features and more combination of features will become an important issue in our work.

(41)

6. References

[1] MÜLLER M., RÖDER T., CLAUSEN M. 2005. Efficient content-based retrieval of motion capture data. ACM Transactions on Graphics 24, 3, 677–685.

[2] KOVAR, L., AND GLEICHER, M. 2004. Automated extraction and parameterization of motions in large data sets. ACM Transactions on Graphics 23, 3, 559–568.

[3] KOVAR, L., AND GLEICHER, M. 2003. Flexible automatic motion blending with registration curves. In Proc. ACM SIGGRAPH 2003 / Eurographics Symposium on Computer Animation, Eurographics Association, 214–224.

[4] KOVAR L., GLEICHER M., PIGHIN F. 2002. Motion graphs. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques, ACM Press, 473-482.

[5] KOVAR L., GLEICHER M. 2003. Flexible automatic motion blending with registration curves. ACM SIGGRAPH / Eurographics Symposium on Computer Animation (Aug. 2003), 214-224.

[6] WITKIN, A., AND POPOVIC, Z. 1995. Motion warping. In Proc. ACMSIGGRAPH 95, ACM Press / ACM SIGGRAPH, Computer Graphics Proc., Annual Conf. Series, 105-108.

[7] POPOVI ´C Z., WITKIN A. P. 1999. Physically based motion transformation. In Proceedings of SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, 11-20.

[8] ARIKAN O. AND D.A. FORSYTHE. 2002. Interactive motion generation from examples. In Proceedings of ACM SIGGRAPH, Annual Conference Series.

[9] JEHEE L. AND SUNG YONG SHIN. 1999. A hierarchical approach to interactive motion editing for human-like figures. In Proceedings of ACM SIGGRAPH, Annual Conference Series, 39–48.

[10] IKEMOTO, L., AND FORSYTH, D. A. 2004. Enriching a motion collection by transplanting limbs. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, 99-108.

[11] POLLICK, F., HALE, J. G., AND MCALEER, P. 2003. Visual perception of humanoid movement. In Proceedings Third International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems 101, 107-114.

[12] REITSMA, P. S. A., AND POLLARD, N. S. 2003. Perceptual metrics for character animation: Sensitivity to errors in ballistic motion. ACM Transactions on Graphics 22(3), 537-542.

[14] HODGINS, J. K., O'BRIEN, J. F., AND TUMBLIN, J. 1998. Perception of human motion with different geometric models. IEEE Transactions on Visualization

(42)

and Computer Graphics 4, 4 (October), 307-316.

[15] HECK R., KOVAR L. AND GLEICHER. M. 2006. Splicing Upper-Body Actions with Locomotion. Computer Graphics Forum (Proceedings Eurographics) [16]N. Al-GHREIMIL, JAMES K. HAHN. 2003. Combined partial motion clips. In Proceedings WSCG

[17] YAMANE K, JAMES J. KUFFNER, and JESSICA K. HODGINS. 2004. Synthesizing animations of human manipulation tasks. ACM Transactions on Graphics.23(3)

[18] ARIKAN O., FORSYTH D. A., O’BRIEN J. 2003. Motion synthesis from annotations. ACM Transactions on Graphics 22, 3, 402–408.

利用局部動作查詢與組合之角色動作合成

國

立

交

通

大

學

多媒體工程研究所

碩

士

論

文