Twists and Exponential Products Formulation

The concept of twists and exponential products for kinematic chains is introduced by Murray [23] and is generalized to the application of 3-D human models by Mikic [13]. We can view the articulated structure of the human body as kinematic chains and the twists framework can be used to describe the rotation of the joints. It gives a concise description of the motion parameters and the motion constraints.

Moreover, the relationship between the parameters and the position of the points in the model is simple. In this section, we present the concept of the twists framework based on Mikic’s formulation.

Consider the rotation of a rigid object about a fixed axis as the simplest case.

Given that the direction of the axis is a unit vector ω   R³ and g  R³ is a point on the axis. After rotation by θ radians, the position of a point on the object is denoted as p(θ). Its velocity is then expressed as

( )θ ( ( )θ )

′ = ×ω −

p p g Eq. 3-17

Figure 3-17 An object rotates θ radians about a fixed axis ω

We can reformulate Eq. 3-17 in the form of the matrix and homogeneous coordinate representation:

ω

0 0 0 1 1

e^ξθ is viewed as the exponential map from the initial position of the point p on the object to its new position after rotating θ radians. Besides, e^ωθcorresponds to the rotation matrix R of the rigid object and t is the translation vector.

We can generalize this formulation to an open kinematic chain with m connected links. These links have their own rotation axes and different rotation angles. Let K(0) be the rigid body transformation which describes the position of the point on the last object of the chain, in terms of the base of the chain for the initial configuration. After the chain rotates Θ=[θ1 θ2 … θm]^T for each link, the transformation K(Θ) can be

where R(Θ) is the rotation matrix and t(Θ) is the translation matrix. The exponential product shown in the Eq. 3-21 can be used to describe the position of the points on an open kinematic chain properly.

Figure 3-18 The rigid body transformation of an open kinematic chain

3.2.1.2 3-D Skeleton Model

In our model, a human body is a 3-D skeleton composed of 12 segments and 23 parameters. as illustrated in Figure 3-19. It is based on the twists formulation and the position of each point in the model can be described using exponential products.

Figure 3-19 3-D human skeleton model world coordinate

Table 3-1 The notation of the lengths of the body parts

The human skeleton model refers to and is modified from Mikic’s 3-D human shape model [13]. It consists of 16 rotation axes for 9 joints and the rotation is formulated using the twists framework that is referred to the torso coordinate system.

For the rotation axes, their corresponding rotation angles are denoted as θ1, θ2, …, θ16, separately. The lengths of the body parts are decided beforehand based on the stature of the target person. To describe the position of each point in the model, we exploit the rigid body transformation and twists formulation. Therefore, we have to define the initial body configuration to manipulate the model by 16 rotation angles. The initial configuration is defined as the pose which is shown in Figure 3-19. The initial values of the parameters and the positions of the joints are listed in Table 3-2.

The positions of the points in the human model can be calculated using twits and exponential products formulation. The values of the positions and rotation parameters are based on the torso-centered coordinate system. Based on the concepts of the twist framework, the position of each point in the human body can be described by the rigid body transformation. For example, the position of the wrist in terms of the torso-centered coordinate is decided by the kinematic chain which consists of the lower arm, the upper arm and the shoulder. The transformation is defined by the function K(0) in the initial configuration. When the body starts to move, the transformation can be adjusted to K(Θ) via exponential products formulation.

Furthermore, we have to decide the transformation between the world and torso-centered coordinate systems to transform the torso-centered system to the world coordinates. The transformation is determined by the rotation and the translation of the torso center relative to the origin of the world coordinate. In Mikic’s design, the rotation axis of the torso is an arbitrary unit vector ω0 in the world coordinates [13].

However, it is not easy to control the orientation of the human model. The orientation of the human model determines which part is the right hand side. This information is important since the motion constraints for the right side and the motion constraints for

the left side are somewhat different. For example, the allowed rotation angle for the right elbow is [0,π]. However, it is [-π,0] for the left elbow. With the motion constraints, we can make our pose estimation more natural and reasonable.

Unfortunately, the Mikic’s method does have the problem in defining the orientation of the model. In Figure 3-20, we illustrate an example to explain this issue.

Table 3-2 The values of the parameters for the initial configuration Joints Position (torso-centered

coordinate) Rotation axis Rotation angle neck p1=

[

]

Figure 3-20 The weakness of Mikic’s method

As shown in Figure 3-20, if the initial pose estimation is in the back of the target person, it has the wrong body orientation. When we rotate the human model to the right orientation, it causes even larger fitting error since the model rotates about the axis with an inclined angle. Therefore, the system may pick up the one with wrong orientation for our pose estimation result. To determine the right side from the left side, Mikic switches the right side and left side of the human model and compares their fitting errors. For the case in Figure 3-20, it happens that the smaller fitting error actually corresponds to the wrong decision. A more natural thinking is that if we can make the model self-spin, the orientation of the human body can be easily manipulated and decided. Hence, we redefine the rotation axis of the human body as the torso stick of the model. Furthermore, the neck position and torso center control the incline of the human model. With self spin, the human model can easily spin to the correct orientation to obtain less fitting error. To sum up, the rotation axis of the human body is decided by the positions of the neck joint and the torso center. That is, the orientation is performed by rotating the human model about the z axis of the torso coordinate.

The total number of the human model parameters is 23 in our case, which include ˆp (the position of the torso center respect to the world coordinate), ₀ ˆp ₁ (the position of the neck joint respect to the world coordinate), θ0 (the spin angle) and 16 rotation angles for each joints. In the following context, we use ^ to indicate the positions respect to the world coordinate. Otherwise, the positions are in terms of the torso-centered coordinate system. The orientation of the torso is decided by the rotation matrix related to ˆp and ₀ ˆp : ₁

ω0

R L

L R

0 = pˆ1− pˆ0

Eq. 3-23 is derived from Eq. 3-20. Furthermore, we can apply the concept of exponential product to the calculation of the positions of each point in the human model. Consider the position p(0) of an arbitrary point in the model for the initial configuration. The rotations which influent the position of p is called the significant rotations by Mikic’s definition [13]. For example, if p refers to the position of the fingertip, the significant rotations include the rotation angles of the wrist, elbow, and shoulder. According to Eq. 3-21, we can find the new position of the point p after the pose change from the initial configuration. Assuming there are m significant rotations for p, the new position of p will be:

Finally, we need to transform the torso-centered coordinate system to the world coordinate system via the rotation matrix of the torso. Hence, Eq. 3-24 becomes:

0 0

We can reformulate Eq. 3-25 in the form of Cartesian coordinate:

0 1 2 2 1 0

ˆ( )Θ = ( ( ( (… _m (0)+ _m)+…)+ )+ )+ ˆ

p R R R R p t t t p Eq. 3-26

3.2.2 PSO Based Pose Estimation

PSO (Particle Swarm Optimization) has the advantages of being capable of dealing with nonconcave and nonlinear cost functions. Moreover, its computational cost is usually very light. This PSO method provides a powerful tool for dealing with an optimization problem in a high dimensional search space. Inspired by [8], we apply PSO to the fitting of the 3-D skeleton model to the extracted skeleton data.