Intention in Tool-handling Task

Fig. 3.1 shows the conceptual diagram of the proposed approach for intention de-duction from demonstration. In Fig. 3.1, the robot first observes a series of human demonstrations and records the corresponding trajectories and environmental states.

From these recorded motion data, the robot searches for the possible intentions that lead to the delicate motions. The derived intentions can then be used to generate new trajectories that respond to new environmental states. Let us take the pouring task shown in Fig. 3.2 as an example. In Fig. 3.2(a), three vessels A, B, and C are arbitrarily located on the table. And, in Figs. 3.2(b)-(c), the operator pours the content from vessel A to vessels B and C, respectively, and then places vessel A back on the table. During the demonstrations, the initial locations of the vessels may vary, and so does the pouring sequence. From the recorded trajectories and corresponding locations of the vessels (environmental states), the proposed approach identifies the intention of the operator, i.e., the portions of the trajectory that

cor-Human

Figure 1 The conceptual diagram of the proposed approach for intention learning by demonstration.

Figure 3.1: Conceptual diagram of the proposed approach.

Human

Fig. 4: Conceptual diagram of the proposed approach.

Fig. 5: A pouring task: (a) the setting of the vessels, (b) pouring vessel A to vessel B, and (c) pouring vessel A to vessel C.

A to vessels B and C, respectively, and then places vessel A back on the table. During the demonstrations, the

initial locations of the vessels may vary, and so does the pouring sequence. From the recorded trajectories and

corresponding locations of the vessels (environmental states), the proposed approach will identify the intention of

the operator, i.e., the portions of the trajectory that correspond to the two pouring actions (delicate motions). With

the derived intention, the robot is then able to execute the pouring task with the vessels located at various locations

and possibly altered pouring sequences.

Before the discussion on the process of intention deduction, we first describe how the motion can be generated

under new environmental states when the human intention has already been derived. We start with the representation

of the intention I. Assume that there are N delicate motions and S objects involved in a demonstrated task. Because

the intention is closely related to the delicate motions of the maneuver, I is formulated as a set of delicate motions, Figure 3.2: A pouring task: (a) the setting of the vessels, (b) pouring vessel A to

vessel B, and (c) pouring vessel A to vessel C.

respond to the two pouring actions (delicate motions). With the derived intention, the robot is then able to execute the pouring task with the vessels located at various locations and possibly altered pouring sequences.

Before the discussion on the process of intention deduction, we first describe how the motion can be generated under new environmental states when the human intention has already been derived. We start with the representation of the intention I. Assume that there are N delicate motions and S objects involved in a demon-strated task. Because the intention is closely related to the delicate motions of the maneuver, I is formulated as a set of delicate motions, D_n(t), associated with the corresponding objects Obj_s:

I = {D₁(t), D₂(t), ..., D_N(t); Obj₁, Obj₂, ..., Obj_S} (3.1) where Dn(t) stands for the part of the demonstrated trajectory for delicate motion n and Obj_s the position and orientation of an object s. Note that, because an object may correspond to one, several, or no delicate motion, the number of delicate motions may not be equal to that of the objects. We then introduce the motion index (M I), which serves as an index linking to I. M I is formulated as an ordered set of the time-point pairs, d_j = {n_j, l_j, s_j}, which provides the starting time n_j,

Motion

Figure 2 The process for motion generation.

Figure 3.3: Process for motion generation.

end time lj, and number of the operated object sj for each of delicate motions D:

M I = {d₁, d₂, .., d_N} (3.2) where M I represents I. Fig. 3.3 shows the process for motion generation. Ac-cording to M I, the motion cutting module locates the delicate motions D_j from the demonstrated motion in order. To respond to the new environmental state, the motion adjustment module moves these D_j to match the new locations of the ob-jects and become D_Gj. Finally, the motion connection module uses the move motion M_Gj to smoothly connect every two D_Gj. As its accuracy is not that critical, M_Gj is generated using the cubic polynomial. With both D_Gj and M_Gj, we now have a feasible trajectory Q_G corresponding to the new environmental state:

Q_G = {M_G1, D_G1, M_G2, D_G2, ..., D_GN, M_{GN +1}} (3.3)

Fig. 3.4(a) shows an example for motion cutting based on the pouring task shown in Fig. 3.2, and Fig. 3.4(b) that of motion generation. In Fig. 3.4(a), the demonstrated trajectory during task execution is projected on the X-Y plane, where the yellow and green rectangles indicate the locations of vessels B and C. The yellow and green trajectories are the delicate motions determined by the motion cutting module according to the given M I. In Fig. 3.4(b), the yellow and green rectangles indicate the locations of vessels B and C in the new environmental state. In re-sponding to these new locations of vessels B and C, the delicate motions identified in Fig. 3.4(a) are transformed to be the yellow and green trajectories by the motion adjustment module. Finally, the three move motions, as the red trajectories, are utilized to smoothly connect the two delicate motions.

0.1 0.2 0.3 0.4 0.5

Figure 4 Examples of (a) motion cutting and (b) motion generation based on the pouring task.

End

Figure 3.4: Examples for (a) motion cutting and (b) motion generation based on the pouring task shown in Fig. 3.2.

Motion

Figure 3.5: Process for M I evaluation: (a) standard M I evaluation and (b) M I evaluation with strategy.