Chapter 5. Experimental Result and Discussion
5.1 Generated Walking Trajectory
This subsection described the result of walking pattern generation, which been illustrated in Section 3.2. In this experiment, we run the walking pattern generator with hand-tuned parameters that describe in Table 5-1.
Table 5-1. Parameter values for generating trajectories.
Parameter
The example of trajectory is done when the robot is given walking forward command which illustrated in Figure 5-1. The CoM frame moved forward in the x-axis direction, followed by the left and right sole frame while the robot got the motion command cmdx
= 0.05, cmdy = 0.00, and cmd = 0.15 rad, as shown in Figure 5-1 (a). The curve in Figure 5-1 (a) indicated that the sole position is moving up to the z-axis and touch down to the ground while in the single support phase.
Figure 5-1 (b) illustrated the resulted trajectory from walking sideways motion while the robot was given motion command by cmdy = 0.015, wherecmdx, and cmd are
33
equals zeros. As shown in Figure 5-1 (b), the CoM frame moved to the left side, followed by the y-axis direction. The movement of the left and right soles also followed the direction of CoM, while the position in the x-axis maintained constant in 0 m.
The diagonal path trajectory yielded when cmdx and cmdy values are given a non-zero, as illustrated in Figure 5-1 (c). In this case, the orientation CoM frame was pointing forward to the x-axis, but the position of the CoM frame was moving in both the x-axis and y-axis.
Figure 5-1 (d) shown a trajectory where the cmdx and cmd components are set to a non-zero value. The CoM trajectory generated a curve path, as shown in Figure 5-1 (d).
In contrast to Figure 5-1 (c), not only the position of the CoM frame was changing, but also the orientation of the CoM frame. In the real environment, this trajectory will result in a turning motion for the robot.
(a) (b)
(c) (d)
Figure 5-1. Generated CoM and sole trajectories (a) walking forward, (b) walking sideways, (c) walking diagonal, and (d) walking with turning.
34 5.2 Training Performance
Figure 5-2 shows the statistics of reward and average success rate during the training process. The training reward statistics are illustrated in Figure 5-2 (a). The red line indicated the median of reward from all individuals, followed by a red band that indicated the 25th and 75th percentile of reward. The blue line in Figure 5-2 (a) visualized the Root Mean Square (RMS) standard deviation from all parameters during training.
On the other hand, Figure 5-2 (b) described success rate performance during training.
As seen in Figure 5-2 (a), the median, on the 25th, and 75th percentile of the reward increased significantly during training. In the first generation, the median of reward was -8.946, with 25th percentile -10 and 75th percentile -6.894. Moreover, the resulted success rate at the beginning of training, about 28%. In the final generation, the median of reward increased to -3.160, as the 25th percentile rises to -3.212, and the 75th percentile goes up to -3.127, which the success rate performance steady at 98 %.
Based on the training record, the reward value remained stable after reaching the 40th generation, while the mean success rate stayed constant above 90 %. The RMS standard deviation decreased significantly and close to zeros in the last generation that indicated the parameters converge. The best parameters yielded from training in the simulation are shown in Table 5-2.
(a) (b)
Figure 5-2. Reward and success rate performance during training.
35
Table 5-2. Result of optimal walking parameters.
Parameter Name
Value
zc 32.220 cm
tstep 0.543 s ratio
DSP 62.382 %
zf 3.628 cm
xc 3.158 cm
ys 0.259 cm
c 12.651 degrees
5.3 Evaluation Performance
The best parameter resulted from each generation is evaluated in simulation to verify the energy reduction based on torque value. We verify from the first until the last generation to study reduction each training iteration.
Figure 5-3 shows the evaluation in the simulation model. The leg torque is shown in Figure 5-3 (a), and the and roll pitch value are illustrated in Figure 5-3 (b). As seen in Figure 5-3 (a), the leg torque decreased significantly from the first to the last generation that indicated the optimization process successfully minimizes the energy.
(a) (b)
Figure 5-3. Evaluation of simulation model (a) torque (b) and roll pitch.
On the other hand, the value decreased from the first generation, and the roll pitch value maintained stable at 0.04 - 0.06. This condition made the robot walk more stable in the last generation. Table 5-3 shown a definite improvement in energy and stability.
36
The reduction of energy can be seen in torque value that reduced about 29.813 %. The improvement of stability can be seen in a success rate that increased about 20 % from the initial and final generation.
Table 5-3. Comparison performance between initial and final generation on simulation.
5.4 Comparison Performance Before and After Optimization
We evaluate the optimization result by comparing the performance of optimized and non-optimized gait in the robot. We used hand-tuned parameters from trial and error experiments for non-optimized gait and got the best parameter in Table 5-2 for optimized gait. In this experiment, we used a statistical approach with four different types of omnidirectional walking, such as forward, backward, sideways, and turning.
Each walking type is applied to the robot with 50 trials on the thin carpet with the starting and finish line. The robot is controlled manually by joystick to start walking from the starting line to the finish line. The distance of the starting and finish line for walking forward and backward was 2 meters, and the distance for walking sideways was 1 meter.
In the turning motion, we run the robot to walk on a circular path with a diameter of 0.74 m. During the experiment, the robot is given a constant motion command. The command for walking forward and backward cmd =x
0.03, 0.03−
. For the walking sideways, we run with both walking to side left and walking in the right direction by giving the command cmd =y
0.01, 0.01−
. For walking on a circular path, we gave the motion command cmd =x 0.03 and cmd =
5, 5−
degrees to walk on clockwise (CW) and counter-clockwise (CCW) direction. We recorded the voltage, current, and IMU sensor data during walking to compare the gait performance as presented in Table 5-4.37
Table 5-4. Comparison before and after optimization.
Walking Type
Before Optimization After Optimization Saving energy
Table 5-4 delivered the performance comparison before and after optimization. The success rate represented the percentage of the successful trial, where the robot reached the finish line without falling. The energy represented by the total consumed electric power of the leg actuator per second.
As described in Table 5-4, before optimization, the average success rate was about 61.5 % that indicated unstable gait and made the robot high chance to fall. The non-optimized gait yielded a high success rate in walking forward only that indicated the parameters were not generalized for other types of walking. From the speed and energy perspective, before optimization, the average energy was about 12.985 W/s, and the average speed about 0.075 m/s. After optimization, the gait performance improved significantly. The success rate maintained a constant of 100 % for a different type of walking that indicated the gait parameters yielded a stable gait that made the robot never falling.
On the other hand, the average consumed energy of 10.4 W/s, which was lower compared to non-optimized gait. However, the average speed reached 0.037 m/s, which was slower compared to non-optimized gait. By comparing the consumed energy before and after optimization, the optimized gait was able to save energy about 19.905 %. As the conclusion of the result visualized in Table 5-4, after optimization, the walking gait was slower, more stable, and less consumed energy compared to non-optimized gait.
38 5.5 Straight Walk with Variable Step Length
We extend the experiment in Section 5.4 by varying step lengths when the robot walked forward and backward to study the effect of changing step length to gait performance. We used step length from range 1 cm to 5 cm with an interval of 1 cm.
The result of the walking forward experiment shown in Table 5-5, and walking backward experiment summarized in Table 5-6.
Table 5-5. Comparison walking forward with variable step length.
Step Length
Based on data in Table 5-5, before optimization, the changing of step length affected the reduction in success rate. The higher step length applied to the robot produced a high chance for the robot to fall. However, after we optimized the gait, the changing of step length did not affect the stability. The robot can walk with minimum to maximum step length with a constant success rate. With the optimized gait, we can reach a maximum speed of 0.072 m/s with the highest step length of 7 cm. On the other hand, the consumed energy maintained stable at around 10.021 – 11.004 W/s, even though the step length was varying.
Table 5-6. Comparison walking backward with variable step length.
Step Length
39
The backward walking result, illustrated in Table 5-6, has a similar result with walking forward. While the variable step length applied to non-optimized gait, it affected the reduction in success rate. By using an optimized gait, the robot able to walk backward from step length 1 cm – 7 cm without falling. However, compared to forward walking, the consumed energy was quite higher, about 10.426 W/s – 11.301 W/s. In contrast, the maximum speed was equal to forward walking about 0.072 m/s.
Table 5-5 and Table 5-6 conclude that the yielded walking parameter from optimization successfully applied to walk with variable step length, which means that the robot can walk stably with low energy at a different speed.
40
Chapter 6. Conclusion and Future Work
This thesis presented a stable, energy-efficient, and omnidirectional gait generation on the humanoid robot. The ZMP preview controller with Bezier function was used to generate a walking gait. Moreover, the CMA-ES algorithm was proposed for optimizing gait parameters in the simulation model. The yielded gait engine was verified in the real robot to measure the stability and consumed energy performance.
Based on an experimental result, the proposed gait generation achieved a stable and energy-efficient gait. The reduction in energy during training about 29.813 % in simulation. On the other hand, stability increases by 20 % in simulation. The optimized gait successfully reduced energy consumption by 19.905 % compared to non-optimized gait. Moreover, the optimized gait yielded a stable performance while it applied to variable-speed and omnidirectional walk.
Even though the gait engine is stable, but it can not guarantee to reject external disturbance cause the gait generation is open loop. In future work, a model-free reinforcement learning will be studied to improve the dynamic balance in the robot.
41
References
[1] S. Kajita et al., "Biped walking pattern generation by using preview control of zero-moment point," in 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422), 2003, vol. 2, pp. 1620-1626 vol.2.
[2] M. Naveau, M. Kudruss, O. Stasse, C. Kirches, K. Mombaur, and P. Souères, "A Reactive Walking Pattern Generator Based on Nonlinear Model Predictive Control," IEEE Robotics and Automation Letters, vol. 2, no. 1, pp. 10-17, 2017.
[3] M. Kasaei, N. Lau, and A. Pereira, "A Fast and Stable Omnidirectional Walking Engine for the Nao Humanoid Robot," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) vol. 11531 LNAI, ed, 2019, pp. 99-111.
[4] K. Harada, S. Kajita, K. Kaneko, and H. Hirukawa, "AN ANALYTICAL METHOD FOR REAL-TIME GAIT PLANNING FOR HUMANOID ROBOTS," International Journal of Humanoid Robotics, vol. 03, no. 01, pp. 1-19, 2006/03/01 2006.
[5] I. W. Park, J. Y. Kim, J. Lee, and J. H. Oh, "Online free walking trajectory generation for biped humanoid robot KHR-3(HUBO)," in Proceedings - IEEE International Conference on Robotics and Automation, 2006, vol. 2006, pp.
1231-1236.
[6] K. Matsuoka, "Mechanisms of frequency and pattern control in the neural rhythm generators," Biological Cybernetics, Article vol. 56, no. 5-6, pp. 345-353, 1987.
[7] K. Matsuoka, "Sustained oscillations generated by mutually inhibiting neurons with adaptation," Biological Cybernetics, Article vol. 52, no. 6, pp. 367-376, 1985.
[8] L. Righetti, J. Buchli, and A. J. Ijspeert, "Dynamic Hebbian learning in adaptive frequency oscillators," Physica D: Nonlinear Phenomena, Article vol. 216, no.
2, pp. 269-281, 2006.
[9] H. Wang, C. Liu, and Q. Chen, "Omnidirectional walking based on preview control for biped robots," in 2016 IEEE International Conference on Robotics and Biomimetics, ROBIO 2016, 2016, pp. 856-861.
[10] S. Wang, M. Hu, H. Shi, S. Zhang, X. Li, and W. Li, "Humanoid robot's omnidirectional walking," in 2015 IEEE International Conference on Information and Automation, ICIA 2015 - In conjunction with 2015 IEEE International Conference on Automation and Logistics, 2015, pp. 381-385.
[11] N. Snafii, A. Abdolmaleki, N. Lau, and L. P. Reis, "Development of an Omnidirectional Walk Engine for Soccer Humanoid Robots," International Journal of Advanced Robotic Systems, Article vol. 12, no. 12, 2015, Art. no. 193.
[12] P. Shen, Z. Liang, and X. Li, "Omnidirectional walk of biped robots in RoboCup3D simulation environment," in 26th Chinese Control and Decision Conference, CCDC 2014, 2014, pp. 2119-2123.
[13] A. Abdolmaleki, N. Shafii, L. P. Reis, N. Lau, J. Peters, and G. Neumann,
"Omnidirectional walking with a compliant inverted pendulum model," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) vol. 8864, ed, 2014, pp. 481-493.
42
[14] N. Shafii, A. Abdolmaleki, R. Ferreira, N. Lau, and L. P. Reis, "Omnidirectional walking and active balance for soccer humanoid robot," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) vol. 8154 LNAI, ed, 2013, pp. 283-294.
[15] J. J. Alcaraz-Jiménez, D. Herrero-Pérez, and H. Martínez-Barberá, "Motion planning for omnidirectional dynamic gait in humanoid soccer robots," Journal of Physical Agents, Article vol. 5, no. 1, pp. 25-34, 2011.
[16] J. Strom, G. Slavov, and E. Chown, "Omnidirectional walking using ZMP and preview control for the NAO humanoid robot," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) vol. 5949 LNAI, ed, 2010, pp. 378-389.
[17] D. Gouaillier, C. Collette, and C. Kilner, "Omni-directional closed-loop walk for NAO," in 2010 10th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2010, 2010, pp. 448-454.
[18] J. Cristiano, D. Puig, and M. A. García, "Generation and control of locomotion patterns for biped robots by using central pattern generators," Journal of Physical Agents, Article vol. 8, no. 1, pp. 40-47, 2017.
[19] K. Moradi, M. Fathian, and S. Shiry Ghidary, "Omnidirectional walking using central pattern generator," International Journal of Machine Learning and Cybernetics, Article vol. 7, no. 6, pp. 1023-1033, 2016.
[20] D. Rodriguez, A. Brandenburger, and S. Behnke, "Combining Simulations and Real-Robot Experiments for Bayesian Optimization of Bipedal Gait Stabilization," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) vol.
11374 LNAI, ed, 2019, pp. 70-82.
[21] V. H. Dau, C. M. Chew, and A. N. Poo, "Achieving energy-efficient bipedal walking trajectory through Ga-based optimization of key parameters,"
International Journal of Humanoid Robotics, Article vol. 6, no. 4, pp. 609-629, 2009.
[22] P. Kormushev, B. Ugurlu, S. Calinon, N. G. Tsagarakis, and D. G. Caldwell,
"Bipedal walking energy minimization by reinforcement learning with evolving policy parameterization," in IEEE International Conference on Intelligent Robots and Systems, 2011, pp. 318-324.
[23] J. Kober and J. Peter, "Policy search for motor primitives in robotics," in Springer Tracts in Advanced Robotics vol. 97, ed, 2014, pp. 83-117.
[24] J. Kober and J. Peters, "Policy search for motor primitives in robotics," Machine Learning, Article vol. 84, no. 1-2, pp. 171-203, 2011.
[25] J. Kober and J. Peters, "Policy search for motor primitives in robotics," in Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference, 2009, pp. 849-856.
[26] J. Li and W. Chen, "Energy-efficient gait generation for biped robot based on the passive inverted pendulum model," Robotica, Article vol. 29, no. 4, pp. 595-605, 2011.
[27] Z. Sun and N. Roos, "An energy efficient dynamic gait for a Nao robot," in 2014 IEEE International Conference on Autonomous Robot Systems and Competitions, ICARSC 2014, 2014, pp. 267-272.
43
[28] N. Kohl and P. Stone, "Policy gradient reinforcement learning for fast quadrupedal locomotion," in Proceedings - IEEE International Conference on Robotics and Automation, 2004, vol. 2004, pp. 2619-2624.
[29] A. A. Saputra, T. Takeda, and N. Kubota, "Efficiency energy on humanoid robot walking using evolutionary algorithm," in 2015 IEEE Congress on Evolutionary Computation, CEC 2015 - Proceedings, 2015, pp. 573-578.
[30] N. Shafii, N. Lau, and L. P. Reis, "Generalized learning to create an energy efficient ZMP-based walking," in Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science), 2015, vol. 8992, pp. 583-595.
[31] N. Hansen, S. D. Müller, and P. Koumoutsakos, "Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES)," Evolutionary Computation, vol. 11, no. 1, pp. 1-18, 2003/03/01 2003.
[32] S. Kajita, H. Hirukawa, K. Harada, and K. Yokoi, "Introduction to Humanoid Robotics," in Springer Tracts in Advanced Robotics, ed: Springer Berlin Heidelberg, 2014.
[33] M. VukobratoviĆ and B. Borovac, "ZERO-MOMENT POINT — THIRTY FIVE YEARS OF ITS LIFE," International Journal of Humanoid Robotics, vol.
01, no. 01, pp. 157-173, 2004/03/01 2004.
[34] N. Hansen, "The CMA evolution strategy: A comparing review," in Studies in Fuzziness and Soft Computing vol. 192, ed, 2006, pp. 75-102.
[35] N. Hansen and A. Ostermeier, "Completely derandomized self-adaptation in evolution strategies," Evolutionary computation, Review vol. 9, no. 2, pp. 159-195, 2001.
44
Autobiography
Eko Rudiawan Jamzuri finished diploma at Politeknik Negeri Batam in 2011 and obtained a Bachelor of Applied Science from Bandung Institute of Technology in 2013.
Currently, he is a master student at the Department of Electrical Engineering, National Taiwan Normal University, and member of Educational Robotics Centre (ERC) National Taiwan Normal University.
45
Academic Achievement
1. International Intelligent RoboSports Competition 2020 (Taiwan) - 1st Place HuroCup Kid-size Humanoid Sprint & Marathon.
2. IEEE/RSJ IROS 2019 (Macau) – 3rd Place Humanoid Robot Application Challenge.
3. Iran FIRA RoboWorldCup Open 2019 (Iran) - 1st Place all-round HuroCup Kid-size Humanoid.