1.1. Background
Humanoid robots have become important types of robots that researchers develop and improve them rapidly. In [1], a description of the possible application using a humanoid robot in real-life is provided. In [2], the authors review over last decade's application and influence of humanoid robots in the social, healthcare, and education domains. Recently, in (2019), the humanoid robot applications in a real-world scenario were chosen as the special topic issues in IEEE Robotics and Automation Magazine (RAM)1. Therefore, the development of humanoid robots offers significant potential in alleviating tedious and tough tasks that currently performed by humans.
The important question with developing a humanoid robot is “Why humanoid robot? Why not the other types of robots?”. The answer can be indicated as the functions of the humanoid robots itself. Three main fundamental functions of a humanoid robot are evaluated on [3]: (i) Humanoid robots are able to work in the human environment, (ii) Humanoid robots are capable to use humans tools, (iii) Humanoid robots are designed structurally similar to a human shape. As mentioned, a humanoid robot is designed to be similar to mankind. It should mimic a human from different aspects such as interaction, perception, locomotion, manipulation, and behavior.
Generally, humanoid robots were expected to work alongside humans, or as an alternative to humans in any circumstances. For example, in heavy-duty work such as civil engineering and hazardous environments construction, Moving Large and Heavy Objects (MLHO) is required. Moreover, in rescue applications, during the evacuation process, it is necessary to remove the large size of debris. Though biped humanoid robots have high mobility like humans, walking with moving objects has a possibility robot may fall, due to relatively disturbance in the Centre of Mass (COM) with suffering
1 https://www.ieee-ras.org/publications/ram/special-issues/humanoid-robot-applications-in-real-world-scenarios
2
serious damage. So far, many humanoid robot development projects with a focus on the MLHO was still a challenging problem [4-12]. These challenges can be summarized into how to develop a stable walking gait on a biped robot while the robot is dragging a large size object. Admittedly, the dragging problem is more challenging than carrying because there are more uncertainties of surface friction which duplicates the complexity of the problem.
1.2. Problem statement
Biped walking humanoid robots may not be stable due to different real-time environment conditions even the desired walking pattern has planned to realize stable walking on the flat floor. However, in the MLHO problem, it is assumed that some objects are too heavy to lift or its shape or size is very hard to carry for a humanoid robot with limited joint torque. Therefore, to deal with this problem, we considered the humanoid robot to pull the object. For this reason, we used the pull motion and then specifically called dragging. This is significant although drag and pulls motion have a similar meaning, however, term dragging is more specific than pull.
The important question in this MLHO motion type, “Why we choose dragging the object rather than pushing the object?”. The answer is illustrated in Figure 1-1, MLHO with dragging motion has more benefit than pushing an object, which is the main target in this thesis is based on that. A study about comparison force on the push and pull an object in flat horizontal surface provided by [13, 14].
Based on Figure 1-1, it shows that there is a difference in friction and forces toward the object between those two tasks. The push motion as shown in Figure 1-1(a), shows the vertical component of the pushing force acts on the object in the vertically downward direction. Therefore, it increases the effective weight of the object and it’s mathematically written in Eq (1-1). Whereas, it also affects the friction force between object and ground. The effective weight W of the object on pushing motion as follow:
sin
W = +m g F (1-1)
Where m is a mass of the object, gis the gravity, Fis the pushing force, and is elevation angle of the force given to the object.
On the other hand, the pulling motion shows the reverse way of the vertical force component acts on the object is in a vertically upward direction. Thus, it reduces the effective weight of the object proof on Eq (1-2) and it also decreases friction between the object and the ground. The effective weight Wof the object on pulling motion as follow:
sin
W = −m g F (1-2)
Based on these two equations, dragging an object on the horizontal plane is easier than pushing. Note that, although pushing the object can be beneficial in different conditions for a humanoid robot, but it is not the objective in this research study.
(a) Pushing object. (b) Dragging object.
Figure 1-1 Comparison motion pose on the moving object.
4
1.3. The objective of the study
In this work, we present an adult-sized bipedal humanoid robot that is capable of moving a large and heavy object. The objectives of this project are divided into two parts. First, proposing a robot vision algorithm on 3D object detection and 2D object instance segmentation, that uses a deep-learning algorithm approached. Furthermore, in 3D object detection, the object will be acquired using a real-time LiDAR scanner on the robot's head to get the 3D data. On the other hand, the 2D instance segmentation will be expected running in real-time and used for floor detection from the robot’s webcam.
Second, proposing a deep reinforcement learning algorithm specifically on the Deep Q-Learning algorithm to improve the robot’s behavior on whole-body manipulation to transporting large size and heavy objects. Therefore, in the training process, we used a simulated robot model and environment on Gazebo2. The advantage of using a gazebo simulator that it can simulate very close to the real environment. As a result, the training resulted can be directly applied to the real robot without any parameter adjustment. This thesis discusses a way of MLHO by a bipedal adult-sized humanoid robot, in which the robot drags different objects including a massive object on various flat surfaces, and walks in a backward direction.
The rest of the thesis is organized as follows. In chapter 2 an overview of the literature review on moving objects using bipedal humanoid robots presented. Chapter 3 explains the methodology of the algorithms to solve MLHO problem, in which the architecture of THORMANG-Wolf robot, vision on the proposed deep learning 3D object classification and floor detection, the bipedal humanoid robot walking control, and the proposed deep reinforcement learning method are presented. Chapter 4 provides the experimental result of the 3D object classification and the proposed method of Deep Q-Network (DQN) on the THORMANG-Wolf robot. Finally, chapter 5 concludes the thesis and shows future work.
2 http://gazebosim.org/
1.4. Limitation of the study
There are four major limitations in this research that could be addressed in future research. First, the research focused on robot vision processing that is based on a deep learning approach. Also, it divided into 3D voxel object classification from LiDAR point cloud data and real-time instances segmentation for floor detection. The second limitation concern of robot manipulation control, it only used static grasp motion for grasping the object. Third, on the robot walking control, it used the original ZMP walking controller provided from ROBOTIS on the THORMANG3 robot. Finally, in robot behavior control, it specifically uses the deep reinforcement learning on the DQN algorithm to learn the control policy of the Centre of Body (CoB) parameter.
6