Part II: 2014 Proposal
II. Content of Research Proposal
1. Background, Significance, Innovation, and Specific Aims Home Service Robots
In the domain of intelligent robot research, we have developed an advanced home service robot in our laboratory. The goal of the robot is to help people and those disabled to facilitate their lives. The robot is expected to interact with humans friendly and to complete a conversation naturally. When some people are happy, angry, or depressed, the robot can encourage and placate them. It means the robot should have the ability to analyze the emotion of people. To serve people at home, the robot should establish a map for the house firstly and can locate itself. In this project, we will complete online simultaneous localization and mapping (Online SLAM), 3D vision system, and improve the face recognition rate and successful rate of grasping objects. Finally, we will manufacture a whole new home service robot, so that the new one and old one can cooperate with each other and complete much more complicated tasks.
Humanoid Robots
Although the humanoid robot has been studies for decades, it is still difficult to develop a low-cost humanoid robot that can freely move in any environment and complete various tasks.
Nowadays, there are several famous adult-sized humanoid robots such as Asimo [1.2.1], HRP-4C [1.2.2], Petman [1.2.3], and CHARLI [1.2.4]. These robots are all high tech robot with high intelligence and robust walking ability, and clearly they cost high expenses and manpower.
Hence, the main objective of this project is to develop two different sized humanoid robots, which are capable of participate in multi-sport events. Furthermore, the cost of the robots in this project can be much lower than the cost of the robots mentioned above. Besides, one of the robots is about 135cm tall and 16kg weight for RoboCup [1.2.5] competition and the other is about 90 cm tall and 12kg weight for FIRA [1.2.6] competition. FIRA and RoboCup are two well-known international robot competitions held annually in different country, and they are good platforms to verify the overall performance of the robot system.
2. Methodology and Advantages Home Service Robots
a. Hardware
The structure of our robot includes a mobile chassis, a laser measurement system, a robotic arm, a microphone, and a video camera. With these components and computers, the robot can do some housework such as folding clothes, moving heavy things, and so on. But in order to have more applications, the robot needs more powerful and useful sensors that receive images, sounds, pressure, and depth information. In addition, it is necessary to increase the mobility of the chassis and give our robot an attractive appearance so that it can be used in our daily life.
All the components that can improve the robot’s capabilities are shown as follows:
(1) Chest: To get the farther object without changing the length of arm, the new robot should have more dimensions at chest. We will add motors and bearings to help rotate and reduce the friction.
(2) Elbow: To grasp heavier object, the robot’s elbow should add motors to enhance the force power, however, it also increases the shoulder’s load. So we will change the position of the motors and let them near the shoulder. A timing belt will be adopted to transport the rotation force.
(3) Finger: To firmly grasp objects, we refer to the structure of D-hand, which can seize a wider range size of objects than the past.
b. Software
The two-dimensional laser range-finder mapping and localization technology we developed in 2013 cannot be processed simultaneously. Therefore, it usually takes a lot of time and efforts to explore new environments. Hence, online simultaneous localization and mapping (SLAM) is an important capability for autonomous home service robots to explore unknown environments. There are some ways to implement online SLAM. The online EM-SLAM[1.1.1]
algorithm relies on an online version of the Expectation Maximization (EM) algorithm[1.1.2], which is an iterative technique to perform maximum likelihood estimation in latent models.
Another algorithm is a hierarchical hybrid method integrating ML and EKF with occupancy grid map and feature-based map [1.1.3]. We expect to achieve the online SLAM that provides convenience to users since it takes much less computational time.
Vision system developed in 2013 mainly applies to 2D vision. So the next step, our laboratory will develop 3D vision system. First, a 3D model will be set up. There are a lot of technologies to build up a 3D model, for example, stereo vision [1.1.4] or point cloud model [1.1.5] [1.1.6]. The former uses more than two cameras to match feature of these 2D images and computes depth information to build up 3D model by geometry. The latter one uses the combination of image and depth sensor [1.1.7] to realize 3D reconstructions. Next, we have to realize 3D tracking [1.1.5] [1.1.8] for exactly tracking the target object in the process of moving object or robots (Fig. 1.1.1). Finally, we choose the best point of grasp configuration [1.1.9] to improve successful rate of grasping objects (Fig. 1.1.2).
Besides the SLAM technique, we also focus on expression recognition and speaker recognition. If the robot can recognize who the person is and call his name, it can attract the person to talk and interaction with the home service robot. Talking to the robot also can be a new way to relax stress. In order to implement these functions, we have to combine facial recognition and voice recognition. With facial expression and intonation recognition, robots can analyze the emotion of the people and response to users appropriately. To implement the expression recognition .We utilize LEM (Line Edge Map) [1.1.10]. A compact face feature is generated for face coding and recognition. The system performances are also compared with the eigenface method [1.1.11][1.1.12] and Embedded HMM [1.1.13][1.1.14]. Combining facial expression with voice, we can enhance the emotion recognition rate [1.1.15][1.1.16]
[1.1.17][1.1.18].
Fig. 1.1.1 3D face tracking [1.1.8]
Fig. 1.1.2 The best point of the final grasp configuration [1.1.9]
Humanoid Robots
Mechanism
12
In order to improve the overall performance of the robot, it is necessary to look after both mechanism and decision-making system. For the mechanism, since the weight of the robot strongly affects the mobility of the robot, some parts of the mechanism, whose material are Al-Mg alloy, are replaced by lighter material such as Acrylonitrile-Butadiene-Styrene (ABS), Polyoxy-Methylene Resin (POM).The specific gravity of ABS is only 1.04, and it is also easy to process and spray painting, so this material is suitable for no-loading parts such as shell and skull shape. The specific gravity of POM is 1.41, and the advantages of POM are high abrasion resistance and low coefficient of friction. These advantages make POM a good choice of gear material. The specific gravity of Al-Mg alloy is 2.70, about twice than POM.
Although plastic materials are helpful to reduce the mass of robot, Al-Mg alloy cannot be replaced in all of the parts due to its strong rigidity compare with plastic materials.
Gear
When studying the adult-sized humanoid robot, it is known that the motors are damaged frequently because the reaction force is too large for motor to stand. In general case, the gear in the gear box of the motor is broken when the motor is severely damaged. Although the gear ratio in motor is almost 1:225, it is still unable to withstand the landing forces. In order to increase torque, the motor speed and running degree of the robot might be reduced. Thus, we propose to create a pair of two additional spur gears with the ratio 1:3. Under ideal condition, the implementation of this extra decelerator would enhance at least 3 times the motor torque and keep motor away from damage. These gears are made of POM which has lower friction values so that it is wear-resisting, and this plastic material also has high level of rigidity.
3-D.O.F. waist
Generally, there are two or less D.O.F. on the robot waist, but still some researchers develop 3-D.O.F. waist structure robot. The 3-D.O.F. waist for upper body of robots makes the motion more flexible. In order to make our robot walking more smoothly and suitability, we think the 3-D.O.F waist is an effective way to redevelop in the next year.
Cam-Shift algorithm
The decision-making system involves image processing and decision-making strategy. For efficient decision-making, the object should be recognized and tracked correctly. Firstly, the image color space is converted from RGB to HSV [1.2.7] and a look up table is established for speeding up the conversion. Then, a simplified concept of Continuously Adaptive Mean-Shift (Cam-Shift algorithm) [1.2.8]-1.2.11] is utilized to make the object tracking and recognition more robust and faster. Cam-Shift is a recursive searching algorithm, which is able to identify and segment not only different types of objects but also the same type of multi-objects, and it is also good for color object tracking.
With the concept of Cam-Shift, an adaptive region of interest is changed with time. The region is constrained with the boundary of the object found in previous frame, and possesses two benefits, one is the reduction of searching time and the other is the lower probability of erroneous judgment. Since the region of interest is updated by the region of object found in previous frame, the searching time is reduced to only a fraction of the original. Because the object moves continuously, the position of the object in the current frame is assumed to be not far away from the last frame. We can say that the object positions are almost few changes if the frame rate of the camera is high enough. Therefore, we search the region of the object found in previous frame is a good way to reduce the probability of the mismatch and avoid tracking the wrong object.
ABC-CPG gait learning method
Central pattern generator (CPG) is adopted to produce a legged locomotion. The model of neural network suggested by Matsuoka in 1985 [1.2.12-1.2.13] is used most widely. After Matsuoka's model adopted in simulating walking pattern of biped robot by Tagga and other
researchers [1.2.14], CPG becomes a powerful tool to generate gait pattern for humanoid robots. Hence, the CPG network is utilized to generate the periodic trajectory of the position of the robot ankle. The trajectory generated by CPG looks like an ellipse as shown in the Fig.
1.2.1, and it makes the robot walk with a specific frequency. Then, the trajectory of the leg joints can be obtained by inverse kinematics. Moreover, the intrinsic parameters are trained by Artificial Bee Colony algorithm (ABC) [1.2.15]. With the proposed ABC-CPG method, the robot can learn to walk more stable and faster.
Fig. 1.2.1 Propulsive motion in the sagittal plane
3. Anticipated Achievements Home Service Robots
The home service robot performs many useful services to humans, for example, doing housework, watching over the kids, and taking care of the elders. However, for those who seldom interact with robot, they may think it is no need to rely on the robot because of the lack of interaction experiences. Therefore, we expect home service robots to approach the person who requires help actively to enhance the opportunity of interaction. The robot will anticipate the person’s future movement trajectories and plan an appropriate path to close the person. After approaching, the robot meets the target person face to face and starts a conversation with him. At the same time, the robot will detect and memorize the person’s face at first sight. Whenever the robot interacts with the person, it learns the person’s behaviors and demands. Second, the robot will patrol the house autonomously when the family is no body at home. If the robot detects some sound in the environment, it will approach it to confirm whether the person is one of the family member or a stranger. When the robot finds a stranger, it will take a picture and inform family members or the police station immediately.
Home service robot can help the handicapped person, children, pregnant woman, and elders.
For example, the robot can help a child who wants to take something from a high place, or help handicapped person open and close a door, or open a refrigerator to take drinks or foods.
Home service robot also can help pregnant woman and the elders carry heavy objects.
Humanoid Robots
Study of adult-sized humanoid robots has become a trend in the world for decades.
Recently, some universities in Taiwan starts to do research on this area. For EE of NCKU, we developed and implemented our first adult-sized humanoid robot, David, in year 2012. We believe both the developments of simulator and real robot are necessary. Hence, the simulator is investigated to analyze the dynamic of the robot and verify the feasibility and effectiveness of the decision making system. Some papers associated with the gait learning method and simulator development will be submitted and published by the IEEE Transactions or top 5%
ranking journals.
Two different sized humanoid robots will be developed and implemented in 2014. One is about 135cm tall and 16kg weight for RoboCup competition and the other is about 90 cm tall and 12kg weight for FIRA competition. They are expected to possess the capability to perform multiple sports such as weight-lifting, basketball, sprint, obstacle run, wall-climbing, penalty kick, lift and carry, marathon and soccer. Since FIRA and RoboCup are good platforms to verify the overall performance of the robot system, the anticipated achievements
14
are set to win the 1st place of one-on-one soccer game in Adult-sized Humanoid League of the RoboCup competition and 1st place of Overall Round in Adult-sized League of the FIRA competition.
Reference
Home Service Robot
[1.1.1] S. Le Corff, G. Fort, and E. Moulines, “Online Expectation Maximization algorithm to solve the SLAM problem,” in Statistical Signal Processing Workshop (SSP), 2011 IEEE, Jun., pp. 225 –228.
[1.1.2] T. K. Moon, “The expectation-maximization algorithm,” IEEE Signal Processing Mag., vol. 13, pp.
47–60, Nov. 1996.
[1.1.3] G. Huang, A. Rad, and Y. Wong, “Online slam in dynamic environments,” in Proc. 12th Int’l Conf. on Advanced Robotics, pp. 262–267, July 2005.
[1.1.4] B. Tippetts, D. Lee, J. Archibald, and K. Lillywhite, “Dense disparity real-time stereo vision algorithm for resource limited systems,” IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 10, pp.
1547–1555, Oct. 2011.
[1.1.5] O. Serdar Gedik and A. Aydın Alatan, “3-D Rigid Body Tracking Using Vision and Depth Sensors”, IEEE Transactions on Cyberbetics, vol. 43, no. 5, OCT. 2013.
[1.1.6] M. Ye, X. Wang, R. Yang, F. Ren, and M. Pollefeys, “Accurate 3-D pose estimation from a single depth image,” in Proc. ICCV, 2011, pp. 731–738.
[1.1.7] S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. Newcombe, P. Kohli, J. Shotton, S. Hodges, D.
Freeman, A. Davison, and A. Fitzgibbon,“KinectFusion: Real-time 3-D reconstruction and interaction using a moving depth camera,” in Proc. ACM Symp. User Interface Software Technol., Oct. 2011, pp.
559–568.
[1.1.8] L. Vacchetti, V. Lepetit, and P. Fua, “Stable real-time 3-D tracking using online and offline information,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 10, pp. 1385–1391, Oct. 2004.
[1.1.9] Vincenzo Lippiello, Fabio Ruggiero, “Visual Grasp Planning for Unknown Objects Using a Multifingered Robotic Hand”, IEEE/ASME Transactions on Mecharonics, vol. 18, no. 3, June 2013.
[1.1.10] Y. Gao and M. K. H. Leung, "Face Recognition Using Line Edge Map", IEEE Transactions on Pattern Analysisand Machine Intelligence, Vol. 24, No.6, pp.764-779,2002.
[1.1.11] E. Mower, M.J. Matarić and S. Narayanan, “A Framework for Automatic Human Emotion Classification Using Emotion Profiles,” IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 5, pp.
1057-1070, July, 2011.
[1.1.12] K. Anderson and P.W. McOwan, “A real-time automated system for the recognition of human facial expressions,” IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 36, no. 1, pp. 96-105, Feb. 2006.
[1.1.13] R. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” In Proc. IEEE, vol. 77, no. 2, pp. 257–286, 1989.
[1.1.14] efian and M. Hayes, ” An embedded hmm-based approach for face detection and recognition,” In Proc.
IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 6, pp.
3553–3556,1999.
[1.1.15] Shahin, “Analysis and investigation of emotion identification in biased emotional talking environments,” IET Signal Processing., vol. 5, no. 5, pp. 461-470, Aug. 2011.
[1.1.16] W. J. Yoon and K. S. Park, “Building robust emotion recognition system on heterogeneous speech databases,” IEEE Trans. Consumer Electron., vol. 57, no. 2, pp. 747-750, May 2011.
[1.1.17] E. Mower, M. J. Mataric and S. Narayanan, “Human Perception of Audio-Visual Synthetic Character Emotion Expression in the Presence of Ambiguous and Conflicting Information,” IEEE Trans.
Multimedia, vol.11, no. 5, pp. 843-855, Aug. 2009.
[1.1.18] Y. Wang and L. Guan, “Recognizing Human Emotional State From Audiovisual Signals*,” IEEE Trans. Multimedia, vol.10, no. 5, pp. 936-946, Aug. 2008.
Humanoid Robot
[1.2.1] Asimo, Available: http://world.honda.com/ASIMO/
[1.2.2] HRP-4C, Available: http://techcrunch.com/tag/hrp-4c/
[1.2.3] Petman, Available:
http://article.wn.com/view/2013/04/08/Petman_robot_rocks_gas_mask_chemical_suit/#/video [1.2.4] CHARLI, Available: http://www.unirel.vt.edu/CHARLI.html
[1.2.5] RoboCup, Available: http://www.robocup.org/
[1.2.6] FIRA, Available: http://www.fira.net/
[1.2.7] A.R. Smith, “Color Gamut Transform Pairs,” In: SIGGRAPH 1978, pp. 12-19, 1978.
[1.2.8] G. R. Bradski, “Computer Video Face Tracking for use in a Perceptual User Interface,” in Proc. IEEE Workshop Applications of Computer Vision, pp. 214-219, 1998.
[1.2.9] P. Hidayatullah, H. Konik, “Camshift Improvement On Multi-Hue Object And Multi-Object Tracking”, in Visual Information Processing (EUVIP), 2011 3rd European Workshop on Digital Object, pp.
143-148, 2011.
[1.2.10] J. Li, F. Li, and M. Zhang, “A real-time detecting and tracking method for moving objects based on color video,” Proc. 6th Int. Conf. Comput. Graphics, Imag. Visualization, pp. 317-322, 2009.
[1.2.11] Yi-Bo Li, Xiao-ling Shen, Shan-shan Bei, “Real-time Tracking Method for Moving Target Based on an Improved Camshift Algorithm”, International Conference on Mechatronic Science, Electric Engineering and Computer, 2011.
[1.2.12] K. Matsuoka, “Sustained Oscillations Generated by Mutually Inhibiting Neurons with Adaptation,”
Biological Cybernetics, vol. 52, pp. 367-376, 1985.
[1.2.13] K. Matsuoka, “Mechanisms of Frequency and Pattern Control in the Neural Rhythm Generators,”
Biological Cybernetics, vol. 56, pp. 345-353, 1987.
[1.2.14] G. Taga, Y. Yamaguehi, and H. Shimizu, “Self-organized Control of Bipedal Locomotion by Neural Oscillators in Unpredictable Environment,” Biological Cybernetics, vol. 65, pp. 147-159, Apr. 1991.
[1.2.15] D. Karaboga and B. Basturk, “A Powerful and Efficient Algorithm for Numerical Function Optimization: Artificial Bee Colony (ABC) Algorithm,” Journal of Global Optimization, vol. 39, no.
3, pp. 459–471, Apr. 2007.
III. International Collaboration
International Collaboration 1. Professor Jong-Hwan Kim
Department of Electrical Engineering and Computer Science, KAIST, Korea IEEE Fellow
FIRA President
Email: [email protected] 2. Professor Jacky Baltes
Computer Science at the University of Manitoba, Winnipeg, Canada.
HuroCup Chair
Organizing Committee Member of Humanoid Soccer League RoboCup Email: [email protected]
16
IV. Anticipated Key Performance Indicators
Please list at least five items for each term.
1. Anticipated Qualitative Performance One new version Home Service Robot, May
Cooperation demonstration between two home service robots
One new Humanoid Robot with 135cm tall and 16kg weight for RoboCup competition One new Humanoid Robot with 90 cm tall and 12kg weight for FIRA competition Hold or co-hold an international conference on robotics related topics
2. Anticipated Quantitative Performance
No. KPI Estimated Performances
1 SCI papers 15
2 Top 5% SCI & SSCI papers 5
3 Robot Competition Awards 3
4 Industry-University Cooperative Research Project 2
5 Conference Paper Awards 1
V. Received Research Funds for Related Research in the Past Three Years
Title of Project Source Period Position Amount
Design and Implementation of Cooperative Service Robots with Imitation and Learning Capabilities
National
Study of Hand-Eye-Foot Coordination Control and Its Application to Adult-Size Decathlon Humanoid Robots: SubProject 3 (100-2221-E-006-087-MY3)
NSC 2011/8/1~
2014/7/31
PI 3,360,000
Design and Implementation of Adult-Size Decathlon Humanoid Robots: Integrated Project
(100-2221-E-006-055-MY3)
NSC 2011/8/1~2 014/7/31
PI 2,817,000
Educational improvement plans by national educational society (Chinese Automatic Control Society) (100-2217-E-546-001-)
NSC 2011/1/1~2 011/12/31
PI 843,000
Scenario Design and Power/Control Module Development for Urban Intelligent Electric Vehicles
Scenario Design and Power/Control Module Development for Urban Intelligent Electric Vehicles