Scenario

2.1 Experimental Setup

The main goal in this research topic is to pick up a moving object from the conveyor to the platform. The associated factors to complete the task will be fully illustrated in the following subsections.

2.1.1 Scene

The scene of our experiment is illustrated in Fig. 2.1. Objects to be picked up are transmitted by a conveyor. The object is moving among whole process. The manipulator is a 7-DoFs robot arm which is designed and assembled in our lab. There is a 3 finger gripper in the tip point to grasp objects on the moving conveyor. The visual sensors are able to sense any difference happening in the environment. In this research, the system is sensor-integrated with two vision sensors including Microsoft Kinect and webcam camera.

Fig. 2.1 An industrial task outlining the pickup of moving objects

doi:10.6342/NTU201703382

2.1.2

For a proper robot assembly, some fundamental functions should be constructed including manipulator control, object recognition and fetching and dynamic factor. The whole process to list will be introduced in Chapter 4. When the manipulator functions, there are some problems robot must handle. In factories, most of the problems are the instant position and orientation of object. In object recognition system, the lighting condition may also affect the color or intensity feature of the object. When this basic factor hasn’t been solved, the robot is unable to hold an object, let alone grasp moving one.

2.2 Procedures

The whole procedures can be separated into two parts: static and dynamic.

In static part, the industrial elements are put on the conveyor and being transmitted to a certain region where the light sensor is mounted. The light sensor in Fig. 2.2 will be trigger and stop the conveyor so that the part will stop and wait for subsequent

Fig. 2.2 The light sensor of the conveyor

operations. At the meantime, the light sensor will send a signal to the computer and start the object recognition algorithm to identify the type as well as the pose of the element.

After the success of the object recognition, the manipulator will go along a pre-taught trajectory to the ready pose and then pick up the object using a pre-taught grasp. The manipulator will then start the operations for the object such as assembling, polishing, painting, etc. After the current task accomplished, the manipulator will send a signal to the conveyor and transmit the next part for the next operation. The cycle then repeats itself.

When the conveyor starts to move, there are several dynamic factors on the conveyor. The vision sensors are used as monitors to sense any variation in the experiment. In this thesis, two ways to describe the relationship between arms and cameras will be illustrated. The first way is eye to hand, shown in Fig. 2.3. Robot eye stands at a relative position to the robot hand. This means that the camera is set near by the robot manipulator and its distance remains solid. When the conveyor start to transmit the objects, the camera will check the instant position and recognize the object at the same time. After receiving new data from vision, the end-effector will run to the target to pick up the object. The other one is eye in hand, shown in Fig. 2.4. Literally, on the tooltip of manipulator, there is an eye on it. This eye helps to track the object immediately. However, the view may include partial body of object. To calculate the area of object in the camera frame, it is important to segment the 640 x 480 view. Only a small region is interested in the procedure. The robot manipulator will track the object three times and grasp it before it falls down to the ground. Therefore, the dynamic factors can be solved by two tracking skill. The vision sensor used in eye to hand is Microsoft Kinect, and the other one in eye in hand is webcam camera.

doi:10.6342/NTU201703382

2.3 Preconditions

In industrial scenarios, the environment is rather controllable compared with household scenarios. As a result, with some assumptions properly made, we can greatly increase the efficiency of the system. Furthermore, we can setup the environment to meet the precondition of some algorithms, in our case the object recognition algorithms,

Fig. 2.3 Eye to hand system (Microsoft Kinect sensor)

Fig. 2.4 Eye in hand system (webcam camera)

so that those powerful algorithms can be adapted to meet our requirement of the system such as high reliability, ow time consumption, and so on.

2.3.1

In industrial scenario, the environment is controllable. We can build fences around the robot which prohibits humans from going inside. The environment can be fixed once the system has been deployed. As a result, if the robot can work in a well -controlled

We assume that objects or components that may occur in the scenario are all rigid bodies. They would not deform or break on account of external forces. Following this assumption, model-based object recognition algorithms can be applied. The states of the object can be described by its type and the 6-DoFs rigid transform in the 3D Cartesian space. The internal states such as the deformation can be neglected under the rigid body assumption. Furthermore, the feasible grasp for picking up the object stably can be described easily. Other factors such as the exerted force of the gripper, the damage of the object and the deformation of the object are not taken into consideration. Only the grasp point is required for describing the proper grasp, which is simply the 6-DoFs pose of the end-effector in the object’s coordinate.

doi:10.6342/NTU201703382