Experimental Results and Comparisons

During the training for first stage neural network, the robotic end-effector holds a red LED as target. The stereo camera maintains target in its binocular fixation by pan-tilt control. The system is updated every 40ms. Arm motion is controlled within a defined range. To construct random and equally distributed movements for two joint angles, joint 0 is moved by constant speed and joint 1 is moved by random speed. The direction of either joints reverses when reaching its boundary. The trajectory of two joints is shown in Figure 5.

Figure 5 Trajectory of arm joint 0 and 1

We could observe that the 2D arrays of neurons are statistically equally distribution and fired. As discussed in previous section, the sensory signals (pan, tilt position and vergence) are not fully explored. By implementing histogram equalization, neurons of each dimension receive roughly same amount of trainings. The Hebbian learning algorithm described in previous section updates the connection between motor and

-40 -30 -20 -10 0 10 20 30

45 50 55 60 65 70 75 80 85 90 95

Arm Joint 

0 Position (degree) Arm Joint 1 Position (degree)

11

sensory signals. The weight is initially randomly assigned and normalized. After training for around 7,000 data sets (7,000 weight updates), it constructs stable weight between 2D neurons of arm joint angles and 3D neurons of sensory signals.

During the training for second stage neural network, the robotic end-effector holds a red LED as hand position. At initial state, the robotic arm is randomly moved in defined range, and the pan-tilt unit performs tracking to locate the red LED in its binocular fixation. The joint angles and visual position of red LED is recorded and this state is denoted as origin. Then the robotic arm is randomly moved in a local region and changes in both joints angle from origin is bounded in [−6, 6] degree range. The changes in the observed hand positions in stereo camera are also recorded to build up the association between two sets of signals. After training in one origin for 100 data sets, the robotic arm moves to another position and repeats above local training. After training for 45 different origins (4,500 weight updates), the trajectory of changes in joint angles is shown in Figure 6. Similarly, the training data span the 2D arrays of neurons.

Figure 6 Trajectory of arm joint 0 and 1

B. Testing Results

During the testing period, the external inputs will not be available, and we could simply set training weight C to be zero.

1. Evaluation of First Stage Neural Network

In order to evaluate the performance of first stage neural network, we keep the target at the hand and disable commanded signal to control joint angles. This allows us to move arm joints to difference positions while analyzing the generated motor signals from neural network. Figure 7 shows the comparision beween actual arm positions and

-8 -6 -4 -2 0 2 4 6 8

-8 -6 -4 -2 0 2 4 6 8

Arm Joint d₀ Position (degree) Arm Joint d1 Position (degree)

12

generated commands by fisrt stage neural network after training. Errors between corresponding signals are calculated by mean square error.

Figure 7 Comparision between actual arm commands (green marked) and generated neuron responses (red marked) in first stage neural network

To futher illustrate the method of histogram equalization in neuron allocation, we also implement method that equally locate neuron in each dimension. Its performance is shown in Figure 8.

Figure 8 Comparision between actual arm commands (green marked) and generated neuron responses (red marked) in first stage neural network without Histogram Equalization

0 1000 2000 3000 4000 5000 6000

-40

0 1000 2000 3000 4000 5000 6000

50

0 1000 2000 3000 4000 5000 6000

-40

0 1000 2000 3000 4000 5000 6000

50

13

One wrong direction occurs when the generated signal from network is in different direction with that of the actual signal and the actual signal is significantly away from origin, e.g. 0.5 degree in our experiment. The wrong direction ratio is the ratio of number of wrong direction over total number of training set.

With the implementation of histogram equalization in neuron allocation, the performances of both joints are improved by 30.3% and 20.2% respectively. If we assume that the distributions of errors in two joints follow Gaussian distribution, then variances σ² for both joints are bounded in 3 degree. When we perform second stage neural network training with range of [-6, 6] degree from origin, it is expected to cover and improve errors in the range of ±3.5σ or equivalent 99.95% of the errors in first stage neural network.

2. Evaluation of Second Stage Neural Network

To analyzing the performance of second stage neuron network, we apply similar strategy as that in training, but disconnecting the generated arm commands by this network with the actual commands. After moving arm randomly to one location, the pan-tilt unit stops tracking and the system sets current stage as origin. Then the arm is controlled randomly move in that local region, and generated arm commands from this network are compared with actual positions (Figure 9):

Figure 9 Comparision between actual arm commands (green marked) and generated neuron responses (red marked) in second stage neural network

As second stage neural network is designed to generate on-line arm commands based on active sensory signals, we also compare the wrong direction ratio defined as follows:

0 20 40 60 80 100 120 140 160 180 200

14

The MSEs in this neuron network are around 1 degree for both joints, however, though on-line control, the error is expected to converge.

In this neural network, we also compare the performance of the network that uses equally spaced neuron allocation (Figure 10):

Figure 10 Comparision between actual arm commands (green marked) and generated neuron responses (red marked) in second stage neural network without histogram equalization

With the implementation of histogram equalization in neuron allocation, the performances of both joints are improved by 13.3% and 16.2% in MSE respectively and 58.1% and 67.9% in wrong direction ratio respectively. We notice that there is dramatic improvement in wrong direction ratio which emphasizes more on the stability of real time control. One plausible explanation is that through neuron re-allocation by histogram equalization, the dense region (around origin) is better clarified and the connection between 3D sensory signals and 2D arm signals is less ambiguity. In addition, the performance of joint 1 is better improved than that of joint 0. We speculate that due to the structure of robotic arm, by same range in joint angle, the sensory signals caused by joint 0 spread into a larger range than that caused by joint 1. When we perform histogram equalization, the center regions of sensory signals, affected by both joint 0 and 1, are better refined. Therefore, the effect of joint 1 is better represented in the network.

3. Evaluation of Merged Neural Networks

To evaluate the performance of two stage neural networks, we introduce two LEDs that blue one is randomly placed in space as target and red one is held by hand of the robotic arm. The vision system keeps on tracking the blue LED target and observes the hand position if it’s seen in camera. At first, we randomly place the target position in reachable space of hand. Then the system will first make use of first stage neural network to move

0 20 40 60 80 100 120 140 160 180 200

Error: 0.98199, Wrong Direction Ratio: 2.5726%

0 20 40 60 80 100 120 140 160 180 200

Error: 1.4055, Wrong Direction Ratio: 4.7933%

Actual Arm Position Training Result

Actual Arm Position Training Result

15

the arm approach the target. Once hand position is found in visual system, the system will switch to second stage neural network and move further closer towards the target. At the beginning of each test, two joints are setting to their minimum bounds respectively. The observed difference between hand and target from camera is shown below (Figure 11):

Figure 11 Performance of two-stage neural network model in visual difference, the magenta marked point indicates the hand position is not seen in stereo camera,

the green marked point indicates the start of second stage neural network and the red marked point indicates the end of one test

Define error as the difference between target and hand position in pixel when the system is stabilized (at the end of each test). Then we measure the MSE of each dimension. The results show that the pixel differences between target and hand position are within 10 pixels. In camera image, it usually takes more than 5x5 pixel size to represent one LED.

By taking the centroid of the LED region, it is hard to further improve the performance.

Besides, when two different color LEDs get close, the light color will be mixed thus it introduces more uncertainty in locating the actual position of hand and target from stereo camera.

To evaluate the generated joint positions by two stage neural networks and the joint positions to reach the target, we run another test that moves the arm randomly in space with red LED held in hand. Then we put the target (blue LED) as close as possible to the hand (red LED). Thus the current joint position of arm can be approximated as that of the

0 100 200 300 400 500 600 700

-50 0 50

frame

Visual Signal Difference of Y with Error: 1.2566

0 100 200 300 400 500 600 700

0 200 400

frame

Visual Signal Difference of X Position in Left Camera with Error: 5.3298

0 100 200 300 400 500 600 700

0 200 400

frame

Visual Signal Difference of X Position in Right Camera with Error: 5.3298

16

corresponding target. Based on this approximation, we could evaluate both the performance in visual coordinates (Figure 12) and joint coordinates (Figure 13).

Figure 12 Performance of two-stage neural network model in visual difference, with similar color notation of Figure 11

Figure 13 Performance of two-stage neural network model in joint angles, with similar color notation of Figure 11

0 100 200 300 400 500 600 700

-50 0 50

frame

Visual Signal Difference of Y with Error: 2.0054

0 100 200 300 400 500 600 700

0 200 400

frame

Visual Signal Difference of X Position in Left Camera with Error: 10.8464

0 100 200 300 400 500 600 700

0 200 400

frame

Visual Signal Difference of X Position in Right Camera with Error: 10.8464

0 100 200 300 400 500 600 700

-40 -20 0 20 40

Joint 1 Position with Error 0.98394

frame

0 100 200 300 400 500 600 700

40 50 60 70 80

Joint 2 Position with Error 0.9408

frame

17

From Figure 12 and 13, we notice that the performance of visual difference is slightly worse. And the estimated mean square errors of both joints are less than 1 degree.

However as we cannot obtain the correct joint angles for that target, estimated errors could only illustrate relatively accurate performance of two stages neural networks.

VI. Conclusion

In this work, we have demonstrated a biologically plausible development of hand-eye coordination for visual target. The two stage neural networks are used to construct the connections between motor signals and sensory signals derived from stereo camera and pan-tilt unit. Histogram equalization method is applied in neuron allocation to maximize the connections between motor and sensory signals. After training both networks by Hebbian learning rule, the vision-guided reaching task is accurately achieved. Further work on this system includes deriving mathematical model for motor and sensory relationship and increasing the dimensionality of the system.

Reference

[1]. Kuperstein M. Neural model of adaptive hand-eye coordination for single postures.

Science, vol. 239, no. 4845, pp.1308–1311, 1988.

[2]. T. Martinetz, H. Ritter, and K. Schulten. Three-dimensional neural netfor learning visuo-motor coordination of a robot arm. IEEE Transactions on Neural Networks, Vol. 1, pp.131-136, 1990.

[3]. M. Marjanovic, B. Scassellati, and M. Williamson. Self-Taught Visually-Guided Pointing for a Humanoid Robot. In Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior, Sep. 1996.

[4]. H. Hashimoto, T. Kubota, W-C. Lo and F. Harashima. A Control Scheme of Visual Servo Control of Robotic Manipulators Using Artificial Neural Network. In Proc. IEEE Int. Conf.

Control and Applications, pp. TA 3 6, 1989.

[5]. E. Salinas and L. F. Abbot. Transfer of Coded Information from Sensory to Motor Networks.

Journal of Neuroscience, vol. 75, no. 10,pp.6461-647, Oct. 1995.

[6]. D. Caligiore, D. Parisi, and G. Baldassarre. Toward an integrated biomimetic model of reaching. In 6th IEEE International Conference on Development and Learning, pp.241-246, 2007.

[7]. Yiwen Wang, Tingfan Wu, Orchard, G., Dudek, P., Rucci, M., Shi, B.E.. Hebbian learning of visually directed reaching by a robot arm. Biomedical Circuits and Systems Conference, 2009. BioCAS 2009. IEEE , pp.205-208, 26-28 Nov. 2009.

[8]. Dzialo, Karen A., Schalkoff, Robert J.. Control Implications in Tracking Moving Objects Using Time-Varying Perspective-Projective Imagery. IEEE Transactions on Industrial Electronics, IE-33, Issue: 3, 1986, p247-253.