Trajectory Accuracy

In this experiment, in order to see how keyframes affect trajectory accuracy, we per-form single robot localization with three robots on the field and compare the result of different input images sets which are extracted from the given sequence of images by us-ing the proposed method with different keyframe selection parameters. The images are captured by robots and localization results are calculated on the simulator. The ground truth system used to provide the benchmark for quantitative comparison is briefly intro-duced. A wide-angle camera was placed over the field to provide the position of the robots’ head which are carried color boards including blue, yellow, and red. Figure 5.9a is an image captured by the overhead camera. The ground truth position is calculated by projecting the center position of the color board from the image coordinates onto the world coordinates.Figure 5.9b shows the labeled blue points we projected from the image of Figure 5.9a to the world coordinates which are marked by red circles. The projection errors for the labeled points are recorded in Table 5.1.

point number 1 2 3 4 5 6 7 8 9

projection error (mm) 10.04 30.90 9.11 3.40 4.97 4.02 6.03 24.14 23.82 Table 5.1: Camera projection errors for labeled points.

Table 5.2 shows the absolute trajectory RMSE (ATE RMSE) and keyframe selection ratio with different thresholds. The adaptive threshold τais defined by

τ_a ≡

which means that if the moving object candidates set is empty, in order to keep more stationary objects, the threshold is set to lower value. By using τa, the keyframe ratio is less than τ = 0.6 about 7%. However, the accuracy is worse and using more number of frame by using τ = 0.7 than using τa. For Robot 58 and Robot 340, trajectories was

improved significantly (for up to 3x ATE RMSE) from odometry data by using keyframes but the most accurate results are calculated by using all frames. For Robot 58, the best performance of keyframe-based localization is about 43% of using all frames, and 33%

for Robot 340. The trajectories are shown in Figure 5.10 and Figure 5.12. The trade-ff between the localization correctness and resource saving is shown in Figure 5.13 for Robot 58, and in Figure 5.15 for Robot 340. They have the same tendency that the the gold cross is at τ ≈ 0.65.

For Robot 235, the best ATE RMSE is by using odometry only and even better than using all frames. The reason is that there are noisy measurements in the interval after the first spinning shown in Figure 5.11, and these measurements make the robot’s be-lieve not to move forward. As a result, the trade-off shown in Figure 5.15 has no gold cross. However, the best performance with the keyframe-based method is 85% of using all frames.

All frames Odometry τ = 0.6 τ_a τ = 0.7 τ = 0.8

Robot 58

52.53 323.51 82.43 85.57 119.94 140.07

100% 0% 26.6% 19.3% 20.8% 12.6%

Robot 235

376.89 327.32 432.47 406.249 416.53 424.74

100% 0% 23.6% 16.9% 18.5% 11.2%

Robot 340

122.49 638.28 204.59 219.72 228.46 263.22

100% 0% 24.9% 17.2% 18.8% 13.3%

Table 5.2: Comparison of absolute trajectory error (ATE) RMSE (unit: mm) and the keyframe percentage under different threshold (τ ) selection.

(a) The images which are selected as keyframes with τ = 0.8.

(b) Entropy over time.

Figure 5.6: The keyframes and entropy with threshold τ = 0.8 in stationary object sce-nario.

Figure 5.7: The timeline of keyframes which are selected by the system against thresholds in moving object scenario.

(a) The images which are selected as keyframes with τ = 0.8. and predicted distance of 900 mm.

(b) Entropy over time.

Figure 5.8: The keyframes and entropy with threshold τ = 0.8 and predicted distance of 900mm in moving object scenario.

(a)

(b)

Figure 5.9: (a) Setting for localization experiment. (b) Camera projection for labeled points (red circles) from (a) against the world coordinates (blue points).

Figure 5.10: Trajectory of Robot 58

Figure 5.11: Trajectory of Robot 235

Figure 5.12: Trajectory of Robot 340

Figure 5.13: The ATE RMSE and keyframe selection ratio of Robot 58.

Figure 5.14: The ATE RMSE and keyframe selection ratio of Robot 235.

Figure 5.15: The ATE RMSE and keyframe selection ratio of Robot 340.

Chapter 6 Conclusion

In this thesis, we designed a keyframe selection mechanism with a utility function as a preprocessing for MR-SLAT. The utility function measures low-level data extracted from an image by incorporating Shannon Entropy theory. If the data extracted from an image has high utility, meaning the image has significant information change compared with the previous keyframe, the image will be selected as the next keyframe and the data extracted from the image will be used for object detection modules as well as for measurement update of localization. Our method can detect the change of scene such as the shape of field line, the number of robots and the distance of robots between two frames. In localization experiment, the trajectory of localization can be significantly improved for up to 3x from odometry by using 25% of image data. when the trajectory error ratio between using all frames and odometry is close to 1, the best performance of the keyframe-based localization system is about 85% of by using all frames.

Bibliography

[1] S. I. Roumeliotis and G. A. Bekey, “Distributed multirobot localization,” IEEE Transactions on Robotics and Automation, vol. 18, no. 5, pp. 781–795, Oct 2002.

[2] A. Howard, M. J. Matark, and G. S. Sukhatme, “Localization for mobile robot teams using maximum likelihood estimation,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 1, 2002, pp. 434–439 vol.1.

[3] G. Ng and K. Ng, “Sensor management – what, why and how,” Infor-mation Fusion, vol. 1, no. 2, pp. 67 – 75, 2000. [Online]. Available:

http://www.sciencedirect.com/science/article/pii/S1566253500000099

[4] A. O. Hero and D. Cochran, “Sensor management: Past, present, and future,” IEEE Sensors Journal, vol. 11, no. 12, pp. 3064–3075, Dec 2011.

[5] H. Wang, G. Pottie, K. Yao, and D. Estrin, “Entropy-based sensor selection heuristic for target localization,” in Third International Symposium on Information Process-ing in Sensor Networks, 2004. IPSN 2004, April 2004, pp. 36–45.

[6] D. S. Zois, M. Levorato, and U. Mitra, “A pomdp framework for heterogeneous sensor selection in wireless body area networks,” in 2012 Proceedings IEEE INFO-COM, March 2012, pp. 2611–2615.

[7] S. Joshi and S. Boyd, “Sensor selection via convex optimization,” IEEE Transactions on Signal Processing, vol. 57, no. 2, pp. 451–462, Feb 2009.

[8] X. Shen and P. K. Varshney, “Sensor selection based on generalized information gain for target tracking in large sensor networks,” IEEE Transactions on Signal Pro-cessing, vol. 62, no. 2, pp. 363–375, Jan 2014.

[9] C. H. Chang, S. C. Wang, and C. C. Wang, “Exploiting moving objects: Multi-robot simultaneous localization and tracking,” IEEE Transactions on Automation Science and Engineering, vol. 13, no. 2, pp. 810–827, April 2016.

[10] C.-C. Wang, C. Thorpe, S. Thrun, M. Hebert, and H. Durrant-Whyte, “Simultaneous localization, mapping and moving object tracking,” The International Journal of Robotics Research, vol. 26, no. 9, pp. 889–916, 2007.

[11] H. Zhang, B. Li, and D. Yang, “Keyframe detection for appearance-based visual slam,” in 2010 IEEE/RSJ International Conference on Intelligent Robots and Sys-tems, Oct 2010, pp. 2071–2076.

[12] G. Lu, Y. Zhou, X. Li, and P. Yan, “Unsupervised, efficient and scalable key-frame selection for automatic summarization of surveillance videos,” Multimedia Tools and Applications, vol. 76, no. 5, pp. 6309–6331, Mar 2017. [Online]. Available:

https://doi.org/10.1007/s11042-016-3263-z

[13] Z. Dong, G. Zhang, J. Jia, and H. Bao, “Keyframe-based real-time camera tracking,”

in 2009 IEEE 12th International Conference on Computer Vision, Sept 2009, pp.

1538–1545.

[14] A. Das and S. L. Waslander, “Entropy based keyframe selection for multi-camera visual slam,” in 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Sept 2015, pp. 3676–3681.

[15] W. J. Beksi and N. Papanikolopoulos, “Point cloud culling for robot vision tasks under communication constraints,” in 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Sept 2014, pp. 3747–3752.

[16] M. Hsiao, E. Westman, G. Zhang, and M. Kaess, “Keyframe-based dense planar slam,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), May 2017, pp. 5110–5117.

[17] “Adaptive dynamic walking and motion optimization for humanoid robots,” Ph.D.

thesis, University of Miami, Department of Computer Science, 2015.