Figure A.1 displays step-by-step illustration of our incremental learning approach.
Figure A.1: Step-by-step illustration of our incremental learning approach.
Reference
[1] Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. Surf: Speeded up robust features.
Computer vision–ECCV 2006, pages 404–417, 2006.
[2] David G Lowe. Distinctive image features from scale-invariant keypoints. Interna-tional journal of computer vision, 60(2):91–110, 2004.
[3] Stefan Hinterstoisser, Cedric Cagniart, Slobodan Ilic, Peter Sturm, Nassir Navab, Pascal Fua, and Vincent Lepetit. Gradient response maps for real-time detection of textureless objects. IEEE Transactions on Pattern Analysis and Machine Intelli-gence, 34(5):876–888, 2012.
[4] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–
2324, 1998.
[5] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information pro-cessing systems, pages 1097–1105, 2012.
[6] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet:
A large-scale hierarchical image database. In Computer Vision and Pattern Recog-nition, 2009. CVPR 2009. IEEE Conference on, pages 248–255. IEEE, 2009.
[7] M. Everingham, S. M. A. Eslami, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zis-serman. The pascal visual object classes challenge: A retrospective. International Journal of Computer Vision, 111(1):98–136, January 2015.
[8] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740–755. Springer, 2014.
[9] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep net-work training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015.
[10] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.
[11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into recti-fiers: Surpassing human-level performance on imagenet classification. In Proceed-ings of the IEEE international conference on computer vision, pages 1026–1034, 2015.
[12] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization.
arXiv preprint arXiv:1412.6980, 2014.
[13] Hossein Azizpour, Ali Sharif Razavian, Josephine Sullivan, Atsuto Maki, and Stefan Carlsson. From generic to specific deep representations for visual recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 36–45, 2015.
[14] George E Dahl, Dong Yu, Li Deng, and Alex Acero. Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Transactions on audio, speech, and language processing, 20(1):30–42, 2012.
[15] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine trans-lation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.
[16] Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernockỳ, and Sanjeev Khu-danpur. Recurrent neural network based language model. In Interspeech, volume 2, page 3, 2010.
[17] Geoffrey E Hinton, Simon Osindero, and Yee-Whye Teh. A fast learning algorithm for deep belief nets. Neural computation, 18(7):1527–1554, 2006.
[18] Ruslan Salakhutdinov and Geoffrey Hinton. Deep boltzmann machines. In Artificial Intelligence and Statistics, pages 448–455, 2009.
[19] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680, 2014.
[20] David Held, Sebastian Thrun, and Silvio Savarese. Robust single-view instance recognition. In Robotics and Automation (ICRA), 2016 IEEE International Confer-ence on, pages 2152–2159. IEEE, 2016.
[21] Giulia Pasquale, Carlo Ciliberto, Lorenzo Rosasco, and Lorenzo Natale. Object identification from few examples by improving the invariance of a deep convolu-tional neural network. In Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on, pages 4904–4911. IEEE, 2016.
[22] Robi Polikar, Lalita Upda, Satish S Upda, and Vasant Honavar. Learn++: An in-cremental learning algorithm for supervised neural networks. IEEE transactions on
systems, man, and cybernetics, part C (applications and reviews), 31(4):497–508, 2001.
[23] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
[24] Suhang Wang, Yilin Wang, Jiliang Tang, Kai Shu, Suhas Ranganath, and Huan Liu.
What your images reveal: Exploiting visual contents for point-of-interest recom-mendation. In Proceedings of the 26th International Conference on World Wide Web, pages 391–400. International World Wide Web Conferences Steering Commit-tee, 2017.
[25] Ning Qian. On the momentum term in gradient descent learning algorithms. Neural networks, 12(1):145–151, 1999.
[26] Tijmen Tieleman and Geoffrey Hinton. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning, 4(2):26–31, 2012.
[27] Roger Ratcliff. Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions. Psychological review, 97(2):285–308, 1990.
[28] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, and Christoph H Lampert. icarl:
Incremental classifier and representation learning. arXiv preprint arXiv:1611.07725, 2016.
[29] Robert M French. Semi-distributed representations and catastrophic forgetting in connectionist networks. Connection Science, 4(3-4):365–377, 1992.
[30] Robert M French. Dynamically constraining connectionist networks to produce dis-tributed, orthogonal representations to reduce catastrophic interference. network, 1111:00001, 1994.
[31] Rupesh K Srivastava, Jonathan Masci, Sohrob Kazerounian, Faustino Gomez, and Jürgen Schmidhuber. Compete to compute. In Advances in neural information pro-cessing systems, pages 2310–2318, 2013.
[32] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Des-jardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks.
Proceedings of the National Academy of Sciences, page 201611835, 2017.
[33] Michael D Muhlbaier, Apostolos Topalis, and Robi Polikar. Learn++. nc: Combin-ing ensemble of classifiers with dynamically weighted consult-and-vote for efficient incremental learning of new classes. IEEE transactions on neural networks, 20(1):
152–168, 2009.
[34] Lee Sang Woo, Heo Min Oh, Kim Jiwon, Kim Jeonghee, and Zhang Byoung Tak.
Dual memory architectures for fast deep learning of stream data via an online-incremental-transfer strategy. arXiv preprint arXiv:1506.04477, 2015.
[35] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
[36] Jimmy Ba and Rich Caruana. Do deep nets really need to be deep? In Advances in neural information processing systems, pages 2654–2662, 2014.
[37] Zhizhong Li and Derek Hoiem. Learning without forgetting. In European Confer-ence on Computer Vision, pages 614–629. Springer, 2016.
[38] Saurabh Gupta, Judy Hoffman, and Jitendra Malik. Cross modal distillation for supervision transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2827–2836, 2016.
[39] Heechul Jung, Jeongwoo Ju, Minju Jung, and Junmo Kim. Less-forgetting learning in deep neural networks. arXiv preprint arXiv:1607.00122, 2016.
[40] Anthony Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connec-tion Science, 7(2):123–146, 1995.
[41] Robert M French, Bernard Ans, and Stephane Rousset. Pseudopatterns and dual-network memory models: Advantages and shortcomings. In Connectionist models of learning, development and evolution, pages 13–22. Springer, 2001.
[42] Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. Visualizing higher-layer features of a deep network. University of Montreal, 1341:3, 2009.
[43] Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson. Un-derstanding neural networks through deep visualization. arXiv preprint arXiv:
1506.06579, 2015.
[44] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. http://www.deeplearningbook.org.
[45] Kevin Lai, Liefeng Bo, Xiaofeng Ren, and Dieter Fox. A large-scale hierarchical multi-view rgb-d object dataset. In Robotics and Automation (ICRA), 2011 IEEE International Conference on, pages 1817–1824. IEEE, 2011.
[46] Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM international confer-ence on Multimedia, pages 675–678. ACM, 2014.
[47] Kevin Lai, Liefeng Bo, and Dieter Fox. Unsupervised feature learning for 3d scene labeling. In Robotics and Automation (ICRA), 2014 IEEE International Conference on, pages 3050–3057. IEEE, 2014.
[48] Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European conference on computer vision, pages 818–833. Springer, 2014.
[49] Ross Girshick. Fast r-cnn. In Proceedings of the IEEE International Conference on Computer Vision, pages 1440–1448, 2015.
[50] Georgios Georgakis, Md Alimoor Reza, Arsalan Mousavian, Phi-Hung Le, and Jana Košecká. Multiview rgb-d dataset for object instance detection. In 3D Vision (3DV), 2016 Fourth International Conference on, pages 426–434. IEEE, 2016.
[51] Ali Borji, Saeed Izadi, and Laurent Itti. ilab-20m: A large-scale controlled object dataset to investigate deep learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2221–2230, 2016.
[52] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(Nov):2579–2605, 2008.
VITA
姓名: 柯達方 性別: 男
生日: 民國 81 年 1 月 13 日 (1992/01/13) 籍貫: 中華民國台北市
學歷:
1. 民國 106 年 國立台灣大學電機工程學研究所畢業
2. 民國 103 年 國立台灣大學工程科學與海洋工程學系畢業
3. 民國 99 年 國立師範大學附屬高級中學畢業
發表著作:
1. Ren C. Luo, Da-Fang Ke, ”Mitigate Catastrophic Forgetting in CNNs for Effec-tive Instance Recognition.” 49th International Symposium on Robotics (ISR 2017Asia), Shanghai, China, July 5-8, 2017.
榮譽事蹟:
2016 年 「2016 長庚醫療財團法人醫療機器人大賽」榮獲團體組冠軍
2016 年 「2016 上銀智慧機器手第九屆實作競賽」榮獲團體組亞軍
2016 年 「2016 智慧機器人創意競賽國產工業機器人組」榮獲團體組季軍