Part III: Execution time

In the previous chapter, we find out that supervised learning neural network can replace the algorithm like morphology operation and detect moving object in two or more special colors. The experiment in the section will show each execution time.

Table 4.6 shows the MONNE’s execution time in different block sizes, Table 4.7 shows the SCNN1’s execution time in different block sizes and Table 4.8 will show the total execution time with different condition.

We find out that fewer outputs is a absolute way and input size must be proportional to the operation radius if we want to get the better performance.

Although MONNE and MONND have different operations output design, with the same neural network structure, the execution time will be almost the same. As Table 4.6 and Table 4.7 shown, MONNE’s execution time is between 0.032sec~0.062sec and SCNN1’s execution time is between 0.036sec and 0.038sec. The executing time of morphology operation and image subtraction is quiet shorter in MATLAB R2010b, only 0.0025sec and 0.0001sec, respectively. As shown in Table 4.8, when system using MONNE, MONND and SCNN1 will let the execution time twice longer and the screen will be discontinuous. Considering the executing time, we don’t use all of the intelligent neural networks in this thesis.

49

Table 4.6 MONNE’s execution time in different block sizes.

Block size Input neuron Hidden neuron Output neuron Extraction time

m=4, n=2 16 10 2 0.046sec

m=5, n=3 25 10 9 0.037sec

m=7, n=5 49 10 9 0.032sec

m=9, n=5 81 10 25 0.062sec

m=9, n=7 81 10 25 0.035sec

Table 4.7 SCNN1’s execution time in different block sizes.

Block size The time to generate input

calculate time The time to generate output

total time

m=1, n=1 0.004 0.03 0.004 0.038sec

m=3, n=3 0.002 0.032 0.002 0.036sec

Table 4.8 Compare average execution time in different condition.

Average time Average frame rate

Without Character Recognition 0.04sec 25

With Character Recognition 0.14sec 6.6

System with MONNE and MONND 0.22~0.27sec 3.7~4.5 System with MONNE,MONND and SCNN1 0.026~0.32sec 3~3.8

50

Chapter 5

Conclusions and Future Works

The purpose of this thesis is to build up a system for children to learn words in an interactive way. In this thesis, the developed intelligent system can recognize the character correctly in a moving word card from a sequence of images.

The system is designed in three steps, including potential object localization, character extraction and character recognition. In the first step, it is required to detect the moving object in special color, or word card, and then determine the location of the word card in the image. A supervised learning neural network (MCNN) is used to extract the color and detect the moving object simultaneously. After applying the MCNN, the region of the word card in green color is extracted from a sequence of images; unfortunately, some noise exists therein. Using morphology operation and connected components labeling (CCL), the noise is removed and the region of the word card could be located correctly.

In the second step, use another supervised learning neural network (GNN) to, and then apply the morphology operation to reduce noise. The word card is thus achieved as a binary image with the shape of the character on it. By generating a plain binary card, the character on the word card can be extracted by subtracting the plain binary card. Besides, the total number of character’s pixels is used to determine whether the result is a character or not. In the third step, a scheme based on a set of concentric circles is adopted to extract the character features, and then feed the features into the third supervised learning neural network (CRNN) to recognize which word it is, the designed neural networks CRNN can robustly identify characters in

51

different translation, size, tilt and angle of rotation. The overall system processing time is about 0.15s.

Three intelligent neural networks are respectively proposed to detect moving word card in special color, to detect color for character extraction and to recognize characters. Besides the three neural networks we proposed, we also find out that supervised learning neural networks can be used to execute the image subtraction algorithm, morphology operation, and the moving object detection in two or more special colors. Related experiments have been shown in Chapter 4. However, due to the requirement of real-time operation, the proposed system does not implement them by the neural networks.

The proposed intelligent system has been demonstrated to be successful in intelligent word card image recognition system, which is an important field in robot research. For the future, some related research should be further investigated such as noise reduction, word card in multiple colors, more characters or even images.

52

Reference

[1] S. Haykin, Neural Networks: A Comprehensive Foundation (2nd Edition), Prentice Hall, 1998 [2] C. G. Looney, Pattern Recognition using Neural Networks: Theory and Algorithm for Engineers

and Scientists, NY, USA: Oxford University Press, 1997.

[3] K. Omar and A. Al-Shatnawi, “A comparative studybetween methods of Arabic baseline detection,” International Conference on Electrical Engineering and Informatics: ICEEI, 2009.

[4] A. E. Bryson and Y. C. Ho, Applied Optimal Control : Optimization, Estimation, and Control, New York, Taylor & Francis, 1975.

[5] P. J. Werbos, Doctoral dissertation: Beyond regression: New tools for prediction and analysis in the behavioral sciences, PhD thesis, Harvard University, 1974.

[6] Y. LeCun, “A Learning procedure for asymmetric network,” In Proceedings of Cognitiva, Paris, pp. 599-604, 1985.

[7] J. Heikkilä and O. Silvén, , “A real-time system for monitoring of cyclists and pedestrians,”

IEEE Proc. on Visual Surveillance, pp. 74-81, 1999.

[8] C. Cédras and M. Shah, “Motion-based recognition: a survey,” Image and Vision Computing .vol.13, No.2, pp.129-155, March, 1995.

[9] A. Neri, S. Colonnese, G. Russo and P. Talone, “Automatic moving object and background separation,” Signal Process. pp.219-232,1998.

[10] C. Rafael, E. Woods and L. Steven, Image Processing: Digital Image Processing Using Matlab(2nd Edition), McGraw-Hill Education, LLC, USA, 2011.

[11] N. Rahman, K. Wei and J. See,: “RGB-H-CbCr Skin Color Model for Human Face Dection,”

International Symposium on Information & Communications Technologies 2006.

53

[12] L. Bretzner, I. Laptev and T. Lindeberg, “Hand Gesture Recognition using Multi-Scale Colour Features, Hierarchical Models and Particle Filtering,” IEEE, Automatic Face and Gesture Recognition (AFGR 02), 2002.

[13] Q. Peng and X. Zhang, Sensitive Image Recognition Technology Based on Eigenvectors, Academic Journal:Southwest Jiaotong University, 2007.

[14] V. Vezhnevets, V. Sazonov and A. Andreeva, A Survey on Pixel- Based Skin Color Detection Techniques, Graphicon: 2003.

[15] A. S. Abdallah, A. L. Abbott and M. A. El-Nasr, “A New Face Detection Technique using 2D DCT and Self Organizing Feature Map,” World Academy of Science, Engineering and Technology, vol. 24, pp. 15-19, 2007.

[16] L. A. Torres-Méndez, J. C. Ruiz-Suárez and G. Gómez, “Translation, Rotation, and Scale-Invariant Object Recognition,” IEEE Transactions on Systems Man and Cybernetics Part C Applications and Reviews, vol. 30, no. 1, 2000.

[17] L. A. Torres-Mendez, “Invariant 2-D object recognition,” Graduate Program in Informatics, ITESM-Campus Morelos, México, 1995.

[18] Y. N. Hsu and H. H. Arsenault, “Optical pattern recognition using the circular harmonic expansion,” Appl. Opt., vol. 21, no. 22, pp. 4016–4019,1982.