3. Complete Characterization of Ultrashort Coherent Optical Pulses with SHG
3.6 Conclusions
We developed a BP ANN which can be invoked to retrieve the spectral phase of a coherent pulse from the measured SHG spectrum. We proposed a setup to be used for the experimental realization of the concept. By using this apparatus, we only need to measure the spectrum of the coherent pulse under study and then combine the spectrum with a variety of spectral phase profiles to prepare the SHG spectra for training the BP ANN. The trained BP ANN can be invoked to retrieve the spectral phase profile for the
SHG crystal
Spectrometer Phase
Modulator Plus
ANN System Phase information
Phase retrieve
Fig. 3.11. Proposed Experimental setup.
the predicted phases can achieve the target profile with more than 90% confidence.
Thanks to the computation efficiency of BP ANN, the technique developed in this study offers a possibility to perform a complete-field characterization of a coherent excitation to a physical system and quantum control of the system in a single setup.
Chapter 4 Real-Time Localization of Nano Objects at the Nanometer Scales
4.1 Introduction
An isolated fluorescent molecule or nano object will be observed like a light spot under an optical microscope. The spatial profile of the light spot simply reveals the point spread function of the optical microscope used. The peak position of the light spot can be determined with an accuracy of 1 nm if the signal-to-noise ratio of the detection is high enough. This impressive feature of nanometer localization with optical microscopy had recently inspired many applications including Fluorescence Imaging with One Nanometer Accuracy (FIONA) [37], sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM) [38], and fluorescence photoactivation localization microscopy (FPALM) [39], etc. Important biophysical mechanisms at the subcelluar scales had been discovered [40].
In the historical point of view, we noticed that a modified Hough transformation had been developed to detect a circular object [41] for recognizing and classifying interesting features present in a phase-contrast (PC) cytological image. Fillard, et al. had invoked the frequency dependence of the argument of Fourier transform to analyze an in-focus two-dimensional Airy disk [42]. Alexander, et al. proposed a method to
eliminate the systematic error in centroid estimation and achieved a subpixel accuracy [43]. It is interesting to know that a diffractive optical element (DOE) [44] had been applied to effectively locate a laser spot on a projection screen. The method could be invoked to achieve nanometer localization for optical microscopy. Anderson had presented an algebraic solution to the problem of localizing single fluorescent particle with sub-diffraction-limit accuracy [45]. Qu, et al. [46] had demonstrated nanometer localization of multiple single-molecules (NALMS) by using fluorescent microscopy and photobleaching properties of fluorophores. Cui, et al. [47] had devised an optimized algorithm useful for localizing light spots in high noise background. A method [48]
combining the radial basis network with anisotropic Gaussian basis function had been used to detect the position of a fluorescent protein. Fillard [49] relied on the Fourier phase frequency dependence to achieve sub-pixel localization accuracy of a light spot.
Enderlein [50] had proposed a method useful for tracking single fluorescent molecules diffusing in a two-dimensional membrane by invoking a rotating laser focus to track the position of the molecule. More information about single-molecular imaging and spectroscopy can be found in [51].
Based on the technical review, we found the major issue in the localization and tracking of nano objects is how to localize these objects accurately and rapidly with minimum invasiveness. To achieve the goal, many algorithms had been developed.
Fitting the light spot to a 2D Gaussian function is the most popular technique in this field. Another useful technique is to retrieve the peak position of a light spot via a center-of-mass approach [4]. However, to invoke these two techniques to localize a light spot with large size often fails to yield an accurate result. Therefore, in this chapter we will develop an ANN model to rapidly localize multiple light spots with high accuracy.
We invoked the feature of function approximation of ANN. We expect that the localization accuracy of ANN can be further improved when more data are accumulated.
Comparing to the 2D Gaussian fitting method, our ANN localization method is also less sensitive to noise influence.
4.2 Data Preparation for Training the Artificial Neuron Network
Localization Model
To train and test the performance of an ANN localization model, we prepared an image of 10 bright spots with 256x256 pixels as shown in Figure 4.1. The brightness profile of the light spots is Gaussian. The main target of this study is to construct a trained ANN model which can be invoked to yield the peak positions with a localization error less than one pixel.
4.3 First Test Run of the Artificial Neuron Network Localization Model
In this study, we constructed a BP ANN for localizing bright spots in an observing region. To serve this purpose, we began at an image of one spot with 25x25=625 pixels shown in Fig. 4.2. We input this image into the BP ANN. The output is the coordinates (x, y) of the spot. Therefore, the BP ANN possesses a total of 625 input nodes, 2 output nodes, and a hidden layer of 30 nodes. The activation function of the hidden layer is chosen to be hyperbolic tangent (tanh), while the activation function of the output layer is sigmoid function. We set the rate constant of learning to be 1.0 and 0.1 for the first and the second weighting layer, respectively.
Fig 4.1. An image of 10 bright spots prepared for training the artificial neuron network localization model.
The performance of the BP ANN trained by 10000 epochs with 1000 samples is shown in Figure 4.3. The distribution of the localization error deduced from the results with a training set of 1000 images or 1000 test images reveals that the localization error can be smaller than one pixel. However, in this example, we did not take into account the noise influence and only single bright spot is included.
0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 0
50 100 150 200
Num. of sample
Localization Error (Pixel)
Data Test
Fig. 4.3. The distribution of the localization error deduced from a run with either
Fig. 4.2. One of the images with 25x25=625 pixels and containing single bright spot, used for training the artificial neuron network localization model.
4.3.1 Difficulty Encountered by the Artificial Neuron Network to
Localize Multiple Light Spots
In the previous section, we found that the BP ANN can be used to successfully localize a single bright spot with a localization error less than one pixel. However, it is difficult to use single BP ANN to searching an area with multiple light spots. In fact, in a real situation, we do not know how many spots are involved in the beginning. Therefore, we try to combine two BP ANNs with the first ANN to determine how many spots are involved in a large region, and divide the region into a series of smaller domains with each containing single spot only. And then the second ANN will take over to localize each spot with high accuracy. The second BP ANN can be the same as the one described in the previous section. Thus, we separate the ANN localization procedure into the ANN searching step and the ANN localization step. We will detail each steps in the following sections.
4.3.2 The ANN Searching Step: Searching Over an Entire Region to
Deduce the Number of Light Spots
In order to detect how many spots involved and their rough locations over an image area, we design a BP ANN to search over the entire region to find out the pixels with an intensity value exceeding a threshold value. Based on this searching result, we
shall be able to deduce the number of light spots and roughly locate their positions.
4.3.3 The ANN Localization Step: Localize a Smaller Region to Yield
the Coordinates with High Accuracy
We had demonstrated in Section 4.3 that a light spot in a region of 25x25 pixels can be localized to subpixel accuracy. For a quantitative analysis, we draw all the coordinates deduced in the learning phase over the image area of 25x25 pixels in Figure 4.4. The average inter-spot distance is about one pixel; the localization error is 0.5 pixel.
0 5 10 15 20 25
0 5 10 15 20 25
Data
Y Pos (pixel)
X Pos (Pixel)
4.4 Training ANN over a Small Image Region with Higher Data Density
Fig 4.4. Distribution of the peak position of light spot taken from 1000 trainingdata over an image area of 25x25 pixels.
achieve this goal, we build a BP ANN and apply it to analyze images with 11x11 pixels as shown in Figure 4.5. Furthermore, to reduce the localization error, we increase the number of training data to 10000.
The BP ANN we created contains 121 input nodes and 2 output nodes. The activation function of the input layer is chosen to be hyperbolic tangent (tanh) and that of the output layer is sigmoid function. The learning rates are 1.0 and 0.1 for the input and output weighting layers, respectively.
Fig. 4.5. One of the training images with 11x11 pixels and single bright spot.
3 4 5 6 7 8 3
4 5 6 7 8
10000 train data
Y Pos(Pixel)
X Pos(Pixel)
The peak positions of the 10000 training data are presented in Figure 4.6. The average inter-spot distance is much smaller than one pixel. After training 7500 epochs, the result of localization error is shown in Figure 4.7.
Fig. 4.6. Distribution of the peak position of light spot taken from 10000 training data over an image area of 11x11 pixels.
0.0 0.2 0.4 0
50 100 150 200 250
Num. of sample
Localization Error (Pixel)
Train data Test dataFrom this figure, we can conclude that the spots in an image area of 11x11 pixels can be localized by the ANN to within an error of 0.1 pixel.
4.5 Comparison between ANN Localization Model and 2D Gaussian
Profile Fitting Method
In the single molecular research with nanometer localization and tracking technique, the two-dimensional Gaussian fitting technique used is often limited to an image size of 11x11 pixels due to the compromise between speed and localization accuracy. Therefore, it is interesting to study the performance comparison between our
Fig. 4.7. The distribution of localization error taken from either 1000 training data or 1000 test data after the ANN has been trained for 7500 epochs.
11x11 pixels.
The typical localization error of the 2D Gaussian fitting method is zero and it takes about 0.03 sec to complete the fitting process for each spot. On the other hand, the average localization error of our ANN is about 0.05 pixel but the localization time for each spot is negligible. The 2D Gaussian fitting method can be rendered into a linear algebraic problem and the computation time will increase rapidly as the number of the unknown parameters increases. Therefore, as show in Fig 4.8, we can find that the difference of the computation time between the two methods becomes significant as the image size and therefore the number of spots involved increases.
0 5 10 15 20
Fig. 4.8. Comparison showing the localization time of our ANN localization model and the 2D Gaussian profile fitting method as a function of the number of spots.
4.6 Noise Influence
For a real application with ANN, we shall consider the influence of noise that is unavoidable in experimentally measured data. In this section, we will study the influence of noise on the performance in more detail.
We prepared a set of test images by adding into the images with a noise level from 10 % to 50 % of the peak height of the light spot. The resulting images after adding noise are shown in Figure 4.9.
We used these images as a template and varies the spot position to generate a training set of 10000 images. We trained an ANN with structure the same as that to be described in Section 4.7. After training, the learning performances are shown in Figure 4.10.
Fig. 4.9. Images showing the influences of noise on the training data. The noise level is set to be 0%, 10 %, 20 % , 30%, 40% and 50 % of the peak height (from top left to right bottom), respectively.
0.0 0.2 0.4 0.6 0.8 1.0
Fig. 4.11. The average localization error as a function of noise level in the test data.
Fig. 4.10. The distribution of localization error taken from 1000 most populated test data among the set of 10000 data used in our ANN model. The test data are affected by noise with 10%, 20%, 30%, 40%, and 50% of peak height.
We can find that the localization error increases as the noise level of the test images is increases. However, as shown in Fig. 4.11, even the noise level has been raised to 50%
the average localization error can still be kept below 1 pixel.
For a fair comparison, we will examine the influence of noise on the 2D Gaussian fitting method by using the same test data set. The result is presented in Figure 4.12.
Because our test spot was generated from 2D Gaussian profile, the resulting localization error with the 2D Gaussian fitting method is zero when noise is negligible. The localization error increases as noise level is increased. If we focus on the cases with the localization error below 1 pixel, the performance with the 2D Gaussian fitting method is almost identical to the ANN. However, as the noise level is above 30%, the probability that the 2D Gaussian fitting method fails to yield a peak position is larger than 20%, leading to that the average localization error becomes larger than 1 pixel as shown in Figure 4.13.
0.0 0.2 0.4 0.6 0.8 1.0
Fig 4.13. The distribution of localization error taken from 1000 most populated test data among the set of 10000 data used in the 2D Gaussian fitting method. The test data are affected by noise with 30%, 40%, and 50% of peak height.
Fig 4.12. The distribution of localization error taken from 1000 most populated test data among the set of 10000 data used in the 2D Gaussian fitting method. The test data are affected by noise with 0%, 10%, 20%, 30%, 40%, and 50% of peak height.
4.7 The ANN Localization Model Suited for Large Area Searching
As mentioned in Section 4.3.1, we divide our ANN localization process into two steps. In the first stage, we use the first ANN to determine how many spots are involved in the entire region. For an image as shown in Figure 4.14, depending on the hardware, the complete localization (shown in Fig. 4.15) of all light spots may take less than 0.3 sec.
Fig. 4.14. An image of 10 bright spots in a size of 256x256 pixels prepared for testing the artificial neuron network localization model.
0 64 128 192 256 0
64 128 192
256 Find
Target
Y Pos(pixel)
X pos (pixel)
In the second stage, we isolate a squared region around each spots within which the pixel value is larger than 90% of the peak value. The images of these isolated domains with each occupying 11x11 pixels are sent to the ANN to locate the peak position of each light spot with high accuracy.
In the following, we will analyze the influence of noise on the performance to assess the potential for a real application. The test image is added with a noise level of 10 % and 20 % of the peak height, respectively. The trained ANN model is used to localize the light spots. The results are shown in Figure 4.16.
Fig. 4.15. The target positions (red circles) and the positions (black dots) retrieved with our ANN localization model.
From these case studies reported in this chapter, we can conclude that localization of light spots with ANN is feasible with a performance better than that with the 2D Gaussian fitting method. This is true, especially when high noise level is involved in the images. The ANN also requires less computation effort in comparison with the Gaussian
Fig. 4.16. The same run as Fig. 4.15 except that the test images are affected with different noise levels: (a) 10%; (b) 30%.
time, this procedure only needs to be done once before its use.
Chapter 5 Conclusions and Future Work of This Thesis
5.1 Conclusion of This Thesis
In this thesis, we aim to implement the learning ability of artificial neuron network into an optical apparatus to accumulate the user experiences and improve the prediction accuracy of the artificial neuron network as more data are collected. We demonstrated that artificial neuron network can improve the measurement accuracy or generate new functionality of an apparatus. We focus on two case studies in optical metrology, including the complete characterization of ultrashort laser pulses and nanometer localization in real time for an optical microscope. From the first case study, we conclude that a trained back-propagation artificially neuron network can be invoked to retrieve the spectral phase profile of a coherent pulse from a measured SHG spectrum.
The technique developed in this study offers a possibility to perform a complete-field characterization of a coherent excitation to a physical system and quantum control of the system in a single setup.
From the second case study, we developed a two-stage ANN strategy by first determining how many spots involved in the entire image. And then we invoked the second ANN to further process a set of squared domains around each spots to localize each spot. We successfully used the two-stage ANN strategy to localize multiple light
spots in a large area with high accuracy. We concluded that localization of light spots with ANN is feasible with a performance better than that with the 2D Gaussian fitting method. This is true, especially when high noise level is involved in the images. In addition, the ANN localization model also requires less computation effort in comparison with the Gaussian fitting method.
5.2 Future Work
Although we have successfully applied ANN in two cases in optical metrology, the following issues related to the current work remain unsolved and worth further study:
(1) How many hidden nodes should one use in an ANN to reach the best performance for a specific application?
(2) Since the ANN is powerful for modeling a nonlinear system, it is possible to combine ANN with other solvers for nonlinear equations such as Landsweber iteration method to enhance the iteration performance.
(3) The initial guesses of the weighting parameters and biases may influence the learning speed and thereby the convergence. How to set up a better initial guess worth further study.
(4) In this thesis, we applied ANN to localize multiple bright spots in an optical microscopic image. It is also very interesting to develop an ANN to recognize a
complex object in an image. A related issue is how to distinguish the object from its environment (i.e., how to determine the boundary of the object?) Or how to derive a criterion to guide a computer to determine the boundary of an object automatically?
References
[1] E. M. Hempe, P. C. J., A. Podlipensky, "Measuring ultrashort laser pulses Intensity Autocorrelation", project for the "Elitestudiengang Physik Regensburg-Erlangen", (2006).
[2] D. J. Kane, R. Trebino, "Characterization of Arbitrary Femtosecond Pulses Using Frequency-Resolved Optical Gating", IEEE Journal of Quantum Electronic, Vol.29, No.2, P.571 (1993).
[3] T. Tanabe, H. Tanabe, Y. Teramura, and F. Kannari, "Spatiotemporal measurements based on spatial spectral interferometry for ultrashort optical pulses shaped by a Fourier pulse shaper", Opt. Soc, Vol.19, No.11, P.2795, (2002).
[4] M. R. Brady, "Subpixel Resolution Schemes for Multiphase Flows", Virginia Polytechnic Institute and State University, Master of Science, ( 2006).
[5] 蘇木春, 張孝德, "機器學習:類神經網路、模糊系統以及基因演算法則", 全華科 技圖書股份有限公司, (2006).
[6] V. V. Zharkova, V. Schetinin, "A Neural-Network Technique for Recognition of Filaments in Solar Images", Lecture Notes in Computer Science, Vol.2773, P.148, (2003).
[7] H. A. Rowley, S. Baluja, T. Kanade, "Neural Network-Based Face Detection", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol.20, No.1, P.23, (1998)
[8] J. Korczak, P. Roger, "Stock timing using genetic algorithms", Applied Stochastic Models in Business and Industry, Vol.18, Issue.2, P.121, (2002).
[9] W. S. McCulloch,W. H. Pitts, "A logical calculus of the ideas immanent in nervous activity", Bulletin of Mathematical Biophysics, Vol. 5, P. 115, (1943)
[10] F. Rosenblatt, "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain", Psychological Review, Vol.65, No.6, P.386, (1958).
[11] Marvin Minsky, Seymour Papert, "Perceptrons: An Introduction to Computational Geometry", MIT Press, (1969).
[12] T. Kohonen, "Self-organized formation of topologically correct feature maps", Biological Cybernetics, Vol.43, P.59, (1982).
[13] J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities", Proc. Natl. Acad. Sci. USA, Vol.79, P.2554, (1982) . [14] D. E. Rumelhart, J. L. McClelland, "Parallel Distributed Processing", MIT Press,
(1986).
[15] G. A. Carpenter,S. Grossberg, D. B. Rosen, "Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system", Neural Networks, Vol.4, Issue.6, P.759, (1991).
[16] L.-N. Hossein, M. Nasiruddin, T. Samad, "Effect of Initial Weights on Back-Propagation and It’s Variations", Systems, Man and Cybernetics, IEEE International Conference, Vol.1, P.218, (1989).
[17] J. K. Kruschke, J. R. Movellan, "Benefits of Gain: Speed Learning and Minimal
[17] J. K. Kruschke, J. R. Movellan, "Benefits of Gain: Speed Learning and Minimal