LSTM sequence length

Due to the different nature of labeling data, information about typhoon intensity class and central pressure values in besttrack dataset have different time of announcement, and therefore optimal sequence lengths for these tasks can be also different. Performing an experiment, similar to the one, which was described in previous tasks, the optimal value of input sequence length was found. The results are shown in Fig. 5.3.1.1.

Fig. 5.3.1.1. Errors distribution for different LSTM input sequence lengths.

Pressure regression prediction vs. ground truth for different sequence lengths charts are shown in Fig. 5.3.1.2.

Fig. 5.3.1.2. Pressure regression prediction vs. ground truth for different sequence lengths.

Examples of central pressure estimation for some sequences for different LSTM input sequence lengths are shown in Fig. 5.3.1.3. .

Fig. 5.3.1.3. Examples of central pressure estimation for some sequences for different LSTM input sequence lengths.

5.3.2. Results.

The outcome of this particular experiment are actually quite outstanding. The combination of the methods, and approaches which were used resulted in the model, with exceptional generalization performance, lead to a very robust network response with very good fitting and low mean error of ​7.42 hPa (10.36 hPa by Chen [10] and 8.30 hPa by Rodes-Guirao [48]). It has been shown that the optimal number of frame sequence for LSTM model is 7, which corresponds to providing 6 hour data as a model’s input.

6. Conclusion

The main goal of this research was aimed at improving techniques of analysis of typhoons, namely estimating typhoon intensity class, estimating typhoon central pressure, and determining the transition from tropical to extratropical cyclone phase.

Following techniques were applied to address the challenges of the project: Deep Learning, Convolutional Neural Nets, Long-Short Time Memory, Ensemble Learning.

To evaluate the achievements of this work, the results of the previous works by Danlan Chen [10] and Lucas Rodes-Guirao, which address the same problems are used as a reference. Following achievements were obtained: improved accuracy for typhoon classification problem (67.50%, vs. 63.92% by Chen and 58% by Rodes-Guirao); error for the pressure estimation task has been decreased (7.42 hPa vs. 10.36 hPa by Chen and 8.30 hPa by Rodes-Guirao); tropical-extratropical cyclone transition estimation accuracy has been improved (97.2% vs. 94.73% by Rodes-Guirao). In the tropical-extratropical transition state estimation task, it also has been shown, that labeled data, which is used for training contains low-quality data around the transition state time.

Apart from the research part, there was a goal of adding new features to a software package, called Pyphoon for an easier way of accessing, manipulating and processing data, which was used in this project. The contribution to the Pyphoon contains developing modules for data management and extraction (pyphoon.db), module for performing nonlinear interpolation of missing frames of typhoon images (pyphoon.interpolation), various experiment fixtures, which were used in this project and various data generators for different tasks, compatible with Keras framework to optimize utilization of storage and computational resources.

7. References

[1] Atlantic Oceanographic and Meteorological Laboratory, Hurricane Research Division. "Frequently Asked Questions: What are the average, most, and least tropical cyclones occurring in each basin?" (​http://www.aoml.noaa.gov/hrd/tcfaq/E10.html​) [2] World Meteorological Organization. “Latest Advisories on Current Tropical Cyclones Hurricanes Typhoons”. (​http://www.wmo.int/​)

[3] Central Weather Bureau. “Disaster Caused by Typhoon and Its Prevention”

(​https://www.cwb.gov.tw​)

[4] Mina Moradi Kordmahalleh et al. “Hurricane Trajectory Prediction via a Sparse Recurrent Neural Network”. In: Proceedings of the 5th International Workshop on Climate Informatics. 2015.

[5] Seungkyun Hong et al. “Globenet: Convolutional neural networks for typhoon eye tracking from remote sensing imagery”. In: arXiv preprint arXiv:1708.03417 (2017).

[6] Matsuoka, Daisuke, et al. "DETECTING PRECURSORS OF TROPICAL CYCLONE USING DEEP NEURAL NETWORKS."

[7] Thomas Loridan, Ryan P Crompton, and Eugene Dubossarsky. “A Machine Learning Approach to Modeling Tropical Cyclone Wind Field Uncertainty”. In: Monthly Weather Review 145.8 (2017), pp. 3203–3221.

[8] John A Knaff et al. “Statistical, 5-day tropical cyclone intensity forecasts derived from climatology and persistence”. In: Weather and Forecasting 18.1 (2003), pp.

80–92.

[9] John A Knaff, Charles R Sampson, and Mark DeMaria. “An operational statistical typhoon intensity prediction scheme for the western North Pacific”. In: Weather and Forecasting 20.4 (2005), pp. 688–699.

[10] Danlan Chen. “Typhoon Intensity Forecasts By Using Deep Learning Technique”.

MA thesis. Montreal, Quebec: School of Computer Science, McGill University, 2017.

[11] Ritchie, Elizabeth A., et al. "Tropical cyclone intensity estimation in the North Atlantic Basin using an improved deviation angle variance technique." Weather and Forecasting 27.5 (2012): 1264-1277.

[12] Dvorak, V. F., 1975: Tropical cyclone intensity analysis and forecasting from satellite imagery. Mon. Wea. Rev., 103, 420–430.

[13] Fetanat, Gholamreza, Abdollah Homaifar, and Kenneth R. Knapp. "Objective tropical cyclone intensity estimation using analogs of spatial features in satellite data."

Weather and Forecasting 28.6 (2013): 1446-1459.

[14] Emanuel, Kerry, et al. "Environmental control of tropical cyclone intensity." Journal of the atmospheric sciences 61.7 (2004): 843-858.

[15] Chu, Jan-Hwa. A Regression Model for the Western North Pacific Tropical Cyclone Intensity Forecast. No. NRL/MR/7541--94-7215. NAVAL RESEARCH LAB MONTEREY CA, 1994.

[16] Digital Typhoon: Typhoon Images and Information

(http://agora.ex.nii.ac.jp/digital-typhoon/)

[17] Pradhan, Ritesh & Aygun, Ramazan & Maskey, Manil & Ramachandran, Rahul &

J. Cecil, Daniel. (2017). Tropical Cyclone Intensity Estimation Using a Deep Convolutional Neural Network. IEEE Transactions on Image Processing. PP. 1-1.

10.1109/TIP.2017.2766358.

[18] Karen Simonyan and Andrew Zisserman. “Very deep convolutional networks for large-scale image recognition”. In: 2014.

[19] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. “Imagenet classification with deep convolutional neural networks”. In: Advances in neural information processing systems. 2012, pp. 1097–1105.

[20] Yann LeCun, Yoshua Bengio, et al. Convolutional networks for images, speech, and time series. 1995.

[21] Ioffe, Sergey, and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift." arXiv preprint arXiv:1502.03167 (2015).

[22] Dietterich, Thomas G. "Ensemble learning." The handbook of brain theory and neural networks 2 (2002): 110-125.

[23] Gers, Felix A., Jürgen Schmidhuber, and Fred Cummins. "Learning to forget:

Continual prediction with LSTM." (1999): 850-855.

[24] Ranjan, Anurag, and Michael J. Black. "Optical flow estimation using a spatial pyramid network." IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Vol. 2. IEEE, 2017.

[25] Kossin, James P., and Christopher S. Velden. "A pronounced bias in tropical cyclone minimum sea level pressure estimation based on the Dvorak technique."

Monthly weather review 132.1 (2004): 165-173.

[26] Typhoon Committee (2015). Typhoon Committee Operational Manual 2015. World Meteorological Organization. Retrieved November 13, 2015

[27] Saffir-Simpson Hurricane Wind Scale Extended Table (​https://www.nhc.noaa.gov/​) [28] Hong Kong Weather Watch (​http://www.hkww.org/weather/index_main.html​) [29] Velden, Christopher, et al. "The Dvorak tropical cyclone intensity estimation technique: A satellite-based method that has endured for over 30 years." Bulletin of the American Meteorological Society 87.9 (2006): 1195-1210.

[30] Dvorak, V. F., 1982: Tropical cyclone intensity analysis and forecasting from satellite visible or enhanced infrared imagery. NOAA National Environmental Satellite Service, Applications Laboratory Training Notes, 42 pp.

[31] Dvorak, V. F., 1984: Tropical cyclone intensity analysis using satellite data. NOAA Tech. Rep. 11, 45 pp.

[32] Erickson, C. O., 1972: Evaluation of a technique for the analysis and forecasting of tropical cyclone intensities from satellite pictures. NOAA Tech. Memo. NESS 42, 28 pp.

[33] Simonyan, Karen, and Andrew Zisserman. "Very deep convolutional networks for large-scale image recognition." arXiv preprint arXiv:1409.1556 (2014).

[34] Filonenko, Alexander, Laksono Kurnianggoro, and Kang-Hyun Jo. "Comparative study of modern convolutional neural networks for smoke detection on image data."

Human System Interactions (HSI), 2017 10th International Conference on. IEEE, 2017.

[35] Deng, Jia, et al. "Imagenet: A large-scale hierarchical image database." Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. Ieee, 2009.

[36] Zhou, Bolei, et al. "Learning deep features for scene recognition using places database." Advances in neural information processing systems. 2014.

[37] RSMC Tokyo-Typhoon Center, Best Track Data

(​https://www.jma.go.jp/jma/jma-eng/jma-center/rsmc-hp-pub-eg/trackarchives.html​) [38] Ha, Taehyeun, Seongjoo Lee, and Jaeseok Kim. "Motion compensated frame interpolation by new block-based motion estimation algorithm." IEEE Transactions on Consumer Electronics 50.2 (2004): 752-759.

[39] Niklaus, Simon, Long Mai, and Feng Liu. "Video frame interpolation via adaptive separable convolution." arXiv preprint arXiv:1708.01692 (2017).

[40] Neil Joshi, Duncan Woodbury. “Deep Motion: A Convolutional Neural Network for Frame Interpolation”, 2017

[41] Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. "U-net: Convolutional networks for biomedical image segmentation." International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, 2015.

[42] Liou, Cheng-Yuan, et al. "Autoencoder for words." Neurocomputing 139 (2014):

84-96.

[43] Ciresan, Dan; Meier, U.; Schmidhuber, J. (June 2012). "Multi-column deep neural networks for image classification". 2012 IEEE Conference on Computer Vision and Pattern Recognition: 3642–3649. arXiv:1202.2745

[44] Jones, Sarah C., et al. "The extratropical transition of tropical cyclones: Forecast challenges, current understanding, and future directions." Weather and Forecasting 18.6 (2003): 1052-1092.

[45] Lawrence, Steve, C. Lee Giles, and Ah Chung Tsoi. What size neural network gives optimal generalization? Convergence properties of backpropagation. 1998.

[46] Krogh, Anders, and Jesper Vedelsby. "Neural network ensembles, cross validation, and active learning." Advances in neural information processing systems.

1995.

[47] NASA, MODIS / LANCE (​https://lance.modaps.eosdis.nasa.gov/​)

[48] Rodés-guirao, Lucas. “Understanding the Typhoon Meteorological Satellite Image Dataset "Digital Typhoon" Through Deep Learning”. MA thesis. UPC Barcelona School of Telecommunications Engineering (2018).

[49] Sepp Hochreiter; Jürgen Schmidhuber (1997). "Long short-term memory". Neural Computation. 9 (8): 1735–1780. doi:10.1162/neco.1997.9.8.1735. PMID 9377276.

[50] Klaus Greff; Rupesh Kumar Srivastava; Jan Koutník; Bas R. Steunebrink; Jürgen Schmidhuber (2015). "LSTM: A Search Space Odyssey". IEEE Transactions on Neural Networks and Learning Systems. 28 (10): 2222. arXiv:1503.04069 Freely accessible.

doi:10.1109/TNNLS.2016.2582924.

[51] Chollet, François. "Xception: Deep learning with depthwise separable convolutions." arXiv preprint (2017): 1610-02357.