In this study, we have adopted the MLP network and a hybrid method that combines MLP network and GA to apply on the well logging data inversion. MLP network is trained by gradient descent method and conjugate gradient method. We also expand the input features in MLP network to design 5 higher-order feature neural nets, and the testing result of higher-order feature neural nets is compared to the testing result of general MLP network.
In order to get more training pattern and better convergence result, each well logging dataset is split into sections. The number of features in each section is corresponding to the number of input nodes of MLP network. We have tested the MLP networks with 10, 20, 40, 100, and 200 input nodes respectively. Experimental results show that the network with 10 input nodes has the smallest average of mean absolute error. Besides, the performance of BP algorithm depends on the parameters and topology of neural networks, so we test the different learning rate, the number of hidden nodes, and the number of hidden layers. As our experiments, the one-hidden layer network with learning rate of 0.6, momentum parameter of 0.4, and the number of hidden nodes of 1.2 times the number of input nodes have the best performance. We then design 5 higher-order feature neural nets using the gradient descent method and conjugate gradient method. The experimental results show that the higher-order feature neural nets always have better performance than general MLP network that without higher-order features. The average training time is shorter and the mean absolute error is smaller. The network HOCG-3 that expands input features using second-order function and third-order function provides the largest improvement among all of 5 higher-order feature neural nets.
The testing results presented show that our proposed higher-order feature neural nets using the conjugate gradient method is an effective alternative to well logging data inversion problem.
We also do the experiments on reversing the input and output of well logging data. This work is divided into two parts. First, we examine the inverse ability of MLP network using different number of input nodes with gradient descent method.
The MLP network with 10 and 20 input nodes can not converged. However, the MLP network with 40 and 50 input nodes converged, and the MLP network with 50 input nodes has better performance, so we use 50 input nodes for HOML and
HOCG. Second, we do experiments on HOML and HOCG and compare the network performance between them. Experimental results show that the average performance of HOCG is better than HOML. Also we notice that the experiments on reversing the input and output of well logging data are more time-consuming than previous normal well logging data inversion tests.
In addition to the higher-order feature neural nets, we use the hybrid method that combines the neural network that using gradient descent and genetic algorithm.
According to our testing result, it was shown that the performance of hybrid method has better performance than general MLP networks. Among the higher-order feature neural nets, HOML-3 with GA is the best combination that has the smallest average of mean absolute error. However, a disadvantage of using genetic algorithm in combination with neural network is the large computation cost required.
According to our experimental results, the network HOCG-3 has smallest average of mean absolute error, so we use HOCG-3 to do the well logging inversion which using the real field logs. The experimental result shows that our proposed higher-order feature neural net is an effective alternative to process the well logging data inversion.
ACKNOWLEDGEMENT
The author is grateful to Prof. Liang C. Shen for the well logging datasets, this paper will not be accomplished without these valuable datasets.
References
[1] P. Lippmann, “An Introduction to Computing with Neural Nets,” IEEE ASSP Magazine Volume 4, Issue 2, Part 1, pp. 4-22, April 1987.
[2] M. T. Hagan, H. B. Demuth, and M. Beale, Neural Network Design, PWS Publishing Co., Boston, 1996.
[3] L. S. Martin, D. Chen, and T. Hagiwara, “Neural Network Inversion of Array Induction Logging Data for Dipping Beds,” SPWLA 42nd Annual Logging Symposium, Paper U1-U11, June 17, 2001.
[4] Z. H. Tan, “Hybrid Evolutionary Approach for Designing Neural Networks for Classification,” Electronics Letters, Volume 40, Issue 15, pp. 955-957, 22 July 2004.
[5] W. Kinnebrock, “Accelerating the standard backpropagation method using a genetic approach,” Neurocomput., Volume 6, pp. 583-588, 1994.
[6] C. P. Erick and K. Chandrika, “Evolving Neural Networks for the Classification of Galaxies,” Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1019-1026, 2002.
[7] R. A. Dilruba, N. Chowdhury, F. F. Liza, and C. K. Karmakar, “Data Pattern Recognition Using Neural Network with Back-Propagation Training,”
International Conference on Electrical and Computer Engineering (ICECE), pp.
451-455, Dec. 2006.
[8] R. Hecht-Nielsen, “Theory of the Backpropagation Neural Network,”
International Joint Conference on Neural Networks (IJCNN), Volume 1, pp.
593-605, 18-22 June 1989.
[9] N. Qian, “On the momentum term in gradient descent learning algorithms,”
Neural Networks, Volume 12, Number 1, pp. 145-151, January 1999.
[10] M. Towsey, D. Alpsan, and L. Sztriha, “Training A Neural Network with Conjugate Gradient Methods,” IEEE International Conference on Neural Networks, Volume 1, pp. 373-378, 27 Nov.-1 Dec. 1995.
[11] D. Goryn and M. Kaveh, “Conjugate Gradient Learning Algorithms for Multilayer Perceptrons,” Proceedings of the Circuits and Systems 32nd Midwest Symposium, Volume 2, pp. 736-739, 14-16 Aug. 1989.
[12] J. R. Shewchuk, “An Introduction to the Conjugate Gradient Method Without the Agonizing Pain,” School of Computer Science Carnegie Mellon University Pittsburgh, Aug. 4, 1994.
[13] C. Charalambous, “Conjugate Gradient Algorithm for Efficient Training of Artificial Neural Networks,” IEE Proceedings on Circuits, Devices and
Systems, Volume 139, Issue 3, pp. 301-310, June 1992.
[14] C. H. Chen and H. Lai, “An Empirical Study of the Gradient Descent and the Conjugate Gradient Backpropagation Neural Networks,” OCEANS ’92.
‘Mastering the Oceans Through Technology’. Proceedings Volume 1, pp.
132-135, October 26-29, 1992.
[15] R. Reed, “Pruning Algorithms-A Survey,” IEEE Transactions on Neural Networks, Volume 4, Issue 5, pp. 740-747, Sept. 1993.
[16] K. Messer and J. Kittler, “Fast Unit Selection Algorithm for Neural Network Design,” 15th International Conference on Pattern Recognition, Volume 2, pp.
981-984, 3-7 Sept. 2000.
[17] D. Whitley, “A Genetic Algorithm Tutorial,” Statistics and Computing, Volume 4, Number 2, pp. 65-85, June 1994.
[18] J. Stender, “Introduction to Genetic Algorithms,” Applications of Genetic Algorithms, IEE Colloquium, pp. 1/1-1/4, 15 Mar 1994.
[19] Y. Sasaki, H. Garis, and P. W. Box, “Genetic Neural Networks for Image Classification,” IEEE International on Geoscience and Remote Sensing Symposium (IGARSS), Volume 6, pp. 3522-3524, 21-25 July 2003.
[20] D. X. Gone and X. G. Ruan, “A Neural Networks for Real Coded Genetic Algorithm,” IEEE International Conference on Robotics, Intelligent Systems and signal Processing, Volume 2, pp. 874-879, 8-13 Oct. 2003.
[21] Z. Michalewicz, “Evolutionary Computation Techniques for Nonlinear Programming Problems,” International Transactions in Operational Research, Volume 1, Number 2, pp. 223-240, 1994.
[22] X. Yao, “Evolving artificial Neural Networks,” Proceedings of the IEEE, Volume 87, Number 9, pp. 1423-1447, Sep. 1999.