Construct Neural Network Model Using Genetic Algorithm 廖鴻翰、陳木松

Construct Neural Network Model Using Genetic Algorithm 廖鴻翰、陳木松

E-mail: [email protected]

ABSTRACT

In this thesis a new Genetic Algorithm to optimize weights and topology of Neural Networks is presented and compared with other learning methods, such as gradient-descent learning algorithm, and other evolutionary system. Since the characteristics of topology space and weight space (one of them is in integer space and the other is in real space) are absolutely different, it is very difficult to optimize both of them at the same time. Cascade-Correlation algorithm (CCA) is a popular supervised learning architecture that dynamically grows layers of hidden neurons, so that the network topology (size, depth) can be determined with network weights at the same time. CCA is a gradient-descent based learning algorithm. It is known that the gradient methods always foiled by local minimum problem. On the other hand, it is powerful on local search but insufficient on global search. Genetic algorithm (GA) is a computationally intensive optimization method. The rewards from the huge computational power are some very desirable properties. One of them is that a global search is performed during the optimization. But unfortunately, GA is insufficient on local search for the reason of poor fine-tuning. In order to optimize network topology and weights more efficiently. It is possible to combine the advantages of them (CCA and GA), and avoid the disadvantages. For the reason, Genetic Algorithm Based Correlation-Construction (GABCC) was developed. The basic concept of GABCC is ''''adaptability''''. By adaptive GA

operators(selection, crossover, mutation), we can evaluate the population more efficiently. Adaptive GA operators have good abilities both on global search and local search. For the reason, it gets better performance than traditional gradient methods shown in the benchmarks we have tested. More over, GABCC combine both of the adaptive GA and correlation-constructed method. So that weights and topology of network can be optimized at the same time

Keywords : 神經網路 ; 權值空間 ; 架構空間 ; 基因演算法

Table of Contents 0

REFERENCES