Chapter 2. Genetic Algorithm Based Fuzzy ID3 Method
2.5 Genetic Algorithm for Fuzzy ID3
From the description above, ò , and the membership functions of all the continuous features of Fuzzy ID3 algorithm are defined by a user. A good selection of fuzzy rule base, ò , and the membership functions are best matched to the database to be processed, would greatly improve the accuracy of the decision tree. To this end, any optimization algorithms seem appropriate for this purpose. In particular, genetic algorithm (GA) based scheme is highly recommended since a gradient computation for conventional optimization approach is usually not feasible for a decision tree. This is because condition-based decision path is nonlinear in nature, and hence its gradient is not defined. Based on the concept, we will introduce genetic algorithm to search best , and the membership functions of all the continuous features for the design of fuzzy ID3.
r, òn
r, òn
òr, òn
GA is adaptive heuristic search method that may be used to solve all kinds of complex search and optimization problems. It is an optimization search mechanism based natural selection process. Its essential mind is to imitate the criterion “survival of the fittest” of the biology. GA is often capable of finding optimal solutions even in the most complex of search spaces or at least it offer significant benefits over other search and optimization techniques. A typical GA operates on a population of solutions within the search space. The search space represents all the possible solutions that can be obtained for the given problem and is usually very complex or even infinite. Every point of the search space is one of the possible solutions and therefore the aim of the GA is to find an optimal point or at least come as close to is as possible.
The advantage of GA is in their parallelism. GA is considering many individuals instead of an individual in a search space. It gets the global optimum rapidly, furthermore avoids the chance to fall into the local optimum.
GA is typically implemented as a computer simulation in which a population of chromosomes of individuals to an optimization problem evolves toward better solutions. Traditionally, solutions are represented in binary as strings of 0s and 1s, but different encodings are also possible. In this thesis, we use 6-bits to represent a parameter. We use GA to tune the thresholds ò , and the parameters of the membership functions of feature values. The membership function of each sub-attribute is assumed to be Gaussian-type and is given by
r, òn
m(x) = exp
( à
(x2ûàö)2 2)
, (2.8) where x is the corresponding feature value of the data point with mean ö and standard deviation . Thus for each membership function, we have two parameters û ö and û to tune. For example, assume we have a data set, which has 4 continuous attributes and 3 classes such that there are 12 membership functions. Each membership function has 2 parameters and there are 2 thresholds of leaf condition in addition. Thus we have 26 parameters to be tuned, and the length of a binary chromosome is 156. There are three operators of genetic algorithm which are reproduction, crossover, and mutation. We briefly describe how to perform these three operators. The flowchart of GA is shown in Fig. 2.4.Fig. 2.4. Flowchart of genetic algorithm.
Reproduction is a process according to the fitness degree of each individual to decide which will be eliminated or copied at next generation, the individual with higher fitness value will be copied in a large number; the individual with lower fitness value will be eliminated. The potential chromosomes of the population are copied into a mating pool depending on their fitness values. The operator is shown in Fig. 2.5.
Fig. 2.5. Reproduction.
To minimize the rule number and maximize the accuracy, let the fitness function ( )
( )
(
Ai Aworst Ravg Rif =100100 − 2+
)
, (2.9) where is the learning accuracy of the individual , and is the worst learning accuracy of all individuals. is the average number of the rules of all individuals, and is the number of the rules of the individual .Ai i Aworst
Ravg
Ri i
Crossover is a process with selecting two potential chromosomes randomly from the matching pool, and exchange bit information mutually to produce two new individuals. Roughly speaking, it hopes to generate greater filial generation by accumulating the superior bit information of parents. An example for crossover is shown as Fig. 2.6.
Fig. 2.6. Crossover.
Mutation is an occasional alteration of a random bit. It is the process that selects randomly string of an individual and selects randomly the mutation point to change the bit information of the string. The probability of this process is controlled by the mutation probability. For binary string, “0” is changed to “1,” and “1” is changed to
“0.” The mutation is shown as Fig. 2.7. Mutation that helps to find an optimal solution to the given problem more reliably, as it prevents GA from finishing the search prematurely with a sub-optimal solution.
Fig. 2.7. Mutation.
After the genetic algorithm above, the system will generate two parameters of Gaussian-type membership function ö and û, and we get the parameters 0.827, and 0.001 of the criterions for tree construction. The membership functions of each continuous attribute for the training set are illustrated in Fig. 2.8.
òr= òn=
(a)
(b)
Fig. 2.8. The membership functions of each attribute for the training set. (a) The membership functions of “temperature.” (b) The membership functions of “humidity.”