www.elsevier.com/locate/fss

## Integrating membership functions and fuzzy rule sets from multiple

## knowledge sources

### Ching-Hung Wang

a_{, Tzung-Pei Hong}

b;*∗, Shian-Shyong Tseng*

c
a_{Chunghwa Telecommunication Laboratories, Chung-Li, 32617, Taiwan, ROC}
b_{Department of Information Management, I-Shou University, Kaohsiung, 84008, Taiwan, ROC}
c_{Institute of Computer and Information Science, National Chiao-Tung University, Hsin-Chu, 30050, Taiwan, ROC}

Received June 1997; received in revised form November 1997

Abstract

In this paper, we propose a GA-based fuzzy knowledge-integration framework that can simultaneously integrate multiple
fuzzy rule sets and their membership function sets. The proposed two-phase approach includes fuzzy knowledge encoding
and fuzzy knowledge integration. In the encoding phase, each fuzzy rule set with its associated membership functions is
rst transformed into an intermediary representation, and further encoded as a string. The combined strings form an initial
knowledge population, which is then ready for integration. In the knowledge-integration phase, a genetic algorithm is used
to generate an optimal or nearly optimal set of fuzzy rules and membership functions from the initial knowledge population.
The hepatitis diagnostic problem was used to show the performance of the proposed knowledge-integration approach. Results
show that the fuzzy knowledge-base resulting from using our approach performs better than every individual knowledge
base. c*
2000 Elsevier Science B.V. All rights reserved.*

Keywords: Expert systems; Fuzzy machine learning; Fuzzy sets; Knowledge acquisition; Membership functions; Operators

1. Introduction

Expert systems have been successfully applied to many elds and have shown excellent performance. Knowledge-base construction remains, however, one of the major costs in building an expert system even though many tools have been developed to help with knowledge acquisition. Building a knowledge-based system usually entails constructing new knowledge bases from scratch. The cost of the eort is high and will become prohibitive as we attempt to build larger and larger systems. Reusing and integrating available

*∗*_{Corresponding author.}

E-mail address: tphong@csa500.isu.edu.tw (T.-P. Hong)

knowledge from a variety of sources, such as do-main experts, historical documentary evidence, cur-rent records, or existing knowledge bases, thus plays an important role in building eective knowledge-based systems [1, 10, 13, 19]. Especially for complex application problems, related domain knowledge is usually distributed among multiple sites, and no sin-gle site may have complete domain knowledge. The use of knowledge integrated from multiple knowledge sources is thus especially important to ensure compre-hensive coverage.

Many knowledge acquisition and integration systems [2, 9, 21] based on the Personal Constructs Psychology (PCP) model [15] or Integrity Con-straints [1, 19] have been developed. Recently, genetic

0165-0114/00/$ - see front matter c*
2000 Elsevier Science B.V. All rights reserved.*
PII: S 0 1 6 5 - 0 1 1 4 ( 9 7 ) 0 0 3 8 5 - 0

algorithms have also been used to derive knowledge from training instances [3, 6, 12]. In [28, 29], Wang et al. proposed a GA-based knowledge-integration strategy that automatically integrates multiple rule sets in a distributed-knowledge environment. A self-integrating knowledge-based brain-tumor diagnostic system that uses this method was also developed [25]. In this paper, we attempt to generalize it to fuzzy domains.

Most knowledge sources or actual instances in real-world applications contain fuzzy or ambiguous information. Especially in domains such as medical or control domains, the boundaries of a piece of infor-mation used may not be clearly dened. Expressions of the domain knowledge using fuzzy descriptions are thus seen more and more frequently. Several researchers have recently investigated automatic generation of fuzzy classication rules and fuzzy membership functions using evolutionary algorithms [3, 16, 18, 23]. These methods can be categorized into the following four types:

1. learning fuzzy membership functions with xed fuzzy rules [14];

2. learning fuzzy rules with xed fuzzy membership functions [23, 30];

3. learning fuzzy rules and membership functions in stages [16] (i.e., rst evolving good fuzzy rule sets using xed membership functions, then tuning membership functions using the derived fuzzy rule sets);

4. learning fuzzy rules and membership functions simultaneously [3, 18].

In this paper, we propose a GA-based fuzzy knowledge-integration framework that can eectively integrate multiple fuzzy knowledge sources into a single knowledge base. The hepatitis diagnostic prob-lem [4] was used to show the performance of the proposed knowledge-integration approach. Results show that the fuzzy knowledge base that results from using our approach performs better than every in-dividual knowledge base. Knowledge integration is thus a successful application of genetic algorithms.

The remainder of this paper is organized as follows. Some GA-based classier systems are reviewed in Section 2. A GA-based fuzzy knowledge-integration framework is proposed in Section 3. The fuzzy knowledge encoding approach used in the proposed framework is explained in Section 4. Our fuzzy

knowledge integration approach is proposed in Section 5. Experiments on the diagnosis of hepatitis are stated in Section 6. Conclusions and future work are given in Section 7.

2. Review of GA-based classier systems

In this section, we review two famous approaches commonly used by genetic algorithms as classier systems, the Michigan approach and the Pittsburgh approach.

2.1. The Michigan approach

Cognitive system one (CS-1) was the rst Michigan genetic classier system. It was devised by Holland and Reitman in 1983 [12]. CS-1 maintains a popula-tion in which each individual is a rule, and is encoded as a xed-length string. A tness function is dened to evaluate the goodness (called the strength) of each rule in the population. The genetic algorithm then op-erates on the level of individual rules and selects good parent rules for mating according to their strength values.

The major problem with the Michigan approach is the simultaneous cooperation and competition of the individual rules within the population. During evolu-tion, rules within the population compete with each other, and the ones with high strength values are se-lected for mating to generate new ospring rules. At the same time, rules within the population must co-operate to solve the given problem. Maintenance and evolution of such a set of co-adapted rules by consid-ering both factors mentioned above is thus vital to the success of the Michigan approach.

2.2. The Pittsburgh approach

Learning system one (LS-1) was the rst Pittsburgh genetic classier system. It was proposed by Smith in 1980 [22]. LS-1 maintains a population in which each individual is a rule set, and is encoded as a variable-length string. A tness function is dened to evaluate the goodness of each rule set in the population. The genetic algorithm then operates on the level of indi-vidual rule sets and selects good parent rule sets for mating according to their tness values.

Fig. 1. A GA-based fuzzy knowledge-integration framework.

The major problem with the Pittsburgh approach is the maintenance and evaluation of a population of rule sets. It often leads to a much greater computa-tional burden (in terms of both memory and process-ing time). Also, since credit assignment occurs on the level of rule sets by a predened evaluation func-tion, we may obtain only the tness value of rule sets. It cannot help us promote the performance of individ-ual rules. This is another problem of the Pittsburgh approach.

Application of genetic algorithms to the Pittsburgh approach is apparently quite dierent applying them to the Michigan approach. For the knowl-edge integration task, representation based on the Pittsburgh approach is preferred since knowledge at each dierent site is a rule set. In [28, 29], we pro-posed a GA-based knowledge-integration framework based on the Pittsburgh approach for integrating un-ambiguous knowledge. In this paper, we extend it to managing fuzzy knowledge integration.

3. A GA-based fuzzy knowledge-integration framework

Here, we propose a GA-based fuzzy knowledge-integration framework that integrates information

from various fuzzy knowledge sources into a single knowledge base. The proposed framework can inte-grate multiple fuzzy rule sets and membership func-tion sets at the same time. The proposed framework is shown in Fig. 1.

Fuzzy rule sets, membership functions, and test ob-jects including instances and historical records may be distributed among various sources. Knowledge from each site might be directly obtained by a group of human experts using knowledge-acquisition tools, or derived using machine-learning methods. Here, we as-sume that all knowledge sources are represented by fuzzy rules since almost all knowledge derived by knowledge-acquisition tools or induced by machine-learning methods may easily be translated into or rep-resented by rules.

The proposed framework maintains a population of fuzzy rule sets with their membership functions, and uses the genetic algorithm to automatically derive the resulting fuzzy knowledge base. It operates in two phases: fuzzy knowledge encoding and fuzzy knowl-edge integration. The encoding phase rst transforms each fuzzy rule set and its associated membership functions into an intermediary representation, which is further encoded as a variable-length string. The integration phase then chooses appropriate strings for “mating”, gradually creating good ospring fuzzy

Fig. 2. Genetic fuzzy knowledge integration.

rule sets and membership function sets. The ospring
fuzzy rule sets with their associated membership
functions then undergo recursive “evolution” until an
optimal or nearly optimal set of fuzzy rules and
mem-bership functions has been obtained. Fig. 2 shows
the two-phases process, where ˜_{R ˜}_{S}1+ MFS1; ˜R ˜S2+

MFS2; : : : ; ˜R ˜Sm+ MFSm are the fuzzy rule sets with

their associated membership function sets, as obtained from dierent sources for integration.

4. Fuzzy knowledge encoding

In order to apply GAs to integration of multiple fuzzy rule sets, we need a powerful description lan-guage to represent complex rule sets and to map them easily into string representations. One of the most pop-ular representation of rules, the conjunctive normal form, is then chosen as our description language to ex-press each fuzzy rule set. Since the fuzzy rule sets with their associated membership functions are obtained from dierent sources, they may dier in size. Repre-sentation of variable-length rule sets is thus preferred here. Each fuzzy rule set with its associated member-ship functions is encoded as a variable-length chro-mosome by the Pittsburgh approach. However, each fuzzy rule set must rst be translated into a uniform intermediary representation to preserve the syntactic and semantic constraints of the fuzzy rule sets before encoding. The steps for translating fuzzy rule sets into intermediary representations are described below. 1. Collect the features and possible values occurring

in the condition parts of the fuzzy rule sets. All fea-tures gathered together comprise the global feature set.

2. Collect classes occurring in the conclusion parts of the fuzzy rule sets. All classes gathered together comprise the global class set.

3. Translate each fuzzy rule into an intermediary rep-resentation that retains its essential syntax and se-mantics. If some features in the global feature set are not used by the fuzzy rule, dummy tests are inserted into the condition part of the fuzzy rule. Each rule in the intermediary representation is then composed of N feature tests and one class pattern, where N is the number of global features collected. 4. Concatenate all intermediary representations of

rules to form the representation of a rule set. An example is given below to demonstrate the translation process of forming intermediary represen-tations.

Example 1. Fisher’s Iris data [8] are used to demon-strate the translation process of forming interme-diary representations. There are three species of Iris Flowers to be distinguished: Setosa, Versicolor and Virginica. A class domain D ower is dened as

D
ower=*{Setosa; Versicolor; Virginica}. Each rule*

is described by four features: Sepal Length (S:L:), Sepal Width (S:W:), Petal Length (P:L:), and Petal Width (P:W:). Each feature has the possible linguistic values shown below.

DS:L:=*{Short; Medium; Long};*

DS:W:=*{Narrow; Medium; Wide};*

DP:L:=*{Short; Medium; Long};*

Assume a fuzzy rule set ˜_{R ˜}_{S}_{q}obtained from a fuzzy
knowledge source has the following four rules:

˜

rq1: IF (P:L: = Short); then Class is Setosa:

˜

rq2: IF (P:L: = Long); then Class is Virginica:

˜

rq3: IF (P:W: = Medium); then Class is Versicolor:

˜

rq4: IF (P:W: = Wide); then Class is Virginica:

After translation, the intermediary representations of these rules would then be constructed as follows:

˜

rq1*0* : IF (S:L: = Short or Medium or Long) and

(S:W : = Narrow or Medium or Wide) and (P:L: = Short) and

(P:W : = Narrow or Medium or Wide); then Class is Setosa:

˜

rq2*0* : IF (S:L: = Short or Medium or Long) and

(S:W : = Narrow or Medium or Wide) and (P:L: = Long) and

(P:W : = Narrow or Medium or Wide); then Class is Virginica:

˜

rq3*0* : IF (S:L: = Short or Medium or Long) and

(S:W : = Narrow or Medium or Wide) and (P:L: = Short or Medium or Long) and (P:W : = Medium); then Class is Versicolor: ˜

rq4*0* : IF (S:L: = Short or Medium or Long) and

(S:W : = Narrow or Medium or Wide) and
(P:L: = Short or Medium or Long) and
(P:W : = Wide); then Class is Virginica:
The tests with underlines are dummy tests.
Also, ˜_{r}_{qi}*0* is logically equivalent to ˜_{r}_{qi}_{, for i = 1; : : : ; 4.}
After translation, each intermediary representation of
the rule is then composed of four feature tests and
one class pattern.

Although the intermediary representation may in-clude irrelevant tests and increase search space dur-ing integration, it can easily map each intermediary rule into a xed-length string representation. The con-dition part of each intermediary rule is a conjunctive form with internal disjunctions that can describe com-plex rules. Irrelevant tests can also be removed by the knowledge decoding process after integration.

After each rule set has been translated into an in-termediary representation, an appropriate data struc-ture must then be designed to encode both the fuzzy

Fig. 3. Membership functions of feature Ai.

rule sets and their membership function sets. Several strategies for representing fuzzy knowledge structures in conceptual learning were proposed [3, 20]. Here, we represent each membership function with two pa-rameters as Parodi and Bonelli [20] did. Membership functions applied to a fuzzy rule set are assumed to be isosceles-triangle functions, as shown in Fig. 3, where cij is the abscissa with the highest membership value

of the jth linguistic value (aij) of feature Ai, and wij

represents half the spread of the membership function. A linguistic value aij of the feature Ai is then

repre-sented as a pair (cij; wij).

Assume that an intermediary rule ˜_{r}_{qk}*0* is composed
of N feature tests and one class pattern. Each
fea-ture test in a fuzzy rule is then encoded as mi pairs

of (c; w)’s, where mi is the number of possible

lin-guistic values of Ai, and the pair of (c; w) represents

one possible linguistic value. If a fuzzy test value “Ai= aij” exists in a rule, the test is then encoded

as _{(c}_{i1}_{; −w}_{i1}_{)(c}_{i2}_{; −w}_{i2}_{) : : : (c}_{ij}_{; w}_{ij}_{) : : : (c}_{i}

mi*; −w*imi).

Only wij is positive, the other w values have minus

signs added to them. Similarly, if the fuzzy test is (“Ai= aij” or “Ai= aik”), then only wij and wik are

positive and the other w values are negative. From the
sign of w, the encoded string can then correctly
repre-sent the test condition of a fuzzy rule. The condition
part of each fuzzy rule is then encoded as PN_{i=1}_{m}_{i}
pairs of (c; w)’s using the proposed encoding methods.
Next, the conclusion part of each fuzzy rule is
en-coded as a bit substring (’1*· · · ’*x), where x is the

number of possible classes. When the rule points to class j, then ’jis set as 1 and the others are set as 0.

The rule ˜_{r}_{qk} is then encoded as shown in Fig. 4.
In Fig. 4, the substring (cqk_{i1}w_{i1}qk*· · · c*qkijwqkij *· · ·*

cqkimiw

qk

imi) represents the membership functions of mi

Fig. 4. String representation of ˜r_{qk}*0*.

Fig. 5. String representation of fuzzy rule set ˜R ˜Sq.
the substring (’qk1 *· · · ’*

qk

x ) represents its output class.

Since cijand wijare both numeric values, fuzzy rules

and their fuzzy membership function sets are then
en-coded as xed-length real-number strings rather than
bit strings. Each fuzzy rule set ˜_{R ˜}_{S}_{q} _{that contains k}
fuzzy rules is then encoded by concatenating strings
of k fuzzy rules (Fig. 5).

An example is given below to demonstrate the en-coding of fuzzy knowledge.

Example 2. Continuing from Example 1, assume the
fuzzy rule set ˜_{R ˜}_{S}_{q} is to be encoded. Assume that the
membership functions used for each feature are as
shown in Fig. 6.

Using the proposed intermediary representation, the
fuzzy rule ˜_{r}_{q1}*0* in Example 1 is encoded as shown in
Fig. 7.

Since feature S:L: in ˜rq1*0* has three disjunctive test

values, Short; Medium and Long, the tests for S:L: are
then encoded as “5.2, 0.9, 6.1, 0.9, 7.0, 0.9” according
to the membership functions given in Fig. 6. S:W: also
has three disjunctive test values, Narrow, Medium and
Wide, and is then encoded as “2.6, 0.6, 3.2, 0.6, 3.8,
0.6”. Similarly, P:W: is encoded as “0.7, 0.6, 1.3, 0.6,
1.9, 0.6”. But P:L:, has only one test value, Short. It is
then encoded as “2.4, 1.5, 3.9,*−1.5, 5.4, −1.5”, where*
the two negative w values indicate “P:L: = Medium”
and “P:L: = Long” are not in the condition part of
rule ˜_{r}_{q1}*0* .

This representation allows genetic operators (de-ned later) to easily integrate multiple fuzzy rule sets and their fuzzy membership function sets at the same

time. Furthermore, since fuzzy membership functions are encoded together with each rule (as opposed to a global collection of membership functions for all rules), rules are permitted to evolve to dierent de-grees of vagueness as Carse et al. proposed [3]. The ad-vantage of this representation is the expressive power for the derived rules to possess their own specicity in terms of the fuzzy sets they relate to. Especially for multi-dimensional domains that cannot generally use a global collection of membership functions for all rules, the use of local fuzzy sets to perform a lin-guistic interpretation of individual rules seems valid to overcome “curse of dimensionality” [3]. However, this advantage is at the cost of an increase of the search space. A more detailed discussion about using local fuzzy sets can be found in [3, 5, 18].

5. Fuzzy knowledge integration

After each fuzzy rule set with its associated mem-bership functions has been encoded as a variable-length string (an individual in the initial population), the genetic-fuzzy knowledge-integration process starts. It chooses good individuals in the population for “mating”, gradually creating better ospring fuzzy rule sets. During evolution, a measure function and a set of test objects are used to evaluate the tness value of each “ospring” fuzzy rule set. The ospring fuzzy rule sets then undergo recursive “evolution” until a really good fuzzy knowledge base has been produced. Domain experts thus need not intervene in

Fig. 6. Membership functions for Example 2.

Fig. 7. String representation of ˜r_{q1}*0*.

the integration process. Notation and denitions used are given as below.

5.1. Notation and denitions

Denition 1. A fuzzy test ˜_{s}_{k}_{is represented as [A}_{k}_{r ],}
where Ak is a feature, r is a relationship, and is a

fuzzy linguistic value. For example “color = reddish” and “height = tall” are both fuzzy tests.

Denition 2. _{u}_{˜s}_{k}( ˜_{e) represents the degree to which}
object ˜_{e is matched by ˜s}_{k}_{. The value of u}_{˜s}_{k}( ˜_{e) ranges}
between 0 and 1; 0 indicates complete exclusion and
1 indicates complete inclusion.

Denition 3. Assume the condition part ˜_{c}_{j}of rule ˜_{r}_{j}
consists of jmtests, ˜sj1*∧ ˜s*j2*∧ · · · ∧ ˜s*jm. The degree of

object ˜_{e matched by ˜c}_{j}is evaluated as
uc˜j( ˜e) = u˜sj1( ˜*e) ∧ u*˜sj2( ˜*e) ∧ · · · ∧ u*˜sjm( ˜e);

or more generally,

uc˜j( ˜e) = u˜sj1( ˜e) u˜sj2( ˜*e) · · · u*˜sjm( ˜e);

where is a t-norm operator.

Denition 4. The classication of an object ˜e judged
by a rule ˜_{r}_{j} ( ˜_{c}_{j}*⇒ ˜*_{j}) is ˜_{}_{j}, with a membership value
uc˜j( ˜e).

Denition 5. The classication of an object ˜_{e judged}
by a rule set ˜_{R ˜}_{S is ˜}_{j}_{; if a rule concluding to}

˜

j ( ˜cj*⇒ ˜*j) has the highest uc˜j( ˜e) among all rules.

If ˜_{e is classied by ˜}_{R ˜}_{S into several classes that have}
the highest degree, the classication of an object ˜_{e is}
then shared among them.

Denition 6. An object is correctly matched by rule
set ˜_{R ˜}_{S if the original object class is equal to the class}
judged by ˜_{R ˜}_{S.}

5.2. Initial population

The proposed fuzzy knowledge-integration method uses a genetic algorithm for integration and optimiza-tion of fuzzy rule sets. The genetic algorithm requires a population of feasible solutions to be initialized and updated during the evolution process. In our approach, the initial population of fuzzy rule sets with their as-sociated membership functions comes from multiple knowledge sources. Each individual in the initial pop-ulation represents a fuzzy rule set with its associated

membership functions. If the initial number of knowl-edge sources is small, some dummy initial rule sets that are randomly generated or duplicated from orig-inal rule sets, are inserted into the population to in-crease the population size.

5.3. Fitness and selection

In order to develop a “good” fuzzy knowledge base
from the initial population, the genetic algorithm
se-lects parent fuzzy rule sets with high tness values
for mating. An evaluation function and a set of test
objects including instances or historical records, are
then used to qualify the derived fuzzy rule set. Rule
set performance is then fed back to the genetic
algo-rithm to control how the solution space is searched to
promote fuzzy rule set quality. Two important factors
are used in evaluating derived fuzzy rule sets, the
ac-curacy and the complexity of the resulting knowledge
structure. Accuracy of a fuzzy rule set ˜_{R ˜}_{S is evaluated}
using test objects as follows:

Accuracy( ˜_{R ˜}_{S)}

=total number of objects correctly matched by ˜R ˜S

total number of objects :

The more data used, the more objective and accurate
the evaluation is. The complexity of the resulting rule
set ( ˜_{R ˜}_{S) is the ratio of rule increase, dened as follows:}
Complexity( ˜_{R ˜}_{S)}

= Number of rules in the integrated rule set ˜R ˜S
[Pm_{i=1}(Number of rules in the initial ˜_{R ˜}_{S}_{i}_{)] =m};
where ˜_{R ˜}_{S}_{i}_{is the ith initial fuzzy rule set, and m is the}
number of initial rule sets. Accuracy and complexity
are combined to represent the tness value of the rule
set. The evaluation function is then dened as follows:
tness( ˜_{R ˜}_{S) =} [Accuracy( ˜R ˜S)]

[Complexity( ˜_{R ˜}_{S)]};

where is a control parameter, representing a trade-o between accuracy and complexity.

5.4. Genetic operators

Genetic operators are very important to success of specic GA applications. Two genetic operators,

crossover and mutation, are used in the genetic fuzzy knowledge-integration framework.

5.4.1. Crossover operator

The crossover operator used here selects crossover points dierently from the way used in the simple ge-netic algorithm. The crossover operator in the simple genetic algorithm chooses the same points for both parent chromosomes, but, the crossover operator used here need not use the same point positions for both parent chromosomes. The crossover points may oc-cur within rule strings or at rule boundaries. The only requirement for crossover points is that they must “match up semantically”. That means, if one parent is cut at a rule boundary, then the other parent must also be cut at a rule boundary. Similarly, if one parent is cut at a point p units to left of a rule boundary, then the other parent must also be cut at a point p units to the left of some other rule boundary. An example of crossover operation is shown below.

Example 3. Assume that parent rule sets ˜_{R ˜}_{S}1 and

˜

R ˜S2, contain, respectively, n and m rules for

classify-ing test objects with two lclassify-inguistic features (F1 and

F2). Features F1 and F2 both have two possible

lin-guistic values. Two classes are to be determined.
As-sume that ˜_{R ˜}_{S}1and ˜R ˜S2are encoded as shown in Fig. 8.

As mentioned above, the crossover points on both
parents must “match up semantically”. If crossover
point cp_{1} is the sixth unit to the left of ˜_{r}_{1i} in ˜_{R ˜}_{S}1

(denoted as cp_{1}_{= (1i; 6)), then crossover point cp}_{2}
for ˜_{R ˜}_{S}2must be the sixth unit to the left of a certain

rule ˜_{r}_{2j}(denoted as cp_{2}_{= (2j; 6)). Thus, the crossover}
operator generates two ospring rule sets, ˜_{O}1and ˜O2,

as shown in Fig. 9.

After ospring fuzzy rule sets have been generated using the crossover operation, the order of fuzzy mem-bership functions may be destroyed, and may need rearrangement according to their center values. An ex-ample is given below to demonstrate the rearrange-ment of membership functions.

Example 4. Assume that two fuzzy rule sets, ˜_{R ˜}_{S}1and

˜

R ˜S2, for the Iris Flower domain, are encoded as shown

in Fig. 10.

The crossover operator generates two ospring rule
sets, ˜_{O}1 and ˜O2. The pairs (2.6, 0.6)(2.5,*−0:4)(3.8,*

Fig. 8. String representation of ˜R ˜S1 and ˜R ˜S2.

Fig. 9. An example of the crossover operation.

Fig. 11. The mutation process for Example 5.

Fig. 12. String representation of ˜r_{qi}*0*.

rearranged in ascending order of membership function
centers to (2.5,*−0:4)(2.6, 0.6)(3.8,−0:9).*

5.4.2. Mutation operator

The mutation operator is used to create a new fuzzy membership function by adding a random value to the center or the spread of an existing fuzzy membership function, say f. Assume that c and w represent the center and the spread of f. The center or the spread of the new derived membership function will be changed to c + or w + by the mutation operation. Mutation at the center of a fuzzy member-ship function may, however, disrupt the order of the feature’s fuzzy membership functions. These fuzzy membership functions then need rearrangement ac-cording to their center values. An example is given below to demonstrate the rearrangement of member-ship functions.

Example 5. Continuing from Example 1, assume a center of the second membership function for feature S:W: in ˜R ˜S1 is chosen for mutation. Assume the

*ran-dom value is −0:7. The mutation process is shown*
in Fig. 11.

The mutation operator generates one ospring, rule
set ˜_{O}1, from ˜R ˜S1, by adding the to the center of

mem-bership function (3.2, *−0:6). The new membership*
function formed is (2.5,*−0:6). The pairs (2.6, 0.6),*
(2.5,*−0:4) and (3.8, −0:9) for S:W: in ˜O*1 are then,

however, out of sequence. They must be rearranged in
ascending order of membership function centers; the
nal result is (2.5,*−0:4), (2.6, 0.6), and (3.8, −0:9).*
As mentioned above, the fuzzy knowledge
integra-tion phase chooses fuzzy rule sets with their associated
membership functions for “mating”, gradually
creat-ing good osprcreat-ing. After the termination criteria are
satised, the best ospring is decoded into the form
of rules. If a fuzzy test in the nal ospring is
repre-sented as (ci1; wi1)(ci2; wi2)*· · · (c*ij; wij)*· · · (c*i_{mi}; wi_{mi})

and each w is positive, the feature is irrelevant for the rule and is removed from the rule condition part. The rule can thus be succinctly interpreted. An example is given below to demonstrate the decoding of the nal ospring.

Example 6. Continuing from Example 1, assume the
fuzzy-rule string ˜_{r}_{qi}*0* (shown in Fig. 12) is to be decoded
into the form of rules.

Fig. 13. Membership functions for P:L: in Example 6.

Since the tests for feature S:L: in ˜rqi*0* are represented

as “5.3, 0.9, 6.2, 0.6, 7.1, 0.8” and their w values are all
positive, feature S:L: is thus irrelevant and is removed
from the condition part of ˜_{r}_{qi}*0*. Similarly, the features
S:W: and P:W: are also removed from the condition
part of ˜_{r}_{qi}*0*_{. The tests in P:L: are represented as “2.5,}
1.3, 3.8, 1.4, 5.5,*−1:5”. Since the last w value (−1:5)*
in the tests of P:L: is negative, the feature P:L: cannot
be thought of an irrelevant one. The tests for P:L:
are then decoded as “P:L: = Short or Medium” and
their associated membership functions are decoded as
shown in Fig. 13.

After decoding, the fuzzy-rule string ˜_{r}_{qi}*0* is thus
rep-resented as below.

IF (P:L: = Short or Medium); then Class is Setosa:

6. Experimental results

The hepatitis diagnostic problem [4] was used as the problem domain to test the performance of the proposed fuzzy knowledge-integration approach. The 155 cases used in these experiments were obtained from Carnegie–Mellon University [4]. The goal of the experiments was to identify two possible classes, Die or Live. Table 1 shows an actual case expressed in term of 19 features and one class.

The 155 cases were rst divided into two groups, a training set and a test set. The training set was used to evaluate the tness of rule sets during the integration process; the test set acted as input events to test the derived rule set, and the percentage of correct predic-tions was recorded. In each run, 70% of the hepatitis cases were selected at random for training, and the remaining 30% of the cases were used for testing. Ten initial rule sets were obtained from dierent groups of experts or derived via machine-learning [24, 26, 27].

Table 1

A case for hepatitis diagnosis

Features Feature values

Age 34 Sex male Steroid high Antivirals high Fatigue high Malaise high Anorexia high

Liver BIG yes

Liver Firm yes

Spleen Palpable yes

Spiders yes Ascites high Varices high Bilirubin 0.90 Alk Phosphate 95 SGOT 28 Albumin 4.0 Protime 75 Histology no Class: Live Table 2

The accuracy of the ten initial fuzzy rule sets

Rule Sets Accuracy (%) No. of rules

1 80.0 3 2 79.1 4 3 73.4 3 4 78.7 4 5 74.6 3 6 74.8 4 7 73.3 3 8 74.3 4 9 80.4 4 10 80.2 4

Table 3

Experimental results for the hepatitis diagnostic problem

Generation CPU time (s) Accuracy Fitness values

0 1 76.88 0.7537 13 4 78.44 0.7690 69 19 81.32 0.7972 124 35 83.28 0.8164 181 52 84.32 0.8266 261 75 85.25 0.8357 414 115 86.88 0.8517 1401 427 88.76 0.5701 2110 560 89.49 0.8773 3110 822 89.65 0.8789 3550 935 89.77 0.8800 3817 1005 91.83 0.9002 4000 1056 92.90 0.9107

The accuracy of the ten initial rule sets was measured using the test instances. The results are shown in Table 2.

Although the ten initial fuzzy rule sets were not accurate enough, they could however act as a set of locally-optimal solutions that provide useful in-formation in the search space. Beginning with these fuzzy rule sets, the genetic algorithm could then reach the (nearly) global optimal solution more rapidly than if it had nothing to refer to. Of course, each initial rule set could rst have been improved by the same GA scheme before integration. We do not however favor this alternative since it can-not use information from the other rule sets, and it would take much time to get good results. Sim-ilarly, we could also have abandoned these initial rule sets and directly applied the genetic algo-rithm to acquire knowledge from training instances. But the same disadvantages would still have been present.

In the experiments, the operation frequency for crossover and mutation was set at 0.9 and 0.04, respectively. Table 3 shows the results for dier-ent generations as to accuracy, integration time, and tness values.

Experimental results also show that executing the proposed approach in more generations yields more accurate results although the spent time increases. Fig. 14 shows the relationship between generations and tness values of resulting rule sets for the pro-posed approach.

Fig. 14. Relationship between tness values and generations. Table 4

A comparison with other learning methods

Methods Accuracy (%)

Our approach 92.9

Assistant-86 [5] 83

Diaconis and Efron’s [8] 80

As the numbers of generations were increased, the resulting tness values also increased, and nally con-verged to a specic value.

The accuracy of some other learning algorithms on the Hepatitis classication problem was examined in [4, 7]. The methods studied were Diaconis and Efron’s statistic method [7] and that of Cestnik et al. Assistant-86 [4]. Table 4 compares the accuracy of our proposed

approach with that of the other learning methods. It can easily be seen that the accuracy of our approach is higher than those of the other learning methods. 7. Conclusions

In this paper, we have shown how fuzzy knowledge-integration can be eectively processed using a genetic algorithm. Experimental results have also shown that our genetic fuzzy knowledge-integration framework is valuable for simultaneously combining multiple fuzzy rule sets and membership function sets. Our approach needs no human experts’ intervention during the inte-gration process. The time required by our approach is thus dependent on computer execution speed, but not on human experts. Much time can thus be saved since experts may be geographically dispersed, and their discussions are usually very time-consuming [11, 17]. Also, our approach is a scalable integration method, that can be applied as well when the number of rule sets to be integrated increases. Integrating a large num-ber of rule sets may increase the validity of the result-ing knowledge base. It is also objective since human experts are not involved in the integration process.

Although the work presented here shows good re-sults, it is only a beginning. Some future investigations are proposed below.

1. Several issues in the eld of knowledge verica-tion remain unresolved. Eectively dealing with knowledge verication issues is another interesting topic.

2. Each derived rule supplied to the classier sys-tem has a dierent degree of vagueness. Design of a new genetic fuzzy knowledge-integration framework that permits rules evolve over a global collection of membership functions is our current research project.

Acknowledgements

The authors would like to thank the anonymous referees for their very constructive comments.

References

[1] C. Baral, S. Kraus, J. Minker, Combining multiple knowledge bases, IEEE Trans. Knowledge and Data Eng. 3 (2) (1991) 208–220.

[2] J.H. Boose, Rapid acquisition and combination of knowledge from multiple experts in the same domain, Future Comput. Systems 1 (1986) 191–216.

[3] B. Carse, T.C. Fogarty, A. Munro, Evolving fuzzy rule based controllers using genetic algorithms, Fuzzy Sets and Systems 80 (1996) 273 –293.

[4] G. Cestnik, I. Konenenko, I. Bratko, Assistant-86: a knowledge-elicitation tool for sophisticated users, in: Bratko, Lavrac (Eds.), Machine Learning, Sigma Press, 1987, pp. 31– 45.

[5] M.G. Cooper, J.J. Vidal, Genetic design of fuzzy controllers: the cart and jointed pole problem, Proc. 3rd IEEE Internat. Conf. Fuzzy Systems IEEE, Piscataway, NJ, 1994, pp. 1332–1337.

[6] K.A. DeJong, W.M. Spears, D.F. Gordon, Using genetic algorithms for concept learning, Machine Learning 13 (1993) 161–188.

[7] P. Diaconis, B. Efron, Computer-intensive methods in stat-istics, Scientic American 248 (1983).

[8] R.A. Fisher, The use of multiple measurements in taxonomic problems, Annual Eugenics 7 (1936) 179–188.

[9] B.R. Gaines, M.L.G. Shaw, Eliciting knowledge and transferring it eectively to a knowledge-based system, IEEE Trans. Knowledge Data Eng. 5 (1) (1993) 4 –14.

[10] B.J. Gragun, H.J. Studel, A decision-table based processor for checking completeness and consistency in rule-base expert-systems, Int. J. Man–Machine Studies 26 (1987) 633– 648. [11] D.M. Hamilton, Knowledge acquisition for multiple site,

related domain expert systems: Delphi process and application, Expert Systems with Applications 11 (3) (1996) 377– 389.

[12] J.H. Holland, J.S. Reitman, Cognitive systems based on adaptive algorithms, in: D.A. Waterman, F. Hayes-Roth (Eds.), Pattern-directed Inference Systems, Academic Press, New York, 1978.

[13] G.J. Hwang, Knowledge elicitation and integration from multiple experts, J. Inform. Sci. Eng. 10 (1) (1994) 99–109. [14] C. Karr, Design of an adaptive fuzzy logic controller using a genetic algorithm, Proc. 4th Int. Conf. Genetic Algorithms, 1991, pp. 450 – 457.

[15] G.A. Kelly, The psychology of personal constructs, Norton, New York, 1995.

[16] J. Kinzel, F. Klawonn, R. Kruse, Modications of genetic algorithms for designing and optimising fuzzy controllers, Proc. 1st IEEE Int. Conf. Evolutionary Computation, IEEE, Piscataway, NJ, 1994, pp. 28 – 33.

[17] A.J. LaSalle, L.R. Medsker, Computerized conferencing for knowledge acquisition from multiple experts, Expert Systems With Applications 3 (1991) 517– 522.

[18] M. Lee, H. Takagi, Integrating design stages of fuzzy systems using genetic algorithms, Proc. 2nd IEEE Int. Conf. Fuzzy Systems, IEEE, San Francisco, 1993, pp. 612– 617. [19] O.K. Ngwenyama, N. Bryson, A formal method for analyzing

and integrating the rule sets of multiple experts, Inform. Systems 17 (1) (1992) 1–16.

[20] A. Parodi, P. Bonelli, A new approach of fuzzy classier systems, Proc. 5th Int. Conf. Genetic Algorithms, Morgan Kaufmann, Los Altos, CA, 1993, pp. 223 –230.

[21] M.L. Shaw, B.R. Gaines, KITTEN: Knowledge initiation and transfer tools for experts and novices, Int. J. Man–Machine Studies 27 (1987) 251–280.

[22] S.F. Smith, A learning system based on genetic adaptive algorithms, Ph.D. Thesis, University of Pittsburgh, 1980. [23] P. Thrift, Fuzzy logic synthesis with genetic algorithms,

Proc. 4th Int. Conf. Genetic Algorithms, Morgan Kaufmann, Los Altos, CA, 1991, pp. 509– 513.

[24] C.J. Tsai, C.H. Wang, T.P. Hong, S.S. Tseng, An inductive learning strategy with fuzzy sets, Proc. Int. Conf. Articial Intelligence, 1996, pp. 63 – 69.

[25] C.H. Wang, T.P. Hong, S.S. Tseng, Self-integrating knowledge-based brain tumor diagnostic system, Expert Systems With Applications 11 (3) (1996) 351– 360.

[26] C.H. Wang, T.P. Hong, S.S. Tseng, Inductive learning fuzzy examples, Proc. 5th IEEE Int. Conf. Fuzzy Systems, IEEE, San Francisco, 1996, pp. 13 –18.

[27] C.H. Wang, J.F. Liu, T.P. Hong, S.S. Tseng, A fuzzy inductive learning strategy for modular rules, Fuzzy Sets and Systems, accepted.

[28] C.H. Wang, T.P. Hong, S.S. Tseng, C.M. Liao, Automatically integrating multiple rule sets in a distributed-knowledge environment, IEEE Trans. Systems Man Cybernet., accepted. [29] C.H. Wang, T.P. Hong, S.S. Tseng, Knowledge integration by genetic algorithms, 7th Int. Fuzzy Systems Association World Congress, 1997, pp. 404–408.

[30] Y. Yuan, H. Zhuang, A genetic algorithm for generating fuzzy classication rules, Fuzzy Sets and Systems 84 (1996) 1–19.