A New Method for Constructing Membership
Functions and Fuzzy Rules from Training Examples
Tzu-Ping Wu and Shyi-Ming Chen,
Senior Member, IEEEAbstract—To extract knowledge from a set of numerical data
and build up a rule-based system is an important research topic in knowledge acquisition and expert systems. In recent years, many fuzzy systems that automatically generate fuzzy rules from numerical data have been proposed. In this paper, we propose a new fuzzy learning algorithm based on the -cuts of equivalence relations and the -cuts of fuzzy sets to construct the membership functions of the input variables and the output variables of fuzzy rules and to induce the fuzzy rules from the numerical training data set. Based on the proposed fuzzy learning algorithm, we also implemented a program on a Pentium PC using the MATLAB development tool to deal with the Iris data classification problem. The experimental results show that the proposed fuzzy learning algorithm has a higher average classification ratio and can generate fewer rules than the existing algorithm.
Index Terms— Fuzzy learning algorithms, fuzzy rules,
knowl-edge acquisition, membership functions, rule-based systems.
I. INTRODUCTION
I
N RECENT years, expert systems have become more and more popular and important in many applications. The purpose of expert systems is to emulate the reasoning process of human experts within a specific domain of knowledge. An expert system consists of a knowledge base, a user interface, and an inference engine [8]. Knowledge engineering plays an important role in the research field of expert systems. Knowl-edge engineering involves knowlKnowl-edge acquisition, knowlKnowl-edge representation, and human–machine interaction. The purpose of knowledge acquisition is to extract knowledge from the opinions of experts or a set of numerical data. It is difficult to build up a conventional mathematical model to elicit knowledge from experts or the sample data in the real-world application due to the fact that it needs to precisely describe all the characteristics of the system and it lacks flexibility. Fuzzy set theory proposed by Zadeh [38] can deal with the vagueness and uncertainty residing in the knowledge possessed by human beings or implicated in the numerical data, and it allows us to represent the system parameters with linguistic terms. Fuzzy rules have been used as a key tool to express knowledge in fuzzy logic and are more adequate and flexible than the traditional IF–THEN rules. In [6], Dubois et al. have surveyedManuscript received November 19, 1997; revised July 15, 1998. This work was supported in part by the National Science Council, Republic of China, under Grant NSC 87-2213-E-009-011.
T.-P. Wu is with the Department of Computer and Information Science, National Chiao Tung University, Hsinchu, Taiwan, R.O.C.
S.-M. Chen is with the Department of Electronic Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan, R.O.C.
Publisher Item Identifier S 1083-4419(99)00768-2.
different possible semantics for fuzzy rules and showed how they can be captured in the framework of fuzzy sets and possibility theory.
There are different approaches to extract knowledge from experts or training examples. These methods are based on neural networks or the fuzzy set theory [9]–[13], [15], [16], [20], [21], [26], [29], [32]. In [32], Wang et al. used a table-lookup scheme to generate fuzzy rules directly from numerical examples and proved that a fuzzy inference system is a universal approximator by the Stone–Weierstrass theorem [33]. In [26], Nozaki et al. presented a heuristic method for generating Takagi–Sugeno–Kang (TSK) fuzzy rules from numerical data, and then translated the consequent parts of TSK fuzzy rules into linguistic representation. In [9], Grauel et al. have investigated the connection between the shape of transfer functions and the shape of membership functions, where membership functions for multi-input of Sugeno con-trollers and designed rules were derived. In [16], Klawonn et al. discussed how fuzzy clustering techniques could be applied to construct a fuzzy controller from the training data. In [11], Hong and Lee have pointed out that the drawbacks of most fuzzy controllers and fuzzy expert systems are that they need to predefine membership functions and fuzzy rules to map numerical data into linguistic terms and to make fuzzy reasoning work. They proposed a method based on the fuzzy clustering technique and the decision tables to derive membership functions and fuzzy rules from numerical data. However, Hong and Lee’s algorithm presented in [11] needs to predefine the membership functions of the input linguistic variables and it simplifies fuzzy rules by a series of merge operations. As the number of variables becomes larger, the decision table will grow tremendously and the process of the rule simplification based on the decision tables becomes more complicated. Thus, we must develop a new algorithm to overcome the drawbacks of the Hong and Lee’s algorithm.
In this paper, we propose a new fuzzy learning algorithm based on the -cuts of fuzzy equivalence relations and the -cuts of fuzzy sets to divide numerical data into different partitions and to automatically derive membership functions for each partition. Based on the hierarchical relationships between different data partitions, the membership functions of the input linguistic variables and the output linguistic variables can be constructed automatically and the fuzzy rules can be generated directly.
The rest of this paper is organized as follows. In Section II, basic concepts and definitions of the fuzzy set theory are reviewed from [8], [17], [22], [36], and [38]. In Section III, we
Fig. 1. The triangular fuzzy setA.
introduce how to use the -cuts of the fuzzy equivalence rela-tion to partirela-tion the numerical data and to derive membership functions of variables from them. And then we can directly induce fuzzy rules from the data partitions. In Section IV, we use an example to illustrate our new algorithm and the experimental results are shown. In Section V, we apply the proposed fuzzy learning algorithm to the Iris classification problem and compare the experimental results to that of Hong and Lee’s algorithm [11]. The conclusions are discussed in Section VI.
II. BASIC CONCEPTS OF FUZZY SETS
The theory of fuzzy sets was proposed by Zadeh in 1965 [38]. A fuzzy set can be characterized by a membership function. Let be the universe of
discourse, . A fuzzy set in
the universe of discourse is a set of ordered pairs , where is the membership function of the fuzzy set , , and indicates the membership degree of in the fuzzy set . If the universe of discourse is an infinite set, then the fuzzy set can be represented by
The triangular fuzzy set shown in Fig. 1 can be rep-resented by a triplet , where and are called the left vertex and right vertex of the triangular fuzzy set , respectively.
The -cut of a fuzzy set in the universe of discourse is defined as follows:
Based on the definition of -cuts, the fuzzy set can be represented as follows:
A fuzzy relation among fuzzy sets is a subset of the Cartesian product and is
denoted by , where
and . The
mem-bership function of the fuzzy relation
is represented by , where and
. Let and be two binary fuzzy
relations with a common fuzzy set . The composition of
and is denoted by and is defined as follows:
where for all .
Let be a fuzzy set of the universe of discourse . A binary fuzzy relation is reflexive if and only
if . A fuzzy relation is
symmetric if and only if
. A fuzzy relation is transitive if and only if . The transitive closure of a binary fuzzy relation
is denoted by and defined as follows:
where and .
If a binary fuzzy relation is reflexive and symmetric, then the binary fuzzy relation is a compatibility relation. If a binary fuzzy relation is reflexive, symmetric, and transitive, then the binary fuzzy relation is called a fuzzy equivalence relation. Assume that is a fuzzy set, then, the -cuts of a binary fuzzy relation can be defined as follows:
where . Thus, every binary fuzzy relation can be represented in terms of its -cut i.e.,
Furthermore, it can be easily shown that a fuzzy equiva-lence relation can be considered to effectively group elements into equivalence classes whose members are similar to each other in some specified degree by taking an -cut , where . Each of these equivalence classes forms a partition of . Two elements of belong to the same block of partition if and only if .
III. A NEWFUZZY LEARNINGALGORITHM
In an -input-single-output fuzzy system, the fuzzy rules have the following general format:
is is
is is
where the variables appearing in the
antecedent parts of the fuzzy rules are called the input linguistic variables, the variable in the consequent part of the fuzzy rule is called the output linguistic variable, the fuzzy sets are called the input fuzzy sets of the input linguistic variable of the fuzzy rule , and the fuzzy set is called the output fuzzy set of the output linguistic variable of the fuzzy rule .
In many real-world applications, the numerical data are easily obtained from the instruments or the environment. Thus, the fuzzy rules can be learned from these numerical data. Assume that there are input–output pairs given as the
numerical training data set of the -input-single-output fuzzy system
(1) where is the value of the th input linguistic variable
of the th input–output pair , and is the value of the output linguistic variable of the th
input–output pair . The value of the
input linguistic variable is called the input value, and the value of the output linguistic variable is called the output value, where and . The input-value set of the input linguistic variable consists of some input values of the input linguistic variable in the input–output pairs of the training data set , where . The output-value set of the output linguistic variable consists of some output values of the output linguistic variable in the input–output pairs of the training data set .
For simplicity, we only consider to construct the multiple-inputs-single-output (MISO) fuzzy model. Since the outputs of the MISO fuzzy model are independent, the general rule structure of the MISO fuzzy model can easily be represented as a collection of the rules of the multiple-inputs-multiple-outputs (MIMO) fuzzy models [22].
As discussed in the previous section, a fuzzy equivalence relation is reflexive, symmetric, and transitive, and it can divide the crisp data into different groups by its -cuts. Instead of finding the fuzzy equivalence relation directly, we can determine a fuzzy compatibility relation (reflexive and symmetric) in terms of an appropriate distance function applied to the given data. Then, a fuzzy equivalence relation can be obtained by the max–min transitive closure of the fuzzy compatibility relation. Before constructing the fuzzy equivalence relation of the training data set , we first use the output values of the output linguistic variable as the key to sort the training data set in an ascending order [11]. Then, the sorted training data set can be obtained as follows:
and (2)
where . We only consider the
output values of the output linguistic variable in the sorted training data set . The fuzzy compatibility relation between the output values and of the output linguistic variable in the sorted training data set can be defined in terms of the Euclidean distance [17] as follows:
(3)
where and are the output values of the output linguistic variable in the sorted training data set , where
and , and is a constant which ensures the compatibility relation . That is, if , then we let . In this paper,
we calculate the value of as follows:
(4) where is the maximum value of the output linguistic variable in the sorted training data set . In general, the relation defined in (3) is a fuzzy compatibility relation, but it is not necessarily a fuzzy equivalence relation. The fuzzy equivalence relation between the output values and of the output linguistic variable in the sorted training data set can be obtained from the max–min transitive closure of the compatibility relation
between the output values and of the output linguistic variable in the sorted training data set .
After the fuzzy equivalence relation between the output values of the output linguistic variable in the sorted training data set has been defined, we can divide the sorted training data set into different partitions based on the -cuts of the fuzzy equivalence relation . Assume that we partition the sorted training data set into
different subsets , and the th subset of the sorted training data can be represented as follows:
and (5)
where is the threshold value that is chosen to divide the sorted training data set adequately, and is the number of subsets obtained from the sorted training data set .
Assume that the th output-value set of the output linguistic variable and the th input-value set of the input linguistic variable are obtained from the th subset
of the sorted training data set , then
(6)
(7) Thus, we can construct the membership functions of the output fuzzy sets of the output linguistic variable from the output values of the output-value set , where . Since the output values in the sorted training set have been divided into different output-value sets
based on the -cuts of the equivalence relation, each output-value set of the output linguistic variable can be thought of as the -cut of the output fuzzy set of the output linguistic variable . That is
and
(8) where is the membership function of the output fuzzy set of the output linguistic variable , and is the th output-value set of the output linguistic variable .
Because the output values of the -cut of the output linguistic variable are permuted in ascending order (i.e., if ), we can simply calculate the average of the minimum and the maximum output values of the -cut set and define it as the center of the output fuzzy set of the output linguistic variable . That is
(9) where is the center of the output fuzzy set ; and are the minimum and the maximum elements of the -cut , respectively. The membership grades of the minimum and maximum elements of the -cut are set to the threshold value , where . According to these three
points , and , we can calculate the
left vertex and the right vertex of the output fuzzy set of the output linguistic variable by interpolation as follows:
(10) (11) where is obtained using (9), and are the minimum and the maximum elements of the -cut , respectively. Thus, the membership function of the output fuzzy set of the output linguistic variable can be represented by the triplet as follows:
if ,
if ,
otherwise.
(12)
For the input linguistic variable , its corresponding input values in the sorted training data set also have concurrently been divided into input-value sets based on the -cuts of the equivalence relation between the output values of the output linguistic variable in the sorted training data set . In the th input-value set of the input linguistic variable , we also sort the input values in the ascending order. The fuzzy equivalence relation between the input values of the th input-value set of the input linguistic variable can be constructed based on the similarity between the input values. We also define the fuzzy compatibility relation between the input values of the input-value set of the input linguistic variable in terms of the Euclidean distance [17] (13) where
, and . denotes the number
of the elements of the input-value set . The value of is calculated as follows:
(14)
where denotes the number of the elements of the input-value set of the input linguistic variable is the maximum value of the input linguistic variable in the input-value set , and . By the max–min transitive closure of the compatibility relation , we can obtain the fuzzy equivalence relation
between the input values of the input-value set of the input linguistic variable . Furthermore, based on the -cuts of the fuzzy equivalence relation, the th input-value set of the input linguistic variable can be divided into different “input-value subsets” of the th input-value set of the input linguistic variable as follows:
and (15)
where is the number of the input-value subsets ob-tained from the th input-value set of the input linguistic variable based on the -cuts of the fuzzy equivalence relation between the input values of the th input-value set of the input linguistic variable . denotes the number of the elements of the input-value set .
Every input-value subset of the input linguistic vari-able can also be thought of as the -cut of the corresponding input fuzzy set of the input linguistic variable . Then, the -cut of the input fuzzy set of the input linguistic variable is defined as follows:
(16) where is the membership function of the input fuzzy set of the input linguistic variable is the threshold value, and . The input values in the -cut are also permuted in ascending order (i.e., , if ). The membership function of the input fuzzy set can be represented
by a triplet , where , and
are the center, the left vertex, and the right vertex of the input fuzzy set of the input linguistic variable , respectively. Then, the triplet can be calculated by the minimum and the maximum elements of the -cut of the input fuzzy set and by interpolation as follows:
(17) (18) (19) where and are the minimum and the maxi-mum elements of the -cut of the input fuzzy set
of the input linguistic variable
, and is the number of the input-value subsets obtained from the th input-value set of the input linguistic variable . Furthermore, the membership function of the input fuzzy set of the
input linguistic variable can be represented by the triplet as follows: if , if , otherwise, (20)
where and , and are the center, the left vertex, and the right vertex of the input fuzzy set of the input linguistic variable , respectively,
, and is the number of the input-value subset obtained from the th input-value set of the input linguistic variable .
Based on (3)–(20), we can partition the input–output pairs of the sorted training data set into the different input-value subset of the input linguistic variable and the output-value set of the output linguistic variable , where
, and . We also
can derive the membership function of the input fuzzy set of the input linguistic variable and the membership function of the output fuzzy set of the output linguistic variable from the data residing in the input-value subset of the input linguistic variable and the output-value set of the output linguistic variable
, respectively.
After deriving the membership functions of the input lin-guistic variables and the output linlin-guistic variables, we con-tinue to generate the fuzzy rules based on the hierarchical relationships between the input-value subsets of the input linguistic variables and the output-value sets of the output linguistic variables. In the following, we take a system with two input linguistic variables , and one output lin-guistic variable as an example to illustrate the hierarchical relationships between the output-value sets and the input-value subsets.
First, the th output-value set of the output linguistic variable and the th input-value set of the input linguistic variable are obtained based on the -cuts of the equivalence relation between the output values of the output linguistic variable in the training data set , where , and is the number of subsets obtained from the sorted training data set . Furthermore, the th input-value subset of the input linguistic variable can be obtained based on the -cuts of the equivalence relation between the input values of the th input-value set of the input linguistic variable , where
, and is the number of input-value subsets obtained from the th input-value set of the input linguistic variable .
There exists the hierarchical relationship between the th output-value set of the output linguistic variable and the th input-value subsets of the input linguistic variable
, where and . Assume that
Fig. 2. The relationship between the output-value set Oj of the output linguistic variableY and the input-value subset I1; j; kof the input linguistic variableX1.
Fig. 3. The relationship between the output fuzzy set Aj of the output linguistic variableY and the input fuzzy set A1; j; kof the input linguistic variableX1.
the input–output pair belongs to the sorted training data set , where . If the input value of the input linguistic variable belongs to the input-value subset , then we can infer that the corresponding output value of the output linguistic variable should belong to the output-value set . In Fig. 2, we can see the hierarchical relationship between the th output-value sets of the output linguistic variable and the th input-value subset of the input linguistic variable . Hence, we can obtain the hierarchical relationship between the th output fuzzy set of the output linguistic variable and the th input fuzzy set of the input linguistic variable based on the hierarchical relationship between the th output-value set of the output linguistic variable and the th input-value subset of the input linguistic variable . The hierarchical relationship between the th output fuzzy set of the output linguistic variable and the th input fuzzy set
of the input linguistic variable is shown in Fig. 3,
where and .
From Fig. 3, the fuzzy rules can be generated based on the hierarchical relationship between the output fuzzy set of the output linguistic variable and the input fuzzy set
of the input linguistic variable as follows: is is is is is is .. . .. . .. . .. . .. . is is is is is is (21)
By the same way, we can also obtain the hierarchical relationship between the th output-value set of the output linguistic variable and the th input-value subset of the input linguistic variable as shown in Fig. 4, where and . The hierarchical relationship between the th output fuzzy set of the output linguistic variable and the th input fuzzy set of the input linguistic variable is shown in Fig. 5, where
and . From Fig. 5, the fuzzy rules can be generated based on the hierarchical relationship between the output fuzzy set of the output linguistic variable and the input fuzzy sets of the input linguistic variable as follows: is is is is is is .. . .. . .. . .. . .. . is is is is is is (22)
In fact, we can see that the fuzzy rules whose antecedent part is “ is ” in (21) and the fuzzy rules whose antecedent part is “ is ” in (22) share the same consequent part “ is .” For a training input–output pair of the sorted training data set , if the input value belongs to the input-value set of the input linguistic variable and if the input value belongs to the input-value set of the input linguistic variable , then the output value belongs to the output-value set of
Fig. 4. The relationship between the output-value set Oj of the output linguistic variableY and the input-value subset I2; j; kof the input linguistic variableX2.
Fig. 5. The relationship between the output fuzzy set Aj of the output linguistic variableY and the input fuzzy set A2; j; kof the input linguistic variableX2.
the output linguistic variable . Thus, if the input–output pair satisfies both the antecedent parts of “ is
” and “ is ” for some and
some , then we can infer that the consequent part of the fuzzy rule is “ is ,” where , is the number of the input-value subsets obtained from the th input-value set of the input linguistic variable , where
.
The hierarchical relationship between the output fuzzy set of the output linguistic variable , the input fuzzy set of the input linguistic variable , and the input fuzzy set of the input linguistic variable can be obtained as shown in Fig. 6, where and
.
The fuzzy rules in (21) and (22) can be combined based on the hierarchical relationship between the output fuzzy set
of output linguistic variable , the input fuzzy set of the input linguistic variable , and the input fuzzy set
of the input linguistic variable as follows:
is is is
(23)
where , and .
Finally, we can generate fuzzy rules from the given numerical training data set. In general, if
Fig. 6. Relationship between the output fuzzy set Aj of the output linguistic variableY , the input fuzzy set A1; j; k of the input linguistic variable X1 and the input fuzzy set A2; j; k of the input linguistic variable X2.
there are input linguistic variables and a single output linguistic variable in a fuzzy system, then the total number of fuzzy rules generated from the training data set based on the proposed fuzzy learning algorithm is
(24)
where is the number of output fuzzy sets of the output linguistic variable , and is the number of input fuzzy sets of the input linguistic variable .
However, it is necessary to check whether some of the generated fuzzy rules are unnecessary or some input fuzzy sets of the input linguistic variables in the antecedent parts of the fuzzy rules are equivalent or redundant. If there are equivalent input fuzzy sets of the same input linguistic variable in the antecedent parts of the generated fuzzy rules, we need to perform the merge operation to simplify the generated fuzzy rules.
Consider the following two fuzzy rules with two input linguistic variables , , and one output linguistic variable
:
is is is
is is is
where and are the output fuzzy sets of the output linguistic variable , and are the input fuzzy sets of the input linguistic variable , and and are the input fuzzy sets of the input linguistic variable . The input fuzzy sets and of the input linguistic variable , and the input fuzzy sets and of the input linguistic variable are shown in Fig. 7.
From Fig. 7, we can see that the similarity degree between the input fuzzy sets and of the input linguistic variable is high. Since the two input fuzzy sets seem to be equivalent, we can generate a new membership function of the new input fuzzy set by merging the membership functions of the input fuzzy sets and of the input linguistic variable . After the new input fuzzy set of the input linguistic variable has been generated, we can replace the input fuzzy sets and in the
Fig. 7. The similarity between input fuzzy sets.
antecedent parts of the fuzzy rules by . The simplified fuzzy rules are listed as follows:
is is is
is is is
There are a lot of methods to measure the similarity or equality degree between two distinct fuzzy sets, such as in [4], [5], [22], [31], and [37]. In this paper, we build the equality relation matrices for the input fuzzy sets of the input linguistic variable based on the method proposed by Lin [22]. The equality degree of two distinct input fuzzy sets and of the input linguistic variable is defined as follows:
(25)
where , and
. is the area of the
intersection of the two distinct input fuzzy sets and . is the area of the union of the two distinct input fuzzy sets and .
By means of the -cuts of the equality, we can obtain those fuzzy sets whose equality degree is higher than the specified
merge their membership functions to generate a new one. As-sume that is greater than the threshold value , where the value of is specified by the user and , then we can merge the two input fuzzy sets and of the input linguistic variable to generate the new input fuzzy set which can be represented by the triplet . The triplet is calculated by averaging the triplets
( , , ) of the input fuzzy set
and the triplet , , of the input fuzzy
set . Hence, the triplet of the
merged input fuzzy set can be obtained as follows:
(26) (27) (28)
The membership function of the input fuzzy set of the input linguistic variable can be represented
by the triplet as follows:
if ,
if ,
otherwise.
(29)
The new fuzzy set of the input linguistic variable can replace those input fuzzy sets of the input linguistic variable whose equality degree is higher than the threshold value in the antecedent parts of the fuzzy rules. Therefore, the number of the input fuzzy sets of the input linguistic variable in the generated fuzzy rules is reduced, and we can obtain simpler and more efficient fuzzy rules.
Based on the method discussed previously, the proposed fuzzy learning algorithm which constructs the membership functions of the input linguistic variables and the output lin-guistic variable and generates fuzzy rules from the numerical training data set is stated as follows.
Input: The training data set contains input–output
pairs of the input
lin-guistic variables , and the output linguistic variable , where . Step 1: Sort the training data set in ascending order
based on the output values of the output linguis-tic variable and obtain the sorted training data set .
Step 2: Construct the equivalence relation between the output values of the output linguistic vari-able in the sorted training data set . Step 3: Divide the sorted training data in the sorted
training set into different output-value
sets of the output linguistic variable and different input-value sets of the input linguistic variable based on the -cuts of the equivalence relations derived from the output values of the output linguistic variable in the sorted training set , where
and .
Step 4: Derive the membership function of the output fuzzy set of the output linguistic variable based on the -cuts of the output fuzzy set with respect to the output-value sets , where .
Step 5: Sort the input values of the input-value set of the input linguistic variable in ascending
order, where and .
Step 6: Construct the equivalence relation between the input values of the th input-value set of the input linguistic variable , where
and .
Step 7: Divide the input values of the input-value set into input-value subsets based on the -cuts of the equivalence relation
, where
, and is the number of input-value subsets obtained from the th input-input-value set of the input linguistic variable . Step 8: Derive the input membership function
of the input fuzzy set
of the input linguistic variable based on the -cut of the input fuzzy set with respect to the input-value subset of the input linguistic variable , where
and ,
and is the number of input-value subsets obtained from the th input-value set
of the input linguistic variable . Step 9: Generate the fuzzy rules based on the
hier-archical relationship between the output fuzzy set of the output linguistic variable and the corresponding input fuzzy set of the input linguistic variable , where
, and .
Step 10: Calculate the equality degree between the input
fuzzy sets and of the input
linguistic variable based on (25), where ,
and .
Step 11: If , , where
is specified by the user,
, and then
Construct the new input fuzzy set of the input linguistic variable by merging the input fuzzy sets
and of the input
TABLE I
THETRAININGDATASET OF THEINSURANCEFEEPROBLEM[11]
Replace the input fuzzy sets and of the input linguistic variable in the antecedent parts of the fuzzy rules by the new input fuzzy set
IV. AN EXAMPLE
In the following, we use the example shown in [11] to demonstrate the proposed fuzzy learning algorithm for con-structing membership functions and generating fuzzy rules from the numerical training data set.
Example 4.1: An insurance company evaluates the insur-ance fee of a person based on the age and the property of the person. Assume that the eight training examples are available as shown in Table I. Each input–output pair consists of two input linguistic variables, “Age” and “Property,” and one output linguistic variable, “Insurance Fee.” The unit of the property is ten thousand dollars. We will learn the membership
functions of the two input linguistic variables “Age” and “Property,” and the output linguistic variable “Insurance Fee” and generate fuzzy rules from the eight training samples based on the proposed algorithm.
[Step 1]: /* Sort the training data set in ascending order according to the output values of the output linguistic variable */
Since the output values of the output linguistic variable “Insurance Fee” of the training data set shown in Table I have been arranged in ascending order, by performing Step 1 of the algorithm, the sorted training data set is shown as follows:
[Step 2]: /* Construct the equivalence relation between the output values of the output linguistic variable */
According to (3), the compatibility relation between the output values of the output linguistic variable “Insurance Fee” is constructed as shown in (30) and (31), at the bottom of the page, where the equivalence relation between the output values of the output linguistic variable “Insurance Fee” in the sorted training data set can be derived by the max–min transitive closure of the compatibility relation as shown in (32), at the bottom of the page.
[Step 3]: /* Divide input–output pairs of the sorted training data set into different subsets based on the
(30)
(31)
-cuts of the equivalence relation between the output values of the output linguistic variable */
Assume that the threshold value is 0.8, then the sorted training set can be divided into three subsets , and based on the 0.8-cut of the equivalence relation as follows:
Furthermore, the output-value sets , and of the output linguistic variable “Insurance Fee,” the input-value sets , and of the input linguistic variable “Age,” and the input-value sets , and of the in-put linguistic variable “Property” are obtained as follows:
[Step 4]: /* Derive the membership functions of the output fuzzy sets of the output linguistic variable from different -cuts of the output fuzzy sets corre-sponding to the output-value sets */
Assume that the threshold value of the -cut of the output fuzzy set of the output linguistic variable “Insurance Fee” corresponding to the output-value set of the output linguistic variable “Insurance Fee” equals to 0.8, where . Then, based on (9)–(11), we can obtain the membership function of the output fuzzy set of the output linguistic variable “Insurance Fee” represented by the triplet , where . The triplet of the output fuzzy set of the output linguistic variable “Insurance Fee” can be obtained as follows:
Thus, the membership function of the output fuzzy set can be represented by the
triplet as follows: if if
otherwise.
The triplet of the output fuzzy set of the output linguistic variable “Insurance Fee” can be obtained as follows:
Thus, the membership function of the output fuzzy set can be represented by the triplet as follows:
if if
otherwise.
The triplet of the output fuzzy set of the output linguistic variable “Insurance Fee” is obtained as follows:
Thus, the membership function of the output fuzzy set is represented by the triplet
as follows: if if
otherwise.
The membership functions of the output fuzzy sets , and of the output linguistic variable “Insurance Fee” are shown in Fig. 8. [Step 5]: /* Sort the input values of the input-value sets of
the input linguistic variable */
The two input linguistic variables “Age” and “Property” are denoted by and . In the input-value sets , and of the input linguistic variable “Age,” the sorted input-value sets , and are the same as the input-value sets , and , respectively. In the input-value sets , and of the input
Fig. 8. The fuzzy sets of the output linguistic variable “Insurance Fee.”
linguistic variable “Property,” the sorted input-value sets are represented by , and . That is
[Step 6]: /* Construct the equivalence relation between the input values of the input-value set of the input linguistic variable */
Based on (13), the compatibility relations , and are derived from the input-value sets , and of the input linguistic variable “Age,” respectively,
where
Inf
where Inf means positive infinity.
The equivalence relations , , and of the input-value sets , , and of the input linguistic variable “Age” can be obtained by the max–min transitive closure of compatibility relations , and , respectively
where
By the same way, the compatibility relations of the input-value sets , and of the input linguistic variable “Property” can be obtained as follows:
Then, the equivalence relations of the sorted input-value sets , and of the input linguistic variable “Property” can be obtained as follows:
[Step 7]: /* Divide the value set into different input-value subsets based on the -cuts of the equiv-alence relation between the input values of the input-value set */
Indeed, the training examples are so few that it is not necessary to partition input values again. The input-value subsets of the input linguistic variables “Age” and “Property” are the same with their corresponding input-value sets, and the number of subsets for each input-value set equals to one.
. Hence, the input-value subsets of the input linguistic variable “Age” are obtained as follows:
The input-value subsets of the input linguistic variable “Property” are obtained as follows:
[Step 8]: /* Derive the membership functions of the input fuzzy sets of the input linguistic variable from the -cut of the input fuzzy set corresponding to the input-value subset */
The input-value subsets can be thought of as the -cut set of the input fuzzy set of the input linguistic variable “Age,”
where , and . Assume that
the threshold value of the -cut set is equal to 0.7. Thus, we can derive the membership function of the input fuzzy set of the input linguistic variable “Age” based on (17)–(19). The
triplet of the input fuzzy
set of the input linguistic variable “Age” can be obtained as follows:
The membership function of the in-put fuzzy set of the input linguistic vari-able “Age” can be represented by the triplet
as follows:
if if
otherwise.
The triplet of the input
fuzzy set of the input linguistic variable “Age” can be obtained as follows:
The membership function of the in-put fuzzy set of the input linguistic vari-able “Age” can be represented by the triplet
as follows:
if if
otherwise.
The triplet of the input
fuzzy set of the input linguistic variable “Age” can be obtained as follows:
where the value of is equal to the average of the larget element of and the smallest element of , i.e.,
Max Min
Fig. 9. The fuzzy sets of the input linguistic variable “Age.”
The membership function of the in-put fuzzy set of the input linguistic vari-able “Age” can be represented by the triplet
as follows:
if if
otherwise. The input fuzzy sets , and of the input linguistic variable “Age” are shown in Fig. 9. The membership functions , , and of the input fuzzy
sets of the input linguistic
variable “Property” can be obtained as follows:
if if otherwise. if if otherwise. if if otherwise. The input fuzzy sets , and of the input linguistic variable “Property” are shown in Fig. 10.
[Step 9]: /* Construct fuzzy rules based on the hierarchical relationships between the output fuzzy sets of the output linguistic variable and the corresponding input fuzzy sets of the input linguistic variables */
Fig. 10. The fuzzy sets of the input linguistic variable “Property.”
Fig. 11. The hierarchical relationship between the output fuzzy setAjof the output linguistic variable “Insurance Fee,” the input fuzzy setA1; j; k of the input linguistic variable “Age,” and the input fuzzy setA2; j; k of the input linguistic variable “Property.”
The hierarchical relationship between the out-put fuzzy set of the output linguistic variable “Insurance Fee,” the input fuzzy set of the input linguistic variable “Age,” and the input fuzzy set of the input linguistic variable “Property” is shown in Fig. 11.
Based on the hierarchical relationship between the output fuzzy set of the output linguistic variable “Insurance Fee,” the input fuzzy set of the input linguistic variable “Age,” and the input fuzzy set of the input linguistic variable “Property,” the fuzzy rules of the Insurance Fee problem are generated as follows: Age is Property is Insurance Fee is Age is Property is Insurance Fee is Age is Property is Insurance Fee is
[Step 10]: /* Calculate the equality degree between the input fuzzy sets of the input linguistic variable */
Based on (25), we can calculate the equality degree between the input fuzzy sets of the input linguistic variable “Age” as follow.
First, the area of the intersection of the two distinct input fuzzy sets and can be calculated from Fig. 9 as follows:
The area of the union of the two distinct input fuzzy sets and can be calculated as from Fig. 9 as shown in (33) at the bottom of the page. Based on (25), the equality degree be-tween fuzzy sets and is obtained as follows:
By the same way, the equality degrees,
and , are
calculated as shown in (34) at the bottom of the next page. The equality matrix of the input fuzzy sets of the input linguistic variable “Age” is obtained as follows:
Based on (25), we can calculate the equality degree between the input fuzzy sets of the input linguistic variable “Property” as shown in (35) at the bottom of the next page. The equality matrix of the input fuzzy sets of the input linguistic variable “Property” is obtained as follows:
Assume that the value of specified by the user is 0.5, then we can see that the equality degrees between the input fuzzy sets of the
input linguistic variables “Age” and “Property” in the equality matrices and , respectively, are less than 0.5. It is not necessary to perform the merge operation.
Finally, the fuzzy rules generated by the pro-posed fuzzy learning algorithm are listed as follows: Age is Property is Insurance Fee is Age is Property is Insurance Fee is Age is Property is Insurance Fee is V. EXPERIMENTAL RESULTS
There are three kinds of flowers in the Iris data [7], i.e., Setosa, Versicolor, and Verginica, where each flower can be identified by the four kinds of input variables, i.e., sepal length, sepal width, petal length, and petal width. The unit of each input variable is centimeters. The Iris data set contains 150 data. Based on the proposed fuzzy learning algorithm, we have implemented a program on a Pentium PC to generate the membership functions and the fuzzy rules from the Iris training data automatically and then evaluate the average accuracy rate of the proposed fuzzy learning algorithm with the testing data set. We use the MATLAB software to develop the program and randomly choose 50% of the Iris data as the training data set and the other 50% as the testing data set. The membership functions of the input variables are derived under the 0.5-cut of the equality relations and shown in Figs. 12–15. The final generated fuzzy rules are shown in Table II.
Fig. 12. Fuzzy sets of the input variable “Sepal Length.”
Fig. 13. Fuzzy sets of the input variable “sepal width.”
Fig. 14. Fuzzy sets of the input variable “petal length.”
The simplified fuzzy rules are as follows. : IF sepal length is AND sepal width is
AND petal length is AND petal width is THEN the flower is Setosa.
(34)
Fig. 15. Fuzzy sets of the input variable “petal width.”
TABLE II
SIMPLIFIEDFUZZYCLASSIFICATIONRULES FOR THEIRISDATA
TABLE III
THEAVERAGEACCURACYRATE OF THE
PROPOSEDFUZZYALGORITHM FOR THEIRISDATA
TABLE IV
A COMPARISON OF THENUMBER OFFUZZYRULES AND THE NUMBER OFINPUTFUZZYSETSBETWEEN
HONG-AND-LEE’SALGORITHM AND THEPROPOSEDALGORITHM
: IF sepal length is AND sepal width is AND petal length is AND petal width is THEN the flower is Versicolor.
: IF sepal length is AND sepal width is AND petal length is AND petal width is THEN the flower is Virginica.
The average accuracy rate of the proposed fuzzy learning algorithm after 200 runs is listed in Table III.
A comparison of the proposed fuzzy learning algorithm and Hong and Lee’s algorithm [11] in terms of the number of the fuzzy rules and the number of the input fuzzy sets are shown in Table IV.
A comparison of the experiment results between Hong and Lee’s algorithm [11] and the proposed algorithm is listed in Table V.
From Tables IV and V, we can see that the average accuracy rate of the proposed algorithm is better than that of Hong and Lee’s algorithm. The number of rules generated by the proposed algorithm is less than the number of rules generated
TABLE V
A COMPARISON OF THEAVERAGEACCURACYRATEBETWEEN
HONG-AND-LEE’SALGORITHM AND THEPROPOSEDALGORITHM
by Hong and Lee’s algorithm. Furthermore, the proposed algo-rithm does not need to predefine any membership functions of the input variables and the output variables. The membership functions and fuzzy rules are generated by the proposed fuzzy learning algorithm from the numerical training data.
VI. CONCLUSIONS
In this paper, we have presented a new fuzzy learning algorithm based on the -cuts of equivalence relations and the -cuts of fuzzy sets to construct membership functions and to generate fuzzy rules from numerical training data. Furthermore, we also apply the proposed algorithm to the Iris data classification problem. Based on the proposed algorithm, we have implemented a program on a Pentium PC using MATLAB Version 4.0 to deal with the Iris data classification problem. The experimental results are compared with the re-sults of Hong and Lee’s learning algorithm [11]. The proposed algorithm is better than the one presented in [11] due to the following facts.
1) The proposed algorithm could get a better average accuracy rate than the one presented in [11]. From the experimental results shown in the previous section, we can see that the average accuracy rate of the proposed algorithm is 96.21%, where the average accuracy rate of Hong and Lee’s algorithm is 95.57%.
2) The proposed algorithm generates fewer fuzzy rules than the one proposed in [11]. From the experimental results shown in Section V, we can see that the number of fuzzy rules generated by the proposed algorithm is 3, but the number of fuzzy rules generated by Hong and Lee’s algorithm is 6.21.
3) We do not need to partition the input spaces and the output spaces into fuzzy regions or predefine any mem-bership functions as shown in [11]. The memmem-bership functions and fuzzy rules can be automatically derived from the numerical training data by the proposed algo-rithm.
REFERENCES
[1] G. Beliakov, “Fuzzy sets and membership functions based on probabil-ity,” Inf. Sci., vol. 91, no. 1, pp. 95–111, 1996.
[2] J. C. Bezdek, Pattern Recognition with Fuzzy Objective Function
Algo-rithms. New York: Plenum, 1981, pp. 65–86.
[3] C. T. Chao, Y. J. Chen, and C. C. Teng, “Simplification of fuzzy-neural systems using similarity analysis,” IEEE Trans. Syst., Man, Cybern. B, vol. 26, pp. 344–354, Apr. 1996.
[4] S. M. Chen, M. S. Yeh, and P. Y. Hsiao, “A comparison of similarity measures of fuzzy values,” Fuzzy Sets Syst., vol. 72, no. 1, pp. 79–89, 1995.
[5] S. M. Chen, “Using fuzzy reasoning techniques for fault diagnosis of J-85 jet engines,” in Proc. 3rd Nat. Conf. Science Technology National
Defense, Taoyuan, Taiwan, R.O.C., 1994, vol. 1, pp. 29–34.
[6] D. Dubois and H. Prade, “What are fuzzy rules and how to use them,”
Fuzzy Sets Syst., vol. 84, no. 2, pp. 169–185, 1996.
[7] R. Fisher, “The use of multiple measurements in taxonomic problems,”
Ann. Eugenics, vol. 7, pp. 179–188, 1936.
[8] J. Giarratano and G. Riley, Expert Systems: Principles and
Program-ming. Boston, MA: PWS, 1994.
[9] A. Grauel and H. Mackenberg, “Mathematical analysis of the Sugeno controller leading to general design rules,” Fuzzy Sets Syst., vol. 85, no. 2, pp. 165–175, 1997.
[10] K. Hattori and Y. Tor, “Effective algorithms for the nearest neighbor method in the clustering problem,” Pattern Recognit., vol. 26, no. 5, pp. 741–746, 1993.
[11] T. P. Hong and C. Y. Lee, “Induction of fuzzy rules and membership functions from training examples,” Fuzzy Sets Syst., vol. 84, no. 1, pp. 33–47, 1996.
[12] T. Ichimura and E. Tazaki, “Learning and extracting method for fuzzy rules using neural networks with modified structure level adaptation,” in Proc. 3rd Int. Conf. Fuzzy Logic, Neural Nets, Soft Computing, Japan, 1994, pp. 87–90.
[13] H. Ishibuchi and H. Tanaka, “Neural network that learn from fuzzy if–then rules,” IEEE Trans. Fuzzy Syst., vol. 1, pp. 85–97, 1993. [14] J. Rudin, Principles of Mathematical Analysis. New York:
McGraw-Hill, 1976.
[15] N. K. Kasabov, “Learning fuzzy rules and approximate reasoning in fuzzy neural networks and hybrid systems,” Fuzzy Sets Syst., vol. 82, no. 2, pp. 135–149, 1996.
[16] F. Klawonn and R. Kruse, “Constructing a fuzzy controller from data,”
Fuzzy Sets Syst., vol. 85, no. 2, pp. 177–193, 1997.
[17] G. J. Klir and B. Yuan, Fuzzy Sets and Fuzzy Logic Theory and
Applications. Englewood Cliffs, NJ: Prentice-Hall, 1995.
[18] Y. Kodratoff and R. S. Michalski, Machine Learning: An Artificial
Intelligence Approach. San Mateo: Morgan Kaufmann, 1990, vol. 3, pp. 113–127.
[19] B. Kosko, Neural Networks and Fuzzy Systems: A Dynamical Systems
Approach to Machine Intelligence. Englewood Cliffs: Prentice-Hall, 1991, pp. 299–335.
[20] R. Langari and L. Wang, “Fuzzy models, modular networks, and hybrid learning,” Fuzzy Sets Syst., vol. 79, no. 2, pp. 141–150, 1996. [21] C. T. Lin and C. S. George Lee, “Neural-network-based fuzzy logic
con-trol and decision system,” IEEE Trans. Comput., vol. 40, pp. 1320–1336, Dec. 1991.
[22] , Neural Fuzzy Systems. Englewood Cliffs, NJ: Prentice-Hall, 1996.
[23] R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, Machine Learning:
An Artificial Intelligence Approach. Los Altos, CA: Morgan Kauf-mann, 1983, vol. 1, p. 41–80.
[24] R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, Machine Learning:
An Artificial Intelligence Approach. Los Altos, CA: Morgan Kauf-mann, 1983, vol. 2, pp. 45–61.
[25] R. S. Michalski and G. Tecuci, Machine Learning: A Multistrategy
Approach. San Francisco, CA: Morgan Kaufmann, 1994, vol. 4. [26] K. Nozaki, H. Ishibuchi, and H. Tanaka, “A simple but powerful
heuristic method for generating fuzzy rules from numerical data,” Fuzzy
Sets Syst., vol. 86, no. 3, pp. 251–270, 1997.
[27] D. Park, A. Kandel, and G. Langholz, “Genetic-based new fuzzy reasoning models with application to fuzzy control,” IEEE Trans. Syst.,
Man, Cybern., vol. 24, pp. 39–47, Jan. 1994.
[28] P. K. Simpson, “Fuzzy min–max neural networks—Part 2: Clustering,”
IEEE Trans. Fuzzy Syst., vol. 1, pp. 32–45, 1993.
[29] J. J. Shann and H. C. Fu, “A fuzzy neural network for rule acquiring on fuzzy control systems,” Fuzzy Sets Syst., vol. 71, no. 3, pp. 345–357, 1995.
[30] K. Uehara and M. Fujise, “Fuzzy inference based on families of -level sets,” IEEE Trans. Fuzzy Syst., vol. 1, pp. 111–124, 1993.
[31] W. J. Wang, “New similarity measures on fuzzy sets and on elements,”
Fuzzy Sets Syst., vol. 85, no. 3, pp. 305–309, 1997.
[32] L. X. Wang and J. M. Mendel, “Generating fuzzy rules by learning from examples,” IEEE Trans. Syst., Man, Cybern., vol. 22, pp. 1414–1427, 1992.
[33] L. X. Wang, Adaptive Fuzzy Systems and Control: Design and Stability
Analysis. Englewood Cliffs, NJ: Prentice-Hall, 1994, pp. 27–28. [34] T. P. Wu and S. M. Chen, “A new method for constructing membership
functions and fuzzy rules from numerical data,” in Proc. 1997 5th Nat.
Conf. Fuzzy Theory Applications, Tainan, Taiwan, R.O.C., 1997, pp.
499–504.
[35] T. Takagi and M. Sugeno, “Fuzzy identification of systems and its applications to modeling and control,” IEEE Trans. Syst., Man, Cybern., vol. 15, pp. 116–132, 1985.
[36] R. R. Yager and D. P. Filev, Essential Fuzzy Modeling and Control. New York: Wiley, 1994.
[37] V. R. Young, “Fuzzy subsethood,” Fuzzy Sets Syst., vol. 77, no. 3, pp. 371–384, 1996.
[38] L. A. Zadeh, “Fuzzy sets,” Inf. Contr., vol. 8, pp. 338–353, 1965. [39] , “Fuzzy logic,” IEEE Comput., vol. 21, pp. 83–91, 1988. [40] , “The concepts of a linguistic variable and its application to
approximate reasoning (I),” Inf. Sci., vol. 8, pp. 199–249, 1975. [41] H. J. Zimmermann, Fuzzy Set Theory and Its Applications. Boston,
MA: Kluwer-Nijhoff, 1991.
Tzu-Ping Wu was born on October 9, 1967, in Kaohsiung, Taiwan, R.O.C. He received the B.S. degree in information engineering from the Feng-Chia University, Taichung, Taiwan, in June 1989, and the M.S. degree in computer and information science from the National Chiao Tung University, Hsinchu, Taiwan, in June 1997.
He is currently with the Chung Shan Institute of Science and Technology, Taoyuan, Taiwan. His current research interests include fuzzy systems, expert systems, and artificial intelligence.
Shyi-Ming Chen (M’88–SM’96) was born on Jan-uary 16, 1960, in Taipei, Taiwan, R.O.C. He re-ceived the B.S. degree in electronic engineering from the National Taiwan University of Science and Technology, Taipei, in June 1982, and the M.S. and Ph.D. degrees in electrical engineering from the National Taiwan University, Taipei, in June 1986 and June 1991, respectively.
From August 1987 to July 1989 and from August 1990 to July 1991, he was with the Department of Electronic Engineering, Fu-Jen University, Taipei. From August 1991 to July 1996, he was an Associate Professor in the Department of Computer and Information Science, National Chiao Tung University, Hsinchu, Taiwan. From August 1996 to July 1998, he was a Professor in the Department of Computer and Information Science, National Chiao Tung University, Hsinchu. Since August 1998, he has been a Professor in the Department of Electronic Engineering, National Taiwan University of Science and Technology, Taipei. His current research interests include fuzzy systems, database systems, expert systems, and artificial intelligence. He has published more than 95 papers in referred journals and conference proceedings.
Dr. Chen was the winner of the 1994 Outstanding Paper Award of the
Journal of Computer and Information Education and the winner of the 1995
Outstanding Paper Award of the Computer Society of the Republic of China. He was the winner of the 1997 Outstanding Youth Electrical Engineer Award of the Chinese Institute of Electrical Engineering, Republic of China. He is a member of the ACM, the International Fuzzy Systems Association (IFSA), and the Phi Tau Phi Scholastic Honor Society. He is an Executive Committee Member of the Taiwanese Association for Artificial Intelligence (TAAI). He is also an Executive Committee Member of the Chinese Fuzzy Systems Association (CFSA). He is an Editor of the Journal of the Chinese Grey
Systems Association. He is listed in International Who’s Who of Professionals.
