Proceedings o f 1993 International Joint Conference on Neural Networks
Fuzzy Pocket Algorithm : a generalized pocket algorithm for classification of fuzzy inputs
Hahn-Ming Lee and Weng-Tang Wang Department of Electronic Engineering National Taiwan Institute of Technology 43, Keelung RD., Sec. 4, Taipei, Taiwan
E-mail : HMLEE@et.ntit.edu.tw Abstract
Perceptron algorithm has been widely adopted in pattem recognition to decide linear decision boundaries.
Pocket algorithm, a perceptron-based algorithm, works well with nonseparable or even contradictory training instances. In this paper, A generalized pocket algorithm, called fuzzy pocket algorithm, that is capable of handling inputs in linguistics terms is proposed. Linguistic terms are represented as LR-type fuzzy sets. LR- type fuzzy sets operations and defuzzification method are utilized. The fuzzy pocket algorithm is suitable of both fuzzy and crisp inputs. Besides, nodes needed for a linguistic term are few and computation load is light.
One sample problem, called knowledge-based evaluator, is considered to illustrate the working of the proposed method. Also, the experimental results are very encouraging.
1. Introduction
Input data for classification problems may be mixture of symbolic and numerical information. However, most learning methods in neural networks are designed for real vectors. There are many applications that input attributes cannot be represented meaningfully or measured directly as a real vector. That is, we have to deal with vague information in the learning process of neural networks. Fuzzy set theory is a helpful method to process linguistic and ambiguous information, and could link the advantage of symbolic and numerical processing [ 11. Therefore, this paper is aimed at incorporating fuzzy set theory into neural network to improve neural network ability of fuzzy input handling.
There are some neural network models [2][3] to deal with this problem. The main drawbacks of these models are heavy computation load and high memory requirement. In this paper, a generalized pocket algorithm [4], called pocket algorithm, is proposed. LR-type fuzzy sets representation and operations [51 [61 are used.
Therefore, memory requirement is few, and computation load is light [5][6]. Besides, if inputs are crisp values, fuzzy pocket algorithm and pocket algorithm are the same. Therefore, fuzzy pocket algorithm is suitable for both crisp and fuzzy inputs.
2. Input representation
LR-representation [5] of fuzzy sets is used to represent input data, This is because LR-representation of fuzzy sets can represent various type of information, and the computation effort of LR-type fuzzy sets operations is decreased to a great deal without limiting the generality beyond acceptable limits [5][6]. In what follows, we use uppercase letters to indicate fuzzy intervals [5][6] and lowercase letters to denote real values. The membership function of a fuzzy interval M can be then represented as :
N
2873
and denoted as M = ( mi, mz. a, b )LR. Where L, R are two reference functions [5]. A trapezoidally shaped membership function, shown in Figure 1, can be indicated as :
I x - m 2 1 1.
L(
y )
= MZ( 0, 1 - I ~ ml - X I ),andR( x - m2 )=Max(O,l--a b b
A
Figure 1. A trapezoidally shaped membership function.
3. The fuzzy pocket algorithm
The pocket algorithm is proposed by Gallant [4]. An important advantage of the pocket algorithm over perceptron algorithm is that it behaves well with nonlinearly separable instances. The LTU node [ 7 ] , shown in Figure 2, is trained by proposed fuzzy pocket algorithm. The fuzzy pocket algorithm is described below :
Let input fuzzy vector X = ( X I , XZ, . . ., XN ), where X I , X2, . . ., XN are fuzzy intervals, and connection weight between input Xi and LTU node is a scalar. The following LR-type fuzzy intervals operations [ 5 ] are used. Let Xi = ( ml, m2, a, b )LR and WLTU,~ = ( C, c, 0,O )LR.
(1) Multiplication : If WLTU,~ > 0, else,
(2) Addition : ( ml, mz, a, b )LR 8 ( W I , w2, c, d )LR = ( ml + W I , m2 + w2, a + c, b + d )LR WLTU,~ 63 XI = ( cmi, cm2, ca, cb )LR;
W L ~ , ~ €3 Xi = ( c m , cml, - cb, - )LR.
The outcome of the fuzzy intervals operations, Y, is also a fuzzy interval. To get a crisp value, one defuzzification method, Center Of Area ( COA for short ), is then applied. That is, let Y = ( mi, m2, a, b )LR
m 2 + b
i * c L ( i )
COA( Y ) = I"' m 2 + h - a , where p( i ) is the membership value of i .
The learning equation of the fuzzy pocket algorithm is : WLm,i = WLTU,~ + desired-out * COA ( Xi ) .
4
Transfer function
1, If COA(Y) > threshold -1, Otherwise
output =
{
&
...
x1
Figure 2. The LTU node.
The comparison between pocket algorithm and fuzzy pocket algorithm is listed in Table 1. If inputs are crisp value, the activation function, transfer function, and learning equation of the fuzzy pocket algorithm are the same as pocket algorithm's. Therefore, the fuzzy pocket algorithm is a generalization of the pocket algorithm,
2874
and is suitable for both fuzzy and crisp inputs.
As illustrated in Table 2, the fuzzy perceptron [8] and the fuzzy pocket algorithm makes an improvement on perceptron-based algorithms. But they utilize fuzzy set theory for different purpose. Fuzzy perceptron relieves the convergence problem by taking advantage of fuzzy membership function. Proposed fuzzy pocket algorithm utilizes LR-type fuzzy sets operations, and defuzzification method to handle ambiguous or uncertain information.
Table 1. The Comparison between the pocket algorithm and the fuzzy pocket algorithm.
Inputs Pocket
Algorithm "m-k FUZZY Pocket Numeric, Algorithm Fuzzy Set
Node's Activation Learning Rule Weighted- S um
LR-type Fuzzy Sets Operations
wi = wi + Desired-out * xi
wi = w i + Desired-out * COA ( x i )
4. Experimental results
To show the effectiveness of the fuzzy pocket algorithm, one experiment was simulated under PDP software environment [9] on SUN/SPARC 11. The 18 instances of knowledge-based evaluator ( KBE ) [ 101 were used as training instances. Inputs of each feature are linguistic terms. The output of the KBE is Suitability, which takes the value : Poor ( -1 ) or Good ( 1 ), indicating that the application of the expert system on a domain is poor or good, respectively. These membership functions used for each attribute are illustrated in Figure 3. For KBE instance 2 and 3, we generate each possible term for the don't care condition. So, total 340 training instances are generated from the 18 training instances.
, ! ! - ,
Worth Value d Employee Acceptance d ; e1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 1 0
Solution Available Easier Solution
Adopted Fuzzy Theory Fuzzy Membership Function Fuzzy
Perceptron
FUZZY Pocket LR-type Fuzzy Sets Operations, Algorithm Defuzzification
1 2 3 4 5 6 7 8 9
Objective
Convergence Problem Fuzzy Inputs Handling
"
Teachability Risk
2 3 4 5 6 7 8 9 1 0 Figure 3. The membership functions used for instances of KBE.
2875
The experimental results of five trials are illustrated in Figure 4. The decision boundary had deteriorated slightly when pocket-run-len is from 250 to 300, and from 350 to 400. This shows that there is no known bound on the number of iteration needed for the pocket weight to get the best performance [4]. The best result we have ever got is 3 errors for these 340 nonlinearly separable instances, when pocket-run-len is 300. For those instances that cannot be classified, previously proposed method [ 111 can be used.
0 50 100 150 200 250 300 350 400 Figure 4. The classification result of the fuzzy pocket algorithm on KBE instances.
Pocket-Run-Len
5. Conclusion
The contributions of fuzzy set theory lies in their methods to model and process uncertain or ambiguous data, so often encountered in real life. Therefore, to enable a neural network classifier to handle real life situation, one may incorporate the fuzzy set theory into neural networks.
Fuzzy set theory including LR-type fuzzy sets operations and defuzzification method are utilized in the proposed fuzzy pocket algorithm. Only four nodes needed to represent a linguistic term. Besides, computation load for LR-type fuzzy set operations is light. As illustrated in the experiment, the fuzzy pocket algorithm can handle fuzzy inputs well. Also, our proposed model can be used as a method to generate a knowledge base of a connectionist expert system [6] with capability of fuzzy inputs handling. The issues of inference method, and explanation method will be explored in the future.
References
W. Pedrycz, “Fuzzy logic in Development of Fundamentals of Pattern Recogniyion,” International Joumal of Approximate Reasoning, vol. 5 , pp. 251-264, 1992.
Hiaso Ishibuchi, Ryosuke Fujioka and Hideo Tanaka, “An Architecture of Neural Networks for Input Vectors of Fuzzy Numbers,” Proceedings of International Conference on Fuzzy System, 1992, pp. 1293- 1300.
Yoichi Hayashi, “A Neural Expert System Using Fuzzy Teaching Input,” Proceedings of International Conference on Fuzzy System, 1992, pp. 1293-1300.
Stephen I. Gallant, “Connectionist Expert System”, Communication. ACM 3 1, pp. 152-169, 1988.
H. J. Zimmermann, Fuzzy Set Theory and Its Applications, Kluwer-Nijhoff Publishing, 1991,
Dibies Dubios and Henri Prade, “Fuzzy Real Algebra : Some Results,” Fuzzy Sets and Systems, vol. 2, Richard P. Lippmann, “An Introduction to Computing with Neural Net,” IEEE ASSP Magazine, pp. 4-22,
1987.
James M. Keller and Douglas J. Hunt, “Incorporating Fuzzy Membership function to Perceptron Algorithm,” IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-7, pp. 693- 699, 1985.
Neural Works Professional II/Plus, Neuralware Inc., 1991.
pp. 327-348, 1979.
[lo] R.Keller, Expert System-Development & Approaches, Prentice-Hall, Inc., 1987.
[ l l ] H.M.Lee and W.T.Wang, “Training of a Neural Network Classifier by Combining Hyperplane with Exemplar Approach,” Proceeding of IEEE International Conference on Neural Networks, 1993,494-499.
2876
