CHAPTER 3 CONCEPT DRIFT FOR FUZZY MEMBERSHIP FUNCTIONS
3.3 Experimental Results
In this part, the results of the experiments show the performance of the proposed concept-drift for fuzzy membership functions (CDMF) algorithm. We used single computer with 3rd generation Intel Core i5-3230M 2.60GHz processor with 4 cores, 4 threads and DDR3-1600Mhz 12 GB random-access memory. The operating system was Microsoft Windows 8.1 Pro, and the programming language was .NET Framework 4.5.1 C# (C# Version 5.0).
A simulation dataset containing 1559 items and 21,556 transactions was used in the experiments. In the data set, the number of purchased items in transactions was first randomly generated, and the purchased items and their quantities in each transaction were then generated. Here, we selected 21,566 transactions from the simulated dataset, and divided into two datasets as databases Dt and Dt+k, where each dataset had 10,733 transactions. The initial cluster size C was set at 3, the fuzziness index value m was set at 2, the linguistic term threshold varies from 1 to 0.1, the membership functions threshold varies from 1 to 0.1,and support threshold varies from 1 to 0.1 Firstly, Figure
3.4 , 3.6 and 3.8 show the proposed approach.
Experiments were first conducted on database to evaluate the numbers of linguistic term concept-drifts items with different thresholds.
Figure 3.4: The number of concept-drift item by the algorithms along with different linguistic term threshold in database
Figure 3.4 shows the proposed algorithm was performed with different pair of databases that were two databases at different locations, the databases of first half with second half of a year, the two databases of the random months, and the database of a random month with whole year. In the experimental results, we can find the influence for customer behavior for a different time that was bigger than a different location. We observed that the short-term databases may contain more special rules, so when we compared these databases, more concept-drifts can be found. In the contrast, since the
0
First Half with Second Half of A Year
Random Months
A Random Month with Whole Year
long-term databases tended to be stable, less concept-drifts will occur. As a result, we consider that the comparison between short-term databases as more preferable. Figure 3.5 showed the executive efficiency for proposed algorithm on database for different linguistic term thresholds varying from 1 to 0.1.
Figure 3.5: The execution efficiency of the four different time and location with different linguistic term threshold in database.
In Figure 3.5, it showed the executive efficiency of the different linguistic term thresholds in database. More concept-drift patterns will be generated when the threshold value increases. Thus, the proposed algorithm will spend more executive time if the process sets a higher value of threshold.
Experimental results were second conducted on database to evaluate the numbers of membership functions concept-drifts items with and different threshold. Figure 3.6
0
First Half with Second Half of A Year
Random Months
A Random Month with Whole Year
shows the effect of different threshold values that identify the number of rules.
Figure 3.6: The number of concept-drift item by the algorithms along with different membership functions threshold in database.
Experiment were made database to evaluate efficiency of the algorithms. Figure 3.7 showed the execution efficiency on database for different membership functions threshold varying from 1 to 0.1.
0 200 400 600 800 1000 1200 1400 1600
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1
Concept-drift Items
Thresholds
Membership Functions
Different Location
First Half with Second Half of A Year
Random Months
A Random Month with Whole Year
Figure 3.7: The execution efficiency of the four different time and location with different membership functions threshold in database.
Finally, experiment were conducted on database to evaluate the numbers of support concept-drifts items with and different threshold. Figure 3.8 shows the effect of different threshold values that identify the number of rules.
0 100 200 300 400 500 600
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1
Execution Time(sec.)
Thresholds
Membership Functions
Different Location
First Half with Second Half of A Year
Random Months
A Random Month with Whole Year
Figure 3.8: The number of concept-drift item by the algorithms along with different support threshold in database.
Experiment were at last made database to evaluate efficiency of the algorithms.
Figure 3.9 showed the execution efficiency on database for different support threshold varying from 1 to 0.1.
0 200 400 600 800 1000 1200 1400 1600 1800
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1
Concept-drift Items
Thresholds
Support
Different Location
First Half with Second Half of A Year
Random Months
A Random Month with Whole Year
Figure 3.9: The execution efficiency of the four different time and location with different support threshold in database.
There are some concept-drift patterns with higher threshold values. However, these patterns are mostly represented by different types of concept-drifts. The proposed algorithm should be set at a suitable threshold value to attain a reasonable number of patterns as well as making a good representation.
0 100 200 300 400 500 600 700 800
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1
Execution Time(sec.)
Thresholds
Support
Different Location
First Half with Second Half of A Year
Random Months
A Random Month with Whole Year
CHAPTER 4
CONCEPT DRIFT FOR FUZZY ASSOCIATION RULES
4.1 Definitions and Review Fuzzy Association Rules
In this chapter, we will present the fuzzy association rules concept-drift patterns mining (CDFAR).
4.1.1 Fuzzy Membership Functions by Fuzzy C-means
In this part, our previous method is introduced to generate the set of membership functions by fuzzy C-means. Each membership function is designed as a triangle and encoded as a pair (c, w). The peak of triangle is located at c and the distance between c and left acme is w.
Membership function plays a role in converting commodity items into something similar with human semantics. Figure 4.1 shows membership functions set. Figure 4.1 consists of three membership functions representing low, medium, and high for each different purchase amount.
3.0 7.0 11.0 1.0
Apple
Low Medium High
Membership value
Quantity
Figure 4.1: Membership functions set that apple.
We combine the data of the two different databases and generate fuzzy membership functions by this data. The fuzzy membership functions which is generated by Fuzzy C-means is fixed and the same in order to apply to the two databases.
4.1.2 Generating Fuzzy Association Rules by Fuzzy Apriori
In this part, we will generate fuzzy membership functions which are produced by the method in chapter 4.1.1 as the input data by fuzzy Apriori.
Table 4.1 is a transaction database. In the fuzzy association rules, the membership functions is applied to turn information into semantic words.
Table 4.1: An example of a transaction database.
ID Expanded Items 1 (A, 3)(C, 6)(E, 9) 2 (B, 4)(C, 7)(D, 10) 3 (B, 2)(C, 5)(E, 8) 4 (C, 1)(E, 14)
membership functions, we can get fuzzy values of different linguistic terms of each item. So the original transaction database can be converted into a database with fuzzy linguistic terms. An example is shown in Table 4.2.
Table 4.2: Table 4.1 after converting the fuzzy database
TID Fuzzy set
1
2 3 4
Next step, fuzzy frequent itemsets are generated by fuzzy linguistic terms, and then the fuzzy association rules are obtained by fuzzy apriori algorithm [50]. The processes of Apriori algorithm as following :
INPUT: n : quantitative database consisting of transaction; a set of membership functions; α : the minimum support threshold.
OUTPUT: The concept-drift rule for fuzzy membership functions.
STEP 1: Transform the quantitative value vij of each item Ij in the i-th transaction into a fuzzy set fij represented as (fij/Rj1+ fij/Rj2+…+ fijh/Rjh) using the given membership functions, where h is the number of fuzzy regions (linguistic terms) for Ij , Rjl is the l-th fuzzy region of Ij, 1 ≤ l ≤ h, and fijl is vij’s fuzzy membership value in region Rjl.
. )
STEP 2: Calculate the scalar cardinality of each fuzzy region (linguistic terms) Rjl in the transaction data as:
(4-1) STEP 3: Check whether the value countjl of the fuzzy region Rjl is larger than or equal
to the minimum count n × α. If the count of a fuzzy region Rjl is equal to or greater than the minimum count, put the fuzzy region in the set of frequent fuzzy regions (L1). That is
(4-2) STEP 4: If L1 is null, then exit the algorithm; otherwise, do the next step.
STEP 5: Set r = 1, where r is used to represent the number of items kept in the current large itemsets.
STEP 6: Generate the candidate set Cr+1 form Lr. Restated, the algorithm joins Lr and Lr under the condition that r+1 items in the two itemsets are the same and the other one is different. Store in Cr+1 the itemsets which have all their sub-r-itemsets in Lr.
STEP 7: Calculate the following sub-steps for each newly formed (r+1)-itemset s with items (s1, s2, …, sr+1) in Cr+1.
(A) For each transaction datum D, calculate its fuzzy value on s as , where is membership value of D is sj. If the
minimum operator is used for the intersection, then
(4-3) (B) Calculate the count of s in the transaction as :
(4-4) (C) If counts is larger than or equal to the minimum support value α, put s in Lr+1. STEP 8: If Lr+1 is null, then do the next step; otherwise, set r = r + 1 and repeat Steps
6 to 7.
STEP 9: Construct the fuzzy association rules for each large q-itemset s with items (s1,
s2, …, sq), q ≥ 2, using the following sub-steps : (A) form all possible association rules as follows :
(4-5) (B) Calculate the confidence values of each fuzzy association rules using :
(4-6)
STEP 10: Output the rules with confidence values larger than or equal to the confidence threshold λ .
4.1.3 Concept-drift Patterns for Fuzzy Association Rules
In this part, , we generalize the original concept-drift patterns in [24] to quantitative transactions. The following different concept drift patterns of fuzzy
sj
association rules are considered. The first one is the fuzzy emergent patterns in which both the conditional and the consequent parts between two fuzzy association rules from two different databases are the same but the fuzzy support values of the conditional or consequent parts are different. The second one is the unexpected change for fuzzy association rules. It considers two rules in different databases with similar change of the condition parts, but their consequent parts are quite different. The last one also considers unexpected change for fuzzy association rules, but it considers the two rules with similar consequent parts and quite different conditional parts. They are described below. The added and perished concept drift patterns are not considered in the paper.
(A) The fuzzy emerging change
In fuzzy emerging patterns, both the conditional and the consequent parts between two fuzzy association rules from two different databases are the same but the fuzzy support values of the conditional or consequent parts are different. There are three kinds of fuzzy support change for fuzzy emerging patterns. The first case is that the fuzzy support values of the conditional terms between two fuzzy association rules are similar but the fuzzy support values of the consequent terms are different. The second case is that the fuzzy support values of the conditional terms are different but the fuzzy support values of the consequent terms are similar. The third case is that the fuzzy support values of the conditional terms and the consequent terms are both different. Two rules
with the similar support values in both the consequent and conditional parts are not considered since they do not change significantly and are thus not emerging patterns.
In order to calculate fuzzy concept-drift patterns, the following formula modified from [24] is adopted to estimate the similarity ps of the premise (conditional) part in two fuzzy association rules:
𝑐𝑐𝑠𝑠 = �
ℓ𝑖𝑖𝑗𝑗× ∑𝑗𝑗∈𝐴𝐴𝑖𝑖𝑖𝑖𝑥𝑥𝑖𝑖𝑗𝑗𝑗𝑗
�𝐴𝐴𝑖𝑖𝑗𝑗� , 𝑖𝑖𝑖𝑖�𝐴𝐴𝑖𝑖𝑗𝑗� ≠ 0 0, 𝑖𝑖𝑖𝑖�𝐴𝐴𝑖𝑖𝑗𝑗� = 0
(4-7)
The notation in this formula is briefly explained as follows:
𝑐𝑐𝑠𝑠𝑖𝑖𝑗𝑗: The degree of premise similarity between two rules rit and rjt+k ,0 ≤ 𝑠𝑠𝑖𝑖𝑗𝑗 ≤ 1,
ℓ𝑖𝑖𝑗𝑗: The degree of attribute match of the premise part between two rules rit
and rjt+k,
�𝐴𝐴𝑖𝑖𝑗𝑗�: The number of attributes common to both conditional parts of rit and rjt+k,
𝑥𝑥𝑖𝑖𝑗𝑗𝑗𝑗: The degree of attribute value (linguistic term) match of the k-th matching
attribute in 𝐴𝐴𝑖𝑖𝑗𝑗.
In the above formula, ℓ𝑖𝑖𝑗𝑗 can be defined by the following formula [24]:
ℓ𝑖𝑖𝑗𝑗 = �𝐴𝐴𝑖𝑖𝑗𝑗�
max��𝑋𝑋𝑖𝑖𝑡𝑡�, �𝑋𝑋𝑗𝑗𝑡𝑡+𝑗𝑗�� , (4-8) where |𝑋𝑋𝑖𝑖𝑡𝑡| and �𝑋𝑋𝑗𝑗𝑡𝑡+𝑗𝑗� are the numbers of attributes in the premise parts of rit and rjt+k, respectively. 𝑥𝑥𝑖𝑖𝑗𝑗𝑗𝑗 represents the degree of attribute value (linguistic term) match
of the k-th matching attribute in 𝐴𝐴𝑖𝑖𝑗𝑗. It can be defined as follows by considering fuzzy
match:
𝑥𝑥𝑖𝑖𝑗𝑗𝑗𝑗 = 1 − �𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖_𝑐𝑐𝑖𝑖𝑠𝑠𝑖𝑖𝑖𝑖𝑖𝑖𝑐𝑐𝑖𝑖𝑖𝑖𝑗𝑗
𝑖𝑖𝑗𝑗− 1 �
𝛼𝛼
, (4-9)
where nk is the number of membership functions the k-th attribute, 𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖_𝑐𝑐𝑖𝑖𝑠𝑠𝑖𝑖𝑖𝑖𝑖𝑖𝑐𝑐𝑖𝑖 is the number of intervals between the linguistic values of the two
same attributes in the two rules rit and rjt+k, and α is a parameter controlling the effect of different linguistic values. For example, if an attribute has only three linguistic terms:
high, middle, low. Then the value of interval_distance between high and middle is 1, and between high and low is 2.
After the premise similarity of fuzzy rules is defined, the similarity cs of the
consequent parts in two fuzzy association rules is designed as follows:
𝑐𝑐𝑠𝑠𝑖𝑖𝑗𝑗 = 𝑐𝑐𝑖𝑖𝑗𝑗 × �1 − �𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖_𝑐𝑐𝑖𝑖𝑠𝑠𝑖𝑖𝑖𝑖𝑖𝑖𝑐𝑐𝑖𝑖𝑖𝑖𝑗𝑗
𝑖𝑖𝑗𝑗− 1 �
𝛼𝛼
�, (4-10)
where cij = 1 if the consequent attributes (not including values) in the two rules rit
and rjt+k are the same, and cij = 0 otherwise.
If both the psij and sij values are equal to or larger than a predefined threshold value T, then the supports of the two rules are then checked according to the three cases
mentioned above.
If the rule similarity measure is less than the threshold, it means the conditional
terms are similar in these two fuzzy association rules and if the rule similarity measure is large than the threshold, it means the conditional terms are quite different.
Below is an example to show the concept above. Assume the membership functions for the purchased amount of apples are the same as those in Figure 4.1. Also assume there are two fuzzy association rules from two different databases are shown in Table 4.3.
Table 4.3: The first case of an emerging pattern for two fuzzy association rules Database Fuzzy Association Rules
Dt (Apple.Low,0.5), (Mike.High,0.6) → (Bread.High,0.7) Dt+k (Apple.Low,0.5), (Mike.High,0.6) → (Bread.High,0.5)
In Table 4.3, both the premise similarity and the conditional similarity between the two association rules are very similar (actually the same), thus we judge their fuzzy support change. The fuzzy support values at the premise parts of the two rules are the same, but the value at the consequent parts of the two rules are different with 0.2. If 0.2 is larger than the threshold, then it is the first case of an emerging pattern.
The two fuzzy association rules in Table 4.4 are another example. Both the premise similarity and the conditional similarity between the two association rules are very similar (actually the same), thus we judge their fuzzy support change. The fuzzy support values at the premise parts of the two rules are different, but the values at the consequent parts of the two rules are the same. It is the second case of an emerging pattern.
Table 4.4: The second case of an emerging pattern for two fuzzy association rules
Database Fuzzy Association Rules
Dt (Apple.Low,0.5), (Mike.High,0.9) → (Bread.High,0.7) Dt+k (Apple.Low,0.5), (Mike.High,07) → (Bread.High,0.7)
At last, Table 4.5 shows an example for an emerging pattern for two fuzzy association rules. Both the premise similarity and the conditional similarity between the two association rules are the same. The fuzzy support values of the premise and consequent terms are different. It is thus the third case of an emerging pattern.
Table 4.5: The third case of an emerging pattern for two fuzzy association rules Database Fuzzy Association Rules
Dt (Apple.Low,0.4), (Mike.High,1) → (Bread.High,0.7) Dt+k (Apple.Low,0.4), (Mike.High,0.6) → (Bread.High,0.9) (B) The fuzzy unexpected change
There are two kinds of fuzzy concept drift patterns for unexpected change. The first one is that the premise similarity between the two association rules is very high, but the consequent similarity of the two rules is not high. The second one is the contrary.
That is, the consequent similarity between the two association rules is very high, but the premise similarity of the two rules is not high. It can be judged by a threshold in a way similar to that for emerging patterns.
Below is an example to show the first case. In Table 4.6, there are two fuzzy association rules. The premise similarity of the two rules is high, which is 1. If there are only three membership functions for Bread, then the consequent similarity of the two rules is 0.5. If the threshold is set at 0.6, then they are thought of as quite different.
It is then the first case of the unexpected change.
Table 4.6: The first case of the unexpected change for two fuzzy association rules
Database Fuzzy Association Rules
Dt (Apple.Low,0.4), (Mike.High,0.9) → (Bread.Middle,0.4) Dt+k (Apple.Low,0.4), (Mike.High,0.9) → (Bread.Low,0.2)
Finally, an example is given to show the concept drift which has different premise terms but similar consequent terms. Table 4.7 shows an example for this case. In this case, the consequent similarity of the two rules is 1, but the premise similarity of the two rules is 0.5. If the threshold is set at 0.6, then they are thought of as quite different.
It is then the second case of the unexpected change
Table 4.7: The second case of the unexpected change for two fuzzy association rules
Database Fuzzy Association Rules
Dt (Apple.Middle,0.6), (Mike.High,0.9) → (Bread.High,0.7) Dt+k (Apple.High,0.8), (Mike.Low,0.3) → (Bread.High,0.7)
4.2 The Proposed CDFAR Mining Algorithm
In this part, the proposed CDFAR approach that combines concept-drift, fuzzy C-means algorithm and fuzzy data mining is described as follows:
INPUT: Two quantitative transaction databases Dt consists of n quantitative transactions and m items at time t, and Dt+k consists of w quantitative transactions and m items at time t+k; The parameters include a support threshold α; A confidence threshold λ; A concept-drift rules sets S; cd:
conditional threshold; cs : consequent threshold; A set of membership functions.
OUTPUT: The fuzzy concept-drift patterns.
STEP 1: The two database generate fuzzy membership functions for each item via the following sub-steps.
(a) Set i = 1, where i is used to keep the identity number of the current item from database. (fuzzy c-means refer to the related words).
(b) The center points of these N clusters are set as the center of fuzzy membership functions for these M linguistic terms.
(c) Output fuzzy membership functions for each linguistic term. An example is shown in Figure 4.1.
(d) Set i = i + 1.
(e) i ≤ I, go to Step (a).
STEP 2: The two database generate fuzzy association rules for each item via the following sub-steps.
(a) A set of fuzzy membership functions by fuzzy C-means
(b) If the item satisfies the condition put it in R large itemsets. (fuzzy apriori refer to the chapter 4.1.2).
(c) Output large itemsets for each fuzzy association rules.
STEP 3: Find the concept-drift rules from the fuzzy association rules of large itemsets between Dt and Dt+k by the following sub-steps.
STEP 4: Set the initial concept-drift rules sets .
STEP 5: Set r = 1, where r is used to keep the identity number of the current rule from database.
STEP 6: Calculate the emerging change for the fuzzy association rules and check the concept-drift rules from two databases Dt and Dt+k by below sub steps.
(a) Set j = 1, where j is used to keep the identity number of the current conditional terms.
(b) Calculate the fuzzy values of conditional terms for each rule sets.
𝑐𝑐𝑠𝑠 = �
(e) Calculate the fuzzy values of consequents term for each rule sets.
𝑐𝑐𝑠𝑠𝑖𝑖𝑗𝑗 = 𝑐𝑐𝑖𝑖𝑗𝑗 × �1 − �𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖_𝑐𝑐𝑖𝑖𝑠𝑠𝑖𝑖𝑖𝑖𝑖𝑖𝑐𝑐𝑖𝑖𝑖𝑖𝑗𝑗
𝑖𝑖𝑗𝑗− 1 �
𝛼𝛼
�, (4-8)
(f) Check the concept-drift rules.
STEP 7: Calculate the unexpected change for the fuzzy association rules and check the φ
= S
concept-drift rules from two databases Dt and Dt+k by below sub steps.
(a) Set j = 1, where j is used to keep the identity number of the current conditional terms.
(b) Calculate the fuzzy values of conditional terms for each rule sets.
𝑐𝑐𝑠𝑠 = �
ℓ𝑖𝑖𝑗𝑗× ∑𝑗𝑗∈𝐴𝐴𝑖𝑖𝑖𝑖𝑥𝑥𝑖𝑖𝑗𝑗𝑗𝑗
�𝐴𝐴𝑖𝑖𝑗𝑗� , 𝑖𝑖𝑖𝑖�𝐴𝐴𝑖𝑖𝑗𝑗� ≠ 0 0, 𝑖𝑖𝑖𝑖�𝐴𝐴𝑖𝑖𝑗𝑗� = 0
(4-7)
(c) set j = j + 1
(d) Check the concept-drift rules.
(e) Calculate the fuzzy values of consequents term for each rule sets.
𝑐𝑐𝑠𝑠𝑖𝑖𝑗𝑗 = 𝑐𝑐𝑖𝑖𝑗𝑗 × �1 − �𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖_𝑐𝑐𝑖𝑖𝑠𝑠𝑖𝑖𝑖𝑖𝑖𝑖𝑐𝑐𝑖𝑖𝑖𝑖𝑗𝑗
𝑖𝑖𝑗𝑗− 1 �
𝛼𝛼
�, (4-8)
(f) Check the concept-drift rules.
STEP 8: Set r = r + 1.
STEP 9: If the item set has not been the processed as well as items, go to Step 6.
STEP 10: Output rule sets S.
4.3 Experimental Results
In this part the results of the experiments to show the performance of the proposed fuzzy association rules concept-drift patterns mining (CDFAR) algorithm. In the experimental results, fuzzy membership functions which are generated by Fuzzy
C-means are fixed and the same in order to apply to the two databases. The experiments were implemented in a computer with Intel Core i5-3230M 2.60GHz processor, 4 threads and 12G RAM. The operating system was Microsoft Windows 8.1 Pro, and the programming language was .NET Framework 4.5.1 C# (C# Version 5.0).
A simulation dataset containing 60 items and 10,000 transactions was used in the experiments. In the data set, the number of purchased items in transactions was first randomly generated, and the purchased items and their quantities in each transaction
A simulation dataset containing 60 items and 10,000 transactions was used in the experiments. In the data set, the number of purchased items in transactions was first randomly generated, and the purchased items and their quantities in each transaction