We try to explore category and association rules of customer questions by applying customer analysis and the combination of data mining and rough set theory

(1)

(2)

(3)

(4)

(5)

Clustering Analysis Data

Mining Rough Set Theory

Dynamic Frequently Asked Questions FAQ

(6)

Abstract

Nowadays the enterprise has regarded the customer relationship management as one of management successful factors, what an enterprise focuses on is customer-oriented service. Therefore the foundation for an enterprise to keep good customer relationship is to listen and solve customer’s questions. With the popularization of network development, the network became the main communication channel between enterprise and customers. Also, customer questions inquired directly through network day by day. Therefore the demand to establishes a mechanism that can automatically respond to customer questions gradually draws attention of the enterprise.

In this research, we first break the customer questions by words processing analysis. Then draw representative keywords with characteristics. We try to explore category and association rules of customer questions by applying customer analysis and the combination of data mining and rough set theory. We use customer questions association intensity to establish a dynamic frequently asked questions (FAQ).

To improve how an enterprise used to construct FAQ from the viewpoint of the enterprise, in this research, we apply actual customer questions from a company as the

(7)

user interaction, we analyze information of the user feedback to achieve a FAQ with learning ability. The contribution of this research is to assist the enterprise to cut down cost to solve the customer common questions, and then to promote good relationship between an enterprise and customers. The purpose is to achieve customer oriented service from the customer’s viewpoint.

Keywords: Frequently Asked Question (FAQ) Data Mining Cluster Analysis Rough Set Theory

(8)

(9)

………...i

Abstract……….ii

………..iv

………...v

……….vii

………viii

………..1

………..2

………..4

………..5

………..7

……….10

……….13

……….19

……….27

……….30

……….41

……….48

……….54

……….56

(10)

………73

………74

………76

………78

A ………85

B CKIP ………..98

C ………...100

D ………...101

(11)

………..6

……….11

……….14

……….……22

……….……27

FAQ ……….…...29

……….…32

……….…39

……….…41

Cosine Coefficient ………...………...51

- ………...53

Precision …..………...68

Recall ………..………69

Accuracy ……….………70

(12)

FAQ ……….3

……….16

……….20

……….23

……….26

……….31

-1………..34

-2………..34

-3………..34

-4………..35

………..36

………..38

………..42

………..45

FAQ ……….48

………..51

……….57

……….60

……….61

……….62

……….63

Precision Recall Accuracy …….66

Precision ……….………68

Recall ………..69

Accuracy ……….70

Precision Recall Accuracy ………..70

(13)

[Walton O. Anderson, Jr, 2001]

(14)

FAQ

FAQ [ 2003] FAQ

FAQ FAQ

Question & Answer, Q&A

FAQ

FAQ FAQ

FAQ

(15)

FAQ

FAQ FAQ

FAQ

FAQ FAQ

FAQ

(16)

FAQ FAQ

FAQ

FAQ Customer Questions Oriented FAQ FAQ

FAQ

FAQ Cluster Analysis Rough Set Theory

FAQ

(17)

FAQ

(18)

(19)

FAQ FAQ

FAQ

Network World 2000 Call Center

[

2001] FAQ FAQ Finder Ask Jeeves

FAQ FAQ FAQ

FAQ Finder[Robin D. Burke, Kristian J. Hammond, Steven L. Lytinen,

(20)

FAQ

FAQ Finder FAQ

FAQ Finder

1. Statistic Term Vector TF

Term Frequency IDF Inverse Document Frequency TF-IDF FAQ

2. Semantic Princeton University

Word Net FAQ

3. Coverage

FAQ FAQ

4.

tT wW cC

m T W C

(21)

T W C

t w c

Ask Jeeves FAQ

FAQ FAQ

11

FAQ Ask Jeeves

[ 2000][ 2003]

FAQ FAQ

FAQ

2000 FAQ

FAQ 2001

(22)

2000 FAQ

[Michael J. A. Berry;

Gorden S.Linoff, 2001]

Knowledge Discovery [Fayyad, U., Gregory, P.S., Padhraic, S,

1996] [Jiawei Han , Micheline

Kamber,2000 ][ Margaret H. Dunham, 2002]

1. Data Cleaning and Integration

Noise

2. Data Selection

(23)

3. Data Transformation

4. Data Mining

5. Pattern Evaluation

Support Value Confidence Value

6. Knowledge Presentation Visualization

Data Selection

Data Transformation

Data Mining

Pattern Evaluation

Knowledge Presentation

Data Base Data Integration Data Cleaning

Data

(24)

Classification Estimation Prediction Affinity Grouping Clustering Analysis [C. Westphal; T. Blaxton, 1998]

1.

Class Description Supervised Learning

2. Class

Discrete

3.

(25)

Time Sequence Analysis

4.

Market Basket Analysis [Cunningham, S.J.; Frank, E., 2001]

Association Rule [R. Agrawal; T. Imielinski; A. Swami, 1993][Kitty S. Y. Chiu, Rober W. P. Luk, Keith C. C. Chan and Korris F. L. Chung, 2002]

Cross Selling

5.

Cluster

K-means

(26)

( )

Cluster Criterion Function

Dissimilarity

[ 2002]

[ 2001]

FAQ

(27)

( )

Hierarchical Clustering Nonhierarchical Clustering [ 2000]

[ 2000]

Single Link Complete Link

Average Link K-means

[Z. Huang, 1997][J. Han and M. Kamber, 2001]

Agglomerative Clustering Methods

Button-up

Inter-cluster Similarity

Minimum Distance [ ,2000]

Ci C_j

(28)

Average Distance

O n2

O n3 Chaining effect

1.

[ 2000]

2.

3. O n³

1.

O n2

O n3

2.

(29)

Supervised Learning Unsupervised Learning

[Steinbach, Karypis and Kumar,2000]

Hierarchical Clustering [Zhao and Karypis,2002] Refinement Clustering [Pantel and Lin,2002]

Fuzzy Means

[Baker and McCallum,1998]

Mean Method [ 2003]

(30)

[Lin and Kondadadi,2001]

[ 2003] Steinbach 2000 Bisecting K-Means

Algorithm K K-Means

Zhao Karypis 2002

K K

[ 2003]

K-Nearest Neighbor 2000

2002 2001

2002

(31)

Rough Set Theory

Rough Set Theory Z.Pawlak 1982

Rough Set

1.

2.

3.

4.

(32)

6.

/

Alexious/1999 A Rough Set Approach to Text

Classification

/2000

Yanyi Yang and T.C.

Chiam/2000

Rule Discovery Based On Rough Set

Theory L.J. Mazlack

and A.

He/2000

A Rough Set Approach in Choosing Partitioning

Attributes

(33)

/2002

Y. Bao, S.

Aoyama, X.

Du, D.

Yamada and N. Ishii/2002

A Rough Set-Based Hybrid Method to Text Categorization

Latent Semantic Indexing, LSI

S.C. Lee and M.J.

Huang/2002

Applying AI Technology and Rough Set Theory for

Mining Association Rules to Support Crime Management

and Fire-Fighting Resources Allocation

(Self- Organization Map, SOM)

/2003

(34)

Approximation Lower / Upper Approximations of a Set

Decision Equivalent Classes Condition Equivalent Classes

[Duntsch, I. And Gediga, G., 1999]

, , ,

S U Q V f U

Records U x₁,x₂,...,x_n Q Attributes

1, 2,..., _m

Q q q q V V q Q f

( , )

f x q x q q

(35)

xi x_j q x_i x_j Equivalence Relation

Door Size Cyl Mileage

T1 T₈

Tuple-Id Door Size Cyl Mileage

T1 2 Camp 4 High

T2 4 Sub 6 Low

T3 4 Camp 4 High

T4 2 Camp 6 Low

T5 4 Camp 4 Low

T6 4 Camp 4 High

T7 4 Sub 6 Low

T8 2 Sub 6 Low

http://www.cis.drexel.edu/faculty/thu

1 2 3 4 5 6 7 8

{ , , , , , , , } U T T T T T T T T

{ , , ,

Q Door Size Cyl Mileage}

{2, 4, , , 4, 6, , } V Camp Sub High Low

1 1 1 1

( , ) 2; ( , ) ; ( , ) 4; ( , )

f T Door f T Size Camp f T Cyl f T Mileage High

(36)

Mileage

[Mileage] {[Mileage High],[Mileage Low]} Mileage

2 4 5 7 8

[Mileage Low] { ,T T T T T, , , }

1 3 6

[Mileage High] { , , }T T T Door Size Cyl

1 2 7 3 5 6 4 8

, , {{ },{ , },{ , , },{ },{ }}

Door Size Cyl T T T T T T T T

X

U [A] Q [A] X

[

[ ]/A Dj

Lower Upper_{[ ]/}_A _Dj Dj

[ ]/A Dj

Lower A] X

[ ]/A Dj

Upper

[ ]/_A _Dj { _i| _i } Lower A A Dj

[ ]/_A _Dj { _i| _i }

Upper A A Dj

X

[ Door D Size

S Cyl C

T1 T₈ ]

A

X DSC

(37)

[_DSC] /[Mileage Low] { ,2 4, 7, }8

Lower T T T T T₅ Mileage Low

T3

T5 DSC T₆ T₃ T₆

Mileage High T₅ Mileage Low

[_DSC]/[Mileage High] { }1

Lower T T₃ T₆ DSC

Mileage High T₅ DSC T₃ T₆ T₅

Mileage Low T₃ T₆ Mileage High

[_DSC]/[_Mileage] { ,2 4, 7, 8, }1

Lower T T T T T

[_DSC]/[Mileage Low] { ,2 4, 5, 7, 8, 3, 6}

Upper T T T T T T T T T T T T₂, ₄, ₅, ₇, ₈

Mileage Low T₃ T₆ Mileage T₅

DSC

[_DSC]/[Mileage High] { ,1 3, 5, 6}

Upper T T T T T T T₁, ₃, ₆ Mileage High

T5 Mileage T₆ DSC

[_DSC]/[_Mileage] { ,2 4, 5, 7, 8, ,1 3, 6} Upper T T T T T T T T

Boundary Regions X Boundary_[_DSC_]/[_Mileage_] { ,T T T₃ ₅, }₆

(38)

Soft Computing

AI [Ivo and Gunther,2000]

[Munakata, T., 1998]

Microscopic, Primarily Numeric Macroscopic, Descriptive and Numeric Deductive Chaos Theory Fuzzy methods

Inductive Neural Networks, Genetic Algorithms

Rough Set Data Analysis

FAQ

(39)

FAQ

(CKIP )

FAQ FAQ

(40)

1.

Stop Words

CKIP CKIP

CKIP B

2. FAQ

Lin Kondadadi 2001

Concept Hierarchy

3.

(41)

4.

5. FAQ FAQ

FAQ TFIDF

FAQ FAQ

(42)

6. FAQ

Q&A pairs FAQ Finder[Robin D. Burke, Kristian J. Hammond, Steven L. Lytinen, 1995; Robin D. Burke, Kristian J. Hammond and Steven L. Lytinen, 1997]

Q&A

Top-5 Q&A

1. CKIP

CKIP

(43)

(D) (VC) (Na)

(VJ) (Neu) (VF)

(COMMACATEGORY) (P) (Nc)

(DE) (Neqa) (Nf)

(Ng) (A) (VH)

CKIP

R1 (D) (VC) (Na) (Na)

R2 (D) (VJ) (Neu) (Na) (Na)

R3 (VJ) (VF) (VC) (Na)

(COMMACATEGORY) (D) (P) (Nc) (VJ)

R4 (D) (VC) (Na) (Na) (VF)

(Na) (VC) (DE) (Na) (Na)

R5 (Neqa) (Nf) (Na) (VC) (Ng)

(COMMACATEGORY) (A) (Na) (D) (VH)

2. CKIP

(44)

0 1 2 2003

FAQ

(45)

CKIP C

CKIP

(Stop Words) (DE) (P) (Di)

(D) (DE)

(Na) (Na)

3.

R3

(46)

-1

NO Question C1 C2 C3 C4 C5 C6 C7 C8 C9

3

(VJ) (VF) (VC) (Na) (COMMACATEGORY) (D) (P) (Nc) (VJ)

C5

(COMMACATEGORY)

-2

3

(VJ) (VF) (VC) (Na) (D) (P) (Nc) (VJ)

C2

C3 C4 C2 (VF)

C3 C4

C2 C3 C4

-3

3

(VJ) (D) (P) (Nc) (VJ)

(47)

C6 (D) C7 (P) C6 C7

-4

3

(VJ) (Nc) (VJ)

4.

0

FAQ

5 5

(48)

(49)

Lin Knodadadi 2001

1.

(50)

2.

” ”

5 5

0.6944 6 6

(51)

3 3

0.5625 4 4

3 3

0.375 4 6

2 3 1 4 5

7 6 5 4 3

2, 3 1 5

2, 3

1, 4 5 2, 3

1, 5, 4 2, 3 1, 5, 4 2

4

(52)

Concept Hierarchy

General Specific

[Mark Sanderson and Burce Croft,1999]

(53)

(54)

Qk Record Q_k N Qs

1~ _i

K K

1~ _i

K K N

Ck

NQk NQ_k N

NQs

NCk

1~ _i

NK NK

1~ _i

NK NK N Mk

[ ]/A Dj

Lower A Dj

(55)

min S _ C Value Num C

1~ _i

NK NK

Qk Q_k Q_k

i

1~ _i

K K Q_k

s

Qk

1~

K K

Ck Q C_k

1~ _i

NK NK

1~ _i

K K

Mk

1~ _i

NK NK

(56)

min S M_k minS

s k

NQ C

( min

_ ( min )

k k

k

Num M S C

C Value

Num M S

)

1.

Q&A

Q&A Top Rank

(57)

Rk Record R_k N Qs

DEC Ck C_k

CEC Ks K_s

1~ _i

K K N

[ ]/A Dj

Lower A Dj

[ ]/A Dj

Upper A Dj

Cf Rij

Rk R_s R _s

s

DEC Ck Q C_k

CEC Qs Sys_type,K₁~K_i R_k Rk

Lower

(58)

Certainly

1 Possibly

0~1

[ ]/A Dj

Upper

[ ]/A Dj

Upper

[ ]/A Dj

Upper

[ ]/A Dj

Upper

Rij

( )

k s

s

Num DEC C CEC K Cf Num CEC K

2.

Reduction

Superfluous Attributes Core Attributes

Superfluous Attributes Cj C D

(59)

Cj

[ ]/[C D] [C Cj]/[D]

Lower Lower

[DSWC]/[Mileage] [SWC] /[Mileage]

Lower Lower

W C W

D D S S

C D

Core Attributes Cj C D

C C Cj

Cj

[ ]/[C D] [C Cj]/[D]

Lower Lower

[DWSC]/[Mileage] [DSC] /[Mileage]

Lower Lower

W

W D S C D S C

FAQ

(60)

FAQ

1 FAQ ID_ FAQ

2 ANS_FILE_NO FAQ

3 QUESTION FAQ

4 FAQ TYPE_ FAQ 5 FAQ TYPEID_ FAQ

6 TERMS FAQ

7 K TERMS_ FAQ

8 TOP WEIGHT_ FAQ

FAQ FAQ

FAQ

_ _

ANS FILE NO

FAQ

FAQ FAQ

_

FAQ TYPE FAQ TYPEID_

_

K TERMS

(61)

FAQ 0.1 1.0 FAQ

_

TOP WEIGHT

Vector Space Model

N N

- (Term Frequency – Inverse Document Frequency TF IDF )

FAQ 0 ~ 1

FAQ FAQ

FAQ TF IDF

,

i j

TF Ci j

Cj Term Frequency i

FAQ j i FAQ j Cj

FAQ j FAQ

,

TFi j

, Ci j

(62)

TF FAQ

IDF FAQ

FAQ

i log

i

IDF N

df IDFⁱ (Inverse Document Frequency) FAQ

i FAQ

i

dfi N

TF IDF _ij _ij log

i

W TF N

df i

FAQ i

FAQ

TF IDF

FAQ

(63)

Y X Y X Y

X Y

Similarity Measure Definition

Cosine Coefficient X Y

X Y

Jaccard Coefficient X Y

X Y

Dic Coefficient 2 X Y

X Y

Cosine Coefficient Cosine

Information Retrieval

1 ( _x1, _x2,... _xn) Q tf tf tf

Cosine Coefficient

2 ( _y1, _y2,..., _ym) Q tf tf tf

(64)

Cosine Coefficient

1

2 2

1 1

cos( , )

N i i i

i N N

i i

t Q F Q F

Q F

FAQ i

Dimension N cosine

Similarity

Coefficient Co-occurrence

Q F_i

Weight

C4- m

(65)

C4-

0 0.5

1

QS QS 4

1 W

4 W

C4- [ ] 4

W1

QS-1 W4 QS-1 W3 QS-1 W2 QS-1

-

(66)

FAQ

Application Service Provider, ASP A A

A

70%

FAQ A

2002 3 29 2003 11 5 2003 12 1 2004

3 17 1243 A

A

(67)

1.

6 26 T100403

?

2.

3.

XX ?

4.

(68)

422

821 521

300

521

43

-1 -1

-2 -3

Q01_1060_1

(69)

-1 -2 -3

1

-1 -2 -3

Q01_0000 357

Q01_1000 128

Q01_1010 35

Q01_1020 29

Q01_1030 23

Q01_1040 18

Q01_1050 16

Q01_1060 7

Q01_1060_1 4

Q01_1060_2 3

Q01_2000 83

Q01_2010 36

Q01_2020 31

(70)

Q01_2030_1 19

Q01_2030_2 12

Q01_2030_3 5

Q01_3000 53

Q01_3010 23

Q01_3020 19

Q01_3030 6

Q01_3040 5

Q01_4000 25

Q01_4010 > 10

Q01_4020 6

Q01_4030 9

Q01_5000 21

Q01_6000 47

Q01_6010 14

Q01_6010_1 8

Q01_6010_2 6

Q01_6020 13

Q01_6030 6

Q02_0000 164

(71)

Q02_1010 41

Q02_1020 32

Q02_1030 21

Q02_1040 17

Q02_1050 53

Q02_1050_1 25

Q02_1050_2 17

Q02_1050_3 11

Precision Recall Accuracy

1.

300

(72)

2.

Q01_1020 17

283 Q01_1020

17 1

Q01_1020 16

282 1

_

SysQue Cmatch _

SysQue Nmatch

_

SysQue Fmatch

_

ManQue Cmatch _

ManQue Nmatch _

Total Que

(ManQue Cmatch_ ManQue Nmatch_ )

(73)

Correct Ratio False Ratio Miss Ratio

Correct Ratio _ _

_ _

SysQue Cmatch SysQue Nmatch ManQue Cmatch ManQue Nmatch

False Ratio

_

SysQue Fmatch ManQue Nmatch

Miss Ratio _ _

_

ManQue Cmatch SysQue Cmatch ManQue Cmatch

3.

300

[ ]/A Dj 0.73 Upper

(74)

Q01_0000 239 61 223 56 5 93.00% 8.197% 6.695%

Q01_1000 83 217 71 211 6 94.00% 2.765% 14.458%

Q01_1010 24 276 21 264 12 95.00% 4.348% 12.500%

Q01_1020 17 283 14 271 12 95.00% 4.240% 17.647%

Q01_1030 16 284 16 273 11 96.33% 3.873% 0.000%

Q01_1040 5 295 5 280 15 95.00% 5.085% 0.000%

Q01_1050 6 294 5 288 6 97.67% 2.041% 16.667%

Q01_1060 7 293 6 280 13 95.33% 4.437% 14.286%

Q01_1060_1 5 295 5 281 14 95.33% 4.746% 0.000%

Q01_1060_2 3 297 3 276 21 93.00% 7.071% 0.000%

Q01_2000 64 236 54 229 7 94.33% 2.966% 15.625%

Q01_2010 13 287 12 278 9 96.67% 3.136% 7.692%

Q01_2020 9 291 9 276 15 95.00% 5.155% 0.000%

Q01_2030 12 288 11 274 14 95.00% 4.861% 8.333%

Q01_2030_1 13 287 10 274 13 94.67% 4.530% 23.077%

Q01_2030_2 8 292 8 282 10 96.67% 3.425% 0.000%

Q01_2030_3 9 291 8 275 16 94.33% 5.498% 11.111%

Q01_3000 33 267 26 257 10 94.33% 3.745% 21.212%

Q01_3010 16 284 14 274 10 96.00% 3.521% 12.500%

Q01_3020 8 292 7 279 13 95.33% 4.452% 12.500%

Q01_3030 5 295 5 283 12 96.00% 4.068% 0.000%

Q01_3040 4 296 4 278 18 94.00% 6.081% 0.000%

Q01_4000 14 286 14 281 5 98.33% 1.748% 0.000%

(75)

Q01_4010 3 297 3 279 18 94.00% 6.061% 0.000%

Q01_4020 5 295 5 281 14 95.33% 4.746% 0.000%

Q01_4030 6 294 6 283 11 96.33% 3.741% 0.000%

Q01_5000 11 289 10 280 9 96.67% 3.114% 9.091%

Q01_6000 22 278 18 271 7 96.33% 2.518% 18.182%

Q01_6010 11 289 9 276 11 95.00% 3.806% 18.182%

Q01_6010_1 6 294 6 281 13 95.67% 4.422% 0.000%

Q01_6010_2 5 295 5 286 9 97.00% 3.051% 0.000%

Q01_6020 8 292 7 280 12 95.67% 4.110% 12.500%

Q01_6030 4 296 4 287 9 97.00% 3.041% 0.000%

Q02_0000 61 239 53 233 6 95.33% 2.510% 13.115%

Q02_1000 47 253 40 244 9 94.67% 3.557% 14.894%

Q02_1010 14 286 13 272 14 95.00% 4.895% 7.143%

Q02_1020 8 292 7 281 11 96.00% 3.767% 12.500%

Q02_1030 6 294 6 280 14 95.33% 4.762% 0.000%

Q02_1040 5 295 5 280 15 95.00% 5.085% 0.000%

Q02_1050 14 286 13 277 9 96.67% 3.147% 7.143%

Q02_1050_1 8 292 7 281 11 96.00% 3.767% 12.500%

Q02_1050_2 3 297 3 287 10 96.67% 3.367% 0.000%

Q02_1050_3 3 297 3 288 9 97.00% 3.030% 0.000%

93.00% 95.52% 8.197% 4.104% 15.385% 7.4314%

(76)

1

95.52% 4.104%

7.4314%

2

4.1043%

3

521

(77)

1.

521 300

2.

Precision Recall

(78)

Ai Bi Ai Bi

Cj Dj Cj Dj

Ai Cj Bi Dj N Ai Bi Cj Dj

Hit

Miss Ai Dj

Bi Cj (AiDj BiCj)

Precision FAQ

Recall

FAQ Accuracy

FAQ

(79)

Precision = Ai Ai Bi FAQ

FAQ

Recall = Ai

Ai Cj FAQ

FAQ

Accuracy = Ai Dj

Ai Bi Cj Dj FAQ

FAQ 3.

Precision 0% 100

0% 100% 300

1 Precision

(80)

Precision Precision

0.11 0.19 1 0.21 0.29 3 0.31 0.39 1 0.41 0.49 1 0.51 0.59 4 0.61 0.69 9 0.71 0.79 10 0.81 0.89 32 0.91 0.99 54

1 173

288

Precision

2 Recall

(81)

Recall Recall

0.11 0.19 2 0.21 0.29 1 0.31 0.39 4 0.41 0.49 3 0.51 0.59 9 0.61 0.69 4 0.71 0.79 25 0.81 0.89 38 0.91 0.99 57

1 145

288

Recall

3 Accuracy

(82)

Accuracy

Accuracy Accuracy

0.81 0.91 6

0.82 0.92 6

0.83 0.93 7

0.84 0.94 11

0.85 0.95 10

0.86 1 0.96 13

0.87 1 0.97 17

0.88 4 0.98 31

0.89 3 0.99 38

0.9 5 1 127

288

Accuracy

Precision Recall Accuracy Precision Recall Accuracy

(83)

300

288 A

1 Precision 1 173 300

173 Precision 93.31%

2 Recall

Recall

Recall 1 145

Recall 90.64%

3 Accuracy 97.53%

(84)

FAQ

Accuracy 1

(85)

(86)

2003

(87)

1.

50 2.

A

1.

(88)

2.

FAQ

(89)

Intention

(90)

1 2000

2 2000 FAQ

3 2000 FAQ

4 2000

5 2001 FAQ

6 2002

7 2003

8 2002

9 2000

(91)

10 2003

11 2002

12 2003

13 CKIP

http://godel.iis.sinica.edu.tw/CKIP/ws/

14 1993

1 A. Chouchoulas and Q. Shen. (1999), “A Rough Set Approach to Text Classification”, In Proc. of the 7^th International Workshop on Rough Sets, Page: 118-127.

2 C. Westphal and T. Blaxton (1998), “Data Mining Solutions-Methods and Tools for Solving Real-World Problems”, John Wiley & Sons.

3 Dick Ng’ambi (2002), “Dynamic “Intelligent Handler” of Frequently Asked Questions”, ACM IUI’02, Januray, Page: 210-211.

4 Dmitri Roussinov and J. Leon Zhao (2002), “Message Sense Maker:

Enginerring a Tool Set for Customer Relationship Management”, Proceedings

(92)

of the 36^th Hawaii International conference on System Sciences (HICSS’03).

5 Dmitri Roussinov and J. Leon Zhao (2002), “Making Sense of CRM Message:

an Interctive Toolset”, Americas Conference on Information System (AIS 2002).

6 Fayyad, U. Gregory, P.S., and Padhraic, S. (1996), “The KDD Process for Extracting Useful Knowledge from Volumes of Data”, Communication of ACM, Vol. 39, No. 11, Page: 27–34.

7 Guoyin Wang (2002), “Extension of Rough Set under Incomplete Information Systems”, IEEE International Conference, Page: 1098-1103.

8 Hu, X and Cercone, N. (1995), “Rough set similarity based learning from databases”, In Proc. of the First International Conference on Knowledge discovery and Data Mining, Page: 162-167.

9 Ivo Duntsch and Gunther Gediga (2000), “Rough set data analysis”, In Proc.

of Encyclopedia of Computer Science and Technology, Page: 281-301.

10 Jiawei Han, Micheline Kamber (2000), “Data Mining Concepts and Techniques”, San Francisco: Morgan Kaufmann Publishers.

11 Kretowski, M. and Stepaniuk, J. (1996), “Selection of objects and attributes in a tolerance rough set approach”, In Proc. of the Poster Session of the ninth International Symposium on Methodologies for Intelligent Systems, Page:

(93)

169-180.

12 Kitty S. Y. Chiu, Rober W. P. Luk, Keith C. C. Chan and Korris F. L. Chung (2002), “Market-Basket Analysis with Principal Component Analysis: An Exploration”, Systems, Man and Cybernetics, 2002 IEEE International Conference.

13 Lin, T.Y. and Cercone, N. (1997), “Rough Sets and data mining-analysis of imprecise data”, Kluwer Academic Publishers.

14 Lawrence J. Mazlack, Aijing He, Yaoyao Zhu and Sarah Coppock (2000), “A Rough Set Approach in Choosing Partitioning Attributes”, In Proc. of the ISCA 13^th International Conference.

15 Mark Sanderson and Bruce Croft (1999), “Deriving concept hierarchies from text”, ACM SIGIR’99, Page: 206-213.

16 Margaret H. Dunham (2002), “Data Mining Introductory and Advanced Topics”, Prentice Hall.

17 Michael J, A. and Berry Gordon Linoff (2001), “Data Mining Techniques for Marketing, Sales, and Customer Support”, Weikeg Publishing Co,, Ltd.

18 Pawlak, Z. (1982), “Rough sets”. International Journal of Computer and Information Science 11, Page: 341-356.

(94)

International Conference of Computer Information Science, Pages: 341-356.

20 Robin D. Burke, Kristian J. Hammond and Steven L. Lytinen (1995), “FAQ Finder: A Case-Based Approach to Knowledge Navigation”, Artificial Intelligence Laboratory, University of Chicago, In Proc. of the 11^th Conference IEEE Artificial intelligence for Applications, Proceedings.

21 Robin D. Burke, Kristian J. Hammond, Vladimir Kulyukin, Steven L. Lytinen, Noriko Tomuro and Scott Schoengerg (1997), “Question Answering From Frequently-Asked Question Files: Experiences with the FAQ Finder System”, Technical Report TR-97-05, University of Chicago, Dept. of Computer Science, June.

22 Sally Jo Cunningham and Eibe Frank (1999), “Market Basket Analysis of Library Circulation Data”, In Proc. of the International Conference on Neural Information Processing, Pages: 825 – 830.

23 Show-Chin Lee and Mu-Jung Hung (2002), “Applying AI technology and rough set theory for mining association rules to support crime management and fir-fighting resources allocation”, Journal of Information, Technology and Society, Page: 65-78.

24 S. M. Weiss and C. V. Apte (2002), “Automated generation of model cases for help-desk applications”, IBM Systems journal, vol 41, no 3, Page: 421-427.

(95)

25 Walton O. Anderson, Jr (2001), “Customer Relationship Management in an E-Business Environment”, In Proc. of the IEMC’01 IEEE International Conference, Page: 311-316.

26 X. Hu, N. Cercone and W. Ziarko (1997), “Generation of Multiple Knowledge from Dtabase Base on Rough Sets Theory”, book of Kluwer Academic Publishers, Page 109-121.

27 Y. Bao, S. Aoyama, X. Du, K.Yamada and N. Ishii (2002), “A Rough Set-Based Hybrid Method to Text Categorization”, In Proc. of the IEEE International Conference on Web Information Systems Engineering, Page:

254-261.

28 Yanyi Yang and T. C. Chiam (2000), “Rule Discovery Based on Rough Set Theory”, In Proc. of the ISIF Conference, Page: 11-16.

29 Tsumoto, S and Tanaka, H. (1996), “A common algebraic framework for empirical learning methods based on rough sets and matroid theory”, Fundamenta Informaticae, 27, Page: 273-288.

30 Z. Huang (1997), “A fast clustering algorithm to cluster very large categorical dataset in data mining”, SIGMOD 1997.

31 Zhen LIU and Minyi GUO (2001), “A Proposal of Integrating Data Mining

(96)

IEEE International Conference, Page: 146-151.

32 AskJeeves,http://www.ask.com/

(97)

A

AccIs

AccLab PR003

Q1 AccIs 91/4/21

Q2 AccIs 91/4/21

Q3 AccIs 91/4/23

Q4 AccIs 91/4/24

Q5 AccIs 91/4/26

Q6 AccIs 91/4/26

Q7 AccIs 91/4/28

Q8 AccIs 91/4/29

Q9 AccIs 91/5/1

Q10 AccIs " ..." 91/5/1

Q11 AccIs 91/5/5

Q12 AccLab 91/5/6

Q13 AccIs 91/5/8

Q14 AccIs ? 91/5/9

(98)

Q16 AccIs 91/5/15

Q17 AccLab 91/5/16

Q18 AccIs " " 91/5/16

Q19 AccIs 91/5/19

Q20 AccIs 91/5/21

Q21 AccIs " " 91/5/25

Q22 AccLab 91/5/27

Q23 AccIs 91/5/27

Q24 AccIs 91/5/29

Q25 AccIs 91/5/29

Q26 AccIs 91/6/3

Q27 AccLab 91/6/3

Q28 AccIs 91/6/3

Q29 AccIs 91/6/7

Q30 AccIs 91/6/7

Q31 AccIs 91/6/11

Q32 AccIs 91/6/12

Q33 AccLab 91/6/13

Q34 AccIs 91/6/13

Q35 AccIs ” >

”

91/6/17

Q36 AccIs " " 91/6/17

Q37 AccLab ? 91/6/18

Q38 AccIs 91/6/21

Q39 AccIs > 91/6/22

Q40 AccIs 91/6/22

Q41 AccLab 91/6/22

Q42 AccIs " >

"

91/6/23

Q43 AccLab " > " 91/6/24

Q44 AccIs ? 91/6/25

Q45 AccIs 91/6/27

(99)

Q47 AccLab 91/6/29

Q48 AccIs 91/6/30

Q49 AccIs "

"

91/6/30

Q50 AccIs ” ” 91/6/30

Q51 AccIs 91/7/2

Q52 AccIs " > " 91/7/2

Q53 AccLab 91/7/3

Q54 AccIs 91/7/4

Q55 AccIs 91/7/4

Q56 AccIs 91/7/6

Q57 AccIs 91/7/7

Q58 AccIs 91/7/9

Q59 AccLab 91/7/10

Q60 AccIs 91/7/11

Q61 AccIs 91/7/12

Q62 AccIs 91/7/12

Q63 AccIs 91/7/13

Q64 AccIs 91/7/15

Q65 AccIs 91/7/15

Q66 AccIs 91/7/18

Q67 AccIs 91/7/27

Q68 AccIs "

"

91/7/27

Q69 AccIs " >

"

91/7/29

Q70 AccIs 91/7/30

Q71 AccIs ”

”

91/7/31

Q72 AccIs 91/8/3

(100)

Q74 AccIs " " 91/8/7

Q75 AccLab ” ” 91/8/7

Q76 AccIs 91/8/11

Q77 AccIs 91/8/12

Q78 AccIs 91/8/14

Q79 AccIs 91/8/14

Q80 AccLab 91/8/15

Q81 AccIs 91/8/17

Q82 AccIs ? 91/8/18

Q83 AccIs 91/8/18

Q84 AccIs 91/8/20

Q85 AccIs "

"

91/8/21

Q86 AccIs 91/8/23

Q87 AccIs 91/8/24

Q88 AccLab 91/8/27

Q89 AccIs ? 91/8/30

Q90 AccIs 91/8/31

Q91 AccIs

?

91/9/7

Q92 AccIs ? 91/9/8

Q93 AccIs 91/9/11

Q94 AccIs 91/9/14

Q95 AccIs ? 91/9/16

Q96 AccIs ? 91/9/18

Q97 AccIs 91/9/23

Q98 AccLab 91/9/23

Q99 AccIs 91/9/26

Q100 AccIs 91/9/26

Q101 AccIs " " 91/9/27

Q102 AccIs 91/9/30

(101)

Q104 AccLab ” ” 91/10/1

Q105 AccIs ? 91/10/2

Q106 AccIs ? 91/10/2

Q107 AccIs 91/10/3

Q108 AccIs ? 91/10/4

Q109 AccIs 91/10/4

Q110 AccIs 91/10/5

Q111 AccIs ” ” 91/10/6

Q112 AccIs 91/10/13

Q113 AccIs ? 91/10/14

Q114 AccIs ? 91/10/16

Q115 AccLab " " 91/10/17

Q116 AccIs "

"

91/10/18

Q117 AccIs ? 91/10/22

Q118 AccIs 91/10/22

Q119 AccIs 91/10/25

Q120 AccIs " " 91/10/25

Q121 AccIs ? 91/10/25

Q122 AccIs ? 91/10/26

Q123 AccIs 91/10/28

Q124 AccIs 91/10/28

Q125 AccIs ? 91/10/29

Q126 AccIs 91/10/30

Q127 AccIs 91/10/31

Q128 AccLab 91/10/31

Q129 AccIs 92/3/21

Q130 AccIs 92/3/21

Q131 AccIs 92/3/22

(102)

Q132 AccIs 92/3/24

Q133 AccIs > 92/3/24

Q134 AccIs 92/3/25

Q135 AccIs

?

92/3/29

Q136 AccIs ? 92/3/29

Q137 AccIs

?

92/4/2

Q138 AccLab 92/4/3

Q139 AccIs

?

92/4/3

Q140 AccIs ? 92/4/8

Q141 AccIs 92/4/8

Q142 AccIs

?

92/4/9

Q143 AccIs

?

92/4/10

Q144 AccIs ? 92/4/11

Q145 AccIs 92/4/13

Q146 AccLab ” >

”

92/4/13

Q147 AccIs 92/4/15

Q148 AccIs 92/4/16

Q149 AccIs 92/4/22

Q150 AccIs 92/4/23

Q151 AccIs 92/4/23

Q152 AccIs ? 92/4/26

Q153 AccIs 92/4/27

(103)

Q154 AccIs

?

92/4/29

Q155 AccIs 92/4/29

Q156 AccIs 92/4/30

Q157 AccIs >

? ?

92/5/1

Q158 AccIs 92/5/2

Q159 AccIs ? 92/5/10

Q160 AccIs 92/5/11

Q161 AccIs 92/5/12

Q162 AccIs

> ?

92/5/15

Q163 AccIs 92/5/17

Q164 AccIs 92/5/18

Q165 AccIs 92/5/20

Q166 AccIs 92/5/20

Q167 AccIs 92/5/20

Q168 AccIs 92/5/21

Q169 AccIs 92/5/21

Q170 AccIs 92/5/23

Q171 AccIs

?

92/5/24

Q172 AccIs 92/5/26

Q173 AccIs 92/5/26

(104)

Q175 AccIs 92/5/29

Q176 AccIs 92/5/30

Q177 AccIs 92/5/30

Q178 AccIs ? 92/6/1

Q179 AccIs 92/6/1

Q180 AccIs " > "

?

92/6/3

Q181 AccIs 92/6/4

Q182 AccIs 92/6/8

Q183 AccIs ? 92/6/8

Q184 AccIs 92/6/14

Q185 AccIs 92/6/15

Q186 AccIs ? 92/6/15

Q187 AccIs 92/6/16

Q188 AccIs ” > ” ? 92/6/18

Q189 AccIs ? 92/6/19

Q190 AccLab 92/6/20

Q191 AccIs 92/6/21

Q192 AccIs ? 92/6/21

Q193 AccIs 92/6/26

Q194 AccLab

?

92/6/27

Q195 AccIs ? 92/6/29

Q196 AccIs 92/6/29

Q197 AccIs 92/6/30

Q198 AccIs 92/6/30

Q199 AccIs 92/7/3

(105)

Q201 AccIs 92/7/10

Q202 AccIs 92/7/15

Q203 AccIs 92/7/16

Q204 AccIs 92/7/16

Q205 AccIs 92/7/18

Q206 AccIs

?

92/7/19

Q207 AccIs ? 92/7/21

Q208 AccIs

?

92/7/21

Q209 AccIs

?

92/7/24

Q210 AccIs 92/7/24

Q211 AccIs 92/7/26

Q212 AccIs ? 92/7/28

Q213 AccIs 92/7/29

Q214 AccIs 92/8/2

Q215 AccIs ? 92/8/3

Q216 AccLab 92/8/5

Q217 AccIs 92/8/14

Q218 AccIs 92/8/16

Q219 AccIs

?

92/8/18

Q220 AccIs 2 92/8/19

Q221 AccIs 92/8/22

(106)

Q224 AccLab

?

92/8/27

Q225 AccIs 92/8/30

Q226 AccLab ? 92/8/31

Q227 AccIs 92/9/3

Q228 AccIs "

"

92/9/4

Q229 AccIs 92/9/6

Q230 AccIs ? 92/9/8

Q231 AccLab 92/9/12

Q232 AccIs

?

92/9/16

Q233 AccIs 92/9/19

Q234 AccIs

?

92/9/23

Q235 AccIs

?

92/9/26

Q236 AccIs 92/9/26

Q237 AccIs 92/10/4

Q238 AccIs 92/10/9

Q239 AccIs 92/10/12

QP1 PR003 91/4/29

QP2 PR003 A4 91/4/30

QP3 PR003 91/5/21

QP4 PR003 91/5/22

QP5 PR003 91/5/26

(107)

QP7 PR003 91/6/26

QP8 PR003 91/6/30

QP9 PR003 91/7/13

QP10 PR003 91/7/28

QP11 PR003 91/8/3

QP12 PR003 91/8/18

QP13 PR003 91/9/3

QP14 PR003

?

91/9/11

QP15 PR003 91/9/16

QP16 PR003

?

91/9/18

QP17 PR003 91/9/23

QP18 PR003 ? 91/9/26

QP19 PR003

?

91/9/29

QP20 PR003 91/10/2

QP21 PR003 91/10/5

QP22 PR003 91/10/8

QP23 PR003 91/10/11

QP24 PR003 91/10/15

QP25 PR003 91/10/15

QP26 PR003 91/10/24

QP27 PR003 91/10/28

QP28 PR003 91/10/29

QP29 PR003 92/4/16

QP30 PR003 92/4/26

QP31 PR003 ? 92/4/29

(108)

QP33 PR003 92/5/14

QP34 PR003 ? 92/5/17

QP35 PR003 92/5/18

QP36 PR003 92/5/21

QP37 PR003 92/5/28

QP38 PR003 92/6/1

QP39 PR003 92/6/19

QP40 PR003 92/6/20

QP41 PR003 92/6/26

QP42 PR003 92/7/3

QP43 PR003

?

92/7/12

QP44 PR003 92/7/17

QP45 PR003

?

92/7/22

QP46 PR003 ? 92/7/23

QP47 PR003 92/8/6

QP48 PR003 92/8/13

QP49 PR003 92/8/16

QP50 PR003 92/8/20

QP51 PR003 ? 92/9/1

QP52 PR003

?

92/9/21 QP53 PR003

?

92/9/2

QP54 PR003 92/9/10

QP55 PR003 ”ASP

”

92/9/12

QP56 PR003 92/9/18

QP57 PR003 ? 92/9/23

QP58 PR003 92/10/2

QP59 PR003 ? 92/10/14

(109)

QP61 PR003 ? 92/11/2

(110)

B CKIP

CKIP

CKIP 1.0 10 CKIP

A V N D

P C T I

A Caa Cab Cba Cbb D Da DE Dfa Dfb Di Dk FW

I Na Nb Nc

(111)

Nd Nep Neqa Neqb Nes Neu Nf Ng Nh P SHI

T VA VAC

VB VC VCL

VD VE VF VG VH VHC

VI VJ VK VL V_2

(112)

C

A

1 16

2 17

3 18

4 19

5 20

6 21

7 22

8 23

9 24

10 25

11 26

12 27

13 28

14 29

15

(113)

D

CKIP

50 k1 k2 k17

k1 k2 k3 k4 k5 k6 k7 k8 k9 k10 k11 k12 k13 k14 k15 k16 k17

1 (Nd) (Neu) (Na) (Ng) (VC) (D) (VC)(Na)

(COM MACA TEGO

RY) (D) (VB) (Ng) (D) (VD) (A) (Nf) (Na)

2 (D) (VC) (Na) (Na) (DE) (D) (VC)(Neu) (Caa) (Na) (Na) (DE) (D) (VC)(Neu) (VH) (QUES TIONC ATEG ORY)

3 (VC) (VF) (VC) (Na) (VC) (Na) (Na) (Ng) (COM MACA TEGO

RY) (D) (VE) (VC) (Neu) (SHI) (VH) (DE) (QUES TIONC ATEG ORY)

4 (Nes) (Nf) (D) (VB) (D) (VC)(VAC) (D) (P) (VJ) (VC) COM MACA TEGO

RY (Cbb) (Da) (VE) (VAC) (D)

5 (Cbb) (VC) (PA RENT HESIS CATE GORY

) (Na) (VC) (Na) (Na) (VC)

(PARE NTHE SISCA TEGO

RY) (Cbb) (Nes) (Neu) (Nf) (Na) (Na) (COM MACA TEGO RY) (D)

6 (Na) (VC) (VC)(Na) (V_2) (VC) (VH) (Na) (Na) (DE) (Na) (COM MACA TEGO

RY) (D) (D) (VHC) (D) (VB)

7 (D) (VB) (VC) (VA) (VC) (Ng) (VC) (PA RENT HESIS CATE GORY

) (VB)

(PARE NTHE SISCA TEGO

RY) (VE) (VK) (D) (VB) (Na) (VC)(VC) (COM

MACA