An Enhanced HMM-Based for Fuzzy Time Series Forecasting Model

(1)

An Enhanced HMM-Based for Fuzzy Time Series Forecasting Model

(Yi-Chung Cheng, Pei-Chih Chen, Chih-Chuan Chen, Hui-Chi Chuang, Sheng-Tun Li)

1

(2)

Presentation Outline

Introduction

Preliminaries and Related Work Model Development

Experiment Results

Conclusion

(3)

Introduction (1/5)

In the big data era, an efficiency forecasting model is very important.

3

Traditional time series is complete developing statistic method, but it can’t deal with vague, vocabulary, or uncertainly data.

Zadeh (1965) proposed the fuzzy theory, which closed

to human think and can description vague and

vocabulary variables.

(4)

Introduction (2/5)

Variety of data type used which can not be solved in traditional statistical methods

12000 14000 16000 18000 20000

high middle

low very low

Linguistic data information: a little cold , old, degree of preference…

Ordinal scale data type: signal, business indicators…

(5)

Introduction (3/5)

5

(6)

12000 14000 16000 18000 20000

1971 1976 1981 1986 1991

Type Scholars

One factor Chen, 1996; Chen & Hsu, 2004; K. Huarng, 2001;S. T. Li & Cheng, 2007, 2009; Song & Chissom, 1993a, 1993b, 1994; Sullivan & Woodall, 1994; Tsaur, Yang, & Wang, 2005

Two factor Chen & Hwang, 2000; Hsu et al., 2003; L. W. Lee et al., 2006; S. T. Li & Cheng, 2010

Multi- factor K. H. Huarng, Yu, & Hsu, 2007

One order Chen, 1996; Chen & Hsu, 2004; Hsu et al., 2003; K. Huarng, 2001; K. H. Huarng et al., 2007; S. T. Li

& Cheng, 2010;Song & Chissom, 1993a, 1993b, 1994; Sullivan & Woodall, 1994; Tsaur et al., 2005

High order

12000 14000 16000 18000 20000

1971 1976 1981 1986 1991

Main factor Second factor

Introduction (4/5)

(7)

Introduction (5/5)

Issues discuss

Traditional fuzzy time series forecasting method cannot forecast with multiple factor data and waste the obtained information.

An event can be affected by many factors, therefore, we present a multi-factor HMM-based forecasting, and utilize more factors to get better forecasting accuracy rate.

7

(8)

Presentation Outline

Introduction

Preliminaries and Related Work

Model Development

Experiment Results

Conclusion

(9)

Fuzzy Time Series (1/3)

Fuzzy Time Series F(t)

is a time series Y t t( )( =1, 2,)

is a fuzzy set which depict the linguistic variable of Y(t) fi

( )(

t t =^{1, 2,}

)

F(t) is composed of the series of 𝑓𝑓_𝑖𝑖 𝑡𝑡

( ) 100,100, 200, 300,

Y t = 

1 1 2 3

( ) , , , , F t = A A A A  Fuzzy Logical Relation (R)

Time Variant: R will change by time

Time Invariant: R will not change by time

3

1 2

1 1 2 3

R

R R

A → →A A → A

1 1 2 3

R R R

A → →A A → A

9

(10)

Definition: first-order model of F(t)

Suppose F(t) is caused by F(t-1) only or by F(t-1) or F(t-2) or…or F(t-k), k>0. This relation is denoted as follows:

𝐹𝐹 𝑡𝑡 = (𝐹𝐹 𝑡𝑡 − 1 ⋃𝐹𝐹 𝑡𝑡 − 2 ⋃ … ⋃𝐹𝐹(𝑡𝑡 − 𝑘𝑘)) ∘ 𝑅𝑅_∘(𝑡𝑡, 𝑡𝑡 − 𝑘𝑘)

where ‘⋃’ is the union operator and ‘∘’ is the composition. 𝑅𝑅_∘(𝑡𝑡, 𝑡𝑡 − 𝑘𝑘) is fuzzy relationship between F(t) and F(t-1) or F(t-2)…or F(t-k).

Definition: 𝑲𝑲^{𝒕𝒕𝒕𝒕} order model of F(t)

Suppose that F(t) is caused by F(t-1), F(t-2),…, and F(t-k) (k>0) simultaneously, then their relations can be represented as:

𝐹𝐹 𝑡𝑡 = (𝐹𝐹 𝑡𝑡 − 1 × 𝐹𝐹 𝑡𝑡 − 2 × ⋯ × 𝐹𝐹(𝑡𝑡 − 𝑘𝑘)) ∘ 𝑅𝑅_𝑎𝑎(𝑡𝑡, 𝑡𝑡 − 𝑘𝑘) where 𝑅𝑅_𝑎𝑎 𝑡𝑡, 𝑡𝑡 − 𝑘𝑘 is relation matrix expressing the fuzzy relationship between F(t) and F(t-1), F(t-2),…, F(t-k).

Fuzzy Time Series (2/3)

(11)

Fuzzy Time Series (3/3)

Song and Chissom (1993) first proposed a complete fuzzy time series forecasting model and divided it into 7 steps.

11

(1) defining universe of discourse

(2) partitioning into several intervals

(3) defining fuzzy sets and linguistic values (4) fuzzifying historical data

(5) building up fuzzy relation (6) linguistic forecasting

(7) defuzzification

(12)

Hidden Markov Model (1/4)

Hidden Markov model (HMM) is a statistical model used to deal with symbols or signal sequences.

HMM is composed of two states and three probability matrices.

Two states are hidden state and observable state.

The hidden states set is defined as

The observable states set is defined as

(13)

Hidden Markov Model (2/4)

The three probability matrices are used to describe the relation between hidden and observable states and normally represented as

13

π is initial state vector, A is hidden state transition matrix, and B is confusion matrix with hidden and observable state.

π , A and B can be defined mathematically as follows:

,

(14)

Hidden Markov Model (3/4)

,

(15)

Hidden Markov Model (4/4)

The probability is the sum of with all possible state sequences, so we obtain:

15

,

(16)

Presentation Outline

Introduction

Preliminaries and Related Work Model Development

Experiment Results

Conclusion

(17)

Research framework

17

Finding the Research Topic

Literature Review

Fuzzy Time Series

Forecasting Method

Hidden Markov Model

HMM Forecasting with Multiple Factor Based on Fuzzy Time Series

Fuzzifying historical data of significant variables

Building HMM model

(18)

Model process(1/7)

Step1. Define the universe of discourse U

Find the minimal and maximal values in the historical data

Where 𝐷𝐷_{𝑚𝑚𝑖𝑖𝑚𝑚} and 𝐷𝐷_{𝑚𝑚𝑎𝑎𝑚𝑚} are the minimum and the maximum in the training dataset, and 𝐷𝐷₁, 𝐷𝐷₂ are the two proper positive integers.

12000 14000 16000 18000 20000

1971 1976 1981 1986 1991

For hidden and observable states, the universal of discourse U is made as follows:

(19)

Step2. Partition the universe of discourse U

Use equal length method to define the interval length

19

Model process(2/7)

(20)

Step3. Define the fuzzy sets

For hidden states, n fuzzy sets can be defined as follows

Step4. Fuzzify the time series

The hidden historical datum is fuzzified as where the membership degree in interval is maximal.

where is the membership degree of belonging to .

Model process(3/7)

For k observable states, fuzzy sets ,…, and can be defined as below:

=

where is the membership degree of belonging to .

(21)

Step5. Constructing an HMM model

The initial state vector π is set to be a 1 x n matrix.

The state transition matrix is a n x n matrix

21

where is the number of the data whose initial states are and .

Model process(4/7)

with and ,where denotes that the number of data whose hidden states is h_i at time t-1 and h_j at time t .

(22)

Model process(5/7)

The confusion matrix is a matrix

The multiple observations HMM can be characterized by the following matrices:

with and ,where means that the number of data whose hidden states is h_i at time t and o_j at time t .

(23)

23

Model process(6/7)

We use dynamic programming to calculate maximum likelihood.

A particular HMM can be characterized by and .

Step6. Forecasting

(24)

Step7. Defuzzification

We conduct fuzzy mean method to defuzzify the fuzzy result, which is expressed as: FM C = ^∑_∑^𝑁𝑁^𝑖𝑖=1 ^𝜇𝜇_𝜇𝜇^𝑖𝑖^𝐶𝐶^𝑖𝑖

𝑁𝑁 𝑖𝑖

𝑖𝑖=1

Model process(7/7)

𝑁𝑁：the amount of fuzzy set;

𝜇𝜇_𝑖𝑖：the 𝑖𝑖^𝑡𝑡𝑡membership degree;

𝐶𝐶_𝑖𝑖：the 𝑖𝑖^𝑡𝑡𝑡 midpoint of interval corresponding to the 𝑖𝑖^𝑡𝑡𝑡 linguistic value

(25)

Presentation Outline

Introduction

Preliminaries and Related Work Model Development

Experiment Results

25

Conclusion

(26)

Experiment and Analysis (1/4)

Demonstrated Dataset

Alishan weather data (from 2004 to 2013)

we use four evaluation indices to evaluate the performance and the calculation is display as MAE, PMAD, MAPE, RMSE.

Contain one hidden factor : average temperature,

three observable factors : (1) average relative humidity level (2) number of rainy days

(3) total sunshine duration.

Training data : 2004 to 2010 Testing data : 2011 to 2013

(27)

27

Experiment and Analysis (2/4)

MAE (Mean Absolute Error)

PMAD (Percentage Mean Absolute Deviation)

MAPE (Mean Absolute Percentage Error)

RMSE (Root Mean Squared Error)

(28)

Compare model

1. Chen (1996)

Forecasting enrollments based on fuzzy time series.

2. Hsu et al. (2003)

A new approach of bivariate fuzzy time series analysis to the forecasting of a stock index.

3. Li & Cheng (2007)

Deterministic fuzzy time series model for forecasting enrollments.

4. Chen (2011)

Handling forecasting problems based on high-order fuzzy logical relationships.

(29)

29

MAE PMAD MAPE RMSE

All 0.877112 0.092451 0.092451 1.350553 Average humidity level 1.23298 0.124322 0.124322 1.699685 Number of rainy days 0.992742 0.106911 0.106911 1.405663 Total sunshine duration 1.170567 0.117814 0.117814 1.577392

Experiment and Analysis (3/4)

The evaluation of Alishan weather with different factors

Forecasting with multi-observables is better than single observable

(30)

6 8 10 12 14 16

Average temperature

Real Data Proposed Model

MAE RMSE PMAD MAPE

Proposed Model 1.0496 1.3937 0.0927 0.1097 Chen(1996) 1.3159 1.6584 0.1145 0.1316 Hsu et al. (2003) 1.2397 1.5269 0.1079 0.1251 Li & Cheng (2007) 2.0443 2.7463 0.1779 0.2078

Experiment and Analysis (4/4)

The evaluation of Alishan weather

(31)

Presentation Outline

Introduction

Preliminaries and Related Work Model Development

Experiment Results

31

Conclusion

(32)

Conclusion

Multi-observables HMM model - forecasting with multi- observables is better than single observable and traditional fuzzy time series forecasting model

The previous study focused on constructing relationship by Markov or HMM, the drawback is only deal with one factor or one hidden variable and one observable variable.

The proposed model solves the problem and demonstrates the indicators that “predicting with more factors can

improve the forecasting result”.

(33)

33