CHAPTER 2 LITERATURE REVIEW
2.3 Envelope Module
國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
same time and explore an acceptable nonlinear relationship between explanatory variables and the response in the presence of outliers.
2.3 Envelope Module
Huang et al. (2014) propose an envelope bulk mechanism integrated with the SLFN to cope with outlier detection problem. This outlier detection algorithm is performed with an envelope bulk whose width is 2𝜀𝜀. The 𝜀𝜀 is changed from a tiny value (10−6) to a non-tiny value (1.96) due to the envelope module. The value changes to 1.96 similarly according to the 5% significance level in given the distribution is normal. The standard to distinguish whether the instance is outlier or not is the instance’s residual is greater than ε ∗ γ ∗ σ, where σ is the standard deviation of the residual of the current reference observations and γ is a constant that is equal to or greater than 1.0, depending on the user’s stringency in the outlier detection. The smaller the γ value is, the more stringent the outlier detection is.
Furthermore, if our requirements are stricter, we also can modify the ε value to an appropriate value.
In brief, this envelope module allow us to wrap the response elements seen as inliers in the envelope. Vice versa, the response as outliers won’t wrap in the envelope.
The quantity of the inliers is decided by the ε and γ. The stricter parameter is, the less inliers inside the envelope. In other aspect of outliers, there will be more potential outliers determined by the envelope module.
26
‧
Table 2:The outlier detection with the envelope module (Adapted from Huang et al., 2014). There are N training data.
Step 1: Use the first m+1 reference observations in the training data set to set up an acceptable SLFN estimate with one hidden node. Set n = m+2.
Step 2: If n > N*(1 – k), STOP.
Step 3.1: Use the obtained SLFN to calculate the squared residuals regarding all N training data.
Step 3.2: Present the n reference observations (xc, yc) that are the ones with the smallest n squared residuals among the current squared residuals of all N training data.
Step 4: If all of the smallest n squared residuals are less than ε (the envelope width), then go to Step 7; otherwise, there is one and only one squared residual that is larger than ε.
Step 5: Set 𝐰𝐰� = 𝐰𝐰.
Step 6: Apply the gradient descent mechanism to adjust weights w of SLFN. Use the obtained SLFN to calculate the squared residuals regarding all training data. Then, either one of the following two cases occurs:
(1) If the envelope of obtained SLFN does contain at least n observations, then go to Step 7.
(2) If the envelope of obtained SLFN does not contain at least n observations, then set 𝐰𝐰 = 𝐰𝐰� and apply the augmenting mechanism to add extra hidden nodes to obtain an acceptable SLFN estimate.
Step 7: Implement the pruning mechanism to delete all of the potentially irrelevant hidden nodes; n + 1 n; go to Step 2.
As stated in Huang et al. (2014), the envelope module in Table 2 executes the following procedure:
In step 1, we try to use the first set up an acceptable SLFN. Then set n = m+2.
In step 2, there is a stopping criteria. In this adapted version, where k can be referred to the percentage of potential outlier. Clearly, at least (1-k) % data will be
27
‧
國立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
wrapped into the envelope. For example, if there is approximately at least 95%
non-outliers and at most 5% outliers, the SLFN will take 95% data into consideration while building the SLFN.
In step 3, we try to calculate the squared residuals and determines the input sequence of reference observations in this stage. Furthermore, the input sequence is determine by the residual between the observations with the current SLFN which has already learned n -1 data. The squared residuals will be calculated in every stage, and the input sequence of reference observations will changed according to the squared residuals.
The modeling procedure implemented by Step 6 and Step 7 that adjusts the number of hidden nodes adopted in the SLFN estimate and the associated w to evolve the fitting function f and its envelope to contain at least n observations at the nth stage.
That is, at the nth stage, Step 3 presents the n reference observations that are the observations with the smallest n squared residuals among the current N squared residuals and are used to evolve the fitting function. Step 3 adopts the concept of forward selection, ordering the residuals of all N observations and then augmenting the reference subset gradually by including extra observations one by one to determine the input sequence of the reference observations. With this, some of the reference observations at the early stages might not stay in the set of reference observations at the later stages, although most of them will.
The modeling procedure implemented by Step 6 to Step 7 requires proper values of w and p so that the obtained envelope contains at least n observations at the end of the nth stage. Specifically, at the beginning of Step 6, the gradient descent mechanism
28
‧
Step 5. Thus, we return to the previous SLFN estimate, and then the augmenting mechanism proposed by Tsaih and Cheng (2009) recruit two extra hidden nodes to obtain an acceptable SLFN estimate. In order to decrease the complexity of the fitting function f, the reasoning mechanism (Windham, 1995) is proposed in Step 7 to delete potentially irrelevant hidden nodes.The envelope module results in a fitting function with an envelope that includes almost the majority of training data, and the outliers are expected to be included at later stages. In this study, we name the instances in this stage as potential outliers.
Then, we use the deviance information as the extra information to define whether the potential outliers is need to be the regarded as outlier candidates. Specifically, here we adopt both the deviance information and the order information to identify the potential outliers.
Regarding the order information for identifying the outliers, we treat the last k%
data as potential outliers. Namely, if n ≥ N *(1 - k) AND the residual is greater than ε, then this data is recorded as the outlier candidate.
The setting of the ε value depends on the user’s perception of the data and its associated outliers. For example, the perceptions are that the error is normally distributed, with a mean of 0 and a variance of 1, and the outliers are the points that have residuals that are greater than 1.96 (when the absolute value is taken). These perceptions are similar to the setting in the regression analysis that corresponds to a 5% significance level. Given that the error terms follow the normal distribution. Then, the user can set the ε value of the proposed envelope module to 1.96 and define the outliers as the points that have residuals that are greater than 1.96.
29
‧
國立 政 治 大 學