3.3 Simulation Studies
3.3.2 Phase I Application
The simulation results for Phase I analysis are presented in this section. Since the distribution of the profile data is assumed to be multivariate normal and the design points are the same for all profiles, multivariate control charts could be utilized directly. One of the charts is the well-known Hotelling’s T2 control chart.
For a sample{y1, . . . , ym} from a p-variate normal distribution, the Hotelling’s T2
Table 3.1: The shifts in mean and/or variance-covariance ma-trix of the OC models
1.0 1.5 2.0 2.5 3.0 3.5
Figure 3.2: Plots of IC and OC samples from Model (a)(top row), (b)(middle row) and (c)(bottom row).
statistic is defined as
Ti2 = (yi− ¯y)′B−1(yi− ¯y) for i = 1, . . . , m,
where ¯y and B are the sample mean and sample variance-covariance matrix, re-spectively. In Phase I analysis, the T2 statistic is approximately distributed as a beta distribution (Tracy et al., 1992), so the control limit can be conveniently set at
(m− 1)2
βα,p/2,(m−p−1)/2,
Table 3.2: The type-I and type-II error rates and their standard errors (in parentheses) of OC Model (a) for α = 0.05 and δ2 = 0.
pI pII
δ1 T2 CS(2) CS(3) CS(4) CS(5) T2 CS(2) CS(3) CS(4) CS(5)
0.625 0.0749 0.0722 0.0692 0.0706 0.0683 0.9162 0.9095 0.9155 0.9168 0.9210 (.0005) (.0005) (.0005) (.0005) (.0004) (.0013) (.0014) (.0013) (.0013) (.0013) 1.250 0.0735 0.0688 0.0677 0.0682 0.0666 0.8976 0.8552 0.8765 0.8869 0.8967 (.0005) (.0005) (.0004) (.0005) (.0004) (.0014) (.0018) (.0016) (.0015) (.0014) 1.875 0.0709 0.0645 0.0643 0.0654 0.0644 0.8711 0.7561 0.8082 0.8348 0.8514 (.0005) (.0005) (.0004) (.0005) (.0004) (.0016) (.0024) (.0021) (.0019) (.0018) 2.500 0.0704 0.0642 0.0624 0.0636 0.0629 0.8347 0.5908 0.6944 0.7479 0.7836 (.0005) (.0005) (.0004) (.0005) (.0004) (.0019) (.0033) (.0030) (.0027) (.0024) 3.125 0.0680 0.0677 0.0642 0.0639 0.0629 0.7935 0.3343 0.4850 0.6030 0.6827 (.0005) (.0005) (.0004) (.0005) (.0004) (.0023) (.0035) (.0041) (.0039) (.0035) 3.750 0.0672 0.0727 0.0687 0.0675 0.0646 0.7466 0.1174 0.2091 0.3313 0.4693 (.0005) (.0005) (.0005) (.0005) (.0004) (.0028) (.0021) (.0035) (.0051) (.0054) 4.375 0.0667 0.0759 0.0730 0.0729 0.0698 0.6885 0.0295 0.0557 0.0950 0.1689 (.0005) (.0005) (.0005) (.0005) (.0005) (.0037) (.0009) (.0014) (.0024) (.0044) 5.000 0.0669 0.0771 0.0742 0.0744 0.0727 0.6057 0.0066 0.0130 0.0210 0.0344 (.0005) (.0005) (.0005) (.0005) (.0005) (.0058) (.0004) (.0006) (.0008) (.0014)
where α is the nominal type-I error probability, and βα,a,b is the upper α quantile of the beta distribution with shape parameters a and b. The observations with values of T2 statistic larger than the control limit are considered as OC cases.
Suppose the IC profiles are from Np(µ, Σ), where µ and Σ are given in (3.12) and (3.13), respectively, and the OC profiles are from the models described in Table 3.1. Various values of δ1 and δ2 are considered. For each scenario, a sample of 450 IC and 50 OC profiles is generated. Let the false-alarm rate α = 0.05. The type-I and type-II error rates accompanied with their standard errors (in the parentheses) for the six OC conditions are summarized in Tables 3.2 - 3.7, respectively. Each value of error rates is obtained by averaging 1,000 replications.
Table 3.3: The type-I and type-II error rates and their standard errors (in parentheses) of OC Model (a) for α = 0.05 and δ1 = 0.
pI pII
δ2 T2 CS(2) CS(3) CS(4) CS(5) T2 CS(2) CS(3) CS(4) CS(5)
0.875 0.0721 0.0688 0.0668 0.0676 0.0663 0.8507 0.8045 0.7928 0.8082 0.8246 (.0005) (.0005) (.0005) (.0005) (.0004) (.0016) (.0019) (.0020) (.0019) (.0018) 1.750 0.0719 0.0691 0.0676 0.0682 0.0665 0.7812 0.7016 0.6761 0.7007 0.7240 (.0005) (.0005) (.0004) (.0005) (.0005) (.0020) (.0022) (.0022) (.0023) (.0022) 2.625 0.0713 0.0685 0.0677 0.0682 0.0664 0.7265 0.6314 0.6014 0.6285 0.6533 (.0005) (.0005) (.0005) (.0005) (.0004) (.0022) (.0023) (.0023) (.0023) (.0023) 3.500 0.0724 0.0698 0.0681 0.0681 0.0666 0.6744 0.5759 0.5427 0.5694 0.5951 (.0005) (.0005) (.0005) (.0005) (.0004) (.0024) (.0023) (.0024) (.0024) (.0024) 4.375 0.0721 0.0698 0.0686 0.0687 0.0679 0.6283 0.5312 0.4965 0.5220 0.5459 (.0005) (.0005) (.0005) (.0005) (.0005) (.0025) (.0024) (.0025) (.0025) (.0025) 5.250 0.0720 0.0698 0.0684 0.0686 0.0679 0.5913 0.4941 0.4639 0.4867 0.5053 (.0005) (.0005) (.0005) (.0005) (.0004) (.0026) (.0024) (.0025) (.0025) (.0025) 6.125 0.0721 0.0706 0.0691 0.0693 0.0676 0.5619 0.4641 0.4320 0.4551 0.4767 (.0005) (.0005) (.0005) (.0005) (.0004) (.0026) (.0024) (.0024) (.0024) (.0024) 7.000 0.0726 0.0712 0.0696 0.0698 0.0678 0.5327 0.4403 0.4105 0.4313 0.4526 (.0005 ) (.0005) (.0005) (.0005) (.0005) (.0026) (.0023) (.0023) (.0024) (.0024)
Consider the OC Model (a), the type-I and type-II error rates of applying the Hotelling’s T2 control chart and CS chart are given in Tables 3.2 and 3.3. The value in the parentheses of the CS chart is the number of the chosen PCs. It is noticed that the type-I error rates of both Hotelling’s T2 and CS charts are a little bit larger than the nominal value 0.05. We figure that this is mainly caused by the iterative procedure of removing the observations exceeding the trial control limit from the historical dataset. Note that, in each iteration, each IC observation has a certain probability to be removed, hence the false-alarm rate would accumulate and exceed the nominal value as the iteration proceeds. The large size of our
Table 3.4: The type-I and type-II error rates and their standard errors (in parentheses) of OC Model (b) for α = 0.05 and δ2 = 0.
pI pII
δ1 T2 CS(2) CS(3) CS(4) CS(5) T2 CS(2) CS(3) CS(4) CS(5)
0.625 0.0736 0.0727 0.0694 0.0674 0.0670 0.9030 0.9128 0.9156 0.9011 0.9065 (.0005) (.0005) (.0005) (.0005) (.0004) (.0014) (.0013) (.0013) (.0014) (.0014) 1.250 0.0705 0.0714 0.0682 0.0634 0.0637 0.8561 0.8860 0.8871 0.8163 0.8337 (.0005) (.0005) (.0004) (.0004) (.0004) (.0018) (.0015) (.0015) (.0021) (.0020) 1.875 0.0673 0.0691 0.0657 0.0620 0.0623 0.7848 0.8490 0.8425 0.6187 0.6811 (.0005) (.0005) (.0004) (.0004) (.0004) (.0023) (.0019) (.0019) (.0037) (.0034) 2.500 0.0666 0.0686 0.0632 0.0687 0.0677 0.6867 0.8065 0.7836 0.1924 0.2733 (.0005) (.0005) (.0004) (.0005) (.0005) (.0037) (.0024) (.0026) (.0035) (.0053) 3.125 0.0686 0.0685 0.0647 0.0761 0.0749 0.4950 0.7613 0.6071 0.0255 0.0302 (.0005) (.0005) (.0005) (.0005) (.0005) (.0083) (.0034) (.0056) (.0008) (.0009) 3.750 0.0747 0.0687 0.0755 0.0763 0.0752 0.2183 0.6852 0.1199 0.0018 0.0020 (.0006) (.0005) (.0005) (.0005) (.0005) (.0099) (.0070) (.0067) (.0002) (.0002) 4.375 0.0772 0.0696 0.0780 0.0769 0.0752 0.0941 0.5971 0.0008 0.0000 0.0000 (.0005) (.0005) (.0005) (.0005) (.0005) (.0074) (.0096) (.0004) (.0000) (.0000) 5.000 0.0783 0.0705 0.0784 0.0764 0.0751 0.0523 0.5202 0.0000 0.0000 0.0000 (.0005) (.0005) (.0005) (.0005) (.0005) (.0058) (.0109) (.0000) (.0000) (.0000)
false-alarm rate, because a large sample size results in more iterations. Apart from this, the type-I error rates of both charts are steadily around 0.07 as the size of the shift increases gradually. In addition, the type-I error rates of the CS chart are similar in spite of the different number of PCs. Tables 3.4 - 3.7 present the simulation results of OC Models (b) and (c). The type-I error rates are similar to that of Model (a).
Tables A.1 - A.3 in Appendix A.1 give the proportion of the total variation explained by K PCs for K = 1, . . . , 4. According to the principle of parsimonious-ness in choosing K, one should choose the minimum number of the PCs for which the total variation explained has reached a prespecified satisfactory level. If we set
the satisfactory level at 95%, then for Model (a), one may select K = 2 for most of cases under δ1 = 0 or δ2 = 0. However, K = 2 does not perform the best in terms of the type-II error rates for the cases under δ1 = 0. Moreover, choosing larger K does not necessarily give a better result for type-II error rates. The reason is: if we include some additional PCs that are more than needed, the superfluous PCs would dilute the significance of the meaningful PCs in the charting statistic T02; and since the chosen K usually is not large, a few superfluous PC scores would reduce the power of the T02 part of the CS chart significantly. Meanwhile, tak-ing away few of the PCs from the complementary space has a little effect to the T12 statistic since the number of the “non-effective” PCs is often large. On the other hand, if we choose a K too small such that the primary space is not large enough to approximate the IC profiles well then the power of the CS chart would be reduced. Therefore, choosing an appropriate K is an important issue. Since Phase I analysis is an off-line operation, practitioners can try various values of K and inspect the OC cases detected carefully, then pick the one that gives the most reasonable results. For the example of Model (a), one may choose K = 2 or 3 based on the parsimoniousness principle, but can also try K = 2, 3 or 4 to see which one is better.
For type-II error rates, the CS chart outperforms the Hotelling’s T2 chart in most of the OC conditions regardless of the chosen K when the OC cases are generated from Model (a). That is because the charting statistic of the Hotelling’s T2 chart puts equal weights on all the design points. Therefore, it detects the shift in a particular direction less efficiently. That is, the Hotelling’s T2 chart sacrifices the detecting power to trade for a comprehensive monitoring on shifts in all dimensions. On the other hand, the CS chart puts more weights on the effective PCs thus it combines the information in a more efficient way if the shift is mainly in the primary space.
Consider the OC cases from Model (b), when the shift is in the complementary space, choosing too small K, say 2, gives the CS chart a poor ability in detecting
Table 3.5: The type-I and type-II error rates and their standard errors (in parentheses) of OC Model (b) for α = 0.05 and δ1 = 0.
pI pII
δ2 T2 CS(2) CS(3) CS(4) CS(5) T2 CS(2) CS(3) CS(4) CS(5)
0.875 0.0723 0.0719 0.0682 0.0665 0.0661 0.8497 0.8756 0.8769 0.8271 0.8361 (.0005) (.0005) (.0005) (.0005) (.0005) (.0016) (.0015) (.0015) (.0019) (.0018) 1.750 0.0719 0.0718 0.0686 0.0666 0.0661 0.7832 0.8207 0.8205 0.7359 0.7506 (.0005) (.0005) (.0005) (.0005) (.0004) (.0020) (.0019) (.0018) (.0021) (.0021) 2.6250 0.0714 0.0708 0.0683 0.0665 0.0663 0.7251 0.7731 0.7700 0.6629 0.6805 (.0005) (.0005) (.0005) (.0004) (.0004) (.0023) (.0021) (.0022) (.0023) (.0023) 3.500 0.0719 0.0717 0.0684 0.0664 0.0666 0.6740 0.7282 0.7257 0.6048 0.6212 (.0005) (.0005) (.0005) (.0005) (.0005) (.0024) (.0023) (.0024) (.0024) (.0024) 4.375 0.0718 0.0718 0.0686 0.0677 0.0674 0.6290 0.6862 0.6819 0.5562 0.5747 (.0005) (.0005) (.0005) (.0004) (.0005) (.0025) (.0025) (.0025) (.0025) (.0024) 5.250 0.0721 0.0712 0.0684 0.0676 0.0676 0.5931 0.6501 0.6463 0.5247 0.5390 (.0005) (.0005) (.0005) (.0005) (.0005) (.0025) (.0024) (.0025) (.0025) (.0025) 6.125 0.0721 0.0717 0.0693 0.0684 0.0675 0.5611 0.6187 0.6103 0.4919 0.5068 (.0005) (.0005) (.0005) (.0005) (.0004) (.0026) (.0025) (.0026) (.0025) (.0025) 7.000 0.0724 0.0717 0.0695 0.0686 0.0679 0.5317 0.5875 0.5798 0.4665 0.4819 (.0005) (.0005) (.0005) (.0005) (.0005) (.0026) (.0026) (.0027) (.0023) (.0024)
OC observations. However, it can be improved through choosing an appropriate K (choose K ≥ 4 in the case of δ1 = 0 or δ2 = 0).
Consider the Model (c), the CS chart performs better than the Hotelling’s T2 chart for the mean and variance-covariance matrix shifts when choosing K = 2∼ 5 and K ≥ 4, respectively. Since the shift is in both of the primary and complemen-tary spaces, the CS chart is quite sensitive in detecting the OC observations.
To sum up, generally, for a given type-I error rate, one can obtain a better results in terms of type-II error rate than the Hotelling’s T2 chart by using the CS chart if the number of effective PCs is selected appropriately in Phase I applica-tions.
Table 3.6: The type-I and type-II error rates and their standard errors (in parentheses) of OC Model (c) for α = 0.05 and δ2 = 0.
pI pII
δ1 T2 CS(2) CS(3) CS(4) CS(5) T2 CS(2) CS(3) CS(4) CS(5)
0.625 0.0736 0.0716 0.0684 0.0675 0.0668 0.9026 0.9009 0.9084 0.8993 0.9055 (.0005) (.0005) (.0005) (.0005) (.0004) (.0014) (.0014) (.0014) (.0014) (.0014) 1.250 0.0704 0.0666 0.0655 0.0634 0.0635 0.8539 0.8303 0.8526 0.8089 0.8298 (.0005) (.0005) (.0004) (.0004) (.0004) (.0018) (.0019) (.0018) (.0022) (.0021) 1.875 0.0675 0.0620 0.0614 0.0624 0.0618 0.7836 0.7098 0.7599 0.6023 0.6730 (.0005) (.0004) (.0004) (.0004) (.0004) (.0023) (.0027) (.0024) (.0040) (.0034) 2.500 0.0669 0.0629 0.0601 0.0701 0.0683 0.6872 0.4935 0.6058 0.1562 0.2440 (.0005) (.0005) (.0004) (.0005) (.0005) (.0039) (.0041) (.0038) (.0033) (.0052) 3.125 0.0685 0.0704 0.0652 0.0761 0.0747 0.4992 0.1580 0.2886 0.0148 0.0197 (.0005) (.0005) (.0005) (.0005) (.0005) (.0080) (.0038) (.0054) (.0006) (.0007) 3.750 0.0752 0.0764 0.0742 0.0760 0.0743 0.2125 0.0103 0.0194 0.0007 0.0010 (.0006) (.0005) (.0005) (.0005) (.0005) (.0098) (.0006) (.0013) (.0001) (.0001) 4.375 0.0774 0.0775 0.0755 0.0764 0.0741 0.0928 0.0006 0.0008 0.0000 0.0000 (.0005) (.0005) (.0005) (.0005) (.0005) (.0074) (.0001) (.0001) (.0000) (.0000) 5.000 0.0783 0.0778 0.0756 0.0759 0.0742 0.0518 0.0000 0.0000 0.0000 0.0000 (.0005) (.0005) (.0005) (.0005) (.0005) (.0059) (.0000) (.0000) (.0000) (.0000)