國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
40
第六章. 總結
總結表 5-1 至 5-6 的結果可以發現，在挑選變數時，迴歸模型中兩自變數的相 關係數|r |越大，則 suppressor 變數被剃除的機率越小。再者，在挑選變數時，12 backward elimination 相對於 forward selection 來的好。由於我們的模擬實驗僅止 於兩個自變數的情況，因此結果無法延伸至多個自變數的情況。儘管如此，我們 還是可以注意到，當存在 suppressor 變數時，藉由 stepwise regression procedure、
forward selection 或 backward elimination 來選取變數時，未必是恰當的方式。
文獻中一個與 enhancement 與 suppression 相關的主題為迴歸模型中自變數的相 對重要性(relative importance)，而探討相對重要性的主要工具之一為 Lindeman, Merenda and Gold (1980)一文中所引進的方法，後續多半簡稱為 LMG 法。另一 種方法則是 Feldman (2005)所介紹 proportional marginal variance decomposition 的 方法、或簡稱為 PMVD 法。這兩者皆是以 semipartial correlation coefficient 的變 化，來決定相對重要變數。此外， Pratt(1987)則是利用 2
R r
dj
bj j 來作為判斷依據，這意味著迴歸係數及相關係數也可以作為判斷相對重要變數的工具。這種思 維方式，某種程度上與定義 suppressor 變數的方法是一致的。因此我們相信在相 對重要變數的探討與 suppressor 變數之間，一定存在著某些關聯，這或許值得作 為未來深思並探討的一個課題。
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
41
參考文獻
Bertrand, P.V., and Holder, R.L. (1988). “A quirk in multiple regression: The whole regression can be greater than the sum of its parts.” The Statistician, 37, 371-374.
Currie, I., and Korabinski, A. (1984). “Some comments on bivariate regression.” The Statistician, 33, 283–292.
Cohen, J., and Cohen, P. (1975). Applied multiple regression/correlation analysis for the behavioral sciences, New Jersey: Lawrence Erlbaum Associates.
Conger, A.J. (1974). “A revised definition for suppressor variables: a guide to their identification and interpretation.” Educational and Psychological Measurement, 34, 35-46
Darlington, R.B. (1968). “Multiple regression in psychological research and practice.”
Psychological Bulletin, 69,161-182
Dayton, M. (1972). “A method for constructing data which illustrate a suppressor variable.” The American Statistician, 26, 36.
Feldman, B. (2005), “Relative Importance and Value.” Unpublished manuscript(Version 1.1, March 19 2005).
Friedman, L., and Wall, M. (2005). “Graphical views of suppression and
multicollinearity in multiple linear regression.” The Educational and Psycgological Measurment, 2005, 59, 127-137.
Hamilton, D. (1987). “Sometimes
1 2
2 2 2
yx yx
R
r
r : Correlated variables are not always redundant.” The American Statistician, 41, 129-132.Holling, H. (1983). “Suppressor structures in the general linear model.” Educational and Psychological Measurement, 43, 1-9.
Horst, P. (1941). “The role of prediction variables which are independent of the criterion.” In Horst, P.(Ed.): The prediction of personal adjustment . Social Science
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
42
Research Bulletin, 48, 431-436.
Lindeman, R. H., Merenda, P. F., and Gold, R. Z. (1980). Introduction to Bivariate and Multivariate Analysis. Glenview, IL: Scott, Foresman.
Lynn, H.S. (2003). “Suppression and confounding in action.” The American Statistician, 57, 58-61.
Lutz, G. (1983). “A method for construction data which illustrate there types of suppressor variable.” Educational and Psychological Measurement, 43, 373-377.
Newton, R. G. and Spurrel, D. J. (1967). “Examples of the use of elements for clarifying regression analysis.” Applied Statistics, 16, 165-172.
Pratt, J. W. (1987). “Dividing the indivisible: Using simple symmetry to
partitionvariance explained”, in T. Pukkila and S. Puntanen (eds.), Proceedings of the Second International Conference in Statistics (University of Tampere,
Tampere,Finland) pp. 245–260.
Schey, H.M. (1993). “The relationship between the magnitudes of SSR(x2) and SSR(x2|x1): A geometric description.” The American Statistician, 47, 26-30.
Shieh, G. (2001). “The inequality between the coefficient of determination and the sum of squared simple correlation coefficients.” The American Statistician, 55, 121–124.
Shieh, G. (2006). “Suppression situation in multiple linear regression.” The Educational and Psychological Measurement, 66,435-447.
Smith, R.L., Ager, J.W., and Williams, D. L. (1992). “Suppressor variables in
multiple regression/correlation.” Educational and Psychological Measurement, 52, 17-29.
Velicer, W. (1978). “Suppressor variables and the semipartial correlation coefficient.”
Educational and Psychological Measurement, 38, 953-958.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
43
附錄
模擬程式碼如下:
n=30
x1=rnorm(n,1,3) x2=rnorm(n,2,4)
data1=data.frame(x1,x2) lin1=lm(x2~x1,data=data1) res=residuals(lin1)
T=(x1-mean(x1))/sd(x1) S=(res-mean(res))/sd(res) y=T+rnorm(n,0,1)
x=(T+S)/2 xt=S
xr=0.1*T-0.2*S xn=0.1*T+0.4*S
#stepwise regression procedure 迴圈
n=30n1=10000 k=NULL k1=NULL k2=NULL
out1=matrix(0,nrow=1,ncol=n1) out2=matrix(0,nrow=1,ncol=n1)
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
44
pv2.1=NULL pv1.2=NULL for(i in 1:n1){
x1=rnorm(n,1,3) x2=rnorm(n,2,4)
data1=data.frame(x1,x2) lin1=lm(x2~x1,data=data1) res=residuals(lin1)
T=(x1-mean(x1))/sd(x1) S=(res-mean(res))/sd(res) y=T+rnorm(n,0,1)
x=(T+S)/2 x2=0.1*T+0.4*S y=(y-mean(y))/sd(y) x=(x-mean(x))/sd(x) xr=(x2-mean(x2))/sd(x2) r1=cor(y,x)
r2=cor(y,x2) r12=cor(x,x2)
ry1.2=(r1-r2*r12)/(1-r12^2)^(0.5) ry2.1=(r2-r1*r12)/(1-r12^2)^(0.5) R2=(r1^2+r2^2-2*r1*r2*r12)/(1-r12^2) t1=(r1^2*(n-2)/(1-r1^2))^(1/2)
t2=(r2^2*(n-2)/(1-r2^2))^(1/2) k=1-pt(t1,n-2)
if(k<0.05){
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
45
t2.1=(ry2.1^2*(n-3)/(1-R2))^(1/2) k2=1-pt(t2.1,n-3)
pv2.1=c(pv2.1,k2) if(k2<0.05) { out2[1,i]=1
t1.2=(ry1.2^2*(n-3)/(1-R2))^(1/2) k1=1-pt(t1.2,n-3)
pv1.2=c(pv1.2,k1)
if(k1<0.05) out1[1,i]=1 else out1[1,i]=0 }
else out2[1,i]=0 }
}
length(which(out1==1))/length(which(out2==1))
length(which(out2==1))/length(pv2.1)
#forward selection
n=30n1=10000 k=NULL k1=NULL k2=NULL
out2=matrix(0,nrow=1,ncol=n1) pv2.1=NULL
pv1.2=NULL
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
46
for(i in 1:n1){
x1=rnorm(n,1,3) x2=rnorm(n,2,4)
data1=data.frame(x1,x2) lin1=lm(x2~x1,data=data1) res=residuals(lin1)
T=(x1-mean(x1))/sd(x1) S=(res-mean(res))/sd(res) y=T+rnorm(n,0,1)
x=(T+S)/2 x2=0.1*T+0.4*S y=(y-mean(y))/sd(y) x=(x-mean(x))/sd(x) xr=(x2-mean(x2))/sd(x2) r1=cor(y,x)
r2=cor(y,x2) r12=cor(x,x2)
ry1.2=(r1-r2*r12)/(1-r12^2)^(0.5) ry2.1=(r2-r1*r12)/(1-r12^2)^(0.5) R2=(r1^2+r2^2-2*r1*r2*r12)/(1-r12^2) t1=(r1^2*(n-2)/(1-r1^2))^(1/2)
t2=(r2^2*(n-2)/(1-r2^2))^(1/2) k=1-pt(t1,n-2)
if(k<0.05){
t2.1=(ry2.1^2*(n-3)/(1-R2))^(1/2) k2=1-pt(t2.1,n-3)
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
47
pv2.1=c(pv2.1,k2) if(k2<0.05) out2[1,i]=1 else out2[1,i]=0 }
}
length(pv2.1)/n1
length(which(out2==1))/length(pv2.1)
#backward elimination
n=30n1=10000 k=NULL k1=NULL k2=NULL k1.2=NULL k2.1=NULL pv2.1=NULL pv1.2=NULL pv1=NULL pv2=NULL for(i in 1:n1){
x1=rnorm(n,1,3) x2=rnorm(n,2,4)
data1=data.frame(x1,x2) lin1=lm(x2~x1,data=data1) res=residuals(lin1)
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
48
T=(x1-mean(x1))/sd(x1) S=(res-mean(res))/sd(res) y=T+rnorm(n,0,1)
x=(T+S)/2 x2=0.1*T+0.4*S
y=(y-mean(y))/sd(y) x=(x-mean(x))/sd(x) xr=(x2-mean(x2))/sd(x2) r1=cor(y,x)
r2=cor(y,x2) r12=cor(x,x2)
R2=(r1^2+r2^2-2*r1*r2*r12)/(1-r12^2) ry1.2=(r1-r2*r12)/(1-r12^2)^(0.5) ry2.1=(r2-r1*r12)/(1-r12^2)^(0.5) t1.2=(ry1.2^2*(n-3)/(1-R2))^(1/2) k1.2=1-pt(t1.2,n-3)
t2.1=(ry2.1^2*(n-3)/(1-R2))^(1/2) k2.1=1-pt(t2.1,n-3)
t1=(r1^2*(n-2)/(1-r1^2))^(1/2) t2=(r2^2*(n-2)/(1-r2^2))^(1/2) k1=1-pt(t1,n-2)
k2=1-pt(t2,n-2)
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
49
if(k1.2<k2.1){
if(k2.1<0.05)
pv2.1=c(pv2.1,k2.1) else if(k1<0.05) pv1=c(pv1,k1)
}
if(k1.2>k2.1){
if(k1.2<0.05)
pv1.2=c(pv1.2,k1.2) else if(k2<0.05) pv2=c(pv2,k2) }
}