The impact of ignoring random features of predictor and moderator variables on sample size for precise interval estimation of interaction effects

(1)

The impact of ignoring random features of predictor

and moderator variables on sample size for precise interval

estimation of interaction effects

Gwowen Shieh

Published online: 7 May 2011 # Psychonomic Society, Inc. 2011

Abstract The influence of the joint distribution of predictor and moderator variables on the identification of interactions has been well described, but the impact on sample size determinations has received rather limited attention within the framework of moderated multiple regression (MMR). This article investigates the deficiency in sample size determina-tions for precise interval estimation of interaction effects that can result from ignoring the stochastic nature of continuous predictor and moderator variables in MMR. The primary finding of our examinations is that failure to accommodate the distributional properties of regressors can lead to underesti-mation of the necessary sample size and distortion of the desired interval precision. In order to take account of the randomness of regressor variables, two general and effective procedures for computing sample size estimates are presented. Moreover, corresponding programs are provided to facilitate use of the suggested approaches. This exposition helps to correct drawbacks in the existing techniques and to advance the practice of reporting confidence intervals in MMR analyses.

Keywords Moderation . Precision . Sample size

Moderated multiple regression (MMR) has been extensively employed to study the interaction effects between predictor

and moderator variables in management, psychology, educa-tion, and related disciplines. It follows from the comprehen-sive reviews of Stone-Romero, Alliger, and Aguinis (1994), Aguinis (1995), Aguinis and Stone-Romero (1997), Aguinis, Beaty, Boik, and Pierce (2005), and relatedwork that most of the methodological research in MMR has been concerned with the statistical power of hypothesis testing for detecting moderating effects. Although null hypothesis significance testing is useful in various applications, the dichotomous accept–reject decision ignores other useful information in its analysis. As an alternative, the notion of interval estimation has been stressed in studies such as Hahn and Meeker (1991), Steiger and Fouladi (1997), and Smithson (2003). Accordingly, the inferential procedures of interval estimators are strongly recommended by Wilkinson and the American Psychological Association Task Force on Statistical Inference (1999), as well as the Publication Manualof the American Psychological Association (American Psychological Associ-ation,2001). Since confidence intervals constructed with the desired reliability are more informative about the location of a targeted parameter, they should be the best reporting strategy in practical study. However, the methodological artifacts and statistical implications associated with interval estimation of moderating effects have received little attention within the framework of MMR.

The interactional formulation of MMR can be viewed as a special case of the statistical linear models, and so the inferential procedures of hypothesis testing and interval estimation of moderation can be conducted with standard methods and software packages for linear regression analysis. In this article, we consider the simple interaction models with criterion variable Y, predictor variable X, moderator variable Z, their cross-product term XZ, and a normal error termε in the formulation of Y ¼ bIþ X bX þ ZbZþ XZbXZþ ", where both the predictor X and moderator Z are continuous

Electronic supplementary material The online version of this article (doi:10.3758/s13428-011-0103-y) contains supplementary material, which is available to authorized users.

G. Shieh (*)

Department of Management Science, National Chiao Tung University, 1001 Ta Hsueh Road,

Hsinchu, Taiwan 30050, Republic of China e-mail: [email protected]

(2)

variables. Naturally, the special consideration of continuous moderator and predictor variables incurs the important notion of two different regression formulations. Because of the inherent nature of continuous measurements of the two regressor variables, they are not typically fixed in advance and are available only after the data has been collected. In order to recognize this stochastic feature of regressor variables, the appropriate strategy is to consider a random regression or unconditional formation, rather than a fixed or conditional setting, in experimental designs where the factors are under the control of investigators. The intrinsic appro-priateness and theoretical properties of fixed and random regression models have been discussed in Cramer and Appelbaum (1978) and Sampson (1974). Essentially, the inferential procedures of hypothesis testing and interval estimation are the same under both fixed and random formulations. The distinction between the two modeling approaches, however, becomes crucial when power, cover-age probability, and corresponding sample size calculations are to be made.

In the context of MMR, the distinct formulations of fixed and random modeling of the simple interaction models were especially emphasized in Shieh (2009,2010). Specif-ically, Shieh (2009) considered the power calculation and sample size determination for significance tests of moder-ating effects. The procedure takes account of the critical factors of strength of moderator effect, magnitude of error variation, and distributional property of predictor and moderator variables. On the other hand, Shieh (2010) incorporated the random nature of continuous moderator and predictor variables into two approaches to sample size computation for precise interval estimation of interaction effects. One approach provides the necessary sample size so that the designated interval for the least squares estimator of moderating effects attains the specified coverage probabil-ity. The other approach gives the sample size required to ensure, with a given tolerance probability, that a confidence interval of moderating effects will be within a specified range. The vital discrepancies between the conditional and unconditional settings in power and precision analyses are also closely evaluated in Shieh (2009, 2010). The results reveal substantial detrimental effects of failing to account for the randomness of predictor and moderator variables. Thus, the conventional fixed modeling formulation may not be applicable to the MMR with continuous regressor variables, because ittends to give insufficient sample sizes and inevitably leads to poor statistical performance.

Notably, there is a considerable recent literature pertain-ing to the illuminatpertain-ing applications of precise interval estimation in multiple linear regression (see Kelley 2008; Krishnamoorthy & Xia2008; Kelley & Maxwell2008; and the references therein). Instead of a direct accept-or-reject conclusion in a simple hypothesis test, it is arguable that

confidence intervals generally provide more information about the interested parameter value with a quantitative bound and assurance level. Accordingly, researchers should become methodologically conscious that the mere statistical significance of a targeted parameter is inadequate to warrant the conclusion that the effect is substantial and practically important. In view of the relatively scarce description of interval estimation in MMR, it is prudent to contribute to the documentation and examination of confidence intervals in different perspectives. The precision considerations in Shieh (2010) may not be the only criteria of practical importance. Just as in the instance of Kelley and Maxwell, two other useful principles related to the statistical properties of a confidence interval deal with the control of expected width and tolerance probability of interval width within a designated value. However, the explication of Kelley and Maxwell is confined to the case of a fixed regressor modeling framework, so that the corresponding sample size procedures may not be appro-priate for the interval estimation of interaction effects between continuous predictor and moderator variables. Moreover, it is also noted in Kelley and Maxwell that the sampling distribution of an estimated regression coefficient depends on whether the regressors are of fixed or random nature. Thus, the difference should be properly recognized in sample size planning for the simple interaction models described above. The aim of this article is to contribute to the design of MMR study by illustrating how the specification of inherent features associated with continuous predictor and moderator variables in sample size calculations influences the resulting precision of confidence intervals for the prescribed two interval width appraisals in the inference of an interactive effect. Consequently, the presentation here complements the work of Shieh (2010) with distinct precision criteria in sample size determinations.

Due to the complexity within the random regression framework, there appears to be a lack of applicable sample size procedures in the literature that accommodate the considerations of expected confidence interval width and tolerance probability of interval width within a designated value. Therefore, it is essential to extend the development and exposition of sample size methodology for precise interval estimation of interaction effects in two distinct aspects. One method gives the minimum sample size, such that the expected confidence interval width is within the designated bound. The other provides the sample size needed to guarantee, with a given tolerance probability, that the width of a confidence interval will not exceed the planned range. It is important to realize that the simplicity of a fixed setup may be appealing for inducing computa-tional shortcuts, but it does not involve all of the key factors in sample size calculation and, thus, is generally error prone. Accordingly, theoretical implications and numerical

(3)

examinations are presented to demonstrate that the sample size procedures for fixed regression, although they share many similarities with those for random regression, have some distinct disadvantages in MMR applications.

Confidence intervals of interaction effects

Consider the simple interaction model or MMR model within the fixed modeling framework:

Yi¼ bIþ XibX þ ZibZþ XiZibXZþ "i; ð1Þ where Yiis the value of the response variable Y; Xiand Ziare the values of the continuous predictor X and moderator Z;εi are iidN(0,σ2) random errors for i = 1,…, N; and βI,βX,βZ, andβXZare unknown parameters. To examine the existence and magnitude of a moderating effect, we are concerned with the distributional property for the least squares estimator bb_XZ Nðb ; V bbXZÞ

ofβXZ, where VðbbXZÞis the variance of bb_XZ and one useful expression is Vð bbXZÞ ¼ s2=SSEXZ where SSEXZ¼ 1 R2XZ

S2

XZis the residual sum of squares for the regression of the product term XZ on the X and Z variables, R2

XZ is the corresponding coefficient of determination, S_XZ2 ¼ i¼1PN Ui U

2

, Ui = XiZi, i = 1, …, N, and U¼iNP¼ 1Ui=N. Moreover, the natural estimator bVðbbXZÞ

o f bVðbbXZÞ i s bVðbbXZÞ ¼ bs2=SSEXZ, w h e r e bs2¼ 1 R2

ð ÞS2

Y= N 4ð Þ is the usual unbiased estimator of σ2, R2 is the coefficient of determination of the model in Eq.1,

S2 Y ¼ N P i¼ 1 Yi Y 2

,and Y¼iNP¼ 1Yi=N. For inferential

purpo-ses, we focus on the interval estimation procedure of β_XZ here. According to the standard results (Rencher, 2000, Section 8.6), a 100(1− α)% confidence interval of βXZ is

bbXZ tN4; a=2 bs2=SSEXZ n o1=2 ; bbXZþ tN4; a=2 bs2=SSEXZ n o1=2 ; where tN − 4, α/2 is the 100(1 − α/2)th percentile of the t distribution with N - 4 degrees of freedom. The half-width of the 100(1− α)% confidence interval is denoted by

H¼ tN4; a=2 bs2=SSEXZ

1=2_: _ð2Þ

Thus, the actual half-width H depends on the sample size N, confidence coefficient 1 -α, variance estimate bs2_{, and} observed values of predictor and moderator variables through the quantity of SSEXZ.

Due to the nature of continuous measurements encountered in practical research, the regressor variables typically cannot be controlled and are available only after observation. Hence,

in order to extend this concept and applicability to MMR, the continuous predictor and moderator variables {(Xi, Zi), i = 1, …, N} in Eq. 1 are assumed to have a joint probability functiong (Xi, Zi) with finite moments. Moreover, the form of g(Xi, Zi) does not depend on any of the unknown parameters (βI,βX,βZ,βXZ)orσ2. Consequently, bothbs2and SSEXZare realized values of random variables. It readily follows from Eq. 2 that the statistical properties ofbs2 and SSEXZ jointly determine the underlying distributional feature for the half-width H of a confidence interval.

When planning a study, researchers wish to ensure that the confidence interval is narrow enough to produce meaningful findings despite the stochastic nature of an interval width. In order to obtain an informative interval, it is necessary to specify not only the required confidence level and desired precision, but also the appropriate sample size. The most common approach is to determine the required sample size such that the expected half-width of a 100(1− α)% confidence interval is within the designated bound

E H½ ¼ d; ð3Þ

where the expectation E[H] is taken with respect to the joint distribution of bs2 _{and SSE}

XZ, and δ(>0) is a constant. Alternatively, one may compute the sample size needed to guarantee with a given tolerance probability that the width of a 100(1 − α)% confidence interval will not exceed the planned range

PfH < wg ¼ 1 g; ð4Þ

where 1− γ is the specified tolerance level and ω (> 0) is a constant. Note that these two concerns of expected magni-tude and tolerance probability of half-width have been discussed in Kupper and Hafner (1989) for simpler situations of one- and two-sample problems and in Kelley and Maxwell (2008) for multiple linear regression. Although the notion of expected width is widely covered in standard texts for sample size determination, the assurance of tolerance probability approach is recommended by Kupper and Hafner. However, it is noteworthy that the two principles are closely related to the two distinct criteria of unbiasedness and consistency in statistical point estimation. Therefore, each measure is of theoretical importance and practical interest in its own right. Furthermore, although the results are not completely comparable, it typically requires a larger sample size to meet the necessary assurance of tolerance probability than the control of a designated expected width.

Sample size procedures for precise interval estimation

It was mentioned above that Kelley and Maxwell (2008, p. 173) focused on the situation in which regressor

(4)

variables are fixed.The particular conditional design assumes that the same set of regressor values would be examined for repeated studies. Since bs2 _{is an unbiased} estimator of σ2, the use of population notation in the expression of VðbbXZÞ in Kelley and Maxwell (2008, Equation 34) implicitly assumes that SSEXZ¼: N 4ð Þ 1ð r2

XZÞs2XZor E SSE½ XZ ¼: N 4ð Þ 1 r2XZ

s2

XZ, where r2XZ is the coefficient of determination for the regression of product term XZ on the X and Z variables and s2_XZ is the variance of the product term XZ. Although the form of VðbbXZÞinvolves some unstated clarifications, the formulation suggests a simplified approximation for E½bVðbbXZÞ

¼: s2_{= N 4}_ð _{Þ 1 r}2 XZ s2 XZ

under a random regression setup. Hence, the corresponding approximate expected half-width is E H½ ¼: tN4; a=2 s2= N 4ð Þ 1 r2XZ s2_XZ 1=2_: _ð5Þ

Accordingly, the sample size NEHS needed for the expected half-width of a 100(1− α)% confidence interval to fall within the designated bound δ is the minimum integer N, such that

t_{N4; a=2}s2= N 4ð Þ 1 r 2_XZs_XZ2 1=2 d: ð6Þ A similar expression is presented in Kelley and Maxwell (Equation 35). On the other hand, an analogous argument applies for the tolerance probability consideration given in Eq. 4. Thus, P Hf < wg ¼: P tN 4; a=2 bs2= N 4fð Þ 1ð

h n r2 XZÞs2XZg1=2< wg, or equivalently, P Hf <wg ¼: P K < w=t _{N 4; a=2}2ðN 4Þ2 1 r2XZ s2_XZ=s2 n o ; ð7Þ where K¼ N 4ð Þbs2=s2 x2ðN 4Þ and χ2(υ) is a chi-square distribution withυ degrees of freedom. The sample size NPHS required to guarantee with a given tolerance probability (1 − γ) that the width of a 100(1 − α)% confidence interval will not exceed the planned rangeω is the smallest integer N such that

X2ðN 4; gÞ w=tN 4; a=2 2 N 4 ð Þ2 _{1 r}2 XZ s2_XZ=s2 ð8Þ whereχ2(N− 4, γ) is the 100(1 − γ)th percentile of a χ2(N− 4) distribution. The formula in Kelley and Maxwell (Equation 35) basically provides the same inequality, although the degrees of freedom of the chi-square distribution there was incorrectly expressed as N − 1 instead of N − 4. To compute the necessary sample sizes, a standard iterative search can be conducted to find NEHSand NPHSwith the inequalities given in Eqs. 6 and 8, respectively. As noted above, these two simplified procedures rely primarily on the straightforward approximation of E SSE½ XZ ¼: ðN 4Þ 1 r2XZ

s2 XZ, and they can alternatively be viewed as the exact methods under

the fixed regressor setting as in Kelley and Maxwell. However, their usage in the random regression scenario raises natural concerns about the resulting accuracy, for the obvious reason that the underlying variability of the residual sum of squares SSEXZis ignored. The potential deficiency of the sample sizes NEHSand NPHSfor precise interval estimation of interaction effects will be examined later in the numerical investigations. The prescribed discussion emphasizes that the underly-ing statistical properties of a confidence interval half-width are uniquely defined by the joint distribution of the regressors. Accordingly, the underlying property of predic-tor and moderapredic-tor variables should be incorporated into the sample size procedures as much as possible. In order to provide a unified and feasible technique, we adopt the large-sample approach to avoid reliance on anyspecific distributional assumption of the predictor and moderator variables. To this end, the theoretical properties of the suggested asymptotic approximation to the distribution of W»¼ SSEXZ= N 1ð Þ is presented in Eq. 11 in the

Appendix. The results facilitate the proposed sample size procedures for constructing precise confidence intervals under the two precision criteria described in Eqs.3and 4.

With the approximate expected half-width presented in Eq.12, the sample size NEHP needed for the expected half-width of a 100(1− α)% confidence interval to fall within the designated boundδ is the minimum integer N, such that tN 4; a=2ðN 1Þ1=2E½ E Wbs » 1=2

h i

d: ð9Þ

In contrast, the approximate probability in Eq.13suggests another useful approach to sample size determination. Consequently, the sample size NPHP required to guarantee, with a given tolerance probability (1− γ), that the width of a 100(1− α)% confidence interval will not exceed the planned rangeω is the smallest integer N, such that

E½Φ wð »W»Þ 1 g: ð10Þ

Clearly, the two inequalities in Eqs. 9 and 10are more involved than the corresponding formulas in Eqs.6 and8. Instead of substituting the term SSEXZ with a constant estimate, as in the simplified procedures, the extra complexity in the suggested formulations is employed to reflect the embedded stochastic characteristic of SSEXZ. The usefulness of the sophisticated inequalities is shown next in the numerical example.

Numerical example

To illustrate a typical sample size problemmost frequently encounteredin the planning stage of an MMR study, we

(5)

consider the hypothetical research framework presented in Shieh (2010) for assessing interaction effects between the length of time in the position (X) and managerial ability (Z) on the self-assurance of managers (Y). Due to the prospective nature of advanced research planning, the general guideline suggests that a successful pilot study can offer plausible and reasonable planning values of the vital characteristics for calculating the necessary sample size. On the basis of the pilot data in Table 5 in Shieh (2010), the empirical distribution of the 60 observed config-urations of predictor and moderator variables may be utilized to reconstruct or approximate the actual distributional feature of the two variables. However, the prescribed sample size procedures differ in their use of the pilot information and lead to substantially distinct results. The key difference is that the simplified method treats the regressor variables as constants, but it is more reasonable to regard them as random variables, and so the suggested procedures assume they were drawn from a multivariate population.

For the 60 observed values of Xi = (Xi, Zi, XiZi)T and empirical probability 1/60 for i = 1, . . . , 60, the moment matrices for the quantities in theAppendixcan be obtained by

2¼:XX60 i¼1 Xi=60;@ ¼: X X60 i¼ 1 Xi2 ð Þ Xð i2ÞT=60; and =¼:XX60 i¼ 1 Xi2 ð Þ Xð i2ÞT Xð i2Þ Xð i2ÞT h i =60:

Moreover, it follows that m_W»¼ 1 r 2_XZs2

XZ¼: 1:2348 and s2

W»¼: 22:6511= N 1ð Þ, as shown in Shieh (2010). In planning a research study with these prior inputs, the minimum sample size needed to control the expected half-width of a 95% confidence interval of interaction effects within the designated bound 0.15 can be computed with the inequalities in Eqs.6and9. The resulting sample sizes are

NEHS = 145 and NEHP = 156, and the corresponding approximate expected half-widths given in Eq. 12 are 0.1560 and 0.1497, respectively. Hence, the sample size NEHS determined by the simplified method is too small to satisfy the desired precision bound. This phenomenon should continue to exist in other settings. Moreover, the smallest sample sizes required to guarantee with a given tolerance probability of .90 that the width of a 95% confidence interval will not exceed the planned range 0.15 are NPHS = 165 and NPHP = 216, based on the inequalities in Eqs. 8 and 10, respectively. Notably, the approximate tolerance probabilities obtained with Eq. 13 are .6762 and .9019. The sizable difference between NPHP− NPHS= 216− 165 = 51 yields the major deficit of .9019 − .6762 = .2257 in tolerance probability.

This numerical investigation exemplifies the fundamen-tal deficiency that overlooking the stochastic nature in regressor variables may lead to a serious underestimation of the sample size required to obtain a designated expected half-width or to ensure the adequate probability of achieving the desired half-width for the confidence inter-vals. To facilitate the application of the proposed approaches, the SAS/IML (SAS Institute, 2008) programs employed to perform the sample size calculations are available as supplemental materials from brm.psycho-nomic-journals.org/content/supplemental. Users can easily identify the statements containing the key values in this exposition and then modify the program to accommodate their own specifications.

Simulation study

To further evaluate the performance of the sample size formulas with respect to the prescribed precision criteria in Eqs. 3 and 4 under various parameter specifications, the MMR model defined in Eq. 1 with bivariate normal predictor, and moderator variables is used as the base for a

Table 1 Computed sample size, approximate expected half-width, andsimulatedexpected half-widthfor the half-width of a 90% two-sided interval ofbbXZ at specified half-width δ = 0.15with bivariate

normal predictor and moderator variables (βXZ= 1,σ2= 1,μX=μZ=

0,s2

X =s2Z= 1, correlationρ)

ρ Simplified Method Proposed Method

NEHS Approximate Expected

Half-Width

Simulated Expected Half-Width

Error NEHP Approximate Expected

Half-Width Simulated Expected Half-Width Error .1 125 0.1499 0.1551 −0.0052 129 0.1497 0.1525 −0.0028 .3 117 0.1494 0.1555 −0.0061 122 0.1497 0.1520 −0.0023 .5 103 0.1493 0.1570 −0.0078 111 0.1492 0.1508 −0.0015 .7 87 0.1496 0.1598 −0.0102 97 0.1498 0.1496 0.0002 .9 73 0.1492 0.1623 −0.0131 83 0.1492 0.1494 −0.0002

(6)

Monte Carlo exposition. A similar bivariate normal assump-tion was made for related MMR treatments in McClelland and Judd (1993), O’Connor (2006), and Shieh (2009,2010). Specifically, the coefficient parameters and variance of the simple interaction model are set asβI=βX=βZ=βXZ= 1 andσ2= 1, respectively. Moreover, the predictor and moderator (X, Z) variables are jointly normally distributed with mean (0, 0), variance (1, 1), and correlation ρ. The correlation parameter ρ between the predictor variable and moderator variable is set at the five levels of .1–.9 in increments of .2.

With the specifications described above, the simulation study is conducted in two steps. First, under the selected values of coefficient parameters, error, distribution config-urations of the bivariate predictor, and moderator distribution, the estimates of sample sizes required for precise interval estimation are computed with confidence levels (1− α) = .90 and .95,δ = ω = 0.15 and 0.20, and tolerance probability (1 − γ) = .90. These levels were selected to represent reasonably the range of specifications and sample sizes used in typical research settings.The sample sizes NEHS and NEHP required for controlling the expected half-width are presented in Tables 1, 2, 3 and 4, while those sample sizes, NPHS and NPHP, ensuring the probability of a bounded confidence

interval are summarized in Tables 5, 6, 7 and 8. It can be readily seen that the computed sample sizes NEHP are generally larger than NEHSfor controlling the expected half-width in Tables1,2,3and4, although the difference is more pronounced between the sample size estimates NPHS and NPHPfor the assurance of tolerance probability in Tables5,6,

7 and8. With the reported sample sizes NEHSand NPHS, the actual values of the approximate expected half-width and approximate tolerance probability given in Eqs. 5 and 7, respectively, are also calculated. Similarly, the approximated expected half-width and approximated tolerance probability associated with computed sample sizes NEHP and NPHP are calculated on the basis of Eqs. 12 and 13, respectively. As was expected, the resulting approximate expected half-width is slightly less than the designated expected half-width, whereas the approximate tolerance probability is marginally greater than the chosen tolerance probability.

In the second step, the accuracy of the sample size procedures is examined through a Monte Carlo simulation study. Estimates of the true expected half-width and tolerance probability associated with a given sample size N and various parameter configurations are computed through a Monte Carlo simulation of 10,000 independent

Table 2 Computed sample size, approximate expected half-width, andsimulatedexpected half-widthfor the half-width of a 90% two-sided interval ofbbXZ at specified half-width δ = 0.20with bivariate

0,s2

Half-width

Simulated expected Half-width

Half-Width Simulated Expected Half-Width Error .1 73 0.1997 0.2111 −0.0114 78 0.1991 0.2029 −0.0038 .3 68 0.1998 0.2128 −0.0130 75 0.1986 0.2011 −0.0025 .5 60 0.1999 0.2174 −0.0175 68 0.1997 0.2019 −0.0022 .7 52 0.1983 0.2203 −0.0220 60 0.1982 0.2014 −0.0031 .9 44 0.1979 0.2258 −0.0279 52 0.1954 0.2009 −0.0055

Table 3 Computed sample size, approximate expected half-width, and simulated expected half-width for the half-width of a 95% two-sided interval ofbbXZ at specified half-width δ = 0.15with bivariate

0,s2

Half-Width

(7)

data sets. For each replicate, N sets of predictor and moderator values are generated from the designated bivariate normal distribution. These values of predictor and moderator, in turn, determine the mean responses for generating N normal outcomes with the simple interaction model. Next, the half-width estimate H is computed, and the simulated expected half-width is the mean of the 10,000 replicates of H. Alternatively, the simulated tolerance probability is the proportion of the 10,000 replicates whose values of H are less than or equal to the specified bound ω. The adequacy of the sample size procedure for precise interval estimation is deter-mined by the following formula: error ¼ approximated expected half ‐ width simulated expected half ‐ width, or error ¼ approximated tolerance probability simulated tolerance probability. The simulated expected half-width, simulated tolerance probability, and associated error are also summarized in Tables1,2,3,4,5,6,7 and8.

Examination of the sample sizes in these tables reveals the general pattern that when all other factors remain constant, the sample size increases with increasing confi-dence level (1− α), with decreasing half-width bound δ or ω and with decreasing correlation ρ. Therefore, for both simplified and proposed methods, the largest sample size

NEHS= 176 and NEHP= 179 for (1− α) = .95, δ = 0.15 and ρ = .1 in Table3, whereas the smallest sample sizes NEHS= 44 and NEHS= 52 for (1− α) = .90, δ = 0.20 and ρ = .9 in Table2. Accordingly, the largest and smallest sample sizes NPHSare 198 and 54, and NPHPare 232 and 84 in Tables7 and 6, respectively.

Furthermore, as can be seen from the errors in Tables1, 2,3and4concerning the precision of expected half-width, the inequality in Eq.6for computing the sample size NEHS produces accurate expected half-width for the cases ofδ = 0.15 in Tables1and3. Most of the resulting absolute errors are less than 0.01, with only two exceptions (0.0102 and 0.0131) in Table1. In contrast, the performance associated with the situations of δ = 0.20 in Tables 2 and 4 degrade slightly but are still reasonable, with the absolute errors between 0.0073 and 0.0279. In short, the simplified approach tends to give reliable yet marginally smaller than required sample sizes. However, the accuracy improves with larger sample sizes, because large errors occur with smaller sample sizes. Comparatively, the errors of the expected half-width associated with sample sizes NEHP in Tables 1, 2, 3 and 4 clearly show that the inequality of Eq. 9 performs extremely well because all absolute errors are less than or equal to 0.0055 for the 20 cases examined

Table 4 Computed sample size, approximate expected half-width, and simulated expected half-width for the half-width of a 95% two-sided interval ofbbXZat specified half-width δ = 0.20 with bivariate

0,s2

Half-Width

Table 5 Computed sample size, approximate tolerance probability, and simulated tolerance probability for the half-width of a 90% two-sided interval ofbbXZat specified half-widthω = 0.15 and tolerance

probability 0.90 with bivariate normal predictor and moderator variables (βXZ= 1,σ2= 1,μX=μZ= 0,s2X =s2Z= 1, correlationρ)

NPHS Approximate Tolerance

Probability

Simulated Tolerance Probability

Error NPHP Approximate Tolerance

Probability Simulated Tolerance Probability Error .1 144 .9059 .6650 0.2409 173 .9023 .8990 0.0033 .3 135 .9116 .6456 0.2660 167 .9012 .9099 −0.0087 .5 119 .9047 .6120 0.2927 155 .9001 .9115 −0.0114 .7 102 .9049 .6075 0.2974 140 .9025 .9198 −0.0173 .9 86 .9019 .5884 0.3135 123 .9020 .9232 −0.0212

(8)

here. Although the simplified inequality of Eq.6is accurate enough for practical use, it is consistently outperformed by the improved formula in Eq.9.

On the other hand, the performance of the simplified method is extremely poor and disturbing for the consideration of tolerance probability of bounded interval half-width. The discrepancies between the approximate tolerance probabilities and simulated tolerance probabilities in Tables5, 6, 7, and 8 range from 0.2409 to 0.3321. Hence, the inequality in Eq.8severely overestimates the attained tolerance probabil-ity, and thus it underestimates the necessary sample size to meet the selected criterion. Similar findings were reported in Kupper and Hafner (1989) for interval estimation of one- and two-sample problems. In the present case, the parameter value of Nð 4Þ 1 r2

XZ

s2

XZis used in place of SSEXZ, so the variability in SSEXZ has been neglected in sample size calculations. The repercussionsof ignoring the random feature of predictor and moderator variables on sample size calculation are detrimental and substantial. Hence, the simplified procedure should not be used in such a random regression setting, because it can lead to under allocation of sample size or overconfidence in the interval precision. Regarding the behavior of the proposed method, an

inspection of Tables 5, 6, 7 and 8 shows that the corresponding differences between the approximated and simulated tolerance probabilities are fairly small. Since the considered approach useslarge sample approximation, the accuracy is affected, to some extent, for those situations with small sample sizes. The largest two deviations of −0.0258 and −0.0287 occur with the sample sizes 94 and 84 in Table 6 for ρ = .7 and .9, respectively. Obviously, the advantage of the proposed procedure over the simplified method persists in the case of ensuring the required tolerance level. Due to the behavior of the simplified method, the accurate performance of the suggested inequality outweighs the extra computational requirement. In light of these detailed empirical comparisons, the proposed methods are clearly superior to the simplified procedures in sample size calculations for precise interval estimation.

Concluding remarks

The purpose of the present article was to discuss the sample size issues surrounding the use of confidence intervals for the inference of interaction effects in MMR

Table 6 Computed sample size, approximate tolerance probability, and simulated tolerance probabilityfor the half-width of a 90% two-sided interval ofbbXZ at specified half-width ω = 0.20and tolerance

probability .90 with bivariate normal predictor and moderator variables (βXZ= 1,σ2= 1,μX=μZ= 0,s2X =s2Z= 1, correlationρ)

Probability

Probability Simulated Tolerance Probability Error .1 87 .9082 .6508 0.2574 110 .9033 .9050 −0.0017 .3 82 .9157 .6348 0.2809 108 .9040 .9144 −0.0104 .5 73 .9150 .5991 0.3159 102 .9019 .9208 −0.0189 .7 63 .9134 .5930 0.3204 94 .9035 .9293 −0.0258 .9 54 .9178 .5857 0.3321 84 .9024 .9311 −0.0287

Probability

Probability Simulated Tolerance Probability Error .1 198 .9020 .6598 0.2422 232 .9021 .8984 0.0037 .3 185 .9050 .6453 0.2597 223 .9025 .9022 0.0003 .5 163 .9004 .6271 0.2733 205 .9011 .9040 −0.0029 .7 140 .9097 .6096 0.3001 182 .9000 .9163 −0.0163 .9 118 .9114 .6074 0.3040 159 .9016 .9179 −0.0163

(9)

studies. The focus is on the information contained in the joint probability function of continuous predictor and moderator variables in a simple interaction model. During the planning stage for MMR research with limited resources, it is important to consider all possible features. We demonstrate that sample size estimates for precise interval estimation of interaction effects will generally be inadequate and misleading if they are based solely on the anticipated characteristics of the regressor variables. Thus, the simplified procedure based on a fixed modeling setup should not be applied indiscrimi-nately. With increased computing power and the general availability of statistical software, computational simplic-ity is no longer an adequate criterion. An appropriate approach should involve all of the critical factors in sample size determination. Therefore, a more prudent strategy is to account for the stochastic behavior in regressor variables. This article gives explicit formulas for calculating the necessary sample size with respect to the considerations of expected confidence interval half-width and tolerance probability of interval half-half-width within a designated value. The proposed approaches have clear advantages in the flexibility of the joint distribution of predictor and moderator variables and the unification of a normal approximation for ease of computation. More important, the performance of the suggested methods appears to be remarkably good for the range of model specifications considered in the present article. The proposed methodology not only facilitates the advocated practice of interval procedures, but also further reinforces the potential usefulness of MMR analysis.

Author Note The author thanks the editor, Gregory Francis, and the two anonymous reviewers for their valuable comments on earlier drafts of the article.

Appendix

The properties of the confidence interval half-width It follows from the standard assumption in Eq. 1 for the simple interaction model that the half-width of the 100(1 – α)% confidence interval is H¼ tN 4; s=2 bs2=SSEXZ

n o1=2

. Hence, the statistical property of H is determined by the joint distribution ofbs2 _{and SSE}

XZ. Since the distribution ofbs2 s2_{= N 4}_ð _Þ

x2_ð_N₄_{Þ does not depend on the predictor} and moderator variables,bs2_{and SSE}

XZare independent. The remaining issue in describing the feature of H is to attain the distribution of SSEXZ. Note that the distribution of SSEXZis somewhat more complex, and an explicit expression generally does not exist. However, much of the complexity is considerably simplified if we consider the asymptotic phenomenon.

For ease of discussion, the moments of the explanatory vectors Xi= (Xi, Zi, XiZi)Tare defined as

2¼ E X½ ;i X X ¼ E Xð i2Þ Xð i2ÞT h i and = ¼ E Xð i2Þ Xð i2ÞT Xð i2Þ Xð i2ÞT h i ; where the expectationsare taken with respect to the joint probability density function g(Xi, Zi) of (Xi, Zi), and ⊗ represents the Kronecker product. Analogous to the practical standpoint of Shieh (2009) for providing a generally useful and versatile solution without being specifically confined to any particular joint probability function g(Xi, Zi), we consider the large-sample distribution of W»¼ SSEXZ= N 1ð Þ, W»Nðm_W»; s2_W»; ð11Þ w h e r e mW»¼ 1= cTΣ 1cÞ; s2W»¼ v2W»= N 1ð Þ; v2W»¼ m4 W» cTΣ 1 cTΣ 1Þ< Σ 1c Σ 1cÞ m2W»g , c =

Probability

Probability Simulated Tolerance Probability Error .1 118 .9006 .6522 0.2484 145 .9033 .8973 0.0060 .3 111 .9093 .6320 0.2773 141 .9036 .9160 −0.0124 .5 98 .9014 .6049 0.2965 132 .9029 .9218 −0.0189 .7 85 .9127 .6043 0.3084 119 .9008 .9212 –0.0204 .9 72 .9097 .5940 0.3157 106 .9030 .9294 −0.0264

(10)

(0, 0, 1)Tis a 3 × 1 row vector, andΣ and Ψ are defined above. It is noted in Shieh (2009) that the mean value μW* is equivalent to the extra variance of the product XZ after controlling for X and Z. Therefore, it is more informative to express it as mW»¼ 1 r2XZ

s2_XZ here. As SSEXZ¼ N 1

ð ÞW»is the residual sum of squares for the regression of product term XZ on the X and Z variables, the values of both SSEXZand W* are presumably nonnegative. It appears that the probability of negative W*, P(W* < 0), is often small enough so that the large sample normal approximation of W* is nearly adequate for practical purposes. Thus, the evaluations with respect to g(Xi, Zi) are transformed to and approximated by the corresponding assessments with respect to W*.

According to the aforementioned theoretical results, the expected half-width can be simplified as

E H½ ¼: tN 4; a=2ðN 1Þ1=2E½bsE W» 1=2

h i

ð12Þ It follows from the fact thatbs2_{has a multiple of a chi-square} distribution that

E½ ¼ s 2= N 4bs f ð Þg1=2Γ N 4fð Þ=2g:

Since there is no analytic expression for E[W*− 1/2], the actual quantity needs to be evaluated with numerical integration with respect to the normal distribution in Eq. 11. In addition, the probability P{H < ω} for ω > 0 can be rewritten as P Hf < wg¼:P Kf < w»W»g; W h e r e K¼ N 4ð Þbs2_=s2_x2_ð_N₄_{Þ a n d w}»_¼ N 1 ð Þ N 4ð Þw2 =t2

N 4; a=2s2g. To permit computatio-nal simplifications, the approximation is alternatively expressed as

P Hf < wg ¼: E½Φ wð »W»Þ; ð13Þ whereΦ(c) = P{K < c}, c > 0, is the cumulative density function ofχ2(N− 4). Since all related functions of normal and chi-square distributions are readily embedded in con-temporary statistical packages, the expressions for approxi-mate values of E[H] and P{H <ω} given in Eqs. 12) and 13), respectively,can be readily implemented.

References

Aguinis, H. (1995). Statistical power problems with moderated multiple regression in management research. Journal of Management, 21, 1141–1158.

Aguinis, H., Beaty, J. C., Boik, R. J., & Pierce, C. A. (2005). Effect size and power in assessing moderating effects of categorical

variables using multiple regression: A 30-year review. The Journal of Applied Psychology, 90, 94–107.

Aguinis, H., & Stone-Romero, E. F. (1997). Methodological artifacts in moderated multiple regression and their effects on statistical power. The Journal of Applied Psychology, 82, 192– 206.

American Psychological Association. (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: Author.

Cramer, E. M., & Appelbaum, M. I. (1978). The validity of polynomial regression in the random regression model. Review of Educational Research, 48, 511–515.

Hahn, G. J., & Meeker, W. Q. (1991). Statistical intervals: A guide for practitioners. New York: Wiley.

Kelley, K. (2008). Sample size planning for the squared multiple correlation coefficient: Accuracy in parameter estimation via narrow confidence intervals. Multivariate Behavioral Research, 43, 524–555.

Kelley, K., & Maxwell, S. E. (2008). Sample size planning with applications to multiple regression: Power and accuracy for omnibus and targeted effects. In P. Alasuuta, L. Bickman, & J. Brammen (Eds.), The Sage handbook of social research methods (pp. 166–192). Newbury Park, CA: Sage.

Krishnamoorthy, K., & Xia, Y. (2008). Sample size calculation for estimating or testing a nonzero squared multiple correla-tion coefficient. Multivariate Behavioral Research, 43, 382– 410.

Kupper, L. L., & Hafner, K. B. (1989). How appropriate are popular sample size formulas? American Statistician, 43, 101– 105.

McClelland, G. H., & Judd, C. M. (1993). Statistical difficulties of detecting interactions and moderator effects. Psychological Bulletin, 114, 376–390.

O’Connor, B. P. (2006). Programs for problems created by continuous variable distributions in moderated multiple regression. Organizational Research Methods, 9, 554–567.

Rencher, A. C. (2000). Linear models in statistics. New York: Wiley. Sampson, A. R. (1974). A tale of two regressions. Journal of the

American Statistical Association, 69, 682–689.

SAS Institute (2008). SAS/IML User’s Guide, Version 9.2. Cary, NC: SAS Institute Inc.

Shieh, G. (2009). Detecting interaction effects in moderated multiple regression with continuous variables: Power and sample size considerations. Organizational Research Methods, 12, 510–528.

Shieh, G. (2010). Sample size determination for confidence intervals of interaction effects in moderated multiple regression with continuous predictor and moderator variables. Behavior Research Methods, 42, 824–835.

Smithson, M. (2003). Confidence intervals. Thousand Oaks, CA: Sage.

Steiger, J. H., & Fouladi, R. T. (1997). Noncentrality interval estimation and the evaluation ofstatistical models. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 222–257). Mahwah, NJ: Erlbaum.

Stone-Romero, E. F., Alliger, G. M., & Aguinis, H. (1994). Type II error problems in the use of moderated multiple regression for the detection of moderating effects of dichotomous variables. Journal of Management, 20, 167–178.

Wilkinson, L., & the American Psychological Association Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594–604.