The last component of the survey design is to consider how many additional short surveys are required or how to determine the number of long versus short surveys based on a fixed budget. Answers to this question depend on several factors simultaneously. First we
need to understand the effect of sample size, differentiating by and , on the overall variance level.
nC nI
In model 1, we have assumed missing completely at random (MCAR), so that
i
. Therefore, the variance of weighted average spending can be rewritten as
)
From equation 6, we can infer that when the sample size of complete cases on
spending ( ) is doubled, while holding all else constant, the variance of average spending is reduced by roughly 50%, or vice versa. The exact value will depend on the number of
incomplete cases and the variance associated with segment shares. Also, if the sample size on the complete cases is fixed (e.g., 400), more samples ( ) are allocated to the segment with a larger spending variation would reduce the overall variance to a greater extent because
would decrease.
nC
On the contrary, holding complete cases and other parameters constant, the addition of short surveys will not reduce the estimator variance as efficiently as the long surveys do.
This is because a short survey ( ) influences only the precision of segment shares, the second component of equation 6. By holding all else constant, the level of variation (
nI
Tˆ ) with
regards to the short survey converges at a rate of (nI−1) with probability one (equation 7).
Variance (Tˆ ) ≈
nI
b k a
+ + (7)
By holding pi,ui,nc,σiconstant, we can treat a,b,k>0and as constant
Using model 1 with complete case ( ) of 400 as an example, we plot the relationship between the sample variance and the number of incomplete cases ( ) in Figure 2. When increases from zero to 1000, 2000, and 3000, the variance reduces to 423, 415, and 412, respectively. As the number of continues to expand, the variance gradually approaches 404.75. A convex plot in figure 2 clearly indicates that the marginal error reduction rate is much higher in the early stage and becomes inefficiently lower as expands. Based on the specified parameters in model 1, the best combination of versus is approximately 1: 5, since beyond this point, the contribution of on variance reduction is negligible. The
addition of short surveys therefore should not be unlimited. To determine the appropriate sample size of , researchers should take into consideration of survey cost per , the expected variance level of spending, and the marginal error reduction rate, simultaneously.
nC
nI
nI
nI
n
nI
nI
nC I
nI nI
Besides marginal variance reduction rate, the maximum level of error reduction in adding short surveys is another key factor to be considered. In model 1, the spending variance can be reduced by 14% if the segment is deterministic while the variance in model 2 can be reduced up to 20% (Table 2 and 3). To understand the cause of differences, we compare the spending variances when is given at a finite number versus is infinity (Equation 8).
When tends to infinity, the segment shares are determined and no variation is introduced, which sets the basis for precision comparison.
nI nI
nI
Comparison of estimator variance
= variance when nIis given at a fixed number / variance when nItends to infinity
= ⎥⎥
Plotting in the default parameters from Table 1, this ratio from equation 8 is equal to (
1 ). In other words, in model 1, the estimator variance without obtaining
additional information on segment shares ( = 0) is 1.157 times the estimator variance when segment shares are known. If is equal to , the estimator variance is 1.078 times that w segment shares are deterministic. The level of variance reduction is greatly influenced by the
segment share variance,
nI
arger this component is, the more
worthwhile to make extra effort to obtain additional information through nI.
− =
Therefore, for model 1, the maximum level of variance reduction from segmentation will depend on the relative segment shares across groups ( ), the average spending by each group (
pi
μi), number of complete cases ( ) and spending variation within groups ( ). For model 2, when the response rate is different across groups3, the variance reduction level would then depend on and the probability of giving full spending information,
nC σi2
Based on the above discussion, it is recommended to design expenditure surveys by implementing two waves of data gathering, one for collecting the spending and segment information and the other wave for collecting segmentation only. If the expenditure study is aimed for a longer period, the first step is to get a rough idea for the level of segment shares,
average spending, and variance by user groups in the study population through long surveys ( ). Obtaining such information would then help researchers to calibrate the appropriate number of short surveys ( ) by taking into consideration the associated variance reduction level and operating costs per collected survey.
nC
nI
7. CONCLUSION
Adopting the segmentation strategy to estimate total visitor consumption has several advantages in terms of policy management, evaluation, and sampling design. However, this approach requires two sets of information – average spending per segment as well as segment share so that total visitation can be apportioned to each subgroup. Both variables generally rely on visitor surveys for estimation. To gather this information, Stynes (1999) and Stynes and White (2006) recommend conducting a short survey to investigate visitor segments in addition to regular visitor surveys (full survey) on spending. This paper expands their discussion further in two directions. First, a statistical formula is provided to compute the estimator variance by taking into account the stochastic nature of spending and segment shares. This formula provides a convenient step to understand the variance level introduced by segmentation. Using simulation analysis, we demonstrate that acquiring more information on the segment share helps to reduce the variance of the weighted average spending so that better precision is attained. Secondly, the addition of segment information will not reduce the estimator variance in a linear pattern so that there is no need to generate an excessive number of short surveys. The key is to allocate the short surveys to be representative of the time and location of recreational usage across the study region. Optimization analysis can then be performed based on expected level of variance reduction, subject to constraint of time, manpower, and survey materials, to reach a balance between data quality and operating cost between full and short surveys.
Acknowledgements: Constructive comments from Dr. Ariel Rodriguez at Arizona State University and anonymous referees are highly appreciated. Financial support from the Taiwan National Science Council under NSC 98-2410-H-390 -029 -SS2 is acknowledged.
8. REFERENCE
Baird, B. F. (1989). Managerial decisions under uncertainty: An introduction to the analysis of decision making. New York: Wiley-Interscience.
Breen, H., Bull, A., & Walo, M. (2001). A comparison of survey methods to estimate visitor expenditure at a local event. Tourism Management, 22, 473-479.
Diaz-Perez, F. M., Bethencourt-Cejas, M., & Alvarez-Gonzalez, J. A. (2005). The segmentation of canary island tourism markets by expenditure: Implications for tourism policy. Tourism Management, 26(6), 961-964.
Efron, B. (1979). Bootstrap methods: Another look at the Jackknife. Annals of Statistics, 7(1), 1-26.
English, D. B. K. (2000a). Calculating confidence intervals for regional economic impacts of recreation by bootstrapping visitor expenditure. Journal of Regional Science, 40(3), 523-539.
English, D. B. K. (2000b). A simple procedure for generating confidence intervals in tourist spending profiles and resulting economic impacts. The Journal of Regional Analysis
& Planning, 30(1), 59-74.
Fleming, W. R., & Toepper, L. (1990). Economic impact studies: Relating the positive and negative impacts to tourism development. Journal of Travel Research, 29(1), 35-42.
Frechtling, D. C. (2006). An assessment of visitor expenditure methods and models. Journal of Travel Research, 45(1), 26-35.
Groves, R. M. (1989). Survey errors and survey costs. New York: John Wiley & Sons, Inc.
Hays, W. L. (1994). Statistics (5th ed.). Belmont, CA: Wadsworth/ Thomson Learning.
Kish, L. (1995). Survey sampling. New York: John Wiley & Sons, Inc.
Koc, E., & Altinay, G. (2007). An analysis of seasonality in monthly per person tourist spending in Turkish inbound tourism from a market segmentation perspective.
Tourism Management, 28(1), 227-237.
Kotler, P. (1980). Principles of marketing (1st ed.). New Jersey: Prentice-Hall.
Laesser, C., & Crouch, G. I. (2006). Segmenting markets by travel expenditure patterns: The case of international visitors to Australia. Journal of Travel Research, 44(4), 397-406.
Lai, S. Z., & Wong, K. F. (2008). Computation of joint confidence intervals for the weighted average. Kaohsiung, Taiwan: National University of Kaohsiung.
Leeworthy, V. R., Wiley, P. C., English, D. B. K., & Kriesel, W. (2001). Correcting response bias in tourist spending surveys. Annals of Tourism Research, 28(1), 83-97.
.htm
Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). New Jersey: Wiley-Interscience.
Lohr, S. L. (1999). Sampling: Design and analysis. Pacific Grove, CA: Brooks/Cole Publishing Company.
Loomis, J. (2007). Correcting for on-site visitor sampling bias when estimating the regional economic effects of tourism. Tourism Economics, 13(1), 41-47.
Mok, C., & Iverson, T. J. (2000). Expenditure-based segmentation: Taiwanese tourists to Guan. Tourism Management, 21(3), 299-305.
Orens, A., & Seidl, A. (2004). Winter tourism and land development in Gunnison, Colorado.
Fort Collins, CO: Department of Agricultural and Resource Economics, Colorado State University.
Ormer, C. V., Littlejohn, M., & Gramann, J. H. (2001). Olympic National Park visitor study:
University of Idaho.
Pannell, D. J. (1997). Sensitivity analysis of normative economic models: Theoretical framework and practical strategies. Agricultural economics, 16(2), 139-152.
Pol, A. P., Pascual, M. B., & Vazquez, P. C. (2006). Robust estimators and bootstrap
confidence intervals applied to tourism spending. Tourism Management, 27(1), 42-50.
Research Resolutions & Consulting Ltd. (2005). Guidelines for measuring on-site spending at gated and ungated events and festivals Appendices I - IV. Retrieved
from http://www.rpts.tamu.edu/impacts/page3_Gated
Rylander, R. G., Propst, D. B., & McMurtry, T. R. (1995). Nonresponse and recall biases in a survey of traveler spending. 33(4), 39-45.
Simmons, T., & Gramann, J. H. (2001). Badlands National Park visitor study: University of Idaho.
Smith, S. L. J. (2000). New developments in measuring tourism as an area of economic activity. In W.C.Gartner & D. W. Lime (Eds.), Trends in outdoor recreation, leisure and tourism (pp. 225-234). New York: CAB International.
Stynes, D. J. (1999). Guidelines for measuring visitor spending. Retrieved from https://www.msu.edu/course/prr/840/econimpact/pdf/ecimpvol3.pdf
Stynes, D. J., & Sun, Y.-Y. (2003). Economic impacts of National Park visitor spending on gateway communities, systemwide estimates for 2001. East Lansing, MI: Department of Park, Recreation and Tourism Resources, Michigan State University.
Stynes, D. J., & Sun, Y.-Y. (2004). Economic impacts of national heritage area visitor spending; Summary results from seven national heritage area visitor surveys. East
Lansing, MI: Department of Community, Agriculture, Recreation and Resource Studies, Michigan State University.
Stynes, D. J., & Sun, Y.-Y. (2005a). Economic impacts of visitors to Grand Canyon National Park, 2003. East Lansing, MI: Department of Park, Recreation and Tourism
Resources, Michigan State University.
Stynes, D. J., & Sun, Y.-Y. (2005b). Impacts of visitor spending on the local economy:
Arches National Park 2003. East Lansing, MI: Department of Park, Recreation and Tourism Resources, Michigan State University.
Stynes, D. J., & White, E. M. (2006). Reflections on measuring recreation and travel spending. Journal of Travel Research, 45(1), 8-16.
Sun, Y.-Y., & Stynes, D. J. (2005). Survey biases from three National Park Service visitor studies. Paper presented at the Annual Travel & Tourism Research Association Conference, New Orleans, Louisiana, USA.
Tkaczynski, A., Rundle-Thiele, S. R., & Beaumont, N. (2009). Segmentation: A tourism stakeholder view. Tourism Management, 30(2), 169-175.
Weiler, S., Loomis, J., Richardson, R., & Shwiff, S. (2002). Driving regional economic models with a statistical model: Hypothesis testing for economic impact analysis. The Review of Regional Studies, 32(1), 97-111.
Wilton, J. J., & Nickerson, N. P. (2006). Collecting and using visitor spending data. Journal of Travel Research, 45(1), 17-25.
Wood, D., & Hughes, M. (2006). Tourism accommodation and economic contribution on the Ningaloo Coast of Western Australia. Tourism and Hospitality Planning &
Development, 3, 77-88.
World Tourism Organization. (1995). Collection of tourism expenditure statistics. Madrid:
World Tourism Organization.
Table 1 Parameters for simulation
Segments
Population
Prob. of obtaining incomplete spending information (φ) Mean Variance Size Model 1 Model 2 Group 1 67.19 8359 15,000 0.33 0.50 Group 2 134.85 10217 5,000 0.33 0.20 Group 3 446.56 243185 65,000 0.33 0.33 Group 4 160.41 13760 15,000 0.33 0.40 Total 100,000
Weighted average 331.69
Table 2 Summary results of model 1 with 200 and 400 complete data
Study
design nC,nI
Simulation mean
Simulation variance
Percentage of variance reduction
Estimated variance
Empirical coverage
1 200,0 332.46 965.37 950.52 0.939 2 200,n* 332.37 919.18 4.8% 907.75 0.938 3 200,800 332.33 858.15 11.1% 846.14 0.940 4 200,infinity 332.32 831.70 13.8% 819.80 0.939
1 400,0 331.81 470.85 471.73 0.948 2 400,n* 331.83 450.39 4.3% 450.64 0.949 3 400,1600 331.77 418.46 11.1% 420.40 0.948 4 400,infinity 331.72 405.56 13.9% 407.40 0.947
Table 3. The summary results of model 2 with 200, 400 complete data
Study design nC,nI
Simulation mean
Simulation variance
Percentage of variance reduction
Mean of estimated variance of
each run
Empirical coverage
1 200,0 342.83 967.42 970.31 0.945 2 200,n* 331.94 867.29 10.4% 868.02 0.942 3 200,800 331.98 806.31 16.7% 810.97 0.945 4 200,infinity 331.98 787.15 18.6% 784.88 0.943
1 400,0 342.85 480.99 485.29 0.939 2 400,n* 331.87 426.89 11.2 % 433.81 0.948 3 400,1600 331.86 398.85 17.1 % 405.36 0.948 4 400,infinity 331.87 385.84 19.8 % 392.40 0.948
Figure 1. Scenarios under the segmentation approach
Figure 2. The relationship between var(Tˆ ) and number of incomplete cases (nI)
Note: the asymptotic line equals to 404.75
Footnote
1 For detailed sampling methods and sample size, please refer to Ormer, Littlejohn, and Gramann (2001) and Simmons and Gramann (2001).
2 This report (Research Resolutions & Consulting Ltd, 2005) provides a precaution that
“excluding cases with no spending information” should be considered only if the target number of completions per cell is achieved. However, increasing sample size without targeting nonresponse does not reduce nonresponse bias (Lohr, 1999).
3 For model 2, it is more difficult to analyze the relationship between estimator variance and number of income cases (nI), because the assumption of nC,i ≈nC*pidoes not hold due to different response rate of full survey by individual segment (γi). Therefore, it must be analyzed case by case. In the example of model 2 with = 200, due to large sample theorem,
This implies that
)
estimator variance without obtaining additional information on segment shares ( = 0) is 1.163 times the estimator variance when segment shares are known. If is equal to , the estimator variance is 1.082 times that when segment shares are deterministic.
nI
nI nC
Appendix
Theorem 1 can be discussed in two parts and the proof is provided below.
Theorem 1.1：Suppose (Xj,Wj,Cj), j =1,2,K,nbe independent, identically distributed
in distribution, where
∑
= independent, identically distributed random variables which has the samedistribution as W1. Also,Wj, j =n+1,K,n+nI and (Xj,Wj,Cj), j=1, 2, are independent. Let
n
whenevern≥N.
The proof of Theorem 1.1 is similar as the proof of Theorem 1.2. Thus, we give only the proof of Theorem 1.2 here.
Proof：Let 1, 2, be independent, identically distributed 2m-dimensional random vectors each with mean
=
By Central Limit Theorem,
)
in distribution.
Suppose, L
covariance matrix of
∑*
Thus, even if L=1, we also have
M in distribution.
Finally, since and
lim in probability, it follows that
]
= in distribution, where
2
By weak law of large numbers,
i
Further details can be found in the technical report by Lai & Wong (2008), available upon request.