We take flat priors on both θ1and θ2and use the implied density (29) to validate convergence of MCMC samples. With WinBUGS Lunn et al. (2000) we obtain the posterior MCMC samples of sizes 10000 (burn-in=20000). In this example, the number of burn-in needs to be large to converge. Figures 17(a2) and (b2) show that the implied density (29) with s = 2 and s = 4 is close to the density of MCMC samples. Here [Σt]22 = 0.01247 and θˆt2 = 154.4327, and the posterior means of qi(Zt2), i = 1, ..., 4 from the MCMC samples in Figure 17(b2) are (0.450, 0.290, 0.699, 1.128) and ˆR is 1.0111 for θ2. Given sample size t = 9, we randomly draw 9 observations from Hurn et al. (1945). Figures 17(a1) and (b1) show that the implied density (29) with s = 2 and s = 4 is close to the density of MCMC samples. Here [Σt]22 = 0.00713 and ˆθt2 = 194.6036, and the posterior means of qi(Zt2), i = 1, ..., 4 from the MCMC samples in Figure 17(b1) are (0.315, 0.250, 1.143, 2.170).
4.6 Mixture Normal
In this section we consider mixture normal models where the posterior density of the pa-rameter may have multimodes. Suppose that a vector of observations y = (y1, ..., yt) is drawn from a mixture distribution of two normals:
p(y|θ) = w1φ(y; θ1, σ21) + w2φ(y; θ2, σ22), (43) where φ(y; θ, σ2) is normal density with mean θ and variance σ2, w1+ w2 = 1, w1 and θ1
are unknown parameters of interest, θ2, σ12, and σ22 are assumed known. The conjugate prior distributions are (w1, w2) ∼ Beta(α, β) and θi ∼ φ(µi, τi2) for i = 1, 2. For detailed discussions of mixture distributions, see Gilks et al. (1995, chapter 24) and Minka (2001).
It is convenient to introduce unobserved indicator variables ζi with
ζi =
1 if the ith observed is drawn from φ(θ1, σ21) 0 otherwise.
Given w1, each unobserved variable ζi follows Bernoulli distribution with mean w1.
‧
So, the full conditional distribution for ζi is Bernoulli distribution with mean ri. To con-struct the full conditionals for w1 and θ1, by (44),So, the full conditional for w1 is a Beta distribution, and the full conditional for θ1 is a normal distribution with mean b/(1/100 + h) and variance 1/(1/100 + h), where h =∑
ζi
and b = ∑
ζiyi. The Gibbs sampler is easy to apply for the mixture normals because the full conditional posterior distributions - π(ζ|θ1, w1, y), π(w1|θ1, ζ, y), π(θ1|w1, ζ, y)
-‧
have standard forms and can be easily sampled from. One cycle of the Gibbs sampler is described below.
The loglikelihood and the second partial derivatives are
`t(θ) = and generate a data of sample size 20. Then, we take the prior of θ1as N(0,100) and use the implied density (29) to validate convergence of MCMC samples. We run R function to obtain posterior samples of θ1by Gibbs sampler with sample sizes 5000 (burn-in=5000). The initial value for Gibbs sampler is (θ1(0), w(0)1 ) = (−2, 0.5). Figure 18 shows that the implied density (29) with s = 2 and s = 4 is very close to the MCMC result. For this data, the posterior distribution has a single mode and the posterior means of qi(Zt2) with Zt2= [Σt]22(θ1− ˆθt1), i = 1, . . . , 4 from the Gibbs samples are (−0.00076, 0.0786, 0.0184, 0.1693) and ˆR is 1.0024 for θ1. Next we let σ22= 10, meanwhile generating another data of sample size 20. We obtain
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
the posterior samples of θ1 by Gibbs sampler with sample sizes 5000 (burn-in=5000). The initial value for Gibbs sampler is also (θ1(0), w(0)1 ) = (−2, 0.5). The posterior samples have two modes. Figure 19(a) shows that the density of MCMC samples is close to the exact density by numerical integration and 19(b) shows the implied density (29) failed to diagnose for this example. The posterior means of qi(Zt2), i = 1, . . . , 4 from the Gibbs samples are (−2.643, 86.271, −925.97, 5850.66) and ˆR is 1.0016 for θ1.
We have a further discussion on this example and propose a method that transforms Zt to another pivotal quantity in chapter 5.
5 Concluding Remarks
We have compared second order Bayesian asymptotics and found some agreements and some disagreements. We found that our O(t−1/2) term is arithmetically equivalent to Johnson’s, but the O(t−1) term is not. Since the derivation is tedious and difficult to detect errors, simulation studies are conducted to further compare these expansions. The simulations confirmed that the two expressions for O(t−1/2) term yield close results, and revealed that our O(t−1) term gives better performance than Johnson’s. Note that the emphasis here is on comparison of the two expansions, rather than the regularity conditions for the expansions.
We also conduct simulations to compare the order O(t−3/2) expansions with Tierney and Kadane (1986) result.
Since the asymptotic posterior distribution depends on observed data, we try different samples to see its effect on the asymptotic approximations. Here we have a further discussion from both analytical and numerical (experimental) result.
Analytical results. In the appendix of Tierney and Kadane (1986), Laplace’s method provides an approximation for integrals of the form ∫
etL(θ)dθ when t is large. If L has a unique maximum at ˆθ and σ2 =−1/∇2L(ˆθ), then
∫
etL(θ)dθ =√
2πσt−1/2etL(ˆθ)(1 + a t + b
t2 + O(t−3)), (46)
where, setting Lk=∇kL(ˆθ), the constants a and b are given by a = 1
8σ4L4+ 5 24σ6L23
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
and
b = 1
48σ6L6+ 35
384σ8L24+ 7
48σ8L3L5+35
64σ10L23L4+ 385
1152σ12L43.
Result (46) remains valid if L is replaced by a sufficiently well-behaved sequence L(t) of functions. In this case the coefficients a and b may depend on t, but this dependence will be suppressed. If a and b do indeed depend on t, we will assume regularity conditions for the sequence L(t) that insure that a and b are bounded in t.
In Weng (2010a) (see Theorem 2, page 752), to ensure the marginal posterior density (Eq(23)), the following condition is required:
(A1) For each r > 0, Eξt(kZtkr) = O(1).
Here O(1) means convergence of a sequence of real numbers as t → ∞. However, it is difficult to judge O(1) from the posterior moments of Zt.
Numerical (experimental) result. Consider the same logit model in Section 4.1.2. Given sample sizes t = 12 and t = 15, we randomly draw three samples from 24 observations, respectively, and the comparisons of the density (Eq(23)) with exact density (numerical integration) are in Figures 20 and 21 with s = 2. Figure 20 contains the results of three different samples for t = 12. The posterior means of the three samples Zt2r, r = 1, . . . , 10 from numerical integration are in Table 1.1, where the seventh to tenth posterior moments of Zt2 for the three samples are large and it does not seem to satisfy the condition (A1).
The approximations are not well in Figures 20(a), (b) and (c).
Figure 21 contains the results of three different samples for t = 15. The posterior means of the three samples Zt2r, r = 1, . . . , 10 from numerical integration are in Table 1.2, where the seventh to tenth posterior moments of Zt2 for the three samples are also large and it does not seem to satisfy the condition (A1). In this example we found that the posterior moments of the first sample are relatively smaller than the other two samples. So, the approximations are not well in Figures 21(b) and (c); however, the approximation is well in Figure 21(a).
Now we consider the same observed data from the previous paragraph. The comparisons of the density (Eq(23)) and the MCMC density with both s = 2 and s = 7 are in Figures 22 and 23, where the sample sizes are t = 12 and t = 15, respectively. The posterior means of the three samples Zt2r, r = 1, . . . , 6 from MCMC samples are in Table 2.1, where the fifth
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
to sixth posterior moments of Zt2in the first sample are larger than the other two samples.
The first sample does not seem to satisfy the condition (A1) and the approximation is not well in Figure 22-1. However, the posterior moments of the second and third samples are relatively smaller than the first sample and the approximations are well in Figures 22-2 and 22-3.
The posterior means of the three samples Zt2r, r = 1, . . . , 6 from MCMC samples are in Table 2.2, where the fifth to sixth posterior moments of Zt2 in the third sample are larger than the other two samples. The third sample does not seem to satisfy the condition (A1) and the approximation is not well in Figure 23-3. However, the posterior moments of the first and second samples are relatively smaller than the third sample and the approximations are well in Figures 23-1 and 23-2.
posterior moments (exact density)
Zt2 Zt22 Zt23 Zt24 Zt25 Zt26 Zt27 Zt28 Zt29 Zt210 data
1 -0.55 1.76 -3.53 11.94 -37.29 139.1 -521.6 2100.7 -8596.0 36413.0 2 -0.44 1.58 -2.68 9.40 -26.51 99.36 -351.4 1406.9 -5562.5 23389.6 3 -0.27 1.43 -1.72 7.50 -17.20 72.31 -226.5 966.7 -3549.9 15526.2
Table 1.1 sample size t = 12.
posterior moments (exact density)
Zt2 Zt22 Zt23 Zt24 Zt25 Zt26 Zt27 Zt28 Zt29 Zt210 data
1 -0.19 1.28 -1.10 5.74 -10.14 47.47 -124.4 565.8 -1845.8 8401.6 2 -0.54 1.70 -3.44 11.34 -35.62 130.9 -491.2 1961.0 -8013.8 33771.0 3 -0.75 2.14 -5.39 18.01 -62.90 239.6 -949.5 3908.4 -16496.7 71105.2
Table 1.2 sample size t = 15.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
posterior moments (MCMC density) Zt2 Zt22 Zt23 Zt24 Zt25 Zt26 data
1 -0.51 1.75 -3.56 13.26 -47.28 208.5 2 -0.39 1.54 -2.29 8.37 -20.85 79.53 3 -0.24 1.36 -1.48 6.46 -13.14 54.34
Table 2.1 sample size t = 12.
posterior moments (MCMC density) Zt2 Zt22 Zt23 Zt24 Zt25 Zt26 data
1 -0.14 1.20 -0.79 4.67 -6.23 32.96 2 -0.52 1.64 -3.22 10.59 -33.25 123.8 3 -0.78 2.45 -7.41 30.61 -142.1 743.2
Table 2.2 sample size t = 15.
In Section 4.2.1, we have described the binomial model Y ∼ Bin(t, θ) with prior Beta(α, β) of θ for multi-stage data and compared the posterior density (Eq(37)) with the exact distri-bution. Now we try different priors to see its effect on the approximations. We compared the exact posterior distribution of θ given the first-stage data y1 with posterior density (Eq(37)) for four different priors; the results are in Figures 24(a), (b), (c) and (d), respec-tively. Next, we try four different second-stage data for each of the settings in Figures 24(a), (b), (c) and (d). The results are in Figures 25 to 28. We have some observations from Fig-ure 25 to FigFig-ure 28. First, the approximated posterior density (Eq(37)) is near the exact distribution when the MLE of second-stage data does not differ much from the posterior mode in the previous data, see Figures 25(a)(b), 26(a)(b), 27(a)(b) and 28(a)(b). Secondly, the approximation (Eq(37)) may not perform well and may have some negative values when the posterior mode in the previous data greatly differs from the MLE of second-stage data;
see Figures 25(c), 26(c), 27(c) and 28(c). Thirdly, the problem of negative values by ap-proximated posterior density (Eq(37)) may be improved when the sample size increases;
see Figures 25(d), 26(d), 27(d) and 28(d). Our experiments on multi-stage data showed that the analytic approximations are reasonably well when the MLE of second-stage data
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
does not differ much from the posterior mode in the previous data. However, when the MLE of second-stage data greatly differs from the posterior mode in the previous data, the approximations may be not well; the larger the sample size is, the better the result will be.
We also proposed a graphical method for validating convergence of MCMC. Our method is based on a Bayesian Edgeworth expansion for the posterior distribution. The method has been tested on some GLM models and mixture normal models, but the results are mixed.
One problem is that in some occasions the posterior densities seem to have heavier tails.
In fact, for a Bayesian Edgeworth expansion to be valid, the posterior moments need to be finite. So, intuitively a density with heavy tails may cause some problems. However, in a simulation sample, all moments can be calculated and are finite; therefore, it is difficult to judge whether the real moments are finite or not.
When the posterior moments of Zt are large, the asymptotic result is often not well.
We transform Zt to Wt = w ∗ Zt, where 0 < w < 1. Since Wt has smaller moments, we expect the asymptotic result for Wt to be better. By (22), Ztp involves only θp so that Pξt(θp ≤ ap) = Pξt(Ztp ≤ z∗p) for zp∗ = [Σt]pp(ap − ˆθtp). Let ϕp = wθp. We obtain Pξt(ϕp ≤ wap) = Pξt(Wtp ≤ w∗p) and Pξt(Wtp ≤ wp∗) = Pξt(Ztp ≤ w∗p/w), where w∗p = wzp∗ = [Σt]pp(wap− wˆθtp). Then, by (21) and (23), we derive the marginal posterior density for ϕp:
ξϕt(bp) = 1 w[Σt]pp
φ(w∗p
w ) + ∑
k∈{1,...,3s}
k6=3s−1
1 k!qk(wp∗
w )φ(w∗p
w )Eξt(qk(Wtp
w )) + O(t−3s+12 +s)
, (47)
where bp= wap. Two examples below are illustrated.
Logit model (three-parameter case.) Consider the same logit model in Section 4.3.
Here we take the posterior MCMC samples of parameters with sample sizes 10000 (burn-in=5000). Now we take w = 0.5 and to compare the posterior density (Eq(47)) with MCMC densities in Figure 29, where the posterior means of qi(Wt3), i = 1, . . . , 6 from the MCMC samples are (0.2786, -0.5833, -0.3912, 1.2865, 1.1674, -4.8815). The results are well.
Mixture model. Consider the same mixture model in Section 4.6. Here we take the pos-terior samples of θ1 with sample sizes 5000 (burn-in=5000) and Zt2= [Σt]22(θ1− ˆθt1). Now we take w = 0.5, w = 0.1 and to compare the posterior density (Eq(47)) with MCMC
den-‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
sities in Figures 30(a) and (b). In Figure 30(a), the posterior means of qi(Wt2), i = 1, . . . , 6 from the MCMC samples are (-1.3218, 20.8179, -112.77, 3561.29, -31987.67, 1253357). The result is not well. In Figure 30(b), the posterior means of qi(Wt2), i = 1, . . . , 6 from the MCMC samples are (-0.2643, -0.1272, -0.1407, 3.666, -5.229, 19.425) and we obtain better results for larger s.
When the numbers of both burn-in and MCMC sample are small, the asymptotic result for Zt is not well. We transform Zt to Wt to see its effect on the asymptotic result. Some examples below are illustrated.
Logit model (two-parameter case.) Consider the same logit model in Section 4.3. Here we take the posterior MCMC samples of parameters with sample sizes 100 (burn-in=50).
Next we take w = 0.1 and to compare the posterior densities (Eq(47)) with MCMC densi-ties in Figures 31(a) with s = 10 and 31(b) with s = 20. The posterior means of qi(Wt2), i = 1, . . . , 4 from the MCMC samples are (−0.0445, −0.9823, 0.1291, 2.895). The approxi-mations are not well.
Poisson model. Consider the same Poisson model in Section 4.4. Here we take the posterior MCMC samples of parameters with sample sizes 100 (burn-in=50). The posterior means of qi(Zt2), i = 1, . . . , 4 from the MCMC samples are (−0.1730, 0.2623, −1.004, 1.0059) and ˆR is 1.5873 for θ2. The comparisons of the density (Eq(47)) with the density (MCMC) are in Figures 32(a) with s = 10 and 32(b) with s = 20. The approximations are not well.
Now we take w = 0.1 and to compare the posterior densities (Eq(47)) with MCMC densities in Figure 33. The posterior means of qi(Wt2), i = 1, . . . , 4 from the MCMC samples are (−0.0173, −0.9873, 0.0503, 2.9248). The approximations are not well.
Mixture model. Consider the same mixture model in Section 4.6. Here we take the posterior samples of θ1 with sample sizes 50 (burn-in=50). The posterior means of qi(Zt2), i = 1, . . . , 4 from the MCMC samples are (0.84976, 1.8444, 8.3110, 40.6077) and ˆR is 1.1375 for θ1. The posterior samples have two modes. Figure 34(a) shows that the density (MCMC) is not close to the exact density (numerical integration). The comparisons of the density (Eq(47)) with the density (MCMC) are in Figures 34(b) with s = 7 and 34(c) with s = 27.
The approximations are not well. Now we take w = 0.1 and to compare the posterior densities (Eq(47)) with MCMC densities in Figure 35. The posterior means of qi(Wt2),
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
i = 1, . . . , 4 from the MCMC samples are (0.0849,−0.9715, −0.2440, 2.834). The approxi-mations are not well.
References
Albert, J. H. and Chib, S. (1993), “Bayesian analysis of binary and polychotomous response data”, Journal of the American Statistical Association, 88, 669–679.
Brooks, S. P. and Gelman, A. (1998), “General methods for monitoring convergence of iterative simulations”, Journal of Computational and Graphical Statistics, 7, 434–455.
Dellaportas, P. and Smith, A. F. M. (1993), “Bayesian inference for generalized linear and proportional hazards models via gibbs sampling”, Appl. Statist., 42, 443–460.
Erdelyi, A. (1956), Asympotic Expansions, New York: Dover Publications.
Finney, D. J. (1947), “The estimation from individual records of the relationship between dose and quantal response”, Biometrika, 34, 320–334.
Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (1995), Bayesian data analysis, New York : Chapman and Hall.4.
Gelman, A. and Rubin, D. B. (1992), “Inference form iterative simulation using multiple sequences”, Statistical Science, 7, 457–511.
Geman, S. and Geman, D. (1984), “Stochastic relaxation, gibbs distributions and the bayesian restoration of images”, IEEE Trans. Pattn. Anal. Mach. Intel, 6, 721–741.
Gilks, W. R., Richardson, S., and Spiegelhalter, D. J. (1995), Markov Chain Monte Carlo in Practice, London: Chapman and Hall.
Gilks, W. R. and Wild, P. (1992), “Adaptive rejection sampling for gibbs sampling”, Appl.
Statist., 41, 337–348.
Glickman, Mark E. (1993), “Paired comparison models with time-varying parameters”, Ph.D. thesis, Department of Statistics, Harvard University.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
Hinde, J. (1982), Compound Poisson regression models, New York: Springer.
Hurn, M. W., Barker, N. W., and Magath, T. D. (1945), “The determination of prothrombin time following the administration of dicumarol with specific reference to thromboplastin”, Med., 30, 432–447.
Johnson, R. (1967), “An asymptotic expansion for posterior distributions”, Ann. Math.
Statist., 38, 1899–1906.
(1970), “Asymptotic expansions associated with posterior distributions”, Ann.
Math. Statist., 41, 851–864.
Kutner, M. H., Nachtsheim, C. J., and Neter, J. (2004), Applied linear regression models, Boston: McGraw.
Lunn, D.J., Thomas, A., Best, N., and Spiegelhalter, D. (2000), “Winbugs – a bayesian modelling framework: concepts structure and extensibility”, Statistics and Computing, 10, 325–337.
Martin, Andrew D., Quinn, Kevin M., and Park, Jong Hee (2010), “MCMCpack:
Markov chain Monte Carlo (MCMC) Package”, R package version 2.10.0, URL http://CRAN.R-project.org/package=MCMCpack.
McCullagh, P. and Nelder, J. A. (1989), Generalized linear models, New York : Chapman and Hall.
Mendenhall, W. M., Parsons, J. T., Stringer, S. P., Cassissi, N. J., and Million, R. R.
(1989), “T2 oral tongue carcinoma treated with radiotherapy: analysis of local control and complications”, Radiotherapy and Oncology, 16, 275–282.
Minka, T. P. (2001), “A family of algorithms for approximate bayesian inference”, Depart-ment of Electrical Engineering and Computer Science, 1–75.
Myers, R. H. (1990), Classical and Modern Regression with Applications, Boston: PWS-Kent.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
R Development Core Team (2010), “R: A language and environment for statistical com-puting”, R Foundation for Statistical Computing, Vienna, Austria, ISBN 3-900051-07-0, URL http://www.R-project.org.
Tanner, M. A. (1993), Tools for Statistical Inference, New York: Springer.
Tierney, L. and Kadane, J. B. (1986), “Accurate approximations for posterior moments and marginal densities”, Journal of the American Statistical Association, 81, 82–86.
Weng, R. C. (2003), “On Stein’s identity for posterior normality”, Statistica Sinica, 13, 495–506.
(2010a), “A Bayesian Edgeworth expansion by Stein’s Identity”, Bayesian Analysis, 5, 741–764.
(2010b), “A note on posterior distributions”, in JSM Proceedings, Section on Bayesian Statistical Science, 2599–2608, Alexandria, VA: American Statistical Associ-ation.
Weng, R. C. and Hsu, C.-H. (2011), “A study of expansions of posterior distributions”, Communications in Statistics - Theory and Methods.
Weng, R. C. and Tsai, W.-C. (2008), “Asymptotic posterior normality for multiparameter problems”, Journal of Statistical Planning and Inference, 138, 4068–4080.
Woodroofe, M. (1989), “Very weak expansions for sequentially designed experiments: linear models”, Ann. Statist., 17, 1087–1102.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a) (b)
0.0 0.2 0.4 0.6 0.8 1.0
0.01.02.03.0
θ
marginal posterior density
0.0 0.2 0.4 0.6 0.8 1.0
0246
θ
marginal posterior density
Figure 6: Marginal posterior pdf of θ. Beta-Binomial model.
Sequential update. Dotted: Equation (37); Dot-dashed: Exact distribution.
(a)(t1, y1) = (8, 3), p(θ|y1).
(b)(t1, y1) = (8, 3), (t2, y2) = (50, 12) two stage approximation of p(θ|y1, y2).
−0.3 −0.2 −0.1 0.0 0.1
−505101520
θ2
marginal posterior density
Figure 7: Marginal posterior pdf of θ2. Sequential update for logit model. Dotted: Equation (37); Dashed: MCMC(burn-in=5000, mcmc=10000) as Exact.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a) s = 2
−5 0 5 10
0.000.100.200.30
θ1
marginal posterior density
−0.3 −0.2 −0.1 0.0 0.1
024681012
θ2
marginal posterior density
(b) s = 4
−5 0 5 10
0.000.050.100.150.20
θ1
marginal posterior density
−0.3 −0.2 −0.1 0.0 0.1
0246810
θ2
marginal posterior density
(c) s = 20
−5 0 5 10
0.00.10.20.3
θ1
marginal posterior density
−0.3 −0.2 −0.1 0.0 0.1
0510
θ2
marginal posterior density
(d) s = 27
−5 0 5 10
0.00.10.20.3
θ1
marginal posterior density
−0.3 −0.2 −0.1 0.0 0.1
051015
θ2
marginal posterior density
Figure 8: Marginal posterior pdf of θ1 and θ2. Logit model with flat prior.
Dashed: MCMC(burn-in=50, mcmc=100);
Solid: Equation (29) with s = 2, s = 4, s = 20 and s = 27.
R is 1.2309 for θˆ 2.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a) s = 2
−5 0 5 10
0.000.050.100.150.20
θ1
marginal posterior density
−0.3 −0.2 −0.1 0.0 0.1
02468
θ2
marginal posterior density
(b) s = 4
−5 0 5 10
0.000.050.100.150.20
θ1
marginal posterior density
−0.3 −0.2 −0.1 0.0 0.1
02468
θ2
marginal posterior density
Figure 9: Marginal posterior pdf of θ1 and θ2. Logit model with flat prior.
Dashed: MCMC(burn-in=5000, mcmc=10000);
Solid: Equation (29) with s = 2 and s = 4.
R is 1.0020 for θˆ 2.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a) s = 2
−5 0 5 10
−0.10.00.10.20.30.4
θ1
marginal posterior density
−0.3 −0.2 −0.1 0.0 0.1
051015
θ2
marginal posterior density
(b) s = 4
−5 0 5 10
0.00.10.20.30.4
θ1
marginal posterior density
−0.3 −0.2 −0.1 0.0 0.1
051015
θ2
marginal posterior density
Figure 10: Marginal posterior pdf of θ1 and θ2. Logit model with N(0,1) prior.
Dashed: MCMC(burn-in=5000, mcmc=10000);
Solid: Equation (29) with s = 2 and s = 4.
R is 1.0001 for θˆ 2.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a)s = 2
−25 −20 −15 −10 −5 0 5
−0.050.050.150.25
θ1
marginal posterior density
−2 0 2 4 6 8 10
0.00.10.20.30.40.5
θ2
marginal posterior density
−2 0 2 4 6
−0.20.00.20.40.60.8
θ3
marginal posterior density
(b)s = 4
−25 −20 −15 −10 −5 0 5
−0.6−0.20.20.6
θ1
marginal posterior density
−2 0 2 4 6 8 10
−0.40.00.40.8
θ2
marginal posterior density
−2 0 2 4 6
−2−10123
θ3
marginal posterior density
Figure 11: Marginal posterior pdf of θ1, θ2 and θ3. Logit model with flat prior.
Dashed: MCMC(burn-in=5000, mcmc=10000);
Solid: Equation (29) with s = 2 and s = 4.
R is 1.0022 for θˆ 3.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a)s = 2
−25 −20 −15 −10 −5 0 5
0.000.040.080.12
θ1
marginal posterior density
−2 0 2 4 6 8 10
0.000.100.20
θ2
marginal posterior density
−2 0 2 4 6
0.00.10.20.30.4
θ3
marginal posterior density
(b)s = 4
−25 −20 −15 −10 −5 0 5
0.000.040.080.12
θ1
marginal posterior density
−2 0 2 4 6 8 10
0.000.100.20
θ2
marginal posterior density
−2 0 2 4 6
0.00.10.20.30.4
θ3
marginal posterior density
Figure 12: Marginal posterior pdf of θ1, θ2 and θ3 (truncated). Logit model with flat prior.
Dashed: MCMC(burn-in=5000, mcmc=10000);
Solid: Equation (29) with s = 2 and s = 4.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a)s = 2
−25 −20 −15 −10 −5 0 5
−0.20.00.10.20.3
θ1
marginal posterior density
−2 0 2 4 6 8 10
−0.20.00.20.40.6
θ2
marginal posterior density
−2 0 2 4 6
−0.50.00.51.0
θ3
marginal posterior density
(b)s = 4
−25 −20 −15 −10 −5 0 5
−0.20.00.10.20.3
θ1
marginal posterior density
−2 0 2 4 6 8 10
−0.20.00.20.40.6
θ2
marginal posterior density
−2 0 2 4 6
−0.50.00.51.0
θ3
marginal posterior density
(c)s = 14
−25 −20 −15 −10 −5 0 5
−0.10.10.30.5
θ1
marginal posterior density
−2 0 2 4 6 8 10
0.00.20.40.60.81.0
θ2
marginal posterior density
−2 0 2 4 6
0.00.40.81.2
θ3
marginal posterior density
Figure 13: Marginal posterior pdf of θ1, θ2 and θ3 (truncated). Logit model with N(0,1) prior.
Dashed: MCMC(burn-in=5000, mcmc=10000);
Solid: Equation (29) with s = 2, s = 4 and s = 14.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a) (b) (c)
0 5 10
0.000.050.100.150.20
θ1
marginal posterior density
−0.12 −0.08 −0.04 0.00
0510152025
θ2
marginal posterior density
0 2 4 6
0.00.10.20.30.4
θ3
marginal posterior density
(d) (e) (f)
−2 0 2 4 6
0.00.10.20.30.40.5
θ4
marginal posterior density
−1 0 1 2 3 4
0.00.10.20.30.40.50.6
θ5
marginal posterior density
−1 0 1 2 3 4 5
0.00.10.20.30.40.50.6
θ6
marginal posterior density
Figure 14: Marginal posterior pdf of θ1. . . θ6. Logit model with flat prior.
Dashed: MCMC(burn-in=10000, mcmc=10000);
Solid: Equation (29) with s = 4.
R is 1.0114 for θˆ 6.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a1)s = 2 and sample size: t = 16
0.0 0.5 1.0 1.5 2.0
0.00.51.01.5
θ1
marginal posterior density
0.000 0.001 0.002 0.003 0.004
0200400600800
θ2
marginal posterior density
(b1)s = 4 and sample size: t = 16
0.0 0.5 1.0 1.5 2.0
0.00.51.01.5
θ1
marginal posterior density
0.000 0.001 0.002 0.003 0.004
0200400600800
θ2
marginal posterior density
(a2)s = 2 and sample size: t = 32
0.0 0.5 1.0 1.5 2.0
0.00.51.01.5
θ1
marginal posterior density
0.0005 0.0015 0.0025 0.0035
02006001000
θ2
marginal posterior density
(b2)s = 4 and sample size: t = 32
0.0 0.5 1.0 1.5 2.0
0.00.51.01.5
θ1
marginal posterior density
0.0005 0.0015 0.0025 0.0035
02006001000
θ2
marginal posterior density
Figure 15: Marginal posterior pdf of θ1 and θ2. Poisson model with flat prior.
Dashed: MCMC(burn-in=5000, mcmc=10000).
Solid: Equation (29) with s = 2 and s = 4.
R is 1.0004 for θˆ 2 with t=32.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a1)s = 2 and sample size: t = 16
0.0 0.5 1.0 1.5 2.0
0.00.51.01.5
θ1
marginal posterior density
0.000 0.001 0.002 0.003 0.004
0200400600800
θ2
marginal posterior density
(b1)s = 4 and sample size: t = 16
0.0 0.5 1.0 1.5 2.0
0.00.51.01.5
θ1
marginal posterior density
0.000 0.001 0.002 0.003 0.004
0200400600800
θ2
marginal posterior density
(a2)s = 2 and sample size: t = 32
0.0 0.5 1.0 1.5 2.0
0.00.51.01.5
θ1
marginal posterior density
0.0005 0.0015 0.0025 0.0035
02006001000
θ2
marginal posterior density
(b2)s = 4 and sample size: t = 32
0.0 0.5 1.0 1.5 2.0
0.00.51.01.52.0
θ1
marginal posterior density
0.0005 0.0015 0.0025 0.0035
020060010001400
θ2
marginal posterior density
Figure 16: Marginal posterior pdf of θ1 and θ2. Poisson model with N(0,1) prior.
Dashed: MCMC(burn-in=5000, mcmc=10000);
Solid: Equation (29) with s = 2 and s = 4.
R is 1.0011 for θˆ 2 with t=32.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a1)s = 2 and sample size: t = 9
−200 −100 0 100
0.0000.0040.008
θ1
marginal posterior density
−400 0 200 600
0.00000.00100.00200.0030
θ2
marginal posterior density
(b1)s = 4 and sample size: t = 9
−200 −100 0 100
0.0000.0040.008
θ1
marginal posterior density
−400 0 200 600
0.00000.00100.00200.0030
θ2
marginal posterior density
(a2)s = 2 and sample size: t = 18
−100 −50 0 50
0.0000.0050.0100.015
θ1
marginal posterior density
−200 0 200 400
0.0000.0020.004
θ2
marginal posterior density
(b2)s = 4 and sample size: t = 18
−100 −50 0 50
0.0000.0050.0100.0150.020
θ1
marginal posterior density
−200 0 200 400
0.0000.0020.004
θ2
marginal posterior density
Figure 17: Marginal posterior pdf of θ1and θ2. Gamma model(identity link) with flat prior.
Dashed: MCMC(burn-in=20000, mcmc=10000);
Solid: Equation (29) with s = 2 and s = 4.
R is 1.0111 for θˆ 2 with t=18.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a) s = 2 (b) s = 4
1.0 1.5 2.0 2.5 3.0 3.5
0.00.40.81.2
θ1
marginal posterior density
1.0 1.5 2.0 2.5 3.0 3.5
0.00.40.81.2
θ1
marginal posterior density
Figure 18: Marginal posterior pdf of θ1 with ˆR = 1.0024. Mixture normal model.
Dashed: MCMC(burn-in=5000, mcmc=5000);
Solid: Equation (29) with s = 2 and s = 4.
(a) (b)
−5 0 5 10
0.00.10.20.30.4
θ1
marginal posterior density
0 1 2 3 4
−4e+770e+004e+77
θ1
marginal posterior density
Figure 19: Marginal posterior pdf of θ1 with ˆR = 1.0016. Mixture normal model.
Dashed: MCMC(burn-in=5000, mcmc=5000); Solid: Equation (29).
(a)Dot-dashed: Exact distribution by numerical integration; (b)s = 27.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a) (b) (c)
−0.6 −0.4 −0.2 0.0 0.2 0.4
−1005101520
θ2
marginal posterior density
−0.8 −0.4 0.0 0.2 0.4 0.6
02468
θ2
marginal posterior density
−0.6 −0.2 0.2 0.4 0.6
0123456
θ2
marginal posterior density
Figure 20: Marginal posterior pdf of θ2 with t = 12. Logit2p-flat model.
Dotted: Equation (23); Dot-dashed: Exact distribution by numerical integration.
(a), (b) and (c) with s = 2.
(a) (b) (c)
−0.6 −0.4 −0.2 0.0 0.2 0.4
0123456
θ2
marginal posterior density
−0.5 0.0 0.5
−50510
θ2
marginal posterior density
−1.0 −0.5 0.0 0.5
−30−100102030
θ2
marginal posterior density
Figure 21: Marginal posterior pdf of θ2 with t = 15. Logit2p-flat model.
Dotted: Equation (23); Dot-dashed: Exact distribution by numerical integration.
(a), (b) and (c) with s = 2.
‧
Figure 22: Marginal posterior pdf of θ1 and θ2 with t = 12. Logit2p-flat model.
Dashed: MCMC(burn-in=5000, mcmc=10000); Dotted: Equation (23).
(a1) and (b1) with s = 2; (a2) and (b2) with s = 7.
Figure 23: Marginal posterior pdf of θ1 and θ2 with t = 15. Logit2p-flat model.
Dashed: MCMC(burn-in=5000, mcmc=10000); Dotted: Equation (23).
(a1) and (b1) with s = 2; (a2) and (b2) with s = 7.
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a) (b)
0.0 0.2 0.4 0.6 0.8 1.0
0.01.02.03.0
θ
marginal posterior density
0.0 0.2 0.4 0.6 0.8 1.0
0.01.02.03.0
θ
marginal posterior density
(c) (d)
0.0 0.2 0.4 0.6 0.8 1.0
0123
θ
marginal posterior density
0.0 0.2 0.4 0.6 0.8 1.0
−2024
θ
marginal posterior density
Figure 24: Marginal posterior pdf of θ. Beta-Binomial model (t1, y1) = (8, 3).
Dotted: Equation (37); Dot-dashed: Exact distribution.
(a) prior Beta(0.5, 4). (b) prior Beta(2.5, 4). (c) prior Beta(4, 6). (d) prior Beta(6, 2).
(a) (b)
0.0 0.2 0.4 0.6 0.8 1.0
012345
θ
marginal posterior density
0.0 0.2 0.4 0.6 0.8 1.0
0123456
θ
marginal posterior density
(c) (d)
0.0 0.2 0.4 0.6 0.8 1.0
−20246
θ
marginal posterior density
0.0 0.2 0.4 0.6 0.8 1.0
02468
θ
marginal posterior density
Figure 25: Marginal posterior pdf of θ. Beta-Binomial prior Beta(0.5, 4). (t1, y1) = (8, 3).
Dotted: Equation (37); Dot-dashed: Exact distribution.
(a)(t2, y2) = (12, 3). (b)(t2, y2) = (25, 6). (c)(t2, y2) = (20, 10). (d)(t2, y2) = (80, 40).
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a) (b)
0.0 0.2 0.4 0.6 0.8 1.0
01234
θ
marginal posterior density
0.0 0.2 0.4 0.6 0.8 1.0
012345
θ
marginal posterior density
(c) (d)
0.0 0.2 0.4 0.6 0.8 1.0
−1012345
θ
marginal posterior density
0.0 0.2 0.4 0.6 0.8 1.0
0246
θ
marginal posterior density
Figure 26: Marginal posterior pdf of θ. Beta-Binomial prior Beta(2.5, 4). (t1, y1) = (8, 3) Dotted: Equation (37); Dot-dashed: Exact distribution.
(a)(t2, y2) = (10, 3). (b)(t2, y2) = (20, 7). (c)(t2, y2) = (20, 12). (d)(t2, y2) = (60, 36)
(a) (b)
0.0 0.2 0.4 0.6 0.8 1.0
01234
θ
marginal posterior density
0.0 0.2 0.4 0.6 0.8 1.0
0123456
θ
marginal posterior density
(c) (d)
0.0 0.2 0.4 0.6 0.8 1.0
−20246
θ
marginal posterior density
0.0 0.2 0.4 0.6 0.8 1.0
02468
θ
marginal posterior density
Figure 27: Marginal posterior pdf of θ. Beta-Binomial prior Beta(4, 6). (t1, y1) = (8, 3) Dotted: Equation (37); Dot-dashed: Exact distribution.
(a)(t2, y2) = (16, 6). (b)(t2, y2) = (32, 12). (c)(t2, y2) = (20, 12). (d)(t2, y2) = (60, 36)
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a) (b)
0.0 0.2 0.4 0.6 0.8 1.0
01234
θ
marginal posterior density
0.0 0.2 0.4 0.6 0.8 1.0
012345
θ
marginal posterior density
(c) (d)
0.0 0.2 0.4 0.6 0.8 1.0
−1012345
θ
marginal posterior density
0.0 0.2 0.4 0.6 0.8 1.0
02468
θ
marginal posterior density
Figure 28: Marginal posterior pdf of θ. Beta-Binomial prior Beta(6, 2). (t1, y1) = (8, 3) Dotted: Equation (37); Dot-dashed: Exact distribution.
(a)(t2, y2) = (10, 7). (b)(t2, y2) = (20, 12). (c)(t2, y2) = (10, 2). (d)(t2, y2) = (80, 16)
‧ 國
立 政 治 大 學
‧
N a tio na
l C h engchi U ni ve rs it y
(a)s = 2
−20 −15 −10 −5 0 5 10
0.000.050.100.150.20
ϕ1
marginal posterior density
−4 −2 0 2 4 6 8
0.00.10.20.30.4
ϕ2
marginal posterior density
−2 0 2 4 6
0.00.20.40.6
ϕ3
marginal posterior density
(b)s = 4
−20 −15 −10 −5 0 5 10
0.000.050.100.150.20
ϕ1
marginal posterior density
−4 −2 0 2 4 6 8
0.00.10.20.30.40.5
ϕ2
marginal posterior density
−2 0 2 4 6
0.00.20.40.6
ϕ3
marginal posterior density
Figure 29: Marginal posterior pdf of ϕ1, ϕ2 and ϕ3 (Wt= 0.5∗ Zt). Logit model with flat
Figure 29: Marginal posterior pdf of ϕ1, ϕ2 and ϕ3 (Wt= 0.5∗ Zt). Logit model with flat