Large Sample Theory
Homework 1: Bootstrap Method, CLT Due Date: October 3rd, 2004
1. Suppose that someone collects a random sample of size 4 of a particular mea- surement. The observed values are {2, 4, 9, 12}.
(a) Find the bootstrap mean and variance of the above sample.
(b) Find the relationship between sample mean and bootstrap mean.
(c) Find the relationship between sample variance and bootstrap variance.
(d) Consider a general problem in which we have a random sample {x1, . . . , xn}.
Establish the relationships you found in (b) and (c) under this setting.
2. Let X1, . . . , Xn be i.i.d. random variables and ˆθ = ¯X2. (a) Derive the asymptotic distribution of√
n[ˆθ − E2(X)].
(b) Show that the bootstrap variance estimator based on i.i.d. Xi∗’s from Fn is equal to
V¯B = 4 ¯X2ˆc2
n +4 ¯X ˆc3
n2 + cˆ4
n3,
where ˆcj’s are the sample central moments defined by n−1Pni=1(Xi− ¯X)j. (c) Is ¯VB a consistent estimate of the asymptotic variance derived in (a)?
Hint: For (a), you can use Slutsky’s theorem.
3. Let X1, . . . , Xn be i.i.d. from a pdf σ−1f ((x − µ)/σ) on R, where f is known.
Let
Hn(t) = P
√n( ¯X − µ)
S ≤ t
!
and
HˆB(t) = P
√n( ¯X∗− ¯X)
S∗ ≤ t
X1, . . . , Xn
!
be the bootstrap estimator of Hn, where S2 is the sample variance, Xi∗’s are i.i.d. from s−1f ((x − ¯x)/s), given ¯X = ¯x and S = s, and S∗ is the bootstrap analogue of S. Show that Hn(t) = ˆHB(t).
4. Suppose Yi = α + βxi+ i for i = 1, 2, . . ., where the xi are known numbers not all equal and the j are independent random variables with mean 0 and common variances σ2.
(a) Show that the least-squares estimate, ˆβn, of β was consistent provided
Pn
i=1(xi− ¯xn)2 → ∞.
(b) Show that ˆβnis asymptotically normal when the j are identically distributed and
1≤j≤nmax
(xj− ¯xn)2
Pn
i=1(xi− ¯xn)2 → 0 as n → ∞.
5. Suppose we would like to estimate mean (θ) of a population and a random sample {x1, . . . , xn} is being taken. Consider ˆθ = ¯X.
(a) Find the jackknife estimate of θ.
(b) Find the jackknife variance estimate of θ.
6. Let X = (X1, . . . , Xn)T be an n × 1 vector of random variables and let A be an n × n symmetric matrix. If E(X) = θ and V ar(X) = Σ = (σij)n×n, show that
E[XTAX] = tr[AΣ] + θTAθ and
V ar[XTAX] = (µ4− 3µ22)aTa + 2µ22tr(A2) + 4µ2θTA2θ + 4µ3θTAa, where µr = E[(Xi− θi)r] and a is the column vector of the diagonal elements of A.
7. Let Xi be iid and real-valued for i = 1, . . . , n; 4th moments exist. The sample variance is s2.
(a) Find the mean and variance of s2.
(b) How would you estimate V ar(s2) from the sample by the method of mo- ments?
(c) How would you estimate V ar(s2) from the sample by the jackknife?
(d) How would you estimate V ar(s2) if the parent distribution was normal?
How does the normal-theory estimator behave if the parent distribution is not normal?
Hint: You can solve (a) using the results derived in last question.
8. Let X1, . . . , Xn be i.i.d. N (µ, σ2). Let V =Pnj=1(Xj − ¯X)2.
(a) For what constants c1(n) depending on n is c1(n)V an unbiased estimator of σ2?
(b) For what constants c2(n) depending on n is the mean-square error Eσ2[(c1(n)V − σ2)2] minimized for all σ > 0?
9. Find the jackknife estimate of the variance of the median. Is this estimate a good one? You can use either a simulation or an analytic method to address this question by assuming that the unknown population distribution is the uniform distribution on the interval [0, 1].
10. A random variable ˆθn, based on a random sample of size n > 1, is said to be an unbiased estimator of a parameter θ if E(ˆθn) = θ for all θ ∈ Θ, the parameter space. Suppose that ˆθn is unbiased for θ and that V (ˆθn) = O(n−1).
(a) If X1, X2, . . . , Xn is a random sample from N (µ, σ2), show that no unbiased estimator of eµ can be found based on the sample mean ¯X.
(c) Show that E(exp( ¯X)) = exp(µ + σ2/2n) and E(eS2/2n) = exp(σ2/2n) + o(n−1), where S2 =Pni=1(Xi− µ)2/n. Hence find an estimator of exp(µ) that is unbiased apart from terms of smaller order than n−1.
2
