Large Sample Theory

Large Sample Theory

Homework 4: Methods of Estimation, Asymptotic Distribution, Probability and Conditioning Due Date: December 1st

1. The Weibull distribution (after the Swedish physicist Waloddi Weibull, who proposed the distribution in 1939 for the breaking strength of materials), has density function

f (x) = λx^λ−1exp^−x^λ for x, λ > 0.

[As an aside, note that the Weibull arises by assuming y = x^λMfollows an exponential distribution].

a. What is the resulting likelihood function `(λ|x1, . . . , x_n), for λ?

b. What is the resulting log-likelihood function?

c. What is the score function?

d. What is the second derivative of the log-likelihood function?

e. Suppose 5 values, 0.10, 0.25, 0.5, 1, and 2 are observed. Plot the resulting log- likelhood function

f. What is the approximate sample variance?

g. What is an approximate 95% confidence interval for λ?

2. Let X be N (0, θ), 0 < θ < ∞.

a. Find the Fisher information I(θ).

b. If X1, X₂, . . . , X_nis a random sample from this distribution, show that the MLE of θ is an efficient estimator of θ.

3. For Type II censoring, the data consist of the rth smallest lifetimes X₍₁₎ ≤ X₍₂₎ ≤

· · · ≤ X(r) out of a random sample of n lifetimes X1, . . . , Xn from the assumed life distribution. Assuming X₁, . . . , X_n are i.i.d. and have a continuous distribution with p.d.f. f (x) and survival function S(x).

a. Show that the joint p.d.f. of X₍₁₎, X₍₂₎, · · · X_(r)is L_II,1= n!

(n − r)!

" _r Y

i=1

f (x_(i))

#

[S(x_(r)]^n−r.

b. Suppose that X_i is an exponentially distributed random variable with mean θ. De- rive the MLE of θ, ˆθ, and state the condition on r to guarantee consistency of ˆθ.

c. Use EM algorithm to derive the MLE of θ.

4. The normally distributed random variables X1, . . . , X_nare said to be serially correlated or to follow an autoregressive model if we can write

X_i = θX_i−1+ _i, i = 1, . . . , n,

where X0 = 0 and 1, . . . , nare independent N (0, σ²) random variables.

a. Show that the density of (X₁, . . . , X_n) is 1

(2πσ²)^n/2 exp

(

−

Pn

i=1(x_i− θx_i−1)² 2σ²

)

for −∞ < x_i < ∞, i = 1, . . . , n, x₀ = 0.

1

b. Derive MLE of θ and σ². Give a condition on θ so that they are consistent estimates.

5. Let Y_i denote the response of a subject at time i, i = 1, . . . , n. Suppose that Y_i satisfies the following model

Y_i = θ + _i, i = 1, . . . , n

where i can be written as i = ce_i−1+ e_i for a given constant c satisfying 0 ≤ c ≤ 1, and the e_i are independent and identically distributed with mean zero and variance σ², i = 1, . . . , n; ₀ = 0. Let

Y =¯ 1 n

n

X

i=1

Y_i, ˆθ =

n

X

j=1

a_jY_j

where

a_j =

n−j

X

i=0

(−c)^j 1 − (−c)^j+1 1 + c

!

/

n

X

i=1

1 − (−c)ⁱ 1 + c

!2

. a. Show that if ei ∼ N (0, σ²), then ˆθ is the MLE of θ.

b. Show that ¯Y and ˆθ are unbiased.

c. Show that V ar( ¯Y ) ≥ V ar(ˆθ).

d. Show that ¯Y and ˆθ are consistent estimates of θ.

6. Suppose that X₁, . . . , X_n are independent and identically distributed according to a lo- cation family with cdf F (x − θ), with F known and with 0 < F (x) < 1 for all x, but that it is only observed whether each X_ifalls below a, between a and b, or above b where a < b are two given constants.

a. Describe the joint distribution of the observed three outcomes.

b. Let V denote the number of observations less than a. Describe the asymptotic distribution of√

n(V /n − p₁) where p₁ = F (a − θ).

c. Show that ˜Vn = a − F⁻¹(V /n) is a consistent estimate of θ. Derive the asymptotic distribution of√

n( ˜V_n− θ)

7. Let X1, . . . , X_nbe iid with distribution Pθdepending on a real-valued parameter θ, and suppose that E_θ(X) = g(θ) and V ar_θ(X) = τ (θ) < ∞, where g is continuously differentiable function with derivative g⁰(θ) > 0 for all θ. Denote the estimator obtained by the method of moments by ˆθ. ( i.e., ˆθ is the solution of the equation g(θ) = ¯X.)

a. Show that ˆθ is consistent.

b. Derive its asymptotic distribution.

8. Suppose that v_i and u_i, 1 ≤ i ≤ n, are associated with a linear relationship v_i = a + bu_i. Due to data collection error, we can only observe (x_i, y_i) where y_i = v_i+ δ_i and x_i = u_i+ _i. It is known that E(δi) = E(_i) = 0 and δ_i and i are to be independent. Note that y_i = a + bx_i+ (δ_i− b_i) and E(δ_i− b_i) = 0.

a. When V ar(_i) = V ar(δ_i) = σ², show that the least squares estimate of b (based on (x_i, y_i)) is not consistent when n^−1Pn_i=1(u_i− ¯u)²goes to a nonzero constant c.

b. Propose a consistent estimate of b when V ar(δ_i) = 2V ar(_i).

2

9. Let X1, . . . , Xn be iid according to the normal distribution N (θ, 1). Consider the se- quence of estimators

δn =

( X¯ if | ¯X| ≥ n^−1/4 a ¯X if | ¯X| < n^−1/4 Find the asymptotic distribution of√

n(δ_n− θ).

Hint: You may need to derive your answer for θ = 0 and θ 6= 0 separately.

10. Show the following properties of the multivariate normal distribution N_k(µ, Σ) where µ ∈ R^kand Σ is a positive definite k × k matrix. Note that, if X ∼ Nk(µ, Σ), its pdf is

f (x) = (2π)^−k/2[Det(Σ)]^−1/2exp(−(x − µ)^TΣ⁻¹(x − µ)).

(a) The mgf of N_k(µ, Σ) is exp(µ^Tt + t^TΣt/2).

Fact: The mgf of X is defined as E exp(X^Tt).

(b) Let X be a random k-vector having the N_k(µ, Σ) distribution and Y = AX + c, where A is a k × ` matrix of rank ` ≤ k and c ∈ R<sup>`</sup>. Then Y has the N<sub>`</sub>(Aµ + c, A^TΣA) Distribution.

Fact: If X and Y are random k-vectors and their mgf are identical for all t ∈ N_ = {t ∈ R^k : ktk ≤ }, then the distribution of X is identical to that of Y.

(c) A random k-vector X has a k-dimensional normal distribution if and only if for any c ∈ R^k, X^Tc has a univariate normal distribution.

(d) Let X be a random k-vector having the N_k(µ, Σ) distribution. Let A be a k ×` matrix and B be a k × m matric. Then XA and XB are independent if and only if they are uncorrelated.

(e) Let (X^T₁, X^T₂)^T be a random k-vector having the N_k(µ, Σ) distribution with Σ = Σ11 Σ12

Σ₂₁ Σ₂₂

!

,

where X1 is a random `-vector and Σ11is an ` × ` matrix. Then the conditional pdf of X₂ given X₁ is

Nk−`(µ2+ (x1− µ1)Σ⁻¹₁₁Σ12, Σ22− Σ21Σ⁻¹₁₁Σ12), where µ_i = E(X_i), i = 1, 2.

Hint: Consider X₂− µ₂− (X₁− µ₁)Σ⁻¹₁₁Σ₁₂and X₁− µ₁.)

11. Suppose X₁, X₂, and X₃ are multivariate normally distributed with means 1 µ₁ = 1, µ₂ = 0, µ₃ = −2 and covariance structure

σ²(X₁) = 3, σ²(X₂) = 4, σ²(X₃) = 6, σ(X₁, X₂) = 1, σ(X₁, X₃) = −1, σ(X₂, X₃) = 2.

a. What is the distribution of (X₁, X₂) given X₃? b. What is the regression of X₁on X₂ and X₃?

c. What is the conditional variance of X₁given X₂and X₃?

3