Large Sample Theory Homework 3: Probability and Conditioning Due Date: November 10th 1. Let

Large Sample Theory

Homework 3: Probability and Conditioning Due Date: November 10th

1. Let X be a random variable with range {0, 1, 2, . . .}. Show that if E(X) < ∞, then

E(X) =

∞

X

n=1

P (X ≥ n).

2. Let X be a random variable having a c.d.f. F (x). Show that if X ≥ 0, then E(X) =

Z

[1 − F_X(x)]dx;

in general, if E(X) exists, then E(X) =

Z ∞ 0

[1 − F_X(x)]dx −

Z 0

−∞[F_X(x)]dx.

3. Let X₁ and X₂ be independent random variables having the standard normal distribu- tion. Obtain the joint p.d.f. of (Y₁, Y₂), where Y₁ =^qX₁²+ X₂² and Y₂ = X₁/X₂. (a). Are the Y_iindependent?

(b). Box-Muller transformation is often used to transform a two-dimensional contin- uous uniform distribution to a two-dimensional bivariate normal distribution. In this algorithm, it generates X₁ and X₂ which are independent and uniformly distributed 0 and 1. Then convert X1 and X2to Z1and Z2by

Z₁ =^q−2π ln X₁cos(2πX₂), Z₂ =^q−2π ln X₁sin(2πX₂).

Use (a) to show that Z₁ and Z₂ are independent and normally distributed with mean 0 and 1.

4. A median of a random variable Y (or its distribution) is any value m such that P (Y ≥ m) ≥ 1/2, P (Y ≤ m) ≥ 1/2.

(a) Show that the set of medians is a closed interval [m₀, m₁].

(b) Let R(c) = E(|Y − c|). Show that either R(c) = ∞ for all c or R(c) ≥ R(m) for any median m of Y .

(c) Give a condition in terms of the density function of Y at m to ensure that R(c) is continuous at m.

5. Let (X1, . . . , Xn) be a sample from a Poisson P(λ) distribution and let Sm =^Pm_i=1Xi, m ≤ n.

(a) Show that the conditional distribution of X given S_n= k is multinomial M(k, 1/n, . . . , 1/n).

(b) Show that E(Sm|Sn) = (m/n)Sn.

6. Suppose that X has a normal N (µ, σ²) distribution and that Y = X + Z, where Z is independent of X and has a normal N (ν, τ²) distribution.

(a) What is the conditional distribution of Y given X = x?

(b) Using Bayes rule find the conditional distribution of X given Y = y.

7. Suppose that Y has the Poisson distribution P (θ) and P_{X|Y =y} has the binomial distri- bution Bin(y, p). Show that the marginal distribution of X is the Poisson distribution P (pθ).

1

8. Let X1, . . . , Xnbe a sample from the exponential distribution with density e^−u (u > 0).

Find the distribution of^Pn_i=1X_i, and the conditional distribution of X₁, given^Pn_i=1X_i. 9. For any set of numbers x1, . . . , x_nand a monotone function h(·), show that the value of a

that minimizes^Pn_i=1[h(x_i) − h(a)]²is given by a = h⁻¹(^Pn_i=1h(x_i)/n). Find functions h that will yield the arithmetic, geometric, and harmonic means as minimizes.

Recall that the geometric mean of non-negative numbers is (^Qx_i)^1/n and the harmonic mean is [n^−1P(1/x_i)]⁻¹.

10. Let X₁, . . . , X_n be i.i.d. from P with unknown P with unknown mean µ ∈ R and variance σ² > 0, and let g(µ) = 0 if µ 6= 0 and g(0) = 1. Find a consistent estimator of g(µ).

11. Let X₁, . . . , X_nbe i.i.d. N (θ, 1) with θ ≥ 0.

(a) Show that the MLE of θ, ˆθ_n, is ¯X if ¯X > 0 and 0 otherwise.

(b) If θ > 0, show that√

n(ˆθ_n− θ)→ N (0, 1).^L

(c) If θ = 0, the probability is 1/2 that ˆθn = 0 and 1/2 that√

n(ˆθn− θ)→ N (0, 1).^L 12. Suppose that X1, . . . , Xn be i.i.d. random variables from F and that F is unknown but

has a Lebesgue p.d.f. f . A simple estimator of f (t), t ∈ R, is defined as the difference quotient

f_n(t) = F_n(t + λ_n) − F_n(t − λ_n)

2λ_n .

Here Fnis the empirical cdf.

(a) Is f_na density function?

(b) Suppose that f is continuously differentiable at t, λ_n → 0, and nλ_n → ∞. Show that

E[f_n(t) − f (t)]² = f (t) 2nλ_n + o

 1 nλ_n



+ O(λ²_n).

(c) Under nλ³_n → 0 and the conditions of (b), show that

q

nλn[fn(t) − f (t)]→ N^d



0,1 2f (t)



.

(d) Suppose that f is continuous on [a, b], −∞ < a < b < ∞, λ_n → 0, and nλ_n → ∞.

Show that^R_a^bf_n(t)dt →^{P R}_a^bf (t)dt.

13. If a_n(Y_n− c) → H and a^L _n→ ∞, then Y_n → c where H is a continuous distribution.^L 14. Let X₁, . . . , X_n be i.i.d. with E(X_i) = θ, V ar(X_i) = σ² < ∞, and let δ_n = ¯X with

probability 1 − _nand δ_n= A_nwith probability _n. If _nand A_nare constants satisfying

_n→ 0 and _nA_n→ ∞,

then δ_nis consistent for estimating θ, but E(δ_n− θ)² does not tend to zero.

15. Suppose X₁, . . . , X_n have common mean θ and variance σ², and that cov(X_i, X_j) = ρ_i−j. For estimating θ, show that:

(a) ¯X_n is not consistent if ρi−j = ρ 6= 0 for all i 6= j; (For this problem, you can only consider the case that (X₁, . . . , X_n) are multivariate normal.)

(b) ¯X_nis consistent if |ρ_i−j| ≤ M γ^j−i with |γ| < 1.

2

16. Suppose that Xnis a random variable having the binomial distribution Bin(n, p), where 0 < p < 1, n = 1, 2, . . .. Define

Y_n =

( log(X_n/n) X_n ≥ 1

1 X_n = 0.

Show that Y_n ^a.s.→ log p and√

n(Y_n− log p)→ N (0, (1 − p)/p).^d

3