3.6 Inequalities and Identities
Theorem 3.6.1 (Chebychev’s Inequality)
Let X be a random variable and let g(x) be a nonnegative function. Then, for any r > 0, P (g(X) ≥ r) ≤ Eg(X)
r .
Proof:
Eg(X) = Z ∞
−∞
g(x)fX(x)dx
≥ Z
{x:g(x)≥r}
g(x)fX(x)dx (g is nonnegative)
≥ r Z
{x:g(x)≥r}
fX(x)dx
= rP (g(X) ≥ r).
Rearranging now produces the desired inequality. ¤
Example 3.6.2 (Illustrating Chebychev)
let g(x) = (x − µ)2/σ2, where µ = EX and σ2 = VarX. For convenience write r = t2. Then P ((X − µ)2
σ2 ≥ t2) ≤ 1
t2E(X − µ)2 σ2 = 1
t2. Thus,
P (|X − µ| ≥ tσ) ≤ 1 t2. For example, taking t = 2, we get
P (|X − µ| ≥ 2σ) ≤ 1
22 = 0.25.
Example 3.6.3 (A normal probability inequality) If Z is standard normal, then
P (|Z| ≥ t) ≤ r2
π e−t2/2
t , for all t > 0.
1
Write
P (Z ≥ t) = 1
√2π Z ∞
t
e−x2/2dx
≤ 1
√2π Z ∞
t
x
te−x2/2dx (since x/t > 1)
= 1
√2π e−t2/2
t and use the fact that P (|Z| ≥ t) = 2P (Z ≥ t).
Theorem 3.6.4
Let Xα,β denote a gamma(α, β) random variable with pdf f (x|α, β), where α > 1. Then for any constants a and b,
P (a < Xα,β < b) = βf (a|α, β) − f (b|α, β) + P (a < Xα−1,β < b).
Lemma 3.6.5(Stein’s Lemma)
Let X ∼ N(θ, σ2), and let g be a differentiable function satisfying E|g0(X)| < ∞. Then E[g(X)(X − θ)] = σ2Eg0(X).
Proof: The left-hand side is
E[g(X)(X − θ)] = 1
√2πσ Z ∞
−∞
g(x)(x − θ)e−(x−θ)2/(2σ2)dx.
Using integration by parts with u = g(x) and dv = (x − θ)e−(x−θ)2/(2σ2)dx to get E[g(X)(X − θ)] = 1
√2πσ
£− σ2g(x)e−(x−θ)2/(2σ2)|∞−∞+ σ2 Z ∞
−∞
g0(x)e−(x−θ)2/(2σ2)dx¤ .
The condition on g0 is enough to ensure that the first term is 0 and what remains on the right-hand side is σ2Eg0(X). ¤
Example 3.6.6 (Higher-order normal moments)
Stein’s lemma makes calculation of higher-order moments quite easy/ For example, if X ∼
2
N(θ, σ2), then
EX3 = EX2(X − θ + θ) = EX2(X − θ) + θEX2
= 2σ2EX + θEX2 = 2σ2θ + θ(σ2+ θ2)
= 3θσ2+ θ3.
Theorem 3.6.7
Let χ2p denote a chi-squared random variable with p degrees of freedom. For any function h(x),
Eh(χ2p) = pE¡h(χ2p+2) χ2p+2
¢
provided the expectations exist.
Some moment calculations are very easy with Theorem ??. For example, the mean of a χ2p is
Eχ2p = pE¡χ2p χ2p
¢= pE(1) = p,
and the second moment is
E(χ2p)2 = pE¡(χ2p)2 χ2p
¢= pE(χ2p) = p(p + 2).
So Var¡ χ2p¢
= p(p + 2) − p2 = 2p.
Theorem 3.6.8 (Hwang)
Let g(x) be a function with −∞ < Eg(X) < ∞ and −∞ < g(−1) < ∞. Then:
a. If X ∼ P oisson(λ),
E(λg(X)) = E(Xg(X − 1)).
b. If X ∼ negative binomial(r, p),
E((1 − p)g(X)) = E¡ X
r + X − 1g(X − 1)¢ .
Example 3.6.9 (Higher-order Poisson moments)
For X ∼ Poisson(λ), take g(x) = x2 and use Theorem 3.6.8:
E(λX2) = E(X(X − 1)2) = E(X3− 2X2+ X).
3
Therefore, the third moment of a Poisson(λ) is EX3 = λEX2 = 2EX2− EX
= λ(λ + λ2) + 2(λ + λ2) − λ = λ3+ 3λ2+ λ.
For the negative binomial, the mean can be calculated by taking g(x) = r + x, E((1 − p)(r + X)) = E¡ X
r + X − 1(r + X − 1)¢
= EX, so, rearranging, we get
EX = r(1 − p)
p .
4
