### Statistics

The Normal Distribution and its Applications

Shiu-Sheng Chen

Department of Economics National Taiwan University

Fall 2019

### Section 1

### Normal Distributions

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 2 / 47

Normal Random Variables

Normal distribution is also called Gaussian distribution, which is named after German mathematician Johann Carl Friedrich Gauss (1777–1855)

Normal Random Variables

Definition (Normal Distribution)

A random variable X has the normal distribution with two parameters
µ and σ^{2} if X has a continuous distribution with the following pdf:

f (x) = 1 σ√

2π e^{−}^{2}^{1}^{(}^{x−µ}^{σ} ^{)}^{2}

where supp(X) = {x∣ − ∞ < x < ∞}, and π ≑ 3.14159. It is denoted by
X ∼ N(µ, σ^{2})

Via converting from Cartesian to polar coordinates (google

“Gaussian integral”)

∫

∞

−∞

1 σ√

2π e^{−}^{2}^{1}^{(}^{x−µ}^{σ} ^{)}^{2} = 1

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 4 / 47

Standard Normal Random Variables

Definition (Standard Normal Distribution)

A random variable Z is called a standard normal random variable, if µ = 0 and σ = 1 with pdf

1

√2πe^{−}^{2}^{1}^{z}^{2}
It is denoted by Z ∼ N(0, 1)

As a conventional notation, we let ϕ and Φ denote pdf and CDF of a standard normal random variable,

ϕ(z) = 1

√ e^{−}^{1}^{2}^{z}^{2}, Φ(z) = ^{z} ϕ(w)dw.

Normal Distributions

Solid line: N(0, 1) vs. Dashed line: N(0, 9)
Skewness γ^{3}= 0; Kurtosis γ^{4}= 3

−10 −5 0 5 10

0.00.10.20.30.4

x

f(x)

N(0,1)

N(0,9)

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 6 / 47

Stock Returns (S&P500)

**Histogram of Stock Returns**

Density 0.020.040.060.080.100.12

Stock Returns (S&P500)

**Histogram of Stock Returns**

r

Density

−20 −10 0 10

0.000.020.040.060.080.100.12

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 8 / 47

Exchange Rate Changes (British Pound)

**Histogram of Exchange Rate Returns**

Density 0.050.100.15

Exchange Rate Changes (British Pound)

**Histogram of Exchange Rate Returns**

rs

Density

−10 −5 0 5 10

0.000.050.100.15

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 10 / 47

Normal vs. Standard Normal

Theorem

*Let* *Z ∼ N(0, 1) and X = σ Z + µ, then*
X ∼ N(µ, σ^{2})
Proof: by CDF method.

In the same vein, you can show that if X ∼ N(µ, σ^{2}) and Z = ^{X−µ}_{σ} ,
then Z ∼ N(0, 1).

Moment Generating Function

Theorem (MGF)

*Let* *Z ∼ N(0, 1), the MGF of Z is*

M_{Z}(t) = e^{2}^{1}^{t}^{2}
Proof: by definition.

It follows that if X ∼ N(µ, σ^{2}), the MGF of X is
MX(t) = e^{µt+}^{1}^{2}^{σ}^{2}^{t}^{2}
Hence, E(X) = µ, Var(X) = σ^{2}

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 12 / 47

Properties

Theorem (Invariance Under Linear Transformations)
*If* X ∼ N(µ, σ^{2}), then

aX + b ∼ N(aµ + b, a^{2}σ^{2}), a ≠ 0.

Proof: by MGF

Example: a portfolio consisting of a stock and a risk-free asset

Properties

Theorem (Sum of I.I.D. Normal Random Variables)
*If {X*i}^{n}_{i=1}∼^{i.i.d.} N(µ, σ^{2}), and

W = α^{1}X^{1}+α^{2}X^{2}+ ⋯ +αnXn,
*then*

W ∼ N (µ

n

∑

i=1

αi,σ^{2}

n

∑

i=1

α^{2}_{i}) .

Proof: by MGF

Consider two special cases:

αi = 1for all i
αi = _{n}^{1} for all i

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 14 / 47

Properties

Theorem

*If {X*_{i}}^{n}_{i=1}∼^{i.i.d.} N(µ, σ^{2})*, then*
Y =

n

∑

i=1

Xi ∼N(nµ, nσ^{2}).

*If {X*i}^{n}_{i=1}∼^{i.i.d.} N(µ, σ^{2}), then
X =¯ ∑^{n}_{i=1}X_{i}

n ∼N (µ,σ^{2}
n) .

Finding Normal Probabilities

However, sometimes what we have is the N(0, 1) table.

When calculating the probability of a normal distribution, we need to transform it to the standard normal distribution.

Then calculate Φ(a) = P(Z ≤ a) For instance, if X ∼ N(5, 16),

P(X ≤ 3) = P (X − 5

4 ≤

3 − 5

4 ) =P(Z ≤ −0.5) = Φ(−0.5) The following properties will be helpful

P(Z ≤ 0) = P(Z ≥ 0) = 0.5 P(Z ≤ −a) = P(Z ≥ a)

P(−a ≤ Z ≤ 0) = P(0 ≤ Z ≤ a)

TA will teach you how to use the N(0, 1) table

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 16 / 47

Z Table

Using Normal Probabilities to Find Quantiles

Example: Suppose the final grade, X is normally distributed with mean 70 and standard deviation 10. The instructor wants to give 10% of the class an A+. What cutoff should the instructor use to determine who gets an A+?

Clearly, X ∼ N(70, 100), and we want to find the constant q such that P(X > q) = 0.10 or P(X ≤ q) = 0.90.

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 18 / 47

Using Z Table

Find q such that

0.90 = P(X ≤ q) = P (X − 70

10 ≤ q − 70

10 ) =P (Z ≤ q − 70 10 ) According to the Z Table, P(Z ≤ 1.28) = 0.90, we have

q − 70 10 = 1.28 That is,

c = 82.8

Example: Mean-Variance Utility

Suppose that the utility function from wealth W is given by
U(W) = c − e^{−bW}, b > 0

This utility function is increasing and concave

U^{′}(W) = be^{−bW} > 0, U^{′′}(W) = −b^{2}e^{−bW}< 0
We further assume that W ∼ N(µ, σ^{2}), then

E[U(W)] = c − E [e^{−}^{bW}] =c − e^{−}^{b(µ−}^{b}^{2}^{σ}^{2}^{)}=g(µ, σ^{2})
Hence,

∂E[U(W)]

∂µ =g_{µ}> 0, ∂E[U(W)]

∂(σ^{2})

=g_{σ}^{2} < 0

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 20 / 47

Bivariate Normal Random Variables

Definition (Bivariate Normal Distribution)

X and Y are bivariate normal distributed if the joint pdf is
fXY(x, y) = e^{ϖ}

2π√

(1 −ρ^{2})σXσY

where

ϖ = − 1
2(1 −ρ^{2})

[(x − µX

σX

)

2

− 2ρ (x − µX

σX

) (y − µY

σY

) + (y − µY

σY

)

2

]

Bivariate Normal Random Variables It is denoted by

⎡⎢

⎢

⎢⎢

⎣ X Y

⎤⎥

⎥

⎥⎥

⎦

∼N⎛

⎝

⎡⎢

⎢

⎢⎢

⎣ µX

µY

⎤⎥

⎥

⎥⎥

⎦ ,

⎡⎢

⎢

⎢⎢

⎣

σ_{X}^{2} σXY

σXY σ_{Y}^{2}

⎤⎥

⎥

⎥⎥

⎦

⎞

⎠

x

y z

**Bivariate Normal Density**

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 22 / 47

Independent vs. Uncorrelated

Theorem

*Given that* *X and Y have a bivariate normal distribution. X and Y*
*are independent if and only if*

Cov(X, Y) = 0.

Proof: we only need to show the “if” part. Clearly, when Cov(X, Y) = 0, which implies ρ = 0, it can be shown that

f_{XY}(x, y) = f_{X}(x) f_{Y}(y)

Symmetry vs. Asymmetry

Normal distribution is a symmetricdistribution.

In some instances, we require a skewed distribution to characterize the data.

For example,

the size of insurance claims time-until-default (survival time)

We thus introduce a random variable called Chi-square random variable to capture the asymmetrical characteristics.

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 24 / 47

### Section 3

### Chi-square Distribution

Chi-square Distribution

Definition (Chi-square Random Variables)

A random variable X has the Chi-square distribution if the pdf is
f (x) = x^{k}^{2}^{−1}

2^{k}^{2}Γ(^{k}_{2})

e^{−}^{2}^{1}^{x}, supp(X) = {x∣0 < x < ∞}

where k is a positive integer called the degree of freedom.

Γ(⋅) is called a Gamma function:

Γ(α) =∫

∞

0 x^{α−1}e^{−x}dx

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 26 / 47

Chi-square distributions (k = 3)

0 5 10 15 20

0.000.050.100.150.200.25

dchisq(x, df = 3)

Chi-square Distribution

Theorem

*The MGF of a Chi-square random variable is*
M_{X}(t) = ( 1

1 − 2t)

k 2

Proof: by the definition of MGF, and let y = (1 − 2t)x.

Hence,

E(X) = k, Var(X) = 2k

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 28 / 47

Chi-square Distribution

Theorem

*If* Xi ∼χ^{2}(ki) *for* *i = 1, 2, . . . n, and they are independent, then*

n

∑

i=1

Xi ∼χ^{2}(

n

∑

i=1

ki)

That is, the sum X^{1}+X^{2}+ ⋯ +Xn has the χ^{2} distribution with
k^{1}+k^{2}+ ⋯ +kn degrees of freedom.

Proof: by MGF.

Chi-square Distribution

The following theorem links the normal distribution and Chi-square distribution.

Theorem

*Let* *Z ∼ N(0, 1). Then the random variable*
Y = Z^{2}∼ χ^{2}(1)
Proof:

M_{Z}^{2}(t) = E(e^{tZ}^{2}) = ∫

∞

−∞

e^{tz}^{2} 1

√2πe^{−}^{2}^{1}^{z}^{2}dz

= ∫

∞

−∞

1

√2πe^{−}^{1}^{2}^{(1−2t)z}^{2}dz = ( 1
1 − 2t)

1 2

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 30 / 47

Chi-square Distributions

Corollary

*Suppose {Z*^{1},Z^{2}. . . ,Zk} ∼^{i.i.d.} *N(0, 1). Let X = ∑*^{k}_{i=1}Z^{2}_{i}*. Then*
X ∼ χ^{2}(k)

Proof: by the above two theorems.

Degree of freedom: The number of values in the final calculation of a statistic that are free to vary

### Section 4

### Student’s t Distribution

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 32 / 47

Student’s t Distributions

Recall that the Kurtosis for Normal random variables is 3 Monthly S&P 500 Stock Returns (1957:1–2013:9):

Kurtosis =5.51

Daily S&P 500 Stock Returns (1957/1/2–2013/9/30):

Kurtosis =30.75

Fat-tailed/Heavy-tailed

Student’s t distributions

The Student’s t distribution was actually published in 1908 by a British statistician, William Sealy Gosset (1876–1937).

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 34 / 47

William Sealy Gosset

Gosset, was employed at the Guinness Brewing Co., which forbade its staffs publishing scientific papers due to an earlier paper containing trade secrets.

To circumvent this restriction, Gosset used the name “Student”, and consequently the distribution was named Student’s t

distribution.

Guinness and the 1908 Biometrika Paper

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 36 / 47

Student’s t distributions

Definition (Student’s t distribution)

If a random variable X has the following pdf
Γ(^{k+1}_{2} )

Γ(^{k}_{2})
1

√kπ(1 + x^{2}
k )

−^{k+1}_{2}

with support supp(X) = {x∣ − ∞ < x < ∞} and a parameter k, then it is called a Student’s t distribution, and denoted by

X ∼ t(k)

Student’s t distributions

Theorem

*Given two independent random variables:* *Z ∼ N(0, 1) and*
W ∼ χ^{2}(*k). Then*

U = Z

√

W k

∼t(k)

Proof: beyond the scope of this course.

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 38 / 47

Student’s t distribution

Given U ∼ t(k) Moments

E(U) = 0 when k > 1,
Var(U) = E(U^{2}) = k

k − 2 when k > 2.

Note that given W ∼ χ^{2}(k),

E ( 1 W) =

1 k − 2

Student’s t distributions

t distribution is a symmetric distribution.

Limiting distribution

t(k) Ð→ N(0, 1) as k Ð→ ∞.

Special case: k = 1, E[t(1)] = ∞ − ∞ (undefined) t(1) is called astandard Cauchy distribution.

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 40 / 47

Comparison: t(2) vs. N(0,1)

Clearly, the Student’s t distribution has a fat tail.

0.00.10.20.30.4

dnorm(x)

N(0,1)

t(2)

### Section 5 F Distributions

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 42 / 47

F Distribution

Definition (F Distribution)

Random X has the F distribution with n^{1} and n^{2} degrees of freedom
if the probability density function is

Γ(^{n}^{1}^{+}_{2}^{n}^{2})
Γ(^{n}_{2}^{1})Γ(^{n}_{2}^{2})

(n^{1}
n^{2})

n1 2

x^{n1}^{2}^{−1}(1 + n^{1}
n^{2}x)

−^{n1+n2}_{2}

with supp(X) = {x∣0 ≤ x < ∞}. It is denoted by
X ∼ F(n^{1},n^{2})

R.A. Fisher and G. W. Snedecor

Sir Ronald A. Fisher (1890–1962), British statistician, evolutionary biologist, eugenicist, and geneticist

George W. Snedecor (1881–1974), American mathematician and statistician

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 44 / 47

F Distribution (n^{1}= 10, n^{2} = 10)

0.00.20.40.6

df(x, df1 = 10, df2 = 10)

F Distribution

Theorem

*Let* W^{1} *and* W^{2} *be independent Chi-square random variables:*

W^{1}∼ χ^{2}(n^{1}), *and* W^{2}∼ χ^{2}(n^{2}),
*then*

X = W^{1}/n^{1}

W^{2}/n^{2} ∼F(n1,n2)
Proof: beyond the scope of this course.

Shiu-Sheng Chen (NTU Econ) Statistics Fall 2019 46 / 47

F Distribution

Theorem
*If* *t ∼ t(k), then*

t^{2} ∼F(1, k)