Statistics
Random Variables
Shiu-Sheng Chen
Department of Economics National Taiwan University
Fall 2019
Section 1
Random Variables
Random Variables
In practice we are interested in certain numerical measurements pertaining to a random experiment.
For example,
Ω= {H, T}
X=⎧⎪⎪⎪
⎨⎪⎪⎪⎩
1, if H
−1, if T Then X is called arandom variable.
Random Variables: A Formal Definition
Definition
A random variable X is a real-value functionfrom the sample space to the real numbers:
X∶ Ω z→ R
and it assigns to each element ω∈ Ω one and only one real number X(ω) = x.
Small letter x denotes the possible value of a random variable X.
Example: Flip a Fair Coin Twice
The sample space is
Ω= {HH, HT, TH, TT}
Let X be the number of heads.
The mapping is
ω X(ω)
{HH} 2
{HT} 1
{TH} 1
{TT} 0
Example: Flip a Fair Coin Twice
How to assign probability P(X = x)?
For example,
P(X = 1) = P({ω ∶ X(ω) = 1})
= P({HT, TH})
= P({HT} ∪ {TH})
= P({HT}) + P({TH}) = 1 4 +
1 4 =
1 2
Random Variables
In the previous example, the set of all possible values that X can assume are finite.
According to the set of all possible values that a random variable can assume, we define two types of random variables.
(1) Discrete random variables (2) Continuous random variables
Section 2
Discrete Random Variables
Discrete Random Variables
A random variable X is a discrete random variable if:
there are a finite number of possible values of X, or
there are a countably infinite number of possible values of X.
Recall that a countably infinite number of possible values means that there is a one-to-one correspondence between the values and the set of positive integers.
Examples
The number of defective light bulbs in a box of six.
Set of all possible values of X= {0, 1, 2, 3, 4, 5, 6}
The number of tails until the first heads comes up.
Set of all possible values of X= {0, 1, 2, 3, . . .}
We use the probability distribution to describe the likelihood of obtaining the possible values that a random variable can assume.
Probability Distribution
Definition (Probability Distribution)
Let X be a random variable. The probability distribution of X is to specify all probabilities involving X.
One way to specify the probability distribution of discrete random variables is the probability mass function.
Probability Mass Function
Definition (Probability Mass Function)
Given a discrete random variable X. A probability mass function (pmf), f(x) ∶ R ↦ [0, 1] is defined by
f(x) = P(X = x)
A probability mass function is also called a discrete probability density function (discrete pdf).
A preferable notation: fX(x)
Probability Mass Function
Definition (Support)
The support of a random variable X is defined as:
supp(X) = {x ∶ f (x) > 0}
Properties:
x∈supp(X)∑
f(x) = 1
An Example of pmf
𝑓(𝑥)
𝑥1 𝑥2 𝑥3 𝑥4 𝑥5 𝑥6 𝑥
Example 1: Flip a Fair Coin Twice
The sample space is Ω= {HH, HT, TH, TT}
Let X be the number of heads.
Table:Mapping and Probability Distribution
ω P({ω}) X(ω)
T T 1/4 0
TH 1/4 1
HT 1/4 1
HH 1/4 2
x f(x) = P(X = x)
0 1/4
1 1/2
2 1/4
Clearly, supp(X) = {0, 1, 2}, and ∑x∈supp(X) f(x) = 1 Q: let A= {X ≤ 1}, what is P(X ∈ A)?
Example 1: Flip a Fair Coin Twice
Probability mass function (alternative notation)
f(x) = P(X = x) =⎧⎪⎪⎪⎪⎪⎪
⎨⎪⎪⎪⎪⎪⎪⎩
1/4 if x = 0 1/2 if x = 1 1/4 if x = 2
Example 1: Flip a Fair Coin Twice
݂ሺݔሻ
ݔ
Ͳ ͳ ʹ
ͳ Ͷ ͳ ʹ
Example 2: Bernoulli Random Variable
Definition (Bernoulli Random Variable)
A random variable X is said to have a Bernoulli distribution with success probability p if X can only assume the values 0 and 1, with probabilities
f(1) = P(X = 1) = p, f(0) = P(X = 0) = 1 − p We write X∼ Bernoulli(p).
Find its support, and probability mass function.
Example 3: Binomial Random Variables
Definition (Binomial Random Variable)
A random variable Y has the binomial distribution with parameters n and p if the probability mass function is
f(y) = (n
y)py(1 − p)n−y, supp(Y) = {y∣y = 0, 1, 2, . . . , n}
It is denoted by Y ∼ Binomial(n, p)
By the binomial theorem, (a + b)n= ∑ny=0(ny)aybn−y,
∑n y=0
f(y) = 1
Example 3: Binomial Random Variables
Bernoulli vs. Binomial
Bernoulli(p) is used to model a single coin toss experiment.
Binomial(n, p) is used to model the number of heads in a sequence of nindependentcoin toss experiment.
Clearly, they are linked by
Y = X1+ X2+ ⋯ + Xn,
where Y ∼Binomial(n, p), and X1,X2, . . . ,Xn are independent Bernoulli(p) variables.
Probability Distribution
Consider X∼ Bernoulli(0.5)
−0.5 0.0 0.5 1.0 1.5
0.30.40.50.60.7
x
f(x)
Probability Distribution
Consider X∼ Binomial(10, 0.5)
0 2 4 6 8 10
0.000.050.100.150.200.25
f(x)
Section 3
Continuous Random Variables
Continuous Random Variables
A random variable is called continuous if it can takes on an uncountably infinite number of possible values.
The percentage of exam complete after 1 hour
The weight of a randomly selected quarter-pound burger
Continuous Random Variables
Though a continuous variable can take any possible value in an interval, its measured value cannot. This is because no measuring device has infinite precision.
Nevertheless, continuous random variables offer reasonable approximations to the underlying process of interest even though virtually all phenomena are, at some level, ultimately discrete.
How to Assign Probability?
Discrete: flip a coin or roll a die.
How about spinning a spinner?
Let X be the result of the spin.
Warning! Impossible to assign each outcome positive probability.
Why?
How to Assign Probability?
Suppose NOT, and let the spinner be fair.
Each outcome has probability p> 0.
Let A⊂ Ω be an event that contains n distinct outcomes.
⇒ Choose n large enough s.t. p > n1. Then P(X ∈ A) = np > 1 (Big Trouble!)
How to Assign Probability?
Hence p must be zero!
That is, if X is a continuous random variable, P(X = c) = 0
How can P(X = c) = 0 make sense? Can many nothings make something?
Think about the length of a point vs. the length of an interval.
A Zero-probability event is NOT an impossible event.
Continuous Random Variables
Definition (Continuous Random Variables)
A random variable X is continuous if there exists a function f ∶ R ↦ R and for any number a ≤ b,
P(a ≤ X ≤ b) = ∫ab f(x)dx
The function f(⋅) is called the probability density function(pdf).
Probability Density Function
In general, if the support for a continuous random variable is not specified, we assume that
supp(X) = {x ∶ −∞ < x < ∞}
The pdf is nonnegative
f(x) ≥ 0, ∀x The integral over the support of X is one
∫
∞
f(x)dx = 1
Probability Density Function
Since P(X = c) = 0 for any real value c, P(a ≤ X ≤ b)
= P(a < X < b)
= P(a ≤ X < b)
= P(a < X ≤ b)
Probability Mass Function vs. Probability Density Function
pmf (discrete pdf):
f(x) ∶ R ↦ [0, 1], f (x) = P(X = x) pdf:
f(x) ∶ R ↦ R+, f(x) ≠ P(X = x) That is, density is not probability.
Example 1: Uniform Random Variable
Definition (Uniform Random Variable)
A random variable X is said to have a uniform distribution on the interval [l, h] if its pdf is given by
f(x) = 1
h− l, l ≤ x ≤ h We write X∼ U[l, h].
The probability that X falls in the sub-interval [a, b] is P(a ≤ X ≤ b) = b− a
h− l.
Uniform Distribution
0.20.40.60.81.0
dunif(x, min = 0, max = 1)
Section 4
Cumulative Distribution Function
Cumulative Distribution Function
An alternative way to specify the probability distribution is to give the probabilities of all events of the form
{X ≤ x}, x ∈ R
For example, what is the probability that the resulting number by rolling a die is smaller than 3.8?
This leads to the following definition of cumulative distribution function.
Cumulative Distribution Function
Definition (Cumulated Distribution Function)
Given any real variable x, a function F(x) ∶ R ↦ [0, 1]:
F(x) = P(X ≤ x)
is called a cumulated distribution function (CDF), or distribution function.
A preferable notation: FX(x)
It should be emphasized that the cumulative distribution function is defined as above for every random variable X, regardless of whether the distribution of X is discrete, or continuous.
Cumulative Distribution Function
If X is discrete,
F(x) = P(X ≤ x) = ∑
u≤x
P(X = u) = ∑
u≤x
f(u) If X is continuous,
F(x) = P(X ≤ x) = ∫−∞x f(u)du
𝑓(𝑢)
Cumulative Distribution Function
Theorem (Properties of CDF)
Let F(x) be the CDF of a random variable X. Then,
If a< b, then F(a) ≤ F(b) and P(a < X ≤ b) = F(b) − F(a).
limx→−∞F(x) = 0, and limx→∞F(x) = 1 F(x) = limδ→0F(x + δ)
CDF: Discrete Random Variable supp(X) = {x1,x2x3}
1
𝐹(𝑥)
𝑃(𝑋 = 𝑥1) 𝑃(𝑋 = 𝑥1)+𝑃(𝑋 = 𝑥2)
CDF: Continuous Random Variable
0 1
𝑥
𝐹(𝑥)
Example 1: Bernoulli(p)
Given the pmf of X∼Bernoulli(p),
f(x) =⎧⎪⎪⎪
⎨⎪⎪⎪⎩
p, x = 1 1− p, x = 0 The CDF is
F(x) = P(X ≤ x) =⎧⎪⎪⎪⎪⎪⎪
⎨⎪⎪⎪⎪⎪⎪⎩
0, x < 0 1− p, 0 ≤ x < 1 1, 1≤ x
Example 1: Bernoulli(p)
𝑥
0 1
1 𝐹(𝑥)
1 − 𝑃
Example 2: Uniform[l, h]
Given X∼ Uniform[l, h], the pdf is f(x) = 1
h− l The CDF is
F(x) = ∫l x f(u)du = ∫l x 1
h− ldu= x − l h− l
Example 2: Uniform[l, h]
𝑥 1
𝐹(𝑥)
0
𝑙 ℎ
Section 5
Quantiles
Quantiles
Definition (Quantiles)
Let F denote the CDF of a random variable X. The function πp= F−1(p) = inf{x∣F(x) ≥ p}
is called the 100 p-th quantile of X. F−1(⋅) is called the inverse distribution function.
Given p= 0.5, the 50-th quantile, π0.5, is called the median.
If F is strictly increasing,
πp= F−1(p) = {x∣F(x) = p}
