## Statistics

Random Variables

Shiu-Sheng Chen

Department of Economics National Taiwan University

Fall 2019

## Section 1

## Random Variables

Random Variables

In practice we are interested in certain numerical measurements pertaining to a random experiment.

For example,

Ω= {H, T}

X=⎧⎪⎪⎪

⎨⎪⎪⎪⎩

1, if H

−1, if T Then X is called arandom variable.

Random Variables: A Formal Definition

Definition

A random variable X is a real-value functionfrom the sample space to the real numbers:

X∶ Ω z→ R

and it assigns to each element ω∈ Ω one and only one real number X(ω) = x.

Small letter x denotes the possible value of a random variable X.

Example: Flip a Fair Coin Twice

The sample space is

Ω= {HH, HT, TH, TT}

Let X be the number of heads.

The mapping is

ω X(ω)

{HH} 2

{HT} 1

{TH} 1

{TT} 0

Example: Flip a Fair Coin Twice

How to assign probability P(X = x)?

For example,

P(X = 1) = P({ω ∶ X(ω) = 1})

= P({HT, TH})

= P({HT} ∪ {TH})

= P({HT}) + P({TH}) = 1 4 +

1 4 =

1 2

Random Variables

In the previous example, the set of all possible values that X can assume are finite.

According to the set of all possible values that a random variable can assume, we define two types of random variables.

(1) Discrete random variables (2) Continuous random variables

## Section 2

## Discrete Random Variables

Discrete Random Variables

A random variable X is a discrete random variable if:

there are a finite number of possible values of X, or

there are a countably infinite number of possible values of X.

Recall that a countably infinite number of possible values means that there is a one-to-one correspondence between the values and the set of positive integers.

Examples

The number of defective light bulbs in a box of six.

Set of all possible values of X= {0, 1, 2, 3, 4, 5, 6}

The number of tails until the first heads comes up.

Set of all possible values of X= {0, 1, 2, 3, . . .}

We use the probability distribution to describe the likelihood of obtaining the possible values that a random variable can assume.

Probability Distribution

Definition (Probability Distribution)

Let X be a random variable. The probability distribution of X is to specify all probabilities involving X.

One way to specify the probability distribution of discrete random variables is the probability mass function.

Probability Mass Function

Definition (Probability Mass Function)

Given a discrete random variable X. A probability mass function (pmf), f(x) ∶ R ↦ [0, 1] is defined by

f(x) = P(X = x)

A probability mass function is also called a discrete probability density function (discrete pdf).

A preferable notation: fX(x)

Probability Mass Function

Definition (Support)

The support of a random variable X is defined as:

supp(X) = {x ∶ f (x) > 0}

Properties:

x∈supp(X)∑

f(x) = 1

An Example of pmf

𝑓(𝑥)

𝑥1 𝑥2 𝑥3 𝑥4 𝑥5 𝑥6 𝑥

Example 1: Flip a Fair Coin Twice

The sample space is Ω= {HH, HT, TH, TT}

Let X be the number of heads.

Table:Mapping and Probability Distribution

ω P({ω}) X(ω)

T T 1/4 0

TH 1/4 1

HT 1/4 1

HH 1/4 2

x f(x) = P(X = x)

0 1/4

1 1/2

2 1/4

Clearly, supp(X) = {0, 1, 2}, and ∑x∈supp(X) f(x) = 1 Q: let A= {X ≤ 1}, what is P(X ∈ A)?

Example 1: Flip a Fair Coin Twice

Probability mass function (alternative notation)

f(x) = P(X = x) =⎧⎪⎪⎪⎪⎪⎪

⎨⎪⎪⎪⎪⎪⎪⎩

1/4 if x = 0 1/2 if x = 1 1/4 if x = 2

Example 1: Flip a Fair Coin Twice

݂ሺݔሻ

ݔ

Ͳ ͳ ʹ

ͳ Ͷ ͳ ʹ

Example 2: Bernoulli Random Variable

Definition (Bernoulli Random Variable)

A random variable X is said to have a Bernoulli distribution with success probability p if X can only assume the values 0 and 1, with probabilities

f(1) = P(X = 1) = p, f(0) = P(X = 0) = 1 − p We write X∼ Bernoulli(p).

Find its support, and probability mass function.

Example 3: Binomial Random Variables

Definition (Binomial Random Variable)

A random variable Y has the binomial distribution with parameters n and p if the probability mass function is

f(y) = (n

y)p^{y}(1 − p)^{n−y}, supp(Y) = {y∣y = 0, 1, 2, . . . , n}

It is denoted by Y ∼ Binomial(n, p)

By the binomial theorem, (a + b)^{n}= ∑^{n}y=0(^{n}_{y})a^{y}b^{n−y},

∑n y=0

f(y) = 1

Example 3: Binomial Random Variables

Bernoulli vs. Binomial

Bernoulli(p) is used to model a single coin toss experiment.

Binomial(n, p) is used to model the number of heads in a sequence of nindependentcoin toss experiment.

Clearly, they are linked by

Y = X^{1}+ X^{2}+ ⋯ + Xn,

where Y ∼Binomial(n, p), and X^{1},X^{2}, . . . ,Xn are independent
Bernoulli(p) variables.

Probability Distribution

Consider X∼ Bernoulli(0.5)

−0.5 0.0 0.5 1.0 1.5

0.30.40.50.60.7

x

f(x)

Probability Distribution

Consider X∼ Binomial(10, 0.5)

0 2 4 6 8 10

0.000.050.100.150.200.25

f(x)

## Section 3

## Continuous Random Variables

Continuous Random Variables

A random variable is called continuous if it can takes on an uncountably infinite number of possible values.

The percentage of exam complete after 1 hour

The weight of a randomly selected quarter-pound burger

Continuous Random Variables

Though a continuous variable can take any possible value in an interval, its measured value cannot. This is because no measuring device has infinite precision.

Nevertheless, continuous random variables offer reasonable approximations to the underlying process of interest even though virtually all phenomena are, at some level, ultimately discrete.

How to Assign Probability?

Discrete: flip a coin or roll a die.

How about spinning a spinner?

Let X be the result of the spin.

Warning! Impossible to assign each outcome positive probability.

Why?

How to Assign Probability?

Suppose NOT, and let the spinner be fair.

Each outcome has probability p> 0.

Let A⊂ Ω be an event that contains n distinct outcomes.

⇒ Choose n large enough s.t. p > _{n}^{1}.
Then P(X ∈ A) = np > 1 (Big Trouble!)

How to Assign Probability?

Hence p must be zero!

That is, if X is a continuous random variable, P(X = c) = 0

How can P(X = c) = 0 make sense? Can many nothings make something?

Think about the length of a point vs. the length of an interval.

A Zero-probability event is NOT an impossible event.

Continuous Random Variables

Definition (Continuous Random Variables)

A random variable X is continuous if there exists a function f ∶ R ↦ R and for any number a ≤ b,

P(a ≤ X ≤ b) = ∫_{a}^{b} f(x)dx

The function f(⋅) is called the probability density function(pdf).

Probability Density Function

In general, if the support for a continuous random variable is not specified, we assume that

supp(X) = {x ∶ −∞ < x < ∞}

The pdf is nonnegative

f(x) ≥ 0, ∀x The integral over the support of X is one

∫

∞

f(x)dx = 1

Probability Density Function

Since P(X = c) = 0 for any real value c, P(a ≤ X ≤ b)

= P(a < X < b)

= P(a ≤ X < b)

= P(a < X ≤ b)

Probability Mass Function vs. Probability Density Function

pmf (discrete pdf):

f(x) ∶ R ↦ [0, 1], f (x) = P(X = x) pdf:

f(x) ∶ R ↦ R^{+}, f(x) ≠ P(X = x)
That is, density is not probability.

Example 1: Uniform Random Variable

Definition (Uniform Random Variable)

A random variable X is said to have a uniform distribution on the interval [l, h] if its pdf is given by

f(x) = 1

h− l, l ≤ x ≤ h We write X∼ U[l, h].

The probability that X falls in the sub-interval [a, b] is P(a ≤ X ≤ b) = b− a

h− l.

Uniform Distribution

0.20.40.60.81.0

dunif(x, min = 0, max = 1)

## Section 4

## Cumulative Distribution Function

Cumulative Distribution Function

An alternative way to specify the probability distribution is to give the probabilities of all events of the form

{X ≤ x}, x ∈ R

For example, what is the probability that the resulting number by rolling a die is smaller than 3.8?

This leads to the following definition of cumulative distribution function.

Cumulative Distribution Function

Definition (Cumulated Distribution Function)

Given any real variable x, a function F(x) ∶ R ↦ [0, 1]:

F(x) = P(X ≤ x)

is called a cumulated distribution function (CDF), or distribution function.

A preferable notation: F_{X}(x)

It should be emphasized that the cumulative distribution function is defined as above for every random variable X, regardless of whether the distribution of X is discrete, or continuous.

Cumulative Distribution Function

If X is discrete,

F(x) = P(X ≤ x) = ∑

u≤x

P(X = u) = ∑

u≤x

f(u) If X is continuous,

F(x) = P(X ≤ x) = ∫_{−∞}^{x} f(u)du

𝑓(𝑢)

Cumulative Distribution Function

Theorem (Properties of CDF)

*Let* F*(x) be the CDF of a random variable X. Then,*

*If* a*< b, then F(a) ≤ F(b) and P(a < X ≤ b) = F(b) − F(a).*

lim_{x→−∞}F*(x) = 0, and lim*x→∞F(x) = 1
F(x) = limδ→0F(x + δ)

CDF: Discrete Random Variable
supp(X) = {x^{1},x^{2}x^{3}}

1

𝐹(𝑥)

𝑃(𝑋 = 𝑥_{1})
𝑃(𝑋 = 𝑥_{1})+𝑃(𝑋 = 𝑥2)

CDF: Continuous Random Variable

0 1

𝑥

𝐹(𝑥)

Example 1: Bernoulli(p)

Given the pmf of X∼Bernoulli(p),

f(x) =⎧⎪⎪⎪

⎨⎪⎪⎪⎩

p, x = 1 1− p, x = 0 The CDF is

F(x) = P(X ≤ x) =⎧⎪⎪⎪⎪⎪⎪

⎨⎪⎪⎪⎪⎪⎪⎩

0, x < 0 1− p, 0 ≤ x < 1 1, 1≤ x

Example 1: Bernoulli(p)

𝑥

0 1

1 𝐹(𝑥)

1 − 𝑃

Example 2: Uniform[l, h]

Given X∼ Uniform[l, h], the pdf is f(x) = 1

h− l The CDF is

F(x) = ∫_{l} ^{x} f(u)du = ∫_{l} ^{x} 1

h− ldu= x − l h− l

Example 2: Uniform[l, h]

### 𝑥 1

### 𝐹(𝑥)

### 0

### 𝑙 ℎ

## Section 5

## Quantiles

Quantiles

Definition (Quantiles)

Let F denote the CDF of a random variable X. The function
π_{p}= F^{−1}(p) = inf{x∣F(x) ≥ p}

is called the 100 p-th quantile of X. F^{−1}(⋅) is called the inverse
distribution function.

Given p= 0.5, the 50-th quantile, π^{0.5}, is called the median.

If F is strictly increasing,

π_{p}= F^{−1}(p) = {x∣F(x) = p}