5.1 Basic Concepts of Random Samples
Definition 5.1.1
The random variables X1, . . . , Xn are called a random sample of size n from the population f (x) if X1, . . . , Xn are mutually independent random variables and the marginal pdf or pmf of each Xi is the same function f (x). Alternatively, X1, . . . , Xn are called independent and identically distributed (iid) random variables with pdf or pmf f (x). This is commonly abbreviated to iid random variables.
If the population pdf or pmf is a member of a parametric family with pdf or pmf given by f (x|θ), then the joint pdf or pmf is
f (x1, . . . , xn|θ) = Yn
i=1
f (xi|θ),
where the same parameter value θ is used in each of the terms in the product.
Example Let X1, . . . , Xn be a random sample from an exponential(β) population. Specif- ically, X1, . . . , Xn might correspond to the times (measured in years) until failure for n identical circuit boards that are put on test and used until they fail. The joint pdf of the sample is
f (x1, . . . , Xn|β) = Yn
i=1
f (xi|β) = 1
βne−Pni=1xi/β.
This pdf can be used to answer questions about the sample. For example, what is the probability that all the boards last more than 2 years?
P (X1 > 2, . . . , Xn> 2) = P (X1 > 2) · · · P (Xn> 2)
= [P (X1 > 2)]n = (e−2/β)n = e−2n/β.
Random sampling models
(a) Sampling from an infinite population. The samples are iid.
(b) Sampling with replacement from a finite population. The samples are iid.
1
(c) Sampling without replacement from a finite population. This sampling is sometimes called simple random sampling. The samples are not iid exactly. However, if the popu- lation size N is large compared to the sample size n, the samples will be approximately iid.
Example 5.1.3 (Finite population model)
Suppose {1, . . . , 1000} is the finite population, so N = 1000. A sample of size n = 10 is drawn without replacement. What is the probability that all ten sample values are greater than 200? If X1, . . . , X10 were mutually independent we would have
P (X1 > 200, . . . , X10> 200) = ( 800
1000)10 = .107374.
Without the independent assumption, we can calculate as follows.
P (X1 > 200, . . . , X10> 200) =
¡800
10
¢¡200
0
¢
¡1000
10
¢ = .106164.
Thus, the independence assumption is approximately correct.
2
