Third Edition
5.2 Indicator random variables
In order to analyze many algorithms, including the hiring problem, we use indicator random variables. Indicator random variables provide a convenient method for converting between probabilities and expectations. Suppose we are given a sample space S and an event A. Then the indicator random variable IfAg associated with event A is defined as
IfAg D
(1 if A occurs ;
0 if A does not occur : (5.1)
As a simple example, let us determine the expected number of heads that we obtain when flipping a fair coin. Our sample space is S DfH; T g, with Pr fH g D PrfT g D 1=2. We can then define an indicator random variable XH, associated with the coin coming up heads, which is the event H . This variable counts the number of heads obtained in this flip, and it is 1 if the coin comes up heads and 0 otherwise. We write
XH D I fH g D
(1 if H occurs ; 0 if T occurs :
The expected number of heads obtained in one flip of the coin is simply the ex-pected value of our indicator variable XH:
E ŒXH D E ŒI fH g
D 1 Pr fH g C 0 Pr fT g D 1 .1=2/ C 0 .1=2/
D 1=2 :
Thus the expected number of heads obtained by one flip of a fair coin is 1=2. As the following lemma shows, the expected value of an indicator random variable associated with an event A is equal to the probability that A occurs.
Lemma 5.1
Given a sample space S and an event A in the sample space S , let XA D I fAg.
Then E ŒXA D Pr fAg.
Proof By the definition of an indicator random variable from equation (5.1) and the definition of expected value, we have
E ŒXA D E ŒI fAg
D 1 Pr fAg C 0 Pr˚ A D Pr fAg ;
where A denotes S A, the complement of A.
Although indicator random variables may seem cumbersome for an application such as counting the expected number of heads on a flip of a single coin, they are useful for analyzing situations in which we perform repeated random trials. For example, indicator random variables give us a simple way to arrive at the result of equation (C.37). In this equation, we compute the number of heads in n coin flips by considering separately the probability of obtaining 0 heads, 1 head, 2 heads, etc. The simpler method proposed in equation (C.38) instead uses indicator random variables implicitly. Making this argument more explicit, we let Xibe the indicator random variable associated with the event in which the i th flip comes up heads:
Xi D I fthe i th flip results in the event H g. Let X be the random variable denoting the total number of heads in the n coin flips, so that
X D Xn i D1
Xi:
We wish to compute the expected number of heads, and so we take the expectation of both sides of the above equation to obtain
E ŒX D E
" n X
i D1
Xi
# :
The above equation gives the expectation of the sum of n indicator random vari-ables. By Lemma 5.1, we can easily compute the expectation of each of the random variables. By equation (C.21)—linearity of expectation—it is easy to compute the expectation of the sum: it equals the sum of the expectations of the n random variables. Linearity of expectation makes the use of indicator random variables a powerful analytical technique; it applies even when there is dependence among the random variables. We now can easily compute the expected number of heads:
E ŒX D E
Thus, compared to the method used in equation (C.37), indicator random variables greatly simplify the calculation. We shall use indicator random variables through-out this book.
Analysis of the hiring problem using indicator random variables
Returning to the hiring problem, we now wish to compute the expected number of times that we hire a new office assistant. In order to use a probabilistic analysis, we assume that the candidates arrive in a random order, as discussed in the previous section. (We shall see in Section 5.3 how to remove this assumption.) Let X be the random variable whose value equals the number of times we hire a new office as-sistant. We could then apply the definition of expected value from equation (C.20) to obtain
E ŒX D Xn xD1
x Pr fX D xg ;
but this calculation would be cumbersome. We shall instead use indicator random variables to greatly simplify the calculation.
To use indicator random variables, instead of computing E ŒX by defining one variable associated with the number of times we hire a new office assistant, we define n variables related to whether or not each particular candidate is hired. In particular, we let Xibe the indicator random variable associated with the event in which the i th candidate is hired. Thus,
Xi D I fcandidate i is hiredg
By Lemma 5.1, we have that E ŒXi D Pr fcandidate i is hiredg ;
and we must therefore compute the probability that lines 5–6 of HIRE-ASSISTANT
are executed.
Candidate i is hired, in line 6, exactly when candidate i is better than each of candidates 1 through i 1. Because we have assumed that the candidates arrive in a random order, the first i candidates have appeared in a random order. Any one of these first i candidates is equally likely to be the best-qualified so far. Candidate i has a probability of 1= i of being better qualified than candidates 1 through i 1 and thus a probability of 1= i of being hired. By Lemma 5.1, we conclude that
E ŒXi D 1= i : (5.3)
Now we can compute E ŒX :
E ŒX D E
" n X
i D1
Xi
#
(by equation (5.2)) (5.4)
D Xn i D1
E ŒXi (by linearity of expectation)
D Xn i D1
1= i (by equation (5.3))
D ln n C O.1/ (by equation (A.7)) . (5.5)
Even though we interview n people, we actually hire only approximately ln n of them, on average. We summarize this result in the following lemma.
Lemma 5.2
Assuming that the candidates are presented in a random order, algorithm HIRE -ASSISTANThas an average-case total hiring cost of O.chln n/.
Proof The bound follows immediately from our definition of the hiring cost and equation (5.5), which shows that the expected number of hires is approxi-mately ln n.
The average-case hiring cost is a significant improvement over the worst-case hiring cost of O.chn/.
Exercises
5.2-1
In HIRE-ASSISTANT, assuming that the candidates are presented in a random or-der, what is the probability that you hire exactly one time? What is the probability that you hire exactly n times?
5.2-2
In HIRE-ASSISTANT, assuming that the candidates are presented in a random or-der, what is the probability that you hire exactly twice?
5.2-3
Use indicator random variables to compute the expected value of the sum of n dice.
5.2-4
Use indicator random variables to solve the following problem, which is known as the hat-check problem. Each of n customers gives a hat to a hat-check person at a restaurant. The hat-check person gives the hats back to the customers in a random order. What is the expected number of customers who get back their own hat?
5.2-5
Let AŒ1 : : n be an array of n distinct numbers. If i < j and AŒi > AŒj , then the pair .i; j / is called an inversion of A. (See Problem 2-4 for more on inver-sions.) Suppose that the elements of A form a uniform random permutation of h1; 2; : : : ; ni. Use indicator random variables to compute the expected number of inversions.