Fundamentals of Probability

開課系所 名稱及代碼

電 子 工 程 學 系

二年級

必修 選修 上 下

開課科目代碼

D-5002-02802

9

3

課 程 名 稱 (中英文並列)

Fundamentals of Probability

任 課 教 授 (中英文並列)

Yu, Jung-Lang

課程目標

The aim of this course is to present probability in a natural way through interesting and instructive examples and exercises that motivates the theory, definitions, theorems and methodology.

For the students to:

(1) Acquire an understanding of the basic concepts of probability.

(2) Become interested in the theory of probability.

(3) Improve their problem solving skills and develop their proof writing techniques

課程綱要

1. Axioms of Probability. (0.5week) 2. Combinatorial Methods. (0.5W)

3. Conditional Probability and Independence. (2.0W)

4. Distribution Functions and Discrete Random Variables. (2.0W) 5. Special Discrete Variables. (1.0W)

6. Continuous Random Distributions. (1.0W) 7. Special Continuous Distributions. (2.0W) 8.Joint Distributions. (2.5W)

9. More Expectations and Variances. (1.5W)

10. Sums of Independent RVs and Limit Theorems. (1.0W) 先 修 課 程 None

建議續修科目 Signal and System, 指定教材

(用書、器材)

S. Ghahramani, ‘Fundamentals of Probability’, 3rd Edition, 2005 (新 月)

參考書目

R.E. Ziemer, ‘Elements of engineering probability and statistics’, 2nd Edition, 1997 (全華)

P.L. Meyer, ‘Introductory probability and statistical applications’, 2nd Edition, 1970 (中央)

備註 Quizs 40%, Midterm 25% Final 35%

Introduction to Probability Theory

Instructor: Dr Jung-Lang Yu(余金郎). Office:SF709, Tel: 29052102

Lectures: Mon. 08:10-09:00, room SF338, Tue. 10:10-12:00, room SF132.

Office hours: Thu. 14:00-16:00, Fri. 14:00-16:00, room SF709 (by appointment if necessary).

E-mail: [email protected]

Textbook: Fundamentals of Probability by Saeed Ghahramani, 3^rd edition, Prentice- Hall, Inc. The course will cover some but not all of the material.

Material Download: http://www.ee.fju.edu.tw/communication/main.html Prerequisites: Calculus.

Course description: Basic notions of probability, discrete and continuous probability distributions, expectation and conditional expectation, independence, limit theorems and notions of convergence.

My role in this course is to help you learn and understand the material. Please come and see me in my office hours immediately if you have troubles with the material, so we can intervene early. Mathematics is not a selection of disassociated pieces, but the parts build upon each other. Mastering mathematics is not very different from mastering a sport.

You need to practice to get the moves right, and then you need to understand the dynamics to put the different moves together for a coherent strategy. Your part consists of

♦ doing the homework (math is learned by doing, not by watching). It has been shown that Time-on-Task (TOT) (which consists of taking notes, re-reading the material after class, doing homework and studying) has the highest correlation with the final grade!!

♦ to ask if you do not understand. Asking does not mean that you are stupid, but rather that you are smart enough to realize that clarifying what bothers you is an important step in doing well in this (or any) class. For the majority of questions there will be many grateful classmates who had the same question, but did not have the guts to ask.

♦ to take advantage of my office hours. This time is set aside for you to come and ask questions. You can make the most of this time by coming prepared – bring your notes with marks on where you have questions, and show me where you get stuck in solving a particular problem. I can assist you best when you have specific questions.

In terms of studying, I urge you to work with other students. Even if your schedule is such that you cannot meet with somebody else, identify somebody whom you can call. Often hours of frustration can be avoided by talking to somebody else. Think about the problems by yourself first, and then ask about the ones you had trouble with.

In preparation for the class, you should read ahead in the material (find the places which they appear in the book and mark them by red pen). This way you can pay particular attention to anything that did not make sense to you. In addition, it is a good idea keep a note book in which you record definitions, theorems and formulas (this will be a good summary for review before exams and will come in handy in any course in which you may need this material). You may also want to summarize the sections/chapters to help in studying for exams. Another use of this notebook is to record which parts of the material/

homework questions you had trouble with. Often when writing down your difficulties they may clear up. I will not collect/check this notebook, but that should not keep you from making entries.

Course Grade:

Your final grade will be computed as follows:

Quiz 40% ( 4 quizzes during this class ) Midterm Exam 25%

Final Exam 35%

Attendance (I expect you to be on time and to stay until the end of class!) and class participation will be taken into account in borderline cases.

Important date:

Midterm Exam: 12-16 November, 2007 Final Exam: 14-18 January, 2008 Quiz:

1^st: Oct. 18, 2007 2^nd: Nov. 07, 2007 3^rd: Dec. 13, 2007 4^th: Jan. 09, 2008

Homework: The minimum exercises essential for the course are listed in the following.

♦ Chapter 1:

Section 1.2 1,3,5,7 Section 1.4 1,3,7,9,19 Section 1.7 1,5,10

♦ Chapter 2:

Section 2.2 1,3,5,8,9,11,17 Section 2.3 1,5,9

Section 2.4 1,3,5,11,13,15,21,37

♦ Chapter 3:

Section 3.1 1,3,5,11 Section 3.2 1 Section 3.3 1,3,7 Section 3.4 1,3,7,9

Section 3.5 3,5,7,9,15,17,25,27,34,35

♦ Chapter 4:

Section 4.2 1,5,11 Section 4.3 1,3,7,11 Section 4.4 3,5,7 Section 4.5 1,3,5,7 Section 4.6 1

♦ Chapter 5:

Section 5.1 1,3,9,13,17,19 Section 5.2 1,3,5,7,9 Section 5.3 1,5,13,17

♦ Chapter 6:

Section 6.1 1,3,5,7,9 Section 6.2 3,7 Section 6.3 2,3,4,5,7 Review 9,12

♦ Chapter 7:

Section 7.1 1,5,11

Section 7.2 1,9,11,15,17,21 Section 7.3 3,5,6

♦ Chapter 8:

Section 8.1 1,5,7,9,11,13,15,17,25 Section 8.2 1,3,5,9,13,15,17 Section 8.3 1,5,7,11,13

♦ Chapter 10:

Section 10.1 3,5,7 Section 10.2 1,3,5,9,11 Section 10.3 1,3,5 Section 10.4 1,3,5,19

Section 10.5 1,3,5

♦ Chapter 11:

Section 11.1 1,3,4,11,13,15,17 Section 11.2 1,11,15

Section 11.3 1,3,5,7 Section 11.4 1 Section 11.5 1,3,5

Chapter 1. Axioms of Probability

1.1. Introduction

z Probability

) Relative frequency interpretation

) (No. of desired outcomes) / (No. of experiments)

) Counting the number of different ways that a certain event can occur

) ( )

limn

n A n

→∞

1.2. Sample Space and Events

z Sample Space

) Set of all possible outcomes of an (a random) experiment ) Usually denoted by S

) Outcomes: sample points of S

z Event

) Subset of S ) Set of outcome(s)

Example 1.1 For the experiment of tossing a coin once, the sample space S consists of two pints (outcomes), “heads” (H) and “tails” (T). Thus S={H,T}.

Example 1.2 Suppose that an experiment consists of two steps. First a coin is flipped. If the outcome is tails, a die is tossed. If the outcome is heads, the coin is flipped again. The sample space of this experiment is S = {T1, T2, T3, T4, T5, T6, HT, HH}. For this

experiment, the event of heads in the first flip of the coin is E = {HT, HH}, and the event of an odd outcome when the die is tossed is F = {T1, T3, T5}

z Relations between events ) Subset

ƒ E is said to be a subset of F if whenever E occurs F also occurs.

ƒ E⊆

F

ƒ E and F are said to be equal if the occurrence of E implies the occurrence of F, and vice versa.

ƒ E⊆ and F

F

E

ƒ An event is called the intersection of two events E and F if it occurs only whenever E and F occur simultaneously.

ƒ EF or ^{E∩ =F}

{

}

) Union

ƒ An event is called the union of two events E and F if it occurs whenever at least one of them occurs.

ƒ ^{E∪ =F}

{

}

) Complement

ƒ An event is called the complement of the event E if it only occurs whenever E does not occur.

ƒ ^{Ec = − =S E}

{

}

) Difference

ƒ ^{A B− =}

{

}

) Certain Event

ƒ An event is called certain if its occurrence is inevitable

ƒ S

) Impossible Event

ƒ An event is called impossible if there is certainty in its nonoccurrence.

ƒ ∅ (the empty set),

S

ƒ If the joint occurrence of two events E and F is impossible, we say that E and F are mutually exclusive (disjoint).

ƒ EF = ∅

) A set of mutually exclusive events { ,

E E

E

Example 1.7 At a busy international airport, arriving planes land on a first-come,

first-served basis. Let E, F, and H be the events that there are, respectively, at least five, at

most three, and exactly two planes waiting to land. Then

♦ E^c is the event that at most four planes are waiting to land.

♦ F^c is the event that at least four planes are waiting to land.

♦ E is a subset of F^c; that is, if E occurs, then F^c occurs. Therefore, EF^c = E.

♦ H is a subset of F; that is, if H occurs, then F occurs. Therefore, FH = H.

♦ E and F are mutually exclusive; that is, EF = ∅. E and H are also mutually exclusive since EH = ∅.

♦ FH^c is the event that the number of planes waiting to land is zero, one or three.

z Venn Diagram: Figure 1.1

) The shaded regions are the indicated events.

) More relations:

ƒ (

E

E E

E

S EE

ƒ Commutative Laws: E∪ = ∪ , EF FE

F F E

ƒ Associative Laws:

E

F

G

E

F

G E FG

EF G

ƒ Distributive Laws: (

EF

G

E G F

G

E

F G

EF

EG

ƒ Demogan’s First Laws:

1 1

( )^{c c c}, ( ⁿ _i)^{c n} _i^c

i i

E G E G E E

= =

∪ =

∪

∩

ƒ Demogan’s Second Laws:

1 1

( )^{c c c}, ( ⁿ _i)^{c n} _i^c

i i

EG E G E E

= =

= ∪

∩

∪

ƒ

E ES

E F

F

EF

EF

EF are disjoint

1.3. Axioms of Probability

z Definition (Probability Axioms)

Let S be the sample space of a random phenomenon. Suppose that to each event A of S, a number denoted by P(A), is associated with A. If P satisfies the following axioms, then it is called a probability and the number P(A) is said to be the probability of A.

) Axiom 1: ( ) 0

P A

) Axiom 3: If { ,

A A A

i j ,

A A

i j

1 1

P ( _i) ( )_i

i i

A P A

∞ ∞

= =

=

∑

∪

z Note that the axioms of probability are a set of rules that must be satisfied before S and P can be considered a probability model. Probability is a real-valued nonnegative, countably additive function.

z “Equally likely” or “Equally probable”: we say that (A,B) are equally likely if P(A) = P(B). Sample points w₁ and w₂ are equally likely if P w

(

)

(

)

z Theorem 1.1 ( ) 0

P

z Theorem 1.2

Let { ,

A A A

A be a mutually exclusive set of events. Then

1 1

P ( ⁿ _i) ⁿ ( )_i

i i

A P A

= =

=

∑

∪

Note that if A₁ and A₂ are mutually exclusive, then P (A₁∪A₂)=P A( )₁ +P A( )₂ . Also note that 1=P(S)=P (

A

A

P A

P A

P A

P A

Example 1.9 A coin is called unbiased or fair if, whenever it is flipped, the probability of obtaining tails. Suppose that in an experiment is an unbiased coin is flipped. The sample space of such an experiment is ^{S= T, H}

{ }

{ }

{ }

( ) { } ( ) { }

( { } ) ( ) { } ( ) { }

P T,H =P T +P H Hence Axioms 2 and 3 imply that

( ) { } ( { } ) ( ) { } ( ) { }

( ) { } ( ) { } ( ) { }

1=P S =P H,T =P H +P T =P H +P H =2P H .

This gives that P H =1/2 and P T =1/2.

( ) { } ( ) { }

( ) { } ( ) { }

P T = 2P H .Hence

( ) { } ( { } ) ( ) { } ( ) { }

( ) { } ( ) { } ( ) { }

1=P S =P H,T =P H +P T =P H +2P H =3P H . This shows that ^{P H =1/3}

( ) { }

( ) { }

Example1.10 Sharon has baked five loaves of bread that are identical except that one of them is underweight. Sharon’s husband chooses one of these loaves at random. Let

Bi ,1 i 5,≤ ≤ be the event that he choose the i-th loaf. Since all five loaves are equally likely to be drawn, we have

( { }

) ( { }

) ( ^{{ }}

) ( { }

) ( ^{{ }}

)

P B =P B =P B =P B =P B .

But the events

{ } { }

{ }

{ }

{ }

{

}

P B ,B ,B ,B ,B Therefore, by Axioms 2 and 3,

( ) ( { }

) ( { }

) ( ^{{ }}

) ( { }

) ( ^{{ }}

) ( { }

)

1=P S =P B +P B +P B +P B +P B =5 P Bi .

This gives P B

( { }

)

( { }

)

z Theorem 1.3

Let S be the sample space of an experiment. If S has N points that are all equally likely to occur, then for any event A of S,

( )

N A

P A

N

Example 1.11 Let S be the sample space of flipping a fair coin three times and A be the event of at least two heads; then

{ }

S= HHH,HTH,HHT,HTT,THH,THT,TTH,TTT

and A=

{

}

Example 1.12 An elevator with two passengers stops at the second, third, and fourth floors. If it is equally likely that a passengers gets off at any of the three floors. What is the probability that the passengers get off at different floors?

Solution: Let a and b denote the tow passengers and a

b

second floor and b gets off at the fourth floor, with similar representations for other cases.

Let A be the event that the passengers gets off at different floors. Then { 2 2, 2 3, 2 4, 3 2, 3 3, 3 4, 4 2, 4 3, 4 4}

S

a b a b a b a b a b a b a b a b a b

And

A

a b a b a b a b a b a b

Example 1.13 A number is selected at random form the set of natural numbers {1,2,3,4, ,1000}. What is the probability that the number is divisible by 3 ?

Solution: Here the sample space contains 1000 points so N＝1000. Let A be the set of all numbers between 1 and 1000 which are divisible by 3. Then A={3m:1 m 333}≤ ≤ . So N(A)

＝333. Therefore, the probability that a random natural number between 1 and 1000 is divisible by 3 is equal to 333/1000=0.333.

1.4. Basic Theorems

z Theorem 1.4

For any event A, ( ) 1

P A

P A

If A⊆ , then (

B P B A

P BA

P B

P A

If A⊆ , then ( )

B P B

P A

( ) ( ) ( ) ( )

P B

A

P B

P A

P B

A

1 2 3 1 2 3 1 2 1 3

2 3 1 2 3

( ) ( ) ( ) ( ) ( ) ( )

( ) ( )

P A A A P A P A P A P A A P A A P A A P A A A

∪ ∪ = + + − −

− +

1 2 3 4 1 2 3 4 1 2

1 3 1 4

( ) ( ) ( ) ( ) ( ) ( )

( ) ( )

P A A A A P A P A P A P A P A A

P A A P A A

∪ ∪ ∪ = + + + −

− − −

z Inclusion-Exclusion Principle

1

1 1 1

1

P ( ⁿ _i) ⁿ ( )_i ^{n n} ( _{i j})

i i j i

i

A P A ⁻ P A A

= = = +

=

=

∑

∑ ∑

∪

Example 1.15 Suppose that in a community of 400 adults, 300 bike or swim or do both, 160 swim, and 120 swim and bike. What is the probability that an adult, selected at random

from this community, bikes? Answer: 0.65

Example 1.16 A number is chosen at random from the set of numbers {1,2,3,…,1000}.

What is the probability that it is divisible by 3 or 5 (i.e. either 3 or 5 or both)? Answer: .0467

z Theorem 1.7

( ) ( ) ( ^c)

P A

P AB

P AB

Example 1.19 In a community, 32% of the population are male smokers; 27% are female smokers. What percentage of the population of this community smoke?

Answer: 59%

z Odds: in the real world, especially for games of chance, it is common to express probability in terms of odds.

The odds in favor of an event A are r to s if ( ) ( )^c

P A r

P A

s r P A

r s

+ ) The odds against an event A are r to s if ^{( )}

( ) P Ac r

P A = s ( or ( )

s P A

r s

+ )

Thus if the odds in favor of an event A are r to s, then the odds against an event A are s to r.

Example 1.11: the odds in favor of HHH are 1/8 to 7/8 or, equivalently, 1 to 7.

1.5. Continuity of Probability Function

z Continuity for a real-valued function ) lim ( ) ( ),

x c

f x f c c R

→ = ∀ ∈

) equivalent to the sequential criterion lim ( )_n (lim ),_n

n

f x f

x

→∞ = →∞ for every convergent sequence { }

x

) increasing sequence: A sequence { ,

E n

increasing if

E

E

E

E

E

1

lim _{n n}

n n

E

E

→∞ =

=

∪

) decreasing sequence: A sequence { ,

E n

decreasing if

E

E

E

E

E

1

lim _{n n}

n n

E

E

→∞ =

=

∩

z Theorem 1.8 (Continuity of Probability Function)

For any increasing or decreasing sequence of events, { ,

E n

lim ( )_n (lim _n)

n

P E P

E

→∞ = →∞

1.6. Probabilities 0 and 1

z There exist infinitely many events each with probability 1, and infinitely many events each with probability 0.

z

P E

P E

z In random selection of points from a continuous period, say (0,1), the probability of the occurrence of any particular point is 0 (shown in page 30 via an example). let

{ }, (0,1)

E

t t

B

t

P E

P B

1.7. Random Selection of Points from Intervals

z Probability that random selection of points from intervals (a,b) is 0 (Section 1.6) z The probability that a random point selected from (a,b) falls into the interval ( , )

2

a a b

is 1/2. The probability that it falls into ( , ) 2

a b

b

is also 1/2.

z The probability of the event that a random point from (a,b) falls into a subinterval ( , )

α β

b a β α

− . (can be induced from Theorem 1.3) z Definition (uniform distribution)

A point is said to be randomly selected from an interval (a,b) if any two subintervals of (a,b) that have the same length are equally likely to include the point. The probability associated with the event that the subinterval ( , )

α β

b a

β α

− .

Chapter 2. Combinatorial Methods

2.1. Introduction

z Combinatorial Analysis

) Dealing with method of counting ) Enabling us to count systematically

) Some problems in probability can be solved just by counting the total number of sample points and the number of ways that an event can occur when all sample points are equally likely. (c.f. Theorem 1.3)

2.2. Counting Principle

z Theorem 2.1 (Counting Principle)

If the set E contains n elements and the set F contains m elements, there are nm ways in which we can choose first an element of E and then an element of F.

z Theorem 2.2 (Generalized Counting Principle)

Let

E E

E be the sets with

n n

n elements, respectively. Then there are

1 2 k

n n

n

E

E

Example 2.1 How many outcomes are there if we throw five dice? Answer: 6 ⁵

Example 2.2 In tossing four fair dice, what is the probability of at least one 3? Answer:

671/1296~=0.52

Example 2.4 Rose has invited n friends to her birthday party. If they all attend, and each one shakes hands with everyone else at the party exactly once, what is the number of handshakes? Answer: n(n+1)/2

Example 2.6 (Standard Birthday Problem) What is the probability that at least two

students of a class of size n have the same birthday? Compute the numerical values of such

probabilities for n = 23,30,50, and 60. Assume that the birth rates are constant throughout the year and that each year has 365 days. Answer: 0.507, 0.706, 0.970, 0.995.

z Power Set

The set of all subsets of a set A is called the power set of A.

z Theorem2.3 (Number of Subsets of a Set)

) A set with n element has 2ⁿ subsets. (The proof is based on the generalized counting principle.)

Example 2.7 A restaurant advertises that it offers over 1000 varieties of pizza. If, at the restaurant, it is possible to have on a pizza any combination of pepperoni, mushrooms, sausage, green peppers, onions, anchovies, salami, bacon, olives, and ground beef, is the restaurant’s advertisement true? Answer: True.

z Tree Diagrams

) Useful pictorial representations breaking down a complex counting problem into smaller, more tractable ones.

) The number of possible ways that an experiment can be performed is finite.

) Systematically identifying all possible cases.

Example 2.8 Bill and John Keep playing chess until one of them wins two games in a row or three games altogether. In what percent of all possible cases does the game end

because Bill wins three games without winning two in a row? Answer: 10%

Example 2.9 Mark has $4. He decides to bet $1 on the flip of a fair coin four times. What is the probability that (a) he breaks even; (b) he wins money? Answer: 6/16, 5/16.

2.3. Permutations

z Definition

An ordered arrangement of r objects from a set A containing n objects (0< ≤

r n

P .

Notes: !

( 1)( 2) ( 1)

( )!

n r

P n n n n r n

= − − − + =

n r

− and _{n n}

P

n n

n

n n

n

Example 2.10 Three people, Brown, Smith, and Jones, must be scheduled for job interviews. In how many different orders can this be done? Answer: 6.

Example 2.11 Suppose that two anthropology, four computer science, three statistics, three biology, and five music books are put on a bookshelf with a random arrangement.

What is the probability that the books of the same subject are together?

Answer:

P A

Example 2.12 If five boys and five girls sit in a row in a random order, what is the probability that no two children of the same sex sit together?

Answer: ( ) 2 5! 5!/10! 0.008

P A

z Theorem 2.4

The number of distinguishable permutation of n objects of k different types, where

n1 are alike, n₂ are alike , ,

n are alike and

n n

n

n

1 2

!

! ! _k!

n

n

n

n

Example 2.13 How many different 10-letter codes can be made using three a’s, four b’s, and three c’s? Answer: 10!/(3! 4! 3!) 4200× × =

Example 2.15 A fair coin is flipped 10 times. What is the probability of obtaining exactly three heads? Answer:

P A

2.4. Combinations

z Definition

An unordered arrangement of r objects from a set A containing n objects

(0< ≤

r n

z Notes:

) !

( )! !

n r

n n

C r n r r

=⎛ ⎞⎜ ⎟⎝ ⎠= − × (

!

n r

P

=

r

) 1,

0 1 1

n n n n

n n n

⎛ ⎞ ⎛ ⎞ ⎛ ⎞ ⎛ ⎞

= = = =

⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ − ⎟

⎝ ⎠ ⎝ ⎠ ⎝ ⎠ ⎝ ⎠

) For any 1

0 , ,

1

n n n n n

r n r n r r r r

⎛ ⎞ ⎛ ⎞ ⎛ + ⎞ ⎛ ⎞ ⎛ ⎞

≤ ≤ ⎜ ⎟ ⎜⎝ ⎠ ⎝= − ⎟ ⎜⎠ ⎝ ⎟ ⎜ ⎟ ⎜⎠ ⎝ ⎠ ⎝= + − ⎟⎠

Example 2.16 In how many ways can two mathematics and three biology books be selected from eight mathematics and six biology books? Answer: 560

Example 2.18 In a small town, 11 of the 25 schoolteachers are against abortion, eight are for abortion, and the rest are indifferent. A random sample of five schoolteachers is selected for an interview. What is the probability that (a) all of them are for abortion; (b) all of them have the same opinion? Answer: 0.0011, 0.0099

Example 2.20 From an ordinary deck of 52 cards, seven cards are drawn at random and without replacement. What is the probability that at least one of the cards is a king?

Answer: 0.4496

Example 2.21 What is the probability that a poker hand is a full house? A poker hand consists of five randomly selected cards from an ordinary deck of 52 cards. It is a full house if three cards are of one denomination and two cards are of another denomination: for example, three queens and two 4’s.

Answer:

13 12 4 4

1 1 3 2

0.0014 52

5

⎛ ⎞⎛ ⎞⎛ ⎞⎛ ⎞

⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟

⎝ ⎠⎝ ⎠⎝ ⎠⎝ ⎠ ≈

⎛ ⎞⎜ ⎟

⎝ ⎠

Example 2.22 Show that the number of different ways n indistinguishable objects can be placed into k distinguishable cell is 1 1

1

n k n k

n k

+ − + −

⎛ ⎞ ⎛ ⎞

⎜ ⎟ ⎜= − ⎟

⎝ ⎠ ⎝ ⎠

Example 2.23 Let n be a positive integer, and let

x

x

x

n

equation. A vector ( , , , )

x x

x satisfying

x

x

x

n

x is a nonnegative integer. It is

x is a positive

1 2 k

x

x x

n

x

x

x

n

z Theorem 2.5 (Binomial Expansion) For any integer

n

( )

0

1 2 2 1

0 1 2 1

n n n i i

i

n n n n n

x y n x y

i

n n n n n

x x y x y xy y

n n

−

=

− − −

+ = ⎛ ⎞⎜ ⎟

⎝ ⎠

⎛ ⎞ ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ ⎛ ⎞

=⎜ ⎟⎝ ⎠ +⎜ ⎟⎝ ⎠ +⎜ ⎟⎝ ⎠ + +⎜⎝ − ⎟⎠ +⎜ ⎟⎝ ⎠

∑

Example 2.25 What is the coefficient of

x y in the expansion of

x

y

Example 2.26 Evaluate the sum

0 1 2 3

n n n n n

n

⎛ ⎞ ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ ⎛ ⎞

+ + + + +

⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟

⎝ ⎠ ⎝ ⎠ ⎝ ⎠ ⎝ ⎠ ⎝ ⎠. Answer: 2ⁿ.

Example 2.27 Evaluate the sum 2 3

1 2 3

n n n n

n n

⎛ ⎞ ⎛ ⎞ ⎛ ⎞ ⎛ ⎞

+ + + +

⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟

⎝ ⎠ ⎝ ⎠ ⎝ ⎠ ⎝ ⎠.

Answer:

n

( ) ( )

n n! n (n-1)! n-1

i =i = =n .

i i! n-i ! (i-1)! n-i ! i-1

⎛ ⎞ ⎛ ⎞

⎜ ⎟ ⎜ ⎟

⎝ ⎠ ⎝ ⎠

i i

So from Example 2.26, we have

n-1

n n n n

+2 +3 + +n

1 2 3 n

n-1 n-1 n-1 n-1

=n + + + +

0 1 2 n-1

=n 2 .

⎛ ⎞ ⎛ ⎞ ⎛ ⎞ ⎛ ⎞

⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟

⎝ ⎠ ⎝ ⎠ ⎝ ⎠ ⎝ ⎠

⎡⎛ ⎞ ⎛ ⎞ ⎛ ⎞ ⎛ ⎞⎤

⎢⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟⎥

⎝ ⎠ ⎝ ⎠ ⎝ ⎠ ⎝ ⎠

⎣ ⎦

i

Example 2.28 Prove that

2

0

2 ⁿ

i

n n

n

i

⎛ ⎞= ⎛ ⎞

⎜ ⎟ ⎜ ⎟

⎝ ⎠

∑

1 2

1 2

1 2 1 2

1 2

( ) !

! ! !

k k

n n n n

k k

x x x n k

x x x n x x x

n n n

+ + + =

+ + + =

∑

Note that the sum is taken over all nonnegative integers

n n

n such that

1 2 k

n

n n

n

2.5. Stirling’s Formula

z Theorem 2.7 (Stirling’s Formula)

! 2 ^{n n},

n

π nn e

lim ! 1

2 ^{n n}

n

n π nn e

→∞ =

Chapter 3. Conditional Probability and Independence

3.1. Conditional Probability

z The conditional probability of A given B

) Knowing that event B has occurred changes the chances of the occurrence of event A

) Denoted by ^{P A B}

(

)

If ^{P B}

( )

( ) ( )

|

P AB ( )

P A B

=

P B

Example 3.1 In a certain region of Russia, the probability that a person lives at least 80 years is 0.75, and the probability that he or she lives at least 90 years is 0.63. What is the probability that a randomly selected 80-year-old person from this region will survive to become 90? Answer: 0.84

Example 3.2 From the set of all families with two children, a family is selected at random and is found to have a girl. What is the probability that the other child of the family is a girl?

Assume that in a two-child family all sex distributions are equally probable. Answer: 1/3

Example 3.3 From the set of all families with two children, a child is selected at random and is found to be a girl. What is the probability that the second child of this girl’s family is also a girl? Assume that in a two-child family all sex distributions are equally probable.

Answer: 1/2

Example 3.5 We draw eight cards at random from an ordinary deck of 52 cards. Given that three of them are spades, what is the probability that the remaining five are also spades? Answer: 5.44 10× ⁻⁶

( )

8 -6

3

13 39

13 8 -

( )

( | ) 5.44 10 .

8 52

8

X

X X

P A B P AB

P B

⎛ ⎞⎛ ⎞

⎜ ⎟⎜ ⎟

⎛ ⎞ ⎝ ⎠⎝ ⎠

= =⎜ ⎟⎝ ⎠ ⎛ ⎞ ≈ ×

⎜ ⎟⎝ ⎠

∑

z Theorem 3.1

Let S be the sample space of an experiment, and let B be an event of S with P(B)>0.

Then the conditional probability satisfies the probability axioms, i.e., (1) ( | ) 0

P A B

(2) ( | ) 1

P S B

(3) If

A A

A

( )

1 1

| |

i i

i i

P

A B

P A B

=

=

⎛ ⎞

⎜ ⎟=

⎝

∪

∑

z Reduction of Sample Space

Let B be an event of a sample space S with P(B)>0. For A⊆ , define ( )

B Q A

P A B

Q subset B

Q A

Q B

P B B

) If

A A

A

( ) ( )

1 1

1 1

| |

i i i i

i i

i i

Q

A P

A B

P A B

Q A

= =

= =

⎛ ⎞= ⎛ ⎞= =

⎜ ⎟ ⎜ ⎟

⎝

∪

∪

∑ ∑

Thus Q satisfies the probability axioms and is a probability function.

For Q, the sample space is reduced from S to B. ( cf. For P, the sample space is S) Suppose that we are interested in P(E|B) where E⊆ . It is usually easier to compute

S

Example 3.6 A child mixes 10 good and three dead batteries. To find the dead batteries, his father tests them one-by-one and without replacement. If the first four batteries tested are all good, what is the probability that the fifth one is dead? Answer: 3/9

Example 3.7 A farmer decides to test four fertilizers for his soybean fields. He buys 32 bags of fertilizers, eight bags from each kind, and tries them randomly on 32 plots, eight plots from each of field A, B, C, and D, one bag per plot. If from type I fertilizer one bag is tried on field A and three on field B, what is the probability that two bags of type I fertilizer are tried on field D? Answer: 0.43

z For all choices of B, P(B)>0 )

P

B

)

P A B

P A B

) If C⊆ , then (

A P AC B

P A C B

P A B

P C B

A P C B

P A B

)

P A C B

P A B

P C B

P AC B

P A B

P AC B

P AC B

) ¹

1 1 1

1

P ( ⁿ _i| ) ⁿ ( | )_i ^{n n} ( _{i j}| )

i i j i

i

A B P A B ⁻ P A A B

= = = +

=

=

∑

∑ ∑

∪

) For any increasing or decreasing sequence of events, { ,

A n

lim ( _n| ) (lim _n| )

n

P A B P

A B

→∞ = →∞

3.2. Law of Multiplication

z Law of Multiplication

Calculating ^{P AB}

( )

(

)

(

)

( ) ( ) (

)

P AB =P B P A B , ^{P B}

( )

( ) ( ) (

)

P BA =P A P B A , ^{P A}

( )

( ) ( ) ( ) ( ) (

) ( ) (

)

P AB =P BA ⇒P AB =P A P B A =P B P A B

∵

Example 3.9 Suppose that five good fuses and two defective ones have been mixed up.

To find the defective fuses, we test them one-by-one, at random and without replacement.

What is the probability that we are lucky and find both of the defective fuses in the first two tests? Answer: 1/21

Solution: Let D1 and D2 be the events of finding a defective fuse in the first and second tests, respectively. We are interested in P(D1D2). Using (3.5), we get

1 2 1 2 1

2 1 1

( ) ( ) ( | ) = =

7 6 21

P D D

P D P D D

z Extended Law of Multiplication:

( ) ( ) (

) (

)

P ABC =P A P B A P C AB

since

( ) ( ) ( ) ( ) ( ) ( ) ( )

( ) ( )

| |

P BA P CAB

P A P B A P C AB P A P ABC

P A P AB

= =

z Theorem 3.2 Generalized Law of Multiplication If

P A A

A

1 2 1 2 1 3 1 2 1 2 1

( _n) ( ) ( | ) ( | ) ( _n| _n )

P A A

A

P A P A A P A A A P A A A

A