Probability Theory I: Syllabus and Exercise
Narn-Rueih Shieh **Copyright Reserved**
§ This course is suitable for those who have taken Basic Probability;
some knowledge of Real Analysis is recommended( will be reviewed in the course).
§ The content and exercise are adapted from R Durrett: Probability: Theory and Examples, Third Edition(2005, Duxbury) Chapters 1,2,4.
§ Grading: HmWk/OnBb: 20 percent; Exams(mid-term and final): 80 percent.
Notice: failure rate may be up to 20 percent
§ Ex’s of Ch 1: p8 1.6,1.9,1.11(BP),1.12(BP);p11 2.4(RA),2.5(RA),2.7(RA);p12
2.8,2.9;p14 3.6;p15 3.8(!);p21 3.11(BP),3.13(RA),3.14(RA),3.18(RA);p28 4.6(BP),4.9(BP);p34 4.18,4.19,4.20;p37 5.2(look);p38 5.3(look);p45 5.3,5.8;p48 6.3(RA),6.4(RA);p51 6.8;p54 6.13,6.16(RA),6.18;p60 7.4;p68 8.3,8.9;p69 8.10
§ Ex’s of Ch 2: p79 1.2, 1.3,;p83 2.2;p88 2.6,2.8;p89 2.9,2.10,2.11,2.15;p95 3.3,3.4;p99
3.13;p101 3.16;p102 3.20,3.21;p104 3.22,exam3.10;p105 3.23,3.25;p109 3.28;p113 4.1(BP);p114 4.5,4.6,4.7;p119 4.9,4.10,4.11,4.13;p137 6.1;p154 7.2;p159 7.5,7.6,7.8;p163 9.1;p167 9.5,9.6;p170 9.7,9.8
§ Ex’s of Ch 4: p220 1.2; p223 1.3,1.4;p225 1.6,1.7;p226 1.9,1.10(BP),1.11;p229 2.2;p235 2.5,2.6,2.8,2.12;p236 2.14;p241 3.7,3.8;p246 3.12;p251 4.8,4.9;p260 5.2,5.3;p261 5.6,5.7;p262 5.8;p263 6.1;p268 6.3;p271 7.1;p273 7.4,7.6,7.8
Introductory
Why we need Probability Theory? Why we need go beyond probability calculations?!
§ the general definition of random variables(rv’s) and expectations.
§ the precise definition of ”with probability one” (as that in SLLN)
§ the general proof of CLT and its extension.
§ the general definition of conditional expectations and “fair games” (martingales).
Chapter 1: Laws of Large Numbers
§ Kolmogorov’s definition of probability space (Ω, F, P ) as a measure space of total measure 1, probability measure
§ basic properties of pm: monotonicity, subadditivity, monotone continuity
§ examples: discrete, unit interval, finite products, infinite products
§ state space S, a separable complete metric space with Borel σ−algebra B(S)
§ Kolmogorov’s definition of an S-valued random variable(rv) as an S-valued mea- surable mapping.
§ probability distribution of a rv, distribution function of a (real-valued) rv, random vector
§ notation of skipping ω
§ properties of a df
§ characterization of a df, K-existence theorem
§ types of rv’s: in terms of df
§ types of rv’s: in terms of pd
§ decomposition theorem
§ examples: BPR for discrete and absolutely continuous rv’s, one example of sin- gularly continuous rv
§ one important estimate: tail distribution of N(0, 1)
§ a property P holds with probability one (w pr 1)=holds almost sure (a.s.)
§ equivalence of two rv’s; in “pointwise” sense and in distributional sense
§ the σ−algebra generated by a rv, by a family of rv’s(in particular a random sequence)
§ operations of rv’s
§ Kolmogorov’s definition of the expectation EX as the Lebesgue integral of X w.r.t. P over Ω
§ integrability= 1st moment exists
§ properties of E: linear, ordered, additive
§ inequalities from RA: Jensen, H´older, Cauchy-Schwarz,
§ Chebyshev inequality
§ a one-sided Chebyshev inequality and its application(BPR)
§ limit theorems from RA: Fatou’s Lemma, MCT, BCT, LDCT
§ “change of variables” formula
§ Lemma: If Y ≥ 0 and p > 0, E(Yp) =∫ 0∞pyp−1P (Y > 0)dy.
§ BP Review: calculations of mean(expectation), moments(at least 2nd moment)
and variance of some rv’s
§ independence: two events, two rv’s, tow sub- σ-algebras
§ the case of more than two: pairwise and totally
§ criterions for independence
§ (joint) pd of n independent rv’s
§ Eh(X, Y ), for two independent X, Y
§ pd of X + Y , for two independent X, Y ; examples
§ Kolmogorov existence(extension, construction) theorem for independent rv’s, fi- nitely many and infinitely many
§ convergence modes of random sequence: a.s. convergence, convergence in proba-
bility, convergence in the mean(in mean square, in Lp(Ω, P ), 1≥ p < ∞), convergence in distributional sense
§ Cauchy criterion for convergence
§ additive property of variance for uncorrelated rv’s
§ WLLN for L2 uncorrelated random sequence, the iid case
§ Bernstein’s approximation theorem
§ the triangular array and its WLLN
§ examples to be read
§ truncation
§ WLLN for the truncated array
§ WLLN for iid L1 random sequence
§ St Petersburg “paradox”
§ Monte Carlo
§ {An i.o.} : the events happen infinitely often; the events happen all but finite many times
§ Lemma: Xn→ 0 a.s. iff ∀² > 0, P {|Xn| > ² i.o.} = 0.
§ Borel–Cantelli Lemma
§ proof of a RA theorem by BC Lemma
§ SLLN for iid rv’s with 4th moment by BC Lemma
§ BC Lemma for independent events; a zero–one law
§ LLN does not hold when iid rv’s with infinite mean
§ SLLN for pairwise independent rv’s with same distribution and with 1st moment:
an 1981 proof
§ a version of SLLN for iid rv’s with infinite mean
§ SLLN for renewal processes
§ Glivenko–Cantelli theorem for empirical distributions
§ Kolmogorov 0–1 law
§ maximal inequality
§ “one-series” theorem
§ three-series theorem
§ LIL
§ random walks
Chapter 2: Central Limit Theorems
§ BPR: DeMoivre-Laplace CLT by hand proof
§ weak convergence of pm’s
§ real line case: convergence in distributional sense
§ examples: DeM-L, G-C, Xn= X + 1/n, pXp, p↓ 0, Xp := G(p)
§ theorem: weak convergence vs a.s. convergence
§ continuous mapping theorem
§ TFAE theorem for weak convergence
§ Helly’s selection theorem (from RA)
§ tightness and its role in Helly’s theorem
§ a criterion for tightness
§ characteristic function (chf) and its basic properties; why chf is better than mgf (you learn this BP)?
§ some calculations on chf’s
§ fourier-L´evy inversion formula
§ absolutely continuous case in FL formula ( fourier inverse transform in RA)
§ L´evy continuity theorem
§ chf for rv with n-th moment
§ chf for rv with 2nd moment
§ Polya’s criterion of chf’s; α−stable distribution
§ moment problem
§ CLT: iid case
§ CLT for triangular arrays: Lindeberg-Feller theorem
§ Poisson convergence theorem
§ BSP Review: Poisson processes and compound Poisson processes
§ beyond CLT: stable laws(distributions), infinitely divisible laws
§ CLT for multidimensional state space Rd, d > 1
Remark: Chapter 3 on RWs is left to your own study; however the notions of stopping times in this chapter will be given in Chapter 4.
Chapter 4: Martingales
§ BP Review: a “fair game” view of SRW on the line
§ conditional probability and conditional expectation: a RA definition
§ lemma: existence and uniqueness in the above definition
§ conditional expectation w.r.t. a decomposition of Ω
§ conditional expectation of X w.r.t. Y ; comparison of that in BP
§ basic properties of conditional expectations
§ conditional expectation of L2 rv’s: a Hilbert space point-of-view
§ conditional variance
§ filtration: an increasing sequence of sub-σ-algebras of Ω; natural filtration of a random sequence
§ definition of a martingale (fair game process), a submartingale, a supermartingale
§ remarks: continuous-time case; multiparameter-time case
§ stopping (optional) times
§ predictable sequence; the “ stochastic integral” process (H · X)n
§ stopped rv, stopped process; the stopped process is still a martingale
§ upcrossings of a sequence over an interval [a, b]
§ a key property: the upper bound for EUn, Un: the upcrossings up to time n
§ martingale convergence theorem
§ Doob-Meyer decomposition theorem
§ an example from SRW
§ no “bounded infinite oscillations”
§ a conditional form of BC Lemma
§ martingale property of a Galton-Watson branching process
§ a theorem on EXN, N bounded by k
§ Doob’s inequality (1)
§ Doob’s inequality (2)
§ martingale Lp, 1 < p <∞, convergence theorem
§ uniform integrability and L1 convergence: from RA
§ martingale L1 convergence theorem
§ L´evy’s convergence theorem and 0–1 law
§ the stopped process is still u.i.
§ a theorem on EXN, N any stopping time
§ stopped σ-algebra FN; a monotone lemma
§ optional stopping theorem
§ backwards (reversed) martingale and its convergence theorem
§ Hewitt-Savage 0–1 law
§ SLLN as a consequence of backwards martingale convergence theorem
§ square integrable martingales
§ martingale CLT
§ remarks: the continuous-time case, and the multiparameter-time case
