Outline Chapter5SumorRandomVariableandLarge-TermAverage

Chapter 5

Sum or Random Variable and Large-Term Average

Random signal and Process 輔仁大學 電機工程系所

Outline

• Expected value (Mean) of Random Variable

— Sum of independent identically distribution

— Sample Mean

— Chebyshev inequality

• Central Limit Theorem

— CDF, PDF, Proof

• Confidence Interval

— Sample mean, unknow Mean and know variance

— Unknow Mean and variance

— Chi-square R.V. with (n-1) degrees of freedom

— ^{St udent ’ s t - di st r i but i on wi t h}

(n-1) degrees of freedom

5 - 2 Chapter 5: Sum of Random Variable and

Large-Term Average Dr. Sheng-Chou Lin

Expected value (Mean) of Random Variable

: a sequence of R.V.

: Sum of R.V.

•Meanof

•Variance of , X₁  X₂  X_n

S_n = X₁ +X₂+ X+ _n S_n

E X₁+X₂+X_n E X=   E X₁ +    E X₂ + +  _n S_n

Z = X+Y E Z  E X=   E Y+   VAR Z  E Z E= – ² Z 

E X+Y–E X  E Y– ²

=

VAR X  VAR Y+   2COV X Y+  

= 0

_Z²_X²+_Y²



VAR X₁+X₂+X_n

VAR X _k COV X_jX_k

k=1



n j=1



n

+

k=1



n

=

jk

-

^{If are indep.}

-

sum of indep., identically dist. (iid)

•pdfof : are indep.

For n=2,

X₁  X₂  X_n

VAR X₁+X₂+X_n VAR X _k

k=1



n

=

E S  E X_n = ₁+X₂+X_n n= VAR S  AR X_n = ₁+X₂+X_n n= ²

S_n X₁  X₂  X_n Z = X+Y

_Z E ew = ^jwZ E e= ^jwXe^jwY  E e^jwX E e+ ^jwY

= = _Xw _Yw

f_Zz

 = f_X fx  _Yy

Random signal and Process 輔仁大學 電機工程系所

Expected value (Mean) of Random Variable

• For n independent R.V.

- -

-

^{Sum of n iid R.V.’s}

Ex: S_n: Sum of n independentGaussian R.V.’s with m₁, ... , m_n, and ₁²,...,_n²

S_n: Gaussian R.V. with m = m₁+ m₂... + m_n and ²=₁²+. .. +_n²

_S

n w _X

1w _X

nw

= f_S

n z ^–1 _X

1w _X

nw

 

=

_S

nw = _Xw ²

_X

k ew = ^{jw mk–jw2k2 2}

_S

nw e^{jw mk–jw2k2 2}

k=1



n

=

e^{jw m1+ m+ n jw– 212 + + n2 2}

=

Ex: A sum of n i.i.d. Exponential R.V. with

m Erlang R.V.

_Xw 

 jw– ---

= _S

nw 

 jw– --- ⁿ

 =



 jw– --- ^m

Exponential

mErlang R.V

5 - 4 Chapter 5: Sum of Random Variable and

Large-Term Average Dr. Sheng-Chou Lin

Sample Mean

: n i.i.d R.V’s with the same pdf

with .

• Sample mean

estimate

•Mean square errorof the sample mean about= variance of M_n

X₁  X₂  X_n E X  =

M_n 1

n--- X_j

j=1



n

=

E X 

E M  E 1_n n--- X_j

j=1



n _n^{---1 E X j} j=1



n ^

= = =

MSE = E M_n–² 1

n²

---VAR S  n_n ² n² ---

=

=

² ---n

= n²

Chebyshev inequality

Ex: Measurement ; : desired voltage, : Noise voltage with .

• How many measurements P X m– a ²

a² ---



P M_n –E M _n  ²n

² ---





P M_n – 1 ² n² ---

 – = 1–

select n X_j = V+N_j V

N_j  1V=

P M_n –V  1 ² n² ---

 – 1 1

n---

– 0.99

= =

1V  1V= _{at least}

n100

 S_nn

f_M

nm



²n

Random signal and Process 輔仁大學 電機工程系所

Central Limit Theorem

: n i.i.d R.V’s with finite mean and ;

• nlarge; cdf, pdf of S_n Gaussian

• ;

 ,

 standard Normal

X₁  X₂  X_n 

² S_n = X₁+X₂+ X+ _n

E X  = VAR X  = ²

E S  n_n = VAR S  n_n = ² _S

 n = n

Z_n S_n–n

--- n

=

Ex:Orders ata restaurantare i.i.d R.V.’s with= $8, = $2

P[First 100 customers spend more than $840]

• ,

P[Total spent by all customers> $1000]=90%

;

n 8 100=  =800 n² =4100 =400 P S₁₀₀840  P Z= ₁₀₀840 800– 20 

Q 2 

 = 2.2810^–2

P 780 S₁₀₀820 P= –1Z₁₀₀1  1 2Q 2–  

 = 0.682

P S₁₀₀1000  P Z= ₁₀₀1000 8n– 2 n  1000 8n–

---2 n =–1.2815

8n 1.2815– 2 n–1000= 0n129 Q x 0.1=

x=1.2815

5 - 6 Chapter 5: Sum of Random Variable and

Large-Term Average Dr. Sheng-Chou Lin

Central Limit Theorem (CDF)

n = 5 Bernoulli with p = 1/2

n = 25 Bernoulli with p = 1/2

n = 5 Exponential with p = 1/2

n = 50 Exponential with p = 1/2

Random signal and Process 輔仁大學 電機工程系所

Central Limit Theorem (PDF)

n = 5 Binomial with p = 1/2

n = 25 Binomial with p = 1/2

n = 5 Discrete uniform from the set

 5 1 2=  

=2.5

 25 1 2=  

=12.5

(0,1,2...9}

nlarge; pdf of S_n Gaussian

; ,

E X  = VAR X  = ² E S  n_n = _S

n

2 =n²

5 - 8 Chapter 5: Sum of Random Variable and

Large-Term Average Dr. Sheng-Chou Lin

Central Limit Theorem (Proof)

Z_n S_n–n

--- n 1

 n--- X_k–

k=1



n

= =

_Z

n E ew ^jwZn  E jw

 n--- X_k–

k=1



n

 

 

 

= = exp

E e^{iw Xk–  n }

k=1



n

= E e^{iw Xk– n} 

k=1



n

=

E e^{iw X– n} 

 ⁿ

= e^x 1 x x²

2! --- x³

3! --- 

+ + + +

=

E 1 jw

 n--- X– jw ² 2!n²

--- X–² R w

+ + +

1 jw

 n--- EX– jw ² 2!n²

--- E X–²E R w

+ + +

=

1 w²

2n--- E R w+ 

– 1 w–²2n

=

n 

, ,

For Gaussian R.V.

• Characteristic Function of Standard Gaussian as n goes to infinity

• Usually,n20, pdf or cdf of 

Gaussian R.V.

1+x

 ⁿ = 1+nx+1 w–²2 e^x = 1+x x = –w²2

_Z

n 1 ww ² 2n---

– 

 

 ⁿ

= e^–w22

_X ew = ^{jw m jw– 2k2 2}e^–w22

m=0²=1

_Z

nw

Z_n

Random signal and Process 輔仁大學 電機工程系所

Confidence Interval

• Sample mean estimator ,

, How good

• Sample variance estimator



Confidence Interval

• Interval

-

Confidence Interval

-

The width of a confidence interval is a measure of the accuracy

M_n E X  =

M_n 1

n--- X_j

j=1 n



= M_n

V_n² 1 n –1

--- X_j–M_n²

j=1 n



=

l X u X

 

P l X  u X   1 = – 1–

  100 %

•Case 1: unknow Mean and know variance

-

-

Confidential Interval

confidence interval for

-

^{Depends on , , ,}

P –Z M_n–n --- Z n

  = 1 2Q z– 

P M_n z

---n

–  M_n z

---n

  +

=

true mean

sample mean

 2Q Z= _{ 2}

z_{ 2} ---n

– M_n z_{ 2} ---n

 + 1–

  100 %

M_n ² m 1–

5 - 10 Chapter 5: Sum of Random Variable and

Large-Term Average Dr. Sheng-Chou Lin

Confidence Interval

Ex: ; : unknown; : random noise, Gaussian pdf with

• 95% confidence interval for if n =100, sample m Sample mean

-

^{, ,}

X = v+N v N

² = 1V v

M_n = 5.25V

 5 %=  2 = 0.025 z_{ 2} = 1.96

Confidence 5.25 1.96 1

---10

– 5.25 1.96 1

---10

 +

= 5.05 5.45

 

= 95% fall in this area

• Case 1:unknow Mean and know variance

-

-

Confidential Interval

confidence interval for

-

^{Depends on , , ,}

P –Z M_n–n --- Z n

  = 1 2Q z– 

P M_n z

---n

–  M_n z

---n

  +

=

true mean

sample mean

 2Q Z= _{ 2}

z_{ 2} ---n

– M_n z_{ 2} ---n

 + 1–

  100 %

M_n ² m 1–

 2

 2

Random signal and Process 輔仁大學 電機工程系所

Confidence Interval

•Case 1: unknow Mean and variance : iid Gaussian R.V. with unknow mean and unknown variance

W :student’s t-distribution with (n-1) degrees of freedom

X_j's 

²

P M_n z

---n

–  M_n z ---n

  +

P M_n zV_n ---n

–  M_n zV_n ---n

  +

=

sample variance

W M_n– V_nn

--- n M_n– V_n ---

= =

M_n–

   n  n 1–

 ²V_n²

  n 1–

 ^{1 2} ---

=

zero-mean unit-variance Gaussian

Chi-square R.V. with (n-1) degree of freedom

• pdfofstudent’s t-distribution

f_{n 1–}y n 2

 n 1– 2n 1–  --- 1 y²

n 1– ---

+ 

 ^{–n 2}

=

P M_n zV_n ---n

–  M_n zV_n ---n

  +

f_{n 1–}yy d

–z



z

= = 1 2F– _{n 1–} –z

Gaussian pdf Student’s pdf n=4 and n=8

5 - 12 Chapter 5: Sum of Random Variable and

Large-Term Average Dr. Sheng-Chou Lin

Confidence Interval

• Confidential Interval 1-

-

confidence interval for mean

-

^{Table 5.2:}

Ex: A certain device, lifetimeGaussian;

8 devices are tested

-

sample mean10days

-

sample variance4days

-

99% confidence interval ,

 2F= _{n 1–}–z_{ 2}n 1–  1–

  100 %

M_n z_{ 2 n 1–} V_n

--- Mn _n z_{ 2 n 1–} V_n ---n

 + –

=

z_{ 2 n 1–}

n 1– = 7 z_{ 2 7} = 3.499

Confidence Interval 10 3.499 2

---8

– 10 3.499 2 ---8

 + =7.53 12.47 