Chapter 3. Renewal Processes
Prof. Ai-Chun Pang
Graduate Institute of Networking and Multimedia,
Department of Computer Science and Information Engineering,
National Taiwan University, Taiwan
Renewal Processes
} 0 ),
~(
{ ≥
= n t t N
} 0 ),
~ (
{ ′ ≥
′= n t t N
} 0 ),
~(
{ ≥
= n t t N
)
~ t( . n
exp
! ) ) (
)
~ (
( k
t k e
t n P
k
t λ
λ
= −
=
0
t
? )
~ (
lim =
∞
→ n t
t
) ?
~ (
lim =
∞
→ t
t n
t
0 t
1
S~ ~2
S G
0 t
? )
)
~ (
( n t = k =
P
)
~ t( n G
Outline
• Distribution and Limiting Behavior of ˜n(t) – Pmf of ˜n(t) : P (˜n(t) = k) =?
– Limiting time average : lim
t→∞
n(t)˜
t =? (Law of Large Numbers) – Limiting PDF of ˜n(t) (Central Limit Theorem)
• Renewal Function E[˜n(t)], and its Asymptotic (Limiting) behavior – Renewal Equation
– Wald’s Theorem and Stopping time – Elementary Renewal Theorem
– Blackwell’s Theorem
Outline
• Key Renewal Theorem and Applications – Definition of Regenerative Process – Renewal Theory
– Key Renewal Theorem
– Application 1: Residual Life, Age, and Total Life – Application 2: Alternating Renewal Process/Theory – Application 3: Mean Residual Life
• Renewal Reward Processes and Applications – Renewal Reward Process/Theory
– Application 1: Alternating Renewal Process/Theory – Application 2: Time Average of Residual Life and Age
• More Notes on Regenerative Processes
Distribution and Limiting Behavior of ˜n(t)
1
S~ ~2
S ~3
S ~~()1
+ t
Sn t
)
~ t( n
1
~x ~x2 ~x3 ~ tn()
•
)
~(
~
t
Sn
{˜xn, n = 1, 2, . . .} ∼ F˜x; mean ¯X (0 < ¯X < ∞)
N = {˜n(t), t ≥ 0} is called a renewal (counting) process n(t) = max{n : ˜˜ Sn ≤ t}
Distribution and Limiting Behavior of ˜n(t)
n(t)˜
1. pmf of ˜n(t) → closed-form
2. Limiting time average [Law of Large Numbers]:
n(t)˜ t
w.p.1
→ 1
X¯ , t → ∞ 3. Limiting time and ensemble average
[Elementary Renewal Theorem]:
E[˜n(t)]
t
w.p.1
→ 1
X¯ , t → ∞
Distribution and Limiting Behavior of ˜n(t)
4. Limiting ensemble average (focusing on arrivals in the vicinity of t ) [Blackwell’s Theorem]:
E[˜n(t + δ) − ˜n(t)]
δ
w.p.1
→ 1
X¯ , t → ∞ 5. Limiting PDF of ˜n(t) [Central Limit Theorem]:
t→∞lim P
n(t) − t/ ¯˜ X σ√
t( ¯X)−3/2 < y
=
y
−∞
√1
2πe−x22 dx ∼ Gaussian( t
X¯ , σ√
t· ¯X−32)
pmf of ˜n(t)
P [˜n(t) = n] = P [˜n(t) ≥ n] − P [˜n(t) ≥ n + 1]
= P [ ˜Sn ≤ t] − P [ ˜Sn+1 ≤ t]
··· x˜i ∼ F,
··
·
x˜i ∼ F (t) ⊗ F (t) . . . ⊗ F (t) ≡ Fn(t)
= Fn(t) − Fn+1(t) n-fold convolution of F (t)
Limiting Time Average
t→∞lim n(t) =?˜
··· P
t→∞lim n(t) < ∞˜
= P [˜n(∞) < ∞] = P [˜xn = ∞ for some n]
= P
∞
n=1
(˜xn = ∞)
=
∞ n=1
P [˜xn = ∞] = 0
··
· lim
t→∞ n(t) = ˜˜ n(∞) = ∞ w.p.1
Question: What is the rate at which ˜n(t) goes to ∞ ?
)
~ t( n
?
) ?
~( lim→∞ =
t t n
i.e. t
Strong Law for Renewal Processes
Theorem. For a renewal process N = {˜n(t), t ≥ 0} with mean inter- renewal interval ¯X, then
t→∞lim
n(t)˜
t = 1
X¯ , w.p.1 Proof.
)
~ t( n
)
~(
~
t
Sn t ~( ) 1
~
+ t
Sn
t
)
~ t( n
Strong Law for Renewal Processes
··· S˜˜n(t) ≤ t < ˜S˜n(t)+1
⇒ S˜˜n(t)
n(t)˜ ≤ t
n(t)˜ <
S˜˜n(t)+1 n(t)˜ =
S˜˜n(t)+1
n(t) + 1˜ × n(t) + 1˜ n(t)˜
⇒ lim
t→∞
S˜˜n(t) n(t)˜
= ¯X why?
≤ lim
t→∞
t
n(t)˜ < limt→∞
⎡
⎢⎢
⎢⎣
S˜˜n(t)+1 n(t) + 1˜
= ¯X
× n(t) + 1˜ n(t)˜
=1
⎤
⎥⎥
⎥⎦
··
· lim
t→∞
n(t)˜
t = 1 X¯
Renewal Function E[˜n(t)]
Let m(t) = E[˜n(t)], which is called “renewal function”.
1. Relationship between m(t) and Fn
m(t) =
∞ n=1
Fn(t), where Fn is the n-fold convolution of F 2. Relationship between m(t) and F
[Renewal Equation]
m(t) = F (t) +
t
0 m(t − x)dF (x)
3. Relationship between m(t) and L˜x(r) (Laplace Transform of ˜x) Lm(r) = L˜x(r)
r[1 − L˜x(r)]
Renewal Function E[˜n(t)]
→ [Wald’s Equation]
4. Asymptotic behavior of m(t) (t → ∞, Limiting)
→ [Elementary Renewal Theorem]
→ [Blackwell’s Theorem]
Renewal Function E[˜n(t)]
1. m(t) = E[˜n(t)] ←→ F? n (i.e., PDF of ˜Sn)
Let ˜n(t) =
∞ n=1
In, where In =
⎧⎨
⎩
1, nth renewal occurs in [0, t];
0, Otherwise;
m(t) = E[˜n(t)] = E
∞
n=1
In
=
∞ n=1
E[In]
=
∞ n=1
P [nth renewal occurs in [0, t]]
=
∞ n=1
P [ ˜Sn ≤ t]
Renewal Function E[˜n(t)]
··
· m(t) =
∞ n=1
Fn(t)
or m(t) =
∞ n=1
P [˜n(t) ≥ n] =
∞ n=1
P [ ˜Sn ≤ t] = ∞
n=1
Fn(t)
. . . .
As t → ∞, n → ∞, finding Fn is far too complicated
⇒ find another way of solving m(t) in terms of F˜x(t)
Renewal Function E[˜n(t)]
2. m(t) ←→ F? ˜x(t) (i.e., PDF of ˜x)
··· S˜n = ˜Sn−1 + ˜xn, for all n ≥ 1, and ˜Sn−1 and ˜xn are independent,
··
· P [ ˜Sn ≤ t] = t
0 P [ ˜Sn−1 ≤ t − x]dF˜x(x), for n ≥ 2 for n = 1, ˜x1 = ˜S1, P [ ˜S1 ≤ t] = F˜x(x)
··
· m(t) =
∞ n=1
P [ ˜Sn ≤ t] = F˜x(t) +
t
0
∞ n=2
P [ ˜Sn−1 ≤ t − x]dF˜x(x) m(t) = F˜x(t) +
t
0 m(t − x) · dF˜x(x) ⇒ Renewal Equation
Renewal Function E[˜n(t)]
3. Lm(r) ←→ L? ˜x(r) (Laplace Transform of ˜x)
(Laplace Transform of m(t) = Lm(r)) Answer:
Lm(r) = L˜x(r) r[1 − L˜x(r)]
<Homework> Prove it.
4. Asymptotic behavior of m(t):
t→∞lim
m(t)
t = lim
t→∞
E[˜n(t)]
t =?
Stopping Time (Rule)
Definition. ˜N , an integer-valued r.v., is said to be a “stopping time” for a set of independent random variables ˜x1, ˜x2, . . . if event { ˜N = n} is independent of ˜xn+1, ˜xn+2, . . .
Example 1.
• Let ˜x1, ˜x2, . . . be independent random variables,
• P [˜xn = 0] = P [˜xn = 1] = 1/2, n = 1, 2, . . .
• if ˜N = min{n : ˜x1 + . . . + ˜xn = 10}
→ Is ˜N a stopping time for ˜x1, ˜x2, . . .?
Stopping Time (Rule)
Example 2.
• ˜n(t), X = {˜xn, n = 1, 2, 3, . . .},
• S = { ˜Sn, n = 0, 1, 2, 3, . . .},
• ˜Sn = ˜Sn−1 + ˜xn
1
~x ~x2 ~x3
4
~x ~x5
1
S~ ~2
S 3
S~ ~4
S 5
S~
t
)
~ t( n
→ Is ˜n(t) the stopping time of X = {˜xn, n = 1, 2, . . .}?
Stopping Time (Rule)
Example 3. Is ˜n(t) + 1 the stopping time for {˜xn}?
Answer:
whether ˜n(t) + 1 = n (→ ˜n(t) = n − 1) depends on ˜Sn−1 ≤ t < ˜Sn
··
· depends on ˜Sn−1 and ˜Sn, i.e., up to ˜xn
··
· n(t) + 1 is the stopping time for {˜˜ xn}, so is ˜n(t) + 2, ˜n(t) + 3, . . .
Stopping Time - from ˜I
nDefinition. ˜N , an integer-valued r.v. is said to be a stopping time for a set of independent random variables {˜xn, n ≥ 1}, if for each n > 1, ˜In, conditional on ˜x1, ˜x2, . . . , ˜xn−1, is independent of {˜xk, k ≥ n}
Define. ˜In - a decision rule for stopping time ˜N , n ≥ 1 I˜n =
⎧⎨
⎩
1, if the nth observation is to be made;
0, Otherwise
1. ··· N is the stopping time˜
··
· I˜n depends on ˜x1, . . . , ˜xn−1 but not ˜xn, ˜xn+1, . . . 2. ˜In is also an indicator function of event { ˜N ≥ n},
i.e., ˜In =
⎧⎨ 1, if ˜N ≥ n;
Stopping Time - from ˜I
nBecause
• If ˜N ≥ n, then nth observation must be made;
• Since ˜N ≥ n implies ˜N ≥ n − 1 and happily, ˜In = 1 implies I˜n−1 = 1
··
· Stopping time
⎧⎪
⎪⎪
⎨
⎪⎪
⎪⎩
{ ˜N = n}, is independent of ˜xn+1, ˜xn+2, . . . or
I˜n is independent of ˜xn, ˜xn+1, . . .
Wald’s Equation
Theorem. If {˜xn, n ≥ 1} are i.i.d. random variables with finite mean E[˜x], and if ˜N is the stopping time for {˜xn, n ≥ 1}, such that
E[ ˜N ] < ∞. Then,
E
⎡
⎣N˜
n=1
x˜n
⎤
⎦ = E[ ˜N ] · E[˜x]
Proof. Let ˜In =
⎧⎨
⎩
1, if n ≤ ˜N ; 0, otherwise;
E
⎡
⎣
N˜
n=1
x˜n
⎤
⎦ = E
∞
n=1
x˜n · ˜In
=
∞
E[˜xn · ˜In] =
∞
E[˜xn] · E[˜In] (why? stopping time)
Wald’s Equation
= E[˜x]
∞ n=1
E[ ˜In] = E[˜x]
∞ n=1
P ( ˜N ≥ n)
= E[˜x]E[ ˜N ]
. . . .
For Wald’s Theorem to be applied, other than {˜xi, i ≥ 1}
1. ˜N must be a stopping time; and 2. E[ ˜N ] < ∞
Wald’s Equation
Example. (Example 3.2.3 – Simple Random Walk, [Kao])
2 1
{˜xi} i.i.d. with: P (˜x = 1) = p
P (˜x = −1) = 1 − p = q S˜n =
n
x˜k
Wald’s Equation
• Let ˜N = min{n| ˜Sn = 1}
→ ˜N is the stopping time
E[ ˜SN˜] = E[ ˜N ] · E[˜x] = E[ ˜N ] · (p − q)
··· S˜N˜ = 1 for all ˜N
··
· E[ ˜SN˜] = 1
– if p = q, E[ ˜N ] = ∞ ⇒ Wald’s Theorem not applicable – if p > q, E[ ˜N ] < ∞ ⇒ E[ ˜N ] = p−q1
– if p < q, E[ ˜N ] = ∞ ⇒ Wald’s Theorem not applicable
Wald’s Equation
• Let ˜M = min{n| ˜Sn = 1} − 1
··
· S˜M˜ = 0 → E[ ˜SM˜ ] = 0 assume E[ ˜N ] < ∞, p > q, ··
· E[ ˜M ] < ∞ but E[ ˜SM˜ ]
=0
= E[ ˜ M ] finite
(p − q)
Why?
Corollary
Before proving lim
t→∞
m(t)
t → 1
X¯ ,
Corollary. If ¯X < ∞, then
E[ ˜S˜n(t)+1] = ¯X[m(t) + 1]
Proof.
E[ ˜S˜n(t)+1] = E
⎡
⎣˜n(t)+1
n=1
x˜n
⎤
⎦ = ¯X · E[˜n(t) + 1] = ¯X · [m(t) + 1]
Why?
The Elementary Renewal Theorem
Theorem.
m(t)
t → 1
X¯ as t → ∞ Proof.
To prove 1
X¯ ≤ lim
t→∞ inf m(t)
t
1
≤ lim
t→∞sup m(t)
t ≤ 1 X¯
2
1. ··· S˜˜n(t)+1 > t ··
· from Cor., ¯X[m(t) + 1] > t m(t)
t ≥ 1
X¯ − 1
t ·
·· lim
t→∞inf m(t)
t ≥ 1 X¯
The Elementary Renewal Theorem
2. Consider a truncated renewal process
x˜n =
⎧⎨
⎩
x˜n, If ˜xn ≤ M; n = 1, 2, . . . M, otherwise
Let ˜Sn = x˜n, and ˜N(t) = sup{n : ˜Sn ≤ t}. We have that S˜ ˜N(t)+1 ≤ t + M
From the corollary,
[m(t) + 1] ¯X ≤ t + M, where ¯X = E[ ˜xn]
··
· lim
t→∞ sup m(t)
t ≤ 1
X¯
The Elementary Renewal Theorem
But since ˜Sn ≤ ˜Sn → N˜(t) ≥ ˜N (t), m(t) ≥ m(t)
··
· lim
t→∞ sup m(t)
t ≤ 1 X¯
Let M → ∞, ¯X → ¯X
··
· lim
t→∞ sup m(t)
t ≤ 1 X¯
Blackwell’s Theorem
• Ensemble Average.
– to determine the expected renewal rate in the limit of large t, without averaging from 0 → t (time average)
• Question.
– are there some values of t at which renewals are more likely than others for large t ?
∞
+4
t∞ t∞+8 t∞ +12
t
– An example. If each inter-renewal interval {˜xi, i = 1, 2, . . .} takes on integer number of time units, e.g., 0, 4, 8, 12, . . . , then
expected rate of renewals is zero at other times. Such random variable is said to be “lattice”.
Blackwell’s Theorem
– Definitions.
∗ A nonnegative random variable ˜x is said to be lattice if there exists d ≥ 0 such that
∞ n=0
P [˜x = nd] = 1
∗ That is, ˜x is lattice if it only takes on integral multiples of some nonnegative number d. The largest d having this property is said to be the period of ˜x. If ˜x is lattice and F is the distribution
function of ˜x, then we say that F is lattice.
• Answer.
– Inter-renewal interval random variables are not lattice
⇒ uniform expected rate of renewals in the limit of large t.
Blackwell’s Theorem
Theorem. If, for {˜xi, i ≥ 1}, which are not lattice, then, for any δ > 0,
t→∞lim [m(t + δ) − m(t)] = δ X¯ Proof. (omitted)
For non-lattice inter-renewal process {˜xi, i ≥ 1},
1. ··· x˜i > 0 ⇒ No multiple renewals (single arrival)
2. From Blackwell’s Theorem, the probability of a renewal in a small interval (t, t + δ] tends to δ/ ¯X + o(δ) as t → ∞,
··
· Limiting distribution of renewals in (t, t + δ] satisfies
t→∞lim P [˜n(t + δ) − ˜n(t) = 1] = δ
X¯ + o(δ)
Blackwell’s Theorem
t→∞lim P [˜n(t + δ) − ˜n(t) = 0] = 1 − δ
X¯ + o(δ)
t→∞lim P [˜n(t + δ) − ˜n(t) ≥ 2] = o(δ)
Blackwell’s Theorem
⇒
single arrival Stationary Independent Increment Increment
Poisson yes yes yes
Renewal yes yes no
Process
(Non-lattice)