y Ex. 9 Suppose that 2 cards are dealt from a deck of 52
without replacement Let random variables X and Y be the without replacement. Let random variables X and Y be the number of aces and kings that occur. It can be shown that
48 4
4 4
51) )(48 52 ( 4 2 )
1 ( )
1
( = p =
p_{X} _{Y}
51) )( 4 52 ( 4 2 )
1 , 1
( =
p
y Since , X and Y are not independent^{p}^{(}^{1}^{,}^{1}^{)} ^{≠} ^{p}_{X} ^{(}^{1}^{)}^{p}_{Y} ^{(}^{1}^{)}
y Joint pdf of X and Y f(x,y) (continuous)
∫∫
=
∈
∈
B A
dxdy y
x f B
Y A X
P( , ) ( , ) ,
where A and B are two realnumber sets
y X and Y are independent if f(x,y)=f_{X}(x)f_{Y}(y) for all x, y where
∫
−^{∞}∞= f x y dy x
f_{X} ( ) ( , )
being the marginal pdfs of X and Y, respectively
∫
−^{∞}∞= f x y dx y
f_{Y} ( ) ( , )
being the marginal pdfs of X and Y, respectively
y Ex. 10 Suppose that X and Y are jointly continuousEx. 10 Suppose that X and Y are jointly continuous random variables with
⎨⎧24 for ≥ 0, ≥ 0 and + ≤1 )
( xy x y x y
f y Then
⎩⎨
= ⎧
otherwise
0 ) ,
,
( y y y
y x f y Then
2 1
0 1
0 24 24  12 (1 )
)
(x xydy xy x x
f_{X} =
∫
^{−}^{x} = ^{−}^{x} = − for 0≦ x ≦12 1
0 1
0 0
) 1
( 12
 24 24
)
(y xydx xy y y
f_{Y} =
∫
^{−}^{y} = ^{−}^{y} = −∫
for 0≦ y ≦1
y Since , X and Y are not independent y If X and Y are not independent, we say that they are
) ( ) ( )
,
(x y f x f y
f ≠ _{X} _{Y}
dependent.
Return 2 Return 1
y The mean or expected value of a random variable X_{i} (or μ_{i} or E[X ])
or E[X_{i} ])
⎪⎪⎨
⎧
=
∑
^{∞}=1
discrete, is
if )
( _{i}
j
j X
j p x X
x μ
⎪⎪
⎩
= ⎨
∫
−^{∞}∞1
. continuous is
if )
( _{i}
X j
X dx
x xf
μ
i
y The mean is one measure of central tendency.
y Ex. 11 For the demandsize random variable in Ex. 5, the mean is given by
mean is given by
2 5 6
4 1 3
3 1 3
2 1 6
1× 1 + × + × + × =
= μ
y Ex. 12 For the uniform random variable in Ex. 6, the mean 2
6 3
3 6
is given by
∫
^{ }^{1}^{xf} ^{(}^{x}^{)}^{dx}∫
^{ }^{1}^{xdx} ^{1}μ ^{=}
∫
0 xf (x)dx ^{=}∫
0 xdx ^{=} 2 μy Important properties of means:
E[ X] E[X]
y E[cX]=cE[X]
y even if the X_{i}’s are dependent
Wh d d ( l b )
∑
∑
^{n}_{i}= c_{i}X_{i} = ^{n}_{i}= c_{i}E X_{i}E[ _{1} ] _{1} [ ]
Where c or c_{i} denoted a constant (real number)
y The median x_{0.5} of the random variable X_{i}, which is an
l i f l d i d fi d b
alternative measure of central tendency, is defined to be the smallest value of x such that for the
discrete random variable For continuous random variable 5
. 0 )
(x ≥ FXi
discrete random variable. For continuous random variable, as shown in Fig. 4.8.
5 . 0 )
(x_{0}_{.}_{5} = FXi
The median may be a better measure of central tendency than the mean
f_{Xi}(x) when X_{i} can take on very large or
very small values.
Sh d d 0 5
Shaded area = 0.5
x x
x_{0.5}
y The variance of the random variable X_{i} denoted or Var(X ) by
2
σi
Var(X_{i}) by
].
[ ]
[ ]
)
[( ^{2} ^{2} ^{2}
2
i i
i i
i E X μ E X E X
σ = − = −
y The variance is a measure of the dispersion of a random variable about its mean.
σ^{2} small σ^{2} large
μ
μ μ
μ
y Ex. 13 For the demandsize random variable in Ex. 5, the variance is computed as follows:
variance is computed as follows:
6 43 6
4 1 3
3 1 3
2 1 6
1 1 ]
[X ^{2} = ^{2} × + ^{2} × + ^{2} × + ^{2} × = E
12 ) 11
2 (5 6
] 43 [
] [
) (
6 6
3 3
6
2 2
2 − = − =
= E X E X
X Var
y Ex. 14 For the uniform random variable in Ex. 6 the variance is computed as follows:
1
1 1
1
3 ) 1
( ]
[ ^{1}
0 1 2
0 2
2 =
∫
^{x} ^{f} ^{x} ^{dx} =∫
^{x} ^{dx} = XE
12 ) 1
2 (1 3
] 1 [ ]
[ )
(X = E X ^{2} − E^{2} X = − ^{2} = Var
y The variance has the following properties:
V ( ) ≧ 0
y Var(x) ≧ 0
y Var(cX)=c^{2}Var(X)
if h X ’ i d d
∑
∑
^{n} ^{n}y
∑ ∑
if the X_{i}’s are independent=
=
=
i
i i
i Var X
X Var
1 1
) (
) (
y The standard deviation of the random variable X_{i} is denoted to be ^{σ}_{i} ^{=} ^{Var}^{(}^{X}_{i}^{)} ^{=} ^{σ}_{i}^{2}^{.}
y E.g. For a normal random variable, the probability that X_{i} is between and is 0.95.^{μ}_{i} ^{−}^{1}^{.}^{96}^{σ}_{i} ^{μ}_{i} ^{+}^{1}^{.}^{96}^{σ}_{i}
y The covariance between the random variables X_{i} and X_{j}, which is a measure of their dependence will be denoted which is a measure of their dependence, will be denoted by C_{ij} or Cov(X_{i},X_{j}) and is defined by
j i j
i j
j i
i
ij E X μ X μ E X X μ μ
C = [( − )( − )] = [ ]−
y Note that covariances are symmetric, i.e., C_{ij }= C_{ji}. And if i=j, C_{ij} = σ_{i}^{2}.
y Ex. 15 For the joint continuous random variable X and Y in Ex 10 the covariance is computed as
in Ex. 10, the covariance is computed as
∫ ∫
^{−} ^{=}= ^{ }^{1} ^{ }^{1} 2
) (
)
(XY ^{x} xyf x y dydx
E(XY) ^{=}
∫ ∫
0 0 xyf (x, y)dydx ^{=} 15 E∫
^{=}= ^{ }^{1} 2
) ( )
(X xf x dx
E(X ) ^{=}
∫
0 xf (x)dx ^{=} 5E _{X}
∫
^{=}= ^{ }^{1} 2
) ( )
(Y yf y dy
E(Y) ^{=}
∫
0 yf (y)dy ^{=} 5E _{Y}
) 2 )(2 (2 ) 2
( ) ( )
( )
,
( −
=
−
=
−
= E XY E X E Y Y
X
Cov ) 75
)(5 (5 ) 15
( ) ( )
( )
, (
Return 1
y If C_{ij}=0, the random variable X_{i} and X_{j} are said to be uncorrelated (X and X are independent) uncorrelated. (X_{i} and X_{j} are independent)
y Fact: independence implies the uncorrelated result.
I l th i t t t f l
y In general, the converse is not true expect for normal random variable's
If C 0 th iti l l t d
y If C_{ij}>0, then positively correlated.
y If C_{ij}<0, then negatively correlated.
X_{i}>μ_{i} and X_{j}>μ_{j} tend to occur together X_{i}<μ_{i} and X_{j}<μ_{j} tend to occur together
y Correlation ρ_{ij} is defined by
2 1 and
for i j n
ρ = C^{ij} , for and =1,2,..., .
2
2 i j n
σ ρ σ
j i
ij = =
y It can be shown that 1 ≦ ρ_{ij} ≦ 1
If i l 1 h hi hl i i l l d
y If ρ_{ij} is close to +1, then highly positively correlated.
y If ρ_{ij} is close to 1, then highly negatively correlated.
y Ex. 16 For random variables in Ex. 10, it can be shown that Var(X)=Var(Y)=1/25. Therefore,
2 75
/ 2 )
, ) (
( Cov X Y − −
Y X
C 3
2 25
/ 1
75 / 2 )
( )
(
) , ) (
,
( = = σ_{ij} = =
Y Var X
Var
Y X Y Cov
X Cor
4 3 Simulation output data and 4.3 Simulation output data and stochastic processes p
y Most simulation models use random variables as input, the simulation output data are themselves random
simulation output data are themselves random.
y A stochastic process is a collection of "similar" random variables ordered over time
variables ordered over time.
y If the collection consists of finite or countably infinite
d i bl (i X X ) th h di t
random variables (i.e. X_{1}, X_{2}, …), then we have a discrete time stochastic process.
y If th ll ti i {X(t) t
≧
0} th hy If the collection is {X(t), t
≧
0}, then we have a continuoustime stochastic process.y Ex. 17 Consider a singleserver queueing system, e.g., M/M/1 queue with IID interarrival times A A IID M/M/1 queue, with IID interarrival times A_{1}, A_{2},..., IID service times S_{1}, S_{2},..., and customers served in a FIFO manner The set of all possible values that these random manner. The set of all possible values that these random variables can take on is called the state space. We can define the discretetime stochastic process of delays in p y queue D_{1}, D_{2},...as follows:
D_{1}=0 D_{1} 0
D_{i+1}=max{D_{i}+S_{i}A_{i+1},0} for i=1,2,…
y Thus, the simulation maps the input random variables into the output stochastic process D D of interest
the output stochastic process D_{1}, D_{2},... of interest.
y The state space is the set of nonnegative real numbers.
D d D i i l l d
y D_{i} and D_{i+1}, are positively correlated.
y A discretetime stochastic process X_{1}, X_{2},...is said to be covariance stationary if
covariancestationary if
∞
<
<
∞
=
=
2 2
2 f 12 d
and 2
1 for
i
μ 
,...
, i
μ μ_{i}
and C_{i,i+j}=Cov(X_{i}, X_{i+j}) is independent of i.
∞
<
=
= ^{2} ^{2}
2 σ for i 1,2,...and σ σ_{i}
y For a covariancestationary process, we denote the
covariance and correlation between X_{i} and X_{i+j} by C_{j} and ρ_{j}, where
,...
2 , 1 , 0 for
2 ,
2 2
, = = =
= ^{+} j
C C σ
C
σ _{j} C^{i} ^{i} ^{j} ^{j} ^{j}
2 0 2
+ σ C
σ σ_{i} _{i} _{j}
j
y Ex. 18, Consider the output process D_{1}, D_{2},...for a
covariance stationary M/M/1 queue with ρ= λ / ω<1 (λ:
covariancestationary M/M/1 queue with ρ= λ / ω<1. (λ:
arrival rate, ω: service rate) See Fig. 4.10, note that the correlation ρ are positive and monotonically decrease to correlation ρ_{j} are positive and monotonically decrease to zero as j increases.
y In general our experiment indicates that output processes y In general, our experiment indicates that output processes
for queueing systems are positive correlated.
y Note that if X_{1}, X_{2},...is a stochastic process beginning at time 0 in a simulation then it is quite likely not to be time 0 in a simulation, then it is quite likely not to be
covariancestationary. However, for some simulation X_{k+1}, X_{k 2} will be approximately covariancestationary if k is X_{k+2},...will be approximately covariance stationary if k is large enough, where k is the length of the warmup period.
4 4 Estimation of means 4.4 Estimation of means, variances, and correlations ,
y Suppose that X_{1}, X_{2},..., X_{n} are IID random variables with finite population mean μ and finite population variance σ^{2} finite population mean μ and finite population variance σ^{2} y Sample mean
n X
∑
n n X
X ( ) =
∑
^{i}=^{1} ^{i}which is an unbiased estimator of μ, i.e., μ
n X
E[[ (( )])] = μ
y Sample variance
∑
^{n} 21
)]
( ) [
( ^{1}
2 2
−
=
∑
= −n
n X n X
S
n
i i
which is an unbiased estimator of σ^{2} since
2 2( )]
[S n σ E[ ( )] =
y Since Var X X n
Var n n
X
Var ^{n}
i
i n
i
i
2 1 1
) 1 (
1 ) ( )]
(
[ =
∑
=∑
=
=
n σ σ
n n X
n Var
n
i
i
i i
2 2
2 1
2
1 1
) 1 1 (
=
∑
= =it is clear that the bigger the sample size n, the closer should be to μ And the unbiased estimator of
n n
n _{i}_{=}_{1}
) (n X
)]
( [X n should be to μ. And the unbiased estimator of Var is obtained by replacing σ^{2 }of above Eq., resulting in
)]
( [X n Var
)]
(
[ ^{2}
∑
^{n} ^{X} ^{X}) 1 (
)]
( ) [
)] ( (
[ ^{1}
2 2
−
−
=
=
∑
=
n n
n X X
n n n S
X
Var ^{i}
i
y If the X_{i}’s are independent, ρ_{j}=0 for j=1,2,…,n1.
) (
y It has been our experience that simulation output data are almost always correlated
almost always correlated.
y Now, assume that the random variables X_{1}, X_{2},..., X_{n} are from a covariance stationary stochastic process
from a covariancestationary stochastic process.
y Then the sample mean is still an unbiased estimator.
y But the sample variance is no longer an unbiased estimator
y It can be shown that
) ] / 1
2 ( 1 [ )]
( [
1 2 1
2 −
×
=
∑
^{−}= j n ρρ n
S E
n
j j
1 ] 2
1 [ )]
(
[ = − × −
ρ n n
S E
y If ρ_{j} > 0, then , which will lead to serious errors in analysis
2 2( )]
[S n σ E <
errors in analysis
y As for , it can be shown (Prob. 4.17) that _{Var}_{[}_{X} _{(}_{n}_{)]}
1
n
ρ n σ j
n X Var
n
j (1 / ) j ]
2 1 )] [
( [
1
2 +
∑
^{−}=1 −=
y One of estimators of ρ_{j} is ρ = Cˆ ^{j}
ˆ
n X X
n X X
n ρ S
j n
j i i
j
−
−
=
∑
^{−}2
)]
( )][
( ˆ [
) (
j n
n X X
n X
C_{j} ^{i} X^{i} ^{i} ^{j}
=
∑
=1 [ ( −)][ + ( )]ˆ
y We shall see in Chap. 9 that it is often possible to group simulation output data into new "observations" to which simulation output data into new observations to which the formulas on IID observations can be applied.
4 5 Confidence intervals and 4.5 Confidence intervals and hypothesis tests for the mean yp
y Let X_{1}, X_{2},...,X_{n} be IID random variables with finite mean μ and finite variance σ^{2}
μ and finite variance σ^{2}.
y Let Z_{n} be the random variable and F_{n}(z) be the distribution function of Z (i e F (z)=P(Z ≦z))
n σ
μ n
X ( ) ]/ /
[ − ^{2}
be the distribution function of Z_{n }(i.e. F_{n}(z)=P(Z_{n }≦z)).
y Note:
y Mean: μ
y Variance of : σX (n) ^{2} / n
Central limit theorem
y TH. 1 F_{n}(z) → Φ(z) as , where Φ(z) is the
distribution function of a normal random variable with
∞
→ n
distribution function of a normal random variable with μ=0 and σ^{2 }=1.
1
∫
−∞− −∞ < < ∞
=
Φ ^{z} e ^{y} dy z
z π
2
/ for 2
) 1
( ^{2}
y If n is “sufficiently” large, the random number Z_{n} will be approximately X_{i}’s.
y The difficulty is that σ2^{2} is generally unknown.
y However, since the sample variance S^{2}(n) converges σ^{2 }as n gets large. TH. 1 remains true if we replace σ^{2 }by S^{2}(n).
y If the X_{i}'s are normal random variables, then random
variable _{S}^{2}_{(} _{)}
variable
h di ib i i h 1 d f f d A d
n n μ S
n X
t_{n} ( )
/ ] )
( [
− 2
=
has a t distribution with n1 degrees of freedom. And an exact 100(1 α) percent confidence interval for μ is given bby
n n t S
n
X _{n} _{α} ( )
)
( ± _{−}_{1}_{,}_{1}_{−} _{/}_{2} 0< α <1
y In practice, the distribution of the X_{i}'s will rarely by
normal, and the above equation will also be approximate.
f(x)
Shaded area = 1 α
0 z_{1 α/2} x
z_{1 α/2}
y If n is sufficiently large, an approximate 100(1 α) percent confidence interval for μ is given by
confidence interval for μ is given by n
z S n
X _{α} ( )
) (
2 2
/ 1−
±
y If one constructs a very large number of independent
α n )
( _{1} _{/}_{2}
100(1 α) percent confidence intervals, the proportion of these confidence intervals that contain μ should be 1 α.
y Ex. 19 Suppose that the 10 observations 1.2, 1.5, 1.68, 1 89 0 95 1 49 1 55 0 5 and 1 09 are from a normal 1.89, 0.95, 1.49, 1.55, 0.5, and 1.09 are from a normal distribution with unknown mean μ and that our objective is to construct a 90 percent confidence interval for μ Note is to construct a 90 percent confidence interval for μ. Note that
17 0 9
/ ] ) 34 1 09 1 ( )
34 1 2 1 [(
) 10 (
34 1 10
) 09 1 5
1 2 1 ( ) 10 (
2 2
2 + +
= +
+ +
= S
. /
. ...
. .
X
( )
^{10} _{1} _{34} _{1} _{83} ^{0}^{.}^{17} _{1} _{34} _{0} _{24}) 10 (
17 . 0 9
/ ] ) 34 . 1 09 . 1 ( ...
) 34 . 1 2 . 1 [(
) 10 (
2 95
0 9
2 2
2
±
=
±
=
±
=
− +
+
−
= t S X
S
Therefore, we claim with 90 percent confidence that μ is i th i t l [1 10 1 58]
24 . 0 34 . 10 1
83 . 1 34 . 10 1
) 10
( ± t_{9}_{,}_{0}_{.}_{95} ± ±
X
in the interval [1.10 1.58].
y Assume that X_{1}, X_{2},...,X_{n} are normally distributed, we would like to test the null hypothesis H that μ= μ
would like to test the null hypothesis H_{0} that μ= μ_{0}
L ^{S}^{2}^{(}^{n}^{)} h f f h h i
y Let , the form of our hypothesis t t (t t t) f i
n n μ S
n X
t_{n} ( )
/ ] )
(
[ − _{0}
=
test (t test) for μ= μ_{0} is
⎧> t reject H
⎩⎨
⎧
≤
>
−
−
−
−
0 2
/ 1 . 1
0 2
/ 1 . 1
accept"
"
reject


If t H
H t t
α n
α n
n
y The set of all x such that is called the critical region
2 / 1 ,
 1
 x > t_{n}_{−} _{−}_{α}
region.
y The probability that the statistic t_{n} falls in the critical region given H is true is called the level of the test region given H_{0} is true is called the level of the test.
y Typical value of level: 0.05 or 0.1. (for experiment) y Two types of errors can be made:
y Type I error: reject H_{0} when it is indeed true.
y Type II error: accept H_{0} when it is false.
y Ex. 20 For the data of the previous example, suppose that we would like to test the null hypothesis H that μ =1 at we would like to test the null hypothesis H_{0} that μ =1 at level α =0.1. Since
34 0 1
) 10 ( X
95 . 0 , 2 9
10 2.65 1.83
10 / 17 . 0
34 . 0 10
/ ) 10 (
1 ) 10
( t
S
t X − = = > =
= y we reject H_{0}.
4 6 The strong law of large 4.6 The strong law of large numbers
y TH. 2
X (n) → μ w p 1 n → ∞ X ( ) . . 1 as
4 7 The danger of replacing a 4.7 The danger of replacing a
probability distribution by its mean probability distribution by its mean
y One should not use the mean to replace the input distribution for the sake of simplicity
distribution for the sake of simplicity.