• 沒有找到結果。

# Monte Carlo Integration

N/A
N/A
Protected

Share "Monte Carlo Integration"

Copied!
64
0
0

(1)

### Monte Carlo Integration

Digital Image Synthesisg g y Yung-Yu Chuang

with slides by Pat Hanrahan and Torsten Moller

(2)

### Introduction

• The integral equations generally don’t have

analytic solutions so we must turn to numerical analytic solutions, so we must turn to numerical methods.

St d d th d lik T id l i t ti

• Standard methods like Trapezoidal integration or Gaussian quadrature are not effective for

hi h di i l d di ti i t l

high-dimensional and discontinuous integrals.

o

o

e

o

i i

i

i

i

2

o

i

i

i

i

i

s

### 

(3)

• Suppose we want to calculate , but can’t solve it analytically The approximations I

### 

ab f (x)dx

can t solve it analytically. The approximations through quadrature rules have the form

### 

n

i

i

i f x

w I

1

) ˆ (

which is essentially the weighted sum of samples of the function at various pointsp p

(4)

### Midpoint rule

convergence convergence

(5)

### Trapezoid rule

convergence convergence

(6)

### Simpson’s rule

• Similar to trapezoid but using a quadratic polynomial approximation

polynomial approximation

convergence convergence

assuming f has a continuous fourth derivative.

(7)

### Curse of dimensionality and discontinuity

• For an sd f i f function f,

• If the 1d rule has a convergence rate of O(n-r), the sd rule would require a much larger number (ns) of samples to work as well as the 1d one.

Thus, the convergence rate is only O(n-r/s).

• If f is discontinuous, convergence is O(n, g ( -1/s) for ) sd.

(8)

### Randomized algorithms

• Las Vegas v.s. Monte Carlo

L V l i h i h b

• Las Vegas: always gives the right answer by using randomness.

• Monte Carlo: gives the right answer on the average. Results depend on random numbers used, but statistically likely to be close to the right answer.

(9)

### Monte Carlo integration

• Monte Carlo integration: uses sampling to estimate the values of integrals It only estimate the values of integrals. It only

requires to be able to evaluate the integrand at arbitrary points making it easy to implement

arbitrary points, making it easy to implement and applicable to many problems.

If l d it t th t

• If n samples are used, its converges at the rate of O(n-1/2). That is, to cut the error in half, it is

t l t f ti

necessary to evaluate four times as many samples.

• Images by Monte Carlo methods are often noisy.

Most current methods try to reduce noise.

(10)

### Monte Carlo methods

E t i l t

– Easy to implement

– Easy to think about (but be careful of statistical bias)

R b t h d ith l i t d d

– Robust when used with complex integrands and domains (shapes, lights, …)

Efficient for high dimensional integrals – Efficient for high dimensional integrals

– Noisy

– Slow (many samples needed for convergence)

(11)

### Basic concepts

• X is a random variable

A l i f i d i bl i

• Applying a function to a random variable gives another random variable, Y=f(X).

• CDF (cumulative distribution function) }

Pr{

)

(x X x

P  

• PDF (probability density function): nonnegative, sum to 1 dP )(x

} {

) (

sum to 1

i l if d i bl ξ ( id d dx

x x dP

p ( )

)

( 

• canonical uniform random variable ξ (provided by standard library and easy to transform to

th di t ib ti ) other distributions)

(12)

### Discrete probability distributions

• Discrete events

### X

i with probability

i

n

i

1

n

i i

### p

i

• Cumulative PDF (distribution)

i

j

P

p

### 1

• Construction of samples:

1

j i

i

P p

###  

• Construction of samples: U

To randomly select an event, Select

i if

i1

i

i

### 0

Uniform random variable

3

(13)

• PDF p x

Uniform

• CDF

x 1

0

P x

p x dx

P x

X

x

X

d

0

X

p x dx

0 1

P

P

0 1

(14)

### Expected values

• Average value of a function f(x) over some distribution of values p(x) over its domain D distribution of values p(x) over its domain D

f x

### 

f x p x dx

E ( ) ( ) ( )

• Example: cos function over [0, π], p is uniform

D

p f x f x p x dx

E ( ) ( ) ( )

p [ , ], p

cos(x)

cos x 1 dx 0

E p

cos(x)

0 cos x dx 0

(15)

### Variance

• Expected deviation from the expected value

F d l f if i h

• Fundamental concept of quantifying the error in Monte Carlo methods

( ) ( ) 2

## 

)]

(

[ f x E f x E f x

V  

(16)

af (x)

aE

f (x)

E

 

i

i E f X

X f

E

( )

( )

 

i i

af (x)

a2V

f (x)

V

af (x)

a V

f (x)

V

f (x)

E

f (x)

2

E

f (x)

2

V

f (x)

E

##  

f (x)

E

f (x)

V

(17)

### Monte Carlo estimator

• Assume that we want to evaluate the integral of evaluate the integral of f(x) over [a,b]

Gi if d

• Given a uniform random variable Xi over [a,b],

M t C l ti t Monte Carlo estimator

### 

N

X a f

F b ( )

says that the expected

### 

i

i

N f X

F N

1

) (

says that the expected value E[FN] of the

estimator FN equals the estimator FN equals the integral

(18)

### General Monte Carlo estimator

• Given a random variable X drawn from an arbitrary PDF p(x) then the estimator is arbitrary PDF p(x), then the estimator is

N f Xi

F 1 ( )

### 

i i

N N p X

F

1 ( )

• Although the converge rate of MC estimator is O(N( 1/2),) slower than other integral methods, its converge rate is independent of the dimension, making it the only practical method for high

dimensional integral

(19)

### Convergence of Monte Carlo

• Chebyshev’s inequality: let X be a random

variable with expected value μ and variance σ2 variable with expected value μ and variance σ2. For any real number k>0,

2

} 1

|

Pr{| X

k

### 

k

• For example, for k= , it shows that at least

half of the value lie in the interval 2 ( 2, 2)

• Let , the MC estimate FYi f (Xi)/ p(Xi) N becomes

1 N

) ,

(

i

i

N Y

F N

1

1

(20)

### Convergence of Monte Carlo

• According to Chebyshev’s inequality,



  1









 

 

 

 [ ] 2

| ] [

|

Pr N N V FN

F E F

V

###  

Y

Y N N V

Y N V

N Y V F

V N 1 N i 1 N i 1 N i 1

]

[

2

2

### 



  

Plugging into Chebyshev’s inequality

N N

N

N i i i i i i

N ] [

1 2 1

2

1

### 

• Plugging into Chebyshev’s inequality,

### 

 





     2

] 1

[

| 1

|

Pr V Y

I FN

So, for a fixed threshold, the error decreases at



 

 

 

|

|

Pr FN I N

, ,

the rate N-1/2.

(21)

### Properties of estimators

• An estimator FN is called unbiased if for all N That is, the expected value is independent of N.

Q F

E[ N ] 

• Otherwise, the bias of the estimator is defined

as

### 

• If the bias goes to zero as N increases the Q

F E

FN ]  [ N ]

### 

[

• If the bias goes to zero as N increases, the estimator is called consistent

0 ]

[

li

[F ] 0

lim 

N

N

### 

F

Q F

E[ ]  lim E FN Q

N

[ ]

lim

(22)

### Example of a biased consistent estimator

• Suppose we are doing antialiasing on a 1d pixel, to determine the pixel value we need to

to determine the pixel value, we need to evaluate , where is the filter function withI

### 

01w(x) f (x)dx w(x)

### 

1 ( d) 1

filter function with

• A common way to evaluate this is

0 w( dxx) 1

N

###  

N

i N

i i i

N w X

X f

X F 1w

) (

) (

) (

• When N=1, we have

i1w(Xi ) X

f

X

( ) ( )

I dx

x f X

f X E

w

X f

X E w

F

E

1

01

1

1

1 1 [ ( )] ( )

) (

) (

) ] (

[

(23)

### Example of a biased consistent estimator

• When N=2, we have

I dx

x dx w

x w

x f x w x

f x F w

E

###  

01

1

0 1 2

2 1

2 2

1 2 1

) (

) (

) (

) (

) ( ) ] (

[

• However, when N is very large, the bias approaches to zeropp

N

i i i

N

X f X N w

F 1

1

) ( ) 1 (

N

i i

N w X

N1 1 ( ) X

f

N X

( ) ( )

1

li 1

I dx x f x w dx

x w

dx x f x w X

N w

X f X N w

F

E N

i i

i i i

N

N N

1 1 0

0 1 0

1

1 ( ) ( )

) (

) ( ) ( )

1 ( lim

) ( ) ( lim

] [ lim

N i i

N

1 ( )

0

(24)

FN N1

### 

N f ((XXi))

• Carefully choosing the PDF from which samples are drawn is an important technique to reduce

### 

i p Xi

N 1 ( )

are drawn is an important technique to reduce variance. We want the f/p to have a low

variance Hence it is necessary to be able to variance. Hence, it is necessary to be able to draw samples from the chosen PDF.

• How to sample an arbitrary distribution from a

• How to sample an arbitrary distribution from a variable of uniform distribution?

– Inversionve s o – Rejection – Transform

(25)

### Inversion method

• Cumulative probability distribution function

### ( ) Pr( ) P x  X  x

• Construction of samples

Solve for X=P-1(U)

1

Solve for X P (U)

• Must know:

### U

• Must know:

1. The integral of p(x)

2. The inverse function P-11(x)

0

(26)

### Proof for the inversion method

• Let U be an uniform random variable and its CDF is P (x) x We will show that Y P-1(U) has CDF is Pu(x)=x. We will show that Y=P-1(U) has the CDF P(x).

(27)

### Proof for the inversion method

• Let U be an uniform random variable and its CDF is P (x) x We will show that Y P-1(U) has CDF is Pu(x)=x. We will show that Y=P-1(U) has the CDF P(x).

Pr

( )

Pr

( )

### 

( ( )) ( )

Pr Y x P1 U x U P x Pu P x P x

because P is monotonic,

2 1

2 1) ( ) if

(x P x x x

P

Thus, Y’s CDF is exactly P(x).

2 1

2

1) ( )

(

(28)

### Inversion method

• Compute CDF P(x)

• Compute P-1(x)

• Obtain ξξ

• Compute Xi=P-1(ξ)

(29)

### Example: power function

It is used in sampling Blinn’s microfacet model.

nn

(30)

### Example: power function

A

It is used in sampling Blinn’s microfacet model.

• Assume

( ) ( 1) n p x n x

1 1 1

0 0

1

1 1

n

n x

x dx n n

### 

0 0

( ) n 1

P x x

1 1

~ ( ) ( ) n

X p x  X P U U

Trick (It only works for sampling power distribution)

1 2 1

max( , , , n, n ) Y U U U U

1 1

n

1

1

Pr( ) Pr( ) n

i

Y x U x x

(31)

### Example: exponential distribution

useful for rendering participating media.

ax

### p ( ) 

• Compute CDF P(x)

• Compute P-1(x)

• Compute P (x) Obt i ξ

• Obtain ξ

• Compute Xi=P-1(ξ)

(32)

### Example: exponential distribution

useful for rendering participating media.

ax

0

## 

ax

### dxca

• Compute CDF P(x)

## 

0

x

as

ax

• Compute P-1(x)

0

• Compute P (x)

Obt i ξ

1

###   

• Obtain ξ

• Compute Xi=P-1(ξ)

(33)

### Rejection method

• Sometimes, we can’t integrate into CDF or invert CDF

1

I

###  

f x dx invert CDF

0

f

dx dy

• Algorithm

( ) y f x

dx dy

• Algorithm

Pick U1 and U2

Accept

if

Accept

1 if

2

1

### )

• Wasteful? Efficiency = Area / Area of rectangle

• Wasteful? Efficiency = Area / Area of rectangle

(34)

### Rejection method

• Rejection method is a dart-throwing method without performing integration and inversion without performing integration and inversion.

1. Find q(x) so that p(x)<Mq(x) 2. Dart throwing

a. Choose a pair (X, ξ), where X is sampled from q(x) b. If (ξ<p(X)/Mq(X)) return X

• Equivalently, we pick a qu vale tly, we p c a point (X, ξMq(X)). If

it lies beneath (X) it lies beneath p(X) then we are fine.

(35)

### Why it works

• For each iteration, we generate Xi from q. The sample is returned if ξ<p(X)/Mq(X) which

sample is returned if ξ<p(X)/Mq(X), which happens with probability p(X)/Mq(X).

S th b bilit t t i

• So, the probability to return x is x

p x

x p

q ( ) ( )

)

( 

whose integral is 1/M

M x

x Mq

q( ) ( )  whose integral is 1/M

• Thus, when a sample is returned (with

probability 1/M) X is distributed according to probability 1/M), Xi is distributed according to p(x).

(36)

### Example: sampling a unit disk

void RejectionSampleDisk(float *x, float *y) { float sx, sy;, y;

do {

sx = 1.f -2.f * RandomFloat();

sy = 1.f -2.f * RandomFloat();

} while (sx*sx + sy*sy > 1.f)

*x = sx; *y = sy;

*x = sx; *y = sy;

}

π/4～78.5% good samples, gets worse in higher dimensions, for example, for sphere, π/6～52.4%

dimensions, for example, for sphere, π/6 52.4%

(37)

### Transformation of variables

• Given a random variable X from distribution px(x) to a random variable Y y(X) where y is one to to a random variable Y=y(X), where y is one-to- one, i.e. monotonic. We want to derive the

distribution of Y p (y) distribution of Y, py(y).

Py

y

x

Y

y

x

X

x

Px

x

P (x)

• PDF:

dx x dP dx

x y

dPy

x

x

Px(x)

dx dx

dy p

y dy dP

y

p y

x Py(y)

px dx

y dy

dx y y

py

y

1

x dy p

y p

y

p x

dx y y

py

x

(38)

x x

px

X Y sin

2 1 1

y

y x

x x p

x y

py x

x

y

(39)

### Transformation method

• A problem to apply the above method is that we usually have some PDF to sample from not we usually have some PDF to sample from, not a given transformation.

Gi d i bl X ith ( ) d

• Given a source random variable X with px(x) and a target distribution py(y), try transform X into t th d i bl Y th t Y h th to another random variable Y so that Y has the distribution py(y).

• We first have to find a transformation y(x) so that Px(x)=Pyy(y(x)). Thus,

1

y x

(40)

### Transformation method

• Let’s prove that the above transform works.

W fi h h d i bl ( )

We first prove that the random variable Z= Px(x) has a uniform distribution. If so, then

y1

### ( Z )

should have distribution Px(x) from the inversion method.

Thus Z is uniform and the transformation works

Z x

Pr

Px (X ) x

Pr

X Px (x)

### 

Px(Px (x)) x

Pr 1 1

Thus, Z is uniform and the transformation works.

• It is an obvious generalization of the inversion method in which X is uniform and P (x)=x

method, in which X is uniform and Px(x)=x.

(41)

x

y

y

y

(42)

x

y

y

y

2

x

y

y

y1

1

2

### y

y x

Thus, if X has the distribution , then the

### 2

x x

px( ) 

random variable has the distributionY  2ln X  ln 2x

y

y y e

p ( ) 

(43)

### Multiple dimensions

We often need the other way around,

(44)

### Spherical coordinates

• The spherical coordinate representation of directions is i

directions is

sin sin

cos sin

r y

r x

cos r

z

sin

|

| J | r2 sin

| JT r

) , , ( sin

) , ,

(r r2 p x y z

p

(45)

### Spherical coordinates

• Now, look at relation between spherical directions and a solid angles

directions and a solid angles

### d  sin

• Hence, the density in terms of

(46)

### Multidimensional sampling

• Separable case: independently sample X from px and Y from p p(x y) p (x)p (y)

and Y from py. p(x,y)=px(x)py(y)

• Often, this is not possible. Compute the i l d it f ti ( ) fi t

marginal density function p(x) first.

### p ( ) ( )

• Then compute the conditional density function

### p ( ) ( , )

• Then, compute the conditional density function

### p 

• Use 1D sampling with p(x) and p(y|x).

(47)

### Sampling a hemisphere

• Sample a hemisphere uniformly, i.e. p() c

2 ) 1

(

p

( )

1 p

2

1 c

• Sample  first

2

) sin ,

(

p

2 2

• Now sampling 

0

0

(48)

### Sampling a hemisphere

• Now, we use inversion technique in order to sample the PDF’s

sample the PDF s

0

### P

• Inverting these:

0

### P

• Inverting these:

1

1

2 1

(49)

### Sampling a hemisphere

• Convert these to Cartesian coordinate

1

1

2 1 1

2 1

1

12

### y 

Si il d i ti f f ll h

1

###  z

• Similar derivation for a full sphere

(50)

### Sampling a disk

RIGHT  Equi-Areal WRONG  Equi-Areal RIGHT Equi Areal WRONG  Equi Areal

1

2

1

r U

2

r

U2

(51)

### Sampling a disk

RIGHT  Equi-Areal WRONG  Equi-Areal RIGHT Equi Areal WRONG  Equi Areal

1

1

2

r

U2

(52)

### Sampling a disk

• Uniform

) 1

,

(x y

p p(r,) rp(x, y) r

• Sample r first.

d 2

) (

) (

2

• Then sample

r d

r p r

p( ) ( , ) 2

0

• Then, sample .

###  

2 1 )

(

) , ) (

|

(  

r p

r r p

p

• Invert the CDF.

2 )

(r p ) 2

(r r

P(r)  r P(

| r) 

P

) 2

|

( rP

r

1

 2

r

 2

2

(53)

r U 1

2

4 1

U U

(54)

u

( ,1u u)

v

u v

v

u v

u 1

2 1

1 1u 1

1 u v 

0 0 0

0

p u v

p u v

(55)

• Here

and

### v

are not independent! p u v( , ) 2

### p 

• Conditional probability

1

( ) 2 2(1 )

u

d

0 2

( ) u 2(1 ) (1 )

P u

u du   u

0

( ) 2 2(1 )

p u

dv u

0 1 1

u   U

( | ) 1 p v u

0 0 0

( ) 2(1 ) (1 )

P u

### 

u du u

0 1 2

v U U

0 0 0 0 0 0

( | ) ( | ) 1

(1 ) (1 )

o o

v v v

P v u p v u dv dv

u u

( | )

(1 ) p v u

u

0 0

0 0

(1 u ) (1 u )

(56)

2

0 0

2

0 0

2

0

1

(57)

2

0

1

1

1

2

2

2

2

(58)

### Cosine weighted hemisphere

• Malley’s method: uniformly generates points on the unit disk and then generates directions by the unit disk and then generates directions by projecting them up to the hemisphere above it.

Vector CosineSampleHemisphere(float u1 float u2){

Vector CosineSampleHemisphere(float u1,float u2){

Vector ret;

ConcentricSampleDisk(u1 u2 &ret x &ret y);

ConcentricSampleDisk(u1, u2, &ret.x, &ret.y);

ret.z = sqrtf(max(0.f,1.f - ret.x*ret.x - ret.y*ret.y));

ret.y ret.y));

return ret;

} }

r

(59)

### Cosine weighted hemisphere

• Why does Malley’s method work?

U i di k li () r

• Unit disk sampling

• Map to hemisphere p(r,)

) , (sin )

,

(r  

1

y

T x

y T x

T

(60)

1

y

T x

T

T

T

(61)

n

n

2

  /2

n

0 0

0

n

1 cos

n

### p 

Write the words under the correct picture... Listen and answer the following questions.?. 1) What three things did Ken get

 In an ordered binary tree (usually called just a binary tree), if an internal vertex has two children, the first child is called the left child and the second one is called

Root the MRCT b T at its centroid r. There are at most two subtrees which contain more than n/3 nodes. Let a and b be the lowest vertices with at least n/3 descendants. For such

trigonometric identities will simplify the integrand and make the method of integration obvious... Classify the Integrand According to

• Appearance: vectorized mathematical code appears more like the mathematical expressions found in textbooks, making the code easier to understand.. • Less error prone: without

Once you have created a complete dialog with widgets or arbitrary window hier- archy, the Interface Builder generates the source code needed to recreate it programmati- cally and

MR CLEAN: A Randomized Trial of Intra-arterial Treatment for Acute Ischemic Stroke. • Multicenter Randomized Clinical trial of Endovascular treatment for Acute ischemic stroke in

In Section 3, the shift and scale argument from  is applied to show how each quantitative Landis theorem follows from the corresponding order-of-vanishing estimate.. A number

2.1.1 The pre-primary educator must have specialised knowledge about the characteristics of child development before they can be responsive to the needs of children, set

 Promote project learning, mathematical modeling, and problem-based learning to strengthen the ability to integrate and apply knowledge and skills, and make. calculated

Now, nearly all of the current flows through wire S since it has a much lower resistance than the light bulb. The light bulb does not glow because the current flowing through it

Recycling Techniques are Available to Address Specific Pavement Distress and/or Pavement Structural Requirement.. Introduction to Asphalt Introduction

 The nanostructure with anisotropic transmission characteristics on ITO films induced by fs laser can be used for the alignment layer , polarizer and conducting layer in LCD cell.

This kind of algorithm has also been a powerful tool for solving many other optimization problems, including symmetric cone complementarity problems [15, 16, 20–22], symmetric

• About 14% of jobs in OECD countries participating in Survey  of Adult Skills (PIAAC) are highly automatable (i.e., probability  of automation of over 70%).  ..

To look at the most appropriate ways in which we should communicate with a person who has Autism and make it.. applicable into our day to

OpenGL 4.2 Reference card: Blue means!. deprecated

It is always not easy to control the process of construction due to the complex problems and high-elevation operation environment in the Steel Structure Construction (SSC)

According to this viewpoints, the concept of marketing is not confined to external marketing &amp; sale for external customers only, it adds the concept of ”making employee

In order to have a complete and efficient decision-making policy, it has to be done with multi-dimensional reflection and analysis; The multi-criteria decision-making analysis

Since visible light is not capable of detecting interior defects of a solar cell; it is not easy to reveal invisible micro cracks.. At present it is most common to detect

Based on the analytical results of allocating project management knowledge to detailed inspection management activities, this study concludes that it is possible to

After unit cost database of the third level elements is established, it is quite easy to extract cost information of different design alternatives to support value engineering