• 沒有找到結果。

Image-Based Lighting & Textures

N/A
N/A
Protected

Share "Image-Based Lighting & Textures"

Copied!
123
0
0

(1)

2005/6/7

with slides by Alex Efros, Li-Yi Wei and Paul Debevec

(2)

• Final project:

– Checkpoint this Sunday, send me team, topic and brief progress update

– demo on 6/30 (Friday) 10:00am in this room – Report due on 7/3 (Monday) 11:59pm

(3)

(4)

(5)

(6)

Reflectance

• The Bidirectional Reflection Distribution Function

Given an incoming ray and outgoing ray

what proportion of the incoming light is reflected along outgoing ray?

Answer given by the BRDF:

surface normal

(7)

(8)

(9)

o

o

e

o

i i

i i

o

2

i

s

o

o

i

o

i

i

(10)

(11)

Environment maps

Miller and Hoffman, 1984

(12)

o

o

e

o

i i

i i

o

2

i

s

o

2

o

i

d

i

i

i

∫

s

(13)

• Basis Functions are pieces of signal that can be used to produce approximations to a function

Basis functions

(14)

• We can then use these coefficients to reconstruct an approximation to the original signal

Basis functions

(15)

• We can then use these coefficients to reconstruct an approximation to the original signal

=

N i

i i 1

(16)

• Orthogonal Basis Functions

– These are families of functions with special properties

– Intuitively, it’s like functions don’t overlap each other’s footprint

• A bit like the way a Fourier transform breaks a functions into component sine waves

i

j

(17)

(18)

• Represented in spherical coordinates by the function

where l is the band and m is the index within the band

0

0

l l

m l m

l

m l m

l m

l

(19)

(20)

(21)

(22)

0

0

−1

1

11

1

2

22

0

2 1

2

2

2

(23)

(24)

(25)

2

2

2

(26)

(27)

2

(28)

(29)

• Take a function comprised of two area light sources

– SH project them into 4 bands = 16 coefficients

(30)

• We reconstruct the signal

– Using only these coefficients to find a low frequency approximation to the original light source

(31)

• Assumptions

– Diffuse surfaces

– Distant illumination

– No shadowing, interreflection

irradiance is a function of surface normal

o

2

o

i

d

i

i

i

s

(32)

quake light map

(33)

Illumination Environment Map Irradiance Environment Map

L n

Ω

(34)

0

l lm lm

l m l

+

= =−

0

l lm lm

l m l

+

= =−

(35)

l

2 1

2 2

( 1) !

2 ( 2)( 1) 2 !

l

l l l

A l l even

l l

π

=

+ − ⎢

cosine term

(36)

-1

-2 0 1 2

0 1 2

lm

θ ϕ

y z x

xy yz 3z2 1 zx x2 y2

l

m

(37)

-1

-2 0 1 2

0 1 2

lm

θ ϕ

y z x

xy yz 3z2 1 zx x2 y2

l

m

(38)

-1

-2 0 1 2

0 1 2

lm

θ ϕ

y z x

xy yz 3z2 1 zx x2 y2

l

m

(39)

Comparison

Incident illumination

300x300

Irradiance map Texture: 256x256

Hemispherical Integration 2Hrs

Irradiance map Texture: 256x256 Spherical Harmonic

Coefficients 1sec

Time 300 300 256 256 × × × Time 9 256 256∝ × ×

(40)

y

(41)

(42)

(43)

(44)

(45)

(46)

(47)

(48)

(49)
(50)

(51)

Light Probe / Calibration Grid

(52)

real scene real scene

Modeling the Scene

light-based model light-based model

(53)

(54)

(55)

(56)

(57)

(58)

(59)

(60)

(61)

(62)

(63)

(64)

(65)

• Given a finite sample of some texture, the goal is to synthesize other samples from that same texture.

– The sample needs to be "large enough"

synthesis

generated image input image

(66)

(67)

i

i

i-1

i-2

i-3

0

i

i-1

i-2

i-n

ffi Ffi-1

Ff0

fi Ffi-2

Ffi-n

i

S-{i}

i

Ni

(68)

• [Shannon’48] proposed a way to generate English-looking text using N-grams:

– Assume a generalized Markov model

– Use a large text to compute probability

distributions of each letter given N-1 previous letters

precompute or sample randomly

– Starting from a seed repeatedly sample this Markov chain to generate new letters

– One can use whole words instead of letters too.

(69)

• Results (using alt.singles corpus):

"One morning I shot an elephant in my arms and kissed him.”

"I spent an interesting evening recently with a grain of salt"

• Notice how well local structure is preserved!

– Now let’s try this in 2D...

(70)

– Assuming Markov property, what is conditional probability distribution of p, given the neighbourhood window?

– Instead of constructing a model, let’s directly search the input image for all such neighbourhoods to produce a

histogram for p

– To synthesize p, just pick one match at random

Ideally

Infinite sample image

generated image

(71)

finite sample image

Generated image

p

– However, since our sample image is finite, an exact neighbourhood match might not be present

– So we find the best match using SSD error (weighted by a

Gaussian to emphasize local structure), and take all samples within some distance from that match

Using Gaussian-weighted SSD is very important

(72)

(73)

More results

Increasing window size

(74)

french canvas rafia weave

(75)

wood

brick wall

(76)

Failure cases

Growing garbage Verbatim copying

(77)

Inpainting

• Growing is in “onion peeling” order

– within each “layer”, pixels with most neighbors are synthesized first

(78)

(79)

(80)

(81)

(82)

9×9 11×11 41×41

423 s 528 s 739 s

1020 s 1445 s 24350 s

(83)

Summary

– conceptually simple

– models a wide range of real-world textures – naturally does hole-filling

– it’s slow

– it’s a heuristic

(84)

(85)

(86)

(87)

(88)

Results

Random Oriented

Regular Semi-regular

(89)

(90)

(91)

• Treat neighborhoods as high dimensional points

1 2 3 4 5 6 7 8 9 10 11 12

Neighborhood

1 2 3 4 5 6 7 8 9 10 11 12

(92)

(93)

• Time complexity : O(log N) instead of O(N)

– 2 orders of magnitude speedup for non-trivial images

(94)

(95)

Patch-based methods

• Observation: neighbor pixels are highly correlated

Input image

non-parametric sampling

Idea:unit of synthesis = blockunit of synthesis = block

Exactly the same but now we want P(B|N(B))

Much faster: synthesize all pixels in a block at once

Synthesizing a block

(96)

Algorithm

– Pick size of block and size of overlap – Synthesize blocks in raster order

– Search input texture for block that satisfies overlap constraints (above and left)

– Paste new block into resulting texture

• blending

• use dynamic programming to compute minimal error boundary cut

(97)

Input texture

B1 B2

Random placement of blocks

block

B1 B2

Neighboring blocks constrained by overlap

B1 B2

Minimal error boundary cut

(98)

min. error boundary

Minimal error boundary

overlapping blocks vertical boundary

overlap error

(99)

(100)

(101)

(102)

(103)

(104)

(105)

(106)

(107)

(108)

(109)

(110)

(111)

(112)

(113)

(114)
(115)

(116)

(117)

(118)
(119)

(120)
(121)

(122)

(123)

References

• Paul Debevec, Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-based Graphics with Global Illumination and High Dynamic Range

Photography, SIGGRAPH 1998.

• Alexei A. Efros, Thomas K. Leung, Texture Synthesis by Non-parametric Sampling, ICCV 1999.

• Li-Yi Wei, Marc Levoy, Fast Texture Synthesis Using Tree-Structured Vector Quantization, SIGGRAPH 2000.

• Aaron Hertzmann, Charles E. Jacobs, Nuria Oliver, Brian Curless, David H. Salesin, Image Analogies, SIGGRAPH 2001.

• Alexei A. Efros, William T. Freeman, Image Quilting for Texture Synthesis and Transfer, SIGGRAPH 2001.

• Fredo Durand, Julie Dorsey, Fast Bilateral Filtering for the Display of High Dynamic Range Images SIGGRAPH the Display of High Dynamic Range Images, SIGGRAPH 2002. •

– Any set of parallel lines on the plane define a vanishing point – The union of all of these vanishing points is the horizon line. • also called

Modeling and Rendering Architecture from Photographs: A Hybrid Rendering Architecture from Photographs: A Hybrid Geometry- and Image-Based Approach, SIGGRAPH 1996. Tour Into

• Local scene w/o objects, illuminated by model Local scene w/o objects, illuminated by model...

Spacetime Faces: High Resolution Capture for Modeling and Animation, SIGGRAPH 2004. • Paul Debevec, Tim Hawkins, Chris Tchou, Haarm-Pieter Duiker, Westley Sarokin, Mark

• Build a model of average shape and texture, and principal variations using PCA..

Use images to adapt a generic face model Use images to adapt a generic face model. Creating

• Tree lifetime: When the first node is dead in tree T, the rounds number of the node surviving is the lifetime of the tree. The residual energy of node is denoted as E)), where

Creating full view panoramic image mosaics and texture-mapped models, SIGGRAPH 1997, pp251-258. Lowe, Recognising Panoramas,

• Paul Debevec, Rendering Synthetic Objects into Real Scenes:. Bridging Traditional and Image-based Graphics with Global Illumination and High Dynamic

 Retrieval performance of different texture features according to the number of relevant images retrieved at various scopes using Corel Photo galleries. # of top

◦ Action, State, and Reward Markov Decision Process Reinforcement Learning.

Core vector machines: Fast SVM training on very large data sets. Using the Nystr¨ om method to speed up

Core vector machines: Fast SVM training on very large data sets. Multi-class support

These images are the results of relighting the synthesized target object under Lambertian model (left column) and Phong model (right column) with diﬀerent light directions ....

• Johannes Kopf, Michael Cohen, Dani Lischinski, Matt Uyttendaele, Joint Bilateral Upsampling, SIGGRAPH 2007. • Jiawen Chen, Sylvain Paris, Fredo Durand, Real-time Edge-Aware

• Given a finite sample of some texture, the goal is to synthesize other samples from that same is to synthesize other samples from that same texture...

– Camera view plane is parallel to back of volume – Camera up is normal to volume bottom. – Volume bottom

Lance Williams: Brute Force in image space (shadow maps in 1978, but other similar ideas like Z buffer, bump mapping using textures and so on).. Completely

Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, Dimitris Metaxas, “StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks”, ICCV,

– Any set of parallel lines on the plane define a vanishing point. – The union of all of these vanishing points is the

◦ Value function: how good is each state and/or action.. ◦ Policy: agent’s

A Face Recognition Technology Based on Weighted Local Vector Pattern (WLVP) and Bidirectional Recognition Algorithm (BR) is proposed.. The significant texture of