• 沒有找到結果。

# • Texture synthesis

N/A
N/A
Protected

Share "• Texture synthesis"

Copied!
76
0
0

(1)

### Digital Visual Effects g Yung-Yu Chuang

with slides by Alex Efros, Li-Yi Wei, Arno Schedl and Paul Debevec

(2)

(3)

(4)

input image

input image

generated image

### • Given a finite sample of some texture, the goal is to synthesize other samples from that same is to synthesize other samples from that same texture.

The sample needs to be "large enough"

– The sample needs to be large enough

(5)

(6)

i

i-1

i-2

i-3

0

i

i-1

i-2

i-n

ffi Ffi-1

Ff0 Ffi-n Ffi-2

i

S-{i}

i

Ni

fi

i

i

(7)

### English-looking text using N-grams:

– Assume a generalized Markov model

Use a large text to compute probability – Use a large text to compute probability

distributions of each letter given N-1 previous letters

lette s

• precompute or sample randomly

– Starting from a seed repeatedly sample this g p y p Markov chain to generate new letters

– One can use whole words instead of letters too.

(8)

### • Results (using alt.singles corpus):

– "One morning I shot an elephant in my arms and kissed him.”

– "I spent an interesting evening recently with a grain of salt"

### • Notice how well local structure is preserved!

– Now let’s try this for video and in 2D...

(9)

(10)

(11)

(12)

(13)

(14)

(15)

2

i, j

frame i vs.

frame j

(16)

(17)

### p

Infinite sample image

– Assuming Markov property, what is conditional probability Infinite sample image

generated image

g p p y, p y

distribution of p, given the neighbourhood window?

– Instead of constructing a model, let’s directly search the g , y input image for all such neighbourhoods to produce a

histogram for p

– To synthesize p, just pick one match at random

(18)

### SAMPLE

finite sample image

### p

image

Generated image – However since our sample image is finite an exact However, since our sample image is finite, an exact

neighbourhood match might not be present

– So we find the best match using SSD error (weighted by a So we find the best match using SSD error (weighted by a

Gaussian to emphasize local structure), and take all samples within some distance from that match

– Using Gaussian-weighted SSD is very important

(19)

(20)

### More results

Increasing window size

(21)

### More results

french canvas rafia weave

(22)

wood brick wall

(23)

### Failure cases

Growing garbage Verbatim copying

(24)

(25)

### 33 55 77

423 528 739 s

423 s 528 s 739 s

### 99 1111 4141

1020 s 1445 s 24350 s

(26)

### Summary

t ll i l – conceptually simple

– models a wide range of real-world textures ll d h l filli

– naturally does hole-filling

– it’s slow

– it’s a heuristic

(27)

(28)

(29)

(30)

(31)

### Results

Random Oriented

Regular Semi-regular

(32)

(33)

(34)

### • Treat neighborhoods as high dimensional points

1 2 3 4 5 6 7 8 9 10 11 12

11 12

### High dimensional point/vector

1 2 3 4 5 6 7 8 9 10 11 12

(35)

(36)

### • Time complexity : O(log N) instead of O(N)

– 2 orders of magnitude speedup for non-trivial images

images

(37)

(38)

### Patch-based methods

non-parametric sampling

I t i

### BB

• Observation: neighbor pixels are highly correlated

Input image Synthesizing a block

Observation: neighbor pixels are highly correlated

### Idea: unit of synthesis = blockunit of synthesis = block

E tl th b t t P(B|N(B))

### •

Exactly the same but now we want P(B|N(B))

### •

Much faster: synthesize all pixels in a block at once

(39)

### Algorithm

– Pick size of block and size of overlap – Synthesize blocks in raster order

– Search input texture for block that satisfies overlap constraints (above and left)

– Paste new block into resulting texture

• blending

• use dynamic programming to compute minimal error boundary cut

(40)

block

Input texture

B1 B2 B1 B2 B1 B2

Random placement p Neighboring blocks Minimal error of blocks

g g

constrained by overlap boundary cut

(41)

### Minimal error boundary

overlapping blocks vertical boundary

## __ ==

min. error boundary overlap error

(42)

(43)

(44)

(45)

### Texture transfer

• Take the texture from one bj d “ i ” i object and “paint” it onto another object

### similarity to underlying image at that spot

(46)

parmesan parmesan

rice

## +==

(47)

(48)

(49)

(50)

### Inpainting

Growing is in “onion peeling” order

• Growing is in onion peeling order

– within each “layer”, pixels with most neighbors are synthesized first

are synthesized first

(51)

(52)

(53)

(54)

(55)

(56)

### Results

http://research microsoft com/vision/cambridge/i3l/patchworks htm http://research.microsoft.com/vision/cambridge/i3l/patchworks.htm

(57)

(58)

(59)

(60)

(61)

(62)

(63)

(64)

(65)

(66)

(67)
(68)

(69)

(70)

(71)
(72)

(73)
(74)

(75)

(76)

### The end!

substance) is matter that has distinct properties and a composition that does not vary from sample

Lee [2006] uses a difficulty level based on a weighted sum of the techniques required to solve a puzzle, showing from a sample of 10,000 Sudoku puzzles that there is a

Given a connected graph G together with a coloring f from the edge set of G to a set of colors, where adjacent edges may be colored the same, a u-v path P in G is said to be a

• Similar to Façade, use a generic face model and view-dependent texture mapping..

The remaining positions contain //the rest of the original array elements //the rest of the original array elements.

These images are the results of relighting the synthesized target object under Lambertian model (left column) and Phong model (right column) with diﬀerent light directions ....

A marble texture can be made by using a Perlin function as an offset using a Perlin function as an offset to a cosine function. texture = cosine( x +

projected texture Active depth from defocus Active depth from defocus Photometric stereo Photometric stereo.. time of flight time