LightSlice: Matrix Slice Sampling for the Many-Lights Problem

(1)

LightSlice: Matrix Slice Sampling for the Many-Lights Problem

SIGGRAPH Asia 2011

Yu-Ting Wu

(2)

Authors

Jiawei Ou (歐嘉蔚) PhD Student

Fabio Pellacini Associate Prof.

Dartmouth College

(3)

Rendering





 ( , ) ₂ ( , , ) ( , )cos )

,

( o e o s o i i i i i

o p L p f p L p d

L       

Recursive

image plane

(4)

Many-Lights Problem

• A long-standing problem in rendering is to efficiently and

accurately render complex scenes under complex illumination (large area lights, environment lights, indirect illumination …)

(5)

• Many-Lights Problem:

– Convert the illumination into a large number of point lights (150-300K)

Many-Lights Problem

(6)

• Many-Lights Problem:

– Convert the illumination into a large number of point lights (150-300K)

– For each shading point, sum the contributions from all lights

Many-Lights Problem

(7)

• Advantages

– Simulate complex illumination using point lights

• Area lights

• HDR environment maps

• Sun & sky light

• Indirect illumination

– Unify illumination – Break the recursion

Many-Lights Problem

Area lights + Sun/sky + Indirect

(8)

• Problem encountered

– Too many lights

• Brute-force is impractical

Many-Lights Problem

Museum (1.5M triangles) 1024 x 1024 x 9 shading points 153K point lights

Hundreds of Hours!

(9)

• Efficient approximation using a subset of lights

Many-Lights Problem

Reference (hundreds of hours) LightSlice (425.9 sec)

(10)

Related Work

(11)

Related Work (Major)

Multidimensional Lightcuts

B. Walter, A. Arbree, K. Bala, D. P. Greenberg Cornell University

SIGGRAPH 2006

Matrix Row-Column Sampling for the Many-Lights Problem

M. Hasan, F. Pellacini, K. Bala

Cornell University / Dartmouth University SIGGRAPH 2007

(12)

Related Work (Minor)

Relighting

SIGGRAPH ‘06

Animation

EGSR ‘08

Glossy Material

SIGGRAPH Asia ’09, ‘10

(13)

Lightcuts

Multidimensional Lightcuts

B. Walter, A. Arbree, K. Bala, D. P. Greenberg Cornell University

SIGGRAPH 2006

Lightcuts: A Scalable Approach to Illumination B. Walter, S. Fernandez, A. Arbree, M. Donikian, K. Bala, D. P. Greenberg

Cornell University SIGGRAPH 2005

(14)

• Key concepts:

– Light cluster: approximate many lights by a single brighter light (the representative light)

Lightcuts

(15)

• Key concepts:

– Light tree: binary tree of lights and clusters

– A cut: a set of nodes that partition the lights into clusters

Lightcuts

Clusters

Individual Lights

(16)

• Example cuts

Lightcuts

#1 #2 #3 #4

1 2 3 4

1 4

Light Tree

Representative Light

4

(17)

• Example cuts

Lightcuts

Three Cuts

#1 #2 #4 #1 #3 #4 #1 #4

1 2 3 4

1 4

4

1 2 3 4

1 4

4

1 2 3 4

1 4

4

(18)

• Example cuts

Lightcuts

Three Cuts

1 2 3 4

1 4

4

1 2 3 4

1 4

4

1 2 3 4

1 4

4

#1 #2 #4 #1 #3 #4 #1 #4

Good Bad Bad

(19)

• Example cuts

Lightcuts

Three Cuts

1 2 3 4

1 4

4

1 2 3 4

1 4

4

1 2 3 4

1 4

4

#1 #2 #4 #1 #3 #4 #1 #4

Bad Good Bad

(20)

• Example cuts

Lightcuts

Three Cuts

1 2 3 4

1 4

4

1 2 3 4

1 4

4

1 2 3 4

1 4

Good Good Good

4

#1 #2 #4 #1 #3 #4 #1 #4

(21)

• Algorithm

– For each shading point

• Choose a cut to approximate the illumination

• Use perceptual metric to avoid visible artifacts (Weber’s Law)

• From the root, refine light tree until the upper bound of error is smaller than the perceptual error we can accept

Lightcuts

error

< - M

_ub

G

_ub

V

_ub

S

_lights

I

_i

(22)

• Results

Lightcuts

Lightcuts (128s) Reference (1096s)

Kitchen, 388K polygons, 4608 lights (72 area sources) Error x16

(23)

Matrix Row-Column Sampling

Matrix Sampling for Global Illumination

M. Hasan

Cornell University

PhD Dissertation (2010)

Matrix Row-Column Sampling for the Many-Lights Problem

M. Hasan, F. Pellacini, K. Bala

Cornell University / Dartmouth University SIGGRAPH 2007

(24)

• We can interpret the many-lights problem as a matrix of light-pixel formulation

Matrix Representation

Pixels

(2,000,000)

Lights ^(300,000)

(25)

• Problem: compute sum of columns

Matrix Representation

= Σ ( )

300,000 point lights

= Σ ( )

Pixels

(26)

• Key concepts:

– Matrix has structures (low rank)

• Compute small subset of elements

• Reconstruct

Matrix Row-Column Sampling

Repeating patterns / Large areas of near black

(27)

• Key concepts:

– Compute image as a weighted column sum

Matrix Row-Column Sampling

compute very small subset of columns

compute weighted sum

How to choose columns !?

(28)

• Algorithm flow of MRCS

Matrix Row-Column Sampling

Stratfied Uniform Sampling

300 – 600 rows 400 – 4000 columns

(29)

• Compute entire row or column using GPU

Matrix Row-Column Sampling

Column: light to all shading points Row: shading point to all lights

(30)

• Light clustering algorithm

Matrix Row-Column Sampling

Reduced columns:

vectors in high-dimensional space

Visualize

Radius = Norm of reduced column

Clustering

Position = Position of normalized reduced columns in the high-dimensional space

(31)

• Light clustering algorithm

– Metrics

Matrix Row-Column Sampling

Minimize



 k p

Cp ,

, 1

) (

cost



2 ,

) (

cost





-



C j i

j i

j

i x x

C  

where

norms of reduced column squared distance between normalized reduced columns

NP-Hard. Solve by heuristic approach

(32)

• Light clustering algorithm

– Top-down splitting

Matrix Row-Column Sampling

(33)

• Reconstruction

– Estimation of cluster contribution

• Randomly choose representative column j with proportional to its global norm

• Render the full column using ray-tracing

• Estimate its total weight as

• Linear combination to get the final result of pixel color

Matrix Row-Column Sampling

 

pC Rp



Rj

(34)

• Review of MRCS

Matrix Row-Column Sampling

compute rows (GPU)

assemble rows into reduced matrix

cluster reduced columns

choose representatives weighted sum

(35)

• Results

Matrix Row-Column Sampling

Our result: 16.9 sec

(300 rows + 900 columns)

Reference: 20 min

(using all 100k lights)

5x diff

Temple, 2.1M polygons, 1,000,000+ lights

(36)

Matrix Interpretation

Repeating patterns / Large black areas

(37)

• Lights have two common behaviors

– Global lights

• Strong contributions to all pixels

• Typically correspond to direct illumination (sun)

• Appear as bright matrix columns

– Local lights

• Local behavior affects only a few pixels

• Typically correspond to indirect lighting

• Appear mostly as black columns with a small, low, intensity sections

Matrix Interpretation

(38)

• Lightcuts chooses different clusters for each pixel

Matrix Interpretation of Lightcuts

For each row, cluster the lights independently

Work well for scenes with little occlusion (or usually local lighting) Not efficient for scenes with highly occlusion and global lighting

• Ignore visibility term

error

< M

_ub

G

_ub

V

_ub

I

_i

lights

- S

(39)

• MRCS use global clusters for all pixels,

considering how the lights effect the scene

Matrix Interpretation of MRCS

Sample a small number of rows to determine the clusters

Work very well for global lighting (subsampled rows include shadow) Not efficient for local lighting

(with many zero elements along the column)

(40)

Algorithms

(41)

• LightSlice combines the advantages from both lightcuts and matrix row-column sampling

• Key concepts:

– Cluster similar surface samples to form slices in matrix

– The slice of the matrix has significantly lower rank than the whole matrix

Algorithms

(42)

• Flowchart

Algorithms

Spatial Clustering

1400 – 1600 slices 400 columns for each slice

(43)

• Matrix slicing

• Slice sampling

• Initial light clustering

• Per-slice cluster refinement

• Per-slice reconstruction

Algorithms

(44)

• Matrix slicing

– Partition the surface samples based on the geometric proximity

– Use a 6D KD-Tree

Algorithms

(45)

• Slice sampling

– Randomly pick one representative point – Compute full row using raytracing

Algorithms

(46)

• Initial Light Clustering

– The same as Matrix Row-Column Sampling

– The rough clustering (30% of total cluster budget) can capture strong global lighting well

Algorithms

(47)

• Per-Slice Cluster Refinement

– Refine the initial light clustering for each slice to adapt to local lighting

– Avoid image-space discontinuity artifacts

Algorithms

(48)

• Per-Slice Cluster Refinement

– For each slice Sⁱ, assemble a local matrix Lⁱ

– Initialize the light clustering as in last step

– Iteratively split the highest cost cluster in until a maximum number of clusters is reached

Algorithms

Sⁱ

Lⁱ 6D space KD-Tree search

(49)

• Per-Slice Reconstruction

– Render each slice by summing the contribution of each of its clusters

Algorithms

(50)

• Per-Slice Reconstruction

– Estimation of cluster contribution

• Randomly choose representative column j with proportional to its global norm |R_j|

• Render using ray-tracing

• Estimate its total weight as

• Linear combination to get the final result of pixel color

Algorithms



_p _C^Lⁱ ^R^p



^R^j



 k

(51)

Results and Discussion

(52)

• Equal-time / equal-quality comparison

– LightSlice, Lightcuts, and MRCS – Four test scenes

– Error (relative) visualization

– Machine

• Four Intel Xeon 7560 processors (running at 2.27 GHz), each with 8 cores

Results and Discussion

(53)

• Scene1: Sanmiguel

– 1.6 M triangles – 550K VPL

– Resolution: 1024 x 1024 (9)

Results and Discussion

(54)

Equal-Time Comparison

light slice mrcs lightcuts reference

(55)

Equal-Quality Comparison

(56)

Error Plot

Scene: sanmiguel

(57)

• Scene2: Museum

– Resolution: 1024 x 1024 (9)

Results and Discussion

(58)

Equal-Time Comparison

(59)

Equal-Quality Comparison

(60)

• Scene3: Condo

– Resolution: 1200 x 900 (9)

Results and Discussion

(61)

Equal-Time Comparison

slices visualization

2.12% 5.60% 5.70%

(62)

Equal-Quality Comparison

slices visualization 351.9 sec 1621 sec (4.6x) 1972 sec (5.6x)

(63)

• Scene4: Lobby

– 611.5 K triangles – 317K VPL

– Resolution: 1200 x 900 (9)

Results and Discussion

(64)

Equal-Time Comparison

slices visualization

1.14% 2.51% 3.70%

(65)

Equal-Quality Comparison

slices visualization 866.5 sec 2542 sec (2.93x) 2738 sec (3.16x)

(66)

• Parameters selection

– LightSlice avoids spatial discontinuities by using a high number of slices

Limitations

378 slices 400 columns

378 slices 1200 columns

(67)

• Glossy surfaces

– Glossy transports increase the local matrix rank

• Animation

– Left as a future work

• Matrix sparsity

– LightSlice cannot efficiently converge on the

correct solution if the transport matrix is too sparse

Limitations

(68)

• LightSlice

– An algorithm that can efficiently solve the many- light problems

– Take advantages of the global and local behavior of lighting by clustering and rendering per-slice columns in the transport matrix