LightSlice: Matrix Slice Sampling for the ManyLights Problem
SIGGRAPH Asia 2011
YuTing Wu
Authors
Jiawei Ou (歐嘉蔚) PhD Student
Fabio Pellacini Associate Prof.
Dartmouth College
Rendering
( , ) _{2} ( , , ) ( , )cos )
,
( o e o s o i i i i i
o p L p f p L p d
L
Recursive
image plane
ManyLights Problem
• A longstanding problem in rendering is to efficiently and
accurately render complex scenes under complex illumination (large area lights, environment lights, indirect illumination …)
• ManyLights Problem:
– Convert the illumination into a large number of point lights (150300K)
ManyLights Problem
• ManyLights Problem:
– Convert the illumination into a large number of point lights (150300K)
– For each shading point, sum the contributions from all lights
ManyLights Problem
• Advantages
– Simulate complex illumination using point lights
• Area lights
• HDR environment maps
• Sun & sky light
• Indirect illumination
– Unify illumination – Break the recursion
ManyLights Problem
Area lights + Sun/sky + Indirect
• Problem encountered
– Too many lights
• Bruteforce is impractical
ManyLights Problem
Museum (1.5M triangles) 1024 x 1024 x 9 shading points 153K point lights
Hundreds of Hours!
• Efficient approximation using a subset of lights
ManyLights Problem
Reference (hundreds of hours) LightSlice (425.9 sec)
Related Work
Related Work (Major)
Multidimensional Lightcuts
B. Walter, A. Arbree, K. Bala, D. P. Greenberg Cornell University
SIGGRAPH 2006
Matrix RowColumn Sampling for the ManyLights Problem
M. Hasan, F. Pellacini, K. Bala
Cornell University / Dartmouth University SIGGRAPH 2007
Related Work (Minor)
Relighting
SIGGRAPH ‘06
Animation
EGSR ‘08
Glossy Material
SIGGRAPH Asia ’09, ‘10
Lightcuts
Multidimensional Lightcuts
B. Walter, A. Arbree, K. Bala, D. P. Greenberg Cornell University
SIGGRAPH 2006
Lightcuts: A Scalable Approach to Illumination B. Walter, S. Fernandez, A. Arbree, M. Donikian, K. Bala, D. P. Greenberg
Cornell University SIGGRAPH 2005
• Key concepts:
– Light cluster: approximate many lights by a single brighter light (the representative light)
Lightcuts
• Key concepts:
– Light tree: binary tree of lights and clusters
– A cut: a set of nodes that partition the lights into clusters
Lightcuts
Clusters
Individual Lights
• Example cuts
Lightcuts
#1 #2 #3 #4
1 2 3 4
1 4
Light Tree
Representative Light
4
• Example cuts
Lightcuts
Three Cuts
#1 #2 #4 #1 #3 #4 #1 #4
1 2 3 4
1 4
4
1 2 3 4
1 4
4
1 2 3 4
1 4
4
• Example cuts
Lightcuts
Three Cuts
1 2 3 4
1 4
4
1 2 3 4
1 4
4
1 2 3 4
1 4
4
#1 #2 #4 #1 #3 #4 #1 #4
Good Bad Bad
• Example cuts
Lightcuts
Three Cuts
1 2 3 4
1 4
4
1 2 3 4
1 4
4
1 2 3 4
1 4
4
#1 #2 #4 #1 #3 #4 #1 #4
Bad Good Bad
• Example cuts
Lightcuts
Three Cuts
1 2 3 4
1 4
4
1 2 3 4
1 4
4
1 2 3 4
1 4
Good Good Good
4#1 #2 #4 #1 #3 #4 #1 #4
• Algorithm
– For each shading point
• Choose a cut to approximate the illumination
• Use perceptual metric to avoid visible artifacts (Weber’s Law)
• From the root, refine light tree until the upper bound of error is smaller than the perceptual error we can accept
Lightcuts
error
<  M
_{ub}G
_{ub}V
_{ub}S
_{lights }I
_{i }• Results
Lightcuts
Lightcuts (128s) Reference (1096s)
Kitchen, 388K polygons, 4608 lights (72 area sources) Error x16
Matrix RowColumn Sampling
Matrix Sampling for Global Illumination
M. Hasan
Cornell University
PhD Dissertation (2010)
Matrix RowColumn Sampling for the ManyLights Problem
M. Hasan, F. Pellacini, K. Bala
Cornell University / Dartmouth University SIGGRAPH 2007
• We can interpret the manylights problem as a matrix of lightpixel formulation
Matrix Representation
Pixels
(2,000,000)
Lights ^{(300,000) }
• Problem: compute sum of columns
Matrix Representation
= Σ ( )
300,000 point lights
= Σ ( )
Pixels
• Key concepts:
– Matrix has structures (low rank)
• Compute small subset of elements
• Reconstruct
Matrix RowColumn Sampling
Repeating patterns / Large areas of near black
• Key concepts:
– Compute image as a weighted column sum
Matrix RowColumn Sampling
compute very small subset of columns
compute weighted sum
How to choose columns !?
• Algorithm flow of MRCS
Matrix RowColumn Sampling
Stratfied Uniform Sampling
300 – 600 rows 400 – 4000 columns
• Compute entire row or column using GPU
Matrix RowColumn Sampling
Column: light to all shading points Row: shading point to all lights
• Light clustering algorithm
Matrix RowColumn Sampling
Reduced columns:
vectors in highdimensional space
Visualize
Radius = Norm of reduced column
Clustering
Position = Position of normalized reduced columns in the highdimensional space
• Light clustering algorithm
– Metrics
Matrix RowColumn Sampling
Minimize
k p
Cp ,
, 1
) (
cost
2 ,
) (
cost

C j i
j i
j
i x x
C
where
norms of reduced column squared distance between normalized reduced columns
NPHard. Solve by heuristic approach
• Light clustering algorithm
– Topdown splitting
Matrix RowColumn Sampling
• Reconstruction
– Estimation of cluster contribution
• Randomly choose representative column j with proportional to its global norm
• Render the full column using raytracing
• Estimate its total weight as
• Linear combination to get the final result of pixel color
Matrix RowColumn Sampling
pC Rp
RjRj
• Review of MRCS
Matrix RowColumn Sampling
compute rows (GPU)
assemble rows into reduced matrix
cluster reduced columns
choose representatives weighted sum
• Results
Matrix RowColumn Sampling
Our result: 16.9 sec
(300 rows + 900 columns)
Reference: 20 min
(using all 100k lights)
5x diff
Temple, 2.1M polygons, 1,000,000+ lights
Matrix Interpretation
Repeating patterns / Large black areas
• Lights have two common behaviors
– Global lights
• Strong contributions to all pixels
• Typically correspond to direct illumination (sun)
• Appear as bright matrix columns
– Local lights
• Local behavior affects only a few pixels
• Typically correspond to indirect lighting
• Appear mostly as black columns with a small, low, intensity sections
Matrix Interpretation
• Lightcuts chooses different clusters for each pixel
Matrix Interpretation of Lightcuts
For each row, cluster the lights independently
Work well for scenes with little occlusion (or usually local lighting) Not efficient for scenes with highly occlusion and global lighting
• Ignore visibility term
error
< M
_{ub}G
_{ub}V
_{ub}I
_{i }lights
 S
• MRCS use global clusters for all pixels,
considering how the lights effect the scene
Matrix Interpretation of MRCS
Sample a small number of rows to determine the clusters
Work very well for global lighting (subsampled rows include shadow) Not efficient for local lighting
(with many zero elements along the column)
Algorithms
• LightSlice combines the advantages from both lightcuts and matrix rowcolumn sampling
• Key concepts:
– Cluster similar surface samples to form slices in matrix
– The slice of the matrix has significantly lower rank than the whole matrix
Algorithms
• Flowchart
Algorithms
Spatial Clustering
1400 – 1600 slices 400 columns for each slice
• Matrix slicing
• Slice sampling
• Initial light clustering
• Perslice cluster refinement
• Perslice reconstruction
Algorithms
• Matrix slicing
– Partition the surface samples based on the geometric proximity
– Use a 6D KDTree
Algorithms
• Slice sampling
– Randomly pick one representative point – Compute full row using raytracing
Algorithms
• Initial Light Clustering
– The same as Matrix RowColumn Sampling
– The rough clustering (30% of total cluster budget) can capture strong global lighting well
Algorithms
• PerSlice Cluster Refinement
– Refine the initial light clustering for each slice to adapt to local lighting
– Avoid imagespace discontinuity artifacts
Algorithms
• PerSlice Cluster Refinement
– For each slice S^{i}, assemble a local matrix L^{i }
– Initialize the light clustering as in last step
– Iteratively split the highest cost cluster in until a maximum number of clusters is reached
Algorithms
S^{i}
L^{i} 6D space KDTree search
• PerSlice Reconstruction
– Render each slice by summing the contribution of each of its clusters
Algorithms
• PerSlice Reconstruction
– Estimation of cluster contribution
• Randomly choose representative column j with proportional to its global norm R_{j}
• Render using raytracing
• Estimate its total weight as
• Linear combination to get the final result of pixel color
Algorithms
_{p} _{C}^{L}^{i} ^{R}^{p}
^{R}^{j}
kResults and Discussion
• Equaltime / equalquality comparison
– LightSlice, Lightcuts, and MRCS – Four test scenes
– Error (relative) visualization
– Machine
• Four Intel Xeon 7560 processors (running at 2.27 GHz), each with 8 cores
Results and Discussion
• Scene1: Sanmiguel
– 1.6 M triangles – 550K VPL
– Resolution: 1024 x 1024 (9)
Results and Discussion
EqualTime Comparison
light slice mrcs lightcuts reference
EqualQuality Comparison
light slice mrcs lightcuts reference
Error Plot
Scene: sanmiguel
• Scene2: Museum
– 1.5 M triangles – 153K VPL
– Resolution: 1024 x 1024 (9)
Results and Discussion
EqualTime Comparison
light slice mrcs lightcuts reference
EqualQuality Comparison
light slice mrcs lightcuts reference
• Scene3: Condo
– 1.4 M triangles – 305K VPL
– Resolution: 1200 x 900 (9)
Results and Discussion
EqualTime Comparison
light slice mrcs lightcuts reference
slices visualization
2.12% 5.60% 5.70%
EqualQuality Comparison
light slice mrcs lightcuts reference
slices visualization 351.9 sec 1621 sec (4.6x) 1972 sec (5.6x)
• Scene4: Lobby
– 611.5 K triangles – 317K VPL
– Resolution: 1200 x 900 (9)
Results and Discussion
EqualTime Comparison
light slice mrcs lightcuts reference
slices visualization
1.14% 2.51% 3.70%
EqualQuality Comparison
light slice mrcs lightcuts reference
slices visualization 866.5 sec 2542 sec (2.93x) 2738 sec (3.16x)
• Parameters selection
– LightSlice avoids spatial discontinuities by using a high number of slices
Limitations
378 slices 400 columns
378 slices 1200 columns
• Glossy surfaces
– Glossy transports increase the local matrix rank
• Animation
– Left as a future work
• Matrix sparsity
– LightSlice cannot efficiently converge on the
correct solution if the transport matrix is too sparse
Limitations
• LightSlice
– An algorithm that can efficiently solve the many light problems
– Take advantages of the global and local behavior of lighting by clustering and rendering perslice columns in the transport matrix
– 3x – 6x performance improvement compared to previous work