Results and discussions
5.4 Other filters
To demonstrate the flexibility of the proposed framework with respect to dif-ferent filters, in addition to cross bilateral filters, we have also experimented with isotropic Gaussian filters and cross non-local means filters. For isotropic Gaussians, we compare the results with GEM [26] which is specifically designed for optimiz-ing over an isotropic Gaussian filterbank. To be fair, we filter the SURE-estimated MSE using an isotropic Gaussian filter without using scene feature information. As shown in Figure 5.6, results of both methods are comparable and the scale selection maps are similar. This means that our SURE optimization is comparable to the specifically-designed GEM for the isotropic case.
The non-local means filter [5] is a popular method for image denoising. It assigns filter weights based on the similarity between pixel neighborhoods. In the context of rendering, we can further utilize scene features for better results. Thus, the cross
non-local mean filter assigns the weight wij between two pixels i and j as implemen-tation). Other symbols are the same as defined in Section 4.2. Note that we use the patch-based distance only for color information, since patch-based distance for scene features tends to smooth out features. The filtered pixel color ˆci of pixel i is computed as the weighted combination of the colors cj of all neighboring pixels j within a 41× 41 neighborhood.
To demonstrate the utility of SURE-based filter selection, we applied cross non-local means filters in two settings. For the first setting – the global cross non-non-local means filter – we used the same range parameter σr across the whole image. For the second one – the SURE cross non-local means filter – we constructed a cross non-local means filterbank by varying σr and used SURE to select best filters and shot samples. Figure 5.7 shows the comparisons between the two. It is clear that the SURE-based framework significantly alleviates the over-smoothness problem of the global filter, especially in shadows and in the motion blur of the moving car. From our experiments, filtering with cross non-local means filters sometimes generated slightly better results than cross bilateral filters. However, it is about 10 times slower than the cross bilateral filter in our implementation. As a compromise between quality and performance, we opted to use cross bilateral filters for most results in the thesis.
5.5 Limitations
The TEAPOT scene (Figure 5.2) reveals a limitation of our approach. The bump mapped floor contains a large number of very high-frequency textures. At the same time, it suffers from a large amount of MC noise due to the environment lighting and
details well. In addition, as with most reconstruction approaches, our method was susceptible to oversmoothing. Scenes contain difficult light paths, such as complex caustics patterns or highly occluded environments where we can not importance sample the light paths efficiently, can also be challenging because the distribution of the samples are more non-Gaussian and the variance estimations are extremely unreliable in these cases.
MC GEM RPF Our(8spp) Our Reference 44 spp 39.86 spp 8 spp 8 spp 26.48 spp 4096spp
140s 135s 363s 85.1s 149.3s
MSE:2.99E-2 MSE:2.07E-3 MSE:6.10E-3 MSE:3.34E-3 MSE:1.28E-3
Figure 5.1: A comparison on the SIBENIK scene with global illumination and depth of field. The image on the top is our result. GEM adapts poorly to the texture on floor and produces oversmoothed results. RPF detects high dependency between u-v parameters and the color, thus filtering the area heavily and also producing oversmoothed results. The RPF image noise is from the sampling approximation of the bilateral filter.
MC GEM RPF Our Reference
35 spp 23.96 spp 8 spp 8 spp 4096spp
42s 44.3s 374.4s 39.3s
MSE:1.99E-1 MSE:1.71E-1 MSE:2.34E-1 MSE:1.34E-1
Figure 5.2: A scene with a glossy teapot. The image on the top is our result. The floor contains complex texture and bump maps. All methods oversmooth the floor.
RPF also oversmooths the glossy self-reflection of the teapot indicated by the arrow.
MC GEM RPF Our(16spp) Our Reference
68spp 63.84spp 16spp 16spp 62.48spp 8192spp
890.5s 906.2s 1676.1s 280.6s 893.1s
MSE:1.33E-1 MSE:1.76E-2 MSE:3.20E-2 MSE:2.05E-2 MSE:8.32E-3
Figure 5.3: Comparisons on a complex scene SPONZA with global illumination and motion blur. The image on the top is our result. Insets show that GEM does not preserve details with symmetric filters, while RPF tends to oversmooth the shadows.
MC GEM RPF Our(8spp) Our Reference 82 spp 51.82 spp 8 spp 8 spp 38.28 spp 4096 spp
59.9s 61.8s 272.4s 26.5s 60.9s
MSE:2.98E-2 MSE:1.84E-2 MSE:5.79E-2 MSE:3.24E-2 MSE:1.12E-2
Figure 5.4: Comparisons on the TOWN scene with an environment light, an area light, and heavy occlusion. The image on the top is our result. GEM fails to adapt to textures, and RPF does not obtain enough samples to reconstruct the scene within the given time. Also, RPF contains heavy noise due to its sampling bilateral filtering approach. Our method adaptively samples the dark noisy area and preserves details well.
MC GEM RPF Our(low spp) Our
Figure 5.5: This figure visualizes the per-pixel relative error of each method. Scenes from the top: SIBENIK, TEAPOT, SPONZA, TOWN. The images show that overall our method produces lower error compare to other methods. It is worth to note that GEM gives lower error in a few regions, such as the shadowed area of the elevated walkway in the TOWN scene, this is because GEM can have more samples under an equal-time comparison, and our feature buffer does not capture the edge well in these region. Still, our method produces a much higher quality image overall compare to GEM.
GEM (MSE:2.87E-4) Our (MSE:3.56E-4)
Figure 5.6: The proposed SURE-based framework incorporated with an isotropic Gaussian filterbank and compared with GEM [26]. The results and scale selection maps generated by both methods are similar.
Global cross non-local means SURE cross non-local means
MSE:8.50E-2 MSE:1.2E-2
Global SURE Reference Global SURE Reference
Figure 5.7: Comparison of cross non-local means filters without and with based framework. Compared to the global cross non-local means filter, our SURE-based optimization largely alleviates the oversmoothing problem. The sampling rate of the noisy input image is about 41 samples per pixel.