3.3 Painting Effects on Height Field and SVBRDF
3.3.3 Apply Paints on SVBRDF
The method that we use to modify the material SVBRDF can be separated into two parts, the diffuse part and the specular part. For the diffuse part, we use the K-M model to modify the original material ρd to produce the new painted diffuse term ρmd. First, we need to choose a paint type, i.e., the S and K values of the paint, and the paint layer thickness d. The thicker the paint layer is, the more the color of the underlying surface is covered by the paint color. Next, we compute the reflectance Rpl and transmittance Tpl of the paint layer using Equation (3.7) with the S and K of the chosen paint type and the thickness d of the paint layer. After that, we use Equation (3.8) with Rpl and Tpl as R1 and T1 and ρdof the BTF material as R2 to compute the new painted diffuse ρmd :
ρmd = Rpl+ Tpl2ρd
1 − Rplρd. (3.9)
If there are more than one paint layer, we could repeat this procedure for each paint layer from the bottom to top to compute the composite reflectance. The S and K coefficients that we used for our experiments come from two sources, one is the oil paint data from [BWL04]
and the other is the watercolor data from [CAS+97]. Although [BWL04] provides us the 101-wavelength data which is more accurate than standard RGB model, limited to the RGB form of the original BTF data, we have only sampled RGB wavelength from the 101-wavelength data. The three wavelenghs for RGB that we sampled are 652 nm, 532 nm, and 460 nm. While [BWL04] also proposed to use Gaussian quadrature to reduce the 101-wavelength data to 8-wavelength according to the light source spectrum, the 8-wavelengths are automatically decided such that we can not restrict Gaussian quadrature to choose RGB channels, besides we don’t know the light source spectrum of the BTF data. On the other hand, the watercolor data from [CAS+97] follows the RGB model so that we could use it directly as test data. By specifying the desired colors of a layer of paint overtop of both a black and a white background, they compute
3.4 Rendering 20
the S and K values by reversing the K-M equations. Fig. 3.6 shows these S and K values.
For the specular part, since K-M model can not handle the specular term S(ωi , ωo) of the original material, and considering this term into our lighting computation would be too complex for our application, we have decided to drop it and add a new specular term calculated using Fresnel equation. For BTF materials that are closer to diffuse reflection, the influence would be small, but for BTF materials that are closer to specular reflection, dropping the original spec-ular term might darken the final painted result. The specspec-ular reflection of light is motivated from light reaching an interface between two materials of different refractive indices which in our case are the air and the paint. This phenomenon is quantitatively described by the Fresnel equation, so we use it to calculate the new specular effect. We have used Schlick’s Fresnel approximation in our implementation. Since we don’t have actual measurement values of re-fractive indices of paints, these are set to around 1.5 ∼ 2.0. Some sample values of rere-fractive indices of different binding media and pigments could be found in [Bud07].
3.4 Rendering
We have implemented our rendering algorithm using graphics hardware. Although we could render a BTF-mapped object with a paint pattern on it without using the original BTF data, i.e.
using the reconstructed and painted height-fields, the normal map, and the SVBRDF, we think that this would lose some complexity hidden in the original BTF data. Thus we have integrated a BTF rendering algorithm, i.e. the local PCA method[MMK03b], into our rendering system to compensate the lost information during the height field reconstruction and SVBRDF fitting processes. The local PCA method rearrange the BTF data into a BRDF-wise arrangement first, and then perform clustering and principle component analysis(PCA) iteratively to compress the data. The compressed data would be a cluster map, a PCA weights map, and the eigen vectors for each cluster. These are stored as 2D or 3D textures for use in the rendering process. We integrated this method to render the unpainted part of the object surface.
The rendering algorithm is described as follows. At first, we employ some per-pixel
ray-3.4 Rendering 21
Figure 3.6: The S and K coefficients that we use are adopted from [CAS+97].
3.4 Rendering 22
tracing algorithm such as [POC05] to find the view-surface intersection point. By comparing the painted height-field value and the original height-field (i.e. the height field recovered from the original BTF) value at that surface point, we can know if this point is a painted point or an unpainted point (The painted and original height-fields are all stored in texture memory). If it is an unpainted point, we use the local PCA method to approximate the original BTF. If it is a painted point, the Kubelka-Munk model is used to calculate the painted reflectance, and a Fresnel specular term is added. To produce results closer to real situation, we have also adapted the parallax occlusion mapping(POM,[Tat06]) technique to simulate self-shadowing effect for the painted part. The above steps are all performed in the fragment shader. The lower half of Fig. 3.1 shows this rendering workflow.
When using the local PCA method for BTF rendering, we need to take multiple texture samples for each pixel, otherwise the rendering result would be blocky because of texture mag-nification. We have compared the results of taking 5 samples, 9 samples and 25 samples for each pixel, and found that the results are almost the same but the rendering speeds are very different, that is roughly linear to the number of samples, thus we took only 5 samples in our final implementation.
C H A P T E R 4
Results
We have tested our method on several BTF materials. All our programs run on an Intel Core 2 Duo 3.0GHz CPU. We use Matlab to perform photometric stereo normal estimation and Frankot-Chellappa height-field reconstruction. The time needed for these two steps is negli-gible. For the SVBRDF fitting, the most time-consuming step is performing the Levenberg-Marquardt non-linear optimization. It takes around 10 seconds to perform this step for each texel on Matlab, therefore several hours are needed even for a 64 × 64 BTF sample. Thus we have implemented another C++ version fitting program that takes only a few minutes to finish the fitting process for the same BTF sample.
A comparison between the original BTF images and images rendered with reconstructed information are shown in Figure 4.1 and 4.2. The upper rows are the original BTF images and the bottom rows are the reconstructed images. The second rows are the positive part of the original images subtract the reconstructed images and the third rows are the negative part. As can be seen from the figures, the original surface shape and the reconstructed surface shape are a little bit different. In Figure 4.2, the edge of each block seems to be smoother than original. This might be the result of using the Frankot-Chellappa algorithm because its preserving integrability
23
24
nature tends to smooth the reconstructed result. Also, the reconstructed images are darker than the original ones especially for images of lower zenith angles. The reason for this might be inaccuracy in diffuse estimation. For the images of higher zenith angles, errors caused by shadows and specular effects are more noticable. As can be seen from the green boxes in each image of the last column in Figure 4.2, the area in the original image is under shadow while the same area is not under shadow in the reconstructed image. Despite these errors, the reconstructed images still preserve the look and feel of the original BTF images.
After the SVBRDF fitting process, we can use the K-M model to produce the painted BTF images. Figure 4.3 and Figure 4.4 show the results of adding different kinds of paints onto the hole BTF sample. The thickness of the paint in each image is the same everywhere. The shadows are rendered with parallax occlusion mapping [Tat06]. We have also compared the rendered results with alpha blending since alpha blending is an intuitive way to change surface color. Figure 4.5 and Figure 4.6 show these comparisons. The first row in each block shows the painted BTF images with the same viewing and lighting angles but different paint thicknesses.
The second row shows the alpha blending results with alpha 0.25, 0.5, 0.75, and 1. Apparently alpha blending is not a good method for changing the surface color, unless the material is flat.
Next, we show the results of adding paint patterns on 3D models. To render the painted part of the model, we use only the reconstructed information to represent the original material, i.e. the reconstructed height-field, the reconstructed normal map, and the SVBRDF, and the reflectance is calculated using K-M model and Fresnel equation with the paint parameters. To render the unpainted part of the model, we could still use only the reconstructed information of the material. However, as described previously, we have combined the local PCA method to render the unpainted part of the model to fully exploit the complexity of the original BTF.
Before using the local PCA BTF rendering method [MMK03a], the original BTF data need to be compressed. The local PCA compression is another task that needs quite a long time to execute. It depends on the BTF data size and the required compression quality. In our experience, performing 10 iterations for a 64 × 64 BTF data with 51 × 51 view-light pairs takes about 100 minutes. After the compression, the compressed data can be used for rendering.
25
Figure 4.1: Comparison between original BTF images and fitting results. The images are all top-view images and the lights are all from the right side with theta 0, 17, 30, 50, and 65 degrees.
26
Figure 4.2: Another comparison between original BTF images and fitting results. The images are all top-view images and the lights are all from the right side with theta 0, 17, 32, 50, and 65 degrees.
27
Figure 4.3: The top row shows the original BTF images of the hole sample. The second to the last rows are results of painting the hole surface with three different kinds of watercolor paints: Quinacridone Rose(thicknesses 0.2 and 0.4), Cerulean Blue(thicknesses 0.2 and 0.4), and Cadmium Yellow(thicknesses 0.2 and 0.3).
28
Figure 4.4: The top row shows the original BTF images of the hole sample. The second to the last rows are results of painting the hole surface with three different kinds of oil paints:
Alizarin Crimson(thicknesses 0.04 and 0.2), Sap Green(thicknesses 0.1 and 0.4), and Cadmium Yellow(thicknesses 0.04 and 0.08).
29
Figure 4.5: Painted BTF v.s. alpha blending. The leftmost images are the original BTF images.
The first row in each block shows the painted BTF images with the same viewing and lighting angles but with different paint thicknesses. The second row shows the alpha blending results with alpha 0.25, 0.5, 0.75, and 1. The paints used in these images are water color Cadmium Red, French Ultramarine, and Cadmium Yellow from top to bottom.
30
Figure 4.6: Painted BTF v.s. alpha blending. The leftmost images are the original BTF im-ages. The first row in each block shows the painted BTF images with the same viewing and lighting angles but with different paint thicknesses. The second row shows the alpha blending results with alpha 0.25, 0.5, 0.75, and 1. The paints used in these images are oil paints Alizarin Crimson, Cobalt Blue, and Cadmium Yellow from top to bottom.
31
Figure 4.7 and Figure 4.8 show comparisons between renderings using local PCA and using only the reconstructed information for the unpainted part of the model. Notice that although we have used two different rendering methods for the painted part and the unpainted part, the result does not show distinguishable inconsistency. Also, the splash paint pattern looks more realistic than evenly applied paint layer.
Figure 4.9 and Figure 4.10 show the rendering results of another BTF sample on a lizard model with different watercolor and oil paints. Comparing these two images, we can see that images using watercolor and images using oil paint seem to be different only on the colors.
This is because we’ve only changed the K and S coefficients. In order to exhibit more different characteristics between watercolor and oil paint, we also need to consider the flowing pattern caused by different viscosities and the different interactions between paints and surfaces.
In Figure 4.11 and Figure 4.12, results using different refractive indices are demonstrated.
Different refractive indices can be used to simulate wet or dry paints. Notice that different background colors causing different painted appearances. We have chosen a neat purplish red flower pattern which is similar to the pattern of the original BTF material making the paints look more like dyes.
Figure 4.13 and Figure 4.14 show rendering results using a paint brush pattern demonstrat-ing how our method can express the thickness of the paint. A viewer determine how thick the paint is by clues from both geometry and reflectance information. In our framework, the geometrical clue is produced by the painted height field, displacement mapping, and shadow;
the reflectance clue is controlled by the thickness parameter d of the K-M model. For larger d, the painted result is less influenced by the underlying surface color, revealing a purer color from the paint. Combining these two kinds of clues, a viewer can perceive the thickness of the paint. To produce accurate thickness effect, the paint thickness from the painted height field should be consistent to the K-M model input d. Since height field is stored as general image format, i.e. each height value or paint thickness is stored as 0 − 255 on a height field image, this value should be transformed into K-M model thickness parameter d during rendering stage in the fragment shader. We have not yet find a good way to define how thick in reality the paint
32
(a)
(b)
Figure 4.7: A comparison between renderings (a) using local PCA and (b) using only the re-constructed information. The paint is watercolor Cadmium Yellow.
33
(a)
(b)
Figure 4.8: A comparison between renderings (a) using local PCA and (b) using only the re-constructed information. The paint is oil paint Cadmium Yellow.
34
thickness on the height field represent, since we don’t know how thick the original BTF surface is either. In other word, since we don’t know how thick a 1 on height field represent in reality, we can not define how thick a paint thickness value on height field represent in reality. For now, we just use a scaling factor to transform this value into d. In Figure 4.13 and Figure 4.14, the paint thickness in the second image is twice the paint thickness in the first image, and the paint thickness in the third image is trice that in the first image.
Figure 4.15 and Figure 4.16 show examples of multiple paint layers which are combined together using Equation (3.8). The graphics cards we used for our experiments are Nvidia GeForce 9600GT and GTS250. The rendering fps for a 768 × 768 window size is around 10 to 30.
35
Figure 4.9: A splash pattern with different watercolor paints on a lizard model. The paints are Burnt Umber, Quinacridone Rose, and Cadmium Red from top to bottom. The refractive indices are all set to 1.8.
36
Figure 4.10: A splash pattern with different oil paints on a lizard model. The paints are Prussian Blue, Alizarin Crimson, and Titanium White from top to bottom. The refractive indices are all set to 1.8.
37
Figure 4.11: A flower pattern with different refractive indices on a cloth model. The refractive indices are set to 1.8, 2.0, and 2.2 from top to bottom. The paint is watercolor Quinacridone Rose.
38
Figure 4.12: A flower pattern with different refractive indices on a cloth model. The refractive indices are set to 1.8, 2.0, and 2.2 from top to bottom. The paint is oil paint Alizarin Crimson.
39
Figure 4.13: Results of a paint brush pattern. The paint type is watercolor Burnt Umber. The paint thickness in the second image is twice the paint thickness in the first image, and the paint thickness in the third image is trice that in the first image.
40
Figure 4.14: Results of a paint brush pattern. The paint type is oil paint Burnt Sienna from [BWL04]. The paint thickness in the second image is twice the paint thickness in the first image, and the paint thickness in the third image is trice that in the first image.
41
Figure 4.15: An example of using four paint layers. The four images in the bottom row are the scaled paint pattern images. The paints used for four paint patterns from left to right are watercolors: French Ultramarine, Quinacridone Rose, Brilliant Orange, and Phthalo Green.
42
Figure 4.16: An example of using four paint layers. The four images in the bottom row are the scaled paint pattern images. The paints used for four paint patterns from left to right are oil paints: Cobalt Blue, Alizarin Crimson, Burnt Sienna, and Viridian.
C H A P T E R 5
Conclusion
In order to further use the BTF data that are not very easy to acquire, we have proposed the problem of how to simulate the painted appearance of a BTF material in this thesis, and we have developed a simple yet effective method to solve this problem. The method first try to recover neccesary information from the original BTF data, including both geometry and reflectance information, and then use these recovered information and the original data to produce the illusion of a painted surface. Our rendering framework have integrated the local PCA BTF rendering method in order not to waste any information hidden in the raw BTF data. The method is simple and intuitive, yet the results are novel.
Nonetheless, there are still rooms for improvements in each solution step. Here we list some of them. First of all, to simulate the changed appearance of the painted material, we have chosen the Lafortune BRDF model and the Kubelka-Munk reflectance model. Since BTF can be used to represent any kind of materials, the Lafortune model might not suitable for all of them. Is there other suitable BRDF model for this application? If we use other BRDF model, will it be compatible with the K-M model? Also, since there are lots of simplifying assumptions for the K-M model which make the model simple but sometimes not conform to the real conditions,
43
44
how to break these assumptions so that the model can fit into more complex situations? Or is there a better reflectance model for our application? Besides the BRDF and reflectance model, we would like to implement some paint flow simulation algorithm since we didn’t implement one in this work, and we would like to investigate the physical reaction happened when paint is applied to a substrate. This physical process is much more complicated then what we have considered in this thesis. Different paints can have different physical properties, and different substrates can have different responses to the same paint. Some effect such as the capillarity effect is what we have missed in our simulation. To more accurately calculate the reflectance of the painted material, we should know all these detail information. Besides, there have been research considering these effects [LAD08].
Lastly, we would like to extend our work to simulate the aging and weather phenomenon
Lastly, we would like to extend our work to simulate the aging and weather phenomenon