S.-H. Chang1and Y.-C. Lai1†and Y. Niu2and F. Liu2and K.-L. Hua1
1National Taiwan University of Science and Technology, Taiwan
2Portland State University, U.S.A.
Abstract
With the advance of graphics hardware, setting 3D texture as render target is newly available to allow voxelization algorithms to record the existence, color and normal information in a voxel directly without specific encoding and decoding mechanism. In this paper two new voxel-based applications are proposed to take advantage of this new functionality for interactively rendering realistic lighting effects including shadow of objects with complex occlu-sion and refraction and transmisocclu-sion of transparent objects. An absorption coefficient is computed according to the number of surface drawing in each voxel during voxelization and used to compute the amount of light passing through partial occluded complex objects. The refraction and transmission of light passing through transparent objects is simulated by our multiple refraction algorithm using surface normal, transmission coefficient and re-fraction index in each voxel. All these applications can generate the result in real-time without any preprocessing step. Additionally, we also found that the newly available geometry shader can be used to transform a highly com-plex surface-represented scene into a set of high-resolution voxels in only one GPU pass. This possibly improve the efficiency of the voxelization process.
Categories and Subject Descriptors(according to ACM CCS): I.3.5 [Computer Graphics]: Computational Geometry and Object Modeling—Curve, surface, solid, and object representations
1. Introduction
Realism is important for human perception but interactiv-ity is more important for many applications. The refrac-tion and transmission of light as it passes through dif-ferent materials result in many beautiful and intriguing effects. The requirement of interactivity motivates many research to look for hardware-accelerated approximation to simulate refraction effects [DB97,HLFpS99,Ohb03, Oli,LKM01,Ade03,GS04,WD06,OB07,IZT∗07,SZS∗08, WZHB09,CO,LES09]. Among these, Eikonal rendering al-gorithms [IZT∗07,SZS∗08] which transforms edge bound-aries to the gradient of refraction indices for simulating re-fraction and multiple rere-fraction methods [WD06,OB07,CO, LES09] which use image-based algoirhtms to approximate the light transport through a transparent object are popu-lar because they can provide realistic results in a highly in-teractive frame rate. However, there are still limitations in Eikonal methods including complex computation in
tranfor-† Corresponding author, NSC 99-2218-E-011-005-, Taiwan
mation and difficulties in adding absorption and in image-based methods including the requirement of an extra image map per object for estimating traversal distance, the assump-tion of a none-self-occluded object and disability to simulate multiple refraction and total internal reflection effects. In this paper we proposed to use volumetric representation to over-come these limitations. GPU-based voxelization first slices the surface models into a set of unit-sized voxels stored in a 3D volume texture. Then, the multiple-refraction method uses the voxelization results to render the refraction and transmission effect. Our algorithm takes advantage of the flexibility and adjustability of the 3D volume texture to com-pute and store the surface and volume information including normal, refraction index and transmission coefficient for bet-ter approximation of refraction and transmission. The infor-mation allows us to trace a view ray from the camera into the scene and when intersecting with the surface boundary voxels, the normal and refraction index is used to compute the new propagation direction. The process continues until the ray shoots out of the scene and the color indexed by the ray direction and the attenuation accumulated along the path
⃝ The Eurographics Association 2011.c
Shu-Huai Chang & Yu-Chi Lai & Yuzhen Niu & Feng Liu & Kai-Lung Hua / Real-time Realistic Voxel-based Rendering is used to compute the color of the ray. The abilities to
rep-resent the entire scene with a single reprep-resentation without the need of extra support data and simulate the multiple re-fraction and total internal reflection effects of possibly self-occluded transparent objects are the advantages of our voxel-based algorithm. The results show that our algorithm per-form better than image-based methods on rendering a scene with complex deformable objects.
In addition to the refraction and transmission effects, shadow is also important for human perception because shadow gives the sense of existence. Traditional shadow map [Wil78] is the simplest interactive shadow algorithm but it cannot handle transparent and partially-occluded objects.
Transparent shadow proposed by Eisemann et al. [ED06]
uses the voxelization result to estimate the traversal length of light passing through transparent and partially-occluded objects and then the attenuation can be computed with the homogeneous absorption coefficient of the entire scene. The assumption of a single coefficient for the entire scene limits the ability to simulate different degree of occlusion in a com-plex partially-occluded objects which are commonly seen in daily life. Therefore, our algorithm overcomes this limita-tion by computing the absorplimita-tion coefficient according to the type of the object and the density of small geometries in each voxel automatically during voxelization process. Then the shadow of the object can be rendered by attenuating the light using these absorption coefficients along the traversal path. The shadow generated is closer to our perception with negligible extra cost.
In both applications described above volumetric represen-tation is required. Generally, volumetric data stores proper-ties of an object in a set of regular 3D grids. A voxelization algorithm is needed to transform the surface-boundary rep-resentation to a volumetric reprep-resentation before applying volumetric applications and algorithms. In order to achieve interactivity, real-time slicing-based GPU voxelization algo-rithms [CF98,FC00,Lla07,KPT99,DCB∗04] are proposed to slice models into a set of voxels but their efficiency is lim-ited by the requirement of multiple GPU passes . Therefore, encoding slice-based algorithms [KPT99,DCB∗04,ED06, FBP09] are proposed to reduce the number of passes by ex-amining the intersection of each primitive with each voxel grid only once with a special encoding mechanism. Unfor-tunately there are several limitations including the usage of triangles as the represented primitive, the strenuous process of changing encoding and decoding mechanism when appli-cations change voxel resolution and difficulties in recording surfacial and volumetric information. In addition the number of GPU passes for a high-resolution representation still has chance to be more than one. Through the development pro-cess of our voxel-based application, a new GPU-based vox-elization algorithm are found to overcome these limitations by using the geometry shader to slice the geometry mod-els using the clipping plane algorithm [FC00] in single GPU pass. The usage of the geometry shader to voxelize the model
relieves the need of multiple passes in original slicing-based algorithms and enhances the voxelization efficiency. Then, the adjustable 3D volume texture is used to store the slicing result with other surfacial and volumetric information. Since the size of the 3D texture can be easily adjusted according to the need of application and the limitation of graphics hard-ware, this can ease the burden of changing encoding and de-coding mechanism when adjusting the voxel resolution for a general encoding voxelization method. Results show that our algorithm can gain improvement in voxelization efficiency and render all three different lighting effects in real time for a high-resolution voxelization process.
2. Voxelization
The newly available geometry shader is used to slice the surface model into a set of voxels. Before developing our voxelization algorithm we must decide how to store the vol-umetric representation of a model. A uniform-sized voxel structure is schematically similar to a 3D volume texture.
Thus, using a texel in a 3D volume texture is the simplest way to store the volumetric data in a voxel. The ability to adjust the memory size of a texel gives our algorithm the flexibility of computing and storing extra surfacial and volu-metric information such as transmittance, normal and color for generating more realistic lighting effect as described in Section3and 4.
The computation of voxelization is conducted as shown in Fig.1. Generally, a surface-represented triangle is stored as 3 vertices with their position, normal and other informa-tion. When vertices of a triangle are queued into the graph-ics pipeline, the position of a vertex is first transformed into the camera coordinate. Our voxelization algorithm computes which slices from the 3D volume texture have the chance to intersect the triangle. The range of slices which possibly in-tersect the triangle can be calculated with the depth of all three vertices using step 3 and 4 listed in Fig.1. The geom-etry shader duplicates the triangle according to the number of slices in the possible range. At the end of the process the slicing algorithm sets up the far and near clipping plane ac-cording to the index of the destined slice in order to correctly compute the boundary voxels for the triangle. Then a du-plicated triangle is rendered for each proper set of clipping planes.
3. Transparent Shadow Map
Eisemann et al. [ED06] used the voxelization result to esti-mate the passing distance for rendering transparent shadow but their method did not take different degrees of occlusion and material absorption into account. Our algorithm makes an improvement in computing the shadow by attenuating the light along the traversal path by accumulating the absorp-tion varying with the degree of occlusion and material in the following steps:
⃝ The Eurographics Association 2011.c
Shu-Huai Chang & Yu-Chi Lai & Yuzhen Niu & Feng Liu & Kai-Lung Hua / Real-time Realistic Voxel-based Rendering
Detecting the voxels intersected with triangle
1 For each triangle, Tri
2 z0= Z(Tri.V 0), z1= Z(Tri.V 1),z2= Z(Tri.V 2) 3 maxslice= max(z0, z1, z2)/thickness
4 minslice= min(z0, z1, z2)/thickness 5 For i = minsliceto maxslice 6 Planenear= i∗thickness 7 Planef ar= Planenear+ thickness
8 Set Planenearand Planef arto projection matrix 9 If (Intersect(Tri))
10 Rasterize Tri into the slice
Figure 1: This is the pseudo code for computing the bound-ary voxels of a triangle, Tri. V denotes a vertex of a triangle, thickness is the voxel size which is a user specified value, Z() is a function to extract the depth value of a vertex after trans-forming the position of the vertex into the camera coordi-nate, max() / min() computes the maximum/minimum value among the set of input values and Intersect() is a function to test whether the triangle is valid after being culled by the clipping planes.
1. The voxelization camera is set at the position of the light source and aligned with the light direction.
2. Our algorithm voxelizes the scene and computes and stores the absorption of each voxel. The absorption coef-ficient is computed by accumulating the number of writ-ing in each voxel and then this number is multiplied by a user-defined constant to get the absorption coefficient be-cause the number of drawing reflects the degree of partial occlusion in the voxel.
3. During the rendering process, the voxel position, (x, y, s), of the first intersection point from the view is computed.
4. The amount of occlusion can be computed using the fol-lowing equation,∑si=0α(x,y,i) × E(x,y,i) where α() de-scribes the light absorption in this voxel and E() is an occupation flag which 1 represents that the voxel is occu-pied by some object.
4. Refraction and Transmission
Refraction is the change in propagation direction of a light ray when it transports from one medium to another and the light propagation direction change can be described by the Snell’s Law. But single refraction is not enough to describe the light transport through a transparent object because gen-erally a light ray enters and exits an object in a pair and it is a multiple refraction phenomenon.
4.1. Two-surface refraction
Wyman et al. [Wym05] proposed that multiple refraction may be simplified to a two-surface-refraction effect: one happens when light enters the object and the other happens when light exits. The first refraction can use the normal of the intersection point and the incident direction to compute the first refraction direction, ⃗T1. If the traversal distance, d, between the first and the second refraction point can be esti-mated, the second refraction position can be estimated with P2= P1+ d⃗T1where P1and P2are the first and second re-fraction position, and ⃗T1is the refraction direction after the first refraction. Wyman et al. [Wym05] proposed an image-based method to estimate d without considering the first re-fraction direction. We realized that our voxelization result can find a better estimate of d with a similar manner de-scribed in Sec.3, P2 can be computed as described previ-ously and projected into the voxel space to extract the sur-face normal, ⃗N2and refraction index and ⃗T2can be computed with ⃗T1, ⃗N2and refraction index.
4.1.1. Multiple-surface refraction
The two-surface-refraction method cannot render all trans-mittance lighting effects when light hits a transparent ob-ject in a scene. In addition it also has some limit in the al-lowable models and transmittance. Thus, a multiple-surface-refraction algorithm is proposed to simulate the multiple-surface-refractions and reflections inside a scene. The same voxelization pro-cess described in the two-surface-refraction method is used.
When rendering the scene, the view initiates a view ray pass-ing through the center of a pixel. Then, the ray is propa-gated inside the voxel space and every time when the ray hits a boundary voxel, the ray is refracted according to the Snell’s law. In order to properly locate the boundary voxel for refraction, the propagating distance must be set prop-erly to prevent missing the boundary voxel during the traver-sal procedure and wasting efforts in extra propagation. Our implementation chooses the physical distance to propagate through a voxel as the propagation step distance. Then, the position where the next refraction event happens can be com-puted with Pi= Pi−1+ thickness× Tvoxel(⃗Ti)/cosθ where Pi−1is the current position, thickness represents the physical size of the voxel, ⃗Ti−1is the current ray propagation direc-tion, Tvoxel() is a function to transform the ray into the voxel coordinate for locating the voxel which records the normal and transmittance information andθ is the angle between the ray and the dominant component axis of the ray. The next step computes the refracted view ray direction according to the following equation:
⃗Ti= {
re f (⃗Ti−1, Nv(Pi), Tv(Pi)) i f (E(Pi)) = 1
⃗Ti−1 otherwise (1)
where re f () computes the new ray direction according to the Snell’s law, E(Pi) is a flag which indicates whether the voxel at Piis a boundary voxel or not, Nv(Pi) extracts the normal
⃝ The Eurographics Association 2011.c
Shu-Huai Chang & Yu-Chi Lai & Yuzhen Niu & Feng Liu & Kai-Lung Hua / Real-time Realistic Voxel-based Rendering stored in the voxel at Pi and Tv(Pi) extracts the refraction
index stored in the voxel at Pi. The process continues to find the intersection and refracted ray direction until the ray hit the boundary of the volume.
However, there are still problems when applying the al-gorithm due to the finite resolution of voxels. The most fre-quent one is multiple boundary voxels along the traversal path. The surface locality information is used to relieve this issue. We observed that when the angle between the nor-mals of surfaces where the consecutive refractions happens is small, the possibility of misjudge is high. Thus, our al-gorithm uses a threshold to determine whether the refrac-tion mechanism initiates or not and this can reduce a large amount of artifact in this type. The second problem is when a ray hits the corner of two boundary voxels, the algorithm may not find the right refraction point. This problem is simi-lar to the hole problem when solid voxelizing a model. Gen-erally a low pass filter should be able to reduce this problem.
In addition, the filter technique can also reduce the multi-ple intersecting voxel issues described previously. Our al-gorithm proposed another relief to this missing voxel issue based on the observation that a missing voxel issue is much harder to handle properly than a multiple intersected voxel issue. Thus, when slicing the scenes, our voxelization algo-rithm extends the clipping region of the slice to make the ex-tent of a voxel overlap with others’ to increase the chance of multiple intersected voxel situations and reduce the chance of missing voxel situations. Then the angle threshold dis-cussed in the previous paragraph can be used to get a good rendering result.
4.2. Transmittance
When light passes through a medium, the amount of energy passing through will decrease and this phenomenon can be described by transmittance. The simplest method [ED06] to estimate transmittance uses a parameter, transparency, which depends on the traversal length of the light ray through the transparent object. Then, the transmittance is used to deter-mine the amount of transparent blending between the object and the background. However, the method [ED06] is limited to an object with homogeneous material. Our voxelization result contains the surface normal information, the transmit-tance coefficient and refraction index. This allows our ren-dering method to compute transmittance with higher preci-sion by both considering the length and the different trans-parent attenuation of the traversal voxels along the path. The traversal distance between refraction points can be estimated using∥ thickness × Tvoxel(⃗Ti)/cosθ ∥ which is a byproduct of refraction ray estimation. Then the transparent attenua-tion can be computed by transparency =∏Ni=0eσ(di)where N is the total number of refraction points along the traver-sal path andσ() calculates the transparent attenuation of the object [ED06].
5. Results and Discussions 5.1. Voxelization
All the results in this paper are rendered and measured us-ing a computer with ATI HD5850, Intel Core 2 duo E6750 and 2 GB main memory. Our voxelization algorithm is im-plemented with DirectX 11 but the same program is also compatible to DirectX 10. The vertex, geometry and pixel shaders are written using HLSL 4.0. The 3D volume texture is implemented with the format of 2D texture array which is a set of 2D textures with the same format in each pixel and the same resolution for each texture. The array provides the required properties to totally support the need of our al-gorithm and gives our alal-gorithm more freedom in setting the format of each pixel during the implementation process. And a 32-bit RGBA floating point format is chosen to record the voxel information in our current implementation.
Additionally, each graphics hardware device has a differ-ent limitation in the allowable voxelization resolution and the limitation depends on the available GPU memory space.
For example a resolution of 256× 256 × 256 with 32-bit in-formation per voxel requires a memory space of 64MB to store the data. According to the graphics card used in the test, our voxelization algorithm are tested on voxelizing different models with various triangle counts under three different res-olution settings which are 128×128×128, 256×256×256 and 512×512×512. The results are shown in Table1. In ad-dition the time required to voxelize the same set of models under these three resolutions using the algorithm proposed by Ignacio et al. [Lla07] is also shown in Table1. The main
For example a resolution of 256× 256 × 256 with 32-bit in-formation per voxel requires a memory space of 64MB to store the data. According to the graphics card used in the test, our voxelization algorithm are tested on voxelizing different models with various triangle counts under three different res-olution settings which are 128×128×128, 256×256×256 and 512×512×512. The results are shown in Table1. In ad-dition the time required to voxelize the same set of models under these three resolutions using the algorithm proposed by Ignacio et al. [Lla07] is also shown in Table1. The main