Modeling Bidirectional Texture Functions with
Multivariate Spherical Radial Basis Functions
Yu-Ting Tsai, Kuei-Li Fang, Wen-Chieh Lin, Member, IEEE, and Zen-Chung Shih, Member, IEEE
Abstract—This paper presents a novel parametric representation for bidirectional texture functions. Our method mainly relies on two original techniques, namely, multivariate spherical radial basis functions (SRBFs) and optimized parameterization. First, since the surface appearance of a real-world object is frequently a mixed effect of different physical factors, the proposed sum-of-products model based on multivariate SRBFs especially provides an intrinsic and efficient representation for heterogenous materials. Second, optimized parameterization particularly aims at overcoming the major disadvantage of traditional fixed parameterization. By using a parametric model to account for variable transformations, the parameterization process can be tightly integrated with multivariate SRBFs into a unified framework. Finally, a hierarchical fitting algorithm for bidirectional texture functions is developed to exploit spatial coherence and reduce computational cost. Our experimental results further reveal that the proposed representation can easily achieve high-quality approximation and real-time rendering performance.Index Terms—Reflectance and shading models, bidirectional texture functions, parameterization, spherical radial basis functions.
Ç
1
I
NTRODUCTIONR
EAL-WORLD surface reflectance, microscale appearance, and realistic lighting effects are too complicated to be described with simple analytic models. State-of-the-art data-driven rendering algorithms thus synthesize high-quality images from precomputed or measured reflectance data. Over the last decades, there have been tremendous advances in this field. For example, image-based rendering methods [1], [2], [3] generate virtual images from novel view directions by interpolating precaptured images. Since they assume no specific reflectance characteristics of object surfaces, the appearance of real-world objects can be faithfully rendered.Moreover, the pioneering work by Dana et al. [4] further introduced the bidirectional texture function (BTF) to model spatially varying reflectance distributions over a 2D surface. A BTF is a 6D function that combines textures and bidirectional reflectance distribution functions (BRDFs) to account for the appearance of a 2D surface under various illumination and view conditions. Therefore, images rendered from measured BTFs can realistically exhibit complex lighting effects and detailed mesostructures of real-world objects, including the microgeometry of rough surfaces, self-shadows, and multiple light scattering. In addition to rendering applications, BTFs provide realistic texture models for computer vision applica-tions, such as segmentation, robust visual classification,
retrieval or illumination/view invariant methods dealing with images of textured natural materials [5].
Nevertheless, a compact and efficient representation for BTFs remains challenging in practice. The enormous amount of BTF data frequently becomes the performance bottleneck at runtime and prohibits further analysis in computer vision and graphics applications. This challenge thus has stimu-lated the recent development of sophisticated approximation algorithms for large-scale surface appearance data [6], [7], [8], [9], [10], [11], [12]. In addition to the challenge of dealing with tremendous data size, a BTF data set is a mixed effect of various types of physical factors. This high-dimensional nature is so complicated that simple analytic models often fail to describe the multivariate behavior of a BTF.
In this paper, we introduce a novel functional represen-tation to solve the tremendous data size and complex behavior problems in BTF modeling. The complex beha-viors of a reflectance function are described as a weighted sum of the products of several univariate basis functions, which form a multivariate representation. Specifically, we decompose a reflectance field as a linear combination of multivariate spherical radial basis functions (SRBFs), while each multivariate SRBF is constructed from the product of several univariate SRBFs1 [13]. Although the optimization process of such a general model may be difficult, our experimental results demonstrate that a fast and practical implementation is feasible even for large-scale appearance data sets such as BTFs.
To obtain a compact representation, it is also well known that transforming the parameters of a reflectance function into another parametric space, which we refer to as para-meterization, can improve approximation efficiency [9], [14], [15], [16]. However, previous articles have considered only fixed transformation functions, little attention has been paid to a data-dependent method [17], [18]. In this paper, we further propose to learn a set of optimized parameterization . Y.-T. Tsai is with the Department of Computer Science and Engineering,
Yuan Ze University, 135 Yuan-Tung Road, Chung-Li City, Taoyuan, Taiwan 320, R.O.C. E-mail: yttsai@saturn.yzu.edu.tw.
. K.-L. Fang, W.-C. Lin, and Z.-C. Shih are with the Department of Computer Science, National Chiao Tung University, 1001 University Road, Hsinchu, Taiwan 300, R.O.C. E-mail: zean.fang@gmail.com, {wclin, zcshih}@cs.nctu.edu.tw.
Manuscript received 19 Aug. 2009; revised 18 Feb. 2010; accepted 28 Sept. 2010; published online 29 Nov. 2010.
Recommended for acceptance by R. Ramamoorthi.
For information on obtaining reprints of this article, please send e-mail to: tpami@computer.org, and reference IEEECS Log Number
TPAMI-2009-08-0548.
Digital Object Identifier no. 10.1109/TPAMI.2010.211.
1. Throughout this paper, the univariate SRBF is referred to as the original SRBF that was introduced by Tsai and Shih [13].
functions for a given reflectance data set. By using a parametric representation to model the transformation functions, the parameterization process can be tightly integrated into our multivariate optimization framework. Previous fixed transformation methods, such as the halfway vector, thus become special cases in this general framework. It should be noted that the multivariate SRBF representa-tion and optimized parameterizarepresenta-tion only focus on BRDF modeling. For spatially varying materials like BTFs, we adopt the apparent BRDF representation [5], [10], [19], [20]. Since this representation describes a BTF as a set of texelwise BRDFs, we can apply the proposed model to separately approximate the reflectance data of each texel. However, directly optimizing the model parameters of each texel is time-consuming. We thus further propose a hierarchical fitting algorithm to exploit spatial coherence in a BTF and reduce the computational cost.
In summary, this paper makes the following contributions:
. A compact functional representation based on a
linear combination of multivariate SRBFs is intro-duced to efficiently model the complex behaviors of measured reflectance fields. Since our representation is a series of continuous functions, no additional interpolation or filtering techniques are required for rendering reflectance functions from novel illumina-tion and view direcillumina-tions at runtime.
. An automatic parameterization framework is
pro-posed to learn the best parameter transformation function from a given reflectance data set and a given form of transformation with unknown para-meters. It can seamlessly cooperate with our multi-variate representation to improve the approximation efficiency for reflectance functions.
. A hierarchical fitting algorithm for BTFs is presented to exploit spatial coherence and accelerate the approximation process. It is particularly suitable for multiresolution analysis and data-driven render-ing applications due to the inherent mipmap pyramid construction.
. The overall result of this paper is a compact and
hardware-friendly representation for BTFs, which can be easily implemented on modern graphics processing units (GPUs).
The remainder of this paper is organized as follows: First, the literature on parameterization and approximation meth-ods for surface appearance models is reviewed in Section 2. We then describe the main ideas of this paper by introducing the multivariate SRBF representation in Section 3 and the optimized parameterization framework in Section 4. A hierarchical fitting algorithm for BTFs and other practical implementation details, such as the initial guess of the multivariate SRBF representation, parameter optimization process, and runtime rendering, are, respectively, presented in Sections 5 and 6. Finally, we demonstrate and discuss the experimental results in Section 7, and conclude this paper in Section 8 to shed some lights on future research directions.
2
R
ELATEDW
ORKIn this section, we first briefly review some previous parameterization methods for reflectance functions (Sec-tion 2.1). We also summarize three main categories of
modern approximation algorithms for reflectance fields: functional linear models (Section 2.2), nonparametric mod-els (Section 2.3), and probabilistic modmod-els (Section 2.4). Due to limited paper length, our review mostly concentrates on BTFs. For a comprehensive survey on BTF modeling in computer vision and graphics, interested readers may further refer to [5].
2.1 Parameterization of Reflectance Functions
Halfway [15] and reflected vector [21] parameterizations for BRDFs have been shown to be effective in modeling highly specular materials. Stark et al. [22] also proposed several physically interpretable parameterizations for isotropic BRDFs. Although their method naturally forms a bary-centric coordinate system that contains some geometric information, it does not provide data-dependent parame-terizations for different real-world BRDFs. Namely, the parameterizations proposed in [22] are all fixed. In recent years, various fixed parameterizations have been applied to approximate spatially varying surface appearance [9], [16], further demonstrating their promising potentials. In gen-eral, parameterization is beneficial to reduce the dimen-sionality of reflectance functions, which leads to a compact and low-dimensional representation for surface appear-ance. It also greatly increases the data coherence that can be exploited by various approximation algorithms.
Nevertheless, previous parameterization techniques are limited to fixed transformation functions. It is unknown which parameter transformation would perform the best for the reflectance data at hand. Although Cole [17] introduced an automatic and data-dependent parameterization method for BRDFs, this approach is limited to linear transforma-tions. By contrast, we propose to learn the best parameter-ization functions from data, within a certain nonlinear functional form, by introducing additional parameters into our appearance representation. The proposed method thus successfully combines parameterization and reflectance model estimation in a unified framework to fill the gap between these two problems that have been solved separately in previous methods.
2.2 Functional Linear Models
Functional linear analysis of materials has received great attention in real-time rendering applications due to its compactness and efficiency. The main concept is to expand a reflectance function as a linear combination of simple basis functions. In this category, choosing an appropriate basis is one of the major research issues as it significantly influences image quality and rendering performance. Previous ap-proaches have modeled (spatially varying) reflectance fields using parametric kernels [7], [23], [24], [25], polynomials [26], [27], radial basis functions [12], [16], [28], [29], spherical harmonics [21], [30], [31], and wavelets [32]. In general, kernel and radial basis functions provide efficient runtime rendering performance and high image quality for all-frequency materials. However, they usually require compu-tationally expensive nonlinear optimization for parameter estimation, which becomes even worse and impractical when modeling materials with spatially varying reflectance. In this paper, we propose to describe a reflectance function as a weighted sum of the products of several
univariate basis functions, and take a step further to search for data-dependent parameterization for illumination and view variations. This general weighted sum-of-products representation not only models the complex multivariate behavior of reflectance functions, but also includes various popular parametric reflection models as its subset. More-over, under the proposed hierarchical optimization frame-work, our experimental results show that the time-consuming nonlinear parameter fitting process for BTFs can be robustly preconditioned and accelerated by a bottom-up approach, leading to a multiresolution repre-sentation and an efficient offline algorithm.
2.3 Nonparametric Models
Nonparametric models can be regarded as functional models that do not have predefined forms of basis functions. In this category, an appropriate basis is learned from data for an accurate representation, rather than prior information specified by researchers. The most popular approaches in computer vision and graphics include clustering and dimensionality reduction techniques, such as variants of principal component analysis [6], [14], [19], [33], [34], [35], [36], [37], [38], matrix factorization [9], [18], [39], [40], [41], tensor approximation [11], [42], and vector quantization [43], [44].
Although nonparametric methods are data-driven mod-els that yield accurate and flexible representations, the amount of compressed data is still cumbersome when compared to other categories of approximation algorithms. For BTFs, it is also difficult to achieve real-time performance for runtime analysis and rendering in computer vision and graphics applications. Additionally, special interpolation or estimation techniques are required to synthesize surface appearance from novel illumination and view directions that are not sampled in raw data. By contrast, our algorithm provides not only a higher compression ratio but also a real-time rendering rate with comparable image quality. Furthermore, novel view and illumination directions can be easily handled by our continuous multivariate model and parameterization, and spatial mipmap texture filtering for runtime rendering is fully supported and inherent in our hierarchical appearance representation.
2.4 Probabilistic Models
In this category, spatial correlations among appearance data are described with probability density functions so that similar appearance data can be synthesized from estimated parameters of distributions and noise maps [45], [46]. Recently, Haindl and Filip [8] further proposed a multiscale probabilistic BTF model based on the casual autoregressive random field and combined range maps to enhance the surface roughness of rendered objects.
Although probabilistic models can achieve a high compression ratio, their main goal is efficient and seamless BTF synthesis, not an optimal reconstruction of the original BTF data. Additionally, the runtime rendering process is slow and currently not GPU-friendly. By contrast, our algorithm can be easily implemented on modern GPUs and provides a better trade-off among compression ratio, image quality, and rendering performance.
3
M
ULTIVARIATESRBF
S3.1 Mathematical Formulation
Let ! and denote two points on the unit sphere SSm in IRmþ1. A univariate spherical radial basis function [13] on SSmis defined as a function Gðcos Þ ¼ Gð! Þ that depends on the geodesic distance between ! and . A popular example of univariate SRBFs is the univariate Gaussian SRBF kernel2:
Gðcos jÞ ¼ Gð! jÞ ¼ eð!Þ; ð1Þ
where 2 IR represents the bandwidth parameter that controls the concentration of a univariate SRBF, and is also known as the SRBF center. A univariate spherical function Fð!Þ 2 IR thus can be approximated with a linear combina-tion of J univariate SRBFs as
Fð!Þ X
J
j¼1
jGð! jjjÞ; ð2Þ
where j2 IR denotes the basis coefficient of the
jth univariate SRBF.
Nevertheless, there are two problems when applying (2) to model a reflectance function. First, the appearance of real-world materials is an effect of various physical factors. Whether these factors are visible or hidden, the observed reflectance distribution is often a function of at least two different variables, e.g., illumination and view directions. However, (2) is a univariate model that only takes a single direction on SSm into account. This suggests that a multi-variate representation may be more favorable to describe the complex behaviors of a reflectance function. Second, even though we can apply (2) to, respectively, model the reflection distribution for each illumination/view direction, the outcome is a discrete and often noncompact representa-tion. It is nontrivial to generalize this representation to estimate the distributions from novel illumination/view directions.
To represent the appearance of a reflectance field under different physical conditions, we can construct a multivariate SRBF from the product of several symmetric univariate SRBFs. For complex or heterogenous materials, multiple multivariate SRBFs can be further linearly mixed to derive a general weighted sum-of-products model. More formally, let ¼ f!ngNn¼1 and ¼ fngNn¼1denote two N-element point sets, with !n and n on the unit sphere SSmn in IRmnþ1. We define a multivariate SRBF on the Cartesian product space SSm1 SSmN as
Gj¼ G!1; . . . ; !Nj1; . . . ; N¼ YN n¼1
Gð!n nÞ; ð3Þ and the multivariate Gaussian SRBF kernel thus corre-sponds to
Gj; ¼ eP
N
n¼1ðnð!nnÞnÞ; ð4Þ
2. It is easy to verify that the normalized univariate Gaussian SRBF on SSm
is equivalent to the von Mises-Fisher distribution. This suggests that various techniques developed for von Mises-Fisher distributions can be applied to univariate Gaussian SRBFs with only minor modifications.
where ¼ fngNn¼1 is the set of bandwidth parameters of the involved univariate SRBFs, and is also called the SRBF center set. Similarly, an N-variate function F ðÞ 2 IR, with each variable defined on SSmn, can be approximated by a
weighted sum-of-products representation,
FðÞ X J j¼1 jG jj; j ¼X J j¼1 j YN n¼1 G!n j;njj;n ; ð5Þ where j¼ fj;ngNn¼1 and j¼ fj;ngNn¼1 are, respectively, the center set and the bandwidth set of the jth multi-variate SRBF.
3.2 Example
Consider a BRDF ð!l; !vÞ 2 IR, where !l and !v, respec-tively, denote the illumination and view directions on SS2. Based on (5), we can approximate ð!l; !vÞ as
ð!l; !vÞ XJ j¼1 jG !l j;!ljj;!l G!v j;!vjj;!v : ð6Þ
Note that (6) is similar to many factorization-based repre-sentations for BRDFs, e.g., principal component analysis [14] and nonnegative matrix factorization [47], but our multi-variate representation, like other parametric models, is more compact and it is also more intuitive to interpret or edit the derived parameters. This relation to matrix factorization methods also suggests the potential of applying our multi-variate SRBF representation to approximate reflectance functions. As we will present in Section 4, (6) can be further extended into a more general model than previous methods when combined with optimized parameterization.
4
O
PTIMIZEDP
ARAMETERIZATION4.1 Mathematical Formulation
Previous articles have reported that fixed parameterization for a reflectance function, such as the halfway and difference vectors [15], can significantly influence the performance of approximation algorithms. However, since a predefined parameterization method often relies on certain assumptions of material properties, it may be inadequate to handle various real-world reflectance data. For example, the halfway parameterization tends to align the specular peak of a reflectance function, but the shadowing and masking effects of microfacets are ignored. This situation will become even worse for a real-world BTF since it is usually measured over a rough surface with complex mesostructures and light scattering properties.
To overcome the disadvantages of fixed parameteriza-tion, we propose to learn a set of optimized transformation functions for a given reflectance data set. Since our goal is to obtain a compact reflectance representation, we choose to model the transformation functions using parametric equations. This particularly allows the parameterization process to be tightly integrated into our multivariate SRBF framework. Although the derived optimal solution is constrained to a certain functional form, our experimental results show that even a linear combination of the parameters of a reflectance function, followed by projection onto the unit sphere, can be more effective than previous fixed parameterization approaches. Finding the truly
optimal parameterization using nonparametric models thus is left as our future work.
More formally, let ðjÞ 2 SSm be a transformation function that depends on a given set of parameterization coefficients ¼ fi2 IRgIi¼1 , where I denotes the total number of parameterization coefficients in and is specified by users. We would like to find an optimal solution to so that a multivariate spherical function F ðÞ 2 IR can be efficiently approximated by transforming it into another univariate spherical function Fpð ðjÞÞ 2 IR that is more suitable for univariate SRBF expansions (2):
FðÞ ¼ Fp ðjÞ XJ
j¼1
jG ðjÞ jjj: ð7Þ From (7), it is also intuitive to extend the same concept to
transform F ðÞ into an Np-variate spherical function
FpðÞ 2 IR as FðÞ ¼ FpðÞ ¼ Fp 1ðj1Þ; . . . ; NpðjNpÞ ^FpðÞ ¼ XJ j¼1 j YNp n¼1 G nðjnÞ j;njj;n ; where ¼ f nðjnÞg Np n¼1 is a set of Np transformation functions, n¼ fi;ngIi¼1n specifies the parameterization coefficient set with In elements for the nth transformation
function, and ^FpðÞ denotes the approximate multivariate
SRBF representation of FpðÞ. Note that the number of
variables of FpðÞ, namely, Np, is not necessarily identical to that of F ðÞ, but, rather, can be specified by users. This flexibility particularly allows our representation to accu-rately model various complex behaviors of a real-world reflectance function.
In summary, we combine the proposed multivariate SRBF representation and optimized parameterization to derive the parameterized multivariate SRBF representation (8), and solve its parameters by minimizing the following objective function: E¼ Eerrþ Eaddl; ð9Þ where Eerr ¼ Z SSm1 Z SSmN FðÞ ^FpðÞ 2 d!1 d!N ð10Þ is the expected squared error between F ðÞ and ^FpðÞ, and Eaddl denotes the additional energy terms that should also be minimized for a robust and satisfying solution. For more details about Eaddl and the practical algorithm for solving (9), please refer to Sections 5 and 6.
4.2 Example
We again take the BRDF ð!l; !vÞ for an example. Based on (8), if we choose a trivariate SRBF representation ðNp¼ 3Þ, (6) can be expressed as ð!l; !vÞ XJ j¼1 j Y3 n¼1 G nð!l; !vjnÞ j;njj;n; ð11Þ while each transformation function can be modeled with the normalization of a linear combination of !land !vas follows:
nð!l; !vjnÞ ¼
1;n!lþ 2;n!v 1;n!lþ 2;n!v
2; ð12Þ
where 1;n and 2;nare the parameterization coefficients of
the nth transformation function and k k 2 denotes the
‘2 norm of a vector. Note that when J ¼ 1, (11) is similar to the mathematical formulation of homomorphic factoriza-tion [40], but our multivariate representafactoriza-tion allows a linear combination of multiple multivariate functions to be used to model heterogeneous materials, which is particularly important for representing BTF data sets. Moreover, (12) was inspired by the fact that many common parameteriza-tions, such as the halfway, illumination, and view vectors, are its special cases. It is also noteworthy that these three parameterizations were employed in the implementation of homomorphic factorization [40], which further implies the practical effectiveness of our model.
5
H
IERARCHICALF
ITTINGA
LGORITHMIn the previous two sections, we have introduced the multivariate SRBF representation (Section 3) and optimized parameterization (Section 4) to model a single reflectance function. To extend our method to model a BTF, we represent the BTF as a set of texelwise BRDFs and, respectively, approximate the reflectance data of each texel. However, this brute-force approach is actually time-consuming even with our GPU-based implementation (Section 6.2). Similarly to the multiresolution reflectance framework in [29], we present a hierarchical fitting algorithm to reduce the computational cost and preserve spatial coherence, while simultaneously constructing the mipmap pyramid for high-quality rendering on GPUs at runtime.
5.1 Overview
Our hierarchical fitting algorithm operates on a given BTF pyramid fBigIi¼0h and an initial guess of each texel at the coarsest level. As illustrated in Fig. 1, our algorithm consists of a sequence of upsampling and optimization stages from the coarsest level B0 to the finest level BIh. For a pyramid
level i > 0, the upsampling stage (Section 5.2) derives the initial solution of each texel at level i from ~Bi1, where ~Bi1 denotes the optimized results of level i1. Instead of using
traditional image interpolation techniques, such as bicubic or Lanczos filtering, we propose a joint least-squares upsampling algorithm by exploiting the relation between Biand Bi1to assist the resampling of ~Bi1.
After that, the optimization stage (Section 5.3) updates the initial solution of each texel at level i based on our parameterized multivariate SRBF representation. To take advantage of hardware texture filtering during runtime rendering, we introduce additional spatial smoothness energy terms in the objective function, which will constrain the parameter coherence of adjacent texels. The procedure “HierarchyOptimize” in Table 1 summarizes the pseudo-code of the overall fitting process. Note that finding an appropriate initial guess for the coarsest level is nontrivial. We postpone the discussion of this issue to Section 6.1. For implementation details about the procedure “Optimize” in Table 1, please refer to Section 6.2.
5.2 Upsampling Stage
Given the optimized parameters of pyramid level i1, namely, ~Bi1, an appropriate initial solution of each texel at level i is derived in the upsampling stage. This initial solution significantly influences the quality and
computa-tional cost for approximating Bi. However, since some
details of Bimay be lost when downsampled to Bi1during BTF pyramid construction, traditional image interpolation techniques are inadequate for a high-quality upsampling from ~Bi1. Our key observation is that because both Biand Bi1are available in this stage, their relation can be employed to “jointly” derive the initial guess of ~Bifrom ~Bi1. This relies on the assumptions that the relation between ~Biand ~Bi1is
Fig. 1. Approximate a BTF using the proposed hierarchical fitting algorithm.
TABLE 1
similar to that between Biand Bi1, and ~Bi1approximates Bi1with low reconstruction errors.
Specifically, a BTF texel t in Bican be approximated with a linear combination of a set of texels in Bi1:
BiðtÞ X t02N
i1ðtÞ
wt0Bi1ðt0Þ; ð13Þ
where Ni1ðtÞ represents the set of participating texels in Bi1 for t, and wt0 denotes the blending weight of a texel
t02 Ni1ðtÞ. In current implementation, we heuristically determine Ni1ðtÞ as the neighboring texels of Bi1ðtÞ within a user-defined window. Since both Bi and Bi1are known in this stage, the unknown blending weights can thus be derived by solving an unconstrained linear least-squares problem, and then applied to compute the initial solution of ~Bi as ~ BiðtÞ ¼ X t02N i1ðtÞ wt0B~i1ðt0Þ: ð14Þ
It should be noted that reconstructing the interpolated parameters of participating texels may not exactly corre-spond to interpolating their reconstructed values. For example, since the center and bandwidth sets are related to the exponents of multivariate Gaussian SRBFs, (14) will linearly blend these two sets of different texels in the logarithmic space, which is definitely not equivalent to the weighted summation of the Gaussian SRBFs of each texel. However, they would be close to each other if the spatial variations in the parameters of participating texels are smooth. Since the model parameters of level i1 were already updated with spatial smoothness energy terms in the optimization stage of previous iteration, we have found that the proposed joint least-squares upsampling algorithm works very well in practice.
5.3 Optimization Stage
In this stage, the initial guess of each texel at pyramid level i is individually updated to obtain a locally optimal solution that minimizes the objective function in (9). Similarly to the mixture model of [29], [48], we introduce an additional smoothness energy term Eaddl in (9) to guarantee spatial coherence in the derived parameters. Previous articles [29], [48] proposed aligning only centers of Gaussian/spherical functions, but we have found that other model parameters should also be appropriately aligned in our experiments. There are three main reasons for this. First, the alignment can be regarded as a regularization process that avoids overfitting. Second, the smoothness energy terms particu-larly allow much faster convergence for the optimization process. Third, linear interpolation, instead of nonlinear filtering [29], [48], on model parameters for efficient runtime performance (Section 6.3) can be employed if all of the model parameters are appropriately aligned. This would cause some slight loss of high-frequency features in the fitted and rendered results, but one may consider it as a trade-off between runtime performance and image quality.
In this paper, Eaddl is thus defined as follows:
Eaddl¼ Eþ Eþ Eþ ;E;þ E; ð15Þ E¼ X t02N0 iðtÞ XJ j¼1 ðjðtÞ jðt0ÞÞ2; ð16Þ E¼ X t02N0 iðtÞ XJ j¼1 XN n¼1 ð1 j;nðtÞ j;nðt0ÞÞ; ð17Þ E¼ X t02N0 iðtÞ XJ j¼1 XN n¼1 ðj;nðtÞ j;nðt0ÞÞ2; ð18Þ E;¼ X t02N0 iðtÞ XJ j¼1 XN n¼1 kj;nðtÞj;nðtÞ j;nðt0Þj;nðt0Þk22; ð19Þ E¼ X t02N0 iðtÞ XNp n¼1 XIn j¼1 ðj;nðtÞ j;nðt0ÞÞ2; ð20Þ where E, E, E, E;, and E are, respectively, the smoothness energy terms of basis coefficients, centers, bandwidths, and parameterization coefficients, N0iðtÞ de-notes the set of participating texels at levels i and i1 for t, and , , , ;, and are, respectively, the user-defined weights for E, E, E, E;, and E.
In this way, the smoothness energy terms in (15) will guide the model parameters of t to approach those of N0iðtÞ. Specifically, E, E, E;, and Ewill penalize large squared errors between the basis coefficients, bandwidths, and parameterization coefficients of t and N0iðtÞ, while E will minimize the cosine of angular differences between the centers of t and N0iðtÞ.
Note that (19) is specially designed for multivariate Gaussian SRBFs as their centers and bandwidths are highly coupled with each other. Moreover, the SRBF parameters in (15)-(20) should depend on the level index i and texel t, but we drop them for notational simplicity. Similarly to Ni1ðtÞ in the upsampling stage, N0iðtÞ is defined as the “valid” neighboring texels of t at levels i and i1 within a user-defined window, while a valid texel is referred to as the texel whose model parameters have ever been optimized. Since a change in the model parameters of t will influence those of N0iðtÞ, the above process is repeated until the parameters of each spatial location at level i converge or a user-defined maximum number of passes is reached.
6
I
MPLEMENTATIOND
ETAILS6.1 Initial Guess
Since the approximation quality of the proposed SRBF representation with optimized parameterization (8) signifi-cantly depends on the initial guess of model parameters, we propose a heuristic technique to determine an effective initial guess that reduces approximation errors and compu-tational cost. For the initial guess of parameterization coefficients, we have found that previous fixed parameter-izations generally provide an appropriate starting point if they are special cases of the adopted transformation functions. Take (12) for an example; the initial guess of the first three parameterization coefficient sets can be explicitly
set to the halfway, illumination, and view parameterizations, while the remainders are randomly generated.
Once the initial values of parameterization coefficients are determined, the initial guess of basis coefficients, center sets, and bandwidth sets can be estimated by treating a multi-variate reflectance function as multiple unimulti-variate functions, and iteratively processing one variable of the reflectance function at a time. Specifically, the key idea is that if we collect all of the reflectance data of a single variable (say !n), the resulting data set will be the observations of a univariate spherical function. Therefore, we can separately apply the scattered univariate SRBF representation [13] to approx-imate the observations of each univariate function, but additionally constrain that the representations for different data sets should employ the same center and bandwidth sets. After carefully examining the derived parameters, it is obvious that the basis coefficients form the observations of another multivariate spherical function without dependence on !n. The above process can thus be repeatedly performed to remove a single variable at each iteration until all model parameters are obtained.
Note that one can always process the variables of a multivariate function in an arbitrary order. It is also feasible to find the optimal order for a small number of variables by a brute-force approach. However, we do not consider this issue in current implementation. Developing an efficient technique for determining the optimal order is left as a possible research direction in the future.
6.2 Optimization Process
In our current implementation, we apply the L-BFGS-B solver [49], [50] to optimize the parameters of the proposed model. Instead of solving all the model parameters at the same time, we employ an iterative alternating least-squares method (the procedure “Optimize” in Table 1) that updates only one set of parameters at each step, while leaving the others unchanged. This scheme often yields better results, since the four sets of model parameters, including coeffi-cient, center, bandwidth, and parameterization coefficient sets, are highly coupled with each other.
During each iteration, the gradient computation is performed on GPUs using NVIDIA CUDA [51]. The computed results are then transferred from GPUs to the host memory for the L-BFGS-B solver to update model parameters on CPUs. Since the gradient computation is one of the main performance bottlenecks in the optimization process, we have found that this approach can reduce the computation time by a factor of 2 to 5.
6.3 Runtime Rendering
The rendering process of approximated BTFs based on multivariate SRBFs is quite simple and intuitive. To utilize mipmap texture filtering on GPUs, we concatenate the SRBF parameters at each level into several two-dimensional texture arrays,3with one (or more if necessary) texture array for one category of SRBF parameter sets.4For mesostructure
synthesis, we apply appearance-space texture synthesis [52] on raw data to obtain the spatial coordinate texture S.
The rendering process thus consists of the following steps:
1. For the current pixel p, sample the synthesized
texture S for the BTF spatial coordinates tp.
2. Sample the texture(s) for all of the SRBF parameters that correspond to tp.
3. The shading color of pixel p is then given by
performing the reconstruction according to the adopted multivariate SRBF representation.
For a pixel, note that we do not reconstruct the shading color of each participating texel and then perform mipmap filtering, but instead we filter the SRBF parameters of each participating texel first and reconstruct the final shading color. When the derived SRBF parameters of each BTF texel are smooth enough, this approach usually increases the rendering performance by a factor of about 6 for trilinear mipmap filtering without noticeable artifacts. In general, the performance gain strongly depends on the utilized filtering technique. The more sophisticated the filtering technique is, the more performance gain our approach can achieve.
7
R
ESULTS ANDD
ISCUSSION7.1 Experimental Results
The experiments of multivariate SRBF representation and optimized parameterization were conducted on a work-station with an Intel Core 2 Extreme QX9650 CPU, an NVIDIA GeForce GTX 280 graphics card, and 8 gigabytes of main memory. The measured BTFs were provided in courtesy of the University of California at San Diego [19], University of Bonn [36], and Dr. Xin Tong. The multivariate Gaussian SRBFs (4) were adopted to represent BTFs. In general, we have found no significant difference between various types of SRBFs for approximating BTFs, but Gaussian SRBFs are locally compact basis functions and handle most common cases very well. Additionally, the SRBF center and bandwidth sets were constrained to be the same for red, green, and blue channels of a BTF texel since separately fitting the data of each channel only slightly reduces approximation errors, but increases computational cost and storage space by a factor of 2 or more.
Table 2 compares the statistics of the proposed model and tensor approximation for modeling hierarchical BTF data sets, which includes the experimental results of N-mode singular value decomposition (N-SVD) [11], [42], [53], traditional fixed (Fix.), and optimized (Opt.) para-meterization. In this table, all compressed data were stored as half-precision (16-bit) floating point numbers [54], and the quality of compressed data is measured by signal-to-mean squared error ratio (“SE ratio”). Moreover, T1 and T2 denote different parameter settings of tensor approximation used for comparison under similar compression ratio and rendering speed, respectively.
User-defined constants in the proposed model, such as the number of SRBFs (J), the number of variates (Np), and the weights of smoothness energy terms, were manually tuned in the current implementation. We first conduct the experiment of a BTF at the coarsest level 0 with J ¼ 8 and 3. If texture arrays are not supported in the graphics application
programming interface, we may tile the mipmap of each parameter texture into one or more “big” textures. However, one should be careful not to include texels out of the tile boundaries in the texture filtering. This can be achieved by clamping texture coordinates into an appropriate range before sampling.
4. One may pack the SRBF center and bandwidth sets into one two-dimensional texture array to slightly reduce texture access time.
Np¼ 3, and gradually increase J or Npif the approximation error is large. Note that this only takes a little time to determine J and Npsince there are only a few (usually one) BTF texels at level 0. After that, the weights of smoothness energy terms are resolved and fine tuned in proportion to
the ‘2 norm of the apparent BRDF data for a BTF texel.
Typically, these weight values are in the interval from 105 to 102. The final weight values of each BTF are also listed in Table 2.
In our experiments, at most three traditional parameter-izations: halfway, illumination, and view directions were employed in the fixed parameterization. As for optimized parameterization, we instead utilized the parameterization function defined in (12). If the number of variates was more than three, namely, Np> 3, the optimization stage at the coarsest level 0 was performed multiple times to account for the randomness of the initial guess. Then, only the SRBF parameters with the lowest approximation errors for BTF texels at level 0 were employed in the subsequent upsampling and optimization stages at higher levels.
Note that we skip the reduction of spatial resolution for the results of tensor approximation since our multivariate SRBF representation can only handle directional variables. For fast BTF rendering based on tensor approximation, it is usually better not to reduce the spatial domain of a BTF. Therefore, we believe that this will not introduce an unfair comparison among these methods.
Fig. 2 demonstrates the reconstructed BTF images of the proposed model and compares them with those of tensor approximation based on N-SVD. From this figure and Table 2, the proposed optimized parameterization gener-ally outperforms the traditional fixed approach in terms of approximation errors and visual quality, especially when the BTF data sets exhibit complex mesostructures, specular reflectance, or sharp shadows. As shown in Fig. 2b, multivariate SRBFs tend to capture more sharp features in a BTF, such as specular highlights, which tensor approximation fails to preserve under similar rendering rate. In Section 7.2, we will further discuss the advantages and disadvantages of multivariate SRBF representation and tensor approximation in detail.
Fig. 3 demonstrates the images of optimized parameter-ization coefficients. In the experiments, we have found that parameterization coefficients tend to align with illumination and geometric features of a BTF, especially shadows, specular highlights, and uneven surfaces. For example, one can roughly perceive the distribution of surface normals of Hole and Wool from Fig. 3. For diffuse-like BTFs such as Wool, surface geometry often dominates the spatial variation of parameterization coefficients, and the solution of parameter-ization coefficients is generally not far away from our initial guess (halfway, illumination, and view directions) when the object surface is rather flat. As for BTFs with specular effects like Hole, illumination features are also very important to the spatial variation of parameterization coefficients. In high-light and shadow-covered regions, parameterization coeffi-cients generally changes more quickly. These interesting findings particularly open a connection between the results of our approach and surface normal estimation of a BTF, or even material/face recognition.
Fig. 4 further plots the squared error ratio of the BTF Hole versus the number of SRBFs based on the proposed model. This figure particularly shows the scalability of the proposed multivariate SRBF representation. The approx-imation error of a BTF can be gradually reduced by increasing the number of SRBFs. Although we currently do not have a theoretical proof on how to decide the number of SRBFs, our framework allows one to incremen-tally increase the number of SRBFs by updating previous optimized results. From our experiments, we have found that the more lighting and geometric saliences exhibited in the BTF (for example, more rapidly spatially or angularly varying shadows and specular highlights or rougher object surfaces), the larger the number of SRBFs needed to achieve a low approximation error, even for approximations with-out smoothness energy terms.
Table 3 and Fig. 5, respectively, present the runtime performance and rendered images of the proposed model and tensor approximation for various BTF data sets. In our experiments, the screen resolution and the number of directional light sources were, respectively, set to 640 480 and 2. In Table 3, we list the resolution of the synthesized coordinate texture of each BTF in the row “Coord. texture.”
TABLE 2
Statistics of the Proposed Model and Tensor Approximation for Multiresolution BTF Approximation
Note that the statistics in the row “Total data” also include the amount of synthesized texture data. Fig. 5 demonstrates that our approach achieves much faster rendering speed while maintaining comparable image quality compared to tensor approximation under similar compression ratio. For comparison under similar rendering speed, one can find that the optimized parameterization has smaller approximation error and compressed data size than the tensor approxima-tion (T2 in Tables 2 and 3). In particular, when a BTF data set exhibits specular reflectance and sharp shadows, the optimized parameterization outperforms the fixed parame-terization and tensor approximation (Fig. 6). In general, all
experimental results show that the proposed model can achieve a better trade-off between rendering performance and image quality than tensor approximation, especially when rendering time is a critical issue at runtime.
Due to the access overhead of additional parameters, the rendering performance of optimized parameterization is slightly slower than that of fixed parameterization, but there are no significant differences between them. For a medium-size model, both methods can easily achieve real-time rendering rates at runreal-time. Moreover, the runreal-time performance of multivariate SRBF representation is typi-cally faster than that of tensor approximation. This is owing to that tensor approximation needs extra computational Fig. 3. Images of optimized parameterization coefficients. (a) Hole. (b) Wool.
Fig. 4. Plot of the squared error ratio versus the number of SRBFs based on the proposed model for multiresolution BTF approximation (Hole). Fig. 2. Reconstructed BTF images of the proposed model and tensor
approximation. From left to right: Raw data, tensor approximation, fixed parameterization, optimized parameterization. In each subfigure, from top to bottom: Reconstructed images, absolute difference images scaled by 2. (a) Carpet. (b) Hole. (c) Impalla.
overhead and auxiliary texture data for efficient interpola-tion at runtime. By contrast, the proposed multivariate SRBF representation is a continuous parametric model, hence no additional interpolation techniques for smooth transitions across different illumination (or view) directions are required.
In Fig. 7, we also compare the effects of smoothness energy terms (Section 5.3) and hardware mipmap filtering accelera-tion. Figs. 7c and 7d were, respectively, generated using the approximated results of the proposed model without/with smoothness energy terms. In general, including the smooth-ness energy terms in (15) will significantly reduce the
Fig. 5. Comparison of rendered images under similar compression ratio. From top to bottom: Raw data, tensor approximation (T1), fixed parameterization, optimized parameterization. This figure illustrates that our approach can achieve similar rendering quality with much faster rendering speed under similar compression ratio. For the configurations of parameterizations and runtime rendering, please refer to Tables 2 and 3. (a) Bunny with Carpet. (b) Bunny with Hole. (c) Bunny with Sponge. (d) Cloth with Wool.
TABLE 3
computational cost of the hierarchical fitting process while only slightly increasing the approximation error of a BTF.
Note that for Figs. 7c and 7d, we disabled the built-in trilinear mipmap filtering of GPUs, reconstructed the shading color of each participating texel, and then per-formed mipmap filtering in shaders. By contrast, Fig. 7e was rendered by enabling hardware trilinear mipmap filtering to interpolate SRBF parameters first and then reconstructing the final shading color. This will greatly increase rendering performance without noticeable defects on image quality when the derived SRBF parameters of a BTF are smooth enough.
7.2 Discussion and Limitations
There are some previous articles that employed basis functions similar to SRBFs to approximate a reflectance function or BTF [12], [28], [29]. The most significant difference between our approach and previous approaches is that we propose multivariate SRBFs to achieve an accurate and efficient representation, while the basis functions adopted in previous work are univariate. This particularly allows us to multilaterally combine various directional factors to describe a reflectance function, which was ignored in previous approaches. Moreover, Green et al. [28] proposed to parameterize a BRDF so that it can be efficiently represented using a mixture of isotropic Gaussians for light transport problems, but their fixed parameterizations may
not be adequate to represent complex real-world reflectance functions. By contrast, our approach utilizes optimized parameterization that is data-dependent. Tan et al. [29] and Wang et al. [12] also applied a mixture of isotropic/spherical Gaussians to represent the distribution of normals in the traditional microfacet model. Specifically, their methods [12], [29] rely on physical assumptions, such as Fresnel and shadowing terms, to derive the simple normal distribution function for reflectance data fitting, which additionally needs to estimate surface normals for the BTF of a rough surface. By contrast, we do not assume any physical properties of the available BTF data, and implicitly incorporate the traditional physical terms into our model. In this way, a single multivariate SRBF can be regarded as data dependently including/approximating many tradi-tional physical properties in terms of different variates.
Table 4 compares the features of multivariate SRBFs and tensor approximation for BTFs. Comparisons of these two compression methods are similar to the traditional debates between parametric and nonparametric models in the statistics and machine learning communities. In general, multivariate SRBFs (parametric models) lead to more efficient rendering performance at runtime, while tensor approximation (nonparametric models) provides a more accurate representation for visual data sets.
An additional advantage of multivariate SRBFs is that approximating a hierarchical set of multiresolution BTFs can
Fig. 7. Effects of smoothness energy terms and hardware texture filtering. (a) Whole rendered result using raw data, (b) raw data, (c) without smoothness, (d) with smoothness, (e) with smoothness and hardware texture filtering. The squared error ratio, compression time, and rendering performance of each approximated BTF are as follows: (c) 0.59 percent, 502.71 hr., 54.27 FPS; (d) 0.81 percent, 5.21 hr., 54.43 FPS; (e) 0.81 percent, 5.21 hr., 311.9 FPS.
Fig. 6. Comparison of rendered images under similar rendering speed. (a) The whole rendered image of optimized parameterization. (b), (c), (d), and (e) The enlarged images generated by different models. One can observe that the optimized parameterization better preserves specular effects and sharp shadows. For the configurations of approximation methods and runtime rendering, please refer to Tables 2 and 3. (a) Optimized. (b) Raw. (c) Tensor (T2). (d) Fixed. (e) Optimized.
be achieved using the proposed hierarchical fitting algo-rithm, while sophisticated runtime mipmap texture filtering can also be readily performed on GPUs by including smoothness energy terms in the objective function. Never-theless, approximating a BTF with multivariate SRBFs is computationally expensive (typically several hours) due to the nonlinear optimization process, even if our implementa-tion already achieves considerable acceleraimplementa-tion with GPUs. On the contrary, tensor approximation usually takes only tens of minutes to decompose a given BTF data set since only linear algebra operations are needed. Thus, there is always a trade-off between offline and runtime costs for these two categories of compression methods.
For the proposed optimized parameterization, it is possible to adopt other parameterization functions other than (12). Any other parameterization functions can be easily integrated into the proposed model as long as they will not lead to difficult gradient computation in the optimization process. For instance, one may formulate the parameteriza-tion funcparameteriza-tions as symmetric 3 3 matrix transformaparameteriza-tions [23] or linear transformations in the spherical coordinate system [17]. It may also be effective to first apply linear or nonlinear projections to obtain some special parameteriza-tions [15], [16], [18], [21], [22], and then linearly combine them to construct final parameterization functions. These can be considered as generalizations of previous traditional parameterizations. In the current implementation, we choose (12) simply because it includes popular fixed parameteriza-tions (halfway, illumination, and view direcparameteriza-tions) as its subset and results in efficient parameter estimation.
There are also some disadvantages and limitations of the proposed model:
. As with many alternating optimization algorithms,
the stability of estimated SRBF parameters is influ-enced by the initial guess and the alternating order.
. User-defined constants in the proposed model need
to be manually tuned for different BTFs.
. The smoothness energy terms (Section 5.3) and
linear interpolation on model parameters at runtime (Section 6.3) inevitably decrease approximation quality and result in some artifacts. It is expected that high-frequency features in BTFs will be slightly smoothed or even lost after approximation.
In the current implementation, we propose a heuristic approach to determine a reasonable initial guess and ignore the effects of the alternating order. We have also found that bounding the values of SRBF parameters can increase the stability of estimated parameters. In our experience, it is recommended to bound coefficients in the interval ½bmax;þbmax, where bmax is the maximum absolute value of input BTF data, bandwidths in the interval ½32; þ32, and parameterization coefficients in the interval ½1; þ1.
Moreover, when multivariate Gaussian SRBFs are adopted and rendering performance is not a major concern, one may instead align only SRBF centers and parameteriza-tion coefficients ((17) and (20)) to preserve more high-frequency features in BTFs. Nevertheless, the nonlinear filtering technique [48] should be applied to interpolate SRBF centers and bandwidths at runtime, while linear interpolation is employed for other model parameters. Note that this can improve image quality, but rather reduce runtime performance.
8
C
ONCLUSIONS ANDF
UTUREW
ORKIn this paper, we have introduced a novel data representa-tion for BTFs. Based on multivariate SRBFs, reflectance functions can be modeled in their intrinsic spherical domain to avoid artifacts that result from false boundaries, distor-tions, and unnecessary parameterization. Most existing methods were not originally developed to handle spherical functions and deficient in this important feature. Therefore, they have to model a spherical function in an inappropriate domain rather than the unit hypersphere. Moreover, while a sum-of-products model with multivariate SRBFs provides an intrinsic and efficient description of multivariate spherical functions, optimized parameterization overcomes the major disadvantage of traditional fixed parameteriza-tions by learning data-dependent transformation funcparameteriza-tions. Experimental results reveal that multivariate SRBFs and optimized parameterization can be seamlessly integrated to obtain a practical solution of photorealistic BTF rendering at real-time rates. Finally, our approach for computing the optimal parameterization and SRBF coefficients of BTF data can be potentially applied to several computer vision problems, such as surface normal estimation, material classification, and object recognition when the appearances of materials or objects under various illumination and view directions are available.
In the current model, the proposed multivariate SRBF representation only employs one type of SRBFs. An obvious question is whether we can utilize more than one type of SRBFs to approximate observations on the unit hypersphere or not. The answer is certainly “yes.” We may achieve this simply by weightedly summing different types of multi-variate SRBFs. For a multimulti-variate SRBF, it can even be constructed from the product of different types of uni-variate SRBFs. Nevertheless, the real question is whether or not this sophisticated approach will outperform the current SRBF representation? This may need more experiments to reach a final conclusion.
The proposed framework of optimized parameterization relies on a predefined parametric model. Hence, its optimality is built upon a specific functional form. Finding
TABLE 4
Feature Comparisons between Multivariate SRBFs and Tensor Approximation for BTF Modeling
*Recall that multivariate SRBFs can only handle directional variables, but tensor approximation can reduce the dimensionality of all kinds of variables at the same time. We currently do not apply any other approximation methods to compress the spatial domain of SRBF parameters for a BTF.
the truly optimal parameterizations for a given visual data set is still a challenging problem. We plan to investigate this issue using nonparametric models in the future. Moreover, the proposed optimized parameterization framework only supports directional variables, but real-world visual data sets may also depend on other types of physical factors. These nondirectional factors should not be excluded from parameterization. For example, we may additionally take the spatial variables of a BTF into account for parameter-ization. In this way, spatially varying characteristics of the BTF can be implicitly modeled in a unified framework.
A
CKNOWLEDGMENTSThe authors would like to thank the anonymous reviewers for profound comments and suggestions, Dr. Xin Tong for providing BTF data, and Jia-Yin Ji for preparing the model Cloth. This work was supported in part by the National Science Council of Taiwan under Grant No. NSC96-2221-E-009-152-MY3, NSC99-2628-E-009-178, and NSC100-2218-E-155-002.
R
EFERENCES[1] S.J. Gortler, R. Grzeszczuk, R. Szeliski, and M.F. Cohen, “The Lumigraph,” Proc. ACM SIGGRAPH, pp. 43-54, 1996.
[2] M. Levoy and P. Hanrahan, “Light Field Rendering,” Proc. ACM SIGGRAPH, pp. 31-42, 1996.
[3] S.M. Seitz and C.R. Dyer, “View Morphing,” Proc. ACM SIGGRAPH, pp. 21-30, 1996.
[4] K.J. Dana, B. Van Ginneken, S.K. Nayar, and J.J. Koenderink, “Reflectance and Texture of Real-World Surfaces,” ACM Trans. Graphics, vol. 18, no. 1, pp. 1-34, 1999.
[5] J. Filip and M. Haindl, “Bidirectional Texture Function Modeling: A State of the Art Survey,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 11, pp. 1921-1940, Nov. 2009. [6] O.G. Cula and K.J. Dana, “Compact Representation of
Bidirec-tional Texture Functions,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 1041-1047, 2001.
[7] D.B. Goldman, B. Curless, A. Hertzmann, and S.M. Seitz, “Shape and Spatially-Varying BRDFs from Photometric Stereo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 6, pp. 1060-1071, June 2010.
[8] M. Haindl and J. Filip, “Extreme Compression and Modeling of Bidirectional Texture Function,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 10, pp. 1859-1865, Oct. 2007. [9] J. Lawrence, A. Ben-Artzi, C. DeCoro, W. Matusik, H. Pfister, R.
Ramamoorthi, and S. Rusinkiewicz, “Inverse Shade Trees for Non-Parametric Material Representation and Editing,” ACM Trans. Graphics, vol. 25, no. 3, pp. 735-745, 2006.
[10] G. Mu¨ller, R. Sarlette, and R. Klein, “Data-Driven Local Coordinate Systems for Image-Based Rendering,” Computer Graphics Forum, vol. 25, no. 3, pp. 369-378, 2006.
[11] M.A.O. Vasilescu and D. Terzopoulos, “TensorTextures: Multi-linear Image-Based Rendering,” ACM Trans. Graphics, vol. 23, no. 3, pp. 336-342, 2004.
[12] J. Wang, P. Ren, M. Gong, J. Snyder, and B. Guo, “All-Frequency Rendering of Dynamic, Spatially-Varying Reflectance,” ACM Trans. Graphics, vol. 28, no. 5, pp. 1-10, 2009.
[13] Y.-T. Tsai and Z.-C. Shih, “All-Frequency Precomputed Radiance Transfer Using Spherical Radial Basis Functions and Clustered Tensor Approximation,” ACM Trans. Graphics, vol. 25, no. 3, pp. 967-976, 2006.
[14] J. Kautz and M.D. McCool, “Interactive Rendering with Arbitrary BRDFs Using Separable Approximations,” Proc. 10th Eurographics Workshop Rendering, pp. 247-260, 1999.
[15] S. Rusinkiewicz, “A New Change of Variables for Efficient Brdf Representation,” Proc. Ninth Eurographics Workshop Rendering, pp. 11-22, 1998.
[16] T. Zickler, R. Ramamoorthi, S. Enrique, and P.N. Belhumeur, “Reflectance Sharing: Predicting Appearance from a Sparse Set of Images of a Known Shape,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 8, pp. 1287-1302, Aug. 2006. [17] F.H. Cole, “Automatic BRDF Factorization,” master’s thesis,
Harvard Univ., 2002.
[18] P. Peers, K. Vom Berge, W. Matusik, R. Ramamoorthi, J. Lawrence, S. Rusinkiewicz, and P. Dutre´, “A Compact Factored Representa-tion of Heterogeneous Subsurface Scattering,” ACM Trans. Graphics, vol. 25, no. 3, pp. 746-753, 2006.
[19] M.L. Koudelka, S. Magda, P.N. Belhumeur, and D.J. Kriegman, “Acquisition, Compression, and Synthesis of Bidirectional Texture Functions,” Proc. Third Int’l Workshop Texture Analysis and Synthesis, pp. 59-64, 2003.
[20] G. Mu¨ller, J. Meseth, M. Sattler, R. Sarlette, and R. Klein, “Acquisition, Synthesis, and Rendering of Bidirectional Texture Functions,” Computer Graphics Forum, vol. 24, no. 1, pp. 83-109, 2005.
[21] R. Ramamoorthi and P. Hanrahan, “Frequency Space Environ-ment Map Rendering,” ACM Trans. Graphics, vol. 21, no. 3, pp. 517-526, 2002.
[22] M.M. Stark, J. Arvo, and B.E. Smits, “Barycentric Parameteriza-tions for Isotropic BRDFs,” IEEE Trans. Visualization and Computer Graphics, vol. 11, no. 2, pp. 126-138, Mar./Apr. 2005.
[23] E.P.F. Lafortune, S.-C. Foo, K.E. Torrance, and D.P. Greenberg, “Non-Linear Approximation of Reflectance Functions,” Proc. ACM SIGGRAPH, pp. 117-126, 1997.
[24] D.K. McAllister, A. Lastra, and W. Heidrich, “Efficient Rendering of Spatial Bi-Directional Reflectance Distribution Functions,” Proc. Graphics Hardware ’02, pp. 79-88, 2002.
[25] G.J. Ward, “Measuring and Modeling Anisotropic Reflection,” Proc. ACM SIGGRAPH, pp. 265-272, 1992.
[26] T. Malzbender, D. Gelb, and H.J. Wolters, “Polynomial Texture Maps,” Proc. ACM SIGGRAPH, pp. 519-528, 2001.
[27] J. Meseth, G. Mu¨ller, and R. Klein, “Reflectance Field Based Real-Time, High-Quality Rendering of Bidirectional Texture Func-tions,” Computers & Graphics, vol. 28, no. 1, pp. 105-112, 2004. [28] P. Green, J. Kautz, and F. Durand, “Efficient Reflectance and
Visibility Approximations for Environment Map Rendering,” Computer Graphics Forum, vol. 26, no. 3, pp. 495-502, 2007. [29] P. Tan, S. Lin, L. Quan, B. Guo, and H.-Y. Shum,
“Multi-resolution Reflectance Filtering,” Proc. Eurographics Symp. Render-ing, pp. 111-116, 2005.
[30] P.-P.J. Sloan, X. Liu, H.-Y. Shum, and J. Snyder, “Bi-Scale Radiance Transfer,” ACM Trans. Graphics, vol. 22, no. 3, pp. 370-375, 2003. [31] P.-P.J. Sloan, J. Kautz, and J. Snyder, “Precomputed Radiance
Transfer for Real-Time Rendering in Dynamic, Low-Frequency Lighting Environments,” ACM Trans. Graphics, vol. 21, no. 3, pp. 527-536, 2002.
[32] W.-C. Ma, S.-H. Chao, Y.-T. Tseng, Y.-Y. Chuang, C.-F. Chang, B.-Y. Chen, and M. Ouhyoung, “Level-of-Detail Representation of Bidirectional Texture Functions for Real-Time Rendering,” Proc. Symp. Interactive 3D Graphics and Games, pp. 187-194, 2005. [33] W.-C. Chen, J.-Y. Bouguet, M.H. Chu, and R. Grzeszczuk, “Light
Field Mapping: Efficient Representation and Hardware Rendering of Surface Light Fields,” ACM Trans. Graphics, vol. 21, no. 3, pp. 447-456, 2002.
[34] G. Mu¨ller, J. Meseth, and R. Klein, “Fast Environmental Lighting for Local-PCA Encoded BTFs,” Proc. Computer Graphics Int’l, pp. 198-205, 2004.
[35] K. Nishino, Y. Sato, and K. Ikeuchi, “Eigen-Texture Method: Appearance Compression and Synthesis Based on a 3D Model,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1257-1265, Nov. 2001.
[36] M. Sattler, R. Sarlette, and R. Klein, “Efficient and Realistic Visualization of Cloth,” Proc. Eurographics Symp. Rendering, pp. 167-178, 2003.
[37] P.-P.J. Sloan, J. Hall, J.C. Hart, and J. Snyder, “Clustered Principal Components for Precomputed Radiance Transfer,” ACM Trans. Graphics, vol. 22, no. 3, pp. 382-391, 2003.
[38] D.N. Wood, D.I. Azuma, K. Aldinger, B. Curless, T. Duchamp, D. Salesin, and W. Stuetzle, “Surface Light Fields for 3D Photo-graphy,” Proc. ACM SIGGRAPH, pp. 287-296, 2000.
[39] L. Latta and A. Kolb, “Homomorphic Factorization of BRDF-Based Lighting Computation,” ACM Trans. Graphics, vol. 21, no. 3, pp. 509-516, 2002.
[40] M.D. McCool, J. Ang, and A. Ahmad, “Homomorphic Factoriza-tion of BRDFs for High-Performance Rendering,” Proc. ACM SIGGRAPH, pp. 171-178, 2001.
[41] F. Suykens, K. vom Berge, A. Lagae, and P. Dutre´, “Interactive Rendering with Bidirectional Texture Functions,” Computer Graphics Forum, vol. 22, no. 3, pp. 463-472, 2003.
[42] H. Wang, Q. Wu, L. Shi, Y. Yu, and N. Ahuja, “Out-of-Core Tensor Approximation of Multi-Dimensional Matrices of Visual Data,” ACM Trans. Graphics, vol. 24, no. 3, pp. 527-535, 2005.
[43] T.K.-H. Leung and J. Malik, “Representing and Recognizing the Visual Appearance of Materials Using Three-Dimensional Tex-tons,” Int’l J. Computer Vision, vol. 43, no. 1, pp. 29-44, 2001. [44] J. Filip, M.J. Chantler, P.R. Green, and M. Haindl, “A
Psychophy-sically Validated Metric for Bidirectional Texture Data Reduc-tion,” ACM Trans. Graphics, vol. 27, no. 5, pp. 1-11, 2008. [45] M. Haindl and J. Filip, “Fast BTF Texture Modelling,” Proc. Third
Int’l Workshop Texture Analysis and Synthesis, pp. 47-52, 2003. [46] M. Haindl, J. Grim, P. Pudil, and M. Kudo, “A Hybrid BTF Model
Based on Gaussian Mixtures,” Proc. Fourth Int’l Workshop Texture Analysis and Synthesis, pp. 95-100, 2005.
[47] J. Lawrence, S. Rusinkiewicz, and R. Ramamoorthi, “Efficient BRDF Importance Sampling Using a Factored Representation,” ACM Trans. Graphics, vol. 23, no. 3, pp. 496-505, 2004.
[48] C. Han, B. Sun, R. Ramamoorthi, and E. Grinspun, “Frequency Domain Normal Map Filtering,” ACM Trans. Graphics, vol. 26, no. 3, 2007.
[49] R.H. Byrd, P. Lu, J. Nocedal, and C. Zhu, “A Limited Memory Algorithm for Bound Constrained Optimization,” SIAM J. Scientific Computing, vol. 16, no. 5, pp. 1190-1208, 1995. [50] C. Zhu, R.H. Byrd, P. Lu, and J. Nocedal, “Algorithm 778:
L-BFGS-B: Fortran Subroutines for Large-Scale Bound-Constrained Opti-mization,” ACM Trans. Math. Software, vol. 23, no. 4, pp. 550-560, 1997.
[51] NVIDIA, “NVIDIA CUDA: Compute Unified Device Architec-ture,” http://www.nvidia.com/cuda/, 2010.
[52] S. Lefebvre and H. Hoppe, “Appearance-Space Texture Synth-esis,” ACM Trans. Graphics, vol. 25, no. 3, pp. 541-548, 2006. [53] L. De Lathauwer, B. De Moor, and J. Vandewalle, “On the Best
Rank-1 and Rank-ðR1; R2; . . . ; RnÞ Approximation of
Higher-Order Tensors,” SIAM J. Matrix Analysis and Applications, vol. 21, no. 4, pp. 1324-1342, 2000.
[54] ILM, “Technical Introduction to OpenEXR,” Industrial Light & Magic, http://www.openexr.com/documentation.html, 2006.
Yu-Ting Tsai received the BS and MS degrees in electronics engineering from National Chiao Tung University in 2000 and 2002, respectively, and the PhD degree in computer science from National Chiao Tung University in 2009. Cur-rently, he is an assistant professor in the Department of Computer Science and Engineer-ing at Yuan Ze University. His research interests include computer graphics, computer vision, machine learning, and signal processing.
Kuei-Li Fang received the BS degree from National Dong Hua University in 2005 and the MS degree from National Chiao Tung University in 2008, respectively. He is interested in real-time rendering and game development. After graduation from National Chiao Tung University, he decided to devote himself to the game industry.
Wen-Chieh Lin received the BS and MS degrees in control engineering from National Chiao-Tung University, Hsinchu, Taiwan, in 1994 and 1996, respectively, and the PhD degree in robotics from Carnegie Mellon Uni-versity, Pittsburgh, Pennsylvania, in 2005. Since 2006, he has been with the Department of Computer Science and the Institute of Multi-media Engineering, National Chiao-Tung Uni-versity, as an assistant professor. His current research interests include computer graphics, computer animation, and computer vision. He is a member of the IEEE and the ACM.
Zen-Chung Shih received the BS degree in computer science from Chung-Yuan Christian University in 1980, the MS degree in 1982 and the PhD degree in computer science from National Tsing Hua University, Taiwan, in 1985. Currently, he is a professor in the Department of Computer Science and Institute of Multimedia Engineering at National Chiao Tung University in Hsinchu. He is a member of the IEEE and the ACM. His current research interests include procedural texture synthesis, nonphotorealistic render-ing, real-time renderrender-ing, and stylized rendering.
. For more information on this or any other computing topic, please visit our Digital Library at www.computer.org/publications/dlib.
