網格化簡與混合式顯像技術及其在即時顯像的應用

全文

(1)國立交通大學資訊工程系博士論文. 網格化簡與混合式顯像技術及其在即時顯像的應用 Mesh Simplification and Hybrid Rendering Techniques for Real-Time Rendering. 研究生：陳治君指導教授：莊榮宏. 教授. 中華民國九十五年十月.

(2) 網格化簡與混合式顯像技術及其在即時顯像的應用 Mesh Simplification and Hybrid Rendering Techniques for Real-Time Rendering. 研究生：陳治君. Student：Chih-Chun Chen. 指導教授：莊榮宏. Advisor：Jung-Hong Chuang. 國立交通大學資訊工程系博士論文. A Dissertation Submitted to Department of Computer Science College of Computer Science National Chiao Tung University in partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in. Computer Science October 2006 Hsinchu, Taiwan, Republic of China. 中華民國九十五年十月.

(3) 網格化簡與混合式顯像技術及其在即時顯像的應用. 學生：陳治君. 指導教授：莊榮宏教授. 國立交通大學資訊工程學系博士班. 摘. 要. 即時顯像是互動應用，如：遊戲、虛擬實境以及虛擬模擬中核心的技術。即時顯像技術包含了幾何為基礎、影像為基礎以及混合式顯像技術。這本論文中我們首先提出兩個全新的幾何顯像式技術，而後提出一個混和式顯像系統架構。在幾個以幾何為基礎的即時顯像之關鍵技術中，多層次模型建構技術為其中極其重要而且相當廣為流傳的即時顯像技術。我們發展出一個全新、簡單且有效的貼圖適應性調整機制來消除在漸進式網格模型配合貼圖時產生的貼圖扭曲現象。以此機制為基礎，我們提出一個新的幾何化簡之曲面與曲面間誤差估測方式，能與現有的方法相競爭甚至更好。混合式顯像技術在遊戲及虛擬實境中已被證實為非常有用的幾何式顯像的輔助技術。我們提出一個混合式顯像機制，利用多層次模型與貼圖並探索可見性的區域連貫性，以額外的儲存空間及預先讀取來做影像品質與顯像效率的取捨。為了支援漸進式網格模型的貼圖，我們常讓整個漸進式網格序列共享一個共有的貼圖。這個共有貼圖能用適當的幾何參數化方法考慮幾何與貼圖壓縮值，甚至考慮幾何邊折疊時的貼圖誤差得來。我們發現即使用很好的參數化貼圖，漸進式網格模型貼上貼圖時很容易產生明顯的貼圖扭曲現象，這主要是因幾何的變化與貼圖硬體的線性內差所致。在這論文中，我們提出一個全新、簡單且很有效率的方法針對每個幾何邊折疊來調整貼圖內容，以消除貼圖扭曲現象。貼圖之適應性調整機制及其反運算都是區域性及累進式運算且能以現有的圖學加速卡直.

(4) 接支援。我們另提出索引貼圖的機制來降低當調整貼圖時可能造成取樣不足進而產生的影像糊化現象。實驗結果顯示，在測試的例子中，貼圖扭曲現象幾乎都能被貼圖之適應性調整機制所消除。基於貼圖之適應性調整機制，我們提出一個新的以映像為基礎的誤差估測法，在不好的參數貼圖下，能夠比 APS 或 QEM 提供更精確的幾何誤差量測。從實驗結果得知，我們提出的方法比 APS 好上許多，而幾乎接近 QEM 的結果。在我們所提出的混合式顯像架構下，場景先被分割成一些蜂巢式的空間，再對蜂巢內的物體用一般的顯像技術來顯像，而蜂巢外的物體則用多層次貼圖幾何配合投影式貼圖法來顯像。多層次貼圖幾何為一以物體為基礎，由原始幾何依據所得到的深度影像化簡而得來，而深度影像則是在蜂巢與其鄰近蜂巢中心點顯像取得。利用這個多層次貼圖幾何，許多在混合式顯像技術中常被發現的問題，如：因物體與物體間遮檔而產生的洞的問題，以及因解析度不一致所產生的裂縫問題都能被消除。而因為物體自身遮檔產生的洞的問題，也能被限制在使用者指定的範圍內。在我們的顯像架構下，數個由上百萬個三角片所組成的複雜場景，都可達到高於每秒鐘 600 個畫面的平均顯像速率，而且只有些微的影像失真。.

(5) MESH SIMPLIFICATION AND HYBRID RENDERING TECHNIQUES FOR REAL-TIME RENDERING. A Dissertation Submitted to Department of Computer Science College of Computer Science National Chiao Tung University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in. Computer Science. Student: Chih-Chun Chen Adviser: Prof. Jung-Hong Chuang. October 2006. Hsinchu, Taiwan, Republic of China.

(6) c Copyright by Chih-Chun Chen 2006. All Rights Reserved. ii.

(7) I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy.. Prof. Jung-Hong Chuang. Principal Adviser. I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy.. Prof. Zen-Chung Shih. I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy.. Assistant Prof. I-Chen Lin. Approved for the University Committee on Graduate Studies.. iii.

(8) Acknowledgements I would like to thank my wife and my family. Without their supports I may not be able to get the Ph.D. degree. I deeply appreciate my adviser Prof. J. H. Chuang not only for his creative suggestions and discussions on my studies, but also his insightful advisements for the philosophy of life. Finally, I also like to thank all the members of Computer Graphics & Geometry Modeling Lab. for coloring my life during the studies.. This dissertation is dedicated to my wife Mei-Ying and my child who will be born around Dec. 2006.. iv.

(9) Contents Acknowledgements. iv. 1 Introduction. 1. 1.1. Main Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 3. 1.2. Dissertation Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 4. 2 Related Work. 5. 2.1. Real-Time Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 5. 2.2. Mesh Simplification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 6. 2.2.1. Level-of-detail modeling . . . . . . . . . . . . . . . . . . . . . . . . .. 6. 2.2.2. Progressive meshes . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 7. 2.2.3. Error metric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 10. 2.2.4. Incremental vs. total error metric . . . . . . . . . . . . . . . . . . . . .. 14. 2.2.5. Vertex-to-plane vs. surface-to-surface error . . . . . . . . . . . . . . .. 15. 2.2.6. Texture distortion . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 15. 2.3. Visibility Culling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 17. 2.4. Hybrid Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 17. 2.4.1. Region-based sprite . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 18. 2.4.2. Environment-based mesh . . . . . . . . . . . . . . . . . . . . . . . . .. 18. 2.4.3. Object-based Textured LOD mesh . . . . . . . . . . . . . . . . . . . .. 19. 3 Texture Adaptation for Progressive Meshes. v. 20.

(10) 3.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 20. 3.2. Texture Adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 23. 3.2.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 23. 3.2.2. Cell correspondence between two consecutive levels . . . . . . . . . .. 24. 3.2.3. Texture adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 27. 3.2.4. Indexing map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 27. Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 30. 3.3.1. Preprocessing time . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 32. 3.3.2. Reversibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 32. 3.3.3. Multiple charts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 37. 3.3. 4 A New Mapping-Based Error Metric. 38. 4.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 38. 4.2. A New Mapping-Based Error Metric . . . . . . . . . . . . . . . . . . . . . . .. 39. 4.2.1. Maximum error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 40. 4.2.2. Average error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 41. Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 42. 4.3. 5 Hybrid Rendering Based on Viewcell Dependent Textured LOD 5.1. 5.2. 5.3. 49. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 49. 5.1.1. View-cell dependent Textured LOD Modeling . . . . . . . . . . . . . .. 50. 5.1.2. System overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 51. Hybrid Rendering Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 52. 5.2.1. Preprocessing phase . . . . . . . . . . . . . . . . . . . . . . . . . . .. 54. 5.2.2. Run-time phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 64. Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 70. 5.3.1. Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 70. 5.3.2. Image quality measurement . . . . . . . . . . . . . . . . . . . . . . .. 72. 5.3.3. Mesh simplification . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 73. 5.3.4. Run-time performance . . . . . . . . . . . . . . . . . . . . . . . . . .. 75. vi.

(11) 5.3.5. Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 79. 6 Conclusion. 84. Bibliography. 86. vii.

(12) List of Tables 3.1. Speedup by packing texture adaptations. . . . . . . . . . . . . . . . . . . . . .. 31. 3.2. The entire run-time cost of simplifying PM w/ and w/o texture adaptation. . . .. 31. 3.3. The preprocessing time for constructing the entire PM sequence of the test models. 34. 4.1. The preprocessing time for constructing the entire PM sequence w/o texture adaptation under different error metrics. . . . . . . . . . . . . . . . . . . . . .. 4.2. The preprocessing time for constructing the entire PM sequence w/ texture adaptation under different error metrics. . . . . . . . . . . . . . . . . . . . . .. 5.1. 47. 48. Maximum ratio of side faces seen from a point inside the cell under different FOVs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 55. 5.2. Object and scene statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 71. 5.3. Simplification performance under different self-occluding-error tolerance Ts . . .. 73. 5.4. Simplification performance under different projected edge-length tolerance Tl . .. 75. 5.5. 4M scene under different cell sizes 50 and 100. . . . . . . . . . . . . . . . . .. 75. 5.6. Performance of the three configurations on a 8M-scene. . . . . . . . . . . . . .. 76. 5.7. Performance of configuration C under different scene complexities. . . . . . .. 76. 5.8. Preprocessing time for different scene complexities. . . . . . . . . . . . . . . .. 79. 5.9. Storage and loading time under different scene complexities (Ts = 5, Tl = 4.5). .. 82. viii.

(13) List of Figures ecol. 2.1. Illustration of the full edge collapse UV −→ V0 . . . . . . . . . . . . . . . . . .. 8. 2.2. Half-edge collapsing of UV −→ V. . . . . . . . . . . . . . . . . . . . . . . . .. ecol. 8. 2.3. Half-edge collapsing of UV −→ U. . . . . . . . . . . . . . . . . . . . . . . . .. ecol. 9. 2.4. The accuracy vs. mesh size plot results from a mesh simplification [35]. . . . .. 10. 2.5. Incremental error is propagated onto per-face bounding box for the total error evaluation [14, 15]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 14. 3.1. Texture distortion introduced by geometry simplification. . . . . . . . . . . . .. 21. 3.2. Texture distortion introduced by edge collapses. . . . . . . . . . . . . . . . . .. 22. 3.3. Texture mapping progressive meshes with texture adaptation.. . . . . . . . . .. 22. 3.4. Mesh parameterizations before and after an edge collapse. . . . . . . . . . . .. 25. 3.5. The nearest pair of points Ai and Ai−1 of two edges, one edge UN is from level. i and another is from level i − 1. . . . . . . . . . . . . . . . . . . . . . . . . . 3.6. 26. Edges overlay in the one-ring neighborhood of u (left), and the edges partition the neighborhood into 9 cells (right).. . . . . . . . . . . . . . . . . . . . . . .. 26. 3.7. Cell correspondence between partitions of T i (left) and T i−1 (right). . . . . . .. 27. 3.8. Accelerate texture adaptation by triangle-fan and triangle strip setups. . . . . .. 28. 3.9. Blurred artifacts introduced by texture adaptation (left) and minimized by indexing mapping (right).. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 28. 3.10 Each texel on indexing map is the texture coordinate referring to the texel on original map. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 29. 3.11 Indexing mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 29. ix.

(14) 3.12 Use one single indexing map to access to all the original maps. . . . . . . . . .. 30. 3.13 Performance difference of indexing map in 8-bit (center) and 16-bit (left) precision. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 30. 3.14 Textured images of parasaur head model. Top row of (b),(c),(d): simplified by QEM of 5D without texture adaptation, middle row: by APS without texture adaptation, bottom row: by QEM of 5D with texture adaptation. . . . . . . . .. 33. 3.15 Normal mapped parasaur head. left: 7685 polygons, center: 499 polygons, and right: 499 polygons with texture adaptation. . . . . . . . . . . . . . . . . . . .. 34. 3.16 Textured images of horse model. Left: original model of 8160 polygons with parameterized texture map, center: simplified model of 800 polygons without texture adaptation, right: simplified model of 800 polygons with texture adaptation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 34. 3.17 Left: original model of 17483 polygons, center: simplified model of 500 polygons without texture adaptation, right: simplified model of 500 polygons with texture adaptation.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 35. 3.18 From left to right, mesh with swirled texture coordinates and its color texture map, original mesh (7688 polygons), simplified mesh (399 polygons), and the same simplified mesh with texture adaptation. . . . . . . . . . . . . . . . . . .. 35. 3.19 (a) The top image is the original mesh, and the bottom image is the mesh that has been applied three iterations of both fine-to-coarse and coarse-to-fine transformation with the texture adaptation. (b) The zoomed images, on the left is the original mesh and on the right is the resulting mesh. . . . . . . . . . . . . . . .. 36. 4.1. The correspondence of xi and xi−1 is established via the cell correspondence. .. 40. 4.2. xn is adapted to xi by a sequence of texture adaptations I n → I n−1 → · · · → I i . .. 42. 4.3. The geometric error of simplified models without texture adaptation (measured by Metro). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 4.4. 44. The simplified parasaur head model without texture adaptation (polygon count: 7685 → 500) by QEM, APS and the new error metric. . . . . . . . . . . . . . . x. 44.

(15) 4.5. The simplified bunny head model without texture adaptation (polygon count: 17483 → 500) by QEM, APS and the new error metric. . . . . . . . . . . . . .. 4.6. The simplified zebra model without texture adaptation (polygon count: 8160 → 800) by QEM, APS and the new error metric. . . . . . . . . . . . . . . . . . .. 4.7. 46. The error distribution (in percentage) of the simplified parasaur head with texture adaptation, QEM: dark blue, APS: pink, and the new error metric: yellow. .. 4.9. 45. The geometric error of the simplified models with texture adaptation (measured by Metro). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 4.8. 45. 47. The simplified parasaur head model with texture adaptation (polygon count: 7685 → 500) by QEM, APS and the new error metric. . . . . . . . . . . . . . .. 47. 4.10 The simplified bunny head model with texture adaptation (polygon count: 17483 → 500) by QEM, APS and the new error metric. . . . . . . . . . . . . . . . . . .. 48. 4.11 The simplified zebra model with texture adaptation (polygon count: 8160 → 800) by QEM, APS and the new error metric. . . . . . . . . . . . . . . . . . .. 48. 5.1. Preprocessing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 54. 5.2. The maximum self-occluding error occurs at the position V 0 . . . . . . . . . . .. 56. 5.3. The derivations of SVMesh (a) and MVMesh (b). . . . . . . . . . . . . . . . .. 58. 5.4. (a) is the original mesh (65, 491 polygons) of a bunny viewed at one cell away (cell size 50), and (b-d) are SVMeshes for the bunny at 7 (259 polygons), 8 (254), and 9 (239) cells away. The upper-right bunnies are the projected images.. 59. 5.5. Testing depth variation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 59. 5.6. (a) is the original mesh (65, 491 polygons) of a bunny viewed at one cell away (cell size 50), (b-g) are MVMeshes of the bunny at 1 (1, 605 polygons), 2 (945), 3 (554), 4 (392), 5 (330), 6 (306) cells away. The upper-right indicates actual projected images. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 62. 5.7. Regional back-face culling. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 63. 5.8. Repeat clustering.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 64. 5.9. Run-time phase.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 65. xi.

(16) 5.10 Re-projection from source to destination image, T1 and T2 are the camera matrix of source image I1 and destination image I2 , respectively. . . . . . . . . . . . .. 68. 5.11 The cached image is a part of a source image. . . . . . . . . . . . . . . . . . .. 68. 5.12 The smaller angle is, the more accurate result is. . . . . . . . . . . . . . . . . .. 70. 5.13 Popping effects occur during the transition between view cells. . . . . . . . . .. 70. 5.14 Bird’s eye view of the 8M-scene. . . . . . . . . . . . . . . . . . . . . . . . . .. 72. 5.15 Distribution of SVMesh and MVMesh for the scenes 4M-50-Ts -4.5. . . . . . .. 74. 5.16 MVMeshes of bunny for different Tl .. . . . . . . . . . . . . . . . . . . . . . .. 74. 5.17 Rendered images by configuration B and C. . . . . . . . . . . . . . . . . . . .. 77. 5.18 Rendered images by configuration B and C at another view. . . . . . . . . . . .. 78. 5.19 Run-time statistics of configuration C on scene 8M-50-5-4.5: The frame rates with prefetching under a cold cache and a warm cache, and without prefetching.. 80. 5.20 Run-time statistics of configuration C on scene 8M-50-5-4.5: The polygon count, texture requirements, and image quality. . . . . . . . . . . . . . . . . .. 81. 5.21 Cases of cell transition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 83. xii.

(17) Abstract Real time rendering has been a kernel technology for interactive applications such as game, virtual reality (VR), and visual simulation. Real time rendering technologies can be geometrybased, image-based, or hybrid. In this thesis, we first present two novel techniques in geometrybased rendering and then propose a hybrid rendering framework. Among several key technologies in geometry-based real time rendering, level-of-detail (LOD) modeling has been a vital representation and a very popular technique in real-time applications. We develop a novel, simple, and effective texture adaptation scheme to eliminate the texture distortion commonly observed in mapping textures to progressive meshes. Based on this scheme, we propose a new surface-to-surface error metric, aiming to offer a simplification error measurement that is competitive to or even better than the existing methods. Hybrid rendering has been proven to be a very useful supplement to geometry-based technologies used in game and VR. We present a hybrid rendering scheme that explores the locality of visibility at the cost of extra storage and prefetching, and makes a tradeoff between image quality and rendering efficiency by using textured LOD meshes. To support texture mapping progressive meshes (PM), we usually allow the whole PM sequence to share a common texture map. Such a common texture map can be derived by using appropriate mesh parameterizations that may consider the minimization of geometry stretch, texture stretch, or even the texture deviation introduced by edge collapses. We have found that even with a well parameterized texture map, the texture mapped PM still reveals apparent texture distortion due to geometry changes and the nature of linear interpolation used by texture mapping hardware. In this dissertation, we propose a novel, simple, and efficient approach that adapts texture content for each edge collapse, aiming to eliminate texture distortion. A texture adaptation and its reverse operation are local and incremental operations that can be fully supported by graphics hardware. We also propose the mechanism of indexing mapping to reduce blurred artifacts due to the under-sampling that might be introduced by the texture adaptation..

(18) Experimental results have demonstrated that texture distortion is almost eliminated in tested examples. Based on the texture adaptation scheme, we propose a new mapping-based error metric that is able to provide much more accurate measurement of simplification error than APS or even QEM in the presence of badly parameterized texture maps. From the experimental results, we have observed that the proposed error metric outperforms the well-known error metric APS, and is better than or almost similar to QEM, depending on either maximum, mean, or RMS error used in measuring the approximation error. In the proposed hybrid rendering framework, the object space is first subdivided into cells. For each cell, inside objects are rendered as normal while outside objects are rendered as textured LOD meshes using projective texture mapping. The textured LOD meshes is object based and derived from the original mesh based on the captured depth images viewed at the centers of the cell and its adjacent cells. With this textured LOD mesh, problems commonly found in hybrid rendering, such as hole problems due to occlusion among objects and the gap problems due to resolution mismatch, can be avoided. Moreover, the size of holes due to self-occlusion is constrained to be within a user-specified tolerance. Several scenes with millions of polygons have been tested and higher than 600 FPS has been achieved with a little loss of image quality.. ii.

(19) Chapter 1 Introduction For last decades, computer graphics has become more widely used in many applications, including visualization, surgical simulation, military training, virtual shopping, movie creation, and 3D games. Not only because the graphics hardware becomes more and more powerful and cheaper, but also many techniques have been developed to eliminate the difficulty in achieving realistic and real-time rendering. Despite tremendous strides in graphics hardware, the demand of realism seems to grow faster than the advance of graphics hardware. To achieve realism, very complex 3D scenes may be necessary. Such complex scenes, however, may be too complex to be rendered in realtime with any available graphics hardware to date. On the other hand, from a laser scans of real objects to procedurally generated geometries, the input models can be too complex to be rendered in realtime. In fact, many polygons in a model are redundant since these polygons are projected to a portion of a screen pixel and do not help much in improving rendering quality. One possible answer to above question is the level-of-detail modeling, known as LOD in short, which aims to provide satisfactory rendering quality with less number of polygons. Although LOD modeling has been studied for about 30 years (the most remarkable paper can be tracked back to the work of James Clark [10]), LOD modeling still is an important research topic in computer graphics. Texture mapping is a simple and efficient method for recovering surface details of a simplified 1.

(20) model. Applying texture maps onto simplified meshes, however, often encounters some serious texture distortion. Many approaches have been proposed to minimize such a distortion, but all fail to solve it successfully. In this dissertation, we address the texture distortion problem and investigate what are the sources of the texture distortion, and propose a novel and simple texture adaptation scheme, to eliminate the texture distortion. In LOD modeling, meshes are simplified by a sequence of simplification operations. Usually, the sequence is ordered by the cost of the simplification operation, which measures the error between the meshes before and after simplification. The use of an accurate error metric can in general lead to a simplification that performs better in preserving geometric features and shape. While often producing satisfactory simplification results, the quadric error metric (QEM) proposed by Garland and Heckbert [29] is not geometrically accurate. On the other hand, the appearance preserving simplification (APS) proposed by Cohen et al. [15] is more geometrically accurate, but its accuracy often suffers from the incorrect mapping introduced by badly parameterized texture maps. A more accurate error metric that is efficient and performs better than current well known error metrics is desirable. Based on the texture adaptation scheme, we introduce a new mapping-based error metric which is a variant of APS but much more accurate than APS. Although LOD modeling techniques reduce the complexity of polygonal meshes, it may still not be able to render an extreme complex scene with acceptable quality at a real-time frame rate. Other approaches, including visibility culling and image-based rendering, can be integrated to further speed up the rendering. While some important visual features such as silhouettes can only be known at run-time and better preserved by view-dependent LOD techniques, a precomputed view-cell dependent LOD might be a compromise between the view independent and view dependent LOD. We develop a view-cell dependent textured LOD modeling for all objects outside a view cell. The construction of the viewcell dependent textured LOD modeling takes into account hybrid rendering, silhouette preserving, back-face culling, and even occlusion culling. We introduce a hybrid rendering system that is based on view-cell dependent. 2.

(21) textured LOD modeling and space subdivision, and is able to exploit the spatial and temporary coherence. The proposed system successfully is capable of rendersing complex scenes in realtime with only little image-quality loss.. 1.1. Main Contributions. The main contributions of this dissertation research are the following: • Propose a novel, simple, and effective texture adaptation scheme to eliminate the texture distortion often observed in texture mapped simplified meshes. It provides a very useful and automatic tool for model artists to design models and textures without concerning the texture distortion in mapping textures to simplified meshes. Moreover, the proposed indexing map successfully reduces the blurred artifacts introduced by the texture adaptation. • Propose a new mapping-based error metric that is able to accurately measure the simplification error even in the presence of badly parameterized texture maps. The mapping of two points on two consecutive levels of mesh is established via the operation of texture adaptation. Such a mapping can be easily extended to the mapping between the original mesh and the simplified mesh by using the indexing map. Experiments have shown that the proposed total/incremental maximum error metric performs better than APS, and better than or almost similar to QEM, depending on how the approximation error is measured. • Propose a hybrid rendering scheme that successfully adapts advantages of both imageand geometry-based rendering techniques. It is able to render extremely complex scenes at a real-time frame rate while preserving silhouettes and surface details of objects. An object-based textured LOD mesh representation is proposed to represent closer objects as well as distant scene in a unified representation. Compared to the environment-based mesh approach that is designed for distant scene. our approach has less popping effects 3.

(22) introduced in the transition between different representations. The proposed representation not only successfully removes the folding problems, gap problems, and rubber-sheet artifacts, but also limits the hole problem due to self occlusion. Moreover, the proposed representation together with scene subdivision is easily integrated with visibility culling techniques to exploit spatial coherence.. 1.2. Dissertation Organization. We give an overview and related works of LOD modeling and hybrid rendering in Chapter 2. The texture distortion and texture adaptation are described in detail in Chapter 3. In Chapter 4 a new mapping-based error metric for mesh simplification is presented and comparison to some well-known error metrics is discussed. The proposed hybrid rendering scheme is described in Chapter 5. Finally, a conclusion and possible future work are stated in Chapter 6.. 4.

(23) Chapter 2 Related Work 2.1. Real-Time Rendering. There are extensive researches in the field of real-time rendering, ranging from LOD modeling, visibility culling, image-based rendering, and hybrid rendering. LOD modeling simplifies geometric meshes of unnecessary complexity into appropriate simplified meshes that are suitable for some viewing conditions for the efficiency of rendering and processing. However, it is difficult to attain visual fidelity in general. Visibility culling techniques cull out polygons that are invisible from the current view before entering the graphics rendering pipeline. The task includes back-face culling that culls out back-facing polygons, view-frustum culling that culls out polygons or objects outside the view frustum, and occluding culling that culls out polygons occluded by other objects. The image-based rendering approaches use the current view image under some constraint to generate new view image when a viewer changes his or her viewpoint. It has the advantages of that its rendering cost is independent on the scene complexity, and of that its potential to produce photo-realistic rendering images. However, it suffers from the problem of limited viewing degree of freedom, which is trivial to geometry-based rendering. On the other hand, LOD. 5.

(24) modeling alone often reveals the problem of losing surface details. Thus, several hybrid rendering schemes have been proposed to combine traditional geometric and image-based rendering techniques to retain advantages of both rendering scheme.. 2.2. Mesh Simplification. The geometric meshes created by geometry modeling, e.g. CAD design tools, and laser scanning systems [17] are rarely optimized for rendering efficiency. The input models may be too complex to be rendered efficiently and may be in unnecessarily high detail for some viewing conditions. These complex models require simplification process to reduce complexity for efficient rendering or further processing. Mesh simplification already has been studied intensively in the last decade. Its ultimate goal is to reduce the complexity of a geometric mesh at the cost of undetectable visual difference, and to provide a tradeoff between fidelity (image quality) and speed (rendering efficiency). Luebke et al. [43] gave a complete and intensive review of model simplification. Additional comprehensive surveys on LOD modeling can also be found in [11, 12, 28, 34, 45]. Here, we review the works that address the reduction of texture distortion resulting from model simplification and address issues of some popular error metrics.. 2.2.1 Level-of-detail modeling The technique that generates multi-resolution representations of a model and select a representation of appropriate resolution for a particular viewing condition is called level-of-detail modeling. For example, a detailed mesh is used when the viewer is close to the object, and a coarser approximation is substituted when the viewer is distant from the object. The multiresolution representation can be derived in preprocessing or at run time. The former is termed view independent LOD modeling and the latter is called view-dependent LOD modeling. Viewindependent LOD modeling usually generates a set of meshes of different levels, each of which 6.

(25) has resolution reduced by a factor of two from the mesh of upper level. In total, the LOD modeling for a model requires only twice as much storage space as the model. For view-independent LOD modeling, instantaneous switching between two successive levels of detail may lead to perceptible popping effects. Popping effects can in general be minimized by blending or morphing. The blending employs alpha-blending to fade between the two successive levels of detail. Both of two successive representations are rendered, and the rendered images are alpha-blended according to the distance to the viewing position [46]. Such blending requires extra cost on rendering two successive representations of the same object, and may still have some blending artifacts if two representations differ at the silhouettes. On the other hand, the morphing gradually changes the shape between two successive levels of detail. It requires the knowledge of the vertex correspondence between two successive representations. Turk [56] proposed a method to have such correspondence by mutual tessellating the two successive levels of detail. Hoppe [35] presented a more elegant approach referred as geomorph that has more smooth transition between levels of detail. The detail description is given in the next subsection.. 2.2.2 Progressive meshes Hoppe [35] proposed a continuous level-of-detail representation, called progressive mesh or PM in short. The term of “continuous” emphasizes the ability to have smooth transition between two successive levels. By limiting the difference between two successive levels to a local region, it is easy to establish the vertex correspondence required for morphing. While discrete LOD modeling can provide only discrete approximation, PM is capable of providing a continuous representation that has less popping effects. PM simplifies an initial mesh M = Mn to a coarse mesh M0 by applying a series of n edge collapses: M = Mn. ecoln−1. ecol. ecol. −→ · · · −→1 M1 −→0 M0 .. As shown in Figure 2.1, an edge-collapse operation collapses two adjacent vertices U and V 7.

(26) into a single vertex V0 . Verteices U and V, edge UV, and adjacent faces 4UVL and 4VUR. vanish afterreplacements the collapse. When the new vertex V0 differs from U and V, this operation, denoted PSfrag. L. L ecol. U R. V0. V. R ecol. Figure 2.1: Illustration of the full edge collapse UV −→ V0 . ecol. as UV −→ V0 , is referred to as a full-edge collapse. Contrarily, a half-edge collapse operation. collapses edge UV to either vertex V or vertex U, and is denoted as ecol. UV −→ V or ecol. UV −→ U, respectively. The half edge collapse operation collapses U to V, and also removes one vertex, three edges, and two faces from a mesh (see Fig. 2.2). Although full-edge collapse can po-. PSfrag replacements. ecol U. V. V ecol. Figure 2.2: Half-edge collapsing of UV −→ V.. tentially produce a better approximation of the original model under the same polygon count, it needs to modify the vertex buffer and obstructs the vertex caching for hardware acceleration [48]. As a consequence, many descendent works use the half-edge collapse in practice because its simplicity in implementation and its efficiency in rendering. The inverse operation of an edge collapse is the vertex split that splits a vertex into two vertices and forms a new edge. By applying a series of n vertex splits, the coarse mesh M 0 can be refined. 8.

(27) to the initial mesh M: vsplitn−1. vsplit. vsplit. 1 0 M0 −→ M1 −→ · · · −→. Mn = M .. Under a network environment, the latency can be greatly reduced by transmitting the coarse version of a mesh at the very beginning and then progressively transmitting the remaining details. One can see a progressively refined approximations to the model as data is being incrementally received. Before introducing the error metrics that evaluate the error introduced by an edge-collapse operation and guide the mesh simplification process, we briefly describe the conventional framework for mesh simplification. To construct a PM for a polygonal mesh, the mesh is gradually simplified by a sequence of edge collapses in an order of increasing cost, evaluated by a specific error metric. This is supported by constructing a priority queue in an increasing cost for all the edges of the mesh and then the edge with the lowest cost will be popped out from the priority queue for each edge collapse operation. Since the cost of edge collapsing is directional, collapsing U to V will have different cost from collapsing V to U; see the difference between Figure 2.2 and Figure 2.3. Moreover, since a vertex U can only be collapsed to one of its one-ring neighboring vertices, it is not necessary to put all of its one-ring neighboring edges into the priority queue. Instead, we put only the directional edge that has the smallest cost among the neighboring edges into the ecol. priority queue. In practice, an infinite cost is usually assigned to a candidate edge UV −→ V. that violates some constraints such as face flip and parametric flip.. PSfrag replacements. ecol U. U. V. ecol. Figure 2.3: Half-edge collapsing of UV −→ U.. For each edge collapse operation, the edge UV with the lowest cost (error) is popped out from. 9.

(28) the priority queue and collapsed to V. After collapsing edge UV to V, the cost of V’s one-. ring neighbors should be re-evaluated and the priority queue should be updated accordingly. For efficiency consideration, one can postpone the evaluation until necessary. It is called lazy evaluation, proposed in [13], in which the cost of a directional edge in the priority queue is not re-evaluated until it is popped from the priority queue with a TRUE lazy flag set at the time one of its neighbors was collapsed. The lazy evaluation scheme has been reported to reduces the number of edge cost evaluation to about a factor of 3.2 [14] with only a little sacrifice in accuracy.. 2.2.3 Error metric The sequence of edge collapses for producing progressive meshes must be chosen carefully since it determines the quality of the approximating mesh M i , for 0 ≤ i < n. Figure 2.4 shows the plot for the accuracy of simplified meshes in different polygon counts. A good simplification algorithm is expected to produce meshes that have a curve close to the optimal curve in dashed line.. Figure 2.4: The accuracy vs. mesh size plot results from a mesh simplification [35]. However, it has been proven that finding the optimal edge collapse sequence is NP-hard for both convex polytopes [20] and polyhedral terrains [1]. Many greedy approaches have been proposed to approximate the optimal solution as much as possible by using priority queue strategy and a good error metric to guide the mesh simplification. 10.

(29) The simplest error metric is the vertex-to-plane distance. If the equation of a given plane p is represented by p=. . V=. . a b c d. T. x y z 1. T. and the position of vertex V is. ,. then the vertex-to-plane distance is VT p = ax + by + cz + d. The squared vertex-to-plane distance is then derived as (VT p)2 = (VT p)(pT V) = VT (ppT )V = VT K p V, where. . a2. ab     ab b2 T K p = pp =    ac bc   ad bd. . ac ad    bc bd  .  2 c cd    2 cd d. The sum of squared distance from vertex V to a set of planes P is then. ∑ VT KpV = VT ( ∑ Kp)V.. p∈P. p∈P. Garland and Heckbert [29] introduced the quadric error metric (QEM) for measuring vertex-toplane distance and provided a scheme to compute the total vertex-to-plane error For a vertex U, we store the matrix QU that is the sum of K p over all one-ring neighboring planes P of U, i.e., QU =. ∑ Kp,. p∈P. where P is the set of planes neighboring to U. The vertex-to-plane error for a vertex V to the 11.

(30) set of planes P is simply defined as VT QU V. ecol. When applying a full-edge collapse UV −→ V0 , the plane sets of vertices U and V are propa-. gated to the collapsed vertex V0 and in consequence Q V0 = Q V + Q U . If the set of one-ring neighboring planes of U and V are disjoint, the plane set propagation is equivalent to a union. Otherwise, each overlapped plane is counted multiple times. In fact, a plane is counted at most three times since it contributes only to the vertices of its defining triangle. QEM has a problem that we gets different squared distance under different triangulations. In general, densely tesselated mesh gets larger squared distance than sparsely tesselated one. This may be a problem for the mesh that has different tessellation densities from region to region. In addition to geometric error, human eye may be more sensitive on other attributes, such as colors, normals, textures, and etc. Garland and Heckbert extended their previous work to meshes with attributes by extending the quadric error metric from 3D to higher dimensions [30]. For the case of a mesh with a texture map, the dimension of Q is five, two for additional s and t coordinates. However, the Cartesian distance in geometry space may not be equivalent to the distance in texture coordinate space. While it is easy to compute the vertex-to-plane distance, the surface-to-surface distance is more likely to be able to reveal the true error introduced in mesh simplification. Bajaj and Schikore [3] proposed a method to evaluate surface-to-surface error. Both of the one-ring neighboring edges of V0 (on Mi−1 ) and the one-ring neighboring faces of UV (on Mi ) are mapped to a projection. plane using a planar projection. The projection plane is the average plane of the one-ring neighboring faces of V. To have a less distorted mapping, the proportion of the projected edge length is constrained to be preserved in the mapping. It is shown that the maximum surface-to-surface in the local region is at one of the intersections of the projected one-ring neighboring edges of 12.

(31) V0 and UV. Therefore, finding the maximum surface-to-surface error amounts to finding the. maximum among these intersections. The problem of this approach is that a valid projection may not always exist. Cohen et al. [13] proposed a more robust method to find a valid projection for an edge collapse. Their descendent work [15] uses a global parameterization as the bijective (1-to-1 and onto) mapping between the one-ring neighboring faces of UV and the one-ring neighboring. faces of V0 . It is assumed that the input is a mesh with parameterized texture maps. Since the parametrization is a bijective mapping function. F (X) = x, where X is a 3D position and x is its corresponding parameter in 2D, it can be used to define point correspondence before and after collapsing an edge. Note that, an edge is in general collapsed to a new vertex and, moreover, the mapping between 2D parameter and 3D surface point will be changed after an edge collapse since such a mapping is derived by the interpolation of triangle’s vertices. In consequence, the bijective mapping F varies while mesh M i is simplified to Mi−1 , i = n, . . . , 1. For two consecutive levels of mesh, Mi and Mi−1 , an incremental mapping-based error metric called APS is defined as the distance between two points, one on Mi and another on Mi−1 , that are mapped to the same parameter in 2D domain. That is, APS is −1 eincr (x) = kFi−1 (x) − Fi−1 (x)k.. (2.1). from Fi to Fi−1 . The total mapping error (distance) of a parametric point x can be defined as etotal (x) = kFn−1 (x) − Fi−1 (x)k.. (2.2). However, finding Fn−1 (x) is not a trivial task. Cohen et al. proposed a evaluation scheme for total error. Incremental error is propagated and accumulated using the per-face bounding boxes (Figure 2.5). Although the error metric APS is geometric in natural, its projected screen distance. 13.

(32) can be viewed as a measurement for the so called texture deviation. Such an extension implies that APS takes into account not only geometric error but also texture deviation. One problem about APS is that it relies on the mapping provided by the parameterized texture map and hence its performance usually depends on how well the parameterization is.. Figure 2.5: Incremental error is propagated onto per-face bounding box for the total error evaluation [14, 15]. Lindstrom and Turk [42] proposed an image-based approach to measure the error introduced in mesh simplification. The mean squared error of the difference between the images of the original model and images of the simplified model is computed for a set of views. Since images are taken into consideration, this image-driven simplification performs better than others in preserving the texture content.. 2.2.4 Incremental vs. total error metric Although authors of [41] claimed and demonstrated that their incremental and memory-less error metric can still output measurably good approximations, one may still need a total error metric for error bounded simplifications needed for some applications such as medical and scientific applications. Using the incremental error metric makes no guarantee on the error bound between the simplified mesh and the original mesh, and may result in serious problems in some applications. 14.

(33) On the other hand, the evaluation of the total error usually requires to maintain some information of the original mesh throughout the mesh simplification process and therefore is more computationally expensive than the evaluation of an incremental error.. 2.2.5 Vertex-to-plane vs. surface-to-surface error Again, surface-to-surface distance may reveal more truth than the vertex-to-plane distance for mesh simplification. In additional to mapping-based error metric, the Hausdorff distance can also be used to measure surface-to-surface distance. The Hausdorff distance is a distance between two point sets, said A and B, and is defined as. H(A, B) = max(h(A, B), h(B, A)),. where h(A, B) = max min ka − bk. a∈A b∈B. Note that, h(A, B) 6= h(B, A). Since surface is a type of continuous point set, the Hasudorff distance is adopted by many error measuring tools to measure the surface-to-surface distance. For example, Metro [9] measures the Hausdorff distance between two surfaces by using sampling.. 2.2.6 Texture distortion Using texture maps is the most practical and the simplest way to recover surface details for simplified meshes. Moreover, texture mapping has a direct support from graphics hardware that lets it be a standard in real time rendering applications. However, we often observe texture distortion in mapping textures to simplified meshes. Following is a brief review of the approaches that have been proposed to minimize such texture distortion. To associated attributes with vertices of a mesh surface requires a bijective mapping that is usually derived by the parameterization [26, 27, 37, 47, 48]. 15.

(34) Cignoni et al. [8] presented a method to preserve surface attributes in texture maps for each level of detail. Requiring one texture map dedicated for each level of detail, however, implies the need for large storage space and long processing time. APS [15] uses a texture deviation error metric to measure the distance between two points on two consecutive levels and correspond to the same texture value. Sander et al. [48] proposed a mesh papameterization that takes into account the geometry and texture stretch as well as texture deviation introduced by edge collapses. The texture map derived from such a parameterization is then shared by the entire PM sequence. Kim and Wohn [38] proposed a method to minimize texture distortion by generating a texture map that can be mapped to all levels of detail in a pre-processing stage. A distortion metric is then used to guide the mesh simplification. Sander et al. [47] proposed a signal-specialized parameterization to minimize texture stretch of the parameterization by allocating more texture samples to areas of higher signal frequency. Khodakovsky et al. [37] proposed a globally smooth parameterization with low distortion. Xu et al. [57] proposed a texture information driven simplification in which texture image frequency distribution and texture mapping distortion energy are combined to evaluate the simplification error. Several papers have been proposed to minimize texture distortion problem in a coarse-to-fine process. Eckstein et al. [25] proposed a coarse-to-fine optimization to generate a proper texture coordinate for the newly refined vertex to support texture mapping multi-resolution meshes. It always guarantees a solution by adding Steiner vertices. Zhou et al. [61] proposed a technique called TextureMontage to seamlessly map multiple texture images onto an arbitrary 3D model. One part of their work is to derive texture coordinates through optimization for all the vertices of the original mesh from the coarse texture coordinate assignments on the base mesh. Both texture stretch and texture color continuity are taken into account in this process. However, all the existing approaches only minimize the texture distortion, they do not solve the problem from the sources. In addition to the geometric error introduced in mesh simplification, we found that the nature of the linear interpolation in hardware texture mapping is the major source of texture distortion. We will address the problem in depth in Chapter 3.. 16.

(35) 2.3. Visibility Culling. For very complex scenes, visibility culling is another commonly used technique to further reduce the number of polygons sent to the rendering pipeline. Visibility culling, as we have been doing in the traditional rendering pipeline, falls into two categories. The first is the back face culling, aiming to perform the culling using the polygon clusters or its hierarchy. Kumar et al. proposed a sub-linear algorithm for hierarchical back-facing culling [39]. Zhang et al. improved the sub-linear algorithm by introducing normal mask which reduces the per polygon back-face test to only one logical AND operation [58]. The second category is the occlusion culling, aiming to cull away the polygons occluded by other objects. Occlusion culling can be done in processing or at run time. Shadow frusta [36], hierarchical Z-buffer [33], and hierarchical occlusion map [59] are popular run-time occlusion culling techniques. To avoid the inevitable run-time overhead, several region-based conservative occlusion culling methods aiming to derive the so called potentially visible set of polygons have been proposed. Cohen-Or et al. [16] proposed a preprocessing algorithm for regional occlusion culling, but its performance depends heavily on a single strong occluder. Durand et al. [24] proposed extended projection operations and Schaufler et al. [49] proposed blocker extension to handle occluder fusion for multiple occluders.. 2.4. Hybrid Rendering. Geometry-based rendering based on LOD modeling and visibility culling alone usually still cannot meet interactive requirement for very complex scenes. Image-based rendering (IBR) has been a well-known alternative. IBR takes parallax into account, and renders a scene realistically by interpolating neighboring reference views [5, 6, 44, 52]. IBR is usually efficient since its computation cost is independent of the scene complexity. It is, however, often constrained by the limited viewing degree of freedom, and the problems including folding, gap, and hole. LDI [54] is a good try to eliminate hole problems due to the visibility changes. LDI structure 17.

(36) is more compact in the sense that redundant information has been reduced when several neighboring reference images are composed into a single LDI. However, splatting is necessary in the rendering to reduce to the gap problem. Lumigraph [32] and light field rendering [40] have been proposed to reduce the 7D plenoptic function to a 4D function for static scenes. However, both methods require large amount of storage for the extremely large number of images. Hybrid rendering is basically a geometry-based rendering that integrates IBR techniques, aiming to replace the rendering of geometry for certain scene regions or objects by the rendering of pre-generated or cached images. In the following subsections, we describe four types of hybrid rendering.. 2.4.1 Region-based sprite Image caching proposed in [50, 53] combines geometry-based rendering and IBR, aiming to achieve an interactive frame rate for complex static scenes. The images of subdivided regions are cached as planar sprites and are rendered in a back-to-front order using texture mapping. The cached planar sprite needs to be updated if its parallax error exceeds a user-specified tolerance. The cached texture possesses no depth and, in turns, limits its life cycle.. 2.4.2 Environment-based mesh The life cycle of the cached sprite can be longer if the so called depth mesh is used instead of images as proposed in [18, 19, 55]. The depth mesh of a region or a background is derived from the captured depth images by first triangulating the image’s pixel space using depth and then re-projecting the 2D triangulation into the object space to represent the background or distant scene. Such depth meshes are then rendered via pipeline. In such approaches, folding problems and gaps resulting from the resolution changes can be eliminated; however, the hole problems due to occlusion among objects and self-occluding still remain. Moreover, disjointed objects. 18.

(37) might be rendered as a connected object. Geometric accuracy might be a problem when reprojecting the 2D triangulation back to 3D space since depth meshes derived from the depth images are in pixel resolution. While objects near to depth meshes are normally represented as original geometry or LOD models, image popping due to the representation transition may occur. In [23], Decoret et al. proposed multi-layered impostors to constrain visibility artifacts between objects to a given size, and a dynamic update scheme to improve the gap due to the resolution mismatch. However, it still encounters hole problems due to self occlusion, and requires a special hardware architecture for an efficient dynamic updated. In [2], an interactive massive model rendering system using geometric and image-based acceleration is proposed, in which distant objects are represented by textured depth meshes and near objects by LOD models. The method proposed in [60] integrates LOD and visibility computation and is suitable for scenes with high-depth complexity and very dynamic scene.. 2.4.3 Object-based Textured LOD mesh Shade et al. [54] described a paradigm in which objects could be represented by environment map, planar sprite, sprite with depth, layered depth image (LDI), and polygonal mesh, depending on their distances to the viewer. The objects are rendered differently, but with different image qualities. In consequence, the transition between different representations may produce noticeable popping effects, especially for the transition between an image-based primitive and a geometry-based primitive.. 19.

(38) Chapter 3 Texture Adaptation for Progressive Meshes 3.1. Introduction. Mesh simplification has been an active area of research in real-time graphics. The ultimate goal of mesh simplification is to generate a simplified mesh of low polygon count that preserves the fidelity of the original mesh. Texture mapping has been very useful in enhancing shaded images with more surface or color details. For a given mesh and its associated texture map, there are several possibilities of applying the texture map to the simplified meshes. One way is to have a texture map for each simplified mesh, which requires more artist work on texture design and more storage, especially for progressive meshes (PM). A more practical way is to have the entire PM sequence share a common texture map; however, when applying texture mapping to PMs, serious texture distortions are often observed. To reduce texture distortion, several schemes have been proposed. One is to consider texture deviation as an error metric, implying that edge collapses with higher texture deviation are more likely to be retained. This, however, cannot prevent texture distortion introduced by edge collapses that have been performed. In addition to using the metric of texture deviation, the texture map can be derived by using appropriate. 20.

(39) parameterizations that take into account the minimization of geometry and texture stretch, as well as the texture deviation introduced by edge collapses. It is observed that even with a very well parameterized texture map, the texture mapped PM still reveals significant texture distortion due to geometry changes and the nature of linear interpolation employed by texture mapping hardware. Let’s consider the 2D case shown in Figure 3.1, where edges AC and BC are simplified to AB.. At the bottom, T is the texture map on which A maps to 0, B maps to 1, and C maps to 0.3. On the textured image of AB, we see the blue color shares a portion of AB that is much smaller than. expected. Figure 3.2 shows a planar polygonal mesh M = M2 and the texture map T associated with it. The textured images of simplified meshes M1 and M0 reveal serious texture distortion, as shown in the bottom row of the figure. In this example, geometry remains the same, but the nature of linear interpolation for texture mapping affects the textured image. Essentially, the texture coordinate within a triangle is piecewise linear but is no longer linear when crossing edges of the triangle. Such texture distortion has been routinely observed for texture mapping PMs using any existing technique.. C PSfrag replacements A. B T 0. 1. 0.3. Figure 3.1: Texture distortion introduced by geometry simplification. We present a novel, simple, and efficient approach that adapts texture content for each edge collapse, aiming to effectively eliminate the texture distortion introduced by geometry changes and the nature of linear interpolation employed by texture mapping hardware. The texture adaptation applied for each edge collapse is local to the region affected by the edge collapse and is applied to the adapted texture resulting from the previous edge collapse. The texture 21.

(40) M2 = M. T. M1. T. M0. T. PSfrag replacements. Figure 3.2: Texture distortion introduced by edge collapses. adaptation is invertible, that is, the backward texture adaptation can be performed for a vertex split. Both texture adaptation and backward texture adaptation can be fully supported by texture mapping hardware. Once the necessary correspondence in the partition of texture space is built during the course of PM construction, the texture adaptation or its inverse can be applied on the fly before rendering the simplified or refined model with texture map. We have observed that the proposed texture adaptation is capable of eliminating texture distortion in a time that is almost negligible. Figure 3.3 depicts that the textured images of M 1 and M0 with texture adaptation are indistinguishable from the textured image of the original mesh.. PSfrag replacements M2 = M. T2 = T. T1. M1. M0. T0. Figure 3.3: Texture mapping progressive meshes with texture adaptation.. 22.

(41) 3.2. Texture Adaptation. 3.2.1 Overview For a given polygon model M with a texture map T , the texture adaptation locally and incrementally adapts the texture map in the course of edge collapses that construct a PM of M. The goal is to eliminate the texture distortion introduced by the edge collapses. For a PM sequence, we then have a texture map T i associated with each reduced model Mi , that is, M = Mn T. = Tn. ecoln−1. ecol1. −→ · · · −→. M1 T1. ecol0. −→. M0 T0. .. The texture adaptation operation from T i to T i−1 is invertible, that is, for a T i−1 associated with Mi−1 , T i of Mi can be derived by doing the inverse of the texture adaptation that brings T i−1 to T i , which we call backward texture adaptation. Consequently, for the sequence of vertex splits from M0 , we have M0 T0. vsplit0. −→. M1 T1. vsplit1. vsplitn−1. −→ · · · −→. Mn = M Tn = T. .. We will see that the texture adaptation and its inverse, backward texture adaptation, involve the same operations, which can be done very efficiently by graphics hardware. And the texture adaptation or its inverse is applied to texture map and is performed on the fly while the model is simplified or refined, respectively. For an edge collapse ecoli−1 that reduces Mi to Mi−1 , the texture adaptation is local to the region Ri−1 that is the neighborhood of the collapsed edge in texture space and is performed incrementally from T i . Since the boundary of Ri−1 remains fixed, what the texture adaptation does is basically find an appropriate texel of T i for each texel of T i−1 , all in the region Ri−1 . We will see in the next subsection that the region Ri−1 can be respectively partitioned into the same number of cells for T i and T i−1 , and within each of these cells, the texture coordinates are piecewise linear. The correspondence between texels of T i and T i−1 is then simplified to 23.

(42) the correspondence between cells of T i and T i−1 . Once all pairs of corresponding cells are found, the texture adaptation can be performed by hardware texture mapping the cell of T i to the corresponding cell in T i−1 . Since cell correspondence is identical for each texture adaptation and its inverse, backward texture adaptation texture maps the cell of T i−1 to the corresponding cell in T i . Essentially, texture adaptation is a re-sampling process that might pose problems of under sampling and introduce blurred artifacts. Indexing mapping is proposed to minimize these blurred artifacts. Indexing mapping involves an additional texture map, called the indexing map, that stores texture coordinates to the original texture map. During edge collapsing, texture adaptation is applied to this indexing map, leaving the original texture map alone. Only during rendering, the original texture map is mapped to the model via texture coordinates derived from the indexing map. The cell partition and cell correspondence are derived while constructing the progressive meshes. The cell partition and cell correspondence are stored along with the information for each edge collapse and vertex split. In rendering the progressive meshes, the texture adaptation or backward texture adaptation is performed on the fly while the model is simplified or refined, respectively. In the following sections, half-edge collapse is used in the illustration of the proposed method. Extension to the full-edge collapse is straightforward.. 3.2.2 Cell correspondence between two consecutive levels For a given mesh M, its texture map T is obtained by parameterizing M onto the texture plane. The parameterization of M is a one-to-one mapping F that maps each vertex V of M to a point v on the texture plane. For a PM using half-edge collapse, the parameterization of M i is a subset of that for M. Figure 3.4 depicts the mapping of the neighborhood of an edge UV on the texture. plane, before and after UV is collapsed.. 24.

(43) Mi. Mi−1. PSfrag replacements. ecoli−1. U. V. V. F i−1. Fi Ri−1. u. T v. Figure 3.4: Mesh parameterizations before and after an edge collapse. When we overlay the edges in the region Ri−1 before and after edge collapse ecoli−1 that reduces Mi to Mi−1 , edges on the overlay may intersect each other and hence partition R i−1 into cells, as shown in Figure 3.6, where un intersects vw at a and another pair of edges intersect at. b and Ri−1 is partitioned into 9 cells. However, the two intersecting edges on the texture plane are generally not coplanar in 3D space. Nevertheless, for each pair of intersecting edges we can compute the pair of nearest points, one on an edge and one on another edge. Taking the mesh in Figure 3.6 as an example, we compute the pair of the nearest points A i and Ai−1 , see Figure 3.5, where Ai is on UN of Mi and Ai−1 on VW of Mi−1 . The points Ai and Ai−1 can be derived by. minimizing the distance of points on the two edges, which amounts to solving the linear system . . . . .  ~u ·~u −~v ·~u   αi   (V − U) ·~u    = , ~u ·~v −~v ·~v αi−1 (V − U) ·~v where ~u =. N − U,. ~v = W − V. The solution of the above system is the pair of parameters αi and αi−1 , 0 < αi , αi−1 < 1. The texture coordinate of Ai , denoted as ai , is derived by interpolating u and n, while the texture. 25.

(44) coordinate of Ai−1 , denoted as ai−1 , is derived by interpolating v and w, as follows: ai. = u + αi (n − u),. ai−1 = v + αi−1 (w − v).. (3.1). It is apparent that ai is generally not equal to ai−1 . Note that v j represents the texture coordinate of vertex V on M j , for j = n, . . . , 1, 0, and since every vertex V of M remains fixed in the course of half-edge collapsing, v j remains the same for all V of M. Since U is collapsed to V, we need to find its texture coordinate ui−1 on T i−1 . First, we find the point Ui−1 on Mi−1 that is nearest to U and identify the triangle containing Ui−1 . Then we compute the barycentric coordinates of Ui−1 with respect to the triangle and finally derive the texture coordinate u i−1 from the texture coordinates PSfrag of triangle vertices using the barycentric coordinates. replacements W. Ai−1. U. Ai. V. N. Figure 3.5: The nearest pair of points Ai and Ai−1 of two edges, one edge UN is from level i and another is from level i − 1.. w. PSfrag replacements. u. b. a n v Figure 3.6: Edges overlay in the one-ring neighborhood of u (left), and the edges partition the neighborhood into 9 cells (right). After deriving ai , ai−1 , bi , bi−1 , and ui−1 , we move a to ai and b to bi to form the partition for Mi and, similarly, move a to ai−1 , b to bi−1 , and u to ui−1 to form the partition for Mi−1 . See Figure 3.7. Two cells in these two region partitions are said to be in correspondence if they correspond to the same cell before moving the points a, b, and u to designated texture coordinates. For example, 4ai vn and 4ai−1 vn are in correspondence. 26.

(45) PSfrag replacements. ecoli−1. u w. bi. ui−1 w. bi−1. n. n v. ai vspliti−1. v. ai−1. Figure 3.7: Cell correspondence between partitions of T i (left) and T i−1 (right).. 3.2.3 Texture adaptation After cell correspondences for all cells in Ri−1 are found, the texture adaptation from T i to T i−1 is performed for each pair of corresponding cells. To adapt the texture from cell 4a i bi ci of T i to cell 4ai−1 bi−1 ci−1 of T i−1 , we let the former be the source texture, the latter be the target polygon, and then apply hardware texture mapping. Similarly, for the backward texture adaptation from T i−1 to T i , 4ai−1 bi−1 ci−1 is the source texture, and 4ai bi ci is the target polygon. Care must be taken to prevent the so called parametric folding in the process of texture adaptation by ensuring that areas of both cells 4ai bi ci and 4ai−1 bi−1 ci−1 are positive. Note that, since we maintain all Ti , i = n, . . . , 1, 0, in a single physical texture map, we need a temporary map to support the texture adaptation. It is worth mentioning that the texture adaptation for all pairs of corresponding cells can be accelerated by using triangle-fan and triangle strip setups, as shown in Figure 3.8. Its cost can be further minimized by packing triangle fans and strips into arrays and using draw array commands, such as glDrawArrays() or glDrawElements(). Moreover, it can be run in parallel with the edge collapse operation.. 3.2.4 Indexing map Texture adaptation is basically a re-sampling process of a texture map. When a texture area of higher frequency gets adapted to a smaller texture area, blurred artifacts due to under-sampling 27.

(46) virtual edge PSfrag replacements u. u. v. v. Figure 3.8: Accelerate texture adaptation by triangle-fan and triangle strip setups. of texture samples might appear, as shown in Figure 3.9. To prevent such problems, we propose the mechanism of indexing mapping that uses an indexing map I to store in each texel the texture coordinate referring to the original texture map T . All the texture adaptation operations are applied to the indexing map, leaving the original texture map alone. Initially, the indexing y+0.5 map I n for Mn stores in each texel the coordinate itself, that is, I n (x, y) = ( x+0.5 w , h ), where. w × h is the resolution of the indexing map. For texture adaptation, we derive I i−1 from I i in the same way as we do for deriving T i−1 from T i , for i = n, . . . , 2, 1, Figure 3.10 shows the indexing map with each texel value s and t coordinates, which is visualized as red and green color respectively.. Figure 3.9: Blurred artifacts introduced by texture adaptation (left) and minimized by indexing mapping (right). Figure 3.11 illustrates an example of indexing mapping. The surface M is texture mapped with T . When M is simplified to M0 , T is adapted to T 0 and, in this example, the texels on T are re-sampled to two texels on T 0 . As a result, one gets a blurred textured image while 28.

(47) Figure 3.10: Each texel on indexing map is the texture coordinate referring to the texel on original map. texture mapping T 0 onto M0 . With the proposed indexing mapping, the texel values of I 0 are the coordinates of T , and are used to access to the original texels on T . Therefore, blurred artifacts are reduced. PSfrag replacements T. I0. T0. T M. M0. M0. Figure 3.11: Indexing mapping. The use of indexing map is also advantageous when more than one texture map, such as a combination of a color map, a normal map, and a bump map, see Figure 3.12, are associated with the model since texture adaptation is applied to the indexing map only, leaving all the maps untouched. Indirect accessing of a texture map is not supported by graphics APIs such as OpenGL. Fortunately, it can be implemented using the fragment shader. Another issue is the precision of the indexing map. In our experience, a texture map of 16-bit precision is sufficient to deliver acceptable image quality and performance for texture adaptation. See the example shown in Figure 3.13.. 29.

(48) Figure 3.12: Use one single indexing map to access to all the original maps.. Figure 3.13: Performance difference of indexing map in 8-bit (center) and 16-bit (left) precision.. 3.3. Experimental Results. In our experiments, we used a Pentium4 3.0Ghz platform with an nVIDIA GeForce 6800GT graphics card. All test models are using a 1024 × 1024 16-bit floating indexing map. We use the render-to-texture feature to avoid transferring texture maps between graphics hardware and the host CPU. In our experiments, we observed that the render-to-texture setup is the most expensive operation in texture adaptation even with the help of framebuffer objects (FBOs). To minimize the setup cost, we simply collect texture adaptations that are applied to disjoint regions and perform adaptations together. When we simplify the model from M j to M i , where j > i, we start with the set COLLECTION containing only the texture adaptation from T j to T j−1 and the set R = {R j }. We then check, in the sequence of k = j − 1, . . . , i + 1, to see if 30.

(49) Rk overlays with any element of R. If not, Rk is inserted into R and the texture adaptation from T k to T k−1 is put into COLLECTION. After doing this cycle, we perform all the texture adaptations collected in COLLECTION with one common setup. We repeatedly perform the packing of texture adaptations until all the texture adaptations are done. A similar approach is applied to the refinement process. We have observed that 13.73 and 20.20 texture adaptations on average are packed to share a common setup for the parasaur head and bunny head, respectively, and such a simple approach achieves about a 13.66× speedup factor for the parasaur head model and 17.51× for bunny head model, as depicted in Table 3.1, and the entire run-time cost is also listed in Table 3.2. Moreover, the setup can be further shared among objects in an environment with multiple objects. Table 3.1: Speedup by packing texture adaptations. w/o packing original. model. parasaur head bunny head. # of packs. adaptations. adaptations. avg. time (ms). 3, 611 8, 545. 0.7581 0.7616. 263 423. simplified. model. model. (polygons). (polygons). 7, 835 17, 483. 499 500. w/ packing. # of texture. per pack. avg. time (ms). speedup factor. 13.73 20.20. 0.0555 0.0435. 13.66× 17.51×. # of texture. Table 3.2: The entire run-time cost of simplifying PM w/ and w/o texture adaptation. run-time cost (ms). model. parasaur head bunny head. original model (polygons). simplified model (polygons). w/o texture adaptation. w/ texture adaptation. 7, 835 17, 483. 499 500. 52.99 137.16. 253.42 508.87. Experimental tests have been done on several models. For PM construction, a quadric error metric (QEM) of five dimensions [30] is applied to all tested models. Figure 3.14 demonstrates the power of texture adaptation on the parasaur head at bottom row. A comparison with QEM of 5D without texture adaptation and APS [15] without texture adaptation is shown at top and middle row, respectively. The texture map for the parasaur head is derived using signal-specialized parameterization [47]. Texture distortion becomes noticeable when the mesh is simplified to 2000 polygons and becomes obvious for meshes of 1000 and 499 polygons. We have observed that texture distortion is almost eliminated by texture adaptation even in the textured image of the 31.