B. 研究步驟 – 移動補償時間濾波之改良與可調式移動資訊
B.4 可調式移動資訊
在文獻[23]中，提出一個適用於 MC-EZBC 之可調式移動向量方法。在此計畫中，我 們將此概念做沿伸與適當的修改，提出一個可適用於AVC 移動估測之可調式移動向量 技術。
在傳統的小波轉換中，擁有空間軸、時間軸及畫質之可調式設計，移動資訊在空間 軸與畫質可調式設計中是無法切割的。當可允許傳送之位元率太低時，抽取器(Extractor or Puller)很可能會因移動資訊過大，而沒有足夠可分配的位元率進行小波係數切割而失 敗。此外，在非常低位元率的情況下，我們會希望在省去部份的移動資訊將較多的位元 率分配給小波係數，以換取較好的畫質。因此，在移動估測之後，我們對移動資訊進行 切割。
在 AVC 的畫面間預測中，基本的處理大小為 16x16 之 macroblock。每一個 macroblock 可再細分為16x16, 16x8, 8x16, 8x8 ,8x4, 4x8 及 4x4，並且所對應到的移動向量具有 1/4 像素之精確度。我們依下列步驟將移動向量做切割。
第一步：進行 16x16 大小之整數像素精確度之移動搜尋，所產生之移動向量為“基 本層”之移動向量。
第二步：進行 16x16 與 8x8 大小之 1/2 像素精確度之移動搜尋。與基本層之差值將 被保留編碼，稱“第一加強層”。
第三步：進行所有搜尋方塊大小之 1/4 像素精確度之移動搜尋。將其與基本層與第 一加強層之和的差值將被保留編碼,稱“第二加強層”。
第四步：將所有的移動向量階層分別利用 CABAC 編碼
Base layer
1st enh. layer
2nd enh. layer
Original Proposed
Original
¼ pixel accuracy Proposed
Base layer: i nteger pi xel accuracy 1st enh. layer: ½ pixel accuracy 2nd enh. layer: ¼ pixel accuracy
Base layer
1st enh. layer
2nd enh. layer
Original Proposed
Original
¼ pixel accuracy Proposed
Base layer: i nteger pi xel accuracy 1st enh. layer: ½ pixel accuracy 2nd enh. layer: ¼ pixel accuracy
圖 23: The base and enhancement layer motion vectors.
如圖 23 所示，原始單層的移動向量被分為三層。每張畫面的移動資訊都會在移動 補償時間濾波的過程中被分成基本層與適當的加強層，因此，所有的移動向量資訊都會 被集中，如圖 24 所示。因為所有時間軸階層之移動資訊基本層必需被傳送，以產生所 有時間軸解析度畫面，所以，基本層資料非常重要不可遺失。
B
D. 結論
目前的可調式技術可分為時間軸、空間軸及畫質可調式編碼。畫面間小波轉換視訊 編碼為一個利用小波轉換達成完全可調式(full scalable)編碼之嶄新技術。DCT 混合式 (hybrid)架構為近年來最廣為被使用的視訊編碼方式。然而，由於過去在小波視訊編碼上 的幾項困難問題近五、六年獲得解決，包括分數精確度之移動補償技術等。如今小波視 訊編碼已成為可在高位元率與H.264 具有相似表現。
在低位元率時，畫面間小波轉換視訊編碼尚不如H.264。這是由於 H.264 使用混合 式架構並且可針對特定位元率做最佳化，而畫面間小波轉換視訊編碼可適用於各位元率 之位元資料流(bitstream)切割，因此，在最低位元率上之表現並未如同 H.264 可最佳化。
在許多的情況下，我們也注意到了由於大量的移動資訊對在低位元率下之壓縮資料所造 成的不利影響。
在低位元率時，我們亦發現畫面間小波視訊轉換編碼在一些特定的測試影像無法表 現得很好，而這些影像內容都具大幅度的移動。雖然分數精確度之移動估測可改善編碼 效率，但在移動補償上並未做最佳化。時間軸之低通影像有時會有移動估測造成的缺陷。
本計畫中所提出之技術可有效改善編碼效率。我們利用了在AVC 中高效率之移動估 測來進行移動補償時間濾波，以及利用I-block 偵測以改善在時間軸低通影像之畫質表 現。此外，所提出的多階層式移動估側可達成可調式移動資訊，並且在低位元率之下有 相當明顯之編碼改善。畫面間小波視訊編碼仍有許多結構與參數有待進一步最佳化。
這部分成果分成兩部分，分別在2004 年 3 月與 7 月提案 MPEG 標準組織。3 月之 提案為參加scalable video coding Call-for-Proposal 競賽，在 14 個提案中經視覺主觀評 審，成績中等。在國際大企業環伺下，此結果似乎尚可。7 月之提案為參加 Core Ex-periments，改善目前的 Reference Model。MPEG 標準組織在未來兩年將持續改良 Reference Model，最後成為標準。
參考文獻
[1] ISO/IEC JTC1/SC29/WG11 N3747. MPEG-4 Overview - (V.16 – La BauleVersion), Contribution for La Baule, October 2000.
[2] C.J. Tsai, M. van der Shaar and Y.K. Lim, “Working Draft 3.0 of ISO/IEC TR2100-12
Multimedia Test Bed for Resource Delivery,” ISO/IEC JTC1/SC29/WG11
MPEG2003/M10299, Hawaii, December 2003.[3] ISO/IEC JTC1/SC29/WG11 N3850. ISO/IEC 14996-1 ,COR1, AMD1.
[4] ISO/IEC JTC1/SC29/WG11 N2614 MPEG-4 Intellectual Property Management &
Pro-tection (IPMP) Overview & Applications.
[5] ISO/IEC JTC1/SC29/WG11 N5068, Study of FPDAM ISO/IEC 14496-1:2001/AMD3,
Jul. 2002.
[6] ISO/IEC JTC1/SC29/WG11 N5333, MPEG-21 Requirements v.14, Dec. 2002.
[7] ISO/IEC JTC1/SC29/WG11, N5535, Requirement for MPEG-21 Intellectual Property
Management and Protection, Pattaya, Mar. 2003.
[8] ISO/IEC JTC1/SC29/WG11, Part 5 – Reference Software – Systems (ISO/IEC 14496-5
Systems)
[9] ISO/IEC JTC1/SC29/WG11 N4702, MPEG-4 IPMP Extension Reference Software
Ar-chitecture based on IM1, Jeju, Mar. 2002.
[10] ISO/IEC JTC1/SC29/WG11 N4850, MPEG-2 and MPEG-4 IPMP Extension Reference
Software Architecture based on IM1, May. 2002.
[11] J. Liu, et al., “WD1.0 of ISO/IEC 13818-5:1997/AMD2:2003 MPEG-2 IPMP Reference
Software,” ISO/IEC JTC1/SC29/WG11 M9840, Trondheim, July 2003.
[12] Y. Lim, C. Xu, and D.D. Feng, “Web-based image authentication using invisible fragile
watermark,” in Conference in Research and Practice in Information Technology, 2002,
pp.31-34.[13] C.S. Shieh, H.C. Huang, F.H. Wang, and J.S. Pan, “Genetic watermarking based on
transform domain techniques,” Pattern Recognition, 2004, pp.555-565.
[14] X. Sun, F. Wu, S. Li, W. Gao, and Y.-Q. Zhang, “Seamless switching of scalable video
bitstreams for efficient streaming,” IEEE Trans. On Multimedia, 2004, pp.291-303.
[15] J.M. Almeida, D.L. Eager, M.K. Vernon, and S.J. Wright, “Minimizing delivery cost in
scalable content distribution systems,” IEEE Trans. On Multimedia, 2004, pp.356-365.
[16] R. Parviainen and P. Parnes, “Large scale distributed watermarking of muticast media
through encryption,” in Proceedings of the International Federation for Information
Processing, Communications and Multimedia Security Joint Working Conference IFIP TC6 and TC11, 2001, pp.149-158.[17] X. Xu, S. Dexter, and A.M. Eskicioglu, “A hybrid scheme for encryption and
water-marking,” IS&T/SPIE Symposium on Electronic Imaging 2004, Security, Steganography,
and Watermarking of Multimedia Contents IV Conference, 2004, pp.723-734.[18] Digital video broadcasting project (DVB): http://www.dvb.org (2004).
[19] Data Encryption Standard (DES): http://www.itl.nist.gov/fipspubs/fip46-2.htm (1993) [20] V. Chande and N. Farvardin, “Progressive transmission of images over memoryless noisy
channels,” IEEE Journal on Selected Areas in Communications, 2000, pp.850-860.
[21] P. Chen and J. W. Woods, "Comparison of MC-EZBC and H.26L TML 8 on Digital Cinema Test Sequences," ISO/IEC JTC1/SC29/WG11, MPEG2002/8130, Cheju Island, March 2002
[22] P. Chen, Fully scalable subband/wavelet coding, Ph.D. thesis, Rensselaer Polytechnic Institute, Troy, New York, May 2003.
[23] H.-M. Hang, S. S. Tsai, and Tihao Chiang, “Motion information scalability for MC- EZBC”, ISO/IEC/JTC1 SC29/WG11 doc. No. M9756, July 2003
計畫成果自評
本計畫有以下幾類成果。第一類為MPEG-4 IPMP System 與 Interframe Wavelet 所發 展出的技術、經驗及成品與國際 MPEG 標準直接相關，極具實用價值，可促進國內工 業研發技術開發。第二類為將上述技術提案至 MPEG 標準組織，有助我國技術之進入 國際舞台，共有六篇MPEG 標準提案。其中 IPMP System 在 2004 年 7 月加入 MPEG-21 Multimedia Test Bed 中，已成為 Test Bed 軟體的一部份，全案期望在 2005 年左右完成標 準化。Interframe Wavelet 在 2004 年 3 月與 7 月提案 MPEG 標準組織，參加 scalable video coding Call-for-Proposal 競賽，與後續 Reference Model 之改進。第三類為計畫執行過程 所獲得之研究成果論文四篇，已發表於國內外學術會議。其四，參與計畫之同學可獲 得國際多媒體最先進的MPEG-4 與 MPEG-21 相關技術及多媒體系統設計經驗，畢業後 進入產業，直接有助於產業界開發新產品，提昇我國工業技術能力。達到人才培育之 目的。
綜合評估：本計畫產出相當多具有學術與應用價值的成果，特別是直接參與國際標 準會議，在國際上展示成果。並培育高科技人才培育，整體成效良好。已發表學術論 文四篇，碩士學位論文一冊，以及六篇MPEG 標準提案如下表。
Publications
(1) H.-M. Hang, “Next generation MPEG video and system,” (invited talk), 2003 Workshop
on Consumer Electronics, Nov. 27 – 28, Tainan, Taiwan 2003.
(2) H.-K. Hsu, H.-C. Huang, and H.-M. Hang, ``An enhanced entropy coding scheme for interframe wavelet,'' in 2004 Conf. on Computer Vision, Graphics, and Image Processing, Hualin, Taiwan, Aug. 2004.
(3) C.-Y. Tsai, H.-K. Hsu, H.-C. Huang, H.-M. Hang and G.-Z. Wu, “Enhanced motion esti-mation for interframe wavelet video coding,” IEEE International Conf. on Image
Process-ing '04, SProcess-ingapore, Oct. 2004
(4) F.-C. Chang, H.-C. Huang and H.-M. Hang, “Combined Encryption and Watermarking Approaches for Scalable Multimedia Coding,” Pacific Rim Conference on Multimedia
2004, Tokyo Japan, Dec. 2004
(5) Chen-Wei Fan 范振韋, MPEG-4 IPMPX Design and Implementation on MPEG-21
Test-bed, MS Thesis, NCTU, June 2004.
MPEG Standard Contributions
1. C.-Y. Tsai, H.-C. Chuang, J.-H. Chen, J.-C. Ma, C.-Y. Liu, C.-W. Fan, F.-C. Chang, C.-N.
Wang, C.-J. Tsai, Tihao Chiang, S.-Y. Lee, and H.-M. Hang, “ISO/IEC JTC1/SC29/WG11 M10160: Scalable Multimedia Streaming Test Bed for Media Coding and Testing in Streaming Environments,” October 2003 (66th, Brisbane, Australia)
2. C.-N. Wang, C.-Y. Tsai, H.-C. Chuang, J.-H. Chen, J.-C. Ma, C.-N. Chiu, C.-Y. Liu, C.-W.
Fan, F.-C. Chang, C.-J. Tsai, Tihao Chiang, S.-Y. Lee, and H.-M. Hang, “ISO/IEC
JTC1/SC29/WG11 M10298: Scalable Multimedia Streaming Test Bed for Media Coding and Testing in Streaming Environments,” December 2003 (67th, Kona, Hawaii, USA)
3. C.-Y. Tsai, H.-K. Hsu, H.-M. Hang, and T. Chiang, “ISO/IEC JTC1/SC29/WG11 M10569/S08, Response to Cfp on Scalable Video Coding Technology: Proposal S08 -- A Scalable Video Coding Scheme Based on Interframe Wavelet Technique,” March 2004 (68th, Munich, Germany)
4. H.-C. Huang, W.-H. Peng, Y.-C. Lin, C.-N. Wang, T. Chiang and H.-M. Hang, “ISO/IEC JTC1/SC29/WG11 M10569/S07, Response to Cfp on Scalable Video Coding Technology:
Proposal S07 -- A Robust Scalable Video Coding Technique,” March 2004 (68th, Munich, Germany)
5. C.-N. Wang, C.-H. Li, C.-W. Fan, Y.-T. Shih, J.-P. Ho, C.-L. Lin, F.-C. Chang, G.-C. Li, C.-N. Chiu, C.-Y. Tsai, H.-C. Chuang, J.-H. Chen, J.-C. Ma, Chi-Yu Liu, C.-C. Chen, C.-J.
Tsai, Tihao Chiang, S.-Y. Lee, and H.-M. Hang, “ISO/IEC JTC1/SC29/WG11 M11117:
Scalable Multimedia Streaming Test Bed for Media Coding and Testing in Streaming Envi-ronments,” July 2004 (69th, Redmond, Washington, USA)
6. H.-K. Hsu, C.-Y. Tsai, H.-C. Huang, H.-M. Hang, and T. Chiang, “ISO/IEC JTC1/SC29/WG11 M10934: Response to Core Experiments in SVC 1b Spatial Transform
& Entropy Coding,” July 2004 (69th, Redmond, Washington, USA)
附錄
(1) H.-K. Hsu, H.-C. Huang, and H.-M. Hang, “An enhanced entropy coding scheme for in-terframe wavelet,” in 2004 Conf. on Computer Vision, Graphics, and Image Processing, Hualien, Taiwan, Aug. 2004.
(2) C.-Y. Tsai, H.-K. Hsu, H.-C. Huang, H.-M. Hang and G.-Z. Wu, “Enhanced motion esti-mation for interframe wavelet video coding,” IEEE International Conf. on Image
Proc-essing '04, Singapore, Oct. 2004
(3) F.-C. Chang, H.-C. Huang and H.-M. Hang, “Combined Encryption and Watermarking Approaches for Scalable Multimedia Coding,” Pacific Rim Conference on Multimedia
2004, Tokyo Japan, Dec. 2004
AN ENHANCED ENTROPY CODING SCHEME FOR INTERFRAME WAVELET Han-Kuang Hsu, Hsiang-Cheh Huang, and Hsueh-Ming Hang
Department of Electronics Engineering,
National Chiao Tung University, Hsinchu, Taiwan, R.O.C.
[email protected] ABSTRACT1
An enhanced entropy coding scheme is incorporated into the interframe wavelet coding architecture in this paper. Interframe wavelet coding has the advantage of SNR, temporal, and spatial scalability, and is a potential candidate for the on-going MPEG-21 scalable video coding (SVC) standard. Motion-Compensated Temporal Filtering (MCTF) and Wavelet Transform Coding are two most essential components in the interframe wave-let coding architecture. The arithmetic entropy subsys-tem is an indispensable element in Wavelet Transform Coding. It produces the final output bitrate. In this paper, we modify the entropy coding syntax/scheme originally specified in the MPEG SVC Core Experiment (CE) reference software. We observe some bit savings of this technique in our simulations based on the conditions specified by the MPEG core experiments; however, the full potential of this technique is yet to be further ex-plored.
1. INTRODUCTION
Video compression is an essential element in multime-dia applications. Conventional video coding systems, including MPEG-1, MPEG-2, H.261 and H.263 interna-tional standards, employ the so-called hybrid coding structure. In these schemes, the reconstructed previous frame is used to predict the current frame after motion compensation.
The on-going MPEG-21 Scalable Video Coding (SVC) standard employs a new approach different from the hybrid coding structure, Motion-Compensated Tem-poral Filtering (MCTF) with Wavelet Transform Cod-ing, to achieve SNR, spatial, and temporal scalability.
Ohm first proposed a motion-compensated t+2D coding structure [1], as shown in Figure 1 [2]. The major dif-ference between the hybrid coding and the t+2D coding is that the latter does not contain the closed-loop (inter-frame) DPCM. In addition, the t+2D coding scheme fulfills for the scalable video coding requirements. One
1 This work is supported in parts by National Science Council (Taiwan, ROC) under Grant NSC 92-2219-E-009-003 and OES, Industrial Technology Research Institute (Taiwan, ROC) under Grant 93B-05.
of the improved and highly efficient realizations of this concept is the interframe wavelet video coder proposed by Woods and his co-workers [2]. This scheme is called Motion Compensated Temporal Filtering – Embedded Zero Block Coding (MCTF-EZBC or MC-EZBC). Its architecture is shown in Figure 2 [4][6]. Essentially, the same basic structure was adopted by the MPEG com-mittee in March 2004 as the first reference model of SVC.
Figure 1. Block diagram of t+2D transform coding system.
Figure 2. The interframe wavelet video coder.
In this paper, we focus on improving the entropy coding scheme in the aforementioned interframe wave-let coding structure. As illustrated in Figure 1 and Figure 2, no matter how the motion compensation is performed in SVC, entropy coding is a must to further reduce the bits in the output bitstream.
The motivation behind our scheme is the observation of clusters of “1”-bits on the wavelet coefficient bit-planes. Thus, we develop our entropy coding based on the quadtree concept. At the end, we compare the per-formance between our proposed scheme and that in [7]
and [8], and show a somewhat better performance of the proposed scheme.
This paper is organized as follows. In Sec. 2, we out-line the motivation behind our proposed scheme. The conventional 3D EBCOT technique is summarized in
Spatial
Sec. 3. In Sec. 4, we describe the coding process of new entropy coding scheme by modifying the existing 3D EBCOT. The changes on CE software for integrating the proposed algorithm are described in Sec. 5. Simula-tion results are shown in Sec. 6, in which we compare the results with the existing scheme. We conclude this paper in Sec. 7.
2. MOTIVATION
In the SVC core experiment one software, the 3D EBCOT entropy coding procedure is applied after MCTF and spatial transform [7][8]. We observe that high energy wavelet coefficients often cluster together [10][9]. In order to save coding bits, we propose a modified coding procedure as described in Section 3.
Essentially we construct another layer that records the bitplane locations of the Significant Bits (SB) of all coefficients. We observe bit savings of this technique in our simulation; however, the full potential of this tech-nique is yet to be further explored.
3. 3D EMBEDDED BLOCK CODING WITH OPTIMAL TRUNCATION SCHEME
In the MPEG SVC core experiment reference software [8], the coefficients are coded by the 3D Embedded Block Coding with Optimal Truncation (3D EBCOT) process after the temporal and spatial subband transform.
Each subband, generated by temporal and spatial trans-forms, is divided into 3D codeblocks, which is coded independently. Next, the entropy coding module is ap-plied to these codeblocks. It encodes each bitplane se-quentially using context-based arithmetic coding.
Three coding operations are employed to encode the samples in a bitplane: [8] [8]
¾ Zero Coding (ZC): When a sample is not yet sig-nificant in the previous bitplane, this primitive
¾ Zero Coding (ZC): When a sample is not yet sig-nificant in the previous bitplane, this primitive