H.264 基線解調器之數位訊號處理平台實現以及四分域為基礎的抗錯與錯誤修補演算法

國

立

交

通

大

學

電子工程學系電子研究所碩士班

碩

士

論

文

H.264 基線解調器之數位訊號處理平台實現以及

四分域為基礎的抗錯與錯誤修補演算法

H.264 Baseline Decoder Implementation and

Quadro-Field Based Error Resilience and Error

Concealment Techniques

指導教授：王聖智 博士

研 究 生：陶世軒

H.264 基線解調器之數位訊號處理平台實現以及四分域為基

礎的抗錯與錯誤修補演算法

H.264 Baseline Decoder Implementation and Quadro-Field

Based Error Resilience and Error Concealment Techniques

研 究 生: 陶世軒

S t u d e n t: Shih-Hsuan Tao

指導教授: 王聖智

A d v i s o r: Shengjyh Wang

國 立 交 通 大 學

電子工程學系電子研究所碩士班

碩 士 論 文

A Thesis

Submitted to Department of Electronics Engineering & Institute of Electronics National Chiao Tung University

in partial Fulfillment of the Requirements for the Degree of Master

in

Electronics Engineering September 2005

HsinChu, Taiwan, Republic of China

H.264 基線解調器之數位訊號處理平台實現以及四分域為

基礎的抗錯與錯誤修補演算法

研究生: 陶世軒

指導教授: 王聖智 博士

國立交通大學

電子工程學系 電子研究所碩士班

摘要

在本文中，我們針對 H.264 視訊編碼的解調端實現在數位訊號處理平台上 面，以及以四分域為基礎提出了新的錯誤抵抗與錯誤修補的技術。H.264 是 ITU-T 提出的一種新的影像編碼標準，主要是利用一些新加入的特徵以及功能，可以達 到高壓縮率的效果。但是因為如此，所以會增加一些運算的時間以及複雜度。所 以在本文中，我們有利用 CCS 來針對 H.264 基線的解調器作複雜度的分析，以及 把 H.264 基線解調器有效率的實現在數位訊號處理平台上面。除此之外，因為現 在無線網路傳輸的頻寬有限，而且變動很大，所以我們也提出了四分域的編碼方 式，可以使得在解調後處理更加容易。在此編碼方式中，我們也提出了三種編碼 的架構。為了減低彼此間的關連性，所以我們採用了其中關聯性以及壓縮率都可 以接受的架構，並且也針對不同的錯誤情形做修補的討論。

H.264 Baseline Decoder Implementation and Quadro-Field

Based Error Resilience and Error Concealment Techniques

Student:

Shih-Hsuan Tao

Advisor:

Dr.

Shengjyh Wang

Institute of Electronics

National Chiao Tung University

Abstract

In this thesis, we implement the H.264/AVC Baseline decoder on a DSP board and propose a new Quadro-Field Based Error Resilience and Error Concealment Technique. H.264 is ITU-T’s new video coding standard. H.264/AVC provides high coding efficiency through added new features and functionalities. These new features and functionalities will directly affect the cost of an H.264/AVC system. In this paper, we use Code Composer Studio (C.C.S) which is a DSP code environment provided by Texas Instruments to profile H.264/AVC computation complexity. Besides, in wireless communications, bandwidth is limited and varying. Here, we propose a Quadro-Field based coding technique and three prediction modes to deal with the error resilience problem and the error concealment problem.

Contents

Chapter 1 Introduction...10

Chapter 2 Introduction to H.264/AVC...11

2.1 The H.264/AVC Codec ... 11

2.1.1 The H.264/AVC Encoder ...12

2.1.2 The H.264/AVC Decoder ...13

2.2 H.264 Structure ...14

2.2.1 The Baseline Profile...14

2.2.1.1 Slices and slice groups ...15

2.2.1.2 Intra prediction...16

2.2.1.3 Inter prediction...16

2.2.1.4 Deblocking Filter ...17

2.2.1.5 Transform and Quantization ...18

2.2.1.6 Entropy coding...18

2.2.2 The Main Profile ...18

2.2.2.1 B slices ...18

2.2.2.2 Interlaced Video...19

2.2.2.3 CABAC...20

2.2.3 The Extend Profile ...20

2.2.3.1 SP and SI slices ...20

2.2.3.2 Data Partition ...20

2.3 Error control techniques in H.264...21

Chapter 3 Environments of DSP Implementation ...22

3.1 The DSP Board ...22

3.2 DSP Chip...23

3.2.1 Central Processing Unit (CPU) ...24

3.2.2 Memory Architecture ...29

Chapter 4 H.264 Decoder Implementation and Optimization ...33

4.1 Profile of H.264 decoder on DSP ...33

4.2 Optimize C/C++ Code ...35

4.2.1 Setting of CCS Compiler ...36

4.2.2 Software Pipelining...37

4.2.3 Using Intrinsics ...38

4.2.4 Packed data Processing ...39

4.2.5 Memory Usage Strategy...39

4.2.6 Optimization Results...40

4.3.1 Over a Single DSP ...42

4.3.1.1 Experiment Results of the Whole System ...43

4.3.2 Over Two DSP’s...47

4.3.2.1 Experiment results of the whole system ...47

Chapter 5 Quadro-Field Coding on H.264 ...50

5.1 Error Control Techniques...50

5.1.1 Error Resilience Tools in H.264...50

5.1.2 Error Concealment Tools ...51

5.2 Quadro-Field Coding ...53

5.3 Error Concealment ...56

5.3.1 One Field Loss ...57

5.3.1.1 Bottom-Right Field ...57

5.3.1.2 Bottom-Left Field ...59

5.3.1.3 Top-Right Field...60

5.3.1.4 Top-Left Field ...61

5.3.2 Two Field Loss...61

5.3.2.1 Spatial interpolation ...61

5.3.2.2 Spatial interpolation & temporal concealment ...64

Chapter 6 Experiment Results ...66

LIST OF FIGURE

FIG 2-1SCOPE OF H.264/AVC STANDARDIZATION[2]... 11

FIG 2-2STRUCTURE OF H.264/AVC VIDEO ENCODER.[2]... 12

FIG 2-3H.264 ENCODER [3] ... 13

FIG 2-4H.264 DECODER [3] ... 13

FIG 2-5H.264 PROFILES [5]... 14

FIG 2-6SUBDIVISION OF A PICTURE INTO SLICES [2]... 15

FIG 2-7SUBDIVISION OF A PICTURE INTO SLICES USING FMO[2]... 15

FIG 2-8FIVE OF NINE PREDICTION MODES [2]... 16

FIG 2-9SEGMENTATIONS OF MACROBLOCK FOR MOTION COMPENSATION [2]... 17

FIG 2-10PERFORMANCE OF THE DEBLOCKING FILTER FOR HIGHLY COMPRESSED PICTURES [2]... 17

FIG 2-11(A) PAST/FUTURE (B) PAST (C) FUTURE [3] ... 19

FIG 2-12PROGRESSIVE AND INTERLACED FRAMES [2] ... 19

FIG 2-13CONVERSION OF A FRAME MACROBLOCK PAIR INTO A FIELD MACROBLOCK PAIR.[2] ... 20

FIG 3-1BLOCK DIAGRAM OF VP3[6]... 23

FIG 3-2BLOCK DIAGRAM OF TMS320DM642[7]... 25

FIG 3-3DATA PATH [7]... 27

FIG 3-4VIDEO PORT BLOCK DIAGRAM [12] ... 32

FIG 4-1DECODER PROGRAM FLOW... 33

FIG 4-2DISTRIBUTION OF CLOCK CYCLE OF EACH FUNCTION... 34

FIG 4-3INTRINSIC FUNCTIONS OF THE TIC6000 SERIES DSP(PART.)[11]... 39

FIG 4-4 CLOCK CYCLE OF DIFFERENT QP ... 44

FIG 4-5FLOWCHART FOR IMPLEMENTATION OVER A SINGLE DSP ... 46

FIG 4-6FLOWCHART FOR IMPLEMENTATION OVER TWO DSP’S... 49

FIG 5-1 ... 52

FIG 5-2 ... 52

FIG 5-3THE PROPOSEDQUADRO-FIELD PACKAGING... 53

FIG 5-4A TYPICAL PREDICTION MODE OF FRAME CODING... 53

FIG 5-5THREE PREDICTION SCHEMES OF QUADRO-FIELD CODING... 56

FIG 5-6INTERPOLATION OF THE LOST PIXEL X ... 57

FIG 5-7SIMULATION RESULTS OF TEST SEQUENCE ‘FOREMAN’... 58

FIG 5-8EXPERIMENT RESULTS OF SPATIAL INTERPOLATION... 58

FIG 5-9EXPERIMENT RESULTS OF SPATIAL INTERPOLATION... 59

FIG 5-10INTERPOLATION OF THE LOST PIXEL X ... 59

FIG 5-11SIMULATION RESULTS OF TEST SEQUENCE ‘FOREMAN’... 60

FIG 5-12SIMULATION RESULTS OF TEST SEQUENCE ‘FOREMAN’... 60

FIG 5-14SIMULATION RESULTS OF TEST SEQUENCE ‘FOREMAN’... 62

FIG 5-15SIMULATION RESULTS OF TEST SEQUENCE ‘FOREMAN’... 62

FIG 5-16SIMULATION RESULTS OF TEST SEQUENCE ‘FOREMAN’... 63

FIG 5-17SIMULATION RESULTS OF TEST SEQUENCE ‘FOREMAN’... 64

FIG 5-18SIMULATION RESULTS OF TEST SEQUENCE ‘FOREMAN’... 65

FIG 5-19SIMULATION RESULTS OF TEST SEQUENCE ‘FOREMAN’... 65

LIST OF TABLE

TABLE 3-1FUNCTIONAL UNITS AND OPERATIONS PERFORMED [8] ...28

TABLE 3-2FUNCTIONAL UNITS AND OPERATIONS PERFORMED [8] ...28

TABLE 3-3DIFFERENCE BETWEEN C62X/C67X AND C64X HPI[10]...30

TABLE 4-1CLOCK CYCLE OF THE MOST COMPLEXITY FUNCTION...34

TABLE 4-2PROFILE OF THE H.264DECODER : DECODE_ONE_MACROBLOCK...35

TABLE 4-3PROFILE OF THE H.264DECODER : READ_ONE_MACROBLOCK...35

TABLE 4-4 COMPILER OPTIONS FOR PERFORMANCE [16]...36

TABLE 4-5COMPILER OPTIONS THAT HAD BETTER BE AVOIDED [16] ...37

TABLE 4-6 CLOCK CYCLE OF THE H.264 DECODER...41

TABLE 4-7 CLOCK CYCLE OF THE H.264 DECODER : DECODE_ONE_MACROBLOCK...41

TABLE 4-8 CODE SIZE OF THE H.264 DECODER :DECODE_ONE_MACROBLOCK...41

TABLE 4-9 CLOCK CYCLE OF THE H.264 DECODER: READ_ONE_MACROBLOCK...42

TABLE 4-10 CODE SIZE OF THE H.264 DECODER: READ_ONE_MACROBLOCK...42

TABLE 4-11RELATION BETWEEN BITSTREAM SIZE AND FUNCTION...44

TABLE 4-12 OVERALL SPEED USING OPTIMIZATION AND NO OPTIMIZATION...45

TABLE 4-13OVERALL SPEED WITH OPTIMIZATION AND WITHOUT OPTIMIZATION...48

Chapter 1 Introduction

H.264/AVC video coding is a high coding efficiency video coding standard [1]. Although H.264/AVC is still based on block-based motion compensation and transform coding framework, it provides high coding efficiency through added new features and functionalities. These new features and functionalities will directly affect the cost of an H.264/AVC system. On the other hand, the demand for multimedia services over internet is steadily increasing, especially in wireless communication. In wireless communications, bandwidth is limited and varying. In order to improve the video quality, a robust error resilient video coding scheme is in need.

In this paper, we implement an H.264/AVC baseline decoder on the Video Parallel Programmable Platform (VP3) DSP board, and proposed a Quadro-Field based error resilience and error concealment technique. In Chapter 3, we introduce the VP3 and the DSP TMS320DM642. Then in Chapter 4, we use Code Composer Studio (C.C.S), which is a DSP code environment provided by Texas Instruments to profile H.264/AVC computation complexity and use some optimization techniques specific to TMS320DM642 to accelerate our H.264/AVC decoder. We also implement the H.264/AVC decoder over two DSPs.

In Chapter 5, we proposed a Quadro-Field based coding technique and three prediction modes. The corresponding error concealment method for each Quadro-Field will be discussed. At the end of this thesis, conclusions are to be given.