Chapter 2 Background and Survey of Related Work
3.2 Investigation of Wavelet Video Bit Streams with Data Losses
3.2.2 Multiple-adaptation and Fine-level FEC Using R-λ Curve Fitting Function
Multimedia distribution over heterogeneous networks and devices has become the mainstream enabling technology for new generations of services. For distribution and playback of a video content on various devices under different network conditions, scalable video coding schemes are usually used. A typical approach for scalable coding is to use a layered coding approach such as MPEG-4 Simple Scalable Profile or fine-grained scalable video coding scheme. In these approaches, the bitstream quality of the encoded video is optimized subject to certain bitrate conditions. Adaptation of the encoded video file to a different target bitrate usually results in sub-optimal bitstreams.
A different approach from the layered coding schemes is to design a scalable codec that produces embedded scalable bitstreams without inherent layered structures. The wavelets-based video codecs belong to this category. After the encoding procedure, there is no inherent layer structure for wavelet video bitstreams, video parameters such as resolution, frame rate, and bitrate can be dynamically adapted with fine granularity. If the R-D tradeoff information is embedded in the bitstream, the adaptation process can produce an R-D optimal bitstream at run-time for the target application. One major advantage of wavelet codecs over coarse-granularity layer-based codecs is that wavelet bistreams can
facilitate multiple adaptations. For example, in Fig. 3-8, the video server transmits dynamically adapted scalable bitstreams to two different devices, namely the notebook and the cellular phone. Upon reception of the embedded bitstreams, the notebook plays the high quality bitstream on its screen. In addition, the multiple adaptation application of the notebook truncates (adapts) the received bitstream further and send the new bitstream to another device (the PDA) with tighter channel and device constraints. For the other distribution chain in Fig. 3-8, the cellular phone first receives an adapted bitstream from the server and plays it on its internal large screen. Later, when the user decides to watch the video on the small external screen to conserve power, the video decoder can extract and decode only part of the received bitstream and display a smaller resolution video.
Video Server
1stAdaptation 2nd Adaptation
1st Adaptation 2nd Adaptation
Final receiver Intermediate
receiver/server
Video display on small screen Video Server
1stAdaptation 2nd Adaptation
1st Adaptation 2nd Adaptation
Final receiver Intermediate
receiver/server
Video display on small screen
Figure 3-8: Two examples of multiple adaptation applications where the same video content is adapted several times down the distribution chains
The R-D information of the wavelet codec is the discrete rate-lambda (R-λ) pairs for all coding passes. In addition to multiple adaptations of video data, R-D information is also very useful for fine-level FEC protection of video data. Several frameworks for wavelet based video streaming have been proposed in the literature recently [71, 72, 78, 79].
However, none of existing works allows multiple-adaptation of fine-level of FEC protection data. The system can be configured in multiple adaptations of video data. R-D information is also very useful for fine-level FEC protection of video data. In addition,
fixed amount of FEC protection consumes considerable overhead, if channel errors are less than expected. Section 3.2.1 proposed a fine-level FEC protection/packetization mechanism of wavelet video data, but multiple adaptations of FEC
Table 3-1: The amount of R-D information.
R-D information (bvtes, % in bitstream) Sequence
Name
Resolution/
frame rate/bit
rate(kbps) MSRA Proposed
Method
Table 3-2: R-D information overhead vs. GOP size R-D information (bytes, % in bitstream) Sequence
Name
Resolution/
GOP size/bit
rate(kbps) MSRA Proposed
Method
codes are not considered. Transmission of the R-D information was a non-negligible fraction of the data rate. The R-D information can be used to calculate the degree of importance of the wavelet coefficients given estimated packet loss rate of the channel. The granularity of the protection level can be fine-tuned for different wavelet video coefficient coding passes. Although the proposed technique performs very well in practice, it does not allow for multiple adaptations since the R-D information will be discarded after packetization due to its nontrivial overhead. The amount of R-D information overhead in different bit rates and frame rates is illustrated in Table 3-1. And the amount of R-D information overhead in various GOP sizes is shown in Table 3-2. The average R-D information percentage in the bitstream is ranging from 1.9% to 4.9%.
Figure 3-9: R-λ curve fitting function of STEFAN in block 0 of SSB 0 of the TSB P(Ht, Y) The proposed model translates the discrete R-λ pairs into the R-λ function, where λ is R-D slopes. In addition, the R-λ function is incorporated into the fine-level FEC framework to facilitate multiple adaptations. For this purpose, a R-λ curve fitting function for multiple adaptations was proposed in our previous study [91]. The R-λ curve fitting function is an approximation of the functional relationship between rate R and the Lagrange parameter λ. We assume that the R-λ curve fitting function: λ= αeβR in coding block level can be rewritten as lnλ = lnα + β⋅R, as shown in Fig. 3-9. Therefore, we have an
over-determined system of Eq. (3-3):
where n > 2. A least squares fitting technique used for solving the parameter α and β. Note that for fine-level FEC protection, the degree of protection level s should be based on the importance of the video data. The importance of the coefficients within a coding block in a subband can be ranked based on the R-D information of the coding block. After wavelet decomposition, the subbands can be arranged and indexed from low to high frequencies.
The smaller the index is, the lower the frequency is. Therefore, each coding block in subband i has a temporal subband index ωi and a spatial subband index τi. The importance of the coefficients in a coding pass is first determined by the importance of the coding block it is located in. The importance of a coding block is in turn determined by the subband it is located in. The importance factor, Wi , of a coding block is computed as in Eq.
(3-4):
where T is the maximal temporal level index, Z is the maximal spatial subband index, and U1 is a weighting factor.
The level of FEC protection is defined by the value s, the number of correctable symbols. Without loss of generality, assume that the bitstream of a coding block j is divided into m codeword. The protection level sj,x of the coefficients in coding pass x of coding block j is computed as in Eq. (3-5):
⎪⎩ parameters for the coding block j, Rj,x is the length of the xth RS codeword in coding block j, npl denotes the estimated number of packet losses per second, and ω is a scale factor determined empirically. In Eq. (3-5), the level of protection decreases following fractional bitplane coding pass order, and si,0 ≥ si,1 ≥…≥ si,m-1.
For some multiple-adaptation applications, the second (and above) adaptations may be due to the change of device capabilities instead of channel conditions. For such case, there is no need to re-compute the FEC codes since the level of protection does not change.
However, repacketization may still be necessary for efficient transmission of the re-adapted data.