II. The Architecture Of Uma Multimedia Delivery System
For achieving UMA, we propose a video server that contains the key modules described in MPEG-21 [1]. In this model, we combine the tools as referred to MPEG-4 Fine Granularity Scalability (FGS) [2], MPEG-4 Simple Profile, MPEG-7 [3], Digital Watermarking techniques [4], and Internet protocols.
A. Efficient FGS-to-Simple Transcoding
To fit with the issue of content adaptation according to terminal capability, we propose a real-time transcoding system that converts the FGS bitstreams into Simple Profile bitstreams covers five modules as shown in Figure 1(a).
1) Video Content Capture
The source video is inputted from the image acquisition device and saved in a digital form like YUV or RGB formats.
2) FGS Encoder and Bitstream Archive
Each video signal is encoded as FGS bitstreams and stored in mass storages. When any request from users, the server will send bitstreams directly to the terminals, or pass them through the Transcoder when format conversion is necessary.
3) Real-time Transcoder:
The transcoding depends on the channel conditions, terminal capabilities, and video content features. Assuming the terminals only support the decoding processes of MPEG-4 Simple Profile bitstreams. When justifying the transmission conditions, the server will ask the transcoder to convert the archived FGS bitstreams into Simple Profile bitstreams with specified formats, sizes, and qualities in an adaptive manner. The real-time FGS-to-Simple transcoder [5] is as illustrated in Figure 1 (b).
The reference method to perform FGS-to-Simple transcoding adopts a cascaded architecture that connects a FGS decoder and a Simple Profile encoder. Our primary objective is to simplify this reference architecture and demonstrate the high quality results of our FGS-to-Simple transcoder.
In the proposed architecture, the motion vectors within the FGS base layer bitstream are reused in MPEG-4 Simple Profile encoder. Additionally, the transcoding is processed in DCT-domain that can provide a low-complexity transcoder.
4) Channel Monitor:
[Base + Enh] Storage
Transcoder FGS-to-Simple
Enh Layer Rate Reduction
Send terminal capabilities to server for format conversion
Network
Send network condition to server for rate control
Device 2
[Base + Enh] Storage
Transcoder FGS-to-Simple
Enh Layer Rate Reduction
Send terminal capabilities to server for format conversion
Network
Send network condition to server for rate control
(a) Proposed architecture
VLD VLC
(b) FGS-to-Simple transcoder
Figure 1. The application scenario of the proposed UMA multimedia delivery system that employs the archived FGS bitstreams. Figure (b) is the block diagram of the FGS-to-Simple Transcoder in Figure (a).
It accepts feedback information from the terminals and also estimates the characteristics of channels, which mean the round-trip time, packet lost ratio, bit error rate, and bandwidth. All obtainable information is passed to the server for adapting the content delivery.
5) Terminals:
Priori to receiving the bitstream, the terminal exchanges its capabilities with the server. As shown in Figure 1(a), the terminals of different decoding capabilities including FGS and Simple Profile are supported. Consequently, the terminals receive and reconstruct the demanded video signals.
Thus in our framework, the source video is encoded and archived as FGS bitstreams, which can provide various QoS service like SNR scalable video coding schemes [2]. With the FGS bitstreams saved in FGS BitStream Archive module, the proposed system can serves heterogeneous terminals through the Internet. Moreover, according to Internet and Terminal devices capabilities, the Channel Monitor can adapt the different resources to each Terminal.
B. FGS Streaming on the Internet
The delivery of multimedia information to mobile device over wireless channels and/or Internet is a challenging problem because multimedia transportation suffers from bandwidth fluctuation, random errors, burst errors and packet losses [2]. However, it is even more challenging to simultaneously stream or multicast video over Internet or wireless channels under UMA framework. The compressed video information is lost due to congestion, channel errors and transport jitters. The temporal predictive nature of most compression technology causes the undesirable effect of error propagation.
To address the broadcast or Internet multicast applications, we proposed a novel technique named Stack RFGS (SRFGS) to improved the temporal prediction of RFGS [6]. SRFGS first simplified the RFGS prediction architecture and then generalized its prediction concept as: the information to be coded can be inter-predicted by the information of the previous time instance at the same layer.
With this concept, the RFGS architecture can be extend to multiple layers, which form the stack architecture. While RFGS can only optimize at one operating point, SRFGS can optimize at several operating point to serve much wider bandwidth with superior performance. With the biplane coding and leaky prediction that used in RFGS, SRFGS hold its fine granularity and error robustness.
SRFGS can also support temporal scalability by simply dropping some B-frames in the FGS server.
An optimized MB-based alpha adaptation is proposed to further improve the coding efficiency.
SRFGS has been proposed to MPEG committee in [11] and has been ranked as one of the best in the Report on Call for Evidence on Scalable Video Coding [12].
C. FGS-Based Video Streaming Test Bed for MPEG-21 UMA with Digital Item Adaptation
As shown in Figure 2, we are developing an FGS-based unicast video streaming test bed, which is now being considered by the MPEG-4/21 committee as a reference test bed [7], [8]. The proposed system supports MPEG-21 DIA scheme which leads to a more strict evaluation methodology according to the MPEG committee specified common test conditions for scalable video coding. It provides easy control of media delivery with duplicable network conditions. To provide the best quality of service for each client, we will propose relevant rate control, error protection, and transmission approaches in the content server, network interface, and clients, respectively.