IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 15, NO. 5, MAY 2005 589
Introduction to the Special Issue on
Integrated Multimedia Platforms
M
EDIA communication in the past was associated with a single specific application, typically telephone, TV, or various data transfer on computer networks. With the de-velopment of the Internet in wired or wireless forms, we are witnessing the convergence of different application functional-ities into the same multimedia terminal. Such terminals ideally would support all functionality in different forms or levels depending on the environment in which they are deployed. Currently we can clearly depict four environments: the office, the home, the pocket, and the car. All such “environments” have dedicated terminals that require the implementation of several complex processing standards at different “levels” of performance and with different implementation constraints. In such scenarios new approaches to system design and new implementation challenges appear. The fact that the same processing standards, JPEG, MPEG, and network protocols appear on different terminals seems to facilitate the work of designers customizing basic IP for the specific application. In reality, the lack of high level tools and efficient design methodologies make the work of transforming the “reference standard” behavioral models into implementations according to specific constraints still a very difficult and time consuming task. Much of the time is consumed in “recoding” complex algorithms into implementation languages and reverifying functionality and conformance. Whether the task at hand is porting a multimedia standard onto a given DSP platform, finding the best partitioning between software and hardware coprocessing, defining a new application specified integrated circuit (ASIC) architecture, or minimizing power dissipation for a given performance constraint, we experience that the indi-vidual and combination of design tasks is an extremely difficult problem. New implementation technologies, such as modern reconfigurable hardware, provide new potential advantageous solutions. In reality, they also add new dimensions to the design problem, thus greatly increasing the complexity by offering new ways to realize multiple requirements, while currently not offering appropriate system level tools to assist the designer in exploring several design implementation solutions. As manual optimization becomes impractical, the designer loses the sense of control of the design unless new high-level tools can be developed to increase awareness and efficiency of the solution space exploration.The goal of this Special Issue is indeed to capture recent fundamental developments and achievements in the areas of high-level tool for system-on-a-chip (SoC) design, namely
ar-Digital Object Identifier 10.1109/TCSVT.2005.846889
chitecture exploration, architecture design, algorithms design, design methodology, complexity analysis, and implementation tools.
For this reason, the first part of this Special Issue is dedicated to papers tackling high-level tool issues. Seven papers examine overall architecture design with the emphasis on various design aspects and optimization constraints. Critical path, memory ac-cess latency, memory bandwidth efficiency, data flow optimiza-tion, memory access minimizaoptimiza-tion, and power efficiency are the main topics. The second part of this Special Issue focuses on the optimization of individual modules particularly critical for implementation in an integrated multimedia system and typical common building blocks for some of the various versions of video and image compression standards.
In the first part, the first paper entitled “Evaluation of the Par-allelization Potential for Efficient Multimedia Implementations: Dynamic Evaluation of Algorithm Critical Path” addresses the problem of getting a high-level global view of a complex algo-rithm. The methodology and the associated tool allow the de-signer to dynamically identify the critical path, thus avoiding static code analysis clearly inappropriate in multimedia com-pression implementations tasks and evaluating the paralleliza-tion potential. The two main innovaparalleliza-tions proposed are the pos-sibility to perform the analysis under real input signals and the fact that no limitations are imposed on the usage of program-ming operators and on the usage of any data addressing and memory allocations modes for C and C++.
The second paper entitled “Memory Centric Design of an MPEG-4 Video Encoder” investigates the problem of multi-media system design by combining the algorithmic refinements and transformations with a memory centric optimization methodology so as to derive suitable memory architecture for video encoder systems. The combination of upfront algo-rithmic tuning with memory centric optimizations to transform the video application into a system consisting of functional blocks with localized data processing and a tailored memory hierarchy constitute the essence of the methodology. The re-sulting memory optimized functional description is proposed as the leverage for the cost-efficient mapping of the system on integrated multimedia platforms. Results on a MPEG-4 video encoder design case leading to pipelined system architecture are also reported and analyzed.
On the same issue of designing efficient data flows memory architectures, the third paper entitled “An Efficient Quality-Aware Memory Controller for Multimedia Platform SoC” investigates a new multilayer approach for memory controllers for SoC implementations. Minimization of latency providing different levels of bandwidths is certainly a common and important multimedia constraint. The paper provides results
590 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 15, NO. 5, MAY 2005
of achievable latency minimization of the proposed solution in the case of a STB implementation.
Another important element of consideration for potential op-timization improvements is the cache architecture to be used for multimedia systems. The fourth paper entitled “Software-Con-trolled Cache Architecture for Energy Efficiency” investigates the achievable power dissipation savings on a specific MPEG-2 software decoder when a new software controlled cache archi-tecture is used. The experimental results show considerable gain in energy efficiency when compared to classical cache controls without reducing the performance efficiency.
An example of the high-dimensional design spaces faced nowadays by multimedia system designers is provided in the paper entitled “Power-Rate-Distortion Analysis for Wireless Video Communication Under Energy Constraints.” Using the new dynamic voltage scaling CMOS technology the paper jointly considers for the design the power constraints and the image distortion when reduced complexity algorithms are used by the video encoder. The result is a framework in which the classical video coding rate-distortion behaviors are extended to power-rate-distortion analysis where the goal is to minimize the image distortion under rate and power constraints. Such joint optimization is particularly important for mobile video application terminals.
An overall system optimization tailored to multimedia algo-rithm can also be achieved by means of a flexible instruction set architecture composed by a classical processor instruction set combined with higher level hardware block processing. Such approach is investigated by the paper entitled “Flexible Hetero-geneous Multicore Architectures for Versatile Media Processing Via Customized Long Instruction Words.” The paper studies a flexible reconfigurable controller for sequences of “customized long instruction words” that are able to express a middle-grained parallelism among the higher level hardware blocks and a re-duced-instruction set computer (RISC).
To design sophisticated multimedia system, there are several important tasks such as defining a new architecture specific for a multimedia standard, defining an instruction set for a processor architecture, or guiding the optimization process in terms of control-flow and data-flow for specific architecture. The paper entitled “High-Abstraction Level Complexity Analysis and Memory Architecture Simulations of Multimedia Algorithms” reviews the state of the art of complexity analysis methodolo-gies oriented to the multimedia system design. It demonstrates an ideal integrated tool for the automatic analysis capable of producing complexity results based on rich and customizable metrics, and reports some examples of complexity analysis results from multimedia standards/algorithms.
The last paper of the first part, entitled “Accelerate Video Decoding With Generic GPU” proposes to take advantage of a generic graphics processing unit (GPU) to accelerate the video decoding process by moving some of the decoding functions such as motion compensation to the GPU. The pipeline architec-ture for both central processing unit (CPU) and GPU are specifi-cally designed with such goal and constitute the main innovation proposed in the paper.
In the second part of the Special Issue, the main processing functions or building blocks present in the more common standards are optimized for embedded implementations. They include: motion estimation, discrete cosine transform (DCT) and shape-adaptive transforms. A simplified binary motion estimation engine is studied by the paper entitled “DCT-Based Adaptive Thresholding Algorithm for Binary Motion Estimation” where a binary version of motion es-timation is implemented aiming at low-complexity VLSI implementation.
One of the most distinct features of the MPEG-4 standard is the object-based coding for multimedia information. To achieve the description of video object, the shape-adaptive DCT is a critical module. For mobile applications, the tradeoffs between functionality and power efficiency are challenging. The goal for low power implementations of shape-adaptive DCT modules are studied in the paper entitled “An Energy-Aware IP Core De-sign for the Variable-Length DCT/IDCT Targeting at MPEG-4 Shape-Adaptive Transforms.”
In current times, the challenge of multimedia systems imple-mentations is the need of both flexible architectures and yet ef-ficient implementations. The architecture should be as much as possible reconfigurable but at the same time easy to optimize for multiple functions. The paper entitled “A Method for Designing High-Radix Multiplier-Based Processing Units for Multimedia Applications” proposes such an architecture specific for video and image processing focusing on memory block optimization, pipeline stage parallelization and multiplexed interconnections. It provides an excellent case study for the design of this class of architectures for the implementation of integrated multimedia system.
For most recently adopted multimedia standards, the DCT is a widely used component for a multimedia system. Thus, an efficient implementation for DCT is a critical part for a mul-timedia system implementation. In the last article entitled “A New Time Distributed DCT Architecture for MPEG-4 Hard-ware Reference Model,” the authors presented a very efficient implementation of the DCT module. The proposed time dis-tributed architecture exploits the computational redundancy ef-ficiently.
Although the selection of papers presented in this special issue is certainly of high quality and attacks from different sides the implementation and design challenges, the overview pro-posed is certainly far from being exhaustive and complete. The difficulties of the problems leave many aspects open to further development and advances. The space and time limits of this Special Issue also forced us to narrow the wide investigations and high quality contributions that have been received.
The Guest Editors would like to thank all the people who contributed to this Special Issue including the authors who submitted papers that we were unable to accommodate due to space limits. Many thanks also to the reviewers for their professional and accurate evaluations of the papers provided in a timely manner. The Guest Editors also wish to thank Dr. T. Sikora for giving us opportunity to organize this Special Issue.
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 15, NO. 5, MAY 2005 591
Last but not the least, we owe many thanks to S. Wang for his valuable technical support to the project.
TIHAOCHIANG
Institute of Electronics
National Chiao Tung University Hsinchu 300, Taiwan, R.O.C. MARCOMATTAVELLI
Signal Processing Institute
Ècole Polytechnique Fédérale de Lausanne (EPFL) CH-1015 Lausanne, Switzerland
ROBERTD.TURNEY
Xilinx Research Laboratories Xilinx Inc.
Watertown, WI 53094 USA
Tihao Chiang (S’91–M’95–SM’99) was born in Cha-Yi, Taiwan, R.O.C., in 1965. He received
the B.S. degree in electrical engineering from the National Taiwan University, Taipei, in 1987, and the M.S. and Ph.D. degrees in electrical engineering from Columbia University, New York, in 1991 and 1995, respectively.
In 1995, he joined the David Sarnoff Research Center, Princeton, NJ, as a Member of Technical Staff. Later, he was promoted to Technology Leader and a Program Manager at Sarnoff. While at Sarnoff, he led a team of researchers and developed an optimized MPEG-2 software encoder. For his work in the encoder and MPEG-4 areas, he received two Sarnoff achievement awards and three Sarnoff team awards. Since 1992, he has actively participated in ISO’s Moving Picture Experts Group (MPEG) digital video coding standardization process with particular focus on the scalability/compatibility issue. He is currently the co-editor of part 7 on the MPEG-4 committee. He has made more than 90 contributions to the MPEG committee over the past 11 years. His main research interests are compatible/scalable video compression, stereoscopic video coding, and motion estimation. In September 1999, he joined the faculty at National Chiao-Tung University, Hsinchu, Taiwan, R.O.C. He has published over 50 technical journal and conference papers in the field of video and signal processing. He holds 13 U.S. patents and 30 European and worldwide patents.
Dr. Chiang was a corecipient of the 2001 Best Paper Award from the IEEE TRANSACTIONS ONCIRCUITS ANDSYSTEMS FOR
VIDEOTECHNOLOGY. He was a Guest Editor for IEEE TRANSACTIONS ONCIRCUITS ANDSYSTEMS FORVIDEOTECHNOLOGYand is the Chair of Visual Processing Technical Committee of IEEE Circuits and Systems Society.
592 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 15, NO. 5, MAY 2005
Marco Mattavelli was born in Milano, Italy, on July 18, 1961. He received his Diploma of
elec-trical engineering degree from the Politecnico di Milano, Milano, Italy, in 1987.
In 1988, he joined the Philips Research Laboratories, Eindhoven, The Netherlands, in the framework of EUREKA-95 (HDMAC) project. Main research activities regarded channel and source coding for optical recording and electronic photography. In 1990, he joined the CSA Philips Research Division of Monza, Italy, working on signal processing of TV and HDTV signals. In October 1991, he joined the Signal Processing Laboratory (LTS) of the Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland, where he received the Ph.D. degree in 1996 with the thesis: “Motion analysis and estimation: From ill-posed discrete inverse linear problems to MPEG-2 coding.” At EPFL, he has been involved in various European projects of 4th and 5th Framework (VADIS, COUGAR), research and didactic activities. In 1995, he was Visiting Researcher at the Center of Operational Research and Applied Mathematics, Cornell University, Ithaca, NY. In July 1996, he joined the Integrated System Laboratory (LSI) of EPFL where he has been involved in ATLANTIC, EMPHASIS, OCCAMM, CARROUSO, MOSES,and ENTHRONE European projects of 5th and 6th Framework, all projects dealing with specification and implementation of multimedia systems. He has been involved in several collaboration with industries and in the ISO/IEC JTC1/SC29/WG11 standardization activities (better known as MPEG), for which he is currently Chairman of the Implementation Study Group (ISG). For his work and contributions on the standardization of MPEG-4, he received the ISO/IEC Award in 1998 and in 2001. His major research activities and interests include architectures and systems for video coding, real-time multimedia systems, high-speed image acquisition and video processing, motion analysis and estimation, neural networks for image and signal processing, applications of combinatorial optimization to signal processing. He holds patents in the multimedia and video processing fields. He is the author or coauthor of more than 80 research papers and one book.
Robert D. Turney was born in Chilton, WI, in 1962. He received the B.S, M.S., and Ph.D.
de-grees in electrical engineering from the University of Wisconsin, Milwaukee, in 1989, 1992, and 2005, respectively.
He has taught courses in DSP and VLSI at the Milwaukee School of Engineering and the Uni-versity of Wisconsin from 1989 to 1999 holding Lecturer and Research Associate positions. From 1994 to 1997, he worked at Camtronics Medical Systems as a DSP Systems Engineer designing medical imaging products. In 1997, he joined Xilinx, Watertown, WI, where he currently is a Senior Staff researcher in Xilinx Research Labs specializing in multimedia, video, and image processing algorithms. He has been involved in video processing since 1992 in projects such as real-time lossless compression, image enhancement, and noise reduction for real-time image ac-quisition. Recent activities are related to implementation of new compression standards in the ISO/IEC’s JPEG and MPEG standardization committees, namely JPEG2000 and MPEG-4.
