A fully automated Web-based TV-News System[40, 41] consists of three modules: (1) TV news video acquisition, (2) news content analysis, and (3) user interface for news query, search and retrieval. In this section, video acquisition method, user interface and some content analysis methods such as shot detection and key frame extraction are presented.
The news content analysis schemes will be discussed in Section 6.4 in detail.
6.3.1 An overview of the TV-news system
Using digital multimedia techniques to create TV news programs has been a new trend for news media production system. However, TV news has been broadcasted for years, a lot of TV news contents are saved and preserved in thousands of news videotapes. Thus, an automatic hierarchy news generating system is definitely necessary to produce multimedia contents from these tapes. Although there are difficulties to retrieve contents from video, the maturation of multimedia and pattern recognition techniques signals we are now able to conquer all the problems. In general, shot detection [44], speaker identification[45], video optical character recognition (video OCR)[46, 47], and data mining techniques are needed for video analysis and multimedia content generation.
The flow chart of automatic news content generation is depicted in Fig. 6.2. There are two input sources: Cable TV and World Wide Web. At first, TV-news program video is recorded to produce high-quality video for analysis, and to generate streaming video for web browsing. The news video is fed into modules for story segmentation, key-frame
selection and news information tree generating. Then, if the owner needs to protect the content, watermarks are embedded into key frames and the news video are encrypted with scrambling key, as described in Section 5.3.2. Finally, streaming video, key frames and news information tree are stored in a database. We will discuss these technologies in detail in the following sections.
A user could requests through the proposed TV-news service web site to search story by keywords or browse daily news. The detail of web-interface design will be given in Section 6.3.4.
6.3.2 Multimedia data acquisition
In order to automatically generate necessary contents, news video and scripts are collected at the beginning of the work. Then, the captured data is transformed into suitable format before being applied to the following analysis. For general content analysis work, MPEG-1, a well-defined open standard of fair quality video, is the format we need the most.
Except MPEG-1 video, high quality and well-recorded images are needed for close caption extraction and video OCR. For this purpose, the captured TV frames are sampled into portable pixel map (PPM) images. Besides, the system encodes captured news video into ASF format for transiting video over Internet of various bandwidths. The advancement of computer hardware makes capturing and encoding video into three different video-formats at the same time to be possible. In addition, encoding video into all desired formats simultaneously brings many of advantages, such as processing time efficient and event synchronizing accurate.
A robot like, web searching software was also developed to automatically fetches news scripts from net-news web sites.
6.3.3 Key frame extraction
To efficiently browse news story without downloading a whole news video, a set of key frames are selected from sampled video images. The main idea is that the system firstly
Figure 6.3: An example of spatio-temporal slice. The locations indicated by arrow symbols are just the shot-change locations.
cuts video into several series frame sets, named shots, and then picks frames from each shot. The spatio-temporal slice method, proposed by Wah [44] presents the spatio and temporal relationship of video sequences. Because the shot-change brings clear edges in spatio-temporal slice, the shot change locations can be easily detected by conventional edge detection algorithm. An example of spatio-temporal(ST) slice is shown in Fig 6.3.
Two apparent vertical edges divide ST slice into three pieces. These two vertical lines corresponds to the time lines of shot changes.
To catch motion activities of a news story as much as possible, we extract key frames from each shot, and high motion scenes. Then, watermarks are embedded into key frames by schemes proposed in Section 4.3.1. On the other hand, time lines of shot changes are preserved for fingerprinting procedure proposed in Section 5 so that clients’ fingerprints can be changed along with a shot change.
6.3.4 Web-based user interface
This section presents the design and implementation of web-base user interface. First, we would like to briefly describe three servers in the web TV-News system. These servers are database server, web server, and media server. The data generated in Section 6.4 are stored in the SQL database. Web server accepts users’ requests of looking up contents from database server, and then composes and returns the requested pages to users. When users acquire a news video, web server redirects the request to media server. The media server then supplies required video.
At the prototype web site, users can browse news stories by date, or query desired news story by assigning keywords. After opening the starting page, there are two links
(a) News stories list
(b) Key frames and video of story
Figure 6.4: User interface of the proposed web-based multimedia news archive.
for PC users and PDA users respectively to begin the news service. The following expla-nation is for PC users. The main service page is divided into two partitions - top and bottom frames. Users can specify date and channel on the top frame, and browse news stories list of specified date and channel on the bottom one, as shown in Fig. 6.4(a).
Headlines are listed on the bottom-left. And the images displayed on the bottom-right are the representative key-frame of each news story. Users can select stories by clicking on headlines or representative key-frames. The story selected is shown on the bottom frame (see Fig. 6.4(b)). The key frames of this story are presented on the bottom-right all at once. Besides, a embedded window for playing video is also displayed on the top-left corner.
In addition to browsing related news stories by date and channel, users can assign several keywords for related news stories in the database of the proposed web TV-News system. Stories that matched users’ requirements are then listed at the bottom-left frame.