Architecture of Streaming Server System

In the server workspace, there are five modules, each of which is responsible for some tasks shown in Figure 23 and is listed below.

1. 3GP Parser module

- It is responsible for reading and parsing MPEG-4 compressed video files.

- It handles the seventh task presented in Figure 23.

2. 3GP Creator module

- It is responsible for creating MPEG-4 files (.mp4) and virtual files (.3gp) for clients to access live-captured audiovisual contents.

- It handles the ninth task shown in Figure 23.

3. Sub Server module

- It deals with all the matters about the transmission of streaming data on the server side, including the connection initiations, the connection negotiation, the connection establishments, the streaming of RTP packets, and RTSP related jobs.

- It handles the second to sixth, and thirteenth to fourteenth tasks presented in Figure 23.

4. Server Dialog module

- It is responsible for defining and creating the user interface for the server, and interacting with users.

- It handles the first, and tenth to twelfth tasks shown in Figure 23.

5. SP_encore.lib library

- It is responsible for compressing the captured frames into the MPEG-4 format.

- It handles the eighth task in Figure 23.

How the five modules link to each other is depicted in Figure 26. The top block, the graphical user interface (GUI) represents the server dialogue. The MPEG-4 block is equivalent to the SP_encore.lib library. The gray blocks, as a whole, symbolize the sub server module. By comparing Figure 26 with Figure 25, it is not hard to realize that the architecture of the proposed server system mimics the typical one. However, they do differ in some parts. Beneficially, the streamer in the proposed server is enhanced by the bandwidth smoother, whose implementation details are given in the later section.

IP Network

Figure 26 – Architecture of server system

Stepping a little bit deeper, on the top of Figure 26, the graphical user interface (GUI) is where the program will interact with the user. The user can trigger events by typing in texts in the boxes or pressing down the buttons provided in the GUI. For example, initially, the user can start the streaming service by pressing the start button. This event triggering can take place because the arrow pointing to both the GUI and the streaming service blocks provides means of communication between the two. Because at any time, a client may want to connect to the server in order to view the images captured by one of the cameras (wireless and web cameras) in real-time or stored in the server storage, the GUI block has to encode raw images acquired from the cameras into MPEG-4 format by using the MPEG-4 encoder. While compressing these images periodically and place them on a buffer, the server has to employ 3GP creator module in order to create a virtual file in the disk, so when a client connect to the server, the client will see that there are MPEG-4 files which are associated with different cameras available in the server storage. Whether the client is interested in the stored or live-captured audiovisual contents, the server will use the 3GP parser module to parse the MPEG-4 file (i.e. whether real or virtual, respectively) that the client demands. In turn, the 3GP parser will feed information to both the bandwidth smoothing and the RTP packetizer modules. To the former module, it will provide the presentation timing and the size of each compressed frames. With this information, the former module can calculate a smoother transmission plan according to the algorithm described in the earlier section. To the later module, the 3GP parser will not only provide the information given to the former one, but also offer a single compressed frame data to the packetizer whenever the packetizer is requesting one. The packetizer, as implied by its name, packetizes frames into fixed-size RTP packets in accordance with RTP [2]. In addition, strictly regulated by the bandwidth smoother, which follows the predefine transmission plan precisely, the packetizer feeds the RTP packets to the data streamer in a controlled manner. The data streamer will stream each RTP packet to the client via the established audio/video channels. If the user of the server presses the pause button, or the RTSP state and message processor receives a valid PAUSE request sent by the client via the RTSP channel, the processor will inform the data streamer to stop the streaming process. The data streamer will halt its transmission and perform necessary calculations in order to stream timely as the presentation is resumed.

Moreover, if the user presses the stop button, or the processor receives a valid STOP request, the processor will inform the data streamer to stop the streaming process and end the streaming session. The data streamer will terminate its transmission and release all the resources associated with the session.

4.3.1.1 Threads Used in Streaming Server System

StatusTimeFunc()

Figure 27 – Threads of server system

Having talked about the tasks and the architecture of the server system, it is time to explain how the system can be implemented in software. Intuitively, since lots of tasks have to be handled at the same time by the server program, multiple threads are definitely needed.

Indeed, referring to Figure 27, there are several threads running simultaneously in the server system when the connections are established with clients. Note that the overlaps between the gray ellipses/rectangles and threads (white rectangles) represent the relationships between the shared data (gray shapes) and the threads. Specifically, the following threads will execute once the server starts:

z GUI thread

- Responsible for interacting with the user of the server program.

z Transcoder()

- Responsible for acquiring audiovisual contents from various types of cameras, encoding or transcoding them into MPEG-4 format, and placing the resulted bitstreams onto the buffer where StreamerletIs() thread can access.

z StatusTimeFunc()

- Responsible for collecting and displaying all the clients’ statuses on the GUI.

z OverallListenerThread()

- Responsible for accepting new incoming connections. Every time a new connection is accepted and established successfully, it will create an instance of

“RTSPProcessorThread()”, which from this point, will take control of this connection.

z PacketScheduler()

- Responsible for injecting RTP packets into network via UDP sockets. It is the data streamer for transmission over UDP.

However, the following group of threads will start only when a client is connecting to the server. For each new RTSP session, the system will start a new group of these threads.

z RTSPProcessorThread()

- Responsible for accepting, parsing, extracting, and replying the RTSP messages, and performing necessary state transitions. It is equivalent to the session controller shown in Figure 25.

z StreamerletIs()

- Responsible for packetizing each data frame into RTP packets if UDP is used as the transmission protocol. It acquires frames from the target MPEG-4 file using 3GP parser module and packetizes them according to the predefined schedule for the requested audiovisual content. It places packetized RTP packets onto the buffer where “PacketScheduler()” can access. On the other hand, it TCP is selected as the transmission protocol, this thread will also acquire frames from the target MPEG-4 file according to the predefined schedule for the requested audiovisual content, but however, will not packetize them. Then it will place these acquired frames onto the buffer where “TunnelMain()” can access.

z BandwidthSmoother()

- Responsible for controlling the frequency of packetizing a new frame performed by StearmerletIs(), so the transmission behavior (rate) of the data streamer can be adjusted to improve the overall transmission performance.

Finally, the following thread will be created only when a client who demands that the RTP packets are to be transmitted over TCP is connecting to the server. For each new RTSP session, the system will start a new one.

z TunnelMain()

- Responsible for injecting media data frames into network via TCP sockets. It is the data streamer for transmission over TCP.