S ERVER P ROGRAMMING FOR I NTER -U SER C OMMUNICATION

With the rapid growth of the Internet, applications involving real-time communication among clients have become increasingly important. Consider an example of chat room or game system. One user types a message and others then can read that message in real time.

Since these applications involve inter-user communication, this dissertation calls them inter-user communication applications.

For inter-user communication applications, servers are often used to handle inter-user communication. For example, game servers receive player events (or messages) and then respond (or pass messages) to other players. For inter-user communication applications, server developers generally must consider the following criteria.

1. Minimize the client response time. If the response time is unexpectedly long, interactions may not evolve as expected or users may run out of patience.

2. Ensure high server stability. Server crashes cause all clients connected to that server to become disconnected.

3. Support as many clients concurrently as possible. For example, support thousands of players on a single server.

The first criterion is essential for server programming in inter-user communication applications. To respond to users as rapidly as possible, servers usually hold connections to clients. Servers thus must handle client messages (or events) from all connections concurrently and server developers must handle concurrent events carefully.

Two main programming models exist for concurrent event handling, namely threading and event-driven programming. Threading is a general-purpose technique for managing concurrency. The advantages of threading compared to event-driven programming include: (a) support of context switching among threads, and (b) support of scalable performance on multiple CPUs.

However, some developers and researchers [57, 66] also have observed that threading has some drawbacks compared to event-driven programming. Note that Ousterhout [57]

described the following drawbacks:

1. Difficult to program. Threads generally require synchronous mechanisms (e.g., locks) to access shared data safely. However, incorrect locking may cause deadlocks, making independent module design difficult. Besides, another problem that also increases programming difficulty is that several standard libraries are not thread-safe [51].

2. Hard to debug. For threading, it is difficult for developers to debug the code due to data and timing dependencies. Besides, another problem that also increases debugging difficulty is that thread stack sizes are normally limited [46, 51], causing processes crash when stacks overflow. In contrast, in event-driven programming, the lack of context switching among event handlers makes it quite easy to debug the code by recording and then replaying the sequence of events.

3. Difficult to achieve good performance. Coarse-grain locking yields low concurrency, while fine-grain locking tends to increase lock operations and thus reduce performance.

Since inter-user communication applications are often used to facilitate heavy inter-user communication among numerous clients (say, thousands of players in a game system), it makes the above drawbacks even worse. The first two drawbacks imply that it is hard for threading to satisfy the second criterion (above) of the inter-user communication applications;

and the third drawback indicates that it is hard for threading to satisfy the third criterion. Thus, for the application developers who are more concerned with the second and third criteria and less concerned with the two threading advantages (described above), the event-driven programming model becomes attractive. Hence, this dissertation is motivated to study and design an event-driven framework for inter-user communication applications.

Our framework is based on event-driven programming (rather than threading) for the following reason. In inter-user communication applications, the above three drawbacks of threading (or the second and third criteria) are important as described above, while the two drawbacks of event-driven programming are less important because they can be ignored or alternatively can be solved in this dissertation. First, regarding the two drawbacks of event-driven programming, this dissertation ignores the one, namely not supporting scalable performance on multiple CPUs, because for most inter-user communication applications servers can be separated into several processes to achieve scalable performance. In the case of tabletop games, such as Chess and Bridge, servers can naturally be separated into several processes, e.g., one for each game. Even for most massive multiplayer online games (MMOGs), such as Ultima Online [22], the server system can use several processes each dealing with a single game scene. Second, this dissertation focuses on overcoming the other drawback of event-driven programming: the need to pay attention to the blocking problems due to the use of blocking I/O operations in event handling.

This dissertation addresses two major blocking problems and presents solutions or guidelines. The two blocking problems are described below.

1. Output blocking: This problem occurs on sending messages to clients with corresponding full kernel buffers. The buffer generally becomes full when network traffic is jammed.

This problem frequently is neglected at the start of server development.

2. Request blocking: This problem occurs when a server waits for responses after sending requests to other servers. For example, when a game server attempts to read several game records from a remote database server.

This chapter presents solutions for the above two blocking problems. An output buffering mechanism is presented to solve the output blocking problem, while a service brokering mechanism is presented to solve the request blocking problem. Meanwhile, for the second problem, several system and library calls that may cause the problem are also identified. Both mechanisms are incorporated into the event-driven framework presented in this chapter.

The rest of this chapter is organized as follows. Section 4.2 reviews the event-driven programming model. Section 4.3 describes the output blocking problem and presents solutions. Section 4.4 then describes the request blocking problem and presents solutions.

Section 4.5 presents performance analysis.