In Chapter 3 we outline the token-based load balancing architecture based on which different load-balancing schemes can be conceived. In this chapter, we proceed to describe one specific load-balancing algorithm, namely TBMM (Token-Based Market Mechanism), that dynamically rearrange tokens among servers to improve the overall system utilization.
Distributed load balancing methods can be characterized with four policies [Shivaratri, et al][Eager, et al], namely information policy, transfer policy, location policy, and selection policy. We will also use the four policies to describe our load balancing method. Before going into the details, however, we first summarize these four policies:
Information policy. The information policy determines the kinds of state information to be exchanged among servers. Conventional methods usually exchange information about a single resource, such as the CPU load.
Transfer policy. The transfer policy determines the set of servers that need to adjust.
The most common approach is threshold-based, which calculates an upper threshold Ts
and a lower threshold Tr. The upper threshold Ts may be in the form of Lavgavg+ d, where
avg
Lavg stands for the average resource load of the whole server cluster and d a designated constant, or aLavgavg where a is a constant value greater than 1. Likewise, Tr may be in the form of Lavgavg- d or aLavgavgwhere a < 1.
As shown in Figure 4.1, if the load of a server is greater than Ts, the server is said to be in sender state; if less than Tr, the server is in receiver state. Otherwise, the server is in common state. As the names suggest, a server in sender state tends to send some of its
jobs to another server with a lower load, while a server in receiver state tends to receive jobs from another server with higher load. The servers in common state do not have to do anything. In this paper, we also define a new state, call exchanger state, to deal with multi-resource load imbalance issue within each server.
Location Policy. The location policy concerns the steps needed to find the target server to which a server in sender state can send jobs to, and to find the source server from which a server in receiver state can receive jobs from. A simple heuristics is for a server in sender state to match the server with lowest load, and similarly for a server in receiver state to match the one with highest load.
Selection Policy. Once the server pair is chosen, the selection policy determines which jobs should be sent to or received from the matching server. We investigate three popular job scheduling approaches, called latest arrival job, backfill lowest [Leinberger, et al]
and backfill balance [Leinberger, et al]:
- Latest Arrival Job (LAJ): send the latest arriving job from the sending server to the receiving server,
- Backfill Lowest (BL): find the resource of the receiving server that is most available, and send the job that demands that resource most from the sending server.
- Backfill Balance (BB): send the job which can minimize the (maximum load / average load) measure for the receiving server.
avg Tr Ts
Receiver Common Sender
Figure 4.1: Sender, receiver, and common states
As an example, consider Figure 4.2 in which there are two servers with 3 jobs and 1 job, respectively, and the load of their resources are 13, 15, 14 and 7, 5, 6, respectively, before job exchange (Figure 4.2a). Assume the server on the left is in sender state and has to send a job to the server on the right. If the LAJ selection policy is used, the latest arriving job (with resource requirements 5, 3, and 5) will be selected (Figure 4.2b). In case the BL selection policy is used, since the second resource of the receiving server is least busy, the first job (with resource requirements 5, 7, and 5) of the sending server will be chosen because it demands the second resource most (Figure 4.2c). Finally, when the BB selection policy is used, since the measure of executing the three jobs on the receiving server are 1.16, 1.0, and 1.03, respectively, the second job will be selected (Figure 4.2d).
3
8 12 9 12 8 11 (b) Result of LAJ selection policy
7
Figure 4.2: Example of selection policies
The framework discussed above is for conventional job scheduling problem. In this thesis, only small modification is needed, that is, by replacing the concept of jobs with tokens.
Note that in conventional job schedule problem, jobs enter the system continuously, and leave the system when completed. In our model, however, tokens persist in the system since beginning, even though their number may increase or decrease over time. However, such difference has no impact from load balancing perspective, since we can simply think of tokens as long-running jobs that seldom finish.
In TBMM, each token assigned to a server will be associated with a cost, which is directly proportional to the resource load of that server. We define the cost per resource requirement of token w assigned to server j as:
and K the number of resource types. The reason we define the cost per requirement rather than the total cost of a token assigned to a server is that the former helps improve the load imbalance degree within a server.Below we describe our method in terms of the four policies mentioned above:
Information policy. In our method, servers exchange their entire resource load L ~1j LKj with
each other, where L represents the load of resource k in server j. kj Transfer policy. We define the load imbalance degree for server j as:
imbalance degree between servers. Similarly, the absolute load imbalance degree for server jis defined as the sum of absolute values of load differences: determine the state of a server as follows:
Sender: Ts Bj.
Receiver: Bj Tr.
Exchanger: Tr Bj Ts and T Babs_j.
Common: the rest.
Ts and Tr are thresholds that are also commonly used in conventional methods. The difference is that we add an exchanger state with the threshold T for servers to further improve load balancing of resources inside of a server.
Location policy. For servers in sender or receiver state, the location policy to find matching
server pairs is the same as those in conventional methods. For servers in exchanger state, on the other hand, the policy is different:
Assume server i and j are both in exchanger state, we define the load imbalance degree for the pair before token exchange as:
j abs i
abs
B
B
_
_ (4)and the ideal load imbalance degree after token exchange as:
Our goal is to find the pair of servers i, j such that their imbalance degree before token exchange minus their ideal load imbalance degree afterwards is the largest. For illustration, consider Figure 4.3a in which there are three servers; the ratio of their capacities is 1:3:4 and their current loads compared to the whole system load are +4%, -4%, and +3% respectively.
Before token exchange, Babs_1, Babs_2, and Babs_3 are 4, 4, and 3, respectively. Without considering server capacity, we may tend to pair server 1 with server 2, because the imbalance degree before token exchange is 4 + 4 = 8, and
4.3b), hence the improvement of imbalance degree is 8 – 4 = 4. If we pair server 2 and server 3 instead, the imbalance degree is 4 + 3 = 7 before token exchange, and becomes
(b) Result of pairing server 1 and server 2
+4% 0% 0%
+4% 0% 0%
S1 S2 S3
(c) Result of pairing server 2 and server 3
Figure 4.3: Example of location policy
Selection policy. Employ Equation (1). For each server pair i, j selected by the location policy
among the servers in sender and receiver states, the token, w, to be sent from server i to j is the one that can result in minimum Cj w for server j.
For each server pair i, j where both servers are in exchanger state, we want to pick a token from each server for exchange, and the exchange can benefit both servers overall. We define the benefit of transferring a token w from server i to server j as:
w C
wCi j (6)
Accordingly, the total benefit of exchanging token w1 in server i with token w2 in server j becomes
Ci (w1)Cj(w1)
Cj(w2)Ci(w2)
(7)Furthermore, the benefit should be larger than a threshold T for the exchange to take place.
Without the threshold T, it is possible that two servers in exchanger state may constantly exchange the same pair of token to and from each other without converging.