An efficient multipath routing for distributed computing systems with data replication

An ecient multipath routing for distributed

computing systems with data replication

D.J. Chen

_{, P.Y. Chang}

Department of Computer Science and Information Engineering, National Chiao Tung University, Hsin Chu 30050, Taiwan, ROC

Received 3 February 1999; received in revised form 4 May 1999; accepted 2 August 1999

Abstract

In distributed computing environments, executing a program often requires the ac-cess of remote data ®les. An ecient data routing scheme is thus important for time-critical applications. To ensure a prior desired communication quality, we present a connection-oriented routing scheme, the multipath routing, which allows multiple routes to be established between the source and the destination. Based on the multipath routing scheme, the problem of ®nding a collection of routing paths for an application to minimize its data transmission time is addressed. Such a problem becomes a complex combinatorial one when the application accesses multiple replicated data sources. Since ®nding an optimal solution is computationally infeasible in practice, we thus propose a heuristic method to get a sub-optimal solution. Ó 1999 Elsevier Science Inc. All rights reserved.

Keywords: Virtual circuit; Multipath routing; Flow; Cut

1. Introduction

This paper presents a routing algorithm for time critical applications which need to retrieve remote high-bandwidth data ®le(s) in a LAN-based distributed computing system (DCS). A connection-oriented data forwarding technique,

www.elsevier.com/locate/ins

*_{Corresponding author. Fax: +886-03-572-4176.}

E-mail address: [email protected] (D.J. Chen)

0020-0255/99/$ - see front matter Ó 1999 Elsevier Science Inc. All rights reserved. PII: S 0 0 2 0 - 0 2 5 5 ( 9 9 ) 0 0 0 5 6 - 0

virtual circuit [4,6,7], which enables bandwidth to be reserved through the lifetime of a connection, is used in the network. Since the inherent link delay (including propagation delay and transmission delay) is insigni®cant for bulk data transfer in a high-speed LAN, the design objective of the routing algo-rithm is to maximize data transfer rate (or throughput). For a source-to-des-tination trac session, throughput can frequently be improved by splitting the trac over several paths. The technique of using multiple paths between a source±destination pair is called multipath routing.

Based on the multipath routing scheme, the problem of optimal routing can be described as a multicommodity ¯ow problem if the number of paths in a source±destination session is unlimited and the inherent link delay and control overhead are negligible. Linear programming is hence a feasible technique to solve it. However, since in a DCS data ®les are often replicated to improve system (or program) reliability, the program may acquire data from any replica [1,3]. This problem thus becomes a complex combinatorial one. An exhaustive approach could of course ®nd the optimal solution, but it pays for high computation price. Our previous work has developed an alternate, the critical-cut algorithm, to solve the optimal routing problem [2]. For most experiment cases, the algorithm yields short execution times. Occasionally, it still takes exponential time. This paper thus proposes a heuristic routing algorithm to obtain acceptable routing paths. Short and stable execution time makes the proposed algorithm be applicable to the existent systems.

Assumptions: We assume that a distributed computing system can be de-scribed as an undirected graph with nodes representing computing sites and edges representing communication links. We assume virtual circuits for com-munication between a source and a destination, and multipath routing is supported in the virtual circuit schemes. Intermediate nodes do not bu€er packets, but simply send all received packets immediately. Inherent link delay is negligible. A given ®le cannot be split and distributed to multiple nodes. A communication path cannot have loops.

2. Critical-cut algorithm

This section presents the critical-cut algorithm to ®nd the optimal routing. Proofs of theorems behind the algorithm can be referred in [2].

Notations

source the node which holds data ®les

target the node which issues request for data ®les

cut set of edges such that the removal of the edges separates a connected graph into two disconnected subgraphs

2.1. Cut-trees

The cut-tree theorem was proposed in order to e€ectively ®nd the maximum ¯ow values between all pairs of nodes in a network [5]. For a network with n nodes, to ®nd the maximum ¯ow values between all pairs of nodes, one must compute n  …n ÿ 1†=2 such maximum ¯ow values. If the network topology is a tree, the maximum ¯ow values between any two nodes can be easily determined by ®nding the minimum edge capacity in the unique path connecting the two nodes. Two networks are said to be ¯ow-equivalent if the maximum ¯ow values between all pairs of nodes are the same. The construction of a ¯ow-equivalent tree from a general graph G is presented in [5]. Therein, a tree that is ¯ow-equivalent to G is called the cut-tree of G because each link of the tree repre-sents a minimum cut of G. A cut-tree of G with n nodes shows the n ÿ 1 minimum cuts of G that do not cross each other. Consider the example in Fig. 1. Fig. 1(a) illustrates an original network G. The numbers associated with the links denote the link capacities. Fig. 1(b) depicts the cut-tree of G generated by the cut-tree algorithm. Fig. 1(c) illustrates the cuts corresponding to the links in the cut-tree. These cuts are referred to herein as partitioned cuts since they do not cross each other.

2.2. Critical cut

When data are transferred with maximum rate, the cuts which are saturated are referred to as the critical cuts. Critical cuts can be found by enumerating the

…X ; X † the cut separating the nodes in set X from the other nodes (i.e., the nodes in X ). Note that in presenting a cut by the notation …X ; X †, target is always placed in X throughout this paper

cap …X ; X † capacity of the cut …X ; X †, i.e. sum of the capacities of the edges in set …X ; X †

LV the number of bytes from the data ®les in V, where V

denotes a set of nodes

Fig. 1. The relationship between a graph and a tree. (a) The original network G. (b) The cut-tree of G. (c) The partitioned cuts.

cuts that separate sources from the target. Consider a DCS in Fig. 2, where a program on node N1needs to retrieve data ®les F1, F2, and F3. Assume that F1

on N3, F2on N4, and F3on N2are selected to be transmitted. All the possible

cuts that separate sources from the target are shown as the dotted lines in Fig. 2. The corresponding parameters in determining the critical cut are listed in Table 1. The term L_X=cap…X ; X † denotes the minimum time required to transmit data through the cut …X ; X †. The overall data transfer time is thus bound by cut c2

(critical cut). Data are transferred through cut c2 requires 1.13 s.

In general, enumerating all the cuts as described above to ®nd the critical cut is prohibitive for a large network. Therefore, the number of cuts to be explored must be reduced. The following section describes how to achieve such a re-duction.

2.3. Cut reduction

Some terminology needs to be de®ned before presenting the said cut re-duction.

De®nition 1. Let ciˆ …X ; X † and cjˆ …Y ; Y † be two non-crossed cuts. If X  Y

(or equivalently, X  Y ), ciis said to be an ancestor of cj, and cjis a descendant

of ci. If X \ Y ˆ ;, ciand cj are said to be brothers of each other.

The relationship between any pair of partitioned cuts can be easily identi®ed by the corresponding cut-tree. Let ei and ej be the edges in the cut-tree

re-garding the partitioned cuts ci and cj. By letting the target be the root of the

cut-tree, if there is a path from root to the leaf containing eiand ej, then ciand

cjhave an ancestor/descendent relationship. Notably, cut ciis the ancestor of cj

if ei is closer to the root than ejis in this path. If no such path can be found,

then ci and cj have a brother relationship.

De®nition 2. A partitioned cut c is called the primary cut if c has no ancestor cut or the capacity of c is smaller than the capacity of any ancestor cut of c (except c itself).

Ta ble 1 C uts and the correspo nding param eters for the exa mple in Fig .2 C ut X X LX cap …X ;X † LX =cap …X ;X † c1 {N 1 } {N 2 ,N3 ,N4 } jF1 j‡ jF2 j‡ jF3 jˆ 1500 k 500 k+1 M ˆ 1500 k 1 c2 {N 1 ,N3 } {N 2 ,N4 } jF2 j‡ jF3 jˆ 1300 k 500 k+250 k+400 k ˆ 1150 k 1.13 c3 {N 1 ,N2 ,N3 } {N 4 } jF2 jˆ 500 k 450 k+400 k ˆ 850 k 0.59 c4 {N 1 ,N3 ,N4 } {N 2 } jF3 jˆ 800 k 500 k+250 k+450 k ˆ 1200 k 0.67 c5 {N 1 ,N2 ,N4 } {N 3 } jF1 jˆ 200 k 1 M +250 k+400 k ˆ 1650 k 0.12 c6 {N 1 ,N2 } {M 3 ,N4 } jF1 j‡ jF2 jˆ 700 k 1 M +250 k+450 k ˆ 1700 k 0.41

De®nition 3. Let ci …X ; X † and cj …Y ; Y † be two non-crossed cuts. The

exclusive-or operation, denoted by Å, is de®ned as follows. If ciand cjhave a

brother relationship, then ci cj …X \ Y ; X [ Y †; if ci is an ancestor of cj,

then ci cj …X [ Y ; X \ Y †.

Theorem 1. The critical cut can be factored into primary cuts based on exclusive-or operation and none of these primary cuts is an ancestexclusive-or (exclusive-or descendant) of the others.

The result of Theorem 1 suggests that we may apply exclusive-or operations on primary cuts in order to generated a set of cuts, say S, which contains the critical cut. Procedure Find-Possible-Critical-Cuts realizes the process to gen-erate S.

Procedure (Find-Possible-Critical-Cuts)

// S is the set of possible critical cuts, Initially S is empty. Finally, S holds the possible critical cuts.

// Q is a queue of cuts. Initially Q is empty.

Construct the cut-tree of the original network. In building the cut-tree, only the sources and the target are considered.

Decide primary cuts

Add all primary cuts to set S Enqueue all primary cuts to Q REPEAT

c ˆ dequeue(Q)

FOR each of the primary cuts, cP

Add and enqueue c  cpto S and Q respectively unless

cond.1: c contains a factor cut which is an ancestor, or a descendent, of cP.

cond.2: c \ cpˆ ; // i.e. cuts c and cP have no common edges when they

are presented by sets of edges. END_FOR

UNTIL Q is empty END_PROCEDURE 2.4. Selection of ®le±servers

The routing optimization problem becomes a complicated combinatorial problem if multiple replicated copies of data ®les are dispersed over the net-work. The following theorem averts a situation which all of the combinations are exhausted to obtain the optimal solution.

Theorem 2. Let M1 and M2 be two possible file±server combinations. M1 differs from M2 only in selecting the copy of file F. M1 selects a copy of F from node i; M2 selects a copy of F from node j. If node i is the ancestor node of node j in cut-tree, then TMINM16 TMINM2 where TMINM1 and TMINM2 denote the data

transfer times via the optimal routing paths for M1 and M2, respectively. An ecient method for obtaining the optimal solution is described by the following Find-Minimum-Transmission-Time algorithm. In this algorithm, the ®le±server combination with the smallest ``weight'' to the root (i.e. the target) is processed ®rst. This strategy stems from Theorem 2, suggesting that the copy nearest to the root in the cut-tree tends to have the shortest transmission time. The weight of a ®le-source mapping is de®ned as follows.

De®nition 4. The weight of a ®le±server combination M is de®ned as X

Niselected by M

…d…Ni†

‰  n…Ni†Š;

where d(Ni) is the number of hops (links traversed) from node Nito the root in

the cut-tree, and n(Ni) the number of the required data ®les in node Nifor M.

Eciency of this algorithm follows from the fact that many cuts and ®le± server combinations can be excluded from consideration. Outputs of the al-gorithm include the best ®le±server combination, the critical cut, and the minimum data transmission time. Optimal trac routing can be easily ob-tained by executing an additional maximum ¯ow computation. A new graph can be constructed from the original network by adding a super node S _and

connecting S _{to the source nodes with proper edge capacity assignment.}

Applying the maximum ¯ow algorithm on this new graph, allowS us to obtain the optimal trac routing.

Algorithm (Find-Minimum-Transmission-Time) // MOP the optimal valid ®le±server combination.

// TMINOP the minimum transmission time for MOP.

// CCOP critical cut for MOP.

// VinXbarMthe number of bytes of the requested data in X for the valid ®le±

server combination M, where …X ; X †is a critical cut found for M. Execute Procedure Find-Possible-Critical-Cuts.

Create the set of valid ®le±server combinations, say U. TMINOP ˆ 1; CCOP ˆ ;; MOP ˆ ;

REPEAT

Select a valid ®le±server combination with the minimum weight, say Mi, from U

FOR each valid ®le±server combination Mj in U DO if VinXbarMjP VinXbarMi then remove Mj from U

END_FOR

IF TMINMi< TMINOP THEN

TMINOP ˆ TMINMi; CCOP ˆ …X ; X †; MOP ˆ Mi

END_IF UNTIL U ˆ ;

Calculate routing paths for MOP

END_ALGORITHM 2.5. An illustrative example

Consider a DCS in Fig. 2, where a program running on node needs to re-trieve data ®les F1, F2 and F3. Fig. 3 illustrates the cut-tree and the

corre-sponding non-crossed cuts. In this network, there are two copies of F1in nodes

N2 and N3. According to Theorem 2, the copy of F1 in N3 is selected to be

transmitted since N3is the ancestor of N2. Similarly, there are two copies of F3

in nodes N2 and N4 and the copy in N2 is selected to be transmitted. We thus

obtain an unique ®le±server combination: selecting F1 from N3, F2 from N4,

and F3from N2. Since all the partitioned cuts (c1, c2, and c3) are primary, the

procedure Find-Possible-Critical-Cuts begins by adding cuts c1, c2, and c3to S,

the set of possible critical cuts. Table 2 lists the details of this process to generate other possible critical cuts. Since cuts c1, c2, and c3 have ancestor/

descendent relationship, no more new possible critical cuts can be generated. The minimum data transfer time can be obtained by

TMINOP ˆ MAX jF1j ‡ jF_Cap…c2j ‡ jF3j

1† ; jF2j ‡ jF3j Cap…c2† ; jF2j Cap…c3†   ˆ MAX 1500₁₅₀₀;1300₁₁₅₀;500₈₅₀   ˆ1300₁₁₅₀ˆ 1:13 and cut c2 is the critical cut.

Fig. 3. The tree and the corresponding partitioned cuts for the example in Fig. 2. (a) The cut-tree. (b) Partitioned cuts.

Trac routing of this optimal transmission can be obtained by executing the maximum ¯ow algorithm on the graph in Fig. 4. The graph in Fig. 4(a) is constructed by adding a supernode S_{and the pseudoedges between S}_{and the}

source nodes. The capacities of edges between S_{and nodes N}

2, N3, and N4are

given as 800 k=1:13 ˆ 707, 200 k=1:13 ˆ 176, and 500 k=1:13 ˆ 442 k, respec-tively. Fig. 4(b) presents the solution of the maximum ¯ow from node S _to

node N1, where the supernode and pseudoedges have been removed.

Multi-plying the ¯ow values on the edges by TMINOP allows us to obtain the volume

of data transmitted via the edges. 3. Heuristic routing algorithm

The performance of critical-cut method depends on the cut tree and the distribution of ®les. For the case such as the example in Section 2.5, the cut tree is a linear array and the algorithm therefore yields very short execution time. However, for some other cases, critical cut method may take exponential time to obtain the optimal routing. In practical environments, a routing algorithm with short and stable execution time is important in real-time applications. We thus propose a heuristic method to decide the routing paths. The main concept of the proposed algorithm is quite simple. Each ®le is assigned a proper portion

Table 2

The process to generate S, the set of possible critical cuts

Cut Enter S? Remark

c1 Yes Primary cut

c2 Yes Primary cut

c3 Yes Primary cut

c1Å c2 No c1and c2has ancestor/descendant

c1Å c3 No c1and c3satisfy ancestor/descendant

c2Å c3 No c2and c3satisfy ancestor/descendant

Fig. 4. The way to obtain the trac ¯ow along each edge. (In (b), values in parenthesis denote the

data volume shipped via the edges.) (a) Graph with supernode S_{and pseudoedges. (b) The ¯ow}

of the total available communication capacity for its transfer. The reserved communication capacity for a data ®le is proportional to the size of the ®le. The routing strategy is that all data ®les are routed simultaneously, and, ide-ally, will complete their transfers at the same time. This strategy stems from the optimal routing. Compared with the intuitive approach in which data ®les are routed one after the other, such a simultaneous ®le transfer can result in higher link utilization and can have shorter transmission time.

However, the total available communication capacity (i.e. the capacity of the critical cut) is not easily determined. The algorithm thus uses the available capacity around the target instead to calculate the appropriate communication capacity for each ®le's transfer. For the DCS in Fig. 2, the routing algorithm will initially assign the capacities of 200 …ˆ 1500  200=1500† kbytes/s for F1Õs

transfer, 500 …ˆ 1500  500=1500† kbytes/s for F2Õs transfer, and 800

…ˆ 1500  800=1500† kbytes/s for F3Õs transfer. The algorithm then operates in

a breadth-®rst-search (BFS) order, with the target as the root of the graph, to visit the nodes and try to ®nd the paths with the assigned communication capacities to transmit the data ®les (if any). Unfortunately, it is typically not the case that all of the ®les can be transmitted with the initially assigned ca-pacities. Some ®les that are located on nodes far from the target may fail to get their deserved communication capacities. These ®les will then be transmitted at a lower speed or be pending until the transmission of some other ®le is ®nished and the occupied channels are released. The released capacity will then be al-located to transmit those ®les that have not been completely transferred. This process repeats until all the data ®les are entirely delivered. The algorithm is shown below.

Algorithm (Heuristic multipath routing)

// FS set of data ®les. Initially, all data ®les are in FS. // FS1 set of data ®les. Initially, FS1 is empty.

// NSd set of nodes with distance d, where ``distance'' is the minimum

num-ber of links between the node and the target. // Maxdist the longest distance in the network.

// Cap…Ni; Nk† available capacity of the link connecting nodes Ni and Nk.

Complete ˆ FALSE WHILE NOT Complete

Compute the deserved capacity for each data ®le in FS. FOR d ˆ 1 to Maxdist DO

WHILE NSd6ˆ ; DO

Select a node Ni which have the largest value ofPNK2NSdÿ1Cap…Ni; Nk†,

from set NSd.

WHILE node Niis in NSd DO

IF there are no ®les in FS can be found in Nior there are no paths

Remove node Nifrom set NSd.

ELSE

Transmit ®le Fj(where Fjis in FS and a copy of Fjis found in node Ni)

via established virtual circuit(s) with the capacity as computed for FjÕs

transfer, or with the current maximum communication capacity if FjÕs

de-served communication capacity is not satis®ed (in such case, add Fjto set

FS1).

Remove ®le Fj from set FS.

END_IF END_WHILE END_WHILE END_FOR

IF ®le FjÕs transmission-completed signal is received THEN

Release the communication channels allocated to Fj.

FS ˆ FS [ FS1 ÿ fFjg; FS1 ˆ ;

IF FS ˆ ; THEN Complete ˆ TRUE ELSE

Send signals to pause the transmission of ®les in FS and release the allo-cated channels.

Adjust the sizes of ®les still remain in FS to be the lengths of untransmit-ted portions.

END_IF END_IF END_WHILE END_ALGORITHM

As an example, consider the DCS in Fig. 2, where FS ˆ fF1; F2; F3g;

NS0ˆ fN1g; NS1ˆ fN2; N3g, and NS2ˆ fN4g (see Fig. 5(a)). According to ®le

sizes, the reserved capacities for F1, F2, and F3are 200, 500, and 800 kbytes/s,

respectively. The ®rst ®le transfer will start at a node selected from NS1 (if a

copy of any required data ®les can be found in that node). In this example, N3

is selected since P_NK2NS0Cap…N3; Nk† ˆ Cap…N3; N1† ˆ 1 Mbytes/s, which is

greater than P_NK2NS0Cap…N2; Nk† ˆ Cap…N2; N1† ˆ 500 kbytes/s. F1 in N3 is

then transmitted at the rate of 200 kbytes/s along path N3±N1, and the

esti-mated F1Õs transfer time is 200 k=200 k ˆ 1 s. FS becomes {F2, F3}. Since there

is not any other ®les in FS can be found in N3, N3 is removed from NS1 (see

Fig. 5(b)).

Node N2 is the next one to be explored. File F3 in N2 will be transmitted.

The virtual circuits with 800 kbytes/s (for example, 500 kbytes/s along path N2±N1, 250 kbytes/s along path N2±N3±N1, and 50 kbytes/s along path N2±

N4±N3±N1) are allocated for F3Õs transfer. The estimated transfer time is

can be found in N2, N2 is removed from NS1 and thus NS1 becomes empty

(see Fig. 5(c)).

The algorithm will next explore the nodes in set NS2. Since N4is the only

node in NS2, F2 in N4will be transmitted. The algorithm tries to ®nd path(s)

with the overall capacity of 500 kbytes/s to transmit F2. However, only a path

(N4±N3±N1) with 350 kbytes/s can be found. F2is thus transmitted via such a

path. The algorithm then adds F2to set FS1, and removes F2from FS. Fig. 5(d)

shows the remaining available capacities after allocating capacities for F2Õs

transfer.

Fig. 5. The statuses for the example in Section 4 by employing Heuristic multipath routing

algo-rithm. (a) The initial status. (b) After establishing path N3±N1(200 kbytes/s) for F1Õs transfer. (c)

After establishing paths N2±N1 (500 kbytes/s), N2±N3±N1 (250 kbytes/s), and N2±N4±N3±N1(50

After F1 and F3 have completed their ®le transfers (i.e. 1 s later since their

transmissions begin), Set FS is updated as FS [ FS1 ÿ fF1; F3g ˆ fF2g and, in

addition, size of F2is adjusted to be 150 …ˆ 500 k ÿ 350 k† kbytes. The

com-munication channels allocated to F1, F2, and F3 are released. Sets NS0, NS1,

and NS2 are reinitialized and, by assuming that the current available link

pacities are the same as shown in Fig. 5(a), the reserved communication ca-pacity for F2is recomputed as 1500 kbytes/s.

Again, nodes are visited in the sequence given by the algorithm, and ®nally the untransmitted part of F2 in N4will be transmitted. Since the path(s) with

the overall capacity of 1500 kbytes/s are not found, The remaining 150 kbytes of F2is transmitted by the paths, N4±N2±N1(450 kbytes/s) and N4±N3±N1(400

kbytes/s). The estimated data transfer time for the 150 kbytes is 150 k/(400 k + 450 k) ˆ 0.18 s. When F2is completely transferred, FS becomes empty and

the routing process stops. Thus the total data transfer time, without consid-ering the control signal processing time and the algorithm execution time, is 1 + 0.18 ˆ 1.18 s.

4. Simulation results

To evaluate the performance of the proposed heuristic routing algorithm, we compared the routes found by the heuristic algorithm with the optimal routing paths. The optimal routing paths were obtained by the critical-cut method as described in Section 2. Many factors such as the network topol-ogy, available link capacities, the number of replicated copies of each data ®le, and the dispersal of the program and the data ®les in the sense will in¯uence the performance. In order to fairly evaluate the proposed algorithm, the simulation programs were tested on various scales of randomly generated graphs, with an average node degree in the range of 2.5±3.0. Each linkÕs available capacity and the size of each data ®le were randomly generated as well.

We use the ratio of TOP to THEU to measure the eciency of the routing

paths found by the heuristic routing algorithm. Fig. 6(a) shows that all TOP/

THEU values fall into the range between 0.70 and 0.89, no matter what scale of

the network is or how many copies each data ®le has. The relationship of the number of the replicas versus the algorithm execution time is shown in Fig. 6(b)±(d). Obviously, the execution time of the critical-cut algorithm increased with the growth of the number of replicas. On the other hand, the algorithm execution time of the heuristic approach was not a€ected by such factor. Fig. 6(e) depicts that both algorithms were network size dependent. That is, if the network extends to some scale, both of the routing algorithms are no longer applicable.

5. Conclusion

In this paper, we propose a heuristic multipath routing algorithm on virtual circuit based DCS. Such a routing scheme is appropriate for high-bandwidth demanded data transfers issued by real-time applications. To evaluate how well the algorithm performs, we made experiments on various networks to monitor the algorithm execution time and the communication capacities of the found routing paths. These observed experimental data are compared with the results obtained by the critical-cut algorithm, which is an ecient optimal routing algorithm proposed in our previous work. Experimental results show that the paths found by the heuristic routing algorithm carry about 70±90% data ¯ow of the optimal routing.

Fig. 6. The comparison of execution time between the heuristic routing algorithm and the

critical-cut algorithm. (a) # of replicas vs. TOP/THEU. (b) # of replicas vs. execution time for 10-nodes

graph. (c) # of replicas vs. execution time for 50-nodes graph. (d) # of replicas vs. execution time for 100-nodes graph. (e) # of node vs. execution time (without ®le replication).

The proposed routing algorithm is a centralized routing. One common problem of the centralized routing is that the time for routing calculation will increase with the network size. Fortunately, this problem may be relieved as more and more powerful computers are developed. We learn from the exper-iments that the current personal computers execute the algorithm to calculate the routes (for the 100-nodes network) in 0.2 s. For bulk data transfer, such program execution time is acceptable if data transfer time takes seconds.

References

[1] P.A. Bernstein, V. Hadzilacos, N. Goodman, Concurrency Control and Recovery in Database Systems, Addison-Wesley, Reading, MA, 1987.

[2] P.Y. Chang, D.J. Chen, Optimal routing for distributed computing systems with data replication, in: Proceedings of IEEE International Computer Performance and Dependability Symposium, Urbana-Champaign, USA, 1996.

[3] A. El Abbadi, S. Toueg, Maintaining availability in partitioned replicated databases, ACM Transaction on Database Systems 14 (2) (1989) 264±290.

[4] D. Ferrari, A. Banerjea, H. Zhang, Network support for multimedia: A discussion of the telnet approach, Comput. Networks and ISDN System (1994) 1267±1280.

[5] T.C. Hu, Combinatorial Algorithms, Addison-Wesley, Menlo Park, CA, 1986.

[6] B. Jabbari, A bandwidth allocation technique for high speed networks, in: Proceedings of IEEE GLOBECOMÕ90, vol. 1, San Diego, CA, 1990, pp. 355±359.

[7] A. Lazar, C. Paci®ci, Control of resources in broadband networks with quality of service guarantees, IEEE Commun. Mag. (1991) 66±73.