The reliability analysis of distributed computing systems with imperfect nodes

全文

(1)The Reliability Analysis of Distributed Computing Systems with Imperfect Nodes M IN -S HENG L IN1 , D ENG -J YI C HEN2. AND. M AW-S HENG H ORNG3. 1 Department of Information Management, Tamsui Oxford University College, Tamsui, Taipei,. Taiwan 25103, Republic of China 2 Institute of Computer Science and Information Engineering, National Chiao-Tung University,. Hsin Chu, Taiwan 30050, Republic of China 3 Department of Mathematics Education, National Taipei Teacher College, Taipei, Taiwan 10659,. Republic of China Email: [email protected] The reliability of a distributed computing system depends on the reliability of its communication links and nodes and on the distribution of its resources, such as programs and data files. Many algorithms have been proposed for computing the reliability of distributed computing systems, but they have been applied mostly to distributed computing systems with perfect nodes. However, in real problems, nodes as well as links may fail. This paper proposes two new algorithms for computing the reliability of a distributed computing system with imperfect nodes. Algorithm I is based on a symbolic approach that includes two passes of computation. Algorithm II employs a general factoring technique on both nodes and edges. Comparisons with existing methods show the usefulness of the proposed algorithms for computing the reliability of large distributed computing systems. Received July 12, 1994; revised December 3, 1998. 1. INTRODUCTION A typical distributed computing system (DCS) consists of processing elements (PEs), memory units, data files and programs. These resources are interconnected through a communication network that dictates how information flows between PEs. Programs residing on some PEs can run using data files stored in other PEs. For successful execution of a program, it is essential that communication links between the PE containing the program and other PEs that have the required data files be operational. Distributed program reliability (DPR) is defined as the probability that a distributed program that runs on multiple PEs and needs to communicate with other PEs for remote files will be executed successfully. For example, in the DCS in Figure 1, there are six PEs (n 1 , n 2 , n 3 , n 4 , n 5 , n 6 ) and eight communication links (e1 , e2 , e3 , e4 , e5 , e6 , e7 , e8 ). Program P1 requires data files f1, f2 and f3 to complete its execution, and it is running at node n 2 , which holds the file f1. Hence, program P1 must access the remote files f2 and f3. Since file f2 is resident at node n 3 and file f3 is resident at nodes n 4 and n 5 , the DPR of program P1 can be formulated as DPR (for program P1) = Prob((nodes n 2 , n 3 , and n 4 are connected) OR (nodes n 2 , n 3 , and n 5 are connected)). T HE C OMPUTER J OURNAL,. FIGURE 1. A simple DCS.. Many algorithms [1, 2, 3, 4, 5, 6, 7] have been proposed to analyse the reliability of DCSs, but most of these algorithms assume that all nodes in the DCS are perfect. However, in real problems, the nodes as well as edges may fail. In Prasnna Kumar et al. [3], the minimum file spanning tree (MFST) was proposed to represent the multiterminal connections required to execute a distributed program and a two-pass method for the reliability analysis of a DCS was developed. In this method, all MFSTs are obtained using a breadth-first search method. Since the MFSTs are not Vol. 42, No. 2, 1999.

(2) 130. M.-S. L IN , D.-J. C HEN. disjoint from each other, once it has found all the MFSTs, the algorithm still requires other terminal reliability evaluation algorithms, such as SYREL [8], to generate the reliability expression. Although Prasnna Kumar’s method is elegant, it generates many redundant subgraphs during processing and requires extra time to check and remove them. Thus, it is an inefficient reliability analysis algorithm. In [5], Ke and Wang proposed an algorithm, ENR/KW (evaluating network reliability/Ke and Wang) which employs a different concept that requires one step to give the reliability expression. The basic idea of the ENR/KW algorithm is to partition the graph directly into a set of disjoint subgraphs. Each disjoint subgraph is generated by maintaining a specific directed graph structure to consider the effect of imperfect nodes. Therefore, the DPR computation can then be carried out by summing all these disjoint probability expressions. This technique represents a onestep approach as there is no need to compute multiterminal connections. Some well known reliability-preserving graph reductions, however, are limited to the specific directed graph structure in the EMR/KW algorithm, and this restriction can increase the complexity of the EMR/KW algorithm that would otherwise use them. The other shortcoming of the EMR/KW algorithm is that it cannot be used to evaluate the reliability of a distributed program running from more than one node. This paper proposed two algorithms, namely the SM (symbolic method) and the FM (factoring method), for computing the reliability of DCSs with imperfect nodes. SM is a two-pass method like Prasnna Kumar’s [3]. SM employs the graph expanding procedure used in the FREA (fast reliability evaluation algorithm [1] to find all MFSTs. Since it has been proved that FREA guarantees that no replicated subgraphs will be generated during the expansion of the computation tree, first pass in SM can also guarantee that no replicated subgraphs will be generated. Therefore, SM is more efficient than Prasnna Kumar’s algorithm [3]. The second proposed algorithm, FM (factoring method), is based on the approach of Theologou and Carlier [10] in which a one-pass method of factoring and reduction was proposed to solve the K -terminal reliability problem with imperfect nodes. The K -terminal reliability problem is to determine the probability that a specified set of nodes K ⊆ V are connected, where V is the entire set of nodes in the network. In reality, the DPR problem is a logical OR-ing of Prob{K -terminals are connected}, as shown in Figure 1, but computing the conditional probabilities required could be rather unpleasant. Belovich [9] has proposed approximation methods in this area. Since the K terminal reliability problem does not consider the effect of data file distribution and the set of K target nodes is not specified in a DCS, the factoring and reductions [10, 11] developed to compute the K -terminal reliability cannot be directly applied to the DPR problem. Obviously, if there are no duplicated files, i.e. if there is only one copy of each file, in the DCS, then the DPR problem can be transferred into an equivalent K -terminal reliability problem in which the K set is just the set of nodes that contain the data files T HE C OMPUTER J OURNAL,. AND. M.-S. H ORNG. needed for the programs under consideration. However, data files are usually duplicated in DCSs, so the factoring and reduction methods for the K -terminal problem cannot be directly applied to the DPR problem. General factoring and reduction methods developed for the DPR problem with perfect nodes have been proposed and discussed in [1, 2, 12]; the FM algorithm concerns the case of a DCS with imperfect nodes. 2. NOTATION AND DEFINITIONS In this paper we will use the following notation and definitions. 2.1. Notation D = (V, E, F). li = (ei , u i , vi ) F Ai ME. pi qi H. R(D H ). D−e D+e. An undirected DCS graph with vertex (node) set V , edge set E, and data file set F. Without loss of generality, we identify a program with a special type of data file, i.e. program ⊆ F. The link i that contains the edge ei and its two endpoints u i and vi . The set of files available at node i . A subset of E that represents the edges merged during the process of finding all MFSTs with the SM algorithm. Reliability of node, edge or link i 1 − pi Subset of files of F, i.e. H ⊆ F, where H contains the programs to be executed and all data files needed for the execution of these programs. The DPR of D with a set H of needed files: Pr{all data files in H can be accessed successfully by the executed programs in H }. We omit H when no ambiguities arise. The graph D with edge e deleted. The graph D with edge e = (u, v) contracted so that node u and v are merged into a single node. This new merged node contains all data files and programs that were in nodes u and v.. Using this notation, we can represent the DPR of program P1 that needs data files f1, f2 and f3 for its execution in the example in Figure 1 by R(D H ) where D = (V, E, F) and V = {n 1 , n 2 , n 3 , n 4 , n 5 , n 6 }, E = {e1 = (n 1 , n 2 ), e2 = (n 1 , n 3 ), e3 = (n 2 , n 3 ), e4 = (n 2 , n 4 ), e5 = (n 3 , n 5 ), e6 = (n 4 , n 5 ), e7 = (n 4 , n 6 ), n 8 = (n 5 , n 6 )}, F = {P1, P2, f1, f2, f3, f4, f5}, F An1 = {f4, f5}, F An2 = {P1, f1}, F An3 = {f2}, F An4 = {f3}, F An5 = {f3}, F An6 = {P2, f4}, and H = {P1, f1, f2, f3}. Vol. 42, No. 2, 1999.

(3) R ELIABILITY A NALYSIS. OF. D ISTRIBUTED C OMPUTING S YSTEMS. 131. Algorithm SM Input: the original DCS graph D = (V, E, F) and the set H of needed files Output: the distributed program reliability DPR begin repeat // reduce the original DCS graph D // perform the degree-1 and parallel reductions perform the series and degree-2 reduction until no reductions can be made Let D 0 be the DCS graph after the reduction step FOUND ← ∅ ME ← ∅ // call FIND FST to find FSTs // FIND FST(D 0 , M E) for all s, tFOUND do // remove the FSTs which are not MFSTs // if t ∩ s = s then remove t from FOUND else if t ∩ s = t then remove s from FOUND endif endif od introduce the nodes which are endpoints of the edges in FOUND apply SYREL [8] to all MFSTs stored in FOUND // call the terminal reliability algorithm // output the DPR. end SM FIND FST(D, M E) begin if there are no FSTs in D then return endif // failure case // if there exists one node n such that F An ⊇ H then FOUND ← FOUND ∪ {M E} return // success case // endif for all ei ∈ the set of edges incident on the nodes containing the programs to be executed do // decompose D by Equation (3.1) // FIND FST(D + ei , M E ∪ {ei }) D ← D − ei remove the irrelevant components from D if there are no FSTs in D then return endif od end FIND FST ALGORITHM 1.. D EFINITION 1. A file spanning tree (FST) [3] is a tree whose nodes hold all needed files in H . D EFINITION 2. A minimal file spanning tree (MFST) [3] is a FST such that there exists no other FST that is a subset of it.. FIGURE 2. The subgraphs generated using Equation (3.1).. From the definition of a MFST, the DPR can be written as R(D H ) = Prob(at least one MFST is operational), or ! #m f st [ MFST j R(D H ) = Prob. D EFINITION 4. A working set is a subset of nodes of V such that if and only if all nodes in the working set fail simultaneously then there are no FSTs in the DCS.. j =1. 3. SM ALGORITHM: THE SYMBOLIC METHOD where #mfst is the number of MFSTs for a given needed file set H . D EFINITION 3. A node n is called a reducible node [1] if and only if: (a) the degree of node n is two in the original DCS graph, and (b) node n is not a leaf node of any MFST. T HE C OMPUTER J OURNAL,. When the DPR is computed by a symbolic method, node failure can be accounted for by using the following steps: (i) all MFSTs are derived for the DCS with nodes considered perfect; Vol. 42, No. 2, 1999.

(4) 132. M.-S. L IN , D.-J. C HEN. AND. M.-S. H ORNG. FIGURE 3. The process of finding all MFSTs using SM for the example in Figure 1.. (ii) introduce the nodes which are endpoints of the edges of MFSTs; (iii) a terminal reliability algorithm, such as SYREL [8], is performed on the resulting expression. To find all MFSTs, we can use the following equation proposed in FREA [1] to decompose the original DCS graph into d subgraphs: R(D H ) = pe1 R(D H + e1 ) + qe1 pe2 R(D H − e1 + e2 ) + . . . + qe1 qe2 . . . qed−1 ped R(D H − e1 − e2 − . . . − ed−1 + ed ). (3.1). where {e1 , e2 , . . . , ed } is the set of edges incident to the nodes containing the programs being executed, and where with each subgraph we associate a set M E of edges to store the edges merged in that subgraph. This decomposition operation is performed recursively and the set M E is updated for each induced subgraph until the further induced graph is obtained in which either (a) there exists a node containing all needed data files in H , or (b) there are no FSTs. In the first of these two cases, an FST can be identified and composed by the edges stored in the set M E. After the T HE C OMPUTER J OURNAL,. FSTs covered by other FSTs are deleted, all MFSTs can be found. T HEOREM 3.1. The subgraphs generated by Equation (3.1) are completely disjoint. Proof. According to Equation (3.1), the original graph can be decomposed into d subgraphs as shown in Figure 2. The leftmost branch corresponds to class Te1 , the set of subgraphs including edge e1 . All the other branches correspond to class Te1 , the set of subgraphs without edges e1 . Within Te1 , the second branch splits it into two classes, one to include e2 , another without it, and so on. When the algorithm progresses at each internal node such kinds of splitting occur. This guarantees that no replicated subgraph will be generated. Since the subgraphs generated using Equation (3.1) will be completely disjoint, no duplicate M E sets will be generated during the expansion of the computation tree. Therefore, the SM does not need the CLEAN procedure used in Prasnna Kumar [3] to check and remove duplicate sets. Before the original DCS graph is decomposed by Equation (3.1), the original DCS graph can be reduced to a Vol. 42, No. 2, 1999.

(5) R ELIABILITY A NALYSIS. OF. D ISTRIBUTED C OMPUTING S YSTEMS. 133. Algorithm FM Input: the original DCS graph D = (V, E, F) and the set H of needed files Output: the distributed program reliability DPR begin repeat // reduce the original DCS graph D // perform the degree-1 and parallel reductions perform the series and degree-2 reduction until no reductions can be made Let D 0 be the DCS graph after the reduction step Output(factoring(D 0)) end FM function factoring(D) begin if there are no FSTs in D then return (0) endif if there exists one node n in D such that pn 6= 1 (node n is an imperfect node) and contains the programs to be executed // identify the working set of nodes // then D1 ← D with setting pn = 1; D2 ← D with deleting node n and its adjacent edges return ( pn · factoring(D1) + qn ·factoring(D2)) // Equation (4.2) // endif // call edge factoring // return(e factoring(D)) end factoring function e factoring(D) begin if there exists one node n such that F An ⊇ H then return (1) endif // success case // if there are no FSTs in D then return (0) endif // failure case // repeat // reduction step // perform degree-1 and parallel reductions perform series and degree-2 reductions until no reductions can be made let D 0 be the DCS graph after the reduction step R←0 C ←1 for all link li = (ei , u i , vi ) in the links (edges) incident to the nodes containing the programs to be executed do // Equation (4.3) // R ← R + C · ( pei · pu i · pvi ) · e factoring(D + ei ) ˙ − pei · pu i · pvi ) C ← C (1 D ← D − ei pv0 i ← ( pvi · qei )/(qvi + pvi · qei ) // Equation (4.4) // // Equation (4.4) // pu0 i ← ( pu i · qei )/(qu i + pu i · qei ) remove the irrelevant components from D if there are no FSTs in D then return (R) endif od return(R) end e factoring ALGORITHM 2.. smaller size by the following reduction methods developed for the DPR problem with imperfect nodes. •. Degree-1 reduction. A node is referred to as a degree1 node if it has only one incident edge. Degree1 reduction removes (i) degree-1 nodes that contain none of the needed data files and programs under consideration and (ii) their incident edges. T HE C OMPUTER J OURNAL,. •. •. Irrelevant component deletion. Let D 0 be a connected component of D that is not connected to the rest of the components of D. If there are no FSTs in D 0 , then the component D 0 is irrelevant and can be deleted. Parallel reduction. Let ea = (u, v) and eb = (u, v) be two parallel edges in D. D 0 is obtained by replacing ea and eb with a single edge ec = (u, v) such that pec = 1 − qea · qeb (or pec = pea + peb − pea · peb ).. Vol. 42, No. 2, 1999.

(6) 134. M.-S. L IN , D.-J. C HEN. AND. M.-S. H ORNG. FIGURE 4. The computation tree of FM for the example in Figure 1.. •. •. The parallel reduction for the DPR problem is the same as the parallel reduction for the K -terminal network reliability problem. Series reduction. Let ea = (u, v) and eb = (v, w) be two series edges in a DCS graph D such that degree(v) = 2 and F Av∩ H = ∅, i.e. node v contains no required data files or programs to be executed. Then a DCS graph D 0 is obtained by replacing ea and eb with a single edge ec = (u, w) such that pec = pea · pv · peb . Degree-2 reduction. Suppose node v is a reducible node; then one can apply series reduction on node v and move data files and programs within node v to a node u or w.. The complete SM algorithm can be stated as in Algorithm 1. E XAMPLE 3.1. We use the example in Figure 1 to illustrate our symbolic method. Assume all nodes and edges have the same reliability, 0.9. The process of finding all MFSTs is shown in Figure 3. T HE C OMPUTER J OURNAL,. All MFSTs found in the set FOUND are {(e4 , e9 ), (e4 , e5 , e10 ), (e5 , e9 )}; we then introduce the nodes which are the endpoints of the edges in FOUND. All MFSTs become {(e4 , e9 , n 2 , n 3 , n 4 ), (e4 , e5 , e10 , n 2 , n 3 , n 4 , n 5 ), Applying the terminal reliability (e5 , e9 , n 2 , n 3 , n 5 )}. algorithm SYREL [8] to these MFSTs, we obtain the disjoint terms d1 = pn2 · pn3 · pn4 · qn5 · pe4 · pe9 , d2 = pn2 · pn3 · pn4 · qn5 · pe4 · qe5 · pe9 , d3 = pn2 · pn3 · pn4 · qn5 · pe4 · qe5 · pe9 · pe10 , and d4 = pn2 · pn3 · pn5 · pe5 · pe9 . P The DPR is computed to be 4i=1 di = 0.7736. 4. FM ALGORITHM: THE FACTORING METHOD For the K -terminal reliability of a network with imperfect nodes, a modified factoring method has been proposed in Vol. 42, No. 2, 1999.

(7) R ELIABILITY A NALYSIS. OF. D ISTRIBUTED C OMPUTING S YSTEMS. 135. TABLE 1. File distributions used for comparison in the ARPA network. P1. f1. f2. f3. f4. f5. Set 1 Set 2 Set 3 Set 4 Set 5 Set 6 Set 7 Set 8 Set 9 Set 10. 8, 18 18, 21 5, 18 9, 16 8, 17 8, 15 11, 14 2, 8 17, 18 4, 9. 18, 21 9, 13 3, 8 15, 10 19, 20 4, 9 2, 18 2, 20 4, 12 1, 20. 9, 13 3, 18 11, 4 6, 8 8, 18 6, 12 6, 7 10, 5 4, 5 10, 18. 3, 18 11, 17 2, 19 9, 17 18, 21 20, 21 7, 19 10, 12 13, 15 15, 16. 11, 17 2, 20 8, 10 4, 10 9, 13 1, 12 3, 12 2, 8 12, 14 4, 16. 2, 20 8, 20 2, 19 3, 18 3, 18 4, 20 11, 16 7, 9 2, 21 17, 18. Set 11 Set 12 Set 13 Set 14 Set 15 Set 16 Set 17 Set 18 Set 19 Set 20. 2, 8, 18 8, 9, 11 3, 16, 21 8, 11, 15 5, 11, 18 2, 5, 15 4, 17, 19 5, 9, 13 11, 14, 19 7, 12, 16. 6, 7, 13 4, 10, 16 12, 14, 19 4, 9, 20 5, 6, 10 12, 15, 21 2, 10, 18 2, 3, 18 4, 6, 17 7, 11, 20. 5, 7, 19 3, 5, 15 2, 11, 13 6, 12, 20 2, 5, 8 1, 13, 14 6, 13, 14 8, 11, 17 10, 18, 21 1, 17, 20. 3, 12, 21 7, 11, 12 1, 4, 14 2, 20, 21 12, 14, 17 10, 16, 18 6, 11, 12 2, 19, 20 2, 5, 11 2, 3, 9. 1, 11, 16 6, 8, 11 3, 4, 17 1, 2, 12 5, 11, 19 5, 17, 20 5, 6, 20 8, 18, 20 6, 17, 18 1, 7, 13. 11, 13, 21 7, 10, 19 5, 9, 13 4, 6, 20 4, 15, 18 2, 4, 11 1, 5, 8 18, 20, 21 5, 13, 15 6, 18, 20. Set 21. 1, 16, 18. 9. 2, 6, 12. 10, 14, 19, 21. 3, 4, 7, 15. Set 22. 1. 6, 7, 8, 16, 18. 5, 10, 11, 21. 2, 15. Set 23. 3. 7, 12, 13. 1, 2, 4, 17. 7. 8, 9, 10, 18, 20 12, 13, 14. 21. Set 24. 4. 3, 11, 16, 17. 5, 6, 8, 21. Set 25. 16, 21. 3, 4, 9, 12, 13, 14, 17, 20 5, 6, 11, 14, 15, 16, 19 1, 2, 9, 10, 15, 18, 19 3, 15. 5, 8, 11, 13, 17, 20 19. 1, 10. 2, 7, 8. Set 26. 13, 18. 12, 20. 7. 4, 5, 19. 4, 14, 18, 19, 20 1, 11, 15, 21. Set 27. 10. 8, 12, 13. 2, 21. 4, 7, 15, 16. Set 28. 4, 11. 1, 16, 17, 18. 3, 8. Set 29. 6, 14. 2, 5, 6, 13, 14, 19 2, 9. 5, 6, 9, 11, 12, 13, 17 2, 3, 6, 8, 9, 10, 14, 16, 17 3, 5, 6, 14, 17, 18 7, 15, 20, 21. 3, 8, 18, 21. 1. Set 30. 3, 4, 9, 21. 7, 10. 4, 7, 10, 11, 15, 19 1, 11, 19. 2, 6, 8, 14, 15, 16, 18. 12, 13, 20. 5, 12, 13, 16, 17, 20 5, 17. [10]. This method is very elegant since it is a one-pass method and only requires a small additional cost for the perfect nodes case. However, the DPR problem is very different from and more complicated than the K -terminal reliability problem. Hence, the factoring and reduction methods proposed in [10] cannot be directly used for the DPR problem. If we consider a DCS with imperfect nodes, the first step in the FM algorithm is to select a working set of nodes. We can choose as a working set a set of nodes including the programs to be executed. Since there is usually only one copy of each program to be executed and each program is stored in only one site in the DCS, the size of the working set we choose can be very small. Once the working set has T HE C OMPUTER J OURNAL,. 1, 9, 11, 19, 20 9, 10, 12. FIGURE 5. ARPA network.. been decided, we factor the nodes in this working set just as we do with edges. For example, if we select the working Vol. 42, No. 2, 1999.

(8) 136. M.-S. L IN , D.-J. C HEN. AND. M.-S. H ORNG. TABLE 2. File distributions used for comparison in the Pacific Basin network. P1. f1. f2. f3. f4. Set 1 Set 2 Set 3 Set 4 Set 5 Set 6 Set 7 Set 8 Set 9 Set 10. 5, 15 3, 19 8, 14 10, 13 14, 19 9, 18 4, 19 13, 19 7, 17 4, 15. 3, 4 8, 13 8, 9 10, 15 1, 19 9, 10 10, 15 1, 8 6, 15 3, 10. 12, 14 12, 14 6, 13 3, 13 8, 11 9, 17 2, 4 8, 12 4, 9 6, 8. 4, 6 10, 14 5, 6 9, 19 1, 7 1, 10 8, 18 5, 16 11, 14 6, 18. 11, 15 13, 18 5, 15 8, 15 5, 13 10, 19 4, 18 1, 13 4, 7 3, 14. Set 11 Set 12 Set 13 Set 14 Set 15 Set 16 Set 17 Set 18 Set 19 Set 20. 11, 13, 19 3, 8, 19 7, 9, 12 12, 15, 17 13, 15, 16 1, 4, 18 7, 12, 15 2, 8, 11 2, 7, 9 1, 5, 7. 3, 15, 17 4, 10, 16 11, 13, 14 5, 7, 13 5, 8, 13 5, 12, 16 6, 10, 16 3, 4, 9 13, 14, 19 2, 12, 15. 1, 6, 14 6, 16, 19 1, 12, 17 2, 8, 19 7, 11, 12 5, 13, 19 5, 6, 10 2, 11, 15 4, 7, 13 3, 4, 8. 2, 6, 18 5, 6, 18 10, 14, 18 2, 3, 18 7, 14, 15 6, 16, 18 1, 12, 13 3, 16, 19 7, 8, 17 7, 10, 15. 7, 12, 14 4, 11, 16 4, 14, 18 6, 9, 12 1, 3, 4 4, 10, 13 7, 9, 14 1, 7, 15 1, 6, 15 1, 7, 13. Set 21. 17. 2, 7, 14. 1, 4, 9, 11, 19. 3, 5, 10, 15. Set 22. 6, 8, 12, 13, 16, 18 5, 10, 17, 18. 9, 16. 4, 15. 1, 2, 13, 14. Set 23 Set 24 Set 25. 7, 10, 11, 18 2, 7, 10, 18, 19 1, 5, 13, 18. 12, 16, 19 1, 3, 4, 6, 14, 17 2, 3, 9. 2, 4, 5, 8, 14 11, 15 10, 11. 3, 15, 17 5, 13 4, 8, 17. Set 26 Set 27. 4, 10, 12 12. 3, 9, 14, 16 8, 9, 18. 13, 15, 17, 18 1, 2, 3, 15. 1, 2, 7, 8, 19 4, 5, 7, 11, 17. Set 28 Set 29. 5 14, 17, 18. 3, 6, 7, 8, 11, 12 1, 6, 9, 13 8, 9, 12, 16 6, 7, 12, 14, 15, 16, 19 5, 6, 11, 15 6, 10, 13, 14, 16, 19 7, 8, 9, 15, 18 2, 8, 11, 12. 12, 14, 19 5, 7. 2, 3, 4, 10, 16 1, 3, 4, 6, 16. Set 30. 4, 10, 18. 8, 11, 14, 15. 6, 9, 12, 13, 17, 19. 5, 7, 16. 1, 6, 11, 13, 17 9, 10, 13, 15, 19 1, 2, 3. DCS graph D. The reliability of the original DCS can then be stated as R(D H ) = pn1 · pn2 · R(D H |n 1 and n 2 work) + pn1 · qn2 · R(D H |n 1 works and n 2 failed) + qn1 · pn2 · R(D H |n 1 failed and n 2 works) + qn1 · qn2 · R(D H |n 1 and n 2 failed).. (4.1). Since the set {n 1 , n 2 } is a working set, by the definition of a working set, the whole DCS will fail if nodes n 1 and n 2 fail simultaneously. Hence, Equation (4.1) can be rewritten as. FIGURE 6. Pacific Basin network.. R(D H ) = pn1 · pn2 · R(D H |n 1 and n 2 work) + pn1 · qn2 · R(D H |n 1 works and n 2 failed). set {n 1 , n 2 } to be factored, then there are four possible combinations in the factoring process and four different disjoint subgraphs D 0 will be produced from the original. In Equation (4.2), each subgraph of (i) D with n 1 and n 2 working, (ii) D with n 1 working and n 2 failed, and (iii). T HE C OMPUTER J OURNAL,. + qn1 · pn2 · R(D H |n 1 failed and n 2 works). (4.2). Vol. 42, No. 2, 1999.

(9) R ELIABILITY A NALYSIS. OF. D ISTRIBUTED C OMPUTING S YSTEMS. 137. FIGURE 7. Plots of execution time for various file distributions.. D with n 1 failed and n 2 working has at least one node working, that is, either n 1 or n 2 . These working nodes can be identified with perfect nodes just as the K target nodes are identified with the perfect nodes in the K -terminal problem. Once we have identified the perfect nodes in each subgraph, the factoring technique used in [10] can be applied. T HE C OMPUTER J OURNAL,. to each subgraph D 0 , and the technique can be generalized as follows: R(D 0H ) = pl1 · R · (D 0H + e1 ) + ql1 p˙ l2 · R(D 0H − e1 + e2 ) + . . . + ql1 · ql2 . . .. · pld · R(D 0H − e1 − e2 − . . . − ed−1 + ed ) (4.3). Vol. 42, No. 2, 1999.

(10) 138. M.-S. L IN , D.-J. C HEN. AND. M.-S. H ORNG. TABLE 3. Comparison of running times (in s) for different sets of file distributions. The ARPA network. The Pacific Basin network. MFST1. FM1. SM1. DPR1. MFST1. FM1. SM1. DPR1. Set 1 Set 2 Set 3 Set 4 Set 5 Set 6 Set 7 Set 8 Set 9 Set 10. 31,886 35,613 29,709 79,298 74,879 393,575 2330 4141 47,862 263,853. 12 4 7 23 12 37 2 2 7 32. 52 55 106 576 33 1797 10 5 139 2046. 0.72208 0.70664 0.85759 0.77292 0.62265 0.62333 0.77055 0.84159 0.71358 0.70700. 120,426 336,382 8704 10,128 566,992 122,442 112,050 146,150 51,973 12,201. 3 3 0.7 2 4 1 2 3 3 2. 69 56 2 43 2843 5 17 460 90 144. 0.62321 0.75127 0.90212 0.79601 0.72288 0.84335 0.65175 0.73069 0.86122 0.86894. Set 11 Set 12 Set 13 Set 14 Set 15 Set 16 Set 17 Set 18 Set 19 Set 20. 3671 4652 10,029 43,565 2538 24,679 47,313 11,785 7597 14,706. 73 41 99 32 22 123 83 31 18 27. 311 845 3040 936 100 1097 4093 40 253 583. 0.90007 0.90489 0.87593 0.89481 0.85926 0.81013 0.89348 0.77215 0.82016 0.88712. 44,053 59,003 241,796 313,406 76,356 169,286 202,565 39,632 128,042 56,692. 23 12 37 18 17 38 13 17 32 34. 2559 168 3520 20,305 2672 18,246 1075 934 1637 5532. 0.88738 0.94013 0.93582 0.87653 0.78671 0.89207 0.90548 0.93888 0.91192 0.93303. Set 21 Set 22 Set 23 Set 24 Set 25 Set 26 Set 27 Set 28 Set 29 Set 30. 48,126 42,324 33,687 84,666 15,848 4382 1663 7130 161,932 16,200. 80 1 3 5 19 6 5 88 16 146. 496 8 62 208 1414 50 329 3666 246 1599. 0.77612 0.48504 0.53779 0.70892 0.73106 0.67767 0.76526 0.88650 0.76793 0.83323. 327,540 76,588 219,423 142,629 55,135 104,456 356 44,681 246,909 26,970. 414 141 222 608 330 162 1 6 31 18. 439 6058 6203 4479 2831 39681 156 1206 8060 311. 0.67141 0.82579 0.89497 0.81395 0.79374 0.92176 0.67407 0.81330 0.91986 0.81735. 1 A Sun SPARC system 600 workstation was used to run the program(s).. TABLE 4. File distribution table. F A n1 F A n2 F A n3 F A n4 F A n5 F A n6 F A n7 F A n8 F A n9 F A n10. = {f1, f2, f3} = {f2, f3, f4} = {f3, f4, f5} = {f4, f5, f6} = {f5, f6 ,f7} = {f6, f7, f8} = {f1, f7, f8} = {f1, f2, f8} = {f3, f7, f8} = {f1, f4, f7}. FIGURE 8. Benchmark DCS D8,6 .. where D 0 is a subgraph generated by the process of factoring on the working set of D, the set {l1 = (e1 , v1 , u 1 ), l2 = (e2 , v2 , u 2 ), . . . , ld = (ed , vd , u d )} is the set of links incident to the nodes that are identified with perfect nodes and pli = pei · pvi · pu i is the probability of link i working, for T HE C OMPUTER J OURNAL,. i = 1 to d. In each subgraph of D 0 with edge ei = (u i , vi ) deleted, the reliability of the two endpoints u i and vi , is updated with the new reliability pv0 i = ( pvi · qei )/(qvi + pvi · qei ), and. pu0 i = ( pu i · qei )/(qu i + pu i · qei ) (see [10]). Vol. 42, No. 2, 1999. (4.4).

(11) R ELIABILITY A NALYSIS. OF. D ISTRIBUTED C OMPUTING S YSTEMS. 139. TABLE 5. Number of subgraphs generated and actual execution time against different topologies. Number of subgraphs generated. Execution time (s). DCS. MFST1. ENR/KW2. SM1. FM1. MFST1. ENR/KW2. SM1. FM1. DPR1. D8,4 D10,4 D8,6 D8,7 D10,7 D8,8 D10,8 D10,9. 35 55 306 1159 3443 3225 20,464 131,899. 16 20 72 289 462 1196 2556 17,832. 10 9 32 181 151 1356 1195 8619. 8 7 20 64 60 260 250 1255. <0.01 < 0.01 0.02 0.81 1.30 21.83 103.82 5023.68. <0.01 0.01 0.02 0.06 0.15 0.22 0.82 5.64. <0.01 <0.01 0.01 0.28 0.21 20.66 16.69 183.72. <0.01 <0.01 <0.01 0.01 0.02 0.08 0.09 0.51. 0.8915513 0.8893553 0.8988961 0.8990611 0.8990729 0.8990899 0.8990949 0.8990990. 1 An Intel Pentium P133 PC was used to run the program(s). 2 These values were obtained by Ke and Wang [5].. Equations (4.3) and (4.4) and the reduction methods described in Section 3 can be recursively applied to each induced graph until either (i) the further induced graph with a node contains all needed data files and all programs to be executed or (ii) the further induced graph contains no FSTs. The former case represents success (reliability = 1); the latter case represents failure (reliability = 0). The complete FM algorithm is stated in Algorithm 2.. and time units are required for their algorithms to run the distributed programs under the effects of different topologies of the DCS and file distributions.. E XAMPLE 4.1. To illustrate the FM algorithm, we shall again use the example in Figure 1. Assume the reliability of all the nodes and links is 0.9. The complete computation tree of FM is shown in Figure 4. From Figure 4, the DPR can be computed as. (i) the file distribution on the nodes of the DCS; and (ii) the topology of DCS.. DPR = 0.9 · 0.81 · 0.8948 + 0.9 · 0.1662 · 0.81 = 0.7736. 5. COMPLEXITY AND COMPARISONS 5.1. Complexity analysis It is well known that computing K -terminal reliability in general is N P-hard, or #P-complete [13]. However, for some classes of networks, for example, tree and seriesparallel networks, the K -terminal reliability problems can be computed in linear time by applying well known reductions like series, degree-2, parallel and polygon-tochain reductions [14]. However, the DPR problem is much more complicated than the K -terminal problem, since its computational complexity is dependent not only on the topology of the network but also on the file distributions. Actually we have shown that the DPR problem for seriesparallel, tree and star networks is still N P-hard [15]. Therefore, there exists no polynomial time algorithm to compute the reliability of the distributed program for general distributed computing systems. Naturally, the SM and FM require exponential time, i.e. 2|V |+|E| , in the worst case. An appropriate and rational comparison for these different algorithms can be made based on the counting approach which counts the number of intermediate trees or subgraphs generated during the whole reliability evaluation. From such a comparison, one can tell how much memory space T HE C OMPUTER J OURNAL,. 5.2. Comparisons The algorithms developed in this paper will now be compared with the existing algorithms under the changes in:. 5.2.1. Effect of data file distributions on performance of different algorithms The algorithms were tested using the examples of the ARPA network and the Pacific Basin network. The topologies of the ARPA and Pacific Basin networks are shown in Figures 5 and 6. All edges and nodes have reliability = 0.9. Thirty sets of file distributions, generated randomly, for each of the ARPA and Pacific Basic networks are listed in Tables 1 and 2. In Tables 1 and 2, sets, 1–10 are those file distributions for which two copies of each file are distributed randomly in the network while sets 11–20 are for three copies of each file. We also generate randomly the other kind of file distribution, sets 21–30, where each node contains only one data file. Table 3 gives running times and the reliability obtained for the example of Tables 1 and 2. The execution time is also plotted in Figure 7. From Table 3 and Figure 7, it is clear that the SM and FM algorithms are much more efficient than Prasnna Kumar’s [3] algorithm. 5.2.2. Effect of topology on performance of different algorithms In this study, we want to see the effect of topological configuration on the performance of different algorithms used. Consider the benchmark networks given in ENR/KW [5]. Let Di j be the benchmark DCS with i nodes and node n 1 to node n j being completely connected. Figure 8 depicts the example of D8,6 . The file distributions are given in Table 4. Assume program P1 needs data files { f 1, f 3, f 5} Vol. 42, No. 2, 1999.

(12) 140. M.-S. L IN , D.-J. C HEN. AND. M.-S. H ORNG. FIGURE 9. Plots of the number of subgraphs generated and execution time for topology variations.. for its executions. These topologies and file distributions are the same as those used in ENR/WK [5]. Table 5 shows the number of subgraphs generated and the actual execution time against different topologies based on program P1 as T HE C OMPUTER J OURNAL,. executed at node n 1 . In addition, Figure 9 shows the plots of Table 5. From Table 5 and Figure 9, it is clear that the FM algorithm is much more efficient, compared with the other algorithms, in any of these different topologies. Vol. 42, No. 2, 1999.

(13) R ELIABILITY A NALYSIS. OF. D ISTRIBUTED C OMPUTING S YSTEMS. 6. CONCLUSION In this paper we have proposed two algorithms for computing the reliability of distributed computing systems with imperfect nodes. The first algorithm, SM, is a twopass method that requires the terminal reliability algorithms to compute the reliability values of symbolic expressions obtained from the SM. The second algorithm, the FM, uses a factoring approach that directly computes the reliability without enumerating the symbolic expressions. In addition, the use of various reliability preserving reduction techniques in the SM and FM implies that the size of the graph will be reduced and, therefore, fewer subgraphs will be generated. Comparisons with existing methods on various file distributions and network topologies show the usefulness of the FM algorithm for complex DCSs. REFERENCES [1] Chen, D. J. and Lin, M. S. (1994) On distributed computing systems reliability analysis under program execution constraints. IEEE Trans. Comput., 16, 87–97. [2] Chen, D. J. and Huang, T. H. (1992) Reliability analysis of distributed systems based on a fast reliability algorithm. IEEE Trans. Parallel Distrib. Syst., 3, 139–153. [3] Prasnna Kumar, V. K., Hariri, S. and Raghavendra, C. S. (1986) Distributed program reliability analysis. IEEE Trans. Software Eng., 12, 42–50. [4] Kumar, A., Rai, S. and Agrawal, D. P. (1988) On computer communication network reliability under program execution constraints. IEEE J. Select. Areas Commun., 6, 1393–1399.. T HE C OMPUTER J OURNAL,. 141. [5] Ke, W. J. and Wang, S. D. (1997) Reliability evaluation for distributed computing networks with imperfect nodes. IEEE Trans. Reliability, 46, 342–349. [6] Lopez-Benitez, N. (1994) Dependability modeling and analysis of distributed programs. IEEE Trans. Software Eng., 20, 345–352. [7] Kumar, A. and Agrawal, D. P. (1993) A generalized algorithm for evaluating distributed program reliability. IEEE Trans. Reliability, 42, 416–426. [8] Hariri, S. and Raghavendra, C. S. (1987) SYREL: A Symbolic Reliability Algorithm based on Path and Cutset Methods. IEEE Trans. Computers, 36, 1224–1232. [9] Belovich, S. G. (1995) A design technique for reliable networks under a non-uniform traffic distribution. IEEE Trans. Reliability, 44, 377–387. [10] Theologou, O. R. and Carlier, J. G. (1991) Factoring and reductions for networks with imperfect vertices. IEEE Trans. Reliability, 40, 210–217. [11] Satyanarayana, A., Chang, M. K. (1983) Network reliability and the factoring theorem. Networks, 13, 107–120. [12] Lin, M. S. and Chen, D. J. (1993) General reduction methods for the reliability analysis of distributed computing systems. Comp. J., 36, 631–644. [13] Ball, M. O. (1986) Computational complexity of network reliability analysis: an overview. IEEE Trans. Reliability, 35, 230–239. [14] Satyanarayana, A. and Wood, R. K. (1985) A linear-time algorithm for computing K -terminal reliability in seriesparallel networks. SIAM J. Comput., 14, 818–832. [15] Lin, M. S. and Chen, D. J. (1997) The computational complexity of the reliability problem on distributed systems. Inform. Process. Lett., 64, 143–147.. Vol. 42, No. 2, 1999.

(14)