Reliability evaluation of dependable distributed computing systems based on recursive merge and BDD

Abstract

System reliability evaluation, sensitivity analysis, im-portance measures, failure frequency analysis and optimal design have become important issues for distributed de-pendable computing. Finding all the Minimal File Span-ning Trees (MFST) and avoiding repeatedly computing the redundant MFSTs is the key technique for evaluating the reliability of a distributed computing system (DCS) in pre-vious works. However, identifying all the disjoint MFSTs is difficult and very time consuming for large-scale net-works. Although existing algorithms have been demon-strated that they work fine on medium-scale networks, they have two inherent drawbacks. First, they do not support efficient manipulation of Boolean algebra. The sum-of-disjoint-products method used by them is ineffi-cient in dealing with large Boolean functions. Second, the tree-based partitioning algorithm does not merge isomor-phic sub-problems and therefore, redundant computations cannot be avoided. In this paper, we propose a new effi-cient algorithm for the reliability evaluation of a DCS based on recursive merge and binary decision diagram (BDD). Using the BDD substitution technique, we can easily apply our algorithm to a network with imperfect nodes. The experimental results show a significant im-provement on the execution time compared to previous works.

1. Introduction

The development of computer networking and embed-ded VLSI processing devices has led to an increasing in-terest in distributed computing systems (DCS) in which the computations are distributed among many processing elements (PEs). Distributed computing involves

†

Y.R. Chang is also with Institute of Nuclear Energy Research, Atomic Energy Council, Taiwan.

Acknowledgment: This research was supported by the National Science Council, Taiwan, R.O.C. under Grant NSC 92-2213-E-002-011.

tion among several loosely coupled computers communi-cating over a network. Distributed systems provide cost- effective means for resource sharing and extensibility, and obtain potential increases in performance, reliability, and fault tolerance. A distributed program usually requires one or more of the resources for successful execution, such as PEs, data files, etc. For successful completion of a pro-gram, the local host (the PEs that contain the required files) and the interconnection links must all function correctly. Therefore, the distribution of data files can affect the over-all reliability of the system. Thus, an important problem in distributed system design and analysis is to define and evaluate various reliability measures as well as estimate the effect of program and resource distributions on the re-liability of a system efficiently. This analysis is crucially important for building a reliable distributed computing system.

There were many researchers studying the distributed program reliability (DPR) and distributed system reliabil-ity (DSR). Kumar et al. [1] seems to be the first to present the definition of the DPR and DSR. They constructed a distributed model including edges, nodes and resource files and proposed the Minimal File Spanning Trees (MFST)-based algorithm to evaluate the DPR and DSR. Later, based on MFST, Raghavendra [2] addressed two measures, distributed program-user reliability and distrib-uted system-user reliability, and proposed an algorithm for their evaluations. Kumar [3] also developed a fast algo-rithm to evaluate the DPR and DSR. These methods are 2-step algorithms. First, they need to find all the MFSTs. Second, they convert these MFSTs to a symbolic reliability expression using an existing reliability evaluation algo-rithm like SYREL [4] to compute the disjoint probability. The major drawback with these methods is that finding all the MFSTs has high computational complexity; and prior knowledge about multi-terminal connections is required in order to compute the reliability expression, thereby mak-ing them inapplicable to large systems. To overcome these problems, Kumar [5] proposed a 1-step algorithm GEAR that can avoid computing the redundant MFSTs and reduce

Reliability Evaluation of Dependable Distributed Computing Systems

Based on Recursive Merge and BDD

Yung-Ruei Chang

, Hung-Yau Lin, and Sy-Yen Kuo

Department of Electrical Engineering, National Taiwan University, Taipei, Taiwan

[email protected]

computational time. To further improve the efficiency of reliability assessment, Chen [6][7] proposed FST-SPR and HRFST algorithm based on the cut-set methods for reduc-ing the reliability evaluation complexity. However, apply-ing their methods to the network with imperfect nodes is not easy. Taking the existence of faulty nodes into account, Ke [8] proposed the ENR/KW algorithm to compute the reliability of a distributed computing network with imper-fect nodes. ENR/KW algorithm needs to find the set of mandatory nodes and does not converge the isomorphic subproblems. Later, based on the model of [1], Zang [9] proposed a Binary Decision Diagram (BDD)-based algo-rithm to analyze the dependability of a DCS with imper-fect fault-coverage. The researches in [10][11][12] contin-ued with the study of the DPR and DSR based on the model of Kumar [1].

Finding all the MFSTs and avoiding the computation of generating the redundant MFSTs is the key technique to evaluate the reliability of a DCS in previous works. How-ever, identifying all the disjoint MFSTs is difficult and is very time consuming. Although the algorithms in previous works have been demonstrated with reasonable efficiency on medium-scale networks, they have two inherent draw-backs. First, they do not support efficient manipulation of Boolean algebra. The sum-of-disjoint-products method used by them is inefficient in dealing with larger Boolean functions. Second, the tree-based partitioning algorithm does not consider the convergence of isomorphic sub- problems and therefore, redundant computations cannot be exactly avoided.

Recent literature [13][14][15][16][17][18][19] show that BDD is a very efficient approach for reliability evaluation. In this paper, we propose a BDD-based algo-rithm, named CLK, to compute the reliability of a DCS with both perfect and imperfect nodes. The main idea, which makes the CLK algorithm more efficient than the previous works, is that the BDD representing the Boolean reliability expression of a DCS can be constructed by avoiding the redundant computation of the isomorphic sub-problems during the merging process. Therefore, the reliability can be quickly derived from the BDD. In addi-tion, our method can be integrated with the methodologies that use the BDD to analyze the dependability of a system, such as system availability, system failure frequency, im-portance measures and sensitivity analysis [19].

Section 2 introduces the concepts of BDD and distrib-uted computing systems. Section 3 illustrates an efficient algorithm based on recursive merge and BDD to evaluate the DPR and DSR of a distributed computing system. Based on the BDD substitution technique, our algorithm is applicable to not only a system with perfect nodes but also a system with imperfect nodes. The experimental results on various benchmark networks are shown in Section 4.

Section 5 gives the conclusions and future works.

2. Preliminaries

2.1. Binary decision diagram (BDD)

BDD [13] is based on a disjoint decomposition of a Boolean function called the Shannon expansion. Given a Boolean function f(x1,,xn), then for any i{1,,n};

i i i x x x {™ 1 : 0 1 ˜ ˜ xi i xi i f x f x f (1)

In order to express the Shannon decomposition concisely, the if-then-else (ite) format [20][21] is defined as:

) , , (xi fxi 1 fxi 0 ite f

The way that BDDs are used to represent logical opera-tions is simple. In practice, the BDD is generated by using logical operations on variables. Let Boolean expressions f and g be: ) , , ( ) , , ( ) , , ( ) , , ( 0 1 0 1 0 1 0 1 G G x ite g g x ite g F F x ite f f x ite f j x x j i x x i j j i i

A logic operation between f and g can be represented by BDD manipulations as: ° ¯ ° ® ­ ! ¡ ¡  ¡ ¡ ¡ ¡ ¡ ) ( ordering ) ( ordering ) , , ( ) ( ordering ) ( ordering ) , , ( ) ( ordering ) ( ordering ) , , ( ) , , ( ) , , ( 0 1 0 1 0 0 1 1 0 1 0 1 j i j j i i j i i j i x x G f G f x ite x x g F g F x ite x x G F G F x ite G G x ite F F x ite (2)

where ¸ represents a logic operation such as AND or OR. Fig. 1 illustrates the construction and manipulation steps of a Boolean function. For more details on using the

op-F = (x1and x3) or (x2and x3) Variable Ordering: x1<x2<x3

Evaluation Steps:

x1= declare_var(x1, 1) x2= declare_var(x2, 1) x3= declare_var(x3, 1) T1= BDD_and(x1, x3) T2= BDD_and(x2, x3) F = BDD_or(T1, T2)

0 1 x₁ 0 1 x₂ 0 1 T₁ 0 1 x₃ T₂ 0 1 F 0 1 x1 x2 x₃ x1 x₃ x2 x₃ x1 x2 x₃ x₃ F = (x1and x3) or (x2and x3) Variable Ordering: x1<x2<x3 Evaluation Steps:

x1= declare_var(x1, 1) x2= declare_var(x2, 1) x3= declare_var(x3, 1) T1= BDD_and(x1, x3) T2= BDD_and(x2, x3) F = BDD_or(T1, T2)

0 1 x₁ 0 1 x₂ 0 1 T₁ 0 1 x₃ T₂ 0 1 F 0 1 x1 x1 x2 x2 x₃ x₃ x1 x1 x₃ x₃ x2 x2 x₃ x₃ x1 x1 x2 x2 x₃ x₃ xx₃₃

erations of BDD, please refer to [13].

A useful property of BDD is that all the paths from the root to the leaves are mutually disjoint. If f represents the Boolean expression of the system availability, based on the property of the disjoint decomposition of BDD, the re-liability (or availability) of a system can be recursively evaluated by (1) as } Pr{ } Pr{ } Pr{ } Pr{ } Pr{f xi ˜ fxi1  xi ˜ fxi 0 (3)

where Pr{·} means Pr{·=1} for simplification. For exam-ple, if Pr{xi} is the availability Ai of component i and Ui is

the unavailability of component i, then the system avail-ability A is: 0 1 0 1 (1 ) } Pr{f AiAxi UiAxi AiAxi  Ai Axi A (4)

where Axi1 and Axi 0 represent Pr{fxi 1} and Pr{fxi 0}

respectively. Similarly, the unavailability of a system can be calculated as: 0 1 } Pr{g UiUxi AiUxi U (5)

where g is the system unavailability expression and the dual of f ; i.e. f(x1,x2,,xn){1g(1x1,1x2,, ) , , , ( 1 ) 1xn { g x1 x2  xn , Uxi 1 and Uxi 0 represent } Pr{gxi 1 andPr{gxi 0}respectively.

2.2. Distributed computing system (DCS)

In [1], Kumar et al. modeled a DCS as an undirected graph G[V, E] in which the nodes represent the hosts and the edges represent the communication links, where V is a set of nodes and E is a set of edges. Fig. 2 shows an exam-ple of a six-node DCS. FAi represents the set of files that

could be obtained at node i. PRGi represents the set of

programs that could be run at node i. FNj represents the set

of required files for the successful execution of program j. For example, program P2 could be executed on either n3 or

n4. According to FN2 = {F2, F4}, program P2 can run

suc-cessfully on n3 due to the successful access of the data files

{F2, F4}. However, program P2 could not be run

success-fully on n4 without the successful access of the data {F4}

since only the data {F2} is provided at node n4. A

file-spanning tree (FST) is defined as a spanning tree that connects the root node (the host node that runs the pro-gram under consideration) to some other nodes such that its nodes contain all the required files for the successful execution of the program. An FST is a minimal file-spanning tree (MFST) if there exists no other FST that is a subset of this FST. For instance, program P2 in Fig. 2

will function if it can run at node n3 or n4, and can access

files {F2, F4}. Therefore, {n3} and {n4, x4, n2, x3, n3} are

two FSTs of P2, but the later one is not a MFST since {n3}

is a subset of {n4, x4, n2, x3, n3}. The set of MFSTs of

pro-gram P2 in Fig. 2 are: {n3}, {n4, x4, n2, x5, n5}, {n4, x7, n5},

{n4, x8, n6, x9, n5}.

By the definition of MFST [1], the distributed program reliability (DPR) for program j and the distributed system reliability (DSR) are defined respectively as:

} 1 Pr{ 1





where nmfst is the number of MFSTs belonging to program j

and mmfst is the number of MFSTs over all programs.

3. The CLK algorithm

Finding FSTs and using MFSTs to compute the DPR by the disjoint method in the previous works is difficult when the DCS network becomes large and complex. In this sec-tion, we will develop an efficient algorithm, named CLK, based on the convergence of the isomorphic sub-problems to compute the DPR and the DSR of a distributed com-puting system. With this method, redundant computations can be avoided. The experimental results presented later will show the effectiveness of our approach compared to the previous works [1][6][7][8][9][22]. Moreover, the CLK algorithm has the capability of dealing with large number of Boolean variables using BDD. Therefore, the CLK algorithm is applicable to a large-scale DCS. In this section, we will first discuss the algorithm for a DCS net-work with perfect nodes. Then, using the BDD substitution technique, the algorithm can be easily and efficiently ap-plied to a DCS network with imperfect nodes.

3.1. Algorithm for perfect nodes

Based on the model in [1], a distributed program can be run successfully on a host node if all the required files in the DCS network can be correctly accessed. The basic idea of the CLK algorithm for computing the DPR is to begin

Node 1 FA1: F1 PRG1: P1 Node 3 FA3: F2F4 PRG3: P2 Node 2 FA2: F1F2 PRG2: Ø Node 4 FA4: F2F3 PRG4: P2P3 Node 5 FA5: F4 PRG5: P3 Node 6 FA6: F3 PRG6: Ø FN1= F1F2F3 FN2= F2F4 FN3= F1F3 x1 x2 x3 x4 x5 x6 x7 x8 x9 Node 1 FA1: F1 PRG1: P1 Node 1 FA1: F1 PRG1: P1 Node 3 FA3: F2F4 PRG3: P2 Node 3 FA3: F2F4 PRG3: P2 Node 2 FA2: F1F2 PRG2: Ø Node 2 FA2: F1F2 PRG2: Ø Node 4 FA4: F2F3 PRG4: P2P3 Node 4 FA4: F2F3 PRG4: P2P3 Node 5 FA5: F4 PRG5: P3 Node 5 FA5: F4 PRG5: P3 Node 6 FA6: F3 PRG6: Ø Node 6 FA6: F3 PRG6: Ø FN1= F1F2F3 FN2= F2F4 FN3= F1F3 x1 x2 x3 x4 x5 x6 x7 x8 x9

the merging procedure from a host node s where the pro-gram can run and produce subgraphs by recursively merg-ing the adjacent nodes one by one into the source set (SS). Each subgraph has its corresponding SS. An SS is the set of nodes such that the host node sSS and SS are connected. Therefore, if the SS of a subgraph satisfies the requirement (i.e. contains all the required files), then the merging with adjacent nodes stops. Fig. 3 illustrates the procedure of

recursive merge in the CLK algorithm for evaluating the DPR1 in Fig. 2. The traversed path, which includes the

branches from the top-root graph to any one of the bot-tom-leaf subgraphs in Fig. 3, represents a FST. It can be transformed into the BDD representation by the CLK al-gorithm. Then taking the advantage of the disjoint property of BDD, the DPR can be efficiently calculated.

Based on the above idea, let each graph have its own x1 SS = n1 x1 x2 x4 x5 x₆ x9 x8 x₇ x₃ n₄ n₁ n₂ n3 n5 n₆ x6 x₄ x2 _x 5 x₉ x₈ x7 x3 n4 n₃ n₅ n6 SS = n1n2 x4 x5 x₆ x9 x8 x7 n4 n5 n6 SS = n1n2n3 x1 x₄ x₅ x6 x₉ x₈ x7 x3 n4 n2 n₅ n6 SS = n1n3 n3 x2x3 x5x7 x6 x9 x₈ n5 n6 SS = n1n2n4 x2x3x6 x9 x8 x4x7 n4 n₃ n6 SS = n1n2n5 x4 x1x3x5 x9 x₈ x7 n4 n2 n6 SS = n1n3n5 x9 x8 x5x6x7 n5 n6 SS = n1n2n3n4 n3 x8 x₂x₃x₆ x9 n6 SS = n1n2n4n5 x4x7 x₉ x8 n4 n6 SS = n1n2n3n5 x2x3x6 x4x7x8 n4 n₃ SS = n1n2n5n6 x9 x₈ n6 SS = n₁n₂n₃n₄n₅ x4 x1x3x5 x₇x₈ n4 n2 SS = n1n3n5n6 x1x3x4x5 x9 x₈ n2 n6 SS = n1n3n4n5 x4x7x8 n4 SS = n1n2n3n5n6 x2 x₂+x₃ x₄ x₅ x₁+x₃ x₆ x4 x5+x6 x2+x3+x6 x4+x7 x9 x₁+x₃+x₅ x₇ x₉ x4+x7 x9 # #

# represents an isomorphic graph. + represents a Boolean operation OR.

G1 G2 G3 G4 x1 SS = n1 x1 x2 x4 x5 x₆ x9 x8 x₇ x₃ n₄ n₁ n₂ n3 n5 n₆ SS = n1 x1 x2 x4 x5 x₆ x9 x8 x₇ x₃ n₄ n₁ n₂ n3 n5 n₆ SS = n1 x1 x2 x4 x5 x₆ x9 x8 x₇ x₃ n₄ n₁ n₂ n3 n5 n₆ x6 x₄ x2 _x 5 x₉ x₈ x7 x3 n4 n₃ n₅ n6 SS = n1n2 x6 x₄ x2 _x 5 x₉ x₈ x7 x3 n4 n₃ n₅ n6 SS = n1n2 x6 x₄ x2 _x 5 x₉ x₈ x7 x3 n4 n₃ n₅ n6 SS = n1n2 x4 x5 x₆ x9 x8 x7 n4 n5 n6 SS = n1n2n3 x4 x5 x₆ x9 x8 x7 n4 n5 n6 SS = n1n2n3 x4 x5 x₆ x9 x8 x7 n4 n5 n6 SS = n1n2n3 x1 x₄ x₅ x6 x₉ x₈ x7 x3 n4 n2 n₅ n6 SS = n1n3 x1 x₄ x₅ x6 x₉ x₈ x7 x3 n4 n2 n₅ n6 SS = n1n3 x1 x₄ x₅ x6 x₉ x₈ x7 x3 n4 n2 n₅ n6 SS = n1n3 n3 x2x3 x5x7 x6 x9 x₈ n5 n6 SS = n1n2n4 n3 x2x3 x5x7 x6 x9 x₈ n5 n6 SS = n1n2n4 n3 x2x3 x5x7 x6 x9 x₈ n5 n6 SS = n1n2n4 x2x3x6 x9 x8 x4x7 n4 n₃ n6 SS = n1n2n5 x2x3x6 x9 x8 x4x7 n4 n₃ n6 SS = n1n2n5 x2x3x6 x9 x8 x4x7 n4 n₃ n6 SS = n1n2n5 x4 x1x3x5 x9 x₈ x7 n4 n2 n6 SS = n1n3n5 x4 x1x3x5 x9 x₈ x7 n4 n2 n6 SS = n1n3n5 x4 x1x3x5 x9 x₈ x7 n4 n2 n6 SS = n1n3n5 x9 x8 x5x6x7 n5 n6 SS = n1n2n3n4 x9 x8 x5x6x7 n5 n6 SS = n1n2n3n4 x9 x8 x5x6x7 n5 n6 SS = n1n2n3n4 n3 x8 x₂x₃x₆ x9 n6 SS = n1n2n4n5 n3 x8 x₂x₃x₆ x9 n6 SS = n1n2n4n5 n3 x8 x₂x₃x₆ x9 n6 SS = n1n2n4n5 x4x7 x₉ x8 n4 n6 SS = n1n2n3n5 x4x7 x₉ x8 n4 n6 SS = n1n2n3n5 x4x7 x₉ x8 n4 n6 SS = n1n2n3n5 x2x3x6 x4x7x8 n4 n₃ SS = n1n2n5n6 x2x3x6 x4x7x8 n4 n₃ SS = n1n2n5n6 x2x3x6 x4x7x8 n4 n₃ SS = n1n2n5n6 x9 x₈ n6 SS = n₁n₂n₃n₄n₅ x9 x₈ n6 SS = n₁n₂n₃n₄n₅ x9 x₈ n6 SS = n₁n₂n₃n₄n₅ x4 x1x3x5 x₇x₈ n4 n2 SS = n1n3n5n6 x4 x1x3x5 x₇x₈ n4 n2 SS = n1n3n5n6 x4 x1x3x5 x₇x₈ n4 n2 SS = n1n3n5n6 x1x3x4x5 x9 x₈ n2 n6 SS = n1n3n4n5 x1x3x4x5 x9 x₈ n2 n6 SS = n1n3n4n5 x1x3x4x5 x9 x₈ n2 n6 SS = n1n3n4n5 x4x7x8 n4 SS = n1n2n3n5n6 x4x7x8 n4 SS = n1n2n3n5n6 x4x7x8 n4 SS = n1n2n3n5n6 x2 x₂+x₃ x₄ x₅ x₁+x₃ x₆ x4 x5+x6 x2+x3+x6 x4+x7 x9 x₁+x₃+x₅ x₇ x₉ x4+x7 x9 # #

# represents an isomorphic graph. + represents a Boolean operation OR.

G1

G2 G3 G4

Boolean reliability expression BF(·), i.e. BF(G) represents the Boolean reliability expression of G. Therefore, we get the recursive relationship of BF(·) between graph G and its subgraph G*n as: )] ( ) * ( [ ) ( : i E i N n x n G BF G BF n ss ss  † … † (8)

where n is a node of G, Nss is the set of nodes that are

ad-jacently connected to SS, Ess:n is the set of edges that

con-nect n and SS, G*n is the subgraph of G obtained by merging adjacent node nNss into SS and deleting any

edge connecting n and SS, x /i xi is the Boolean edge

variable representing edge i is functional/failed (i.e. 1

/ 1 i

i x

x ), † is the Boolean OR operation BDD_or(), and … is the Boolean AND operation BDD_and(). For example, in Fig. 3, BF(G1)

>

@ >

@ >

@

3

2 ) ( ) ( )

(x †x † BF G …x † BF G …x

… . If BF(·) is

im-plemented by a BDD, then each graph has its own BDD and the manipulation technique of BDD in (2) could be used recursively for performing Boolean operations on edge variables during the merging procedure. Finally, by the bottom-up procedure, we construct the BDD of the top-root graph representing the Boolean reliability expres-sion of the DPR.

In addition, the CLK algorithm can avoid the redundant computations of isomorphic problems. An SS of a sub-graph implies a certain set of files can be obtained no mater how the nodes in SS are connected. During the merging procedure, the rest of edges outside SS in a sub-graph will not be changed. Therefore, the Boolean reliabil-ity expression representing the traversing path outward SS of the subgraph will not be changed. This means two sub-graphs derived from different merging procedures are isomorphic if their SS are the same since the follow-up traversing path for the rest of the required files will be the same. Therefore, the isomorphic subgraphs have the same Boolean reliability expression BF(·), i.e. the same BDD. We can use the property to avoid redundant computations.

According to the above idea, a hash table is built in the CLK algorithm to record these isomorphic subgraphs and their own BDD. When we traverse into an isomorphic subgraph, we can just retrieve the BDD belonging to the isomorphic subgraph from the hash table and use the BDD for Boolean operations in the merging procedure. There-fore, the redundant computations on isomorphic subgraphs can be avoided. This will save significant execution time.

The algorithm is depicted by the pseudo-code in Fig. 4 and is executed by initializing G to the original graph, SS to empty, and n to the node at which the program can be executed. The algorithm can recursively merge the nodes in the distributed network and build up the BDD that represents the Boolean reliability expression of a DPR. The hash table in the algorithm can avoid the redundant

computations from isomorphic sub-problems. Fig. 3 illus-trates the procedure for the evaluation of the DPR1 in Fig.

2. The diagram contains 8 terminal subgraphs, 6 interme-diate subgraphs, and 2 isomorphic subgraphs marked with #. The larger the scale of a distributed computing system becomes, the larger the number of isomorphic subgraphs is. Therefore, the scheme of using the hash table will bring a significant improvement on the efficiency of the CLK al-gorithm. By the manipulations of BBD_and() and BBD_or() during the merging procedure, the BDD repre-senting the disjoint terms of the Boolean reliability ex-pression of the DPR can be obtained from (8). Thus, the DPR can be recursively derived from the BDD by (4).

3.2. Ordering strategy

One of the key issues in getting full advantage out of BDD is to find a good variable-ordering. The size of BDD and therefore, the efficiency of the whole methodology, strongly depends on the chosen ordering. Unfortunately, the problem of determining the ordering is NP-Complete. However, approximate solutions are relatively easy to find by a local-search algorithm [18]. For fault trees, the order-ing obtained by a depth-first left-most traversal of the tree is often good enough. For terminal-pair reliability net-works, the ordering obtained by a breadth-first traversal of the network starting from its source is a good candidate [14][15][18].

In our experimental results, the following ordering heu-ristics is good for a distributed computing network. We order the variables by the ordering that we use for BDD

RecursiveMerage( G, SS, n ) {

BDD result, tmp1, tmp2; SS = SS + {n};

if ( SS gets all the required files) return BDD_one; if ( SS contains all the nodes) return BDD_zero; if ( SS gets a hit in the hash table ) { result = getBDD_computed_table(SS); } else { result = BDD_zero; for each node niadjacent to SS {

Sub_G = G * ni;

tmp1 = RecursiveMerage( Sub_G, SS, ni); tmp2 = BDD_or( all the edge connected to ni); tmp2 = BDD_and( tmp2, tmp1);

result = BDD_or( result, tmp2); }

insert_computed_table( SS, result); }

return result; }

Figure 4. The recursive merge algorithm for constructing the BDD with perfect nodes.

manipulation in the algorithm shown in Fig. 4 during building up the BDD, i.e. we put the variables in the or-dered list if we need a new variable to represent a compo-nent for BDD manipulation. Therefore, the prior ordering of the variables is used for the BDD manipulation in the algorithm and the prior ordering is in the BDD. For exam-ple, in Fig. 3, the ordering is as following:

8 1 3 2 6 5 9 7 4 x x x x x x x x x        

The final BDD representing the Boolean reliability ex-pression of DPR1 is obtained as shown in Fig. 5. In the

latter section of experimental results, we will make a comparison between the size of BDD generated with our ordering strategy and that with the breadth-first ordering strategy. The experiments show that the CLK algorithm has a great improvement on the size of BDD and is more efficient about 20% in execution time.

3.3. Algorithm for directed graph

The above algorithm is also applicable to directed graphs without modification. But care must be taken as which nodes can be merged to and which nodes can be absorbed into SS. In the directed graph, a node v can be merged into SS only if there is a nodeuSS such that <u, v> is an edge connecting from u to v. The other part of the algorithm for directed graph is the same as that for

undi-rected graph.

3.4. Algorithm for imperfect nodes

A simple but efficient method dealing with node fail-ures is proposed in [23] and can be embedded in any algo-rithm. That approach is based on a simple concept: “the failure of a node implies the failure of its incident edges”. Therefore, after deriving the BDD of a DCS with perfect nodes in Section 3.1, we can apply the BDD substitution technique on it to solve the problems of a DCS with im-perfect nodes. The BDD substitution technique is a useful property in BDD manipulation. The BDD derived in Sec-tion 3.1 consists of only edge variables. In order to reckon the effects of imperfect nodes, the node variables should be included. Because one edge connects two nodes, the Boolean edge variable in the BDD can be replaced by the Boolean edge variable and the other two Boolean node variables. Table 1 illustrates the rules of variable substitu-tion in our example. The substitusubstitu-tion approach in our al-gorithm can be implemented by BDD_and() and BDD_or() in the following:

where x /i xi is the Boolean edge variable representing

whether edge i is functional/failed (i.e. xi 1/xi 1),

i

i n

n / is the Boolean node variable representing node i functional/failed (i.e. ni 1/ni 1 ), IBF(G) is the BDD-based reliability function for a network G with im-perfect nodes, two end nodes j and k represented by nj and

nk respectively are connected by the edge i represented by

xi, and BF|xi 1(G)/BF|xi 0(G) is the BDD-based

reliabil-ity function for a network G with perfect nodes given that edge i is functional/failed. Therefore, we can evaluate the reliability of a DCS with imperfect nodes very efficiently. One thing we need to take care is the BDD ordering of the nodes and edges. Applying the ordering heuristic described in Section 3.2, the variable ordering for Fig. 3 becomes

8 1 1 2 3 2 3 6 5 5 9 6 7 4 4 x n x n x x n x x n x n x x n              

It should be noted that the number of Boolean manipu-lations in the procedure of network merging and BDD con-structing significantly affects the performance of the algorithm. Therefore, we first consider the problem case with perfect nodes to construct the BDD representing a DCS. We manipulate the Boolean operations with only edge variables but no node variables are included to reduce the number of BDD variables. The smaller the size of BDD variables is, the smaller the number of Boolean

ma-x7 x4 x6 x2 1 x1 0 x2 x3 x9 x5 x7 x5 x6 x2 x3 x9 x7 x4 x6 x2 1 x1 0 x2 x3 x9 x5 x7 x5 x6 x2 x3 x9

Figure 5. The BDD generated by the CLK algorithm for DPR1 in Fig. 2 with perfect nodes.

Table 1. Rules of Boolean variable substitution for Fig. 2.

The rules of Boolean variable substitution

x1m n1x1n2 x4m n2x4n4 x7m n4x7n5 x2m n1x2n3 x5m n2x5n5 x8m n4x8n6 x3m n2x3n3 x6m n3x6n5 x9m n5x9n6

nipulations is and thus the faster the algorithm can run. After deriving the BDD representing the Boolean reliabil-ity expression of a DCS with perfect nodes, we apply the BDD substitution technique to take the effect of imperfect nodes into consideration and then obtain the final BDD. This scheme reduces the number of Boolean manipula-tions during the network merging procedure and can save about 20% of the execution time in average.

4. Reliability evaluation of a distributed

program running at more than one node

dis-tributed program j can run. Then, by the Boolean OR op-eration BDD_or() with the BDDj:i for each node i, the

BDDj representing the Boolean reliability expression of

distributed program j can be derived as:

i j h i j BDD BDD j : 1 † (10)

where† represents the Boolean OR operation BDD_or(),

hj is the number of host nodes in the system where

distrib-uted program j can run, and BDDj:i is the BDD

represent-ing the Boolean reliability expression for successfully run-ning distributed program j at node i. Therefore, the DPRj

of distributed program j running at more than one site can be easily evaluated from the BDDj .

4.1. Distributed system reliability (DSR)

In the last section, we can get the BDD of a given dis-tributed program. For reliability analysis of two or more programs executed simultaneously, we use the Boolean AND operation BDD_and() with the BDDj for each

pro-gram j included in the system.

j jBDD

BDD … (11)

where … represents the Boolean AND operation BDD_and(). Similarly, the DSR of a distributed computing system can be derived by (11) and (4).

4.2.

K-terminal network reliability

The evaluation of k-terminal network reliability is a special case of the evaluation of DPR in a DCS. The only difference is that the k terminals are already identified an-tecedently for evaluating the k-terminal network reliability. Therefore, it is easy to use the algorithm for DCS to com-pute the k-terminal network reliability. However, it will be difficult to use the algorithm developed for evaluating the

k-terminal network reliability to compute a DPR in a DCS since the priori knowledge that the k nodes in a DCS should be antecedently identified is not given. For com-puting the reliability of a k-terminal network with perfect or imperfect nodes using the CLK algorithm, all we need to do is to distribute the different k required files on the k

nodes correspondingly, and empty the set of FA at the rest of nodes. Then, the DPR of such distribution of the re-source files in the DCS will be the k-terminal network reliability.

Similarly, the evaluation of terminal-pair network reli-ability is a special case of the evaluation of k-terminal network reliability where k = 2. For computing the reli-ability of a terminal-pair network with either perfect nodes or imperfect nodes, we locate a required file at the source node as well as another required file at the destination node, and empty the set of FA in the rest of nodes. There-fore, the DPR of such distribution of the resource files will be the terminal-pair network reliability.

5. Experimental results

The efficiency of the CLK algorithm is compared with the algorithms in [1][6][7][8][9][22]. The implementation of the CLK algorithm is done with the C/C++ language on a primary-type SUN UltraSPARC workstation. Although the running times were measured on different platforms, the capabilities of data processing are roughly equivalent. The reliabilities of all the edges and nodes are assumed to be 0.9. First, we compare the CLK algorithm with the al-gorithms in [6][7] that were developed based on the cut-set method. Fig. 6 illustrates a complex DCS with eight proc-essing elements in [6][7]. Table 2 shows a comparison between the CLK algorithm and the algorithms in [6][7]. Although they do not provide the execution time in [6][7], we can still compare the complexity by the number of subgraphs generated from different algorithms. The num-ber of subgraphs produced by the CLK algorithm is about 29.09% less than previous results. That means the CLK algorithm has higher computational efficiency. In addition, the CLK algorithm can avoid the redundant computation of isomorphic sub-problems. The high average hit ratio, about 58.85%, of the hash table also makes the CLK algo-rithm very efficient. Further, by the use of the BDD sub-stitution technique, we can efficiently compute the reli-ability of each individual distributed program even with imperfect nodes, while it will be difficult for the algo-rithms in [6][7].

Moreover, for considering the imperfect nodes, we use the benchmarks Gi:j given in [8][9], jd , where i i is the

number of nodes in the network, and j represents that nodes n1,n2, …, nj are fully connected in network G. For

program is located at n1, and files F1,F3,F5 are required to

execute it. Table 3 gives the location of the files. Table 4 shows the comparison of the number of subgraphs and the execution time among the CLK algorithm, KHR [1], ENR/KW [8], and the algorithm in [9]. Due to the conver-gence of isomorphic sub-problems, the CLK algorithm generates far less subgraphs than other algorithms, espe-cially for G10:9. Although Zang et al. [9] provides a method

to analyze the dependability of a DCS with imperfect fault-coverage using BDD, however, the BDD represent-ing the structure function of the DPR is generated via the traditional MFST-searching method.

The CLK algorithm employs the merging method and a hash table to record the isomorphic subgraphs and the corresponding BDDs. Hence the redundant computations of isomorphic subgraphs can be avoided by looking up in the hash table. The average hit rate is about 28.03%. The larger the scale of the DCS becomes, the higher the hit rate is. This makes the CLK algorithm much more efficient. In addition, we apply the BDD substitution technique only after the BDD representation of the DCS with perfect nodes has been constructed. This technique can reduce the time spent in BDD manipulation and therefore, improve the performance of the CLK algorithm.

To compute the terminal-pair network reliability (TPR) for a network with perfect or imperfect nodes, we distrib-ute a required file on the source node as well as another

required file on the destination node, and empty the set of

FA in the rest of nodes. Then, the DPR of such distribution of the resource files will be the TPR. Table 5 shows a com-parison of the CLK algorithm with TPR/NF [22] and ENR/KW [8] for evaluating the TPR with imperfect nodes. The average hit rate is about 55.92%. The results show a great improvement on the execution time, especially for benchmark G10:9. Node 1 FA1: F1F6 PRG1: P4 Node 3 FA3: F3F5 PRG3: P3 Node 2 FA2: F2 PRG2: Ø Node 5 FA5: F4F5 PRG5: Ø Node 6 FA6: F6 PRG6: P1 Node 8 FA8: F3F8 PRG8: Ø FN1= F1F2F5F7F8 FN2= F1F3F5F8 FN3= F1F2F4F6F8 FN4= F2F3F4F5F7F8 Node 4 FA4: F4F8 PRG4: Ø Node 7 FA7: F2F7 PRG7: P2 Node 1 FA1: F1F6 PRG1: P4 Node 1 FA1: F1F6 PRG1: P4 Node 3 FA3: F3F5 PRG3: P3 Node 3 FA3: F3F5 PRG3: P3 Node 2 FA2: F2 PRG2: Ø Node 2 FA2: F2 PRG2: Ø Node 5 FA5: F4F5 PRG5: Ø Node 5 FA5: F4F5 PRG5: Ø Node 6 FA6: F6 PRG6: P1 Node 6 FA6: F6 PRG6: P1 Node 8 FA8: F3F8 PRG8: Ø Node 8 FA8: F3F8 PRG8: Ø FN1= F1F2F5F7F8 FN2= F1F3F5F8 FN3= F1F2F4F6F8 FN4= F2F3F4F5F7F8 Node 4 FA4: F4F8 PRG4: Ø Node 4 FA4: F4F8 PRG4: Ø Node 7 FA7: F2F7 PRG7: P2 Node 7 FA7: F2F7 PRG7: P2

Figure 6. A complex DPS with eight processing elements.

Table 2. The comparison of the number of subgraphs generated from different algorithms.

FST_SPR [6] HRFST [7] CLK Distributed program DPR with perfect nodes # of subgraphs # of subgraphs # of subgraphs DPR with imperfect nodes # of SS hash hits Hit ratio (%) Execution Time in seconds P1 0.9961182 445 113 95 0.700269 165 63.46 0.08 P2 0.9963265 334 124 85 0.766953 107 55.73 0.05 P3 0.9984532 309 118 95 0.777947 141 59.75 0.07 P4 0.9963256 389 140 76 0.761795 89 53.94 0.06 n₄ n₁ n₂ n₃ n₅ n₆ n₇ n₈ n₄ n₁ n₂ n₃ n₅ n₆ n₇ n₈

Figure 7. Benchmark network G8:6.

Table 3. File distribution.

Node Files Node Files

n1 F1F2F3 n6 F6F7F8

n2 F2F3F4 n7 F1F7F8

n3 F3F4F5 n8 F1F2F8

n4 F4F5F6 n9 F3F7F8

Table 6 shows a comparison of the sizes of BDD and the execution times with different ordering strategies. CLK_BFS represents the CLK algorithm with breadth-first searching ordering strategy. CLK_ISO represents the CLK algorithm with the ordering strategy as described in Sec-tion 3.2. The result shows that a breadth-first vari-able-ordering obtained in [14][15][18] is not good enough for distributed computing networks. However, the heuristic ordering strategy depicted in Section 3.2 is a good candi-date for DCS, especially in G10:9.

6. Conclusions

In this paper, we have proposed an efficient algorithm for evaluating the reliability of a distributed computing

system. The CLK algorithm can avoid redundant computa-tions during the evaluation by converging isomorphic sub problems. The experimental results show that the CLK al-gorithm has a great improvement on the execution time compared to the previous works. Based on BDD, the CLK algorithm has the capability to support a large amount of Boolean manipulations and therefore, is applicable to large-scale distributed computing networks. In addition, using the BDD substitution technique, the CLK algorithm can be efficiently applied to a DCS network with imperfect nodes. Moreover, the BDD-based methodologies in [19] for the dependability analysis of systems can be integrated with the CLK algorithm. Based on this approach, re-searches on sensitivity analysis, importance measures, failure frequency analysis or optimal design issues of

dis-Table 4. The comparison of different algorithms for evaluating DPR.

# of subgraphs Time in seconds # of SS Hit ratio

Network KHR [1] ENR/KW [8] CLK KHR [1] ENR/KW [8] ZST [9] CLK hash hits (%) DPR

G8:4 37 16 25 0.030 0.005 0.05 0.02 3 10.71 0.891551 G10:4 55 20 37 0.037 0.014 0.05 0.04 5 11.90 0.889355 G8:6 306 72 55 0.412 0.015 0.06 0.05 12 17.91 0.898896 G8:7 1159 289 63 4.280 0.055 0.07 0.06 17 21.25 0.899061 G10:7 3443 462 175 9.850 0.148 0.08 0.11 65 27.08 0.899057 G8:8 3225 1196 63 36.300 0.222 0.15 0.09 17 21.25 0.899090 G10:8 20464 2556 223 230.000 0.817 0.25 0.21 101 31.17 0.899092 G10:9 131899 17832 255 8000.000 5.640 1.83 0.42 129 33.59 0.899099

Table 5. The comparison of different algorithms for evaluating TPR.

# of subgraphs Time in seconds # of SS Hit ratio Network TPR/NF [22] ENR/KW [8] CLK TPR/NF [22] ENR/KW [8] CLK hash hits (%) TPR

G8:4 157 48 39 0.039 0.012 0.03 17 30.36 0.806942 G10:4 365 226 75 0.157 0.065 0.05 65 46.43 0.701621 G8:6 2823 1839 111 2.083 0.438 0.09 101 47.64 0.809823 G8:7 33085 11883 127 26.000 2.900 0.23 129 50.39 0.809967 G10:7 36513 27077 351 40.700 10.100 0.49 433 55.23 0.809964 G8:8 383563 69918 127 230.000 18.800 0.84 129 50.39 0.809992 G10:8 553431 234847 447 663.000 100.000 1.69 625 58.30 0.809993 G10:9 10405589 2245128 511 13300.000 990.000 16.03 769 60.08 0.809999

Table 6. Different ordering strategies of CLK.

# of Nodes in BDD Time in seconds

Network CLK_BFS CLK_ISO CLK_BFS CLK_ISO

G8:4 56 28 0.03 0.02 G10:4 60 36 0.04 0.04 G8:6 416 74 0.07 0.05 G8:7 1420 164 0.09 0.06 G10:7 1424 192 0.17 0.11 G8:8 3237 445 0.14 0.09 G10:8 5327 535 0.39 0.21 G10:9 22240 2177 1.10 0.42

tributed processing systems will be the focus of our future works.

References

[1] V.K.P. Kumar, S. Hariri, and C.S. Raghavendra, “Distrib-uted Program reliability analysis”, IEEE Trans. Software Engi-neering, vol. 12, pp. 42-50, Jan. 1986.

[2] C.S. Raghavendra, V.K.P. Kumar, and S. Hariri, “Reliability analysis in distributed systems”, IEEE Trans. Computers, vol. 37, no. 3, pp. 352-358, Mar. 1988.

[3] A. Kumar, S. Rai, and D.P. Agrawal, “On computer com-munication network reliability under program execution con-straints”, IEEE Journal of Selected Areas in Communications, vol. 6, pp. 1393-1400, Oct. 1988.

[4] S. Hariri and C.S. Ragavendra, “SYREL: A symbolic reli-ability algorithm based on path and cut set methods”, IEEE Trans. Computers, vol. 36, pp. 1224-1232, Oct. 1987.

[5] A. Kumar and D.P. Agrawal, “A generalized algorithm for evaluating distributed-program reliability”, IEEE Trans. Reliabil-ity, vol. 42, no. 3, pp. 416-426, Sept. 1993.

[6] D.J. Chen and T.H. Huang, “Reliability analysis of distrib-uted systems based on a fast reliability algorithm”, IEEE Trans. Parallel and Distributed Systems, vol. 3, no. 2, pp. 139-154, Mar. 1992.

[7] D.J. Chen, R.S. Chen, and T.H. Huang, “A heuristic ap-proach to generating file spanning trees for reliability analysis of distributed computing systems”, Computers and Mathematics with Application, vol. 34, pp. 115-131, Nov. 1997.

[8] W.J. Ke and S.D. Wang “Reliability evaluation for distrib-uted computing networks with imperfect nodes”, IEEE Trans. Reliability, vol. 46, no. 3, pp. 342-349, Sept. 1997.

[9] X. Zang, H. Sun, and K.S. Trivedi, “Dependability analysis of distributed computer systems with imperfect coverage”, Proc. 29th Ann. Int’l Conf. Fault-Tolerant Computing, (FTCS-29), 1999, pp. 330-337.

[10] M.S. Lin, M.S. Chang, and D.J. Chen, “Efficient algorithms for reliability analysis of distributed computing systems”, Infor-mation Sciences, vol. 117, pp. 89-106, July 1999.

[11] C.D. Lai, M. Xie, K.L. Poh, Y.S. Dai, and P. Yang, “A model for availability analysis of distributed software/hardware systems”, Information and Software Technology, vol. 44, no. 6, pp. 343-350, 2002.

[12] Y.S. Dai, M. Xie, and K.L. Poh, “Reliability analysis of grid computing systems”, Proc. 2002 Pacific Rim Int’l Symp. De-pendable Computing (PRDC’02), 2002, pp. 97-104.

[13] R.E. Bryant, “Graph-based algorithms for Boolean function manipulation”, IEEE Trans. Computers, vol. 35, pp. 677-691, Aug. 1986.

[14] S.Y. Kuo, S.K. Lu, and F.M. Yeh, “Determining termi-nal-pair reliability based on edge expansion diagrams using OBDD”, IEEE Trans. Reliability, vol. 48, no. 3, pp. 234-246, Sept. 1999.

[15] F.M. Yeh, S.K. Lu, and S.Y. Kuo, “OBDD-based evaluation of k-terminal network reliability”, IEEE Trans. Reliability, vol. 51, no. 4, pp. 443-451, Dec. 2002.

[16] S. Minato, “Streaming BDD manipulation”, IEEE Trans. Computers, vol. 51, no. 5, pp. 474-485, May 2002.

[17] Hung-Yau Lin, Sy-Yen Kuo, and Fu-Min Yeh, “Minimal cutset enumeration and network reliability evaluation by recur-sive merge and BDD”, Proc. 8th IEEE Symp. Computers and Communications (ISCC’03), 2003, pp. 1341-1346.

[18] A. Rauzy, “A new methodology to handle Boolean models with loops”, IEEE Trans. Reliability, vol. 52, no. 1, pp. 96-105, Mar. 2003.

[19] Y.R. Chang, S.V. Amari, and S.Y. Kuo, “Computing sys-tem failure frequencies and reliability importance measures using OBDD,” IEEE Trans. Computers, Jan. 2004, (accepted).

[20] A. Rauzy, “New algorithms for fault tree analysis”, Reli-ability Engineering and System Safety, vol. 40, pp. 203-211, 1993.

[21] R.M. Sinnamon, J.D. Andrews, “Improved efficiency in qualitative fault tree analysis”, Quality and Reliability Engineer-ing Int’l, vol. 13, pp. 293-298, 1997.

[22] V.A. Netes and B.P. Filin, “Consideration of node failures in network-reliability calculation”, IEEE Trans. Reliability, vol. 45, no. 1, pp. 127-128, Mar. 1996.

[23] K.K Aggarwal, J.S. Gupta, and K.B. Misra, “A simple method for evaluation of a communication system,” IEEE Trans. Communications, vol. 23, pp. 563-566, May 1975.