Neural networks for optimization problems in graph theory

269

Neural Networks for Optimization Problems in

Graph Theory

Jenn-Shiang Lai and Sy-Yen Kuo

Dept. of Electrical Engineering

National Taiwan University

Taipei,

Taiwan,

R.O.C.

Abstract

Thir paper presenti a novel technique to map the minimum vertex cover and related problems onto the Hopfdd neural networks. The proposed approach can be w e d to find near-optimum solutions for these prob- lem in parallel, and particularly the network algorithm alwayr yields minimal vertex covers. Further, the re- latiomhipa between Boolean equations and arithmetic functions arc presented. Based on these relationship

I,

other NP-complete problems in graph theory can al- so be solved by neural networks. Extensive simulation wan performed and the experimental results demonstrate that the network algorithm outperforms the well-known greedy algorithm for the vertex cover problem,

1

Introduction

The vertex cover problem is to find the smallest set of vertices that covers all the edges in a given graph. This is

a very practical problem [l]. Since this problem is

NP-

complete, several sequential approximation algorithms were propowd [2]. However, these algorithm make de- cirrionr b a e d solely on local information, and they may fail in many situations [l, 31. In this paper, we propose an algorithm based on the neural network to solve this problem by Hopfield d e l [4].

"he rest of thin paper is organised an follows. In Sec- tion 2, we describe the technique to solve the vertex cover problem by the Hopfield network. Sections 3 and 4 de- rive approaches mlving the maximum independent set and maximum clique problem, respectively. Section 5

ahom experimental results for the vertex cover problem. Findly, conclunions are given in Section 6.

2

Vertex Cover Problem

Let G

= (N,

A) be an undirected graph, where

N

is the met of vertices, A is the set of edges, and

IN1

denote the number of verticer. The vertex cover problem is a

problem of finding the smallest subset C

E N

such that for each edge [i, j] E A at least one of a and j belongs

Ing-Yi Chen

Dept.

of

Electronic Engineering

Chung Yuan Christian University

Chungli, Taiwan, R.O.C.

to

C

[5]. Since the goal of this problem is to cover al- l edges of G with as few vertices as possible, selecting each time the single vertex that by itself covers an many of the remaining edges as possible is an attractive strat- egy. Henceforth, the well-known greedy algorithm for thie problem is described as follows: successively select the vertex of largest degree (i.e., adjacent to the largest number of edges) and remove this vertex together with

all of its adjacent edges from the graph until all edges have been removed. Although, the removed set of ver- tices is expected to be a minimum cover, it ia easy to find some situations where this approximation fails to yield a minimum cover. Consider how this scheme be applied to graph of Figure 1 [3]. We first choose vertex I, and then 11, I I I , I V , and V. The resulting vertex cover consists of 5 vertices. But the optimumvertex cov- er, (11, 111, IV, V}, has just 4 vertices. This is because vertex I becomca redundant after selecting vertices I I , I I I , IV and V, but this sequential algorithm can't re- move any redundant vertex in a cover.

Figure 1: An example for the vertex cover problem. For every undirected graph G =

(N,

A), one can find a Hopfield model such that there is a one to one corre- spondence between the minima of the network and the minimal vertex covers of that graph. Let C be a vertex cover and the state V; of neuron i be determined by

if vertex i is in the cover C otherwise

Hence, if (cqj) is the adjacency matrix of graph G, the vertex cover problem can be mathematically stated as

270

Logic

NOT

X

X A N D Y X O R Y

finding the minimum of the following cost function

Boolean Arithmetic

X

1-x

X A Y XY

X V Y X + Y - X Y

-

where V is the logical OR,

x

means the complement of

X ,

and 0

<

7

<

1.

The first expression in bracket goes t o 0 when all edgca are covered by C, and the second bracketed expression is used to minimise the number of vertices 'in C.

As shown in Table 1, the Boolean equation of log- ical variabk can be represented by its corresponding arithmetic function and therefore, the former objective function can be expanded and rearranged aa

E~ =

- C C R j ~ v y + C r ~

1 (since*i=o)

i j#i j

This is in the form of the above Lyapunov function. Hence we can obtain an algorithm bssed on the Hop field network with the external input I j =

ci

taij - 7 to neuron j, and the connection weight wij = -.ij between

the i-th and the j-th neurons.

If the initial states of all neurons are set to be randomly-generated values around 0.5 (say, 0.5 f 0.05),

and the continuous model of the Hopfield network is ap- plied, this neural network approach will stabilize into a minimal vertex cover in parallel. On the other hand, let the state of every neuron be zero initially, by using the faet gradient-descent technique for the discrete Hop field model (i.e., by sequentially updating the state of a neuron which can reduce the greatest amount of sys- tem energy), this network method is converged to find a cover with minimal number of vertices at all times.

Indeed, the fast gradient-descent network method works like the traditional greedy algorithm, 80 it is

a (lnn)-approximation algorithm [l]. Notice that the greedy algorithm can be considered aa the fast gradient- descent algorithm with -y = 0; therefore, if the opti- mum solution consists of n vertices, the minimal ver- tex cover obtained by this algorithm may grow as fast

ae (n

+

n In n). In practice, this sequential network can obtain the minimum cover of the graph in Figure 1, but this approach is very hard to find the optimum solution for the graph in Figure 2. To avoid this situation, the

Figure 2: A 10-vertex l2-edge graph.

discrete network algorithm in adjusted to emulate the continuous one, and the objective function in modified into

r i r 1

where

In this way, when two vertices have the same degree in the remaining graph, the one with higher original degree will be selected (since it can cover more edges and may result in more redundancies being removed). Also, the external input to the j-th neuron should become

and the worst cases in Figure 2 can be elegantly solved. Likewise, the first expression in bracket goes to 0 when all edges in G are covered, so the energy function of a minimal vertex cover C can be deduced into

r

and the larger the minimal vertex cover is, the higher its energy will be (see the following Lemma).

27 1

Lemma. Let the energy function of a minimal cover C

of a graph G

= (N,

A) be ddined am

After removing the edges not incident with C; or C; and thore between C; and C;, we can obtain a graph

H

without changing E c ,

-

Ec, (see Figure 3). In this way, any edge in

H

i incident with a vertex in C1 n Ca and another vertex in Ci or C;. Hence,

0 <_ deg(i) 5 1C;Itcl (j = 1,2) i€C,- and d e d i )

-

C

deg(j)

I

I ~ J ( l ~ 1 -

IC,l-

I G I )

i€C; j€C; Therefore, 0

3

Maximum Independent Set Problem

In this section, we are to derive a similar technique for solving the maximum independent set problem.

Given a graph G = ( N , A ) , the aim of the maximum independent set problem is to find the largest set S of vertices such that no two vertices in S are connected by

an edge. It is well-known that if S is an independent set of G,

N

- S is a vertex cover of G [5]. Further,

N

- S is the optimal solution to the vertex cover problem iff S is the maximal independent set of G. So, the objective function of the maximum independent set problem can be formulated as

r i r 1

9 3 9 3

6

0

Figure 3: (a) A graph G with C1

=

{5,6,7,.

.

.,

11) and

C2

=

{2,3,4,.

.

.,

7). (b) The corresponding graph

H

without changing Ecl

-

Ec,.

and we can have the neural network algorithm to oolve the h u mindependent set problem.

4

Maximum Clique Problem

By definition, for a graph G

=

(NI

A) a maximal c o m plete subgraph is called a clique, and the complement of the graph G is the graph by deleting the edges of G from the complete graph on the same vertices. Accord- ingly, if (aij) and

(kj)

are the adjacency matricea of G and

E,

respectively, we can have bij

=

1

-

qj for i

#

j , and bii

=

.ii

=

0 for all i. Since the independent sets and cliques have the following relationships

1.

X

L a clique of G.

2.

K

is an independent set of

5.

it is easy to get the objective function of the maximum clique problem 8s

and we can obtain a similar algorithm for the maximum clique problem.

5

Experimental Results

The algorithm are implemented in C on Sun SPARC- station 2 for graphs with edge density 5%,

lo%,

25%,

272 Sises(05%) 10 20 30 40 50 Sises(lO%) 10 20 30 40 50

and 50%. Here the edge density is the probability that an edge exists between a pair of vertices [3]. For each sise, ten random graphs are examined and all algorithm-

s were run for 25 times to find the optimum solution of every instance. In this simulation, all initial states for diacrete techniques, such u the greedy algorithm in [2] or the f a d gradient-descent networka (7 = 1/2 or 7 = (1

+

&j)/lNla), are reroe. In the continu- ous model, 7 U eet to 1/3. Table 2 shows the proba-

bilities of finding the minimum vertex covers by these algorithm. Here the sequential (greedy) algorithm is labelled as SA, the fast gradient-descent networks with and FGDN2, reapectively, and the continuous Hopfield approach with 7 = 1/3 in named CHA. Compared with the sequential algorithm, both gradient-descent network- s have higher probabilities to find the optimum solutions in most cues.

7

=

0.5 and 7 =

(Eie,

&j)/lNIa re denoted by FGDNl

SA FGDNl FGDN2 CHA 0.9 0.912 1.0 0.996 0.848 0.828 0.9 0.892 0.568 0.648 0.584 0.608 0.48 0.58 0.688 0.74 0.36 0.396 0.452 0.404 SA FGDNl FGDN2 CHA 0.98 0.98 1.0 0.98 0.56 0.66 0.588 0.488 0.552 0.604 0.664 0.556 0.508 0.572 0.72 0.744 0.288 0.372 0.548 0.428 Sises(25%) 10 SA FGDNl FGDN2 CHA 0.904 0.996 1.0 0.976 20 30 40 50 Sises(50%) 10 20 30 40 50 6 0.752 0.832 0.884 0.76 0.552 0.66 0.772 0.748 0.448 0.536 0.508 0.496 0.404 0.476 0.512 0.48 SA FGDNl FGDN2 CHA 0.936 0.948 0.916 0.908 0.904 0.924 0.9 0.968 0.856 0.884 1.0 0.828 0.796 0.848 0.9 0.952 0.832 0.856 0.868 0.764

Conclusions

In this paper, we have presented a novel method to derive the minimum vertex cover and its companions (maximum independent set and maximum clique prob- lems) by neural networks. The proposed approach can

be used to find good solutions for vertex cover problem in parallel, and the neural networks alwaya converge t c irredundant vertex covers of the given graphs.

The relationship between Boolean equations a n c

arithmetic functions wan also proposed. In addition tc the problem discussed in this article, other NP-completc problem in graph theory can also be mapped onto thc Hopfield neural networks with the same method. For in stance, the bipartite subgraph problem and the graph partitioning of an %vertex graph [6, 71.

A large number of simulations wan performed to e.

valuate and justify our algorithm. Experimental resultt show that the performance of our method is better thsl that of the well-known sequential greedy algorithm, anc due to the inherent properties of Hopfield neural net works, this algorithm is suitable for massively par& execution. Moreover, with the advances in VLSI tech nology, large-scale neural networks may become a v d . able and this method will provide a significant advantsgc over others.

References

[l] C. H. Papadimitriou, and K. Steiglitr, "Combina- torial Optimbation: Algorithms and Complexity," Prentice-Hall, Englewood Cliffs,

N.

J., pp. 358-409, 1982.

[2] D. Johnson, "Approximation algorithm for com- binatorial problems," J. Comput. Syst. Sci., Vol. 9, 1974.

[3] Y. Peng, J. A. Reggia, and T. Li, "A connectionist approach to vertex covering problems," Int. J. Neurad

Syd., Vol. 3, NO. I, pp. 43-56, 1992.

[4] J. J. Hopfield, "Neural networks and physical system with convergent collective computational properties," Proc. Nut. Acad. Sci. U.S., Vol. 79, pp. 2554-2558, 1982.

[5] M. R. Garey and

D.

S. Johnson, "Computers and Intractability: A Guide to the Theory of

NP-

completeness," San Francisco: Freeman, 1979.

[e]

J. S. Lai and S. Y. Kuo, "A Hopfield network alg+ rithm for the bipartite subgraph problem", 3rd Inter- national Conference for Young Computer Scientists, July, 1993.

[7] D. E. Van den Bout and T. K. Miller, 111, uGraph partitioning using annealed neural networks," IEEE

Z h n s . on Neural Network, Vol. 1, No. 2, pp. 192- 203, June 1990.