Approximation Algorithms (Part I)

Approximation Algorithms (Part I)

P : an optimization problem I : an instance of P

A : an algorithm for solving P

V(I) : the value for I that is obtained by A V*(I) : the optimal value for I

A is an approximation algorithm for P, if for each I, A can generate a feasible solution with V(I) close to V*(I).

•

•

•

• Classification

Suppose that A is an approximation algorithm for an optimization problem P.

♣♣♣

♣ A is an absolute approximation algorithm for P if and only if for each I,

|V*(I)−−−−V(I)| ≤≤≤≤ c, where c is a constant.

♣♣

♣♣ A is an h(n)-approximate algorithm for P if and only if for each I,

V * I V I V * I

( ) ( ) ( )

−

−

−

−

≤≤≤≤ h(n), where n is the size of I.

♣♣♣

♣ A is a c-approximate algorithm for P if and only if for each I,

V * I V I V * I

( ) ( ) ( )

−−

−−

≤≤≤≤ c, where c is a constant.

Notice that ^{V * I V I} V * I

( ) ( ) ( )

−−

−−

≤

≤

≤≤ 1, if P is a maximization problem. Hence, c<1 is required for maximization problems.

♣

♣♣

♣ A is an approximation scheme for P if and only if for each given εεεε_>0 and for each I, A can generate a feasible solution with

V * I( )⁻⁻⁻⁻V I( )

≤

≤

≤

≤ εεεε.

♣♣

♣♣ An approximation scheme is referred to as a polynomial time approximation scheme if and only if its time complexity is polynomial to the size of I.

♣♣

♣♣ A polynomial time approximation scheme is further referred to as a fully polynomial time

approximation scheme if and only if its time complexity is also polynomial to ¹

ε εε ε .

For some NP-hard problems, designing efficient and “accurate” approximation algorithms is as hard as designing efficient exact algorithms.

Ex. 0/1 Knapsack

Instance: A finite set U = {u₁, u₂, …, u_n}, a “size” s(u_i)∈∈∈∈Z⁺ and a “value”

v(u_i)∈∈∈∈Z⁺ for each u_i ∈∈∈∈U, and a size constraint b∈∈∈∈Z⁺.

Question: What is the subset U’⊆⊆⊆⊆U such that

i

u U' i

∑

∈

∈

∈

∈

≤

≤

≤≤ b and

i

u U' i

∑

∈

∈

∈

∈

is maximized?

There is an approximation algorithm as follows: examine the elements u_i in nonincreasing order of ⁱ

i

v u s u

( )

( ) and add u_i to U’ if feasible.

For the instance of U={u , u }, v(u )=100,

Consider the following instance: U={u₁, u₂},

v(u₁)=2, v(u₂)=r, s(u₁)=1, s(u₂)=r, and b=r,

where r>2.

⇒

⇒⇒

⇒ U’={u₁}, V(I)=2, and V*(I)=r.

This algorithm is not an absolute approximation algorithm, because |V*(I)−−−−V(I)|=r−−−−2 is not a constant.

This algorithm is a 1-approximate algorithm, because ^{V * I V I}

V * I ( ) ( )

( )

−

−−

− =

− r²

1 ^≤≤ 1. ^≤≤

This algorithm is not a c-approximate algorithm for any c<1.

•

•

•

• Absolute Approximation

Absolute approximation algorithms are the most desirable approximation algorithms.

However, there are very few NP-hard problems whose polynomial-time absolute approximation algorithms are available.

In particular, designing polynomial-time absolute approximation algorithms for some NP-hard problems was shown NP-hard.

Ex. Consider the problem of finding the minimum number d of colors needed to color a planar graph G=(V, E).

Determining whether d=3 is NP-hard, if G is not bipartite and neither V nor E is empty.

Since every planar graph is 4-colorable, an easy absolute approximation algorithm with

|V*(I)−−−−V(I)|≤≤≤≤1 is as follows.

Step 1. Return d=0 if V is empty.

Step 2. Return d=1 if E is empty.

Step 3. Return d=2 if G is bipartite.

Step 4. Return d=4.

The time complexity of the algorithm is dominated by Step 3, which takes O(|V|+|E|) time.

Ex. Consider the following NP-hard problem, which is a restricted subproblem of the well known bin packing problem.

“Given two disks, each of capacity c, and n programs with storage requirements l₁, l₂, …, l_n, respectively, determine the maximum number of programs that can be stored in the two disks (no program stored in two disks).”

There is an O(nlogn) time absolute approximation algorithm with |V*(I)−−−−V(I)|≤≤≤≤1 as follows

Step 1. Arrange the programs in a nondecreasing

For example, if n=4, (l₁, l₂, l₃, l₄)=(2, 4, 5, 6), and c=10, then the first two programs are stored in the first disk and the third program is stored in the second disk.

For the example, an optimal solution is to store the first and third (or fourth) disks in one disk and the other two in the other disk.

♣♣♣

♣ Proof of |V*(I)−−−−V(I)|≤≤≤≤1

Let p be the maximum number of programs stored in a disk of capacity 2c, which happens when programs are stored in a nondecreasing order of l_i’s.

⇒ V*(I) ≤≤≤≤ p

Assume that the p programs stored in the disk have storage requirements l₁ ≤≤≤≤l₂ ≤≤≤≤ … ≤≤≤≤l_p, and p’ is the greatest index with

p' i i

l

=

∑

1

≤

≤≤

≤ c.

Since

p

i i p' +

l

−

=

∑

1

≤

≤≤

≤

p i i p' +

l

=

∑

2

≤

≤

≤≤ c, we have V(I) ≥≥≥≥ p−−−−1

(i.e., store the first p’ programs in the first disk and the (p’+1)th to (p−−−−1)th programs to the second disk).

Therefore, |V*(I)−−−−V(I)| ≤≤≤≤ 1.

When k≥≥≥≥2 disks are used, |V*(I)−−−−V(I)| ≤≤≤≤ k−−−−1 can be proved similarly.

•

•

•

• NP-Hardness of Absolute Approximation

Ex. 0/1 Knapsack

Instance: A finite set U = {u₁, u₂, …, u_n}, a “size” s(u_i)∈∈∈∈Z⁺ and a “value”

v(u_i)∈∈∈∈Z⁺ for each u_i ∈∈∈∈U, and a size constraint b∈∈∈∈Z⁺.

Question: What is the subset U’⊆⊆⊆⊆U such that

i

u U' i

∑

∈

∈

∈

∈

≤

≤

≤≤ b and

i

u U' i

∑

∈

∈

∈

∈

is maximized?

Π Π Π

Π: the problem of designing a polynomial-time absolute approximation algorithm for 0/1 Knapsack.

We show below that ΠΠΠΠ is NP-hard.

It suffices to show that if there exists a polynomial- time absolute approximation algorithm for 0/1 Knapsack, then 0/1 Knapsack can be solved in polynomial time (i.e., 0/1 Knapsack ∝∝∝∝ ΠΠΠΠ).

Suppose that A is a polynomial-time absolute approximation algorithm for 0/1 Knapsack with

|V*(I)−−−−V(I)|≤≤≤≤k, where k is a constant.

Let I be any instance of 0/1 Knapsack, and I% be the instance of 0/1 Knapsack that is obtained by multiplying each “value” (i.e., v(u_i)) of I by k+1.

⇒⇒⇒

⇒ (1) |V*( I% )−−−−V(I% )| is a multiple of k+1.

(2) I and I% have the same optimal solution

When applying A to I% , we have

|V*( I% )−−−−V(I% )|≤≤≤≤k,

which together with (1) assures V(I% )=V*(I% ).

⇒⇒⇒

⇒ A can generate an optimal solution for I%

(and hence an optimal solution for I).

For example, consider I as follows:

U={u₁, u₂, u₃}, (v(u₁), v(u₂), v(u₃))=(1, 2, 3), (s(u₁), s(u₂), s(u₃))=(50, 60, 30), and b=100, for which U’={u₂, u₃} is the optimal solution and V*(I)=5.

When I% changes (v(u₁), v(u₂), v(u₃)) to (1××××5, 2××××5, 3××××5)=(5, 10, 15), the optimal U’ remains the same, but V*( I% )=5××××5=25.

If A can guarantee |V*(I% )−−−−V(I% )|≤≤≤≤4, then A would compute V(I% )=25 and output U’= {u₂, u₃} for I% , which is also optimal for I.

Ex. Clique

Instance: An undirected graph G=(V, E).

Question: What is the size of a maximum clique of G?

Π Π Π

Π: the problem of designing a polynomial-time absolute approximation algorithm for Clique.

In order to show that ΠΠΠΠ is NP-hard, it suffices to show that if there exists a polynomial-time absolute approximation algorithm A for Clique, then Clique can be solved in polynomial time.

Assume that A guarantees |V*(I)−−−−V(I)|≤≤≤≤k for each instance I of Clique, where k is a constant.

Let I denote any instance G=(V, E) of Clique, and I’ denote the instance G’=(V’, E’) of Clique, where G’ contains k+1 copies of G and every two vertices in distinct copies are connected by an edge.

For example, when k=1, the following graph G

1 2 3 4

will induce G’ as follows.

1 2 3 4

1' 2' 3' 4'

⇒

⇒⇒

⇒ The maximum clique in G has size 3 and the maximum clique in G’ has size 2××××3=6.

In general, G has a clique of size q if and only if G’ has a clique of size (k+1)××××q.

When applying A to G’, we have

|V*(I’)−−−−V(I’)|≤≤≤≤k.

Since |V*(I)−−−−V(I)| is a multiple of k+1,

V(I’)=V*(I’) is implied, i.e., A can generate an optimal solution for I’ (and hence for I).

•

•

•

• h(n)-Approximation

Ex. Given m identical processors, denoted by P_i (1≤≤≤≤i≤≤≤≤m), and n jobs, denoted by J_k (1≤≤≤≤k≤≤≤≤n), a schedule is to assign each job with a time interval and a processor for processing.

Each job is not allowed to be processed by more than one processor at the same time.

A schedule is nonpreemptive if each job is required to be processed continuously from start to end by the same processor, and preemptive else.

t_k : the amount of processing time required for J_k;

F_i : the time when P_i completes the processing of all the jobs assigned to it.

It is an NP-hard problem to find a nonpreemptive schedule that can minimize max{F_i | 1≤≤≤≤i≤≤≤≤m}.

An LPT (longest processing time) schedule is to assign a free job with the longest processing time to a processor whenever it becomes available.

For example, the following is an LPT schedule for m=3, n=6, and (t₁, t₂, t₃, t₄, t₅, t₆)=(8, 7, 6, 5, 4, 3).

6 7 8 11

J1

P1

J2

J3 J4

J6

J5

P2

P3

time

⇒

⇒⇒

⇒ max{F1, F₂, F₃}=11, which is minimized.

An LPT schedule for m=3, n=7, and (t₁, t₂, t₃, t₄, t₅, t₆, t₇)=(5, 5, 4, 4, 3, 3, 3) is shown below.

4 5 8 11

J1

P1

J2

J3 J4

J6

J5

P2

P3

time J7

0

This schedule has max{F₁, F₂, F₃}=11, which is not optimal. An optimal schedule is shown as follow.

5 9

J1

P1

J2

J3

J4

J6

J5

P2

P3

time J7

0

For this instance, we have V * I V I

V * I ( ) ( )

( )

−

−

−

− = (11−−−−9)/9 = 2/9.

It takes O(nlogn) time to generate an LPT schedule, which can guarantee

V * I V I V * I

( ) ( ) ( )

−

−

−

− ≤≤≤≤

− m

1 1

3 3 .

The upper bound is tight, because it equals ² 9 as m=3 (refer to the instance above).

♣

♣

♣♣ Proof of ^{V * I V I} V * I

( ) ( ) ( )

−

−

−

− ≤≤≤≤

− m

1 1

3 3

When m=1, the inequality holds (V*(I)=V(I)).

So, assume m≥≥≥≥2 below.

Suppose that the inequality is violated for some m and (t₁, t₂, …, t_n). Besides, it is assumed that n is minimum, while violating the inequality.

f_k : the time when the processing of J_k is finished.

⇒

⇒⇒

⇒ max{fi | 1≤≤≤≤i≤≤≤≤n} = max{F_i | 1≤≤≤≤i≤≤≤≤m}.

♦♦♦

♦ We first show f_n =max{f_i | 1≤≤≤≤i≤≤≤≤n} below.

Suppose f_n’ =max{f_i | 1≤≤≤≤i≤≤≤≤n}, where n’<n.

⇒

⇒⇒

⇒ V(I’)=f_n’ =max{F_i | 1≤≤≤≤i≤≤≤≤m}=V(I),

where I’ denotes the instance of (t₁, t₂, …, t_n’) and I denotes the instance of (t₁, t₂, …, t_n).

Since V*(I’)≤≤≤≤V*(I), we have V * I' V I'

V * I'

( ) ( ) ( )

−

−

−

− ≥≥≥≥ ^{V * I V I} V * I

( ) ( ) ( )

−

−−

− (>

− m

1 1

3 3 ), which contradicts the assumption of n.

♦♦♦

♦ Next we show V*(I)<3××××t_n below,

meaning that at most two jobs are assigned to each processor in an optimal schedule for I.

Since f_n =V(I), the processing of J_n starts at time V(I)−−−−t_n, i.e., all processors are busy between time 0 and time V(I)−−−−t_n.

⇒

⇒⇒

⇒ V(I)−−−−t_n ≤≤≤≤ ⁿ _i m i

∑

=1

1 ××××

⇒ V(I) ≤≤≤≤ ⁿ _i m i

∑

=1

1 ×××× + ^m _n m^−1××××t

⇒ V(I) ≤≤≤≤ V*(I)+ ^m _n

m^−1××××t ( ⁿ _i m i

∑

=1

1 ×××× ≤≤≤≤ V*(I))

⇒ ^{V * I V I} V * I

( ) ( ) ( )

−−

−−

= ^{V I} V * I

( )

1⁻⁻⁻⁻ ( ) ≤≤≤≤ ^{( )}

* ( ) m n

m V I

t

−1 ^××××

×

×

×

×

Now that ^{V * I V I} V * I

( ) ( ) ( )

−−−

− >

− m

1 1

3 3 , we have

− m

1 1

3 3 < ^{( )}

* ( ) m n

m V I

t

−1 ^××××

××

×× ,

It was proved that an LPT schedule is optimal, if any optimal schedule has at most two jobs assigned to each processor (refer to page 568 of Ref. (2)).

⇒

⇒

⇒⇒ ^{V * I V I} V * I

( ) ( ) ( )

−−

−−

= 0,

contradicting the assumption about I!

•

•

•

• c-Approximation

Ex. Bin Packing

Instance: A capacity value b>0 and a finite set U of items whose sizes do not exceed b.

Question: What is the minimum number of bins of equal capacity b to put away all items of U (each item is placed entirely in one bin)?

For example, given b=10 and six items of sizes 5, 6, 3, 7, 5 and 4, an optimal packing is shown

Four heuristics for Bin Packing:

•••

• First Fit (FF):

Arrange the bins in sequence and place each item into the first bin in which it fits.

•

••

• Best Fit (BF):

Arrange the bins in sequence and place each item into the most nearly full bin in which it fits.

••

•• First Fit Decreasing (FFD):

the same as FF, except that the items are placed in a nonincreasing sequence of sizes.

•

••

• Best Fit Decreasing (BFD):

the same as BF, except that the items are placed in a nonincreasing sequence of sizes.

For the example of b=10 and sizes 5, 6, 3, 7, 5, 4,

FF:

5 3

4

6 7 5

BF:

5

3

4

6 7

5

FFD, BFD:

3 4 5

Properties:

(P1) For FF or BF, V(I) ≤≤≤≤ ¹⁷

10 ××××V*(I)+2;

(P2) For FFD or BFD, V(I) ≤≤≤≤ ¹¹

9 ××××V*(I)+4.

The proofs of (P1) and (P2), which are rather lengthy and complex, can be found below.

Johnson, Demers, Ullman, Garey, and Graham,

“Worst-Case Performance Bounds for Simple One-Dimensional Packing Algorithms,” SIAM Journal on Computing, vol. 3, no. 4, 1974, 299-325.

The upper bound of (P1) was further improved to

¹⁷

10 ××××V*(I) below.

Garey, Graham, Johnson, and Yao, “Resource Constrained Scheduling as Generalized Bin Packing,” Journal of Combinatorial Theory (A), vol. 21, 1976, 257-298.

♣♣♣

♣ A simple proof of V(I)<2××××V*(I) (i.e., ^{V * I V I}

V * I ( ) ( )

( )

−

−

−

− <1) for FF

Assume U={u₁, u₂, …, u_n}, and let s_i be the size of u_i, where 1≤≤≤≤i≤≤≤≤n.

When V(I)=1, we have V*(I)=1 (V(I)<2××××V*(I)).

We consider V(I)>1 below.

For even V(I),

s₁ +s₂ + … +s_n > ^{V I( )} 2 ××××b

⇒

⇒⇒

⇒ V*(I) ≥≥≥≥ ^{V I( )} 2 +1

⇒⇒⇒

⇒ V(I) ≤≤≤≤ 2××××V*(I)−−−−2.

For odd V(I),

Ex. Traveling Salesman Problem (TSP)

Instance: A set C of m cities and distances d_i,j >0 for all pairs of cities i, j∈∈∈∈C.

Question: What is the length of a shortest tour that starts at any city, visits each of the other m−−−−1 cities exactly once, and returns to the initial city?

When all d_i,j’s satisfy triangle inequality, i.e., d_i,j ≤≤≤≤d_i,k +d_k,j for all i, j, k∈∈∈∈C, the metric TSP results.

The Euclidean TSP, where d_i,j is the geodesic distance between i and j, is an instance of the metric TSP.

Both the metric TSP and the Euclidean TSP are NP-hard.

♣ A heuristic with V(I)<2××××V*(I)

For example,

1

2

3

4

5

6

Step 1. Find an MST.

1

2

3

4

5

6

Let d* be the total distance of it.

Step 2. Replace each edge of the tree with two opposite arcs.

1

2

3

4

5

6

Step 3. Construct an Euler circuit of total distance 2××××d* on the graph above.

1→→→→2→→→→3→→→→2→→→→4→→→→6→→→→4→→→→5→→→→4→→→→2→→→→1

Step 4. Arbitrarily find a feasible tour from the circuit above.

1→→→→2→→→→3 −−−−−−−−−−−−→→→→ 4→→→→6 −−−−−−−−−−−→−→→→ 5 −−−−−−−−−−−−−−−−−−−−−−−−→→→→ 1

The feasible tour has

V(I) < 2××××d* < 2××××V*(I).

The time complexity is dominated by Step 1, which takes O(nlogn) time for the Euclidean TSP.

♣

♣♣

♣ A heuristic with V(I) < ³

2 ××××V*(I)

Consider the same example above.

Step 1. Find an MST.

1

2

3

4

5

6

Let d* be the total distance of it.

Step 2. Find a maximum matching with minimum cost on the set of odd-degree vertices.

1

2

3

4

5

6

Step 3. Add the edges of the matching to the MST.

(All the vertices are of even degrees.)

1

2

3

4

5

6

Step 4. Construct an Euler circuit on the graph above.

1→→→→3→→→→2→→→→4→→→→6→→→→5→→→→4→→→→2→→→→1

Step 5. Arbitrarily find a feasible tour from the circuit above.

1→→→→3→→→→2→→→→4→→→→6→→→→5−−−−−−−−−−−−−−−−−−−−−−−−−−−−→→→→1

The time complexity is dominated by Step 2, which takes O(m³) time for the Euclidean TSP (refer to Combinatorial Optimization: Networks and Matroids, by E. L. Lawler, 1976).

Suppose that there are 2k odd-degree vertices: a₁, a₂, …, a_2k, in the MST.

(It is also assumed that these 2k vertices appear in some shortest tour with the same sequence.)

M₁ ={(a₁, a₂), (a₃, a₄), …, (a_2k−−−−1, a_2k)}

M₂ ={(a₂, a₃), (a₄, a₅), …, (a_2k−−−−2, a_2k−−−−1), (a_2k, a₁)}

(M₁ and M₂ are two maximum matchings.) c₁ (c₂) : the cost of M₁ (M₂)

c* : the minimum cost of a maximum matching on {a₁, a₂, …, a_2k}

⇒⇒⇒⇒ 2××××c* ≤≤≤≤ c₁ + c₂ (3)

l : the total distance of the Euler circuit

⇒

⇒

⇒⇒ l = c*+d* (4) and V(I) ≤≤≤≤ l (5) (1), (2), (3), (4), (5) ⇒⇒⇒⇒ V(I) < ³

2 ××××V*(I)

Exercise 11. Read Sec. 9-1 of the textbook.

(1) Illustrate the approximation algorithm by an example.

(2) Show V(I) ≤≤≤ 2≤ ××××V*(I).

Exercise 12. Read Sec. 9-6 of the textbook.

(1) Illustrate the approximation algorithm by an example.

(2) Show V(I) ≤≤≤ 2≤ ××××V*(I).

•

•

•

• NP-Hardness of c-Approximation

Ex. TSP

Let ΠΠΠΠ be the problem of designing a c-approximate algorithm for TSP.

We show Hamiltonian Cycle ∝∝∝∝ ΠΠΠΠ below.

Hamiltonian Cycle

Instance: An undirected graph G=(V, E).

Question: Does G contain a Hamiltonian cycle, i.e., an ordering (v₁, v₂, …, v_|V|) of the vertices of G such that (v₁, v_|V|)∈∈∈∈E and

(v_i, v_i+1)∈∈∈∈E for all 1≤≤≤≤i<|V|?