Algorithm Design and Analysis NP Completeness (2)

(1)

Algorithm Design and Analysis NP Completeness (2)

Yun-Nung (Vivian) Chen

http://ada.miulab.tw

(2)

Outline

• Polynomial-Time Reduction

• Polynomial-Time Verification

• Proving NP-Completeness

• 3-CNF-SAT

• Clique

• Vertex Cover

• Independent Set

• Traveling Salesman Problem

(3)

P, NP, NP-Complete, NP-Hard

• P ≠ NP

Complexity

(4)

(3) (2)

(1)

(4)

Non-Deterministic Problem Solving

initial

configuration

(5)

Non-Deterministic Polynomial

polynomial

“solved” in non-deterministic polynomial time

= “verified” in polynomial time

(6)

Polynomial-Time Reduction

Textbook Chapter 34.3 – NP-completeness and reducibility

(7)

First NP-Complete Problem – SAT (Satisfiability)

• Input: a Boolean formula with variables

• Output: whether there is a truth assignment for the variables that satisfies the input Boolean formula

• Stephan A. Cook [FOCS 1971] proved that

▪ SAT can be solved in non-deterministic polynomial time → SAT ∈ NP

▪ If SAT can be solved in deterministic polynomial time, then so can any NP problems → SAT ∈ NP-hard

(8)

Reduction

• Problem A can be reduced (in polynomial time) to Problem B

= Problem B can be reduced (in polynomial time) from Problem A

• We can find an algorithm that solves Problem B to help solve Problem A

• If problem B has a polynomial-time algorithm, then so does problem A

• Practice: design a MULTIPLY() function by ADD(), DIVIDE(), and SQUARE()

What is the complexity of Algorithm for A?

Algorithm for B

? Instance 𝛽 of B Answer for 𝛽 ? Answer for 𝛼 Instance 𝛼 of A

Algorithm for A

(9)

Reduction

• A reduction is an algorithm for transforming a problem instance into another

• Definition

• Reduction from A to B implies A is not harder than B

• A ≤_p B if A can be reduced to B in polynomial time

• Applications

• Designing algorithms: given algorithm for B, we can also solve A

• Classifying problems: establish relative difficulty between A and B

• Proving limits: if A is hard, then so is B

Algorithm to decide B Reduction

Algorithm

Instance 𝛽 of B Yes

Instance 𝛼 of A

Algorithm to decide A No

This is why we need it for proving NP-completeness!

(10)

Questions

• If A is an NP-hard problem and B can be reduced from A, then B is an NP- hard problem?

• If A is an NP-complete problem and B can be reduced from A, then B is an NP-complete problem?

• If A is an NP-complete problem and B can be reduced from A, then B is an NP-hard problem?

(11)

Problem Difficulty

• Q: Which one is harder?

• A: They have equal difficulty.

• Proof:

• PARTITION ≤_p KNAPSACK

• KNAPSACK ≤_p PARTITION

KNAPSACK: Given a set 𝑎₁, … , 𝑎_𝑛 of non-negative integers, and an integer 𝐾, decide if there is a subset 𝑃 ⊆

1, 𝑛 such that σ_𝑖∈𝑃 𝑎_𝑖 = 𝐾.

PARTITION: Given a set of 𝑛 non-negative integers

𝑎₁, … , 𝑎_𝑛 , decide if there is a subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃 𝑎_𝑖 = σ_𝑖∉𝑃𝑎_𝑖.

Polynomial-time reducible?

(12)

Polynomial Time Reduction

• PARTITION ≤_p KNAPSACK

• If we can solve KNAPSACK, how can we use that to solve PARTITION?

• KNAPSACK ≤_p PARTITION

• If we can solve PARTITION, how can we use that to solve KNAPSACK?

KNAPSACK: Given a set 𝑎₁, … , 𝑎_𝑛 of non-negative integers, and an integer 𝐾, decide if there is a subset 𝑃 ⊆

1, 𝑛 such that σ_𝑖∈𝑃 𝑎_𝑖 = 𝐾.

PARTITION: Given a set of 𝑛 non-negative integers

𝑎₁, … , 𝑎_𝑛 , decide if there is a subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃 𝑎_𝑖 = σ_𝑖∉𝑃𝑎_𝑖.

Polynomial-time reducible?

(13)

PARTITION ≤ _p KNAPSACK

• Polynomial-time reduction

• Set

KNAPSACK: Given a set 𝑎₁, … , 𝑎_𝑛 of non-negative integers, and an integer 𝐾, decide if there is a subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃 𝑎_𝑖 = 𝐾.

PARTITION: Given a set of 𝑛 non-negative integers 𝑎₁, … , 𝑎_𝑛 , decide if there is a

subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃 𝑎_𝑖 = σ_𝑖∉𝑃𝑎_𝑖.

5 6 7 8 5 6 7 8

p-time reduction

PARTITION instance KNAPSACK instance with

(14)

PARTITION ≤ _p KNAPSACK

• Set

• Correctness proof: KNAPSACK returns yes if and only if an equal-size partition

5 6 7 8 5 6 7 8

p-time reduction

PARTITION instance KNAPSACK instance with

Algorithm to decide KNAPSACK

P-time Reduction

Instance 𝛽 of KNAPSACK Instance 𝛼 of Yes

PARTITION

Algorithm to decide PARTITION No

(15)

KNAPSACK ≤ _p PARTITION

• Set

• Add ,

5 6 7 8 5 6 7 8

p-time reduction

PARTITION instance 48 52 KNAPSACK instance with

KNAPSACK: Given a set 𝑎₁, … , 𝑎_𝑛 of non-negative integers, and an integer 𝐾, decide if there is a subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃 𝑎_𝑖 = 𝐾.

PARTITION: Given a set of 𝑛 non-negative integers 𝑎₁, … , 𝑎_𝑛 , decide if there is a

subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃 𝑎_𝑖 = σ_𝑖∉𝑃𝑎_𝑖.

(16)

KNAPSACK ≤ _p PARTITION

• Set

• Add ,

• Correctness proof: PARTITION returns yes if and only if there is a subset

Algorithm to decide PARTITION

P-time Reduction

Instance 𝛽 of PARTITION Instance 𝛼 of Yes

KNAPSACK

Algorithm to decide KNAPSACK No

5 6 7 8 5 6 7 8

p-time reduction

PARTITION instance 48 52 KNAPSACK instance with

(17)

KNAPSACK ≤ _p PARTITION

• Set

• Add ,

• Correctness proof: PARTITION returns yes if and only if there is a subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃 𝑎_𝑖 = 𝐾

• “if” direction

𝑎₁ 𝑎₂ 𝑎₃ 𝑎₄ 𝑎₁ 𝑎₄ 𝑎₅ 𝑎₂ 𝑎₃ 𝑎₆

PARTITION returns yes!

(18)

KNAPSACK ≤ _p PARTITION

• Set

• Add ,

• Correctness proof: PARTITION returns yes if and only if there is a subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃 𝑎_𝑖 = 𝐾

• “only if” direction

• Because , if PARTITION returns yes, each set has 4𝐻 + 𝐾

• 𝑎₁, … , 𝑎_𝑛 must be divided into 2𝐻 − 𝐾 and 𝐾

𝑎₆

𝑎₂ 𝑎₃ 𝑎₅ 𝑎₁ 𝑎₄ 𝑎₁ 𝑎₄ 𝑎₂ 𝑎₃

(19)

Reduction for Proving Limits

• Definition

• Reduction from A to B implies A is not harder than B

• A ≤_p B if A can be reduced to B in polynomial time

• NP-completeness proofs

• Goal: prove that B is NP-hard

• Known: A is NP-complete/NP-hard

• Approach: construct a polynomial-time reduction algorithm to convert 𝛼 to 𝛽

• Correctness: if we can solve B, then A can be solved → A ≤_p B

• B is no easier than A → A is NP-hard, so B is NP-hard

If the reduction is not p-time, does this argument hold?

Algorithm to decide B Reduction

Algorithm

Instance 𝛽 of B Yes

Instance 𝛼 of A

Algorithm to decide A No

(20)

Proving NP-Completeness

(21)

Formal Language Framework

• Focus on decision problems

• A language L over σ is any set of strings made up of symbols from σ

• Every language L over σ is a subset of σ^∗

• An algorithm A accepts a string if

• The language accepted by an algorithm A is the set of strings

• An algorithm A rejects a string x if

The formal-language framework allows us to express concisely the relation between decision problems and algorithms that solve them.

(22)

Proving NP-Completeness

• NP-Complete (NPC): class of decision problems in both NP and NP-hard

• In other words, a decision problem L is NP-complete if 1.L ∈ NP

2.L ∈ NP-hard (that is, L’ ≤_p L for every L’ ∈ NP)

L₁ L₂ L₃ :

L

≤_p

all NP problems How to prove L is NP-hard ?

L₁ L₂ L₃ :

≤_p

all NP problems

L known

NPC problem

≤_p

held by definition

Goal: prove polynomial- time reduction

(23)

Polynomial-Time Reducible

• If are languages s.t. , then L2 ∈ P implies L1 ∈ P.

A₂ Transform

function f 𝑥 𝑓 𝑥

A₁

(24)

P v.s. NP

• If one proves that SAT can be solved by a polynomial-time algorithm, then NP = P.

• If somebody proves that SAT cannot be solved by any polynomial-

time algorithm, then NP ≠ P.

(25)

Circuit Satisfiability Problem

• Given a Boolean combinational circuit composed of AND, OR, and NOT gates, is it satisfiable?

• Satisfiable: there exists an assignment s.t. outputs = 1

Satisfiable Unsatisfiable

(26)

CIRCUIT-SAT

• CIRCUIT-SAT can be solved in non-deterministic polynomial time

→ ∈ NP

• If CIRCUIT-SAT can be solved in deterministic polynomial time, then so can any NP problems

→ ∈ NP-hard

• (proof in textbook 34.3)

• CIRCUIT-SAT is NP-complete

CIRCUIT-SAT = {<C>: C is a satisfiable Boolean combinational circuit}

(27)

Karp’s NP-Complete Problems

1. CNF-SAT

2. 0-1 INTEGER PROGRAMMING 3. CLIQUE

4. SET PACKING 5. VERTEX COVER 6. SET COVERING

7. FEEDBACK ARC SET 8. FEEDBACK NODE SET

9. DIRECTED HAMILTONIAN CIRCUIT

10.UNDIRECTED HAMILTONIAN CIRCUIT 11.3-SAT

12.CHROMATIC NUMBER 13.CLIQUE COVER

14.EXACT COVER

15.3-dimensional MATCHING 16.STEINER TREE

17.HITTING SET 18.KNAPSACK

19.JOB SEQUENCING 20.PARTITION

21.MAX-CUT

(28)

Karp’s NP-Complete Problems

(29)

Formula Satisfiability Problem (SAT)

• Given a Boolean formula Φ with variables, is there a variable assignment satisfying Φ

• ∧ (AND), ∨ (OR), ¬ (NOT), → (implication), ↔ (if and only if)

• Satisfiable: Φ is evaluated to 1

(30)

SAT

• Is SAT ∈ NP-Complete?

• To prove that SAT is NP-Complete, we show that

• SAT ∈ NP

• SAT ∈ NP-hard (CIRCUIT-SAT ≤_p SAT)

1) CIRCUIT-SAT is a known NPC problem

2) Construct a reduction f transforming every CIRCUIT-SAT instance to an SAT instance 3) Prove that x ∈ CIRCUIT-SAT iff f(x) ∈ SAT

4) Prove that f is a polynomial time transformation

SAT = {Φ | Φ is a Boolean formula with a satisfying assignment }

(31)

SAT ∈ NP

• Polynomial-time verification: replaces each variable in the formula with the corresponding value in the certificate and then evaluates the expression

initial configuration

polynomial

(32)

SAT ∈ NP-Hard

1)CIRCUIT-SAT is a known NPC problem

2)Construct a reduction f transforming every CIRCUIT-SAT instance to an SAT instance

• Assign a variable to each wire in circuit C

• Represent the operation of each gate using a formula, e.g.

• Φ = AND the output variable and the operations of all gates

(33)

SAT ∈ NP-Hard

• Prove that x ∈ CIRCUIT-SAT ↔ f(x) ∈ SAT

• x ∈ CIRCUIT-SAT → f(x) ∈ SAT

• f(x) ∈ SAT → x ∈ CIRCUIT-SAT

• f is a polynomial time transformation CIRCUIT-SAT ≤_p SAT → SAT ∈ NP-hard

(34)

Polynomial-Time Verification

Chapter 34.1 – Polynomial-time

Chapter 34.2 – Polynomial-time verification

(35)

Abstract Problems

• Example of a decision problem, PATH

• I: a set of problem instances

• S: a set of problem solutions

• Q: abstract problem, defined as a binary relation on I and S

G1, s1, t1, k1 G2, s2, t2, k2

…

Q: PATH

Is there a path with the length ≤ k?

I: <G, src, dest, k>

All graphs with arbitrary src, dest, and the path length k

1 (Yes) 0 (No) S: {0, 1}

(36)

Problem Instance Encoding

• Convert an abstract problem instance into a binary string fed to a computer program

• A concrete problem is polynomial-time solvable if there exists an algorithm that solves any concrete instance of length n in time for some constant k

• Solvable = can produce a solution

Encoder

Abstract Problem Instance

Binary Strings

(Concrete Problem Instance)

(37)

Decision Problem Representation

• I: a set of problem instances

• Q: a decision problem

= a language L over s.t.

str1 str2 str3

…

Q: decision problem I: a set of problem

instances

1 (Yes) 0 (No) S: {0, 1}

以答案為1的instances定義decision problem Q (L = {str1, str3} in this example)

some strings represent no meaningful instances

(38)

P in Formal Language Framework

• An algorithm A accepts a string if

• An algorithm A rejects a string if

• An algorithm A accepts a language L if A accepts every string

• If the string is in L, A outputs yes.

• If the string is not in L, A may output no or loop forever.

• An algorithm A decides a language L if A accepts L and A rejects every string

• For every string, A can output the correct answer.

A decision problem Q can be defined as a language L over s.t.

(39)

P in Formal Language Framework

• Class P: a class of decision problems solvable in polynomial time

• Given an instance x of a decision problem Q, its solution Q(x) (i.e., YES or NO) can be found in polynomial time

• An alternative definition of P:

• P is the class of language that can be accepted in polynomial time

P = {L ⊆ {0,1}* | there exists an algorithm that decides L in polynomial time}

P = {L | L is accepted by a polynomial algorithm}

(40)

Hamiltonian-Cycle Problem

• Problem: find a cycle that visits each vertex exactly once

• Formal language:

• Is this language decidable?

• Is this language decidable in polynomial time?

• Given a certificate – the vertices in order that form a Hamiltonian cycle in G, how much time does it take to verify that G indeed contains a Hamiltonian cycle?

HAM-CYCLE = {<G> | G has a Hamiltonian cycle}

Yes

Probably not

(41)

Verification Algorithm

• Verification algorithms verify memberships in language

Verification Algorithm Is y a Hamiltonian cycle in the graph (encoded in x)?

Certificate y

YES

x is in HAM-CYCLE Input x

There exists a certificate for each YES instance

(42)

Verification Algorithm

• Verification algorithms verify memberships in language

Certificate y

NO

No conclusion Input x

??

Verification Algorithm Is y a Hamiltonian cycle in the graph (encoded in x)?

(43)

Non-Deterministic Polynomial

polynomial

“solved” in non-deterministic polynomial time

= “verified” in polynomial time

(44)

Polynomial-Time Reducible

• If are languages s.t. , then L2 ∈ P implies L1 ∈ P.

A₂ Transform

function f 𝑥 𝑓 𝑥

A

(45)

Proving NP-Completeness

• NP-Complete (NPC): class of decision problems in both NP and NP-hard

• In other words, a decision problem L is NP-complete if 1.L ∈ NP

2.L ∈ NP-hard (that is, L’ ≤_p L for every L’ ∈ NP)

L₁ L₂ L₃ :

L

≤_p

all NP problems How to prove L is NP-hard ?

L₁ L₂ L₃ :

≤_p

all NP problems

L known

NPC problem

≤_p

held by definition

Goal: prove polynomial- time reduction

(46)

Proving NP-Completeness

• L ∈ NPC iff L ∈ NP and L ∈ NP-hard

• Proof of L in NPC:

• Prove L ∈ NP

• Prove L ∈ NP-hard

1) Select a known NPC problem C

2) Construct a reduction f transforming every instance of C to an instance of L 3) Prove that

(47)

• “Computers and Intractability” by Garey and Johnson includes more than 300 NP-complete problems

• All except SAT are proved by Karp’s polynomial-time reduction

(48)

Proving NP-Completeness

Chapter 34.5 – NP-complete problems

(49)

Roadmap for NP-Completeness

• A → B: A ≤_p B

CIRCUIT-SAT SAT

3-CNF-SAT

CLIQUE SUBSET-SUM

VERTEX-COVER HAM-CYCLE

TSP

(50)

3-CNF-SAT Problem

• 3-CNF-SAT: Satisfiability of Boolean formulas in 3-conjunctive normal form (3- CNF)

• 3-CNF = AND of clauses, each of which is the OR of exactly 3 distinct literals

• A literal is an occurrence of a variable or its negation, e.g., x₁ or ¬x₁

→ satisfiable

(51)

3-CNF-SAT

• Is 3-CNF-SAT ∈ NP-Complete?

• To prove that 3-CNF-SAT is NP-Complete, we show that

• 3-CNF-SAT ∈ NP

• 3-CNF-SAT ∈ NP-hard (SAT ≤_p 3-CNF-SAT)

1) SAT is a known NPC problem

2) Construct a reduction f transforming every SAT instance to an 3-CNF-SAT instance 3) Prove that x ∈ SAT iff f(x) ∈ 3-CNF-SAT

3-CNF-SAT = {Φ | Φ is a Boolean formula in 3-conjunctive normal form (3-CNF) with a satisfying assignment }

We focus on the reduction construction from now on, but remember that a full proof requires showing that all other conditions are true as well

(52)

SAT ≤ _p 3-CNF-SAT

a) Construct a binary parser tree for an input formula Φ and introduce a variable y_i for the output of each internal node

(53)

SAT ≤ _p 3-CNF-SAT

b) Rewrite Φ as the AND of the root variable and clauses describing the operation of each node

(54)

SAT ≤ _p 3-CNF-SAT

c) Convert each clause Φ_i’ to CNF

• Construct a truth table for each clause Φ_i’

• Construct the disjunctive normal form for ¬Φ_i’

• Apply DeMorgan’s Law to get the CNF formula Φ_i’’

𝒚_𝟏 𝒚_𝟐 𝒚_𝟐 Φ₁’ ¬Φ₁’

1 1 1 0 1

1 1 0 1 0

1 0 1 0 1

1 0 0 0 1

0 1 1 1 0

0 1 0 0 1

0 0 1 1 0

𝒚_𝟏 𝒚_𝟐 𝒚_𝟐 Φ₁’

1 1 1 0

1 1 0 1

1 0 1 0

1 0 0 0

0 1 1 1

0 1 0 0

0 0 1 1

(55)

SAT ≤ _p 3-CNF-SAT

d) Construct Φ’’’ in which each clause C_i exactly 3 distinct literals

• 3 distinct literals:

• 2 distinct literals:

• 1 literal only:

• Φ’’’ is satisfiable iff Φ is satisfiable

• All transformation can be done in polynomial time

• → 3-CNF-SAT is NP-Complete

(56)

Clique Problem

• A clique in G = (V, E) is a complete subgraph of G

• Each pair of vertices in a clique is connected by an edge in E

• Size of a clique = # of vertices it contains

• Optimization problem: find a max clique in G

• Decision problem: is there a clique with size larger than k

Does G contain a clique of size 4?

Yes Yes No

(57)

CLIQUE ∈ NP-Complete

• Is CLIQUE ∈ NP-Complete?

• Construct a reduction f transforming every 3-CNF-SAT instance to a CLIQUE instance

• a graph G s.t. Φ with k clauses is satisfiable  G has a clique of size k

CLIQUE = {<G, k>: G is a graph containing a clique of size k}

3-CNF-SAT ≤_pCLIQUE

≤_p

(58)

CLIQUE ∈ NP-Complete

• Polynomial-time reduction:

• Let be a Boolean formula in 3-CNF with k clauses, and each 𝐶_𝑟 has exactly 3 distinct literals 𝑙₁^𝑟, 𝑙₂^𝑟, 𝑙₃^𝑟

• For each , introduce a triple of vertices 𝑣₁^𝑟, 𝑣₂^𝑟, 𝑣₃^𝑟 in V

• Build an edge between 𝑣_𝑖^𝑟, 𝑣_𝑗^𝑠 if both of the following hold:

• 𝑣_𝑖^𝑟 and 𝑣_𝑗^𝑠 are in different triples

• 𝑙_𝑖^𝑟 is not the negation of 𝑙_𝑗^𝑠

(59)

3-CNF-SAT ≤ _p CLIQUE

• Correctness proof: Φ is satisfiable → G has a clique of size k

• If Φ is satisfiable

• → Each C_rcontains at least one 𝑙_𝑖^𝑟 = 1 and such literal corresponds to 𝑣_𝑖^𝑟

• → Pick a TRUE literal from each C_r forms a set of V’ of k vertices

• → For any two vertices 𝑣_𝑖^𝑟, 𝑣_𝑗^𝑠 ∈ 𝑉^′(𝑟 ≠ 𝑠), edge (𝑣_𝑖^𝑟, 𝑣_𝑗^𝑠) ∈ 𝐸, because 𝑙_𝑖^𝑟 = 𝑙_𝑗^𝑠 = 1 and they cannot be complements

(60)

3-CNF-SAT ≤ _p CLIQUE

• Correctness proof: G has a clique of size k → Φ is satisfiable

• G has a clique V’ of size k

• → V’ contains exactly one vertex per triple since no edges connect vertices in the same triple

• → Assign 1 to each 𝑙_𝑖^𝑟 where 𝑣_𝑖^𝑟 ∈ 𝑉′ s.t. each C_ris satisfiable, and so is Φ

(61)

Vertex Cover Problem

• A vertex cover of G = (V, E) is a subset V’ ⊆ V s.t. if (w, v) ∈ E, then w ∈ V’ or v

∈ V’

• A vertex cover “covers” every edge in G

• Optimization problem: find a minimum size vertex cover in G

• Decision problem: is there a vertex cover with size smaller than k

Does G have a vertex cover of size 11?

Yes Yes No

(62)

VERTEX-COVER ∈ NP-Complete

• Is VERTEX-COVER ∈ NP-Complete?

• Construct a reduction f transforming every CLIQUE instance to a VERTEX- COVER instance (polynomial-time reduction)

• Compute the complement of G

• Given G = <V, E>, G^cis defined as <V, E^c> s.t. E^c= {(u,v) | (u,v) ∉ E}

• a graph G has a clique of size k  G_c has a vertex cover of size |V| - k

VERTEX-COVER = {<G, k>: G is a graph containing a vertex cover of size k}

CLIQUE ≤_pVERTEX-COVER

≤_p

(63)

CLIQUE ≤ _p VERTEX-COVER

• Correctness proof:

a graph G has a clique of size k → G_c has a vertex cover of size |V| - k

• If G has a clique V’ ⊆ V with |V’| = k

• → for all 𝑤, 𝑣 ∈ 𝐸_𝑐, at least one of 𝑤 or 𝑣 ∉ 𝑉′

• → 𝑤 ∈ 𝑉 − 𝑉′ or 𝑣 ∈ 𝑉 − 𝑉′ (or both)

• → edge 𝑤, 𝑣 is covered by 𝑉 − 𝑉′

• → 𝑉 − 𝑉′ forms a vertex cover of G_c, and |V - V'| = |V| - k

(64)

CLIQUE ≤ _p VERTEX-COVER

• Correctness proof:

G_c has a vertex cover of size |V| - k → a graph G has a clique of size k

• If G_c has a vertex cover V’ ⊆ V with |V’| = |V| - k

• → for all 𝑤, 𝑣 ∈ 𝑉, if 𝑤, 𝑣 ∈ 𝐸_𝑐, then 𝑤 ∈ 𝑉′ or 𝑣 ∈ 𝑉′ or both

• → for all 𝑤, 𝑣 ∈ 𝑉, if 𝑤 ∉ 𝑉′ and 𝑣 ∉ 𝑉′, 𝑤, 𝑣 ∈ 𝐸

• → 𝑉 − 𝑉′ is a clique where |V - V'| = k

(65)

Independent-Set Problem

• An independent set of G = (V, E) is a subset V’ ⊆ V such that G has no edge between any pair of vertices in V’

• A vertex cover “covers” every edge in G

• Optimization problem: find a maximum size independent set

• Decision problem: is there an independent set with size larger than k

Does G have an independent set of size 1?

(66)

IND-SET ∈ NP-Complete

• Is IND-SET ∈ NP-Complete?

• Practice by yourself (textbook problem 34-1)

IND-SET = {<G, k>: G is a graph containing an independent set of size k}

(67)

CLIQUE, VERTEX-COVER, IND-SET

• The following are equivalent for G = (V, E) and a subset V’ of V:

1) V' is a clique of G

2) V-V' is a vertex cover of G_c 3) V' is an independent set of G_c

(68)

Traveling Salesman Problem (TSP)

• Optimization problem: Given a set of cities and their pairwise distances, find a tour of lowest cost that visits each city exactly once.

• Decision problem: is there a traveling salesman tour with cost at most k

(69)

TSP ∈ NP-Complete

• Is TSP ∈ NP-Complete?

• Construct a reduction f transforming every HAM-CYCLE instance to a TSP instance (polynomial-time reduction)

• G contains a Hamiltonian cycle h = <v₁, v₂, …, v_n, v₁>  <v₁, v₂, …, v_n, v₁> is a traveling-salesman tour with cost 0

TSP = {<G, c, k>: G = (V,E) is a complete graph, c is a cost function for edges, G has a traveling-salesman tour with cost at most k}

HAM-CYCLE ≤_pTSP

u

v y

w z

u

v y

w z

≤_p

0

1

(70)

HAM-CYCLE ≤ _p TSP

• Correctness proof: x ∈ HAM-CYCLE → f(x) ∈ TSP

• If Hamiltonian cycle is h = <v₁, v₂, …, v_n, v₁>

• → h is also a tour in the transformed TSP instance

• → The distance of the tour h is 0 since there are n consecutive edges in E, and so has distance 0 in f(x)

• → f(x) ∈ TSP (f(x) has a TSP tour with cost ≤ 0)

u

v y

u

v y

≤_p

0

1

(71)

HAM-CYCLE ≤ _p TSP

• Correctness proof: f(x) ∈ TSP → x ∈ HAM-CYCLE

• After reduction, if a TSP tour with cost ≤ 0 as <v₁, v₂, …, v_n, v₁>

• → The tour contains only edges in E

• → Thus, <v₁, v₂, …, v_n, v₁> is a Hamiltonian cycle

u

v y

w z

u

v y

w z

≤_p

0

1

(72)

TSP Challenges

• Mona Lisa TSP: $1,000 Prize for 100,000-city

(73)

Strategies for NP-Complete/NP-Hard Problems

• NP-complete/NP-hard problems are unlikely to have polynomial-time

solutions (unless P = NP), we must sacrifice either optimality, efficiency, or generality

• Approximation algorithms: guarantee to be a fixed percentage away from the optimum

• Local search: simulated annealing (hill climbing), genetic algorithms, etc

• Heuristics: no formal guarantee of performance

• Randomized algorithms: use a randomizer (random number generator) for operation

• Pseudo-polynomial time algorithms: e.g., DP for 0-1 knapsack

• Exponential algorithms/Branch and Bound/Exhaustive search: feasible only when the problem size is small

• Restriction: work on some special cases of the original problem. e.g., the maximum independent set problem in circle graphs

(74)

74

(75)

Concluding Remarks

• Proving NP-Completeness: L ∈ NPC iff L ∈ NP and L ∈ NP-hard

• Polynomial-time verification

• Step-by-step approach for proving L in NPC:

• Prove L ∈ NP

• Prove L ∈ NP-hard

• Select a known NPC problem C

• Construct a reduction f transforming every instance of C to an instance of L

• Prove that

• Prove that f is a polynomial time transformation L ∈ NP

• Strategies for NP-complete/NP-hard problems

P ≠ NP

P = NP

CIRCUIT -SAT

SAT

3-CNF- SAT

CLIQUE SUBSET- SUM VERTEX

-COVER HAM- CYCLE

TSP

(76)

Concluding Remarks

• Proving NP-Completeness: L ∈ NPC iff L ∈ NP and L ∈ NP-hard

• Polynomial-time verification

• Step-by-step approach for proving L in NPC:

• Prove L ∈ NP

• Prove L ∈ NP-hard

1) Select a known NPC problem C

2) Construct a reduction f transforming every instance of C to an instance of L 3) Prove that

• Strategies for NP-complete/NP-hard problems

CIRCUIT-SAT

SAT

3-CNF-SAT

CLIQUE SUBSET-

SUM

VERTEX- COVER HAM-CYCLE

TSP

(77)

Question?

Important announcement will be sent to

@ntu.edu.tw mailbox & post to the course website

Course Website: http://ada.miulab.tw Email: ada-ta@csie.ntu.edu.tw

Algorithm Design and Analysis NP Completeness (2)