Slides credited from Hsueh-I Lu & Hsu-Chun Hsiao

Slides credited from Hsueh-I Lu & Hsu-Chun Hsiao

 Homework 4 released

 Due on 1/3 (Thur) 14:20 (three weeks later)

 Mini-HW 10 released

 Due on 12/20 (Thur) 14:20

 Next week

 Break

 Watch online videos

 Class 12/27

 A small test for Christmas (optional)

2

 Complexity Classes

 P v.s. NP

 NP, NP-Complete, NP-Hard

 Polynomial-Time Reduction

4



Input: a graph G



Output: a smallest vertex subset of G that covers all edges of G.



Known to be NP-complete

6

 Input: a Graph G and an integer k.

 Output: Does G contain a vertex cover of size no more than k?

 Original problem  optimization problem

 原先的路燈問題是要算出放路燈的方法

 Yes/No  decision problem

 問k盞路燈夠不夠照亮整個公園

8

Non-Deterministic-Vertex-Cover(G, k) set S = {}

for each vertex x of G

non-deterministically insert x to S if |S| > k

output no

if S is not a vertex cover output no

output yes

 If the correct answer is yes, then there is a computation path of the algorithm that leads to yes.

 至少有一條路是對的

 If the correct answer is no, then all computation paths of the algorithm lead to no.

 每一條路都是對的

Non-Deterministic-Vertex-Cover(G, k) set S = {}

for each vertex x of G

non-deterministically insert x to S if |S| > k

output no

if S is not a vertex cover output no

output yes

10

initial configuration

correct answer

polynomial

“solved” in non-deterministic polynomial time

= “verified” in polynomial time



P ⊆ NP



A problem solvable in polynomial time is verifiable in polynomial time as well



Any NP problem can be solved in (deterministically) exponential time?



Yes



Any NP problem can be solved in (deterministically) polynomial time?



Open problem

12

Why?

 http://www.claymath.org/millennium-problems

 Yang–Mills and Mass Gap

 Riemann Hypothesis

 P vs NP Problem

 Navier–Stokes Equation

 Hodge Conjecture

 Poincaré Conjecture (solved by Grigori Perelman)

 Birch and Swinnerton-Dyer Conjecture

14

Grigori Perelman

Fields Medal (2006), declined Millennium Prize (2010), declined



Aug 2010 claimed a proof of P is

not equal to NP.

 problems that are verifiable  solvable

 public-key cryptography will be broken

16

Widespread belief in P ≠ NP

“If P = NP, then the world would be a profoundly different place than we usually assume it to be. There would be no special value in “creative leaps,” no fundamental gap between solving a problem and recognizing the solution once it's found. Everyone who could appreciate a symphony would be Mozart; everyone who could follow a step-by-step argument would be Gauss...” – Scott Aaronson, MIT

Travelling Salesman (2012)

A movie about P = NP

Best Feature Film in Silicon Valley Film Festival 2012

18

 A problem is NP-hard if it is as least as hard as all NP problems.

 In other words, a problem X is NP-hard if the following condition holds:

 If X can be solved in (deterministic) polynomial time, then all NP problems can be solved in (deterministic) polynomial time.

20

 A problem is NP-complete if

 it is NP-hard and

 it is in NP.

 In other words, an NP-complete problem is one of the “hardest”

problems in the class NP.

 In other words, an NP-complete problem is a hardness representative problem of the class NP.

 Hardest in NP  solving one NPC can solve all NP problems (“complete”)

 It is wildly believed that NPC problems have no polynomial-time solution

 good reference point to judge whether a problem is in P

 We can decide whether a problem is “too hard to solve” by showing it is as hard as an NPC problem

 We then focus on designing approximate algorithms or solving special cases

 Class P: class of problems that can be solved in

 Class NP: class of problems that can be verified in

 Class NP-hard: class of problems that are “at least as hard as all NP problems”

 Class NP-complete: class of problems in both NP and NP-hard

P ≠ NP P = NP 21

22

undecidable: no algorithm;

e.g. halting problem

https://www.youtube.com/watch?v=wGLQiHXHWNk

 Halting problem is to determine whether a program 𝑝 halts on input 𝑥

 Proof for undecidable via a counterexample

 Suppose ℎ can determine whether a program 𝑝 halts on input 𝑥

 ℎ(𝑝, 𝑥) = return (p halts on input x)

 Define g(p) = if h(p,p) is 0 then return 0 else HANG

  g(g) = if h(g,g) is 0 then return 0 else HANG

 Both cases contradict the assumption:

1. g halts on g: then h(g,g)=1, which would make g(g) hang

2. g does not halt on g: then h(g,g)=0, which would make g(g) halt

 Which one is in P?

24

Shortest Simple Path Longest Simple Path

Euler Tour Hamitonian Cycle

LCS with 2 Input Sequences

LCS with Arbitrary Input Sequences Degree-Constrained

Spanning Tree

Minimal Spanning Tree

Sudoku is NPC

26

 Minesweeper Consistency: Given a state of what purports to be a Minesweeper games, is it logically consistent?

Textbook Chapter 34.3 – NP-completeness and reducibility

28

 Input: a Boolean formula with variables

 Output: whether there is a truth assignment for the variables that satisfies the input Boolean formula

 Stephan A. Cook [FOCS 1971] proved that

 SAT can be solved in non-deterministic polynomial time

 SAT ∈ NP

 If SAT can be solved in deterministic polynomial time, then so can any NP problems  SAT ∈ NP-hard

30

 Problem A can be reduced (in polynomial time) to Problem B

= Problem B can be reduced (in polynomial time) from Problem A

 We can find an algorithm that solves Problem B to help solve Problem A

 If problem B has a polynomial-time algorithm, then so does problem A

 Practice: design a MULTIPLY() function by ADD(), DIVIDE(), and SQUARE()

Algorithm for B

? Instance 𝛽 of B Answer for 𝛽 ? Answer for 𝛼 Instance 𝛼 of A

Algorithm for A

What is the complexity of Algorithm for A?

 A reduction is an algorithm for transforming a problem instance into another

 Definition

 Reduction from A to B implies A is not harder than B

 A ≤_p Bif A can be reduced to B in polynomial time

 Applications

 Designing algorithms: given algorithm for B, we can also solve A

 Classifying problems: establish relative difficulty between A and B

 Proving limits: if A is hard, then so is B

31

Algorithm to decide B Reduction

Algorithm

Instance 𝛽 of B Yes

Instance 𝛼 of A

Algorithm to decide A No

This is why we need it for proving NP-completeness!

 If A is an NP-hard problem and B can be reduced from A, then B is an NP-hard problem?

 If A is an NP-complete problem and B can be reduced from A, then B is an NP-complete problem?

 If A is an NP-complete problem and B can be reduced from A, then B is an NP-hard problem?

32



Q: Which one is harder?



A: They have equal difficulty.



Proof:



PARTITION ≤

KNAPSACK



KNAPSACK ≤

PARTITION

KNAPSACK: Given a set 𝑎₁, … , 𝑎_𝑛 of non-negative integers, and an integer 𝐾, decide if there is a subset 𝑃 ⊆

1, 𝑛 such that σ_𝑖∈𝑃𝑎_𝑖 = 𝐾.

PARTITION: Given a set of 𝑛 non-negative integers

𝑎₁, … , 𝑎_𝑛 , decide if there is a subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃𝑎_𝑖 = σ_𝑖∉𝑃𝑎_𝑖.

Polynomial-time reducible?

Polynomial-time reducible?



PARTITION ≤

KNAPSACK

 If we can solve KNAPSACK, how can we use that to solve PARTITION?



KNAPSACK ≤

PARTITION

 If we can solve PARTITION, how can we use that to solve KNAPSACK?

34

KNAPSACK: Given a set 𝑎₁, … , 𝑎_𝑛 of non-negative integers, and an integer 𝐾, decide if there is a subset 𝑃 ⊆

1, 𝑛 such that σ_𝑖∈𝑃𝑎_𝑖 = 𝐾.

PARTITION: Given a set of 𝑛 non-negative integers

𝑎₁, … , 𝑎_𝑛 , decide if there is a subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃𝑎_𝑖 = σ_𝑖∉𝑃𝑎_𝑖.

Polynomial-time reducible?

Polynomial-time reducible?

 If we can solve KNAPSACK, how can we use that to solve PARTITION?

 Polynomial-time reduction

 Set

KNAPSACK: Given a set 𝑎₁, … , 𝑎_𝑛 of non-negative integers, and an integer 𝐾, decide if there is a subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃𝑎_𝑖 = 𝐾.

PARTITION: Given a set of 𝑛 non- negative integers 𝑎₁, … , 𝑎_𝑛 , decide if there is a subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃𝑎_𝑖 = σ_𝑖∉𝑃𝑎_𝑖.

5 6 7 8 5 6 7 8

p-time reduction

PARTITION instance KNAPSACK instance with

 If we can solve KNAPSACK, how can we use that to solve PARTITION?

 Polynomial-time reduction

 Set

 Correctness proof: KNAPSACK returns yes if and only if an equal-size

partition exists ³⁶

Algorithm to decide KNAPSACK

P-time Reduction

Instance 𝛽 of KNAPSACK Instance 𝛼 of Yes

PARTITION

Algorithm to decide PARTITION No

5 6 7 8 5 6 7 8

p-time reduction

PARTITION instance KNAPSACK instance with

 If we can solve PARTITION, how can we use that to solve KNAPSACK?

 Polynomial-time reduction

 Set

 Add ,

KNAPSACK: Given a set 𝑎₁, … , 𝑎_𝑛 of non-negative integers, and an integer 𝐾, decide if there is a subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃𝑎_𝑖 = 𝐾.

PARTITION: Given a set of 𝑛 non- negative integers 𝑎₁, … , 𝑎_𝑛 , decide if there is a subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃𝑎_𝑖 = σ_𝑖∉𝑃𝑎_𝑖.

5 6 7 8 5 6 7 8

p-time reduction

PARTITION instance

48 52

KNAPSACK instance with

 If we can solve PARTITION, how can we use that to solve KNAPSACK?

 Polynomial-time reduction

 Set

 Add ,

 Correctness proof: PARTITION returns yes if and only if there is a

subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃 𝑎_𝑖 = 𝐾 ³⁸

5 6 7 8 5 6 7 8

p-time reduction

KNAPSACK instance with PARTITION instance

48 52

Algorithm to decide PARTITION

P-time Reduction

Instance 𝛽 of PARTITION Instance 𝛼 of Yes

KNAPSACK

Algorithm to decide KNAPSACK No

 Polynomial-time reduction

 Set

 Add ,

 Correctness proof: PARTITION returns yes if and only if there is a subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃 𝑎_𝑖 = 𝐾

 “if” direction

𝑎₁ 𝑎₂ 𝑎₃ 𝑎₄ 𝑎₁ 𝑎₄ 𝑎₅ 𝑎₂ 𝑎₃ 𝑎₆

PARTITION returns yes!

 Polynomial-time reduction

 Set

 Add ,

 Correctness proof: PARTITION returns yes if and only if there is a subset 𝑃 ⊆ 1, 𝑛 such that σ_𝑖∈𝑃 𝑎_𝑖 = 𝐾

 “only if” direction

 Because , if PARTITION returns yes, each set has 4𝐻 + 𝐾

 𝑎₁, … , 𝑎_𝑛 must be divided into 2𝐻 − 𝐾 and 𝐾

40

𝑎₆

𝑎₂ 𝑎₃ 𝑎₅ 𝑎₁ 𝑎₄ 𝑎₁ 𝑎₂ 𝑎₃ 𝑎₄

a subset 𝑃 s.t. σ_𝑖∈𝑃𝑎_𝑖 = 𝐾

 Definition

 Reduction from A to B implies A is not harder than B

 A ≤_p Bif A can be reduced to B in polynomial time

 NP-completeness proofs

 Goal: prove that B is NP-hard

 Known: A is NP-complete/NP-hard

 Approach: construct a polynomial-time reduction algorithm to convert 𝛼 to 𝛽

 Correctness: if we can solve B, then A can be solved  A ≤_p B

 B is no easier than A  A is NP-hard, so B is NP-hard

41

Algorithm to decide B Reduction

Algorithm

Instance 𝛽 of B Yes

Instance 𝛼 of A

Algorithm to decide A No

If the reduction is not p-time, does this argument hold?

42

 Focus on decision problems

 A language L over σ is any set of strings made up of symbols from σ

 Every language L over σ is a subset of σ^∗

 An algorithm A accepts a string if

 The language accepted by an algorithm A is the set of strings

 An algorithm A rejects a string x if

The formal-language framework allows us to express concisely the relation between decision problems and algorithms that solve them.

 NP-Complete (NPC): class of decision problems in both NP and NP-hard

 In other words, a decision problem L is NP-complete if

1. L ∈ NP

2. L ∈ NP-hard (that is, L’ ≤_p L for every L’ ∈ NP)

44

L₁ L₂ L₃ :

L

≤_p

all NP problems

How to prove L is NP-hard ?

L₁ L₂ L₃ :

≤_p

all NP problems

L known

NPC problem

≤_p held by

definition

Goal: prove polynomial- time reduction

 If are languages s.t. , then L₂ ∈ P implies L₁ ∈ P.

A₂ Transform

function f 𝑥 𝑓 𝑥

A₁

46

 Given a Boolean combinational circuit composed of AND, OR, and NOT gates, is it satisfiable?

 Satisfiable: there exists an assignment s.t. outputs = 1

Satisfiable Unsatisfiable

 CIRCUIT-SAT can be solved in non-deterministic polynomial time

 ∈ NP

 If CIRCUIT-SAT can be solved in deterministic polynomial time, then so can any NP problems

 ∈ NP-hard

 (proof in textbook 34.3)

 CIRCUIT-SAT is NP-complete

CIRCUIT-SAT = {<C>: C is a satisfiable Boolean combinational circuit}

48



If one proves that SAT can be solved by a polynomial-time algorithm, then NP = P.



If somebody proves that SAT cannot be solved by any

polynomial-time algorithm, then NP ≠ P.

1. CNF-SAT

2. 0-1 INTEGER PROGRAMMING

3. CLIQUE

4. SET PACKING

5. VERTEX COVER

6. SET COVERING

7. FEEDBACK ARC SET

8. FEEDBACK NODE SET

9. DIRECTED HAMILTONIAN CIRCUIT

10. UNDIRECTED HAMILTONIAN CIRCUIT

11. 3-SAT

12. CHROMATIC NUMBER

13. CLIQUE COVER

14. EXACT COVER

15. 3-dimensional MATCHING

16. STEINER TREE

17. HITTING SET

18. KNAPSACK

19. JOB SEQUENCING

20. PARTITION

21. MAX-CUT

50

 Given a Boolean formula Φ with variables, is there a variable assignment satisfying Φ

 ∧ (AND), ∨ (OR), ¬ (NOT), → (implication), ↔ (if and only if)

 Satisfiable: Φ is evaluated to 1

 Is SAT ∈ NP-Complete?

 To prove that SAT is NP-Complete, we show that

 SAT ∈ NP

 SAT ∈ NP-hard (CIRCUIT-SAT ≤_p SAT)

1) CIRCUIT-SAT is a known NPC problem

2) Construct a reduction f transforming every CIRCUIT-SAT instance to an SAT instance

3) Prove that x ∈ CIRCUIT-SAT iff f(x) ∈ SAT

4) Prove that f is a polynomial time transformation

52

SAT = {Φ | Φ is a Boolean formula with a satisfying assignment }

 Polynomial-time verification: replaces each variable in the formula with the corresponding value in the certificate and then evaluates the expression

53

initial configuration

polynomial

1) CIRCUIT-SAT is a known NPC problem

2) Construct a reduction f transforming every CIRCUIT-SAT instance to an SAT instance

 Assign a variable to each wire in circuit C

 Represent the operation of each gate using a formula, e.g.

 Φ = AND the output variable and the operations of all gates

54

3. Prove that x ∈ CIRCUIT-SAT ↔ f(x) ∈ SAT

 x ∈ CIRCUIT-SAT → f(x) ∈ SAT

 f(x) ∈ SAT → x ∈ CIRCUIT-SAT

4. f is a polynomial time transformation CIRCUIT-SAT ≤_p SAT  SAT ∈ NP-hard

 3-CNF-SAT: Satisfiability of Boolean formulas in 3-conjunctive normal form (3-CNF)

 3-CNF = AND of clauses, each of which is the OR of exactly 3 distinct literals

 A literal is an occurrence of a variable or its negation, e.g., x₁ or ¬x₁

56

 satisfiable

 Is 3-CNF-SAT ∈ NP-Complete?

 To prove that SAT is NP-Complete, we show that

 3-CNF-SAT ∈ NP

 3-CNF-SAT ∈ NP-hard (SAT ≤_p 3-CNF-SAT)

1) SAT is a known NPC problem

2) Construct a reduction f transforming every SAT instance to an 3-CNF-SAT instance

3) Prove that x ∈ SAT iff f(x) ∈ 3-CNF-SAT

4) Prove that f is a polynomial time transformation

3-CNF-SAT = {Φ | Φ is a Boolean formula in 3-conjunctive normal form (3-CNF) with a satisfying assignment }

We focus on the reduction construction from now on, but remember that a full proof requires showing that all other conditions are true as well

a) Construct a binary parser tree for an input formula Φ and introduce a variable y_i for the output of each internal node

58

b) Rewrite Φ as the AND of the root variable and clauses describing the operation of each node

c) Convert each clause Φi’ to CNF

 Construct a truth table for each clause Φi’

 Construct the disjunctive normal form for ¬Φi’

 Apply DeMorgan’s Law to get the CNF formula Φi’’

60

𝒚_𝟏 𝒚_𝟐 𝒚_𝟐 Φ₁’ ¬Φ₁’

1 1 1 0 1

1 1 0 1 0

1 0 1 0 1

1 0 0 0 1

0 1 1 1 0

0 1 0 0 1

0 0 1 1 0

0 0 0 1 0

𝒚_𝟏 𝒚_𝟐 𝒚_𝟐 Φ₁’

1 1 1 0

1 1 0 1

1 0 1 0

1 0 0 0

0 1 1 1

0 1 0 0

0 0 1 1

0 0 0 1

d) Construct Φ’’’ in which each clause C_i exactly 3 distinct literals

 3 distinct literals:

 2 distinct literals:

 1 literal only:

 Φ’’’ is satisfiable iff Φ is satisfiable

 All transformation can be done in polynomial time

  3-CNF-SAT is NP-Complete

 Proving NP-Completeness: L ∈ NPC iff L ∈ NP and L ∈ NP-hard

 Step-by-step approach for proving L in NPC:

 Prove L ∈ NP

 Prove L ∈ NP-hard

1) Select a known NPC problem C

2) Construct a reduction f transforming every instance of C to an instance of L

3) Prove that

4) Prove that f is a polynomial time transformation L ∈ NP

62

P ≠ NP

P = NP

Course Website: http://ada.miulab.tw Email: ada-ta@csie.ntu.edu.tw

63

Important announcement will be sent to @ntu.edu.tw mailbox

& post to the course website