Michael Tsai 2013/11/21 NP-Completeness

2013/11/21

NP-Complete

1. 這些問題目前尚未找到可以在 polynomial time內解決的algorithm.

𝑂 𝑛^𝑘 , 𝑓𝑜𝑟 𝑠𝑜𝑚𝑒 𝑘

2. 這些問題目前尚未被證明無法在 polynomial time內解決.

NP P

2011/05/04 為P v.s. NP提出之四十周年紀念!!

Ref: http://blog.computationalcomplexity.org/2011/05/forty-years-of-p-v-np.html 這些問題可以在polynomial time內被verify

P=NP???

Euler tour versus hamiltonian cycle

 Digraph 𝐺 = 𝑉, 𝐸

 Euler tour: 每個邊都走過一遍, 可以重複走過某 個vertex

 Hamiltonian cycle: 一個simple cycle把所有的 vertex都走一遍

Shortest simple path Longest simple path Solvable in polynomial time NP-Complete

Hamiltonian Cycle Euler Tour

2-CNF satisfiability 3-CNF satisfiability

2-CNF example: 𝑥₁ ∨ ¬𝑥₂ ∧ ¬𝑥₁ ∨ 𝑥₃ ∧ ¬𝑥₂ ∨ ¬𝑥₃ CNF: conjunctive normal form

NP-Complete

 一個問題是NP, 而且跟NP裡面的其他問題至少”一樣難”

 (一樣難: 所有NP裡面的問題可以在polynomial time裡 面轉換成NP-complete的問題)

 因為以上, 所以”一解則全解”

 學這個有什麼用?

 雖然還沒有人證明出NP-complete problems無法在 polynomial time裡面解出來

 但是經過了40年了, 沒有任何NP-complete problem被 解出來(polynomial time), 因此證明某問題為NP-

complete某種程度上也證明了它非常難解(或甚至無法 解)

Decision problem v.s.

optimization problem

 Decision problem: 輸出是yes/no (1/0) (可不可以 找到答案)

 Optimization problem: 輸出是最好的解 (可以的 答案中找出最好的那個)

 例子: Shortest path v.s. Path

 NP-Complete只適用於decision problems

optimization problem

 怎麼將optimization problem轉換成decision problem呢?

 對要optimize的值設定一個bound

 將Shortest path轉換成decision problem:

 給定graph G, vertices u & v, integer k, 有沒有從u 到v的路徑使用少於k個edge?

Reduction

用Reduction證明”一樣難”

 假設以下兩樣存在

 則B也沒有polynomial-time algorithm可以解

 使用反證法,假設有

A: no polynomial-time algorithm exist Polynomial-time reduction algorithm:

instances of A  instances of B

B: polynomial-time algorithm Polynomial-time reduction algorithm:

instances of A  instances of B

A: polynomial-time algorithm

矛盾. 所以B也沒有.

為什麼Polynomial time就是”容 易解”?

 Θ(𝑛¹⁰⁰)雖然是polynomial time, 但實務上這麼高次 的多項式並不常見

 通常如果找到一個polynomial-time algorithm, 比較 快的方法很快也會被找到

 通常使用不同的computation model(之後自動機會 教到, 現在可以想像是單CPU v.s. 多CPU的機器), 某 model可用polynomial-time解的問題在另外一個 model也可用polynomial-time解

 Polynomials are closed under addition, multiplication, and composition.

Abstract problem

I: instances of

problem S: solutions

Q: Abstract problem (binary relation)

Decision problem:

S={0,1}

Example: PATH

𝑖 = 𝐺, 𝑢, 𝑣, 𝑘 𝑃𝐴𝑇𝐻 𝑖 = 1

𝑃𝐴𝑇𝐻 𝑖 = 0

if a shortest path from u to v has at most k edges

otherwise

Encoding

A set S of abstract objects

The set of binary strings

encoding:

mapping Polygons, numbers,

graphs, functions, ordered pairs, programs, …

Some more definitions

 Concrete problem: 一個problem的instance是 binary string的set, 則稱為concrete problem.

 An algorithm solves a concrete problem in O(T(n)):

一個problem的instance長度為n (i的長度, 即為 binary string長度), 而此algorithm可在O(T(n))時 間產生解

 A concrete problem is polynomial-time solvable:

有一個𝑂 𝑛^𝑘 𝑓𝑜𝑟 𝑠𝑜𝑚𝑒 𝑘的algorithm可以解此 problem

P的正式定義

The complexity class P:

The set of concrete decision problems

that are polynomial-time solvable

concrete problem

I: instances of problem

S: solutions Q: Abstract problem

(binary relation)

Decision problem:

S={0,1}

e(I): {0,1}*

𝑒: 𝐼 → 0,1 ^∗

e(Q): Concrete problem

Encoding和花的時間有關嗎?

 有!

 極端的例子: unary

 input: integer k, running time: Θ 𝑘

 假設encoding是unary: 11111…1111

 則在這樣的case下

 input length: n running time: Θ(𝑛)

 可是如果以正常的binary encoding表示

 input length: n = log 𝑘 + 1running time: Θ(2^𝑛)

 Encoding決定是Θ(𝑛) or Θ(2^𝑛)!!

k個

 然而如果我們不考慮這麼極端的例子(unary), 大部 分的encoding都不會影響到一個問題是否可以在 polynomial time解決.

 例: 使用三進位數和二進位數是沒有差別的, 因為我 們可以在polynomial time裡面將三進位數轉換成二 進位數.

f: {0,1}*{0,1}*

input output

如果f花polynomial time可以把任何input轉成 output, 則稱為polynomial-time computable

I: instances of

problem S: solutions

Q: Abstract problem

(binary relation) Decision problem:

S={0,1}

𝑒₁: 0,1 ^∗ 𝑒₁: 𝐼 → 0,1 ^∗

𝑒₁(𝑄): Concrete problem 𝑒₂: 𝐼 → 0,1 ^∗

𝑒₂(𝑄): Concrete problem

polynomially related

if:

Then: 𝑒₁(𝑄) ∈ 𝑃 if and only if 𝑒₂(𝑄) ∈ 𝑃

 因為是對稱的, 所以只需要證明一個方向.

 假設𝑒₁(𝑄)可以在𝑂(𝑛^𝑘)時間內解決(for some constant k)

 假設對每個problem instance i, 𝑒₂(𝑖)轉換成𝑒₁(𝑖)需花 𝑂(𝑛^𝑐)(for some constant c), 𝑛 = 𝑒₂ 𝑖

 則解決𝑒₂ 𝑄 (input為𝑒₂(𝑖)) 先花𝑂(𝑛^𝑐)轉換成𝑒₁(𝑖)

 𝑒₁ 𝑖 = 𝑂(𝑛^𝑐)

 再解決𝑒₁(𝑄) (input為𝑒₁(𝑖)), 花𝑂 𝑒₁ 𝑖 ^𝑘 = 𝑂 𝑛^𝑐𝑘

 c, k都是constant, 因此為polynomial time

只要encoding都是”合理的” (“簡要的”)表示方式, 一個問題的複雜度 (能否在polynomial time裡面解掉)不會被encoding影響.

A Formal-language Framework

 An alphabet Σ: a finite set of symbols

 A language 𝐿 over Σ: 使用Σ裡面的symbol組合而 成的字串 (不一定包含全部可能的字串)

 Ex: Σ = 0,1 , 𝐿 (over Σ)={10,11,101,111, … }

 empty string: ϵ

 empty language: ∅

 Σ^∗: the language with all strings over Σ

 Operations on languages:

 Union, intersection

 Complement: 𝐿 = Σ^∗ − 𝐿

 Concatenation of 𝐿₁𝐿₂:

𝐿 = 𝑥₁𝑥₂: 𝑥₁ ∈ 𝐿₁ 𝑎𝑛𝑑 𝑥₂ ∈ 𝐿₂

 Closure or Kleene Star:

𝐿^∗ = 𝜖 ⋃𝐿 ⋃𝐿² ⋃𝐿³⋃…

𝐿^𝑘:concatenation 自己k次

應用formal language framework…

 Decision problem Q的set of instances為Σ^∗

 Σ = {0,1}

 Q的主要特性可以想成是會產生答案為1(yes)的這些

instances

 因此可以把Q重新定義為一個language over Σ = 0,1 , 而𝐿 = {𝑥 ∈ Σ^∗: 𝑄 𝑥 = 1}

 例子: PATH={ 𝐺, 𝑢, 𝑣, 𝑘 : 𝐺 = 𝑉, 𝐸 is an undirected graph, 𝑢, 𝑣 ∈ 𝑉, 𝑘 ≥ 0 is an integer, and 從u到v在G裡面 有一條路徑含有最多k條edge}

input x, A outputs 1

 An algorithm A rejects a string 𝑥 ∈ 0,1 ^∗ if, given input x, A outputs 0

 The language accepted by an algorithm A is the set of strings 𝐿 = {𝑥 ∈ 0,1 ^∗: 𝐴 𝑥 = 1}

 注意: L is accepted by A, 不一定表示𝑥 ∉ 𝐿會被A reject! (ex. 無窮迴圈)

 A language is decided by an algorithm A if every binary string in L is accepted by A and every binary string not in L is rejected by A

 A language is accepted in polynomial time if it is accepted by A and if A accepts x in time 𝑂(𝑛^𝑘) for a constant k and any length-n string 𝑥 ∈ 𝐿.

: PATH

 The language PATH can be accepted in polynomial time. 這句話什麼意思?

 如果 𝐺, 𝑢, 𝑣, 𝑘 可以找到解 (從u到v在G裡面有一條 路徑含有最多k條edge), 則可以在polynomial time裡 面說”有 (1, yes)”!

 可以嗎? 可以! 用BFS找出最短路徑, 然後看有沒有比 k大 (只需polynomial time)

 假設我們設計這個algorithm發現比k大的話就無窮 迴圈, 這樣的話就沒有 decided in polynomial time.

定義complexity class P

 Complexity class: a set of languages, 是不是在其 中由algorithm的complexity measure決定(ex.

running time), 而此algorithm是決定一個string x 是否屬於L.

 使用這個方式, 我們可以重新定義P這個

complexity class:

𝑃 = {𝐿 ⊆ 0,1 ^∗:

there exists an algorithm A that decides L in polynomial time}

𝑃 = {𝐿: 𝐿 is accepted by a polynomial − time algorithm}.

 Proof:

 The class of languages decided by polynomial- time algorithms是the class of languages

accepted by polynomial-time algorithms的subset.

 所以我們只需要證如果L is accepted by a

polynomial-time algorithm, 它也可以decided by a polynomial-time algorithm.

 假設L是被某polynomial-time algorithm A accept.

 我們要利用A做成一個algorithm A’可以decides L.

for a constant c

 對任何input x, A’ 利用A, 先執行𝑐𝑛^𝑘個steps. 如 果這時候A accept x了, A’就accept x. 如果A還沒 accept x, A’就reject x.

 A’使用A的overhead不會超過一個polynomial factor, 所以A’是一個可以decide L的polynomial time algorithm.

Algorithms that

accepts/rejects/decides L

告訴你某個instance是不 是有解(在L裡面)

Algorithms that verify L with a certificate

給你一個certificate (可能 是答案), 可以讓你檢查某 個instance是不是有解(在 L裡面)

給 𝐺, 𝑢, 𝑣, 𝑘 和一條path p, 我們可以檢查path是 不是真的是在G中uv的path, 且長度是不是不 超過k. 此p是一個certificate, 用來幫助algorithm 看此instance是不是屬於PATH.

對PATH來說其實沒有太大差別, 因為本來就可以在polynomial time decide PATH. 但是對於其他問題可能有差別!

G=(V,E) is a simple cycle that contains each vertex in V.

 Not all graph is Hamiltonian (找不到Hamiltonian cycle)

 HAM-CYCLE={ 𝐺 : G is a Hamiltonian graph}

Hamiltonian cycles

 暴力法?

 假設使用adjacency matrix, n= 𝐺 是G的

encoding的長度 (也就是𝑚²,m是G中vertex數)

 檢查所有的vertex permutation需要Ω 𝑚! =

Ω( 𝑛!) = Ω 2 ^𝑛 , not 𝑂 𝑛^𝑘 for any constant k.

 目前還找不到polynomial time algorithm to decide/accept HAM-CYCLE

Verify會簡單一點嗎?

 會!

 假設告訴你某一個graph G是Hamiltonian, 然後 告訴你一個vertex的序列(certificate)可以組成 Hamiltonian cycle.

 則我們可以在polynomial time裡面檢查:

 這個vertex序列是不是真的是G裡面的vertex的 permutation

 vertex序列的相鄰vertices之間是不是在G中有那 個edge

Verification Algorithm

 Verification Algorithm: Algorithm A with two arguments:

 Ordinary input string x

 Binary string y (certificate)

 A verifies an input string x if there exists a certificate y such that A(x,y)=1.

 The language verified by a verification algorithm A is 𝐿 = {𝑥 ∈ 0,1 ^∗: there exists 𝑦 ∈ 0,1 ^∗

such that 𝐴 𝑥, 𝑦 = 1}

 如果x在L裡面, 則一定找得到y.

 如果x不在L裡面, 則一定找不到y.

The complexity class NP

 Complexity class NP: the class of languages that can be verified by a polynomial-time algorithm.

 A language L belongs to NP if and only if there exist a two-input polynomial-time algorithm A and a constant c such that

L= 𝑥 ∈ 0,1 ^∗: there exists a certificate 𝑦 with 𝑦 = 𝑂 𝑥 ^𝑐 such that 𝐴 𝑥, 𝑦 = 1 .

 Then algorithm A verifies L in polynomial time.

What is in NP?

 HAM-CYCLE∈NP

 if L ∈ 𝑃, then L ∈ 𝑁𝑃. Why?

 可以做出一個algorithm是完全不甩certificate的, 就可以模擬出verification algorithm的效果

 意思就是說𝑃 ⊆ 𝑁𝑃.

 但P=NP or not? (尚未得知)

Complexity class co-NP

 class NP is closed under complement? (尚未得知)

 意思就是說𝐿 ∈ 𝑁𝑃的話, 𝐿 ∈ 𝑁𝑃否?

 co-NP: all languages that satisfies 𝐿 ∈ 𝑁𝑃

NP-Complete languages

 “The hardest languages in NP”

 If NP-P is nonempty, then these in NP-Complete are in NP-P (such as HAM-CYCLE)

 Reducibility  解一個破全部, 一箭千雕

Reducibility

 如果Q可以reduce成Q’, 則表示任何一個Q的 instance都可以”換句話說”變成Q’的一個

instance

 一元一次方程式: ax+b=0可以視為一元二次方程

式的特例: 0𝑥² + 𝑎𝑥 + 𝑏 = 0, 解出來可以得到對 應的一元一次方程式解.

 如果一個問題Q可以reduce成另外一個問題Q’, 則Q不會比Q’難解.

Reduction

 𝐿₁ is polynomial-time reducible to 𝐿₂, (寫成 𝐿₁ ≤_𝑃 𝐿₂) if there exists a polynomial-time

computable function f: 0,1 ^∗ → 0,1 ^∗ such that for all 𝑥 ∈ 0,1 ^∗, 𝑥 ∈ 𝐿₁ if and only if 𝑓(𝑥) ∈ 𝐿₂.

 f: reduction function

 用來計算f 的polynomial-time algorithm F:

reduction algorithm