Degrees of Difficulty

Reductions and Completeness

Degrees of Difficulty

• When is a problem more difficult than another?

• B reduces to A if there is a transformation R which for every input x of B yields an equivalent input R(x) of A.

– The answer to x for B is the same as the answer to R(x) for A.

– There must be restrictions on the complexity of computing R.

– Otherwise, R(x) might as well solve B.

∗ E.g., R(x) = “yes” if and only if x ∈ B!

Degrees of Difficulty (concluded)

• We say problem A is at least as hard as problem B if B reduces to A.

• This makes intuitive sense: If A is able to solve your problem B after only a little bit of work (R), then A must be at least as hard.

– If A were easy, it combined with R (which is also easy) would make B easy, too.^a

aThanks to a lively class discussion on October 13, 2009.

Reduction

x R(x) yes/no

R algorithm

for A

Solving problem B by calling the algorithm for problem once and without further processing its answer.

Comments

• Suppose B reduces to A via a transformation R.

• The input x is an instance of B.

• The output R(x) is an instance of A.

• R(x) may not span all possible instances of A.^b

• So some instances of A may never appear in the range of the reduction R.

aContributed by Mr. Ming-Feng Tsai (D92922003) on October 29, 2003.

bR(x) may not be onto; Mr. Alexandr Simak (D98922040) on October 13, 2009.

Reduction between Languages

• Language L₁ is reducible to L₂ if there is a function R computable by a deterministic TM in space O(log n).

• Furthermore, for all inputs x, x ∈ L₁ if and only if R(x) ∈ L₂.

• R is said to be a (Karp) reduction from L₁ to L₂.

• Note that by Theorem 22 (p. 189), R runs in polynomial time.

• Suppose R is a reduction from L₁ to L₂.

• Then solving “R(x) ∈ L₂” is an algorithm for solving

“x ∈ L₁.”

A Paradox?

• Degree of difficulty is not defined in terms of absolute complexity.

• So a language B ∈ TIME(n⁹⁹) may be “easier” than a language A ∈ TIME(n³).

– This happens when B is reducible to A.

• But isn’t this a contradiction if the best algorithm for B requires n⁹⁹ steps?

• That is, how can a problem requiring n⁹⁹ steps be reducible to a problem solvable in n³ steps?

A Paradox? (concluded)

• The so-called contradiction does not hold.

• When we solve the problem “x ∈ B?” via “R(x) ∈ A?”, we must consider the time spent by R(x) and its length

| R(x) |.

• If | R(x) | = Ω(n³³), then answering “R(x) ∈ A?” takes Ω((n³³)³) = Ω(n⁹⁹) steps, which is fine.

• Suppose, on the other hand, that | R(x) | = o(n³³).

• Then R(x) must run in time Ω(n⁹⁹) to make the overall time for answering “R(x) ∈ A?” take Ω(n⁹⁹) steps.

• In either case, the contradiction disappears.

hamiltonian path

• A Hamiltonian path of a graph is a path that visits every node of the graph exactly once.

• Suppose graph G has n nodes: 1, 2, . . . , n.

• A Hamiltonian path can be expressed as a permutation π of { 1, 2, . . . , n } such that

– π(i) = j means the ith position is occupied by node j.

– (π(i), π(i + 1)) ∈ G for i = 1, 2, . . . , n − 1.

• hamiltonian path asks if a graph has a Hamiltonian path.

Reduction of hamiltonian path to sat

• Given a graph G, we shall construct a CNF R(G) such that R(G) is satisfiable iff G has a Hamiltonian path.

• R(G) has n² boolean variables x_ij, 1 ≤ i, j ≤ n.

• x_ij means

the ith position in the Hamiltonian path is occupied by node j.

1

2

3

4

5 6

8 7 9

x₁₂ = x₂₁ = x₃₄ = x₄₅ = x₅₃ = x₆₉ = x₇₆ = x₈₈ = x₉₇ = 1;

π(1) = 2, π(2) = 1, π(3) = 4, π(4) = 5, π(5) = 3, π(6) = 9, π(7) = 6, π(8) = 8, π(9) = 7.

The Clauses of R(G) and Their Intended Meanings

1. Each node j must appear in the path.

• x_1j ∨ x_2j ∨ · · · ∨ x_nj for each j.

2. No node j appears twice in the path.

• ¬x_ij ∨ ¬x_kj for all i, j, k with i 6= k.

3. Every position i on the path must be occupied.

• x_i1 ∨ x_i2 ∨ · · · ∨ x_in for each i.

4. No two nodes j and k occupy the same position in the path.

• ¬x_ij ∨ ¬x_ik for all i, j, k with j 6= k.

5. Nonadjacent nodes i and j cannot be adjacent in the path.

• ¬x_ki ∨ ¬x_k+1,j for all (i, j) 6∈ G and k = 1, 2, . . . , n − 1.

The Proof

• R(G) contains O(n³) clauses.

• R(G) can be computed efficiently (simple exercise).

• Suppose T |= R(G).

• From clauses of 1 and 2, for each node j there is a unique position i such that T |= x_ij.

• From clauses of 3 and 4, for each position i there is a unique node j such that T |= x_ij.

• So there is a permutation π of the nodes such that π(i) = j if and only if T |= x_ij.

The Proof (concluded)

• Clauses of 5 furthermore guarantee that

(π(1), π(2), . . . , π(n)) is a Hamiltonian path.

• Conversely, suppose G has a Hamiltonian path (π(1), π(2), . . . , π(n)),

where π is a permutation.

• Clearly, the truth assignment

T (x_ij) = true if and only if π(i) = j satisfies all clauses of R(G).

A Comment

• An answer to “Is R(G) satisfiable?” does answer “Is G Hamiltonian?”

• But a positive answer does not give a Hamiltonian path for G.

– Providing witness is not a requirement of reduction.

• A positive answer to “Is R(G) satisfiable?” plus a satisfying truth assignment does provide us with a Hamiltonian path for G.

aContributed by Ms. Amy Liu (J94922016) on May 29, 2006.

Reduction of reachability to circuit value

• Note that both problems are in P.

• Given a graph G = (V, E), we shall construct a variable-free circuit R(G).

• The output of R(G) is true if and only if there is a path from node 1 to node n in G.

• Idea: the Floyd-Warshall algorithm.

The Gates

• The gates are

– g_ijk with 1 ≤ i, j ≤ n and 0 ≤ k ≤ n.

– h_ijk with 1 ≤ i, j, k ≤ n.

• g_ijk: There is a path from node i to node j without passing through a node bigger than k.

• h_ijk: There is a path from node i to node j passing through k but not any node bigger than k.

• Input gate g_ij0 = true if and only if i = j or (i, j) ∈ E.

The Construction

• h_ijk is an and gate with predecessors g_i,k,k−1 and g_k,j,k−1, where k = 1, 2, . . . , n.

• g_ijk is an or gate with predecessors g_i,j,k−1 and h_i,j,k, where k = 1, 2, . . . , n.

• g_1nn is the output gate.

• Interestingly, R(G) uses no ¬ gates: It is a monotone circuit.

Reduction of circuit sat to sat

• Given a circuit C, we will construct a boolean

expression R(C) such that R(C) is satisfiable iff C is.

– R(C) will turn out to be a CNF.

– R(C) is a depth-2 circuit; furthermore, each gate has out-degree 1.

• The variables of R(C) are those of C plus g for each gate g of C.

– g’s propagate the truth values for the CNF.

• Each gate of C will be turned into equivalent clauses.

• Recall that clauses are ∧-ed together by definition.

The Clauses of R(C)

g is a variable gate x: Add clauses (¬g ∨ x) and (g ∨ ¬x).

• Meaning: g ⇔ x.

g is a true gate: Add clause (g).

• Meaning: g must be true to make R(C) true.

g is a false gate: Add clause (¬g).

• Meaning: g must be false to make R(C) true.

g is a ¬ gate with predecessor gate h: Add clauses (¬g ∨ ¬h) and (g ∨ h).

• Meaning: g ⇔ ¬h.

The Clauses of R(C) (concluded)

g is a ∨ gate with predecessor gates h and h⁰: Add clauses (¬h ∨ g), (¬h⁰ ∨ g), and (h ∨ h⁰ ∨ ¬g).

• Meaning: g ⇔ (h ∨ h⁰).

g is a ∧ gate with predecessor gates h and h⁰: Add clauses (¬g ∨ h), (¬g ∨ h⁰), and (¬h ∨ ¬h⁰ ∨ g).

• Meaning: g ⇔ (h ∧ h⁰).

g is the output gate: Add clause (g).

• Meaning: g must be true to make R(C) true.

Note: If gate g feeds gates h₁, h₂, . . ., then variable g appears in the clauses for h , h , . . . in R(C).

An Example

∧

[_ [_ [_

∨

[_

∧ ¬

∨

K_ J_ K_ K_ J_ K_

J_ J_

J_

(h₁ ⇔ x₁) ∧ (h₂ ⇔ x₂) ∧ (h₃ ⇔ x₃) ∧ (h₄ ⇔ x₄)

∧ [ g₁ ⇔ (h₁ ∧ h₂) ] ∧ [ g₂ ⇔ (h₃ ∨ h₄) ]

∧ [ g₃ ⇔ (g₁ ∧ g₂) ] ∧ (g₄ ⇔ ¬g₂)

∧ [ g₅ ⇔ (g₃ ∨ g₄) ] ∧ g₅.

An Example (concluded)

• In general, the result is a CNF.

• The CNF has size proportional to the circuit’s number of gates.

• The CNF adds new variables to the circuit’s original input variables.

Composition of Reductions

Proposition 25 If R₁₂ is a reduction from L₁ to L₂ and R₂₃ is a reduction from L₂ to L₃, then the composition R₁₂ ◦ R₂₃ is a reduction from L₁ to L₃.

• Clearly x ∈ L₁ if and only if R₂₃(R₁₂(x)) ∈ L₃.

• How to compute R₁₂ ◦ R₂₃ in space O(log n), as required by the definition of reduction?

The Proof (continued)

• An obvious way is to generate R₁₂(x) first and then feeding it to R₂₃.

• This takes polynomial time.^a

– It takes polynomial time to produce R₁₂(x) of polynomial length.

– It also takes polynomial time to produce R₂₃(R₁₂(x)).

• Trouble is R₁₂(x) may consume up to polynomial space, much more than the logarithmic space required.

aHence our concern below disappears had we required reductions to

The Proof (concluded)

• The trick is to let R₂₃ drive the computation.

• It asks R₁₂ to deliver each bit of R₁₂(x) when needed.

• When R₂₃ wants to read the ith bit, R₁₂(x) will be simulated until the ith bit is available.

– The initial i − 1 bits should not be written to the string.

• This is feasible as R₁₂(x) is produced in a write-only manner.

– The ith output bit of R₁₂(x) is well-defined because once it is written, it will never be overwritten by R₁₂.

Completeness

• As reducibility is transitive, problems can be ordered with respect to their difficulty.

• Is there a maximal element?

• It is not altogether obvious that there should be a maximal element.

– Many infinite structures (such as integers and real numbers) do not have maximal elements.

• Hence it may surprise you that most of the complexity classes that we have seen so far have maximal elements.

aCook (1971) and Levin (1971).

Completeness (concluded)

• Let C be a complexity class and L ∈ C.

• L is C-complete if every L⁰ ∈ C can be reduced to L.

– Most complexity classes we have seen so far have complete problems!

• Complete problems capture the difficulty of a class because they are the hardest problems in the class.

Hardness

• Let C be a complexity class.

• L is C-hard if every L⁰ ∈ C can be reduced to L.

• It is not required that L ∈ C.

• If L is C-hard, then by definition, every C-complete problem can be reduced to L.^a

aContributed by Mr. Ming-Feng Tsai (D92922003) on October 15, 2003.

Illustration of Completeness and Hardness

A₁

A₂

A₃

A₄

L A₁

A₂

A₃

A₄ L

Closedness under Reductions

• A class C is closed under reductions if whenever L is reducible to L⁰ and L⁰ ∈ C, then L ∈ C.

• P, NP, coNP, L, NL, PSPACE, and EXP are all closed under reductions.

Complete Problems and Complexity Classes

Proposition 26 Let C⁰ and C be two complexity classes such that C⁰ ⊆ C. Assume C⁰ is closed under reductions and L is C-complete. Then C = C⁰ if and only if L ∈ C⁰.

• Suppose L ∈ C⁰ first.

• Every language A ∈ C reduces to L ∈ C⁰.

• Because C⁰ is closed under reductions, A ∈ C⁰.

• Hence C ⊆ C⁰.

• As C⁰ ⊆ C, we conclude that C = C⁰.

The Proof (concluded)

• On the other hand, suppose C = C⁰.

• As L is C-complete, L ∈ C.

• Thus, trivially, L ∈ C⁰.

Two Important Corollaries

Proposition 26 implies the following.

Corollary 27 P = NP if and only if an NP-complete problem in P.

Corollary 28 L = P if and only if a P-complete problem is in L.

Complete Problems and Complexity Classes

Proposition 29 Let C⁰ and C be two complexity classes closed under reductions. If L is complete for both C and C⁰, then C = C⁰.

• All languages L ∈ C reduce to L ∈ C⁰.

• Since C⁰ is closed under reductions, L ∈ C⁰.

• Hence C ⊆ C⁰.

• The proof for C⁰ ⊆ C is symmetric.

Table of Computation

• Let M = (K, Σ, δ, s) be a single-string polynomial-time deterministic TM deciding L.

• Its computation on input x can be thought of as a

| x |^k × | x |^k table, where | x |^k is the time bound.

– It is a sequence of configurations.

• Rows correspond to time steps 0 to | x |^k − 1.

• Columns are positions in the string of M .

• The (i, j)th table entry represents the contents of position j of the string after i steps of computation.

Some Conventions To Simplify the Table

• M halts after at most | x |^k − 2 steps.

– The string length hence never exceeds | x |^k.

• Assume a large enough k to make it true for | x | ≥ 2.

• Pad the table with F

s so that each row has length | x |^k. – The computation will never reach the right end of

the table for lack of time.

• If the cursor scans the jth position at time i when M is at state q and the symbol is σ, then the (i, j)th entry is a new symbol σ_q.

Some Conventions To Simplify the Table (continued)

• If q is “yes” or “no,” simply use “yes” or “no” instead of σ_q.

• Modify M so that the cursor starts not at ¤ but at the first symbol of the input.

• The cursor never visits the leftmost ¤ by telescoping two moves of M each time the cursor is about to move to the leftmost ¤.

• So the first symbol in every row is a ¤ and not a ¤_q.

Some Conventions To Simplify the Table (concluded)

• Suppose M has halted before its time bound of | x |^k, so that “yes” or “no” appears at a row before the last.

• Then all subsequent rows will be identical to that row.

• M accepts x if and only if the (| x |^k − 1, j)th entry is

“yes” for some position j.

Comments

• Each row is essentially a configuration.

• If the input x = 010001, then the first row is

| x |^k

z }| {

¤0_s10001G G

· · ·G

• A typical row may look like

| x |^k

z }| {

¤10100_q01110100G G

· · ·G

Comments (concluded)

• The last rows must look like

| x |^k

z }| {

¤ · · · “yes” · · · G

or

| x |^k

z }| {

¤ · · · “no” · · ·G

• Three out of the table’s 4 borders are known:

#DEFGHI

#

#





# 

# 

A P-Complete Problem

Theorem 30 (Ladner (1975)) circuit value is P-complete.

• It is easy to see that circuit value ∈ P.

• For any L ∈ P, we will construct a reduction R from L to circuit value.

• Given any input x, R(x) is a variable-free circuit such that x ∈ L if and only if R(x) evaluates to true.

• Let M decide L in time n^k.

• Let T be the computation table of M on x.

The Proof (continued)

• When i = 0, or j = 0, or j = | x |^k − 1, then the value of T_ij is known.

– The jth symbol of x or F

, a ¤, and a F

, respectively.

– Recall that three out of T ’s 4 borders are known.

The Proof (continued)

• Consider other entries T_ij.

• T_ij depends on only T_i−1,j−1, T_i−1,j, and T_i−1,j+1. T_i−1,j−1 T_i−1,j T_i−1,j+1

T_ij

• Let Γ denote the set of all symbols that can appear on the table: Γ = Σ ∪ {σ_q : σ ∈ Σ, q ∈ K}.

• Encode each symbol of Γ as an m-bit number, where^a m = dlog₂ | Γ |e.

aState assignment in circuit design.

The Proof (continued)

• Let the m-bit binary string S_ij1S_ij2 · · · S_ijm encode T_ij.

• We may treat them interchangeably without ambiguity.

• The computation table is now a table of binary entries S_ij`, where

0 ≤ i ≤ n^k − 1, 0 ≤ j ≤ n^k − 1, 1 ≤ ` ≤ m.

The Proof (continued)

• Each bit S_ij` depends on only 3m other bits:

T_i−1,j−1: S_{i−1,j−1,1} S_{i−1,j−1,2} · · · S_{i−1,j−1,m} T_i−1,j: S_i−1,j,1 S_i−1,j,2 · · · S_i−1,j,m T_i−1,j+1: S_i−1,j+1,1 S_i−1,j+1,2 · · · S_i−1,j+1,m

• There is a boolean function F_{`</sub> with 3m inputs such that S<sub>ij`} = F_`(S_{i−1,j−1,1}, S_{i−1,j−1,2}, . . . , S_{i−1,j−1,m},

S_i−1,j,1, S_i−1,j,2, . . . , S_i−1,j,m,

S_i−1,j+1,1, S_i−1,j+1,2, . . . , S_i−1,j+1,m), where for all i, j > 0 and 1 ≤ ` ≤ m.

The Proof (continued)

• These F_i’s depend only on M ’s specification, not on x.

• Their sizes are fixed.

• These boolean functions can be turned into boolean circuits.

• Compose these m circuits in parallel to obtain circuit C with 3m-bit inputs and m-bit outputs.

– Schematically, C(T_i−1,j−1, T_i−1,j, T_i−1,j+1) = T_ij.^a

aC is like an ASIC (application-specific IC) chip.

Circuit C

T i - 1,j - 1

T _ij

T i - 1,j + 1

T _{i - 1,j}

C

The Proof (concluded)

• A copy of circuit C is placed at each entry of the table.

– Exceptions are the top row and the two extreme columns.

• R(x) consists of (| x |^k − 1)(| x |^k − 2) copies of circuit C.

• Without loss of generality, assume the output

“yes”/“no” appear at position (| x |^k − 1, 1).

• Encode “yes” as 1 and “no” as 0.

The Computation Tableau and R(x)

#DEFGHI

#

#





& & & & & &

& & & & & &

& & & & & &

A Corollary

The construction in the above proof yields the following, more general result.

Corollary 31 If L ∈ TIME(T (n)), then a circuit with O(T²(n)) gates can decide if x ∈ L for | x | = n.