Every n-ary boolean function can be expressed by a boolean expression of size O(n2 n )

• In general, the exponential length in n cannot be avoided (p. 212).

• The size of the truth table is also O(n2ⁿ).^a

aThere are 2ⁿ n-bit strings.

Boolean Circuits

• A boolean circuit is a graph C whose nodes are the gates.

• There are no cycles in C.

• All nodes have indegree (number of incoming edges) equal to 0, 1, or 2.

• Each gate has a sort from

{ true, false, ∨, ∧, ¬, x¹, x₂, . . . }.

– There are n + 5 sorts.

Boolean Circuits (concluded)

• Gates with a sort from { true, false, x1, x₂, . . . } are the inputs of C and have an indegree of zero.

• The output gate(s) has no outgoing edges.

• A boolean circuit computes a boolean function.

• A boolean function can be realized by infinitely many equivalent boolean circuits.

Boolean Circuits and Expressions

• They are equivalent representations.

• One can construct one from the other:

¬ [_L ¬

[_L

[_L ∨ [_M ∨

[_L [_M

[_L ∧ [_M ∧

[_L [_M

An Example

((x₁ x₂ ) (x₃ x₄)) (x₃ x₄))

x₁ x₂ x₃ x₄

• Circuits are more economical because of the possibility of “sharing.”

circuit sat and circuit value

circuit sat: Given a circuit, is there a truth assignment such that the circuit outputs true?

• circuit sat ∈ NP: Guess a truth assignment and then evaluate the circuit.^a

circuit value: The same as circuit sat except that the circuit has no variable gates.

• circuit value ∈ P: Evaluate the circuit from the input gates gradually towards the output gate.

aEssentially the same algorithm as the one on p. 121.

Some

^a

Boolean Functions Need Exponential Circuits

^b

Theorem 16 For any n ≥ 2, there is an n-ary boolean function f such that no boolean circuits with 2ⁿ/(2n) or fewer gates can compute it.

• There are 2²ⁿ different n-ary boolean functions (p. 202).

• So it suffices to prove that the number of boolean circuits with 2ⁿ/(2n) or fewer gates is less than 2²ⁿ.

aCan be strengthened to “Almost all.”

bRiordan & Shannon (1942); Shannon (1949).

The Proof (concluded)

• There are at most ((n + 5) × m²)^m boolean circuits with m or fewer gates (see next page).

• But ((n + 5) × m²)^m < 2²ⁿ when m = 2ⁿ/(2n):

m log₂((n + 5) × m²)

= 2ⁿ



1 − log_{2 n+5}⁴ⁿ² 2n



< 2ⁿ for n ≥ 2.

m choices

n+5 choices

m choices

Claude Elwood Shannon (1916–2001)

Howard Gardner (1987), “[Shan-non’s master’s thesis is] possibly the most important, and also the most famous, master’s thesis of the cen-tury.”

Comments

• The lower bound 2ⁿ/(2n) is rather tight because an upper bound is n2ⁿ (p. 204).

• The proof counted the number of circuits.

– Some circuits may not be valid at all.

– Different circuits may also compute the same function.

• Both are fine because we only need an upper bound on the number of circuits.

• We do not need to consider the outgoing edges because they have been counted as incoming edges.^a

aIf you prove the theorem by considering outgoing edges, the bound will not be good. (Try it!)

Relations between Complexity Classes

It is, I own, not uncommon to be wrong in theory and right in practice.

— Edmund Burke (1729–1797), A Philosophical Enquiry into the Origin of Our Ideas of the Sublime and Beautiful (1757) The problem with QE is it works in practice, but it doesn’t work in theory.

— Ben Bernanke (2014)

Proper (Complexity) Functions

• We say that f : N → N is a proper (complexity) function if the following hold:

– f is nondecreasing.

– There is a k-string TM M_f such that M_f(x) = ^{f(| x |)} for any x.^a

– M_f halts after O(| x | + f(| x |)) steps.

– M_f uses O(f (| x |)) space besides its input x.

• Mf’s behavior depends only on | x | not x’s contents.

• Mf’s running time is bounded by f (n).

aThe textbook calls “” the quasi-blank symbol. The use of M_f(x) will become clear in Proposition 17 (p. 222).

Examples of Proper Functions

• Most “reasonable” functions are proper: c, log n, polynomials of n, 2ⁿ, √n , n!, etc.

• If f and g are proper, then so are f + g, fg, and 2^g.^a

• Nonproper functions when serving as the time bounds for complexity classes spoil “theory building.”

– For example, TIME(f (n)) = TIME(2^f(n)) for some recursive function f (the gap theorem).^b

• Only proper functions f will be used in TIME(f(n)), SPACE(f (n)), NTIME(f (n)), and NSPACE(f (n)).

aFor f (g(n)), we need to add f (n) ≥ n.

bTrakhtenbrot (1964); Borodin (1972). Theorem 7.3 on p. 145 of the textbook proves it.

Precise Turing Machines

• A TM M is precise if there are functions f and g such that for every n ∈ N, for every x of length n, and for every computation path of M ,

– M halts after precisely f(n) steps,^a and

– All of its strings are of length precisely^b g(n) at halting.^c

∗ Recall that if M is a TM with input and output, we exclude the first and last strings.

• M can be deterministic or nondeterministic.

aFully time constructible (Hopcroft & Ullman, 1979).

bThis strong requirement does not seem needed later.

cFully space constructible (Hopcroft & Ullman, 1979).

Precise TMs Are General

Proposition 17 Suppose a TM^a M decides L within time (space) f (n), where f is proper. Then there is a precise TM M^ which decides L in time O(n + f (n)) (space O(f (n)), respectively).

• M^ on input x first simulates the TM M_f associated with the proper function f on x.

• M^f’s output, of length f (| x |), will serve as a

“yardstick” or an “alarm clock.”

aIt can be deterministic or nondeterministic.

The Proof (continued)

• Then M^ simulates M (x).

• M^(x) halts when and only when the alarm clock runs out—even if M halts earlier.

• If f is a time bound:

– The simulation of each step of M on x is matched by advancing the cursor on the “clock” string.

– Because M^ stops at the moment the “clock” string is exhausted—even if M (x) stops earlier, it is precise.

– The time bound is therefore O(| x | + f(| x |)).

The Proof (concluded)

• If f is a space bound (sketch):

– M^ simulates M on the quasi-blanks of M_f’s output string.^a

– The total space, not counting the input string, is O(f(n)).

– But we still need a way to make sure there is no infinite loop even if M does not halt.^b

aThis is to make sure the space bound is precise.

bSee the proof of Theorem 24 (p. 241).

Important Complexity Classes

• We write expressions like n^k to denote the union of all complexity classes, one for each value of k.

• For example,

NTIME(n^k) =^Δ 

j>0

NTIME(n^j).

Important Complexity Classes (concluded)

P =^Δ TIME(n^k), NP =^Δ NTIME(n^k), PSPACE =^Δ SPACE(n^k), NPSPACE =^Δ NSPACE(n^k),

E =^Δ TIME(2^kn), EXP =^Δ TIME(2^nk), NEXP =^Δ NTIME(2^nk),

L =^Δ SPACE(log n), NL =^Δ NSPACE(log n).

Complements of Nondeterministic Classes

• Recall that the complement of L, or ¯L, is the language Σ^∗ − L.

– sat complement is the set of unsatisfiable boolean expressions.

• R, RE, and coRE are distinct (p. 161).

– Again, coRE contains the complements of languages in RE, not languages that are not in RE.

• How about coC when C is a complexity class?

The Co-Classes

• For any complexity class C, coC denotes the class { L : ¯L ∈ C }.

• Clearly, if C is a deterministic time or space complexity class, then C = coC.

– They are said to be closed under complement.

– A deterministic TM deciding L can be converted to one that decides ¯L within the same time or space bound by reversing the “yes” and “no” states.^a

• Whether nondeterministic classes for time are closed under complement is not known (see p. 113).

aSee p. 158.

Comments

• As

coC = { L : ¯L ∈ C }, L ∈ C if and only if ¯L ∈ coC.

• But it is not true that L ∈ C if and only if L ∈ coC.

– coC is not defined as ¯C.

• For example, suppose C = {{ 2, 4, 6, 8, 10, . . . }, . . . }.

• Then coC = {{ 1, 3, 5, 7, 9, . . . }, . . . }.

• But ¯C = 2{ 1,2,3,... } − {{ 2, 4, 6, 8, 10, . . . }, . . . }.

The Quantified Halting Problem

• Let f(n) ≥ n be proper.

• Define

H_f =^Δ { M; x : M accepts input x after at most f (| x |) steps }, where M is deterministic.

• Assume the input is binary as usual.

H

f

∈ TIME(f(n)

³

)

• For each input M; x, we simulate M on x with an alarm clock of length f (| x |).

– Use the single-string simulator (p. 85), the universal TM (p. 137), and the linear speedup theorem (p. 95).

– Our simulator accepts M ; x if and only if M accepts x before the alarm clock runs out.

• From p. 92, the total running time is O(Mk_M² f(n)²), where _M is the length to encode each symbol or state of M and k_M is M ’s number of strings.

• As Mk_M² = O(n), the running time is O(f (n)³), where the constant is independent of M .

H

f

∈ TIME(f(n/2))

• Suppose TM MH_f decides H_f in time f (n/2).

• Consider machine:

D_f(M ) {

if M_Hf(M ; M ) = “yes”

then “no”;

else “yes”;

}

The Proof (continued)

• MH_f(M ; M ) runs in time f (²ⁿ⁺¹₂ ) = f(n), where n = | M |.^a

• By construction, D^f(M ) runs in the same amount of time as M_Hf(M ; M ), i.e., f (n), where n = | M |.

aMr. Hsiao-Fei Liu (F92922019) and Mr. Hong-Lung Wang (F92922085) pointed out on October 6, 2004, that this estimation (and the text’s Lemma 7.2) forgets to include the time to write down M ; M .

The Proof (concluded)

• First, suppose D^f(D_f) = “yes”.

• This implies

D_f; D_f ∈ H^f.

• Thus D^f does not accept D_f within time f (| D^f |).

• But D^f(D_f) stops in time f (| D^f |) with an answer.

• Hence Df(D_f) = “no”, a contradiction

• Similarly, Df(D_f) = “no” ⇒ Df(D_f) = “yes.”

The Time Hierarchy Theorem

Theorem 18 If f (n) ≥ n is proper, then

TIME(f (n))  TIME(f(2n + 1)³).

• The quantified halting problem makes it so.

Corollary 19 P  E.

• P ⊆ TIME(2ⁿ) because poly(n) ≤ 2ⁿ for n large enough.

• But by Theorem 18,

TIME (2ⁿ)  TIME

(2²ⁿ⁺¹)³

⊆ E.

• So P  E.

The Space Hierarchy Theorem

Theorem 20 (Hennie & Stearns, 1966) If f (n) is proper, then

SPACE(f (n))  SPACE(f(n) log f(n)).

Corollary 21 L  PSPACE.

在文檔中 The Proof (continued) (頁 42-73)