• In general, the exponential length in n cannot be avoided (p. 207).
• The size of the truth table is also O(n2n).a
aThere are 2n n-bit strings.
Boolean Circuits
• A boolean circuit is a graph C whose nodes are the gates.
• There are no cycles in C.
• All nodes have indegree (number of incoming edges) equal to 0, 1, or 2.
• Each gate has a sort from
{ true, false, ∨, ∧, ¬, x1, x2, . . . }.
– There are n + 5 sorts.
Boolean Circuits (concluded)
• Gates with a sort from { true, false, x1, x2, . . . } are the inputs of C and have an indegree of zero.
• The output gate(s) has no outgoing edges.
• A boolean circuit computes a boolean function.
• A boolean function can be realized by infinitely many equivalent boolean circuits.
Boolean Circuits and Expressions
• They are equivalent representations.
• One can construct one from the other:
¬ [L ¬
[L
[L ∨ [M ∨
[L [M
[L ∧ [M ∧
[L [M
An Example
((x1 x2 ) (x3 x4)) (x3 x4))
x1 x2 x3 x4
• Circuits are more economical because of the possibility of “sharing.”
circuit sat and circuit value
circuit sat: Given a circuit, is there a truth assignment such that the circuit outputs true?
• circuit sat ∈ NP: Guess a truth assignment and then evaluate the circuit.a
circuit value: The same as circuit sat except that the circuit has no variable gates.
• circuit value ∈ P: Evaluate the circuit from the input gates gradually towards the output gate.
aEssentially the same algorithm as the one on p. 117.
Some
aBoolean Functions Need Exponential Circuits
bTheorem 16 For any n ≥ 2, there is an n-ary boolean function f such that no boolean circuits with 2n/(2n) or fewer gates can compute it.
• There are 22n different n-ary boolean functions (p. 197).
• So it suffices to prove that the number of boolean circuits with 2n/(2n) or fewer gates is less than 22n.
aCan be strengthened to “Almost all.”
bRiordan & Shannon (1942); Shannon (1949).
The Proof (concluded)
• There are at most ((n + 5) × m2)m boolean circuits with m or fewer gates (see next page).
• But ((n + 5) × m2)m < 22n when m = 2n/(2n):
m log2((n + 5) × m2)
= 2n
1 − log2 n+54n2 2n
< 2n for n ≥ 2.
m choices
n+5 choices
m choices
Claude Elwood Shannon (1916–2001)
Howard Gardner (1987), “[Shan-non’s master’s thesis is] possibly the most important, and also the most famous, master’s thesis of the cen-tury.”
Comments
• The lower bound 2n/(2n) is rather tight because an upper bound is n2n (p. 199).
• The proof counted the number of circuits.
– Some circuits may not be valid at all.
– Different circuits may also compute the same function.
• Both are fine because we only need an upper bound on the number of circuits.
• We do not need to consider the outgoing edges because they have been counted as incoming edges.a
aIf you prove it by considering outgoing edges, the bound will not be good. (Try it!)
Relations between Complexity Classes
It is, I own, not uncommon to be wrong in theory and right in practice.
— Edmund Burke (1729–1797), A Philosophical Enquiry into the Origin of Our Ideas of the Sublime and Beautiful (1757) The problem with QE is it works in practice, but it doesn’t work in theory.
— Ben Bernanke (2014)
Proper (Complexity) Functions
• We say that f : N → N is a proper (complexity) function if the following hold:
– f is nondecreasing.
– There is a k-string TM Mf such that Mf(x) = f(| x |) for any x.a
– Mf halts after O(| x | + f(| x |)) steps.
– Mf uses O(f (| x |)) space besides its input x.
• Mf’s behavior depends only on | x | not x’s contents.
• Mf’s running time is bounded by f (n).
aThe textbook calls “” the quasi-blank symbol. The use of Mf(x) will become clear in Proposition 17 (p. 217).
Examples of Proper Functions
• Most “reasonable” functions are proper: c, log n , polynomials of n, 2n, √n , n!, etc.
• If f and g are proper, then so are f + g, fg, and 2g.a
• Nonproper functions when serving as the time bounds for complexity classes spoil “theory building.”
– For example, TIME(f (n)) = TIME(2f(n)) for some recursive function f (the gap theorem).b
• Only proper functions f will be used in TIME(f(n)), SPACE(f (n)), NTIME(f (n)), and NSPACE(f (n)).
aFor f (g(n)), we need to add f (n) ≥ n.
bTrakhtenbrot (1964); Borodin (1972). Theorem 7.3 on p. 145 of the textbook proves it.
Precise Turing Machines
• A TM M is precise if there are functions f and g such that for every n ∈ N, for every x of length n, and for every computation path of M ,
– M halts after precisely f(n) steps, and
– All of its strings are of length precisely g(n) at halting.
∗ Recall that if M is a TM with input and output, we exclude the first and last strings.
• M can be deterministic or nondeterministic.
Precise TMs Are General
Proposition 17 Suppose a TMa M decides L within time (space) f (n), where f is proper. Then there is a precise TM M which decides L in time O(n + f (n)) (space O(f (n)), respectively).
• M on input x first simulates the TM Mf associated with the proper function f on x.
• Mf’s output, of length f (| x |), will serve as a
“yardstick” or an “alarm clock.”
aIt can be deterministic or nondeterministic.
The Proof (continued)
• Then M simulates M (x).
• M(x) halts when and only when the alarm clock runs out—even if M halts earlier.
• If f is a time bound:
– The simulation of each step of M on x is matched by advancing the cursor on the “clock” string.
– Because M stops at the moment the “clock” string is exhausted—even if M (x) stops earlier, it is precise.
– The time bound is therefore O(| x | + f(| x |)).
The Proof (concluded)
• If f is a space bound (sketch):
– M simulates M on the quasi-blanks of Mf’s output string.a
– The total space, not counting the input string, is O(f(n)).
– But we still need a way to make sure there is no infinite loop.b
aThis is to make sure the space bound is precise.
bSee the proof of Theorem 24 (p. 235).
Important Complexity Classes
• We write expressions like nk to denote the union of all complexity classes, one for each value of k.
• For example,
NTIME(nk) =Δ
j>0
NTIME(nj).
Important Complexity Classes (concluded)
P =Δ TIME(nk), NP =Δ NTIME(nk), PSPACE =Δ SPACE(nk), NPSPACE =Δ NSPACE(nk),
E =Δ TIME(2kn), EXP =Δ TIME(2nk),
L =Δ SPACE(log n), NL =Δ NSPACE(log n).
Complements of Nondeterministic Classes
• Recall that the complement of L, or ¯L, is the language Σ∗ − L.
– sat complement is the set of unsatisfiable boolean expressions.
• R, RE, and coRE are distinct (p. 156).
– Again, coRE contains the complements of languages in RE, not languages that are not in RE.
• How about coC when C is a complexity class?
The Co-Classes
• For any complexity class C, coC denotes the class { L : ¯L ∈ C }.
• Clearly, if C is a deterministic time or space complexity class, then C = coC.
– They are said to be closed under complement.
– A deterministic TM deciding L can be converted to one that decides ¯L within the same time or space bound by reversing the “yes” and “no” states.a
• Whether nondeterministic classes for time are closed under complement is not known (see p. 109).
aSee p. 153.