### Boolean Functions

*• An n-ary boolean function is a function*

*f : { true, false }*^{n}*→ { true, false }.*

*• It can be represented by a truth table.*

*• There are 2*^{2}* ^{n}* such boolean functions.

**– We can assign** *true or false to f for each of the 2** ^{n}*
truth assignments.

### Boolean Functions (continued)

Assignment Truth value 1 true or false 2 true or false

... ...

2* ^{n}* true or false

*• A boolean expression expresses a boolean function.*

**– Think of its truth values under all possible truth**
assignments.

### Boolean Functions (continued)

*• A boolean function expresses a boolean expression.*

**–**

*T |= φ, literal y**i* *is true in “row” T* *(y*_{1} *∧ · · · ∧ y**n*).^{a}

*∗ The implicant y*1 *∧ · · · ∧ y**n* **is called the minterm**
over *{ x*1*, . . . , x*_{n}*} for T .*

**– The size**^{b} is *≤ n2*^{n}*≤ 2** ^{2n}*.

**– This DNF is optimal for the parity function, for**
example.^{c}

a**Similar to programmable logic array. This is called the table**
**lookup representation (Beigel, 1993).**

bWe count only the literals here.

cDu & Ko (2000).

### Boolean Functions (continued)

*x*_{1} *x*_{2} *f(x*_{1}*, x*_{2})

0 0 1

0 1 1

1 0 0

1 1 1

The corresponding boolean expression:

(*¬x*^{1} *∧ ¬x*^{2}) *∨ (¬x*^{1} *∧ x*^{2}) *∨ (x*^{1} *∧ x*^{2}*).*

### Boolean Functions (concluded)

**Corollary 15 Every n-ary boolean function can be***expressed by a boolean expression of size O(n2*^{n}*).*

*• In general, the exponential length in n cannot be*
avoided (p. 209).

*• The size of the truth table is also O(n2** ^{n}*).

^{a}

aThere are 2^{n}*n-bit strings.*

### Boolean Circuits

**• A boolean circuit is a graph C whose nodes are the****gates.**

*• There are no cycles in C.*

*• All nodes have indegree (number of incoming edges)*
equal to 0, 1, or 2.

**• Each gate has a sort from**

*{ true, false, ∨, ∧, ¬, x*^{1}*, x*_{2}*, . . . }.*

**– There are n + 5 sorts.**

### Boolean Circuits (concluded)

*• Gates with a sort from { true, false, x*1*, x*_{2}*, . . . } are the*
**inputs of C and have an indegree of zero.**

**• The output gate(s) has no outgoing edges.**

*• A boolean circuit computes a boolean function.*

*• A boolean function can be realized by infinitely many*
equivalent boolean circuits.

### Boolean Circuits and Expressions

*• They are equivalent representations.*

*• One can construct one from the other:*

¬ [_{L} ¬

[_{L}

[_{L} ∨ [_{M} ∨

[_{L} [_{M}

[_{L} ∧ [_{M} ∧

[_{L} [_{M}

### An Example

*((x*_{1} * x*_{2} *) (x*_{3} *x*_{4})) *(x*_{3} *x*_{4}))

*x*_{1} *x*_{2} *x*_{3} *x*_{4}

*• Circuits are more economical because of the possibility*
of “sharing.”

### circuit sat and circuit value

**circuit sat: Given a circuit, is there a truth assignment**
such that the circuit outputs true?

*• circuit sat ∈ NP: Guess a truth assignment and then*
evaluate the circuit.^{a}

**circuit value: The same as circuit sat except that the**
circuit has no variable gates.

*• circuit value ∈ P: Evaluate the circuit from the input*
gates gradually towards the output gate.

aEssentially the same algorithm as the one on p. 119.

### Some

^{a}

### Boolean Functions Need Exponential Circuits

^{b}

**Theorem 16 For any n***≥ 2, there is an n-ary boolean*
*function f such that no boolean circuits with 2*^{n}*/(2n) or*
*fewer gates can compute it.*

*• There are 2*^{2}^{n}*diﬀerent n-ary boolean functions (p. 199).*

*• So it suﬃces to prove that the number of boolean*
circuits with 2^{n}*/(2n) or fewer gates is less than 2*^{2}* ^{n}*.

aCan be strengthened to “Almost all.”

bRiordan & Shannon (1942); Shannon (1949).

### The Proof (concluded)

*• There are at most ((n + 5) × m*^{2})* ^{m}* boolean circuits with

*m or fewer gates (see next page).*

*• But ((n + 5) × m*^{2})^{m}*< 2*^{2}^{n}*when m = 2*^{n}*/(2n):*

*m log*_{2}*((n + 5)* *× m*^{2})

= 2^{n}

1 *−* log_{2} _{n+5}^{4n}^{2}
*2n*

*< 2*^{n}*for n* *≥ 2.*

*m choices*

*n+5 choices*

*m choices*

### Claude Elwood Shannon (1916–2001)

Howard Gardner (1987), “[Shan- non’s master’s thesis is] possibly the most important, and also the most famous, master’s thesis of the cen- tury.”

### Comments

*• The lower bound 2*^{n}*/(2n) is rather tight because an*
*upper bound is n2** ^{n}* (p. 201).

*• The proof counted the number of circuits.*

**– Some circuits may not be valid at all.**

**– Diﬀerent circuits may also compute the same**
function.

*• Both are ﬁne because we only need an upper bound on*
the number of circuits.

*• We do not need to consider the outgoing edges because*
they have been counted as incoming edges.^{a}

aIf you prove it by considering outgoing edges, the bound will not be good. (Try it!)

*Relations between Complexity Classes*

It is, I own, not uncommon to be wrong in theory and right in practice.

— Edmund Burke (1729–1797),
*A Philosophical Enquiry into the Origin of Our*
*Ideas of the Sublime and Beautiful (1757)*
The problem with QE is
it works in practice,
but it doesn’t work in theory.

— Ben Bernanke (2014)

### Proper (Complexity) Functions

**• We say that f : N → N is a proper (complexity)****function if the following hold:**

**–** *f is nondecreasing.*

**– There is a k-string TM M*** _{f}* such that

*M*

_{f}*(x) =*

^{f(| x |)}*for any x.*

^{a}

**–** *M*_{f}*halts after O(| x | + f(| x |)) steps.*

**–** *M*_{f}*uses O(f (| x |)) space besides its input x.*

*• M**f*’s behavior depends only on *| x | not x’s contents.*

*• M**f**’s running time is bounded by f (n).*

aThe textbook calls “*” the quasi-blank symbol. The use of M*_{f}*(x)*
will become clear in Proposition 17 (p. 219).

### Examples of Proper Functions

*• Most “reasonable” functions are proper: c, log n,*
*polynomials of n, 2** ^{n}*,

*√n , n!, etc.*

*• If f and g are proper, then so are f + g, fg, and 2** ^{g}*.

^{a}

*• Nonproper functions when serving as the time bounds*
for complexity classes spoil “theory building.”

**– For example, TIME(f (n)) = TIME(2*** ^{f(n)}*) for some

**recursive function f (the gap theorem).**^{b}

*• Only proper functions f will be used in TIME(f(n)),*
*SPACE(f (n)), NTIME(f (n)), and NSPACE(f (n)).*

a*For f (g(n)), we need to add f (n)* *≥ n.*

bTrakhtenbrot (1964); Borodin (1972). Theorem 7.3 on p. 145 of the textbook proves it.

### Precise Turing Machines

**• A TM M is precise if there are functions f and g such***that for every n* *∈ N, for every x of length n, and for*
*every computation path of M ,*

**–** *M halts after precisely f(n) steps,*^{a} and

* – All of its strings are of length precisely g(n) at*
halting.

^{b}

*∗ Recall that if M is a TM with input and output,*
we exclude the ﬁrst and last strings.

*• M can be deterministic or nondeterministic.*

aFully time constructible (Hopcroft & Ullman, 1979).

bFully space constructible (Hopcroft & Ullman, 1979).

### Precise TMs Are General

**Proposition 17 Suppose a TM**^{a} *M decides L within time*
*(space) f (n), where f is proper. Then there is a precise TM*
*M*^{}*which decides L in time O(n + f (n)) (space O(f (n)),*
*respectively).*

*• M*^{}*on input x ﬁrst simulates the TM M** _{f}* associated

*with the proper function f on x.*

*• M*^{f}*’s output, of length f (| x |), will serve as a*

“yardstick” or an “alarm clock.”

aIt can be deterministic or nondeterministic.

### The Proof (continued)

*• Then M*^{}*simulates M (x).*

*• M*^{}*(x) halts when and only when the alarm clock runs*
*out—even if M halts earlier.*

*• If f is a time bound:*

* – The simulation of each step of M on x is matched by*
advancing the cursor on the “clock” string.

**– Because M*** ^{}* stops at the moment the “clock” string

*is exhausted—even if M (x) stops earlier, it is precise.*

**– The time bound is therefore O(**| x | + f(| x |)).

### The Proof (concluded)

*• If f is a space bound (sketch):*

**–** *M*^{}*simulates M on the quasi-blanks of M** _{f}*’s output
string.

^{a}

**– The total space, not counting the input string, is**
*O(f(n)).*

**– But we still need a way to make sure there is no**
inﬁnite loop.^{b}

aThis is to make sure the space bound is precise.

bSee the proof of Theorem 24 (p. 237).

### Important Complexity Classes

*• We write expressions like n** ^{k}* to denote the union of all

*complexity classes, one for each value of k.*

*• For example,*

*NTIME(n** ^{k}*) =

^{Δ}

*j>0*

*NTIME(n*^{j}*).*

### Important Complexity Classes (concluded)

P =^{Δ} *TIME(n*^{k}*),*
NP =^{Δ} *NTIME(n*^{k}*),*
PSPACE =^{Δ} *SPACE(n*^{k}*),*
NPSPACE =^{Δ} *NSPACE(n*^{k}*),*

E =^{Δ} TIME(2^{kn}*),*
EXP =^{Δ} TIME(2^{n}^{k}*),*
NEXP =^{Δ} NTIME(2^{n}^{k}*),*

L =^{Δ} *SPACE(log n),*
NL =^{Δ} *NSPACE(log n).*

### Complements of Nondeterministic Classes

*• Recall that the complement of L, or ¯L, is the language*
Σ^{∗}*− L.*

**–** sat complement is the set of unsatisﬁable boolean
expressions.

*• R, RE, and coRE are distinct (p. 158).*

**– Again, coRE contains the complements of languages***in RE, not languages that are not in RE.*

*• How about coC when C is a complexity class?*

### The Co-Classes

*• For any complexity class C, coC denotes the class*
*{ L : ¯L ∈ C }.*

*• Clearly, if C is a deterministic time or space complexity*
*class, then* *C = coC.*

**– They are said to be closed under complement.**

* – A deterministic TM deciding L can be converted to*
one that decides ¯

*L within the same time or space*bound by reversing the “yes” and “no” states.

^{a}

*• Whether nondeterministic classes for time are closed*
under complement is not known (see p. 111).

aSee p. 155.

### Comments

*• As*

co*C = { L : ¯L ∈ C },*
*L ∈ C if and only if ¯L ∈ coC.*

*• But it is not true that L ∈ C if and only if L
∈ coC.*

**– co***C is not deﬁned as ¯C.*

*• For example, suppose C = {{ 2, 4, 6, 8, 10, . . . }, . . . }.*

*• Then coC = {{ 1, 3, 5, 7, 9, . . . }, . . . }.*

*• But ¯C = 2**{ 1,2,3,... }* *− {{ 2, 4, 6, 8, 10, . . . }, . . . }.*

### The Quantified Halting Problem

*• Let f(n) ≥ n be proper.*

*• Deﬁne*

*H** _{f}* =

^{Δ}

*{ M; x : M accepts input x*

*after at most f (| x |) steps },*

*where M is deterministic.*

*• Assume the input is binary as usual.*

*H*

*f*

*∈ TIME(f(n)*

^{3}

### )

*• For each input M; x, we simulate M on x with an alarm*
*clock of length f (| x |).*

**– Use the single-string simulator (p. 83), the universal**
TM (p. 135), and the linear speedup theorem (p. 93).

**– Our simulator accepts M ; x if and only if M accepts***x before the alarm clock runs out.*

*• From p. 90, the total running time is O(**M**k*_{M}^{2} *f(n)*^{2}),
*where ** _{M}* is the length to encode each symbol or state of

*M and k*

_{M}*is M ’s number of strings.*

*• As **M**k*_{M}^{2} *= O(n), the running time is O(f (n)*^{3}), where
*the constant is independent of M .*

*H*

*f*

*
∈ TIME(f(n/2))*

*• Suppose TM M*^{H}_{f}*decides H*_{f}*in time f (n/2).*

*• Consider machine:*

*D*_{f}*(M )* *{*

**if** *M*_{H}_{f}*(M ; M ) = “yes”*

**then “no”;**

**else “yes”;**

*}*

*• D*^{f}*on input M runs in the same time as M*_{H}* _{f}* on input

*M; M, i.e., in time f(*

^{2n+1}_{2}

*) = f(n), where n = | M |.*

^{a}

aA student pointed out on October 6, 2004, that this estimation for-
*gets to include the time to write down M ; M .*

### The Proof (concluded)

*• First,*

*D*_{f}*(D** _{f}*) = “yes”

*⇒ D**f**; D*_{f}*
∈ H**f*

*⇒ D*^{f}*does not accept D*_{f}*within time f (| D*^{f}*|)*

*⇒ D*^{f}*(D** _{f}*) = “no”

*as D*

_{f}*(D*

_{f}*) runs in time f (| D*

^{f}*|),*a contradiction

*• Similarly, D**f**(D** _{f}*) = “no”

*⇒ D*

*f*

*(D*

*) = “yes.”*

_{f}### The Time Hierarchy Theorem

**Theorem 18 If f (n)***≥ n is proper, then*

*TIME(f (n))* * TIME(f(2n + 1)*^{3}*).*

*• The quantiﬁed halting problem makes it so.*

**Corollary 19 P** * E.*

*• P ⊆ TIME(2*^{n}*) because poly(n)* *≤ 2*^{n}*for n large enough.*

*• But by Theorem 18,*

TIME (2* ^{n}*) TIME

(2* ^{2n+1}*)

^{3}

*⊆ E.*

*• So P E.*

### The Space Hierarchy Theorem

**Theorem 20 (Hennie & Stearns, 1966) If f (n) is***proper, then*

*SPACE(f (n))* * SPACE(f(n) log f(n)).*

**Corollary 21 L** * PSPACE.*

### Nondeterministic Time Hierarchy Theorems

**Theorem 22 (Cook, 1973) NTIME(n*** ^{r}*)

*NTIME(n*

*)*

^{s}*whenever 1*

*≤ r < s.*

**Theorem 23 (Seiferas, Fischer, & Meyer, 1978) If***T*_{1}*(n) and T*_{2}*(n) are proper, then*

*NTIME(T*_{1}*(n))* * NTIME(T*^{2}*(n))*
*whenever T*_{1}*(n + 1) = o(T*_{2}*(n)).*

### The Reachability Method

*• The computation of a time-bounded TM can be*
represented by a directed graph.

*• The TM’s conﬁgurations constitute the nodes.*

*• There is a directed edge from node x to node y if x*
*yields y in one step.*

*• The start node representing the initial conﬁguration has*
zero in-degree.

### The Reachability Method (concluded)

*• When the TM is nondeterministic, a node may have an*
out-degree greater than one.

**– The graph is the same as the computation tree**
earlier.

**– But identical conﬁgurations are merged into one**
node.

*• So M accepts the input if and only if there is a path*
from the start node to a node with a “yes” state.

*• It is the reachability problem.*

### Illustration of the Reachability Method

### yes

### yes Initial

### configuration

### Relations between Complexity Classes

**Theorem 24 Suppose f (n) is proper. Then***1. SPACE(f (n))* *⊆ NSPACE(f(n)),*

*TIME(f (n))* *⊆ NTIME(f(n)).*

*2. NTIME(f (n))* *⊆ SPACE(f(n)).*

*3. NSPACE(f (n))* *⊆ TIME(k*^{log n+f(n)}*).*

*• Proof of 2:*

**– Explore the computation tree of the NTM for “yes.”**

**– Speciﬁcally, generate an f (n)-bit sequence denoting***the nondeterministic choices over f (n) steps.*

### Proof of Theorem 24(2)

*• (continued)*

**– Simulate the NTM based on the choices.**

**– Recycle the space and repeat the above steps.**

**– Halt with “yes” when a “yes” is encountered or “no”**

if the tree is exhausted.

**– Each path simulation consumes at most O(f (n))***space because it takes O(f (n)) time.*

**– The total space is O(f (n)) because space is recycled.**

### Proof of Theorem 24(3)

*• Let k-string NTM*

*M = (K, Σ, Δ, s)*

*with input and output decide L* *∈ NSPACE(f(n)).*

*• Use the reachability method on the conﬁguration graph*
*of M on input x of length n.*

*• A conﬁguration is a (2k + 1)-tuple*

*(q, w*_{1}*, u*_{1}*, w*_{2}*, u*_{2}*, . . . , w*_{k}*, u*_{k}*).*

### Proof of Theorem 24(3) (continued)

*• We only care about*

*(q, i, w*_{2}*, u*_{2}*, . . . , w*_{k−1}*, u*_{k−1}*),*

*where i is an integer between 0 and n for the position of*
*the first cursor.*

*• The number of conﬁgurations is therefore at most*

*| K | × (n + 1) × | Σ |*^{2(k−2)f(n)}*= O(c*^{log n+f(n)}_{1} ) (2)
*for some c*_{1} *> 1, which depends on M.*

*• Add edges to the conﬁguration graph based on M’s*
transition function.

### Proof of Theorem 24(3) (concluded)

*• x ∈ L ⇔ there is a path in the conﬁguration graph from*
the initial conﬁguration to a conﬁguration of the form
*(“yes”, i, . . .).*^{a}

*• This is reachability on a graph with O(c*^{log n+f(n)}_{1} )
nodes.

*• It is in TIME(c*^{log n+f(n)}*) for some c > 1 because*
*reachability ∈ TIME(n*^{j}*) for some j and*

*c*^{log n+f(n)}_{1} *j*

*= (c*^{j}_{1})^{log n+f(n)}*.*

aThere may be many of them.

### Space-Bounded Computation and Proper Functions

*• In the deﬁnition of space-bounded computations earlier*
(p. 109), the TMs are not required to halt at all.

*• When the space is bounded by a proper function f,*
computations can be assumed to halt:

**– Run the TM associated with f to produce a***quasi-blank output of length f (n) ﬁrst.*

**– The space-bounded computation must repeat a**

*conﬁguration if it runs for more than c** ^{log n+f(n)}* steps

*for some c > 1.*

^{a}

aSee Eq. (2) on p. 240.

### Space-Bounded Computation and Proper Functions (concluded)

*• (continued)*

**– So an inﬁnite loop occurs during simulation for a**
*computation path longer than c** ^{log n+f(n)}* steps.

**– Hence we only simulate up to c*** ^{log n+f(n)}* time steps
per computation path.

### A Grand Chain of Inclusions

^{a}

*• It is an easy application of Theorem 24 (p. 237) that*
L *⊆ NL ⊆ P ⊆ NP ⊆ PSPACE ⊆ EXP.*

*• By Corollary 21 (p. 232), we know L PSPACE.*

*• So the chain must break somewhere between L and EXP.*

*• It is suspected that all four inclusions are proper.*

*• But there are no proofs yet.*

aWith input from Mr. Chin-Luei Chang (B89902053, R93922004, D95922007) on October 22, 2004.

### What Is Wrong with the Proof?

^{a}

*• By Theorem 24(2) (p. 237),*
NL *⊆ TIME*

*k*^{O(log n)}

*⊆ TIME (n*^{c}^{1})
*for some c*_{1} *> 0.*

*• By Theorem 18 (p. 231),*

*TIME (n*^{c}^{1}) * TIME (n*^{c}^{2}) *⊆ P*
*for some c*_{2} *> c*_{1}.

*• So*

NL *
= P.*

aContributed by Mr. Yuan-Fu Shao (R02922083) on November 11, 2014.

### What Is Wrong with the Proof? (concluded)

*• Recall from p. 222 that TIME(k** ^{O(log n)}*) is a shorthand

for

*j>0*

TIME

*j*^{O(log n)}*.*

*• So the correct proof runs more like*
NL *⊆*

*j>0*

TIME

*j*^{O(log n)}

*⊆*

*c>0*

*TIME (n*^{c}*) = P.*

*• And*

NL *
= P*
no longer follows.

### Nondeterministic and Deterministic Space

*• By Theorem 6 (p. 116),*

*NTIME(f (n))* *⊆ TIME(c*^{f(n)}*),*
an exponential gap.

*• There is no proof yet that the exponential gap is*
inherent.

*• How about NSPACE vs. SPACE?*

*• Surprisingly, the relation is only quadratic—a*
polynomial—by Savitch’s theorem.

### Savitch’s Theorem

**Theorem 25 (Savitch, 1970)**

*reachability ∈ SPACE(log*^{2} *n).*

*• Let G(V, E) be a graph with n nodes.*

*• For i ≥ 0, let*

*PATH(x, y, i)*

*mean there is a path from node x to node y of length at*
most 2* ^{i}*.

*• There is a path from x to y if and only if*
*PATH(x, y,* *log n)*
holds.

### The Proof (continued)

*• For i > 0, PATH(x, y, i) if and only if there exists a z*
*such that PATH(x, z, i* *− 1) and PATH(z, y, i − 1).*

*• For PATH(x, y, 0), check the input graph or if x = y.*

*• Compute PATH(x, y, log n) with a depth-ﬁrst search*
*on a graph with nodes (x, y, i)s (see next page).*^{a}

*• Like stacks in recursive calls, we keep only the current*
*path’s (x, y, i)s.*

*• The space requirement is proportional to the depth of*
the tree (*log n) times the size of the items stored at*
each node.

aContributed by Mr. Chuan-Yao Tan on October 11, 2011.

### The Proof (continued): Algorithm for PATH *(x, y, i)*

1: **if i = 0 then**

2: **if x = y or (x, y) ∈ E then**

3: **return true;**

4: **else**

5: **return false;**

6: **end if**

7: **else**

8: **for z = 1, 2, . . . , n do**

9: **if PATH(x, z, i − 1) and PATH(z, y, i − 1) then**

10: **return true;**

11: **end if**

12: **end for**

13: **return false;**

14: **end if**

### The Proof (continued)

[\ Q

[] Q ]\ Q

ØÙ

ØÙ ØÙ

### The Proof (concluded)

*• Depth is log n, and each node (x, y, i) needs space*
*O(log n).*

*• The total space is O(log*^{2} *n).*