### Turing Machines with Multiple Strings

*• A k-string Turing machine (TM) is a quadruple*
*M = (K, Σ, δ, s).*

*• K, Σ, s are as before.*

*• δ : K × Σ*^{k}*→ (K ∪ {h, “yes”, “no”}) × (Σ × {←, →, −})** ^{k}*.

*• All strings start with a .*

*• The ﬁrst string contains the input.*

*• Decidability and acceptability are the same as before.*

*• When TMs compute functions, the output is the last*

### A 2-String TM

δ

#1000110000111001110001110

#111110000

### palindrome Revisited

*• A 2-string TM can decide palindrome in O(n) steps.*

**– It copies the input to the second string.**

**– The cursor of the ﬁrst string is positioned at the ﬁrst**
symbol of the input.

**– The cursor of the second string is positioned at the**
last symbol of the input.

**– The symbols under the cursors are then compared.**

**– The two cursors are then moved in opposite**
directions until the ends are reached.

**– The machine accepts if and only if the symbols under**

δ

#ababbaabbaabbaabbaba

#ababbaabbaabbaabbaba

### palindrome Revisited (concluded)

*• The running times of a 2-string TM and a single-string*
*TM are quadratically related: n*^{2} *vs. n.*

*• This is consistent with the extended Church’s thesis (p.*

66).

**– “Reasonable” models are related polynomially in**
running times.

### Configurations and Yielding

*• The concept of conﬁguration and yielding is the same as*
*before except that a conﬁguration is a (2k + 1)-tuple*

*(q, w*_{1}*, u*_{1}*, w*_{2}*, u*_{2}*, . . . , w*_{k}*, u*_{k}*).*

**– w**_{i}*u*_{i}*is the ith string.*

**– The ith cursor is reading the last symbol of w*** _{i}*.

**– Recall that**

*is each w*

*’s ﬁrst symbol.*

_{i}*• The k-string TM’s initial conﬁguration is*

*(s,*

*2k*

*, x**,**, ,**, , . . . ,**, ).*

Time seemed to be the most obvious measure of complexity.

— Stephen Arthur Cook (1939–)

### Time Complexity

*• The multistring TM is the basis of our notion of the*
time expended by TMs.

*• If a k-string TM M halts after t steps on input x, then*
**the time required by M on input x is t.**

*• If M(x) =, then the time required by M on x is ∞.*

### Time Complexity (concluded)

**• Machine M operates within time f(n) for f : N → N***if for any input string x, the time required by M on x is*
*at most f (| x |).*

**–** *| x | is the length of string x.*

**• Function f(n) is a time bound for M.**

### Time Complexity Classes

^{a}

*• Suppose language L ⊆ (Σ − {*

*})** ^{∗}* is decided by a

*multistring TM operating in time f (n).*

*• We say L ∈ TIME(f(n)).*

*• TIME(f(n)) is the set of languages decided by TMs*

*with multiple strings operating within time bound f (n).*

**• TIME(f(n)) is a complexity class.**

**– palindrome is in TIME(f (n)), where f (n) = O(n).**

*• Trivially, TIME(f(n)) ⊆ TIME(g(n)) if f(n) ≤ g(n) for*
*all n.*

### Juris Hartmanis

^{a}

### (1928–)

### Richard Edwin Stearns

^{a}

### (1936–)

aTuring Award (1993).

### The Simulation Technique

**Theorem 3 Given any k-string M operating within time***f (n), there exists a (single-string) M*^{}*operating within time*
*O(f (n)*^{2}*) such that M (x) = M*^{}*(x) for any input x.*

*• The single string of M*^{}*implements the k strings of M .*

### The Proof

*• Represent conﬁguration (q, w*1*, u*_{1}*, w*_{2}*, u*_{2}*, . . . , w*_{k}*, u** _{k}*) of

*M by this string of M*

*:*

^{}*(q,w*_{1}^{}*u*_{1} * w*_{2}^{}*u*_{2} * · · · w*^{}_{k}*u*_{k}* ).*

**–** * is a special delimiter.*

**– w**_{i}^{}*is w** _{i}* with the ﬁrst

^{a}and last symbols “primed.”

**– It serves the purpose of “,” in a conﬁguration.**^{b}

aThe ﬁrst symbol is of course *.*

bAn alternative is to use (*q, w*_{1}^{}*|u*1 * w*_{2}^{}*|u*2 * · · · w*_{k}^{}*|u*_{k}* ) by*
priming only * in w** _{i}*, where “

*|” is a new symbol.*

### The Proof (continued)

*• The ﬁrst symbol of w*_{i}* ^{}* is the primed version of

*:*

*.*

^{}**– Recall TM cursors are not allowed to move to the left**

of * (p. 23).*

**– Now the cursor of M**^{}*can move between the*
*simulated strings of M .*^{a}

*• The “priming” of the last symbol of each w**i* ensures that
*M*^{}*knows which symbol is under each cursor of M .*^{b}

aThanks to a lively discussion on September 22, 2009.

bAdded because of comments made by Mr. Che-Wei Chang (R95922093) on September 27, 2006.

### The Proof (continued)

*• The initial conﬁguration of M** ^{}* is

*(s,* * *^{}*x*

*k − 1 pairs*

^{}* · · · *^{}* ).*

**–** * ^{}* is double-primed because it is the beginning and
the ending symbol as the cursor is reading it.

^{a}

**– Again, think of it as a new symbol.**

aAdded after the class discussion on September 20, 2011.

### The Proof (continued)

*• We simulate each move of M thus:*

*1. M*^{}*scans the string to pick up the k symbols under*
the cursors.

**– The states of M*** ^{}* must be enlarged to include

*K*

*× Σ*

*to remember them.*

^{k}^{a}

**– The transition functions of M*** ^{}* must also reﬂect it.

*2. M** ^{}* then changes the string to reﬂect the overwriting

*of symbols and cursor movements of M .*

aRecall the TM program on p. 31.

### The Proof (continued)

*• It is possible that some strings of M need to be*
lengthened (see next page).

**– The linear-time algorithm on p. 37 can be used for**
each such string.

*• The simulation continues until M halts.*

*• M*^{}*then erases all strings of M except the last one.*^{a}

aBecause whatever appears on the string of *M** ^{}* will be considered the
output. So

*s and*

^{}*s need to be removed.*

^{}### The Proof (continued)

^{a}

### string 1 string 2 string 3 string 4

### string 1 string 2 string 3 string 4

aIf we interleave the strings, the simulation may be easier. Con-
tributed by Mr. Kai-Yuan Hou (B99201038, R03922014) on September
22, 2015. This is similar to constructing a single-string *multi-track TM*
in, e.g., Hopcroft & Ullman (1969).

### The Proof (continued)

*• Since M halts within time f(| x |), none of its strings*
*ever becomes longer than f (| x |).*^{a}

*• The length of the string of M*^{}*at any time is O(kf (| x |)).*

*• Simulating each step of M takes, per string of M,*
*O(kf (| x |)) steps.*

* – O(f (| x |)) steps to collect information from this*
string.

* – O(kf (| x |)) steps to write and, if needed, to lengthen*
the string.

a

### The Proof (concluded)

*• M*^{}*takes O(k*^{2}*f (| x |)) steps to simulate each step of M*
*because there are k strings.*

*• As there are f(| x |) steps of M to simulate, M** ^{}* operates

*within time O(k*

^{2}

*f (| x |)*

^{2}).

^{a}

aIs the time reduced to *O(kf(| x |)*^{2}) if the interleaving data structure
is adopted?

### Simulation with Two-String TMs

We can do better with two-string TMs.

**Theorem 4 Given any k-string M operating within time***f (n), k > 2, there exists a two-string M*^{}*operating within*
*time O(f (n) log f (n)) such that M (x) = M*^{}*(x) for any input*
*x.*

### Linear Speedup

^{a}

**Theorem 5 Let L***∈ TIME(f(n)). Then for any > 0,*
*L* *∈ TIME(f*^{}*(n)), where f*^{}*(n)* *= f (n) + n + 2.*^{Δ}

See Theorem 2.2 of the textbook for a proof.

aHartmanis & Stearns (1965).

### Implications of the Speedup Theorem

*• State size can be traded for speed.*^{a}

*• If the running time is cn with c > 1, then c can be made*
arbitrarily close to 1.

*• If the running time is superlinear, say 14n*^{2} *+ 31n, then*
the constant in the leading term (14 in this example)
can be made arbitrarily small.

**– Arbitrary linear speedup can be achieved.**^{b}

**– This justiﬁes the big-O notation in the analysis of**
algorithms.

a*m*^{k}*· |Σ|** ^{3mk}*-fold increase to gain a speedup of

*O(m). No free lunch.*

b

### P

*• By the linear speedup theorem, any polynomial time*
*bound can be represented by its leading term n** ^{k}* for

*some k*

*≥ 1.*

*• If L ∈ TIME(n*^{k}*) for some k* **∈ N, it is a polynomially****decidable language.**

**– Clearly, TIME(n*** ^{k}*)

*⊆ TIME(n*

*).*

^{k+1}*• The union of all polynomially decidable languages is*
denoted by P:

P =^{Δ}

*k>0*

*TIME(n*^{k}*).*

Philosophers have explained space.

They have not explained time.

— Arnold Bennett (1867–1931),
*How To Live on 24 Hours a Day (1910)*
I keep bumping into that silly quotation
attributed to me that says
640K of memory is enough.

— Bill Gates (1996)

### Space Complexity

*• Consider a k-string TM M with input x.*

*• Assume non-*

is never written over by
.^{a}

**– The purpose is not to artiﬁcially reduce the space**
needs (see below).

*• If M halts in conﬁguration*

*(H, w*_{1}*, u*_{1}*, w*_{2}*, u*_{2}*, . . . , w*_{k}*, u*_{k}*),*

**then the space required by M on input x is**

*k*
*i=1*

*| w*_{i}*u*_{i}*|.*

### Space Complexity (continued)

*• Suppose we do not charge the space used only for input*
and output.

*• Let k > 2 be an integer.*

**• A k-string Turing machine with input and output***is a k-string TM that satisﬁes the following conditions.*

**– The input string is read-only.**^{a}

**– The cursor on the last string never moves to the left.**

*∗ The output string is essentially write-only.*

**– The cursor of the input string does not wander oﬀ**
into the

s.

### Space Complexity (concluded)

*• If M is a TM with input and output, then the space*
*required by M on input x is*

*k−1*

*i=2*

*| w**i**u*_{i}*|.*

**• Machine M operates within space bound f(n) for***f :* *N → N if for any input x, the space required by M*
*on x is at most f (| x |).*

### Space Complexity Classes

*• Let L be a language.*

*• Then*

*L* *∈ SPACE(f(n))*

*if there is a TM with input and output that decides L*
*and operates within space bound f (n).*

*• SPACE(f(n)) is a set of languages.*

**– palindrome ∈ SPACE(log n).**^{a}

*• A linear speedup theorem similar to the one on p. 93*
exists, so constant coeﬃcients do not matter.

If she can hesitate as to “Yes,”

she ought to say “No” directly.

— Jane Austen (1775–1817),
*Emma (1815)*

### Nondeterminism

^{a}

**• A nondeterministic Turing machine (NTM) is a***quadruple N = (K, Σ, Δ, s).*

*• K, Σ, s are as before.*

*• Δ ⊆ K × Σ × (K ∪ {h, “yes”, “no”}) × Σ × {←, →, −} is*
*a relation, not a function.*^{b}

* – For each state-symbol combination (q, σ), there may*
be multiple valid next steps.

**– Multiple lines of code may be applicable.**

**– But only one will be taken.**

aRabin & Scott (1959).

b D95723006) on September 23,

### Nondeterminism (continued)

*• As before, a program contains lines of code:*

*(q*_{1}*, σ*_{1}*, p*_{1}*, ρ*_{1}*, D*_{1}) *∈ Δ,*
*(q*_{2}*, σ*_{2}*, p*_{2}*, ρ*_{2}*, D*_{2}) *∈ Δ,*

...

*(q*_{n}*, σ*_{n}*, p*_{n}*, ρ*_{n}*, D** _{n}*)

*∈ Δ.*

*• But we cannot write*

*δ(q*_{i}*, σ*_{i}*) = (p*_{i}*, ρ*_{i}*, D** _{i}*)

as in the deterministic case (p. 24) anymore.

### Nondeterminism (concluded)

*• A conﬁguration yields another conﬁguration in one step*
*if there exists a rule in Δ that makes this happen.*

*• There is only a single thread of computation.*^{a}

**– Nondeterminism is not parallelism, multiprocessing,**
multithreading, or quantum computation.

aThanks to a lively discussion on September 22, 2015.

### Michael O. Rabin

^{a}

### (1931–)

aTuring Award (1976).

### Dana Stewart Scott

^{a}

### (1932–)

### Computation Tree and Computation Path

## Ø\HVÙ V

## ØQRÙ

## Ø\HVÙ K

## K

### Decidability under Nondeterminism

*• Let L be a language and N be an NTM.*

**• N decides L if for any x ∈ Σ**^{∗}*, x* *∈ L if and only if there*
is a sequence of valid conﬁgurations that ends in “yes.”

*• In other words,*

**– If x***∈ L, then N(x) = “yes” for some computation*
path.

**– If x***∈ L, then N(x) = “yes” for all computation*
paths.

### Decidability under Nondeterminism (continued)

*• It is not required that the NTM halts in all computation*
paths.^{a}

*• If x ∈ L, no nondeterministic choices should lead to a*

“yes” state.

*• The key is the algorithm’s overall behavior not whether*
it gives a correct answer for each particular run.

*• Note that determinism is a special case of*
nondeterminism.

aSo “accepts” may be a more proper term. Some books use “decides”

### Decidability under Nondeterminism (concluded)

*• For example, suppose L is the set of primes.*^{a}

*• Then we have the primality testing problem.*

*• An NTM N decides L if:*

* – If x is a prime, then N (x) = “yes” for some*
computation path.

**– If x is not a prime, then N (x)***= “yes” for all*
computation paths.

aContributed by Mr. Yu-Ming Lu (R06723032) on March 7, 2019.

### Complementing a TM’s Halting States

*• Let M decide L, and M*^{}*be M after “yes”* *↔ “no”.*

*• If M is a deterministic TM, then M** ^{}* decides ¯

*L.*

**– So M and M*** ^{}* decide languages that complement
each other.

*• But if M is an NTM, then M** ^{}* may not decide ¯

*L.*

**– It is possible that M and M**^{}*accept the same input x*
(see next page).

**– So M and M**^{}*may accept languages that are not*
even disjoint.

### Ø\HVÙ [

### ØQRÙ

### Ø\HVÙ K

### K

### ØQRÙ [

### Ø\HVÙ

### ØQRÙ K

### K

### Time Complexity under Nondeterminism

**• Nondeterministic machine N decides L in time f(n),***where f :* *N → N, if*

**– N decides L, and**

**– for any x***∈ Σ*^{∗}*, N does not have a computation path*
*longer than f (| x |).*

*• We charge only the “depth” of the computation tree.*

### Time Complexity Classes under Nondeterminism

*• NTIME(f(n)) is the set of languages decided by NTMs*
*within time f (n).*

*• NTIME(f(n)) is a complexity class.*

### NP (“Nondeterministic Polynomial”)

*• Deﬁne*

NP =^{Δ}

*k>0*

*NTIME(n*^{k}*).*

*• Clearly P ⊆ NP.*

*• Think of NP as eﬃciently verifiable problems (see p.*

333).

**– Boolean satisﬁability (p. 119 and p. 194).**

*• The most important open problem in computer science*
is whether P = NP.

### Simulating Nondeterministic TMs

Nondeterminism does not add power to TMs.

**Theorem 6 Suppose language L is decided by an NTM N***in time f (n). Then it is decided by a 3-string deterministic*
*TM M in time O(c*^{f(n)}*), where c > 1 is some constant*

*depending on N .*

*• On input x, M goes down every computation path of N*
using depth-ﬁrst search.

**– M does not need to know f (n).**

* – As N is time-bounded, the depth-ﬁrst search will not*
run indeﬁnitely.

### The Proof (concluded)

*• If any path leads to “yes,” then M immediately enters*
the “yes” state.

*• If none of the paths lead to “yes,” then M enters the*

“no” state.

*• The simulation takes time O(c*^{f(n)}*) for some c > 1*
because the computation tree has that many nodes.

**Corollary 7 NTIME(f (n)))***⊆*

*c>1* *TIME(c*^{f(n)}*).*^{a}

aMr. Kai-Yuan Hou (B99201038, R03922014) on October 6, 2015:

*c>1*TIME(*c** ^{f(n)}*)

*⊆ NTIME(f(n)))?*

### NTIME vs. TIME

*• Does converting an NTM into a TM require exploring*
all computation paths of the NTM as done in Theorem 6
(p. 116)?

*• This is a key question in theory with important practical*
implications.

### A Nondeterministic Algorithm for Satisfiability

*φ is a boolean formula with n variables.*

1: **for i = 1, 2, . . . , n do**

2: *Guess x*_{i}*∈ { 0, 1 }; {Nondeterministic choices.}*

3: **end for**

4: *{Veriﬁcation:}*

5: **if φ(x**_{1}*, x*_{2}*, . . . , x*_{n}**) = 1 then**

6: “yes”;

7: **else**

8: “no”;

9: **end if**

### Computation Tree for Satisfiability

Ø\HVÙ

ØQRÙ ØQRÙØ\HVÙ Ø\HVÙ ØQRÙØQRÙØQRÙØ\HVÙ
[_{}

[_{}
[_{}
[_{}

[_{}

[_{}
[_{}
[_{}

### Analysis

*• The computation tree is a complete binary tree of depth*
*n.*

*• Every computation path corresponds to a particular*
truth assignment^{a} out of 2* ^{n}*.

*• Recall that φ is satisﬁable if and only if there is a truth*
*assignment that satisﬁes φ.*

aEquivalently, a sequence of nondeterministic choices.

### Analysis (concluded)

*• The algorithm decides language*

*{ φ : φ is satisﬁable }.*

**– Suppose φ is satisﬁable.**

*∗ There is a truth assignment that satisﬁes φ.*

*∗ So there is a computation path that results in*

“yes.”

**– Suppose φ is not satisﬁable.**

*∗ That means every truth assignment makes φ false.*

*∗ So every computation path results in “no.”*

### The Traveling Salesman Problem

*• We are given n cities 1, 2, . . . , n and integer distance d**ij*

*between any two cities i and j.*

*• Assume d**ij* *= d** _{ji}* for convenience.

* • The traveling salesman problem (tsp) asks for the*
total distance of the shortest tour of the cities.

^{a}

*• The decision version tsp (d) asks if there is a tour with*
*a total distance at most B, where B is an input.*^{b}

aEach city is visited exactly once.

bBoth problems are extremely important. They are equally hard (p. 404 and p. 502).

### A Shortest Path

### A Nondeterministic Algorithm for tsp (d)

1: **for i = 1, 2, . . . , n do**

2: *Guess x**i* *∈ { 1, 2, . . . , n }; {The ith city.}*^{a}

3: **end for**

4: *{Verification:}*

5: * if x*1

*, x*2

*, . . . , x*

*n*are distinct and

_{n−1}*i=1* *d**x*_{i}*,x*_{i+1}**≤ B then**

6: “yes”;

7: **else**

8: “no”;

9: **end if**

aCan be made into a series of log_{2} *n binary choices for each x** _{i}* so
that the next-state count (2) is a constant, independent of input size.

Contributed by Mr. Chih-Duo Hong (R95922079) on September 27, 2006.

### Analysis

*• Suppose the input graph contains at least one tour of*
*the cities with a total distance at most B.*

**– Then there is a computation path for that tour.**^{a}
**– And it leads to “yes.”**

*• Suppose the input graph contains no tour of the cities*
*with a total distance at most B.*

**– Then every computation path leads to “no.”**

aIt does not mean the algorithm will follow that path. It just means such a computation path (i.e., a sequence of nondeterministic choices) exists.

### Remarks on the P = NP Open Problem

^{?}

^{a}

*• Many practical applications depend on answers to the*
P = NP question.^{?}

*• Veriﬁcation of password should be easy (so it is in NP).*

**– A computer should not take a long time to let a user**
log in.

*• A password system should be hard to crack (loosely*
speaking, cracking it should not be in P).

*• It took logicians 63 years to settle the Continuum*
Hypothesis; how long will it take for this one?

a B96902079, R00922018) on

### Nondeterministic Space Complexity Classes

*• Let L be a language.*

*• Then*

*L* *∈ NSPACE(f(n))*

*if there is an NTM with input and output that decides L*
*and operates within space bound f (n).*

*• NSPACE(f(n)) is a set of languages.*

*• As in the linear speedup theorem,*^{a} constant coeﬃcients
do not matter.

aTheorem 5 (p. 93).