A 2-String TM

Turing Machines with Multiple Strings

• A k-string Turing machine (TM) is a quadruple M = (K, Σ, δ, s).

• K, Σ, s are as before.

• δ : K × Σ^k → (K ∪ {h, “yes”, “no”}) × (Σ × {←, →, −})^k.

• All strings start with a .

• The first string contains the input.

• Decidability and acceptability are the same as before.

• When TMs compute functions, the output is the last

A 2-String TM

δ

#1000110000111001110001110

#111110000

palindrome Revisited

• A 2-string TM can decide palindrome in O(n) steps.

– It copies the input to the second string.

– The cursor of the first string is positioned at the first symbol of the input.

– The cursor of the second string is positioned at the last symbol of the input.

– The symbols under the cursors are then compared.

– The two cursors are then moved in opposite directions until the ends are reached.

– The machine accepts if and only if the symbols under

δ

#ababbaabbaabbaabbaba

#ababbaabbaabbaabbaba

palindrome Revisited (concluded)

• The running times of a 2-string TM and a single-string TM are quadratically related: n² vs. n.

• This is consistent with the extended Church’s thesis (p.

66).

– “Reasonable” models are related polynomially in running times.

Configurations and Yielding

• The concept of configuration and yielding is the same as before except that a configuration is a (2k + 1)-tuple

(q, w₁, u₁, w₂, u₂, . . . , w_k, u_k).

– w_iu_i is the ith string.

– The ith cursor is reading the last symbol of w_i. – Recall that  is each w_i’s first symbol.

• The k-string TM’s initial configuration is

(s,

 2k 

, x,, ,, , . . . ,, ).

Time seemed to be the most obvious measure of complexity.

— Stephen Arthur Cook (1939–)

Time Complexity

• The multistring TM is the basis of our notion of the time expended by TMs.

• If a k-string TM M halts after t steps on input x, then the time required by M on input x is t.

• If M(x) =, then the time required by M on x is ∞.

Time Complexity (concluded)

• Machine M operates within time f(n) for f : N → N if for any input string x, the time required by M on x is at most f (| x |).

– | x | is the length of string x.

• Function f(n) is a time bound for M.

Time Complexity Classes

• Suppose language L ⊆ (Σ − {

})^∗ is decided by a multistring TM operating in time f (n).

• We say L ∈ TIME(f(n)).

• TIME(f(n)) is the set of languages decided by TMs

with multiple strings operating within time bound f (n).

• TIME(f(n)) is a complexity class.

– palindrome is in TIME(f (n)), where f (n) = O(n).

• Trivially, TIME(f(n)) ⊆ TIME(g(n)) if f(n) ≤ g(n) for all n.

Juris Hartmanis

(1928–)

Richard Edwin Stearns

(1936–)

aTuring Award (1993).

The Simulation Technique

Theorem 3 Given any k-string M operating within time f (n), there exists a (single-string) M^ operating within time O(f (n)²) such that M (x) = M^(x) for any input x.

• The single string of M^ implements the k strings of M .

The Proof

• Represent configuration (q, w1, u₁, w₂, u₂, . . . , w_k, u_k) of M by this string of M^:

(q,w₁^ u₁
w₂^ u₂
· · ·
w^_ku_k

).

–
is a special delimiter.

– w_i^ is w_i with the first^a and last symbols “primed.”

– It serves the purpose of “,” in a configuration.^b

aThe first symbol is of course .

bAn alternative is to use (q, w₁^|u1
w₂^|u2
· · ·
w_k^ |u_k

) by priming only  in w_i, where “|” is a new symbol.

The Proof (continued)

• The first symbol of w_i^ is the primed version of : ^. – Recall TM cursors are not allowed to move to the left

of  (p. 23).

– Now the cursor of M^ can move between the simulated strings of M .^a

• The “priming” of the last symbol of each wi ensures that M^ knows which symbol is under each cursor of M .^b

aThanks to a lively discussion on September 22, 2009.

bAdded because of comments made by Mr. Che-Wei Chang (R95922093) on September 27, 2006.

The Proof (continued)

• The initial configuration of M^ is

(s,  ^ x

k − 1 pairs

  

^
· · · ^

).

– ^ is double-primed because it is the beginning and the ending symbol as the cursor is reading it.^a

– Again, think of it as a new symbol.

aAdded after the class discussion on September 20, 2011.

The Proof (continued)

• We simulate each move of M thus:

1. M^ scans the string to pick up the k symbols under the cursors.

– The states of M^ must be enlarged to include K × Σ^k to remember them.^a

– The transition functions of M^ must also reflect it.

2. M^ then changes the string to reflect the overwriting of symbols and cursor movements of M .

aRecall the TM program on p. 31.

The Proof (continued)

• It is possible that some strings of M need to be lengthened (see next page).

– The linear-time algorithm on p. 37 can be used for each such string.

• The simulation continues until M halts.

• M^ then erases all strings of M except the last one.^a

aBecause whatever appears on the string of M^ will be considered the output. So ^s and ^s need to be removed.

The Proof (continued)

string 1 string 2 string 3 string 4

string 1 string 2 string 3 string 4

aIf we interleave the strings, the simulation may be easier. Con- tributed by Mr. Kai-Yuan Hou (B99201038, R03922014) on September 22, 2015. This is similar to constructing a single-string multi-track TM in, e.g., Hopcroft & Ullman (1969).

The Proof (continued)

• Since M halts within time f(| x |), none of its strings ever becomes longer than f (| x |).^a

• The length of the string of M^ at any time is O(kf (| x |)).

• Simulating each step of M takes, per string of M, O(kf (| x |)) steps.

– O(f (| x |)) steps to collect information from this string.

– O(kf (| x |)) steps to write and, if needed, to lengthen the string.

a

The Proof (concluded)

• M^ takes O(k²f (| x |)) steps to simulate each step of M because there are k strings.

• As there are f(| x |) steps of M to simulate, M^ operates within time O(k²f (| x |)²).^a

aIs the time reduced to O(kf(| x |)²) if the interleaving data structure is adopted?

Simulation with Two-String TMs

We can do better with two-string TMs.

Theorem 4 Given any k-string M operating within time f (n), k > 2, there exists a two-string M^ operating within time O(f (n) log f (n)) such that M (x) = M^(x) for any input x.

Linear Speedup

Theorem 5 Let L ∈ TIME(f(n)). Then for any  > 0, L ∈ TIME(f^(n)), where f^(n) = f (n) + n + 2.^Δ

See Theorem 2.2 of the textbook for a proof.

aHartmanis & Stearns (1965).

Implications of the Speedup Theorem

• State size can be traded for speed.^a

• If the running time is cn with c > 1, then c can be made arbitrarily close to 1.

• If the running time is superlinear, say 14n² + 31n, then the constant in the leading term (14 in this example) can be made arbitrarily small.

– Arbitrary linear speedup can be achieved.^b

– This justifies the big-O notation in the analysis of algorithms.

am^k · |Σ|^3mk-fold increase to gain a speedup of O(m). No free lunch.

b

P

• By the linear speedup theorem, any polynomial time bound can be represented by its leading term n^k for some k ≥ 1.

• If L ∈ TIME(n^k) for some k ∈ N, it is a polynomially decidable language.

– Clearly, TIME(n^k) ⊆ TIME(n^k+1).

• The union of all polynomially decidable languages is denoted by P:

P =^Δ 

k>0

TIME(n^k).

Philosophers have explained space.

They have not explained time.

— Arnold Bennett (1867–1931), How To Live on 24 Hours a Day (1910) I keep bumping into that silly quotation attributed to me that says 640K of memory is enough.

— Bill Gates (1996)

Space Complexity

• Consider a k-string TM M with input x.

• Assume non-

is never written over by  .^a

– The purpose is not to artificially reduce the space needs (see below).

• If M halts in configuration

(H, w₁, u₁, w₂, u₂, . . . , w_k, u_k),

then the space required by M on input x is

k i=1

| w_iu_i |.

Space Complexity (continued)

• Suppose we do not charge the space used only for input and output.

• Let k > 2 be an integer.

• A k-string Turing machine with input and output is a k-string TM that satisfies the following conditions.

– The input string is read-only.^a

– The cursor on the last string never moves to the left.

∗ The output string is essentially write-only.

– The cursor of the input string does not wander off into the 

s.

Space Complexity (concluded)

• If M is a TM with input and output, then the space required by M on input x is

k−1

i=2

| wiu_i |.

• Machine M operates within space bound f(n) for f : N → N if for any input x, the space required by M on x is at most f (| x |).

Space Complexity Classes

• Let L be a language.

• Then

L ∈ SPACE(f(n))

if there is a TM with input and output that decides L and operates within space bound f (n).

• SPACE(f(n)) is a set of languages.

– palindrome ∈ SPACE(log n).^a

• A linear speedup theorem similar to the one on p. 93 exists, so constant coefficients do not matter.

If she can hesitate as to “Yes,”

she ought to say “No” directly.

— Jane Austen (1775–1817), Emma (1815)

Nondeterminism

• A nondeterministic Turing machine (NTM) is a quadruple N = (K, Σ, Δ, s).

• K, Σ, s are as before.

• Δ ⊆ K × Σ × (K ∪ {h, “yes”, “no”}) × Σ × {←, →, −} is a relation, not a function.^b

– For each state-symbol combination (q, σ), there may be multiple valid next steps.

– Multiple lines of code may be applicable.

– But only one will be taken.

aRabin & Scott (1959).

b D95723006) on September 23,

Nondeterminism (continued)

• As before, a program contains lines of code:

(q₁, σ₁, p₁, ρ₁, D₁) ∈ Δ, (q₂, σ₂, p₂, ρ₂, D₂) ∈ Δ,

...

(q_n, σ_n, p_n, ρ_n, D_n) ∈ Δ.

• But we cannot write

δ(q_i, σ_i) = (p_i, ρ_i, D_i)

as in the deterministic case (p. 24) anymore.

Nondeterminism (concluded)

• A configuration yields another configuration in one step if there exists a rule in Δ that makes this happen.

• There is only a single thread of computation.^a

– Nondeterminism is not parallelism, multiprocessing, multithreading, or quantum computation.

aThanks to a lively discussion on September 22, 2015.

Michael O. Rabin

(1931–)

aTuring Award (1976).

Dana Stewart Scott

(1932–)

Computation Tree and Computation Path

Ø\HVÙ V

ØQRÙ

Ø\HVÙ K

K

Decidability under Nondeterminism

• Let L be a language and N be an NTM.

• N decides L if for any x ∈ Σ^∗, x ∈ L if and only if there is a sequence of valid configurations that ends in “yes.”

• In other words,

– If x ∈ L, then N(x) = “yes” for some computation path.

– If x ∈ L, then N(x) = “yes” for all computation paths.

Decidability under Nondeterminism (continued)

• It is not required that the NTM halts in all computation paths.^a

• If x ∈ L, no nondeterministic choices should lead to a

“yes” state.

• The key is the algorithm’s overall behavior not whether it gives a correct answer for each particular run.

• Note that determinism is a special case of nondeterminism.

aSo “accepts” may be a more proper term. Some books use “decides”

Decidability under Nondeterminism (concluded)

• For example, suppose L is the set of primes.^a

• Then we have the primality testing problem.

• An NTM N decides L if:

– If x is a prime, then N (x) = “yes” for some computation path.

– If x is not a prime, then N (x) = “yes” for all computation paths.

aContributed by Mr. Yu-Ming Lu (R06723032) on March 7, 2019.

Complementing a TM’s Halting States

• Let M decide L, and M^ be M after “yes” ↔ “no”.

• If M is a deterministic TM, then M^ decides ¯L.

– So M and M^ decide languages that complement each other.

• But if M is an NTM, then M^ may not decide ¯L.

– It is possible that M and M^ accept the same input x (see next page).

– So M and M^ may accept languages that are not even disjoint.

Ø\HVÙ [

ØQRÙ

Ø\HVÙ K

K

ØQRÙ [

Ø\HVÙ

ØQRÙ K

K

Time Complexity under Nondeterminism

• Nondeterministic machine N decides L in time f(n), where f : N → N, if

– N decides L, and

– for any x ∈ Σ^∗, N does not have a computation path longer than f (| x |).

• We charge only the “depth” of the computation tree.

Time Complexity Classes under Nondeterminism

• NTIME(f(n)) is the set of languages decided by NTMs within time f (n).

• NTIME(f(n)) is a complexity class.

NP (“Nondeterministic Polynomial”)

• Define

NP =^Δ 

k>0

NTIME(n^k).

• Clearly P ⊆ NP.

• Think of NP as efficiently verifiable problems (see p.

333).

– Boolean satisfiability (p. 119 and p. 194).

• The most important open problem in computer science is whether P = NP.

Simulating Nondeterministic TMs

Nondeterminism does not add power to TMs.

Theorem 6 Suppose language L is decided by an NTM N in time f (n). Then it is decided by a 3-string deterministic TM M in time O(c^f(n)), where c > 1 is some constant

depending on N .

• On input x, M goes down every computation path of N using depth-first search.

– M does not need to know f (n).

– As N is time-bounded, the depth-first search will not run indefinitely.

The Proof (concluded)

• If any path leads to “yes,” then M immediately enters the “yes” state.

• If none of the paths lead to “yes,” then M enters the

“no” state.

• The simulation takes time O(c^f(n)) for some c > 1 because the computation tree has that many nodes.

Corollary 7 NTIME(f (n))) ⊆

c>1 TIME(c^f(n)).^a

aMr. Kai-Yuan Hou (B99201038, R03922014) on October 6, 2015:



c>1TIME(c^f(n)) ⊆ NTIME(f(n)))?

NTIME vs. TIME

• Does converting an NTM into a TM require exploring all computation paths of the NTM as done in Theorem 6 (p. 116)?

• This is a key question in theory with important practical implications.

A Nondeterministic Algorithm for Satisfiability

φ is a boolean formula with n variables.

1: for i = 1, 2, . . . , n do

2: Guess x_i ∈ { 0, 1 }; {Nondeterministic choices.}

3: end for

4: {Verification:}

5: if φ(x₁, x₂, . . . , x_n) = 1 then

6: “yes”;

7: else

8: “no”;

9: end if

Computation Tree for Satisfiability

Ø\HVÙ

ØQRÙ ØQRÙØ\HVÙ Ø\HVÙ ØQRÙØQRÙØQRÙØ\HVÙ [_ 

[_  [_  [_ 

[_ 

[_  [_  [_ 

Analysis

• The computation tree is a complete binary tree of depth n.

• Every computation path corresponds to a particular truth assignment^a out of 2ⁿ.

• Recall that φ is satisfiable if and only if there is a truth assignment that satisfies φ.

aEquivalently, a sequence of nondeterministic choices.

Analysis (concluded)

• The algorithm decides language

{ φ : φ is satisfiable }.

– Suppose φ is satisfiable.

∗ There is a truth assignment that satisfies φ.

∗ So there is a computation path that results in

“yes.”

– Suppose φ is not satisfiable.

∗ That means every truth assignment makes φ false.

∗ So every computation path results in “no.”

The Traveling Salesman Problem

• We are given n cities 1, 2, . . . , n and integer distance dij

between any two cities i and j.

• Assume dij = d_ji for convenience.

• The traveling salesman problem (tsp) asks for the total distance of the shortest tour of the cities.^a

• The decision version tsp (d) asks if there is a tour with a total distance at most B, where B is an input.^b

aEach city is visited exactly once.

bBoth problems are extremely important. They are equally hard (p. 404 and p. 502).

A Shortest Path

A Nondeterministic Algorithm for tsp (d)

1: for i = 1, 2, . . . , n do

2: Guess xi ∈ { 1, 2, . . . , n }; {The ith city.}^a

3: end for

4: {Verification:}

5: if x1, x2, . . . , xn are distinct and _n−1

i=1 dx_i,x_i+1 ≤ B then

6: “yes”;

7: else

8: “no”;

9: end if

aCan be made into a series of log₂ n binary choices for each x_i so that the next-state count (2) is a constant, independent of input size.

Contributed by Mr. Chih-Duo Hong (R95922079) on September 27, 2006.

Analysis

• Suppose the input graph contains at least one tour of the cities with a total distance at most B.

– Then there is a computation path for that tour.^a – And it leads to “yes.”

• Suppose the input graph contains no tour of the cities with a total distance at most B.

– Then every computation path leads to “no.”

aIt does not mean the algorithm will follow that path. It just means such a computation path (i.e., a sequence of nondeterministic choices) exists.

Remarks on the P = NP Open Problem

• Many practical applications depend on answers to the P = NP question.^?

• Verification of password should be easy (so it is in NP).

– A computer should not take a long time to let a user log in.

• A password system should be hard to crack (loosely speaking, cracking it should not be in P).

• It took logicians 63 years to settle the Continuum Hypothesis; how long will it take for this one?

a B96902079, R00922018) on

Nondeterministic Space Complexity Classes

• Let L be a language.

• Then

L ∈ NSPACE(f(n))

if there is an NTM with input and output that decides L and operates within space bound f (n).

• NSPACE(f(n)) is a set of languages.

• As in the linear speedup theorem,^a constant coefficients do not matter.

aTheorem 5 (p. 93).