Introduction to the Theory of Computation
Final Exam Sample Solutions
Problem 1 (20 pts). Consider f (n) = e−n, g(n) = sin n + 2.
(a) (10 pts) Use the definition of Small-O to check whether f (n) = o(g(n)) or not. Do NOT directly calculate the limit. That is, you need to consider/apply the definition of limit.
(b) (10 pts) Use the definition of Big-O to check whether f (n) = O(g(n)) or not.
Solution.
(a) Yes.
Given real number c > 0, by letting n0 = max(1, b− ln cc + 1), we have en0 > e− ln c = 1
c,
where the inequality is from monotonicity and the fact x − 1 < bxc for any real x. Therefore e−n≤ e−n0 < c ≤ c(sin n + 2), ∀n ≥ n0,
where the last inequality follows from sin n ≥ −1.
(b) Yes. Note that f is decreasing, and f (0) = e0 = 1, so f (n) ≤ 1 for n ≥ 1. On the other hand, since sin n ≥ −1, ∀n, we have g(n) ≥ 1, ∀n. Take n0 = 1, c = 1, then f (n) ≤ 1 ≤ 1 · g(n), ∀n ≥ 1.
Common mistake: For (a) you must give n0, and for (b) you must give c and n0. If not, it’s very likely you don’t get any point.
Problem 2 (10 pts). Suppose f (n) = 2O(n3). Is f (n)3 = 2O(n3)? Please provide detailed explanation.
Solution. Yes. By definition,
∃c, n0 ∈ N s.t. ∀n ≥ n0, f (n) ≤ 2cn3. Take n00 = n0, c0 = 3c. Then
∀n ≥ n00, f (n)3 ≤ (2cn3)3 = 23cn3 = 2c0n3, i.e. f (n)3 = 2O(n3).
Problem 3 (20 pts). Consider the language {w#w | w ∈ {0, 1}∗}.
(a) (15 pts) Design a 2-tape TM with no more than 6 states (including qreject) to recognize the language.
Note that to simplify the figure, you don’t need to draw qreject and the transitions going to it. The strategy should be to
• copy the first w to the second tape
• move head of the second tape to the beginning
• compare contents in the two tapes We require that Γ = {0, 1, #, t}. Note that
• For the two-tape TM, we have
δ : Q × Γ2 → Q × Γ2× {L, R, S}2
• There is no t before the input (b) (5 pts) Simulate
01#01
by using the machine obtained in (a) and the standard TM in Example 3.9 of textbook (see Figure 1). Compare the number of steps needed by the two machines. You need to show every step of the simulation.
Figure 1: State diagram for the Turing machine in Example 3.9
Solution.
(a) We describe the TM with the following state diagram
q1
q2 q3 q4
qaccept 0/t → S/R
1/t → S/R
#/t → S/R
0/t → 0/0, R/R 1/t → 1/1, R/R
#/t → #/#, S/L
#/0 → S/L
#/1 → S/L
#/t → R/R
0/0 → R/R 1/1 → R/R
t/# → S/S
The idea is described as follows.
• At q1, we move the head of the second tape to the right for one step (the transition to q2). By doing so, there will be a leading t to indicate the beginning of the second tape.
• At q2, we repeatedly copy the contents on the first tape to the second tape, and stop copying until the # is met (the transition to q3).
• At q3, we repeatedly move head of the second tape until reaching the beginning t (the transition to q4).
• At q4, we repeatedly check if the right half of the input string on the first tape equals the copied left half on the second tape. We accept if the two heads meet the end of strings at the same time (the transition to qaccept).
Common mistake: If you have a configuration q0 · · · and a transition
δ(q, 0) = (0, L), then the next configuration should still be
q0 · · · rather than
q t 0 · · · .
(b) We give the sequence of running 01#01 on the two-tape TM as follows.
Step 0 → q1 0 1 # 0 1 t
Step 1 → q2 0 1 # 0 1 t q2 t
Step 2 → 0 q2 1 # 0 1 t 0 q2 t Step 3 → 0 1 q2 # 0 1
t 0 1 q2 t Step 4 → 0 1 q3 # 0 1
t 0 q3 1 # Step 5 → 0 1 q3 # 0 1
t q3 0 1 # Step 6 → 0 1 q3 # 0 1
q3 t 0 1 # Step 7 → 0 1 # q4 0 1
t q4 0 1 # Step 8 → 0 1 # 0 q4 1
t 0 q4 1 #
Step 9 → 0 1 # 0 1 q4 t t 0 1 q4 #
Step 10 → 0 1 # 0 1 qaccept t
t 0 1 qaccept #
The sequence of running on the Turing machine in example 3.9 is shown as follows.
q101#01 xq21#01 x1q2#01 x1#q401 x1q6#x1 xq71#x1 q7x1#x1 xq11#x1 xxq3#x1 xx#q5x1 xx#xq51 xx#q6xx xxq6#xx xq7x#xx xxq1#xx xx#q8xx xx#xq8x xx#xxq8t xx#xx t qaccept The former takes 10 steps, while the latter takes 18 steps.
Problem 4 (20 pts). Consider the language {wwR| w ∈ {0, 1}∗}, where wR is the reverse of w.
(a) (15 pts) Give a two-tape non-deterministic TM with no more than 5 states (including qreject) to recognize this language. Note that here we allow S (stay moves) to make the problem easier.
(b) (5 pts) Simulate the path (not the entire tree) that leads to the acceptance of 0110.
Solution.
(a) See the following NTM.
q1 q2 q3 qaccept
0/t → S/R 1/t → S/R
0/t → 0/0, R/R 1/t → 1/1, R/R
0/t → S/L 1/t → S/L
0/0 → R/L 1/1 → R/L
t/t → S/S
t/t → S/S
The idea is described as follows.
• At q1, we either accept (the transition to qaccept), or move the head of the second tape to the right (the transition to q2).
• At q2, we repeatedly copy the contents on the first tape to the second tape, and non-deterministically guess the middle of the input string and stop copying (the transition to q3).
• At q3, we repeatedly check if the right half of the input string on the first tape (reading left to right) equals the copied left half on the second tape (reading right to left). This is true if and only if the input is in the form wwR. So we accept if the two heads meet ts at the same time (the transition to qaccept).
(b) See the following sequence.
Step 0 → q1
0 1 1 0 t
Step 1 → q2 0 1 1 0 t q2 t Step 2 → 0 q2 1 1 0
t 0 q2 t Step 3 → 0 1 q2 1 0
t 0 1 q2 t Step 4 → 0 1 q3 1 0
t 0 q3 1 t Step 5 → 0 1 1 q3 0
t q3 0 1 t Step 6 → 0 1 1 0 q3 t
q3 t 0 1 t
Step 7 → 0 1 1 0 qaccept t
qaccept t 0 1 t
Common mistake: You cannot assume a t before the input.
Problem 5 (30 pts). A CNF hV, Σ, R, Si satisfies that R contains only the following types of rules, Type-V : A → BC
Type-T : A → a Type-E : S → where a ∈ Σ, A, B, C ∈ V , and B, C 6= S. Let
ICN F = {hGi | G is a CNF and |L(G)| = ∞}
be the set of CNFs that generates an infinite number of strings, where |L(G)| is the number of strings in the language L(G) of G. Various ways may be possible to prove that ICN F is decidable, but we would like you to answer the following sub-problems for a step-by-step proof.
(a) (5 pts) Consider a CNF G = hV, Σ, R, Si. If now we consider only type-V rules, then the partial parse trees (not involving type-T and type-E rules) are binary trees where every node is a variable.
For example, let
Ga= h{S, A, B}, {a}, R, Si, where R contains the following rules.
S → AB | A → BB | a B → AA An example of partial parse trees is shown below.
S
A
B A
A B
B A
A
Figure 2: An example of partial parse tree
We say a partial parse tree generates a string w if it can be extended (with all rules in R) to a parse tree for w. For instance, the above partial parse tree can be extended to the following parse tree generating aaaaaa. (Note that we say “if it can be.” That is, a partial parse tree may not be able to generate any string; see sub-problem (b).)
S
A
B A a
A a
B A a
A a
B A a
A a
We define the height of a partial parse tree to be the maximum number of edges in paths starting from the root. Note that the height of the example in Figure 2 is 3.
By the similar reasoning of pumping lemma, if some non-S variable appears at least twice in a path from root, then the looping part between the two occurrences can be repeated as many times as we want, and therefore the partial parse tree can be extended to as large height as we want, and vice versa. For example, the partial parse tree shown above contains paths S − A − B − A where A appears twice, and indeed hGai ∈ ICN F.
• Show that any partial parse tree contains an odd number of nodes.
• For any given CNF hV, Σ, R, Si, find an odd value z so that for any of its partial parse tree, if
#nodes ≥ z,
then some non-S variable must appear at least twice in a path from the root of the partial parse tree.
Hint: You may directly use the fact that if the height of a binary tree is no more than h, then
#nodes ≤ 2h+1− 1. (1)
(b) (5 pts) We are about to argue that partial parse trees generate long strings if the height is large, but there is one subtle issue that needs consideration. Consider
Gb = h{S, A, B}, {b}, R, Si, where R contains the following rules.
S → AB | b A → AA B → b
Gb can have a partial parse tree of height as large as we want (by repeatedly applying the rule A → AA), but L(Gb) = {b}! This is because A is non-deriving, i.e. A cannot derive any string of literals. For example, the following partial parse tree cannot generate any string.
S
A A
A B
Therefore we would like to remove all the non-deriving variables before our argument. Assume we already have a decider D for ECN F = {hGi | G is a CNF and L(G) = Ø}. Use D to design an algorithm that finds out all non-deriving variables of a given CNF.
(c) (5 pts) Given a CNF G, we can apply the algorithm in (b) to find out all the non-deriving variables, remove them from V , and remove all the rules involving any of them, getting a new CNF G0. It’s clear that L(G) = L(G0). Because obtaining G0 from G is a decidable procedure, we can now assume that the CNF G considered has no non-deriving variable.
Show that for a CNF without non-deriving variables, any partial parse tree of height h > 0 can generate at least one string, and that the strings w it generates satisfy |w| ≥ h + 1.
(d) (10 pts) Wrapping up our previous results, given a CNF G without non-deriving variables (by (b)), if there are enough nodes in any partial parse tree of G, by (a) the partial parse tree can be extended to as large height as we want, and therefore by (c) it can generate strings as long as we want.
Use the above properties to derive an algorithm that decides ICN F.
(e) (5 pts) Run your algorithm in (d) on hGai in (a) and hGbi in (b). You don’t need to provide details of running the algorithm in (b).
Solution.
(a) Because the smallest partial parse tree contains only the root S, and each type-V rule applied adds two nodes to the tree, the number of nodes is always odd.
From (1), if
#nodes ≥ 2n+1, then
height ≥ n + 1.
That is, there is a path P from root that contains at least n + 2 nodes. Take n = |V | − 1. The path now contains at least |V | + 1 nodes. Because S can only appear at the root, P contains at least
|V | non-S variables. There are only |V | − 1 non-S variables, so by pigeonhole principle some non-S variable must appear at least twice in the path P .
But |V | > 0 implies 2|V | is even. Therefore we consider trees with at least 2|V |+ 1 nodes.
(b) The following algorithm finds out all the non-deriving variables because A is non-deriving if and only if GA= hV, Σ, R, Ai cannot generate any string of literals, in which case GA is accepted by D.
Input: CNF hV, Σ, R, Si Let VN = Ø;
for A ∈ V do
Let GA= hV, Σ, R, Ai;
if D accepts hGAi then add A to VN; end
return VN;
Remark. The decider for ECN F in the textbook actually finds out all non-deriving variables during the process. Thus instead of calling it |V | times, in fact one call is enough.
(c) Because no variable is non-deriving, we can extend every variable at the leaves of the partial parse tree to a string of literals, and this gives a string generated by the partial parse tree.
Because a partial parse tree involves only type-V rules, applying one rule adds two children to the tree, which increases the number of leaf nodes (of the current tree) by one. A partial parse tree of height h contains a path of h edges, corresponding to h applications of rules. Because the height-0 tree with only the root S has one leaf, for the partial parse tree considered, the number of leaves is at least h + 1. Since h > 0, by the definition of CNF, each leaf variable of the partial parse tree is not S, and each leaf variable cannot yield . So the leaf variables must derive non-empty strings of literals, and thus the generated string is of length at least h + 1.
Common mistake: Some forget to answer the question that any partial parse tree of height h > 0 can generate at least one string.
(d) The algorithm works as follows.
Input: CNF hV, Σ, R, Si
Remove non-deriving variables found by the algorithm in (b), getting CNF hV0, Σ, R0, S0i;
for each partial parse tree T with at most 2|V0|+ 1 nodes do if the height of T is at least |V0| then accept;
end reject;
The algorithm always halts because the number of partial parse trees with at most 2|V0|+ 1 nodes is finite.
Common mistake: According to the problem statement, you need to use “the above properties” to derive the algorithm.
(e)
• hGai
There is no non-deriving variable. So G0 = G, V0 = V , and |V0| = |V | = 3. The partial parse tree provided in (a) has 9 = 23+ 1 nodes, and its height is 3 = |V0|, so hGai is accepted.
• hGbi
A is non-deriving, so after removing A and rules involving A we obtain G0b = h{S, B}, {b}, R0, Si, where R0 contains the following rules.
S → b B → b
Now |V0| = 2, and the only partial parse tree with no more than 22+1 = 5 nodes is the height-0 tree with only the root S. So hGbi is rejected.
