Recursive and Recursively Enumerable Languages

Complements of Recursive Languages

The complement of L, denoted by ¯L, is the language Σ^∗ − L.

Lemma 9 If L is recursive, then so is ¯L.

• Let L be decided by a deterministic M.

• Swap the “yes” state and the “no” state of M.

• The new machine decides ¯L.^a

aRecall p. 118.

Recursive and Recursively Enumerable Languages

Lemma 10 (Kleene’s theorem; Post, 1944) L is recursive if and only if both L and ¯L are recursively enumerable.

• Suppose both L and ¯L are recursively enumerable, accepted by M and ¯M, respectively.

• Simulate M and ¯M in an interleaved fashion.

• If M accepts, then halt on state “yes” because x ∈ L.

• If ¯M accepts, then halt on state “no” because x ∈ L.^a

• The other direction is trivial.

aEither M or ¯M (but not both) must accept the input and halt.

A Very Useful Corollary and Its Consequences

Corollary 11 L is recursively enumerable but not recursive, then ¯L is not recursively enumerable.

• Suppose ¯L is recursively enumerable.

• Then both L and ¯L are recursively enumerable.

• By Lemma 10 (p. 164), L is recursive, a contradiction.

Corollary 12 ¯H is not recursively enumerable.^a

aRecall that ¯H =^Δ { M; x : M(x) = }.

R, RE, and coRE

RE: The set of all recursively enumerable languages.

coRE: The set of all languages whose complements are recursively enumerable.

R: The set of all recursive languages.

• Note that coRE is not RE.

– coRE =^Δ { L : L ∈ RE } = { L : L ∈ RE }.

– RE =^Δ { L : L ∈ RE }.

R, RE, and coRE (concluded)

• R = RE ∩ coRE (p. 164).

• There exist languages in RE but not in R and not in coRE.

– Such as H (p. 144, p. 145, and p. 165).

• There are languages in coRE but not in RE.

– Such as ¯H (p. 165).

• There are languages in neither RE nor coRE.

R coRE RE

H Is Complete for RE

• Let L be any recursively enumerable language.

• Assume M accepts L.

• Clearly, one can decide whether x ∈ L by asking if M : x ∈ H.

• Hence all recursively enumerable languages are reducible to H!

• H is said to be RE-complete.

aPost (1944).

Notations

• The language accepted by TM M is written as L(M).

• If M(x) = for all x, then L(M) = ∅.

• If M(x) is never “yes” nor  (as required by the definition of acceptance), we also let L(M) = ∅.

Nontrivial Properties of Sets in RE

• A property of the recursively enumerable languages can be defined by the set C of all the recursively

enumerable languages that satisfy it.

– The property of finite recursively enumerable languages is

{ L : L = L(M) for a TM M, L is finite }.

– The property of recursiveness is

{ L : L = L(M) for a TM M, L is recursive }.

Nontrivial Properties of Sets in RE (continued)

• A property is trivial if C = RE or C = ∅.

– Answer to a trivial property (about the language a TM accepts) is either always “yes”or always “no.”

– It is either possessed by all recursively enumerable languages or by none.

• Here is a trivial property (always yes): Does the TM accept a recursively enumerable language?^a

• Here is a trivial property (always no): Does the TM accept a language that is finite and infinite?

aOr, L(M) ∈ RE? Formally, { L : L = L(M) for a TM M, L ∈ RE }.

Nontrivial Properties of Sets in RE (continued)

• A property is nontrivial if C = RE and C = ∅.

– In other words, answer to a nontrivial property is

“yes” for some TMs and “no” for others.

– It is possessed by some recursively enumerable languages but not by all.

• Here is a nontrivial property: Does the TM accept an empty language?^a

– Some machines do, but some machines do not.

a L(M) = ∅? That is, does it go into an infinite loop on all inputs?

Nontrivial Properties of Sets in RE (concluded)

• Determining whether a property is trivial may not be trivial!

• Up to now, all nontrivial properties (of recursively enumerable languages) are undecidable.^a

• In fact, Rice’s theorem confirms that.

aSuch as the universal halting problem H^∗ on p. 161.

Rice’s Theorem

Theorem 13 (Rice, 1956) Suppose C = ∅ and C  RE.^a Then the question “L(M) ∈ C?” is undecidable.

• Note that the input is a TM program M.

• Assume that ∅ ∈ C (otherwise, repeat the proof for RE − C.

• Let L ∈ C be accepted by TM ML (recall that C = ∅).

• Let M_H accept the undecidable language H.

– MH exists (p. 144).

The Proof (continued)

• Construct machine M_x(y):

if MH(x) = “yes” then ML(y) else 

• On the next page, we will prove that

x ∈ H if and only if L(Mx) ∈ C. (1) – As a result, the halting problem is reduced to

deciding L(Mx) ∈ C.

– Hence L(M_x) ∈ C must be undecidable,^a and we are done.

aBy Theorem 8 (p. 156).

The Proof (concluded)

• Suppose x ∈ H, i.e., MH(x) = “yes.”

– M_x(y) determines this, and it either accepts y or never halts, depending on whether y ∈ L.

– Hence L(Mx) = L ∈ C.

• Suppose MH(x) =.

– M_x never halts.

– L(Mx) = ∅ ∈ C.

Comments

• Rice’s theorem is about nontrivial properties of the languages accepted by Turing machines.

• It says they are undecidable.

• Rice’s theorem is not about Turing machines’ operations themselves, such as

Does this TM contain 5 states?

Does this TM take more than 1,000 steps on ?

• Both are decidable, and the answers are contingent.

Comments (concluded)

• Rather, it is about

Does this TM accept a language acceptable by one that contains 5 states?

Does this TM accept a language acceptable by one that takes more than 1,000 steps on ?

• Because both properties are nontrivial,^a they are undecidable by Rice’s theorem.

aWhy?

Consequences of Rice’s Theorem

Corollary 14 The following properties of recursively enumerative sets are undecidable.

• Emptiness.

• Nonemptiness.

• Finiteness.

• Recursiveness.

• Σ^∗.

• Regularity.^a

• Context-freedom.^b

aIs it a regular language?

bIs it a context-free language?

Undecidability in Logic and Mathematics

• First-order logic is undecidable (answer to Hilbert’s (1928) Entscheidungsproblem).^a

• Natural numbers with addition and multiplication is undecidable.^b

• Rational numbers with addition and multiplication is undecidable.^c

aChurch (1936).

bRosser (1937).

cRobinson (1948).

Undecidability in Logic and Mathematics (concluded)

• Natural numbers with addition and equality is decidable and complete.^a

• Elementary theory of groups is undecidable.^b

aPresburger’s Master’s thesis (1928), his only work in logic. The direction was suggested by Tarski. Moj¯zesz Presburger (1904–1943) died in a concentration camp during World War II.

bTarski (1949).

Julia Hall Bowman Robinson (1919–1985)

Alfred Tarski (1901–1983)

Boolean Logic

Christianity is either false or true.

— Girolamo Savonarola (1497) Both of us had said the very same thing.

Did we both speak the truth

—or one of us did

—or neither?

— Joseph Conrad (1857–1924), Lord Jim (1900)

Boolean Logic

Boolean variables: x1, x2, . . ..

Literals: x_i, ¬x_i.

Boolean connectives: ∨, ∧, ¬.

Boolean expressions: Boolean variables, ¬φ (negation), φ1 ∨ φ2 (disjunction), φ1 ∧ φ2 (conjunction).

• _n

i=1 φ_i stands for φ1 ∨ φ2 ∨ · · · ∨ φ_n (multiple conjunction).

• _n

i=1 φi stands for φ1 ∧ φ2 ∧ · · · ∧ φn (multiple disjunction).

Boolean Logic (concluded)

Implications: φ1 ⇒ φ2 is a shorthand for ¬φ1 ∨ φ2. Biconditionals: φ1 ⇔ φ2 is a shorthand for

(φ1 ⇒ φ2) ∧ (φ2 ⇒ φ1).

Truth Assignments

• A truth assignment T is a mapping from boolean variables to truth values true and false.

• A truth assignment is appropriate to boolean expression φ if it defines the truth value for every variable in φ.

– { x1 = true, x2 = false } is appropriate to x1 ∨ x2. – { x2 = true, x3 = false } is not appropriate to

x1 ∨ x2.

Satisfaction

• T |= φ means boolean expression φ is true under T ; in other words, T satisfies φ.

• φ1 and φ2 are equivalent, written φ1 ≡ φ2,

if for any truth assignment T appropriate to both of them, T |= φ1 if and only if T |= φ2.

Truth Table

• Suppose φ has n boolean variables.

• A truth table contains 2ⁿ rows.

• Each row corresponds to one truth assignment of the n variables and records the truth value of φ under it.

• A truth table can be used to prove if two boolean expressions are equivalent.

– Just check if they give identical truth values under all appropriate truth assignments.

aPost (1921); Wittgenstein (1922). Here, 1 is used to denote true; 0

A Truth Table

p q p ∧ q

0 0 0

0 1 0

1 0 0

1 1 1

A Second Truth Table

p q p ∨ q

0 0 0

0 1 1

1 0 1

1 1 1

A Third Truth Table

p ¬p

0 1

1 0

Proof of Equivalency by the Truth Table:

p ⇒ q ≡ ¬q ⇒ ¬p

p q p ⇒ q ¬q ⇒ ¬p

0 0 1 1

0 1 1 1

1 0 0 0

1 1 1 1

De Morgan’s Laws

• De Morgan’s laws state that

¬(φ1 ∧ φ2) ≡ ¬φ1 ∨ ¬φ2,

¬(φ¹ ∨ φ²) ≡ ¬φ¹ ∧ ¬φ².

• Here is a proof of the first law:

φ1 φ2 ¬(φ1 ∧ φ2) ¬φ1 ∨ ¬φ2

0 0 1 1

0 1 1 1

1 0 1 1

1 1 0 0

aAugustus DeMorgan (1806–1871) or William of Ockham (1288–

1348).

Conjunctive Normal Forms

• A boolean expression φ is in conjunctive normal form (CNF) if

φ =

n i=1

Ci,

where each clause C_i is the disjunction of zero or more literals.^a

– For example,

(x1 ∨ x2) ∧ (x1 ∨ ¬x2) ∧ (x2 ∨ x3).

• Convention: An empty CNF is satisfiable, but a CNF containing an empty clause is unsatisfiable.

Disjunctive Normal Forms

• A boolean expression φ is in disjunctive normal form (DNF) if

φ =

n i=1

Di,

where each implicant^a or simply term Di is the conjunction of zero or more literals.

– For example,

(x1 ∧ x2) ∨ (x1 ∧ ¬x2) ∨ (x2 ∧ x3).

aD_i implies φ, thus the term.

Clauses and Implicants

• The 

of clauses yields a clause.

– For example,

(x1 ∨ x2) ∨ (x1 ∨ ¬x2) ∨ (x2 ∨ x3)

= x1 ∨ x2 ∨ x1 ∨ ¬x2 ∨ x2 ∨ x3.

• The 

of implicants yields an implicant.

– For example,

(x1 ∧ x2) ∧ (x1 ∧ ¬x2) ∧ (x2 ∧ x3)

= x ∧ x ∧ x ∧ ¬x ∧ x ∧ x .

Any Expression φ Can Be Converted into CNFs and DNFs φ = xj:

• This is trivially true.

φ = ¬φ1 and a CNF is sought:

• Turn φ1 into a DNF.

• Apply de Morgan’s laws to make a CNF for φ.

φ = ¬φ1 and a DNF is sought:

• Turn φ1 into a CNF.

• Apply de Morgan’s laws to make a DNF for φ.

Any Expression φ Can Be Converted into CNFs and DNFs (continued)

φ = φ¹ ∨ φ² and a DNF is sought:

• Make φ1 and φ2 DNFs.

φ = φ1 ∨ φ2 and a CNF is sought:

• Turn φ1 and φ2 into CNFs,^a φ1 =

n1



i=1

Ai, φ2 =

n2



j=1

Bj.

• Set

φ =

n1

 ⁿ²

(Ai ∨ Bj).

Any Expression φ Can Be Converted into CNFs and DNFs (concluded)

φ = φ1 ∧ φ2 and a CNF is sought:

• Make φ1 and φ2 CNFs.

φ = φ1 ∧ φ2 and a DNF is sought:

• Turn φ1 and φ2 into DNFs, φ1 =

n1



i=1

Ai, φ2 =

n2



j=1

Bj.

• Set

φ =

n1



i=1 n2



j=1

(A_i ∧ B_j).

An Example: Turn ¬((a ∧ y) ∨ (z ∨ w)) into a DNF

¬((a ∧ y) ∨ (z ∨ w))

¬(CNF∨CNF)

= ¬(((a) ∧ (y)) ∨ ((z ∨ w)))

¬(CNF)

= ¬((a ∨ z ∨ w) ∧ (y ∨ z ∨ w))

de Morgan

= ¬(a ∨ z ∨ w) ∨ ¬(y ∨ z ∨ w)

de Morgan

= (¬a ∧ ¬z ∧ ¬w) ∨ (¬y ∧ ¬z ∧ ¬w).

Functional Completeness

• A set of logical connectives is called functionally

complete if every boolean expression is equivalent to one involving only these connectives.

• The set { ¬, ∨, ∧ } is functionally complete.

– Every boolean expression can be turned into a CNF, which involves only ¬, ∨, and ∧.

• The sets { ¬, ∨ } and { ¬, ∧ } are functionally complete.^a – By the above result and de Morgan’s laws.

• { nand } and { nor } are functionally complete.^b

aPost (1921).

bPeirce (c. 1880); Sheffer (1913).

Satisfiability

• A boolean expression φ is satisfiable if there is a truth assignment T appropriate to it such that T |= φ.

• φ is valid or a tautology,^a written |= φ, if T |= φ for all T appropriate to φ.

aWittgenstein (1922). Wittgenstein is one of the most important philosophers of all time. Russell (1919), “The importance of ‘tautology’

for a definition of mathematics was pointed out to me by my former pupil Ludwig Wittgenstein, who was working on the problem. I do not know whether he has solved it, or even whether he is alive or dead.”

“God has arrived,” the great economist Keynes (1883–1946) said of him

Satisfiability (concluded)

• φ is unsatisfiable or a contradiction if φ is false under all appropriate truth assignments.

– Or, equivalently, if ¬φ is valid (prove it).

• φ is a contingency if φ is neither a tautology nor a contradiction.

Ludwig Wittgenstein (1889–1951)

Wittgenstein (1922),

“Whereof one cannot speak, thereof one must be silent.”

satisfiability (sat)

• The length of a boolean expression is the length of the string encoding it.

• satisfiability (sat): Given a CNF φ, is it satisfiable?

• Solvable in exponential time on a TM by the truth table method.

• Solvable in polynomial time on an NTM, hence in NP (p. 120).

• A most important problem in settling the “P = NP”^? problem (p. 332).

unsatisfiability (unsat or sat complement) and validity

• unsat (sat complement): Given a boolean expression φ, is it unsatisfiable?

• validity: Given a boolean expression φ, is it valid?

– φ is valid if and only if ¬φ is unsatisfiable.

– φ and ¬φ are basically of the same length.

– So unsat and validity have the same complexity.

• Both are solvable in exponential time by the truth table

Relations among sat, unsat, and validity

Contingent

Valid Unsatisfiable

• The negation of an unsatisfiable expression is a valid expression.

• None of the four problems—satisfiability, unsatisfiability, validity, and contingency—are known to be in P.

Boolean Functions

• An n-ary boolean function is a function

f : { true, false }ⁿ → { true, false }.

• It can be represented by a truth table.

• There are 2²ⁿ such boolean functions.

– We can assign true or false to f for each of the 2ⁿ truth assignments.

Boolean Functions (continued)

Assignment Truth value 1 true or false 2 true or false

... ...

2ⁿ true or false

• A boolean expression expresses a boolean function.

– Think of its truth values under all possible truth assignments.

Boolean Functions (continued)

• A boolean function expresses a boolean expression.

– 

T |= φ, literal yi is true in “row” T(y1 ∧ · · · ∧ y_n).^a

∗ The implicant y1 ∧ · · · ∧ y_n is called the minterm over { x1, . . . , x_n } for T .

– The size^b is ≤ n2ⁿ ≤ 2²ⁿ.

– This DNF is optimal for the parity function, for example.^c

aSimilar to programmable logic array. This is called the table lookup representation (Beigel, 1993).

bWe count only the literals here.

Boolean Functions (continued)

x1 x2 f(x1, x2)

0 0 1

0 1 1

1 0 0

1 1 1

The corresponding boolean expression:

(¬x¹ ∧ ¬x²) ∨ (¬x¹ ∧ x²) ∨ (x¹ ∧ x²).

Boolean Functions (concluded)

Corollary 15 Every n-ary boolean function can be expressed by a boolean expression of size O(n2ⁿ).

• In general, the exponential length in n cannot be avoided (p. 221).

• The size of the truth table is also O(n2ⁿ).^a

aThere are 2ⁿ n-bit strings.

Boolean Circuits

• A boolean circuit is a graph C whose nodes are the gates.

• There are no cycles in C.

• All nodes have indegree (number of incoming edges) equal to 0, 1, or 2.

• Each gate has a sort from

{ true, false, ∨, ∧, ¬, x¹, x², . . . }.

– There are n + 5 sorts.

Boolean Circuits (concluded)

• Gates with a sort from { true, false, x1, x2, . . . } are the inputs of C and have an indegree of zero.

• The output gate(s) has no outgoing edges.

• A boolean circuit computes a boolean function.

• A boolean function can be realized by infinitely many equivalent boolean circuits.

Boolean Circuits and Expressions

• They are equivalent representations.

• One can construct one from the other:

¬ [_L ¬

[_L

[_L ∨ [_M ∨

[_L [_M

[_L ∧ [_M ∧

[_L [_M

An Example

((x₁ x₂ ) (x₃ x₄)) (x₃ x₄))

x₁ x₂ x₃ x₄

• Circuits are potentially more economical because of the possibility of “sharing.”^a

circuit sat and circuit value

circuit sat: Given a circuit, is there a truth assignment such that the circuit outputs true?

• circuit sat ∈ NP: Guess a truth assignment and then evaluate the circuit.^a

circuit value: The same as circuit sat except that the circuit has no variable gates.

• circuit value ∈ P: Evaluate the circuit from the input gates gradually towards the output gate.

aEssentially the same algorithm as the one on p. 120.

Some

Boolean Functions Need Exponential Circuits

Theorem 16 For any n ≥ 2, there is an n-ary boolean function f such that no boolean circuits with 2ⁿ/(2n) or fewer gates can compute it.

• There are 2²ⁿ different n-ary boolean functions (p. 211).

• So it suffices to prove that there are fewer than 2²ⁿ boolean circuits with up to 2ⁿ/(2n) gates.

aCan be strengthened to “Almost all” (Lupanov, 1958).

bRiordan & Shannon (1942); Shannon (1949).

The Proof (concluded)

• There are at most ((n + 5) × m²)^m boolean circuits with m or fewer gates (see next page).

• But ((n + 5) × m²)^m < 2²ⁿ when m = 2ⁿ/(2n):

m log₂((n + 5) × m²)

= 2ⁿ



1 − log_{2 n+5}⁴ⁿ² 2n



< 2ⁿ for n ≥ 2.

m choices

n+5 choices

m choices

Claude Elwood Shannon (1916–2001)

Howard Gardner (1987), “[Shan- non’s master’s thesis is] possibly the most important, and also the most famous, master’s thesis of the cen- tury.”

Comments

• The lower bound 2ⁿ/(2n) is rather tight because an upper bound is n2ⁿ (p. 213).

• The proof counted the number of circuits.

– Some circuits may not be valid at all.

– Different circuits may also compute the same function.

• Both are fine because we only need an upper bound on the number of circuits.

• We do not need to consider the outgoing edges because they have been counted as incoming edges.^a