Cantor’s Theorem (1895)

The point of philosophy is to start with something so simple as not to seem worth stating, and to end with something so paradoxical that no one will believe it.

— Bertrand Russell (1872–1970)

Cantor’s Theorem (1895)

Theorem 9 The set of all subsets of N (2^N) is infinite and not countable.

• Suppose (2^N) is countable with f : N → 2^N being a bijection.^a

• Consider the set B = {k ∈ N : k ∈ f(k)} ⊆ N.

• Suppose B = f(n) for some n ∈ N.

aNote that f(k) is a subset of N.

The Proof (concluded)

• If n ∈ f(n) = B, then n ∈ B, but then n ∈ B by B’s definition.

• If n ∈ f(n) = B, then n ∈ B, but then n ∈ B by B’s definition.

• Hence B = f(n) for any n.

• f is not a bijection, a contradiction.

Georg Cantor (1845–1918)

Kac and Ulam (1968), “[If] one had to name a single person whose work has had the most decisive influence on the present spirit of mathematics, it would almost surely be Georg Cantor.”

Cantor’s Diagonalization Argument Illustrated

f(0) f(1) f(2) f(3) f(4) f(5)

B

0 1 2 3 4 5

A Corollary of Cantor’s Theorem

Corollary 10 For any set T , finite or infinite,

| T | < | 2^T |.

• The inequality holds in the finite T case as k < 2^k.

• Assume T is infinite now.^a

aMr. Kai-Yuan Hou (B99201038, R03922014) on October 13, 2015:

Should we limit T to be countable?

The Proof (concluded)

• | T | ≤ |2^T|.

– Consider f(x) = {x} ∈ 2^T.

– f maps a member of T = { a, b, c, . . . } to the

corresponding member of { { a }, { b }, { c }, . . . } ⊆ 2^T.

• | T | = |2^T|.

– Use the same argument as Cantor’s theorem.

A Second Corollary of Cantor’s Theorem

Corollary 11 The set of all functions on N is not countable.

• It suffices to prove it for functions from N to {0, 1}.

• Every function f : N → {0, 1} determines a subset of N:

{n : f(n) = 1} ⊆ N, and vice versa.

• So the set of functions from N to {0, 1} has cardinality

| 2^N |.

• Cantor’s theorem (p. 139) then implies the claim.

Existence of Uncomputable Problems

• Every program is a finite sequence of 0s and 1s, thus a nonnegative integer.^a

• Hence every program corresponds to some integer.

• The set of programs is therefore countable.

aDifferent binary strings may be mapped to the same integer (e.g.,

“001” and “01”). To prevent it, use the lexicographic order as the map- ping or simply insert “1” as the most significant bit of the binary string before the mapping (so “001” becomes “1001”). Contributed by Mr.

Yu-Chih Tung (R98922167) on October 5, 2010.

Existence of Uncomputable Problems (concluded)

• A function is a mapping from integers to integers.

• The set of functions is not countable by Corollary 11 (p. 145).

• So there are functions for which no programs exist.^a

aAs a nondeterministic program may not compute a function, we consider only deterministic programs for this sentence. Contributed by Mr. Patrick Will (A99725101) on October 5, 2010.

He [Turing] invented the idea of software, essentially[.]

It’s software that’s really the important invention.

— Freeman Dyson (2015)

Universal Turing Machine

• A universal Turing machine U interprets the input as the description of a TM M concatenated with the description of an input to that machine, x.

– Both M and x are over the alphabet of U.

• U simulates M on x so that

U(M; x) = M(x).

• U is like a modern computer, which executes any valid machine code, or a Java virtual machine, which executes any valid bytecode.

aTuring (1936).

The Halting Problem

• Undecidable problems are problems that have no algorithms.

– Equivalently, they are languages that are not recursive.

• We knew undecidable problems exist (p. 146).

• We now define a concrete undecidable problem, the halting problem:

H = {M; x : M(x) =}.

– Does M halt on input x?

H Is Recursively Enumerable

• Use the universal TM U to simulate M on x.

• When M is about to halt, U enters a “yes” state.

• If M(x) diverges, so does U.

• This TM accepts H.

H Is Not Recursive

• Suppose H is recursive.

• Then there is a TM M_H that decides H.

• Consider the program D(M) that calls M_H:

1: if MH(M; M) = “yes” then

2: ; {Writing an infinite loop is easy.}

3: else

4: “yes”;

5: end if

aTuring (1936).

H Is Not Recursive (concluded)

• Consider D(D):

– D(D) =⇒ M_H(D; D) = “yes” ⇒ D; D ∈ H ⇒ D(D) =, a contradiction.

– D(D) = “yes” ⇒ MH(D; D) = “no” ⇒ D; D ∈ H ⇒ D(D) =, a contradiction.

Comments

• Two levels of interpretations of M:^a – A sequence of 0s and 1s (data).

– An encoding of instructions (programs).

• There are no paradoxes with D(D).

– Concepts should be familiar to computer scientists.

– Feed a C compiler to a C compiler, a Lisp interpreter to a Lisp interpreter, a sorting program to a sorting program, etc.

aEckert and Mauchly (1943); von Neumann (1945); Turing (1946).

Cantor’s Paradox

(1899)

• Let T be the set of all sets.^b

• Then 2^T ⊆ T because 2^T is a set.

• But we know^c | 2^T | > | T | (p. 143)!

• We got a “contradiction.”

• Are we willing to give up Cantor’s theorem?

• If not, what is a set?^d

aIn a letter to Richard Dedekind. First published in Russell (1903).

bRecall this ontological argument for the existence of God by St Anselm (1033–1109) in the 11th century: If something is possible but is not part of God, then God is not the greatest possible object of thought, a contradiction.

cReally?

dIt partially answers the question on p. 143n.

Self-Loop Paradoxes

Russell’s Paradox (1901): Consider R = {A : A ∈ A}.

• If R ∈ R, then R ∈ R by definition.

• If R ∈ R, then R ∈ R also by definition.

• In either case, we have a “contradiction.”^b

Eubulides: The Cretan says, “All Cretans are liars.”

Liar’s Paradox: “This sentence is false.”

aE.g., Quine (1966), The Ways of Paradox and Other Essays and Hofstadter (1979), G¨odel, Escher, Bach: An Eternal Golden Braid.

bGottlob Frege (1848–1925) to Bertrand Russell in 1902, “Your dis- covery of the contradiction [. . .] has shaken the basis on which I intended to build arithmetic.”

Self-Loop Paradoxes (continued)

Hypochondriac: a patient (like G¨odel) with imaginary symptoms and ailments.

Sharon Stone in The Specialist (1994): “I’m not a woman you can trust.”

Numbers 12:3, Old Testament: “Moses was the most humble person in all the world [· · · ]” (attributed to Moses).

Self-Loop Paradoxes (concluded)

The Egyptian Book of the Dead: “ye live in me and I would live in you.”

John 14:10, New Testament: “Don’t you believe that I am in the Father, and that the Father is in me?”

John 17:21, New Testament: “just as you are in me and I am in you.”

Pagan & Christian Creeds (1920): “I was moved to Odin, myself to myself.”

Soren Kierkegaard in Fear and Trembling (1843):

“to strive against the whole world is a comfort, to strive

Bertrand Russell (1872–1970)

Karl Popper (1974), “per- haps the greatest philoso- pher since Kant.”

Reductions in Proving Undecidability

• Suppose we are asked to prove that L is undecidable.

• Suppose L^ (such as H) is known to be undecidable.

• Find a computable transformation R (called reduction) from L^ to L such that^a

∀x {x ∈ L^ if and only if R(x) ∈ L}.

• Now we can answer “x ∈ L^?” for any x by asking

“R(x) ∈ L?” because they have the same answer.

• L^ is said to be reduced to L.

aContributed by Mr. Tai-Dai Chou (J93922005) on May 19, 2005.

x R(x) yes/no

R algorithm

for L

algorithm for L

Reductions in Proving Undecidability (concluded)

• If L were decidable, “R(x) ∈ L?” becomes computable and we have an algorithm to decide L^, a contradiction!

• So L must be undecidable.

Theorem 12 Suppose language L₁ can be reduced to

language L₂. If L₁ is undecidable, then L₂ is undecidable.

Undecidability: Special Cases and Subsets

• Suppose L1 can be reduced to L2.

• As the reduction R maps members of L₁ to a subset of L₂,^a we may say L₁ is a “special case” of L₂.^b

• Now suppose L₁ is undecidable and L₁ ⊆ L₂.

• Iis L₂ then undecidable?^c

aBecause R may not be onto.

bContributed by Ms. Mei-Chih Chang (D03922022) and Mr. Kai-Yuan Hou (B99201038, R03922014) on October 13, 2015.

cContributed by Ms. Mei-Chih Chang (D03922022) on October 13, 2015.

Undecidability: Special Cases and Subsets (concluded)

• It depends.

• When L₂ = Σ^∗, L₂ is decidable: Just answer “yes.”

• If L₂ − L₁ is decidable, then L₂ is undecidable – Clearly,

∀x {x ∈ L₁ if and only if x ∈ L₂ − L₁ and x ∈ L₂}.

– Therefore if L₂ were decidable, then L₁ would be.

More Undecidability

• H^∗ = {M : M halts on all inputs}.

– We will reduce H to H^∗.

– Given the question “M; x ∈ H?”, construct the following machine (this is the reduction):^a

Mx(y) {M(x); }

– M halts on x if and only if M_x halts on all inputs.

– In other words, M; x ∈ H if and only if Mx ∈ H^∗. – So if H^∗ were recursive (recall the box for L on p.

161), H would be recursive, a contradiction.

aSimplified by Mr. Chih-Hung Hsieh (D95922003) on October 5, 2006.

Mx ignores its input y; x is part of Mx’s code but not Mx’s input.

More Undecidability (concluded)

• {M; x : there is a y such that M(x) = y}.

• {M; x : the computation M on input x uses all states of M}.

• {M; x; y : M(x) = y}.

Complements of Recursive Languages

The complement of L, denoted by ¯L, is the language Σ^∗ − L.

Lemma 13 If L is recursive, then so is ¯L.

• Let L be decided by M, which is deterministic.

• Swap the “yes” state and the “no” state of M.

• The new machine decides ¯L.^a

aRecall p. 105.

Recursive and Recursively Enumerable Languages

Lemma 14 (Kleene’s theorem) L is recursive if and only if both L and ¯L are recursively enumerable.

• Suppose both L and ¯L are recursively enumerable, accepted by M and ¯M, respectively.

• Simulate M and ¯M in an interleaved fashion.

• If M accepts, then halt on state “yes” because x ∈ L.

• If ¯M accepts, then halt on state “no” because x ∈ L.

• Note that either M or ¯M (but not both) must accept the input and halt.

A Very Useful Corollary and Its Consequences

Corollary 15 L is recursively enumerable but not recursive, then ¯L is not recursively enumerable.

• Suppose ¯L is recursively enumerable.

• Then both L and ¯L are recursively enumerable.

• By Lemma 14 (p. 168), L is recursive, a contradiction.

Corollary 16 ¯H is not recursively enumerable.^a

aRecall that ¯H = {M; x : M(x) =}.

R, RE, and coRE

RE: The set of all recursively enumerable languages.

coRE: The set of all languages whose complements are recursively enumerable.

R: The set of all recursive languages.

• Note that coRE is not RE.

– coRE = { L : L ∈ RE } = { L : L ∈ RE }.

– RE = { L : L ∈ RE }.

R, RE, and coRE (concluded)

• R = RE ∩ coRE (p. 168).

• There exist languages in RE but not in R and not in coRE.

– Such as H (p. 151, p. 152, and p. 169).

• There are languages in coRE but not in RE.

– Such as ¯H (p. 169).

• There are languages in neither RE nor coRE.

R coRE RE

Undecidability in Logic and Mathematics

• First-order logic is undecidable (answer to Hilbert’s (1928) Entscheidungsproblem).^a

• Natural numbers with addition and multiplication is undecidable.^b

• Rational numbers with addition and multiplication is undecidable.^c

aChurch (1936).

bRosser (1937).

cRobinson (1948).

Undecidability in Logic and Mathematics (concluded)

• Natural numbers with addition and equality is decidable and complete.^a

• Elementary theory of groups is undecidable.^b

aPresburger’s Master’s thesis (1928), his only work in logic. The direction was suggested by Tarski. Moj¯zesz Presburger (1904–1943) died in a concentration camp during World War II.

bTarski (1949).

Julia Hall Bowman Robinson (1919–1985)

Alfred Tarski (1901–1983)

Boolean Logic

It seemed unworthy of a grown man to spend his time on such trivialities, but what was I to do? [· · · ] The whole of the rest of my life might be consumed in looking at that blank sheet of paper.

— Bertrand Russell (1872–1970), Autobiography, Vol. I (1967)

Boolean Logic

Boolean variables: x1, x2, . . ..

Literals: xi, ¬xi.

Boolean connectives: ∨, ∧, ¬.

Boolean expressions: Boolean variables, ¬φ (negation), φ1 ∨ φ2 (disjunction), φ1 ∧ φ2 (conjunction).

• _n

i=1 φi stands for φ1 ∨ φ2 ∨ · · · ∨ φn.

• _n

i=1 φi stands for φ1 ∧ φ2 ∧ · · · ∧ φn.

Implications: φ1 ⇒ φ2 is a shorthand for ¬φ1 ∨ φ2.

Biconditionals: φ1 ⇔ φ2 is a shorthand for (φ1 ⇒ φ2) ∧ (φ2 ⇒ φ1).

aGeorge Boole (1815–1864) in 1847.

Truth Assignments

• A truth assignment T is a mapping from boolean variables to truth values true and false.

• A truth assignment is appropriate to boolean expression φ if it defines the truth value for every variable in φ.

– {x₁ = true, x₂ = false} is appropriate to x₁ ∨ x₂. – {x₂ = true, x₃ = false} is not appropriate to

x₁ ∨ x₂.

Satisfaction

• T |= φ means boolean expression φ is true under T ; in other words, T satisfies φ.

• φ₁ and φ₂ are equivalent, written φ₁ ≡ φ₂,

if for any truth assignment T appropriate to both of them, T |= φ₁ if and only if T |= φ₂.

Truth Tables

• Suppose φ has n boolean variables.

• A truth table contains 2ⁿ rows.

• Each row corresponds to one truth assignment of the n variables and records the truth value of φ under that truth assignment.

• A truth table can be used to prove if two boolean expressions are equivalent.

– Just check if they give identical truth values under all appropriate truth assignments.

A Truth Table

p q p ∧ q

0 0 0

0 1 0

1 0 0

1 1 1

A Second Truth Table

p q p ∨ q

0 0 0

0 1 1

1 0 1

1 1 1

A Third Truth Table

p ¬p

0 1

1 0

Proof of Equivalency: p ⇒ q ≡ ¬q ⇒ ¬p

p q p ⇒ q ¬q ⇒ ¬p

0 0 1 1

0 1 1 1

1 0 0 0

1 1 1 1