**ELEMENTS OF THE ** **THEORY OF **

**COMPUTATION **

**Second Edition **

**Harry R. Lewis **

*Gordon McKay Professor of Computer Science * *Harvard University *

*and Dean of Harvard College * *Cambridge, Massachusetts *

**Christos H. Papadimitriou **

### C. *Lester Hogan Professor of Electrical Engineering * *and Computer Science *

*University of California * *Berkeley, California *

### PRENTICE-HALL, Upper Saddle River, New Jersey 07458

Library of Congress Cataloging-in-PubJication Data

Lewis, Harry R.

Elements of the theory of computation I Harry R. Lewis and Christos H. Papadimitriou. - 2nd ed.

p. em.

Includes bibliological references and index.

ISBN: 0-13-26247&-8

I. Machine theory. 2. Formal languages. 3. Computational complexity. 4. Logic, Symbolic and mathematical.

I. Papadimitriou. Christos H. II. Title.

QA267.L49 1998

511.3--<1c21 97-13879

Publisher: Alan Apt

Development Editor: Sondra Chavez

EditorialJProduction Supervision: Barbara Kraemer Managing Editor: Bayani Mendoza DeLeon Editor-in-Chief: Marcia Horton

OP

Assistant Vice President of Production and Manufacturing: David W. Riccardi Art Director: Jayne Conte

Manufacturing Manager: Trudy Pisciotti Manufacturing Buyer: Donna Sullivan Editorial Assistant: Toni Holm

©1998 by Prentice-Hall, Inc.

Simon & Schuster I A Viacom Company Upper Saddle River, New Jersey 07458

All rights reserved. No part of this book may be reproduced, in any form or by any means, without permission in writing from the publisher.

The author and publisher of this book have used their best efforts in preparing this book. These efforts include the development, research, and testing of the theories and programs to determine their effectiveness.

The author and publisher make no warranty of any kind, expressed or implied, with regard to these programs or the documentation contained in this book. The author and publisher shall not be liable in any event for incidental or consequential damages in connection with, or arising out of. the furnishing. performance. or use of these programs.

Printed in the United States of America 1 0 9 8 7 6 5 4 3 2

**ISBN ** **0-13-262478-8 **

Prentice-Hall International (UK) Limited, London Prentice-Hall of Australia Pty. Limited, Sydney Prentice-Hall Canada Inc., Toronto

Prentice-Hall Hispanoamericana, S.A., Mexico Prentice-Hall of India Private Limited, New Delhi Prentice-Hall of Japan, Inc., Tokyo

Simon & Schuster Asia Pte. Ltd., Singapore Editora Prentice-Hall do Brasil, Ltda., Rio de Janeiro

*To our daughters *

**Contents **

**Preface to the First Edition **
**Preface to the Second Edition **
**Introduction **

1 Sets, **Relations, and Languages **
1.1 Sets 5

1.2 Relations and functions 9

1.3 Special types of binary relations 13 1.4 Finite and infinite sets 20

1.5 Three fundamental proof techniques 23 1.6 Closures and algorithms 30

1. 7 Alphabets and languages 42

1.8 Finite representations of languages 47 References 52

**vii **

**ix **
1
5

**2 Finite Automata ** **55 **

2.1 Deterministic finite automata 55 2.2 Nondeterministic finite automata 63 2.3 Finite automata and regular expressions 75 2.4 Languages that are and are not regular 86 2.5 State minimization 92

2.6 Algorithmic aspects of finite automata 102 References 110

**3 Context-free Languages ** **113 **

3.1 Context-free grammars 113 3.2 Parse trees 122

3.3 Pushdown automata 130

3.4 Pushdown automata and context-free grammars 136 3.5 Languages that are and are not context-free 143 3.6 Algorithms for context-free grammars 150

3.7 Determinism and parsing 158 References 175

4 Turing machines

4.1 The definition of a Turing machine 179 4.2 Computing with Turing machines 194 4.3 Extensions of Turing machines 200 4.4 Random access Turing machines 210 4.5 Nondeterministic Turing machines 221 4.6 Grammars 227

4.7 Numerical functions 233 References 243

179

5 Undecidability 245

5.1 The Church-Turing thesis 245 5.2 Universal Turing machines 247 5.3 The halting problem 251

5.4 Unsolvable problems about Turing machines 254 5.5 Unsolvable problems about grammars 258 5.6 An unsolvable tiling problem 262

5.7 Properties of recursive languages 267 References 272

6 Computational Complexity 275

*6.1 The class P * *275 *

6.2 Problems, problems. . . 278 6.3 Boolean satisfiability 288 6.4 The class

*NP *

*292*References 299

7 NP-completeness 301

7.1 Polynomial-time reductions 301 7.2 Cook's Theorem 309

7.3 More NP-complete problems 7.4 Coping with NP-completeness References 350

Index

317 333

353

**Preface to the First Edition **

This book is an introuuction, on the undergraduate level, to the classical and contemporary theory of computation. The topics covered are, in a few words, the theory of automata and formal languages, computability by Turing machines and recursive functions, uncomputability, computational complexity, and math- ematicallogic. The treatment is mathematical but the viewpoint is that of com- puter science; thus the chapter on context-free languages includes a discussion of parsing, and the chapters on logic establish the soundness and completeness of resolution theorem-proving.

**In **the undergraduate curriculum, exposure to this subject tends to come
late, if at all, and collaterally with courses on the design and analysis of al-
gorithms. It is our view that computer science students should be exposed to
this material earlier -as sophomores or juniors- both because of the deeper
insights it yields on specific topics in computer science, and because it serves to
establish essential mathematical paradigms. But we have found teaching to a
rigorous undergraduate course on the subject a difficult undertaking because of
the mathematical maturity assumed by the more advanced textbooks. Our goal
in writing this book has been to make the essentials of the subject accessible
to a broad undergraduate audience in a way that is mathematically sound but
presupposes no special mathematical experience.

The whole book represents about a year's worth of coursework. We have each taught a one-term course covering much of the material in Chapters 1 through 6, omitting on various occasions and in various combinations the sec- tions of parsing, on recursive functions, and on particular unsolvable decision problems. Other selections are possible; for example, a course emphasizing com- putability and the foundations of mechanical logic might skip quickly over Chap- ters 1 through 3 and concentrate on Chapters 4, 6, 8, and 9. However, it is used, our fervent hope is that the book will contribute to the intellectual development

Preface of the next generation of computer scientists by introducing them at an early stage of their education to crisp and methodical thinking about computational problems.

We take this opportunity to thank all from whom we have learned, both teachers and students. Specific thanks go to Larry Denenberg and Aaron Temin for their proofreading of early drafts, and to Michael Kahl and Oded Shmueli for their assistance and advice as teaching assistants. In the spring of 1980 Al- bert Meyer taught a course at M.LT. from a draft of this book, and we thank him warmly for his criticisms and corrections. Of course, the blame for any re- maining errors rests with us alone. Renate D' Arcangelo typed and illustrated the manuscript with her characteristic but extraordinary perfectionism and rapidity.

**Preface to the Second Edition **

*Much has changed in the fifteen years since the Elements of the Theory of Com-*
*putation first appeared -and much has remained the same. Computer science *
is now a much more mature and established discipline, playing a role of ever in-
creasing importance in a world of ubiquitous computing, globalized information,
and galloping complexity -more reasons to keep in touch with its foundations.

*The authors of the Elements are now themselves much more mature and busy *
-that is why this second edition has been so long in coming. We undertook
it because we felt that a few things could be said better, a few made simpler
-some even omitted altogether. More importantly, we wanted the book to
reflect how the theory of computation, and its students, have evolved during
these years. Although the theory of computation is now taught more widely
in absolute terms, its relative position within the computer science curriculum,
*for example vis *

*a *

*the subject of algorithms, has not been strengthened. In fact, the field of the design and analysis of algorithms is now so mature, that its elementary principles are arguably a part of a basic course on the theory of computation. Besides, undergraduates today, with their extensive and early computational experience, are much more aware of the applications of automata in compilers, for example, and more suspicious when simple models such as the Turing machine are presented as general computers. Evidently, the treatment of these subjects needs some updating.*

^{vis }Concretely, these are the major differences from the first edition:

o Rudiments of the design and analysis of algorithms are introduced infor- mally already in Chapter 1 (in connection with closures), and algorith- mic questions are pursued throughout the book. There are sections on algorithmic problems in connection with finite automata and context-free grammars in Chapters 2 and 3 (including state minimization and context- free recognition), algorithms for easy variants of NP-complete problems,

Preface and a section that reviews algorithmic techniques for "coping with

*NP-*

completeness" (special case algorithms, approximation algorithms, back- tracking and branch-and-bound, local improvement, and simulated anneal- ing algorithms).

o The treatment of Turing machines in Chapter 4 is more informal, and the simulation arguments are simpler and more quantitative. A random access Turing machine is introduced, helping bridge the gap between the clum- siness of Turing machines and the power of computers and programming languages.

o We included in Chapter 5 On undecidability some recursive function the- ory (up to Rice's Theorem). Grammars and recursive numerical functions are introduced and proved equivalent to Turing machines earlier, and the proofs are simpler. The undecidability of problems related to context-free grammars is proved by a simple and direct argument, without recourse to the Post correspondence problem. We kept the tiling problem, which we revisit in the NP-completeness chapter.

o Complexity is done in a rather novel way: In Chapter 6, we define no other
time bounds besides the polynomial ones --thus *P *is the first complexity
class and concept encountered. Diagonalization then shows that there are
exponential problems not in *P. *Real-life problems are introduced side-by-
side with their language representations (a distinction that is deliberately
blurred), and their algorithmic questions are examined extensively.

o There is a separate NP-completeness chapter with a new, extensive, and, we think, pedagogically helpful suite of NP-completeness reductions, cul- minating with the equivalence problem for regular expressions --closing a full circle to the first subject of the book. As mentioned above, the book ends with a section on algorithmic techniques for "coping with

*NP-*

completeness."

o There are no logic chapters in the new edition. This was a difficult decision, made for two reasons: According to all evidence, these were the least read and taught chapters of the book; and there are now books that treat this subject better. However, there is extensive treatment of Boolean logic and its satisfiability problems in Chapter 6.

o Overall, proofs and exposition have been simplified and made more informal at some key points. In several occasions, as in the proof of the equivalence of context-free languages and pushdown automata, long technical proofs of inductive statements have become exercises. There are problems following each section.

As a result of these changes, there is now at least one more way of teaching out of the material of this book (besides the ones outlined in the first edition, and the ones that emerged from its use): A semester-length course aiming at the

Preface

coverage of the basics of *both *the theory of computation and algorithms may be
based on a selection of material from Chapters 2 through 7.

We want to express our sincere thanks to all of our students and colleagues who provided feedback, ideas, errors, and corrections during these fifteen years- it is impossible to come up with a complete list. Special thanks to Martha Sideri for her help with the revision of Chapter 3. Also, many thanks to our editor, Alan Apt, and the people at Prentice-Hall-Barbara Kraemer, Sondra Chavez, and Bayani de Leon- who have been so patient and helpful.

In the present second printing. many errors have been corrected. We are in- debted to Carl Smith. Elaine Rich. Ray Miller. James Grimrnelmann. Rocio Guillen.

Paliath Narendran. Kuo-liang Chung Zhizhang Shen. Hua Ren. Charles Wells. Eric Thomson. Eric Fried. Jeremy Dawson. and especially Mikkel Nygaard Hansen. for pointing out errors to us. and to Barbara Taylor-Laino for making sure they were corrected.

Finally, we would appreciate receiving error reports or other comments.

preferably by electronic mail to the address elements(Olcs.berkeley.edu. Con- firmed errors, corrections. and other information about the book can also be obtained by writing to this address.

### ELEMENTS OF THE THEORY OF

### COMPUTATION

**Introduction **

Look around you. Computation happens everywhere, all the time, initiated by everybody, and affecting us all. Computation can happen because computer scientists over the past decades have discovered sophisticated methods for man- aging computer resources, enabling communication, translating programs, de- signing chips and databases, creating computers and programs that are faster, cheaper, easier to use, more secure.

As it is usually the case with all major disciplines, the practical successes
*of computer science build on its elegant and solid foundations. At the basis *
*of physical sciences lie fundamental questions such aH what is the nature of *
*matter? * *and what is the basis and origin of organic life? Computer science *
*has its own set of fundamental questions: What is an algorithm? What can and *
*what cannot be computed? When should an algorithm be considered practically *
*feasible? * For more than sixty years (starting even before the advent of the
electronic computer) computer scientists have been pondering these questions,
*and coming up with ingenious answers that have deeply influenced computer *
*science. *

The purpose of this book is to introduce you to these fundamental ideas,
*models, and results that permeate computer science, the basic paradigms of our *
field. They are worth studying, for many reasons. First, much of modern com-
puter science is based more or less explicitly on them -and much of the rest
should ... Also, these ideas and models are powerful and beautiful, excellent
examples of mathematical modeling that is elegant, productive, and of lasting
value. Besides, they are so much a part of the history and the "collective sub-
conscious" of our field, that it is hard to understand computer science without
first being exposed to them.

It *probably comes as no surprise that these ideas and models are mathemat-*
*ical *in nature. Although a computer is undeniably a physical object, it is also

1

2 Introduction

true that very little that is useful can be said of its physical aspects, such as its
molecules and its shape; the most useful abstractions of a computer are clearly
mathematical, and so the techniques needed to argue about them are necessarily
likewise. Besides, practical computational tasks require the ironclad guarantees
that only mathematics provides (we want our compilers to translate correctly,
our application programs to eventually terminate, and so on). However, the
mathematics employed in the theory of computation is rather different from the
mathematics used in other applied disciplines. It is generally *discrete, in that *
the emphasis is not on real numbers and continuous variables, but on finite sets
and sequences. It is based on very few and elementary concepts, and draws its
power and depth from the careful, patient, extensive, layer-by-Iayer manipula-
tion of these concepts -just like the computer. In the first chapter you will
be reminded of these elementary concepts and techniques (sets, relations, and
induction, among others), and you will be introduced to the style in which they
are used in the theory of computation.

The next two chapters, Chapters 2 and 3, describe certain restricted mod-
els of computation capable of performing very specialized string manipulation
tasks, such as telling whether a given string, say the word *punk, appears in a *
given text, such as the collective works of Shakespeare; or for testing whether
a given string of parentheses is properly balanced --like

### 0

and### (())O,

but not )0. These restricted computational devices (called*finite-state automata and*

*pushdown automata, respectively) actually come up in practice as very useful*and highly optimized components of more general systems such as circuits and compilers. Here they provide fine warm-up exercises in our quest for a formal, general definition of an algorithm. Furthermore, it is instructive to see how the power of these devices waxes and wanes (or, more often, is preserved) with the addition or removal of various features, most notably of non determinism, an in- triguing aspect of computation which is as central as it is (quite paradoxically) unrealistic.

In Chapter 4 we study general models of algorithms, of which the most ba-
sic is the *Turing machine,t a rather simple extension of the string-manipulating *
devices of Chapters 2 and 3 which turns out to be, surprisingly, a general frame-

### t

Named after Alan M. Turing (1912~1954), the brilliant English mathematician and philosopher whose seminal paper in 1936 marked the beginning of the theory of computation (and whose image, very appropriately, adorns the cover of this book). Turing also pioneered the fields of artificial intelligence and chess-playing by computer, as well as that of morphogenesis in biology, and was instrumental*in breaking Enigma, the German naval code during World War II. For more on*his fascinating life and times (and on his tragic end in the hands of official cruelty

*and bigotry) see the book Alan Turing: The Enigma, by Andrew Hodges, New*York: Simon Schuster, 1983.

Introduction 3
work for describing arbitrary algorithms. In order to argue this point, known as
*the Church- Turing thesis, we introduce more and more elaborate models of com-*
*putation (more powerful variants of the Turing machine, even a random access *
*Turing machine and recursive definitions of numerical functions), and show that *
they are all precisely equivalent in power to the basic Turing machine model.

*The following chapter deals with undecidability, the surprising property of *
*certain natural and well-defined computational tasks to lie provably beyond the *
reach of algorithmic solution. For example, suppose that you are asked whether
we can use tiles from a given finite list of basic shapes to tile the whole plane.

If the set of shapes contains a square, or even any triangle, then the answer
is obviously "yes." But what if it consists of a few bizarre shapes, or if some
*of the shapes are mandatory, that is, they must be used at least once for the *
tiling to qualify? This is surely the kind of complicated question that you would
like to have answered by a machine. In Chapter 5 we use the formalism of
*Turing machines to prove that this and many other problems cannot be solved *

*by computers at all. *

*Even when a computational task is amenable to solution by some algorithm, *
*it may be the case that there is no reasonably fast, practically feasible algorithm *
that solves it. In the last two chapters of this book we show how real-life com-
*putational problems can be categorized in terms of their complexity: Certain *
*problems can be solved within reasonable, polynomial time bounds, whereas *
*others seem to require amounts of time that grow astronomically, exponentially. *

In Chapter 7 we identify a class of common, practical, and notoriously difficult
*problems that are called NP-complete (the traveling salesman problem is only *
*one of them). We establish that all these problems are equivalent in that, if one *
of them has an efficient algorithm, then all of them do. It is widely believed that
all NP-complete problems are of inherently exponential complexity; whether
*this conjecture is actually true is the famous P *-:j; *NP problem, one of the most *
important and deep problems facing mathematicians and computer scientists
today.

This book is very much about algorithms and their formal foundations.

However, as you are perhaps aware, the subject of algorithms, their analysis and their design, is considered in today's computer science curriculum quite separate from that of the theory of computation. In the present edition of this book we have tried to restore some of the unity of the subject. As a result, this book also provides a decent, if somewhat specialized and unconventional, introduction to the subject of algorithms. Algorithms and their analysis are introduced informally in Chapter 1, and are picked up again and again in the context of the restricted models of computation studied in Chapters 2 and 3, and of the natural computational problems that they Hpawn. This way, when general models of algorithms are sought later, the reader is in a better position to appreciate the scope of the quest, and to judge its success. Algorithms playa

4 I ntrod uction

major role in our exposition of complexity as well, because there is no better way
to appreciate a complex problem than to contrast it with another, amenable to
an efficient algorithm. The last chapter culminates in a Hection on *coping with *
*NP-completeness, where we present an array of algorithmic techniques that *
have been successfully used in attacking NP-complete problems (approximation
algorithms, exhaustive algorithms, local search heuristics, and so on).

Computation is eSHential, powerful, beautiful, challenging, ever-expanding -and so is its theory. This book only tells the beginning of an exciting Htory.

It is a modest introduction to a few basic and carefully selected topics from the treasure chest of the theory of computation. We hope that it will motivate its readerH to seek out more; the references at the end of each chapter point to good places to start.

**1 ** **Sets, Relations, a nd La nguages **

**1.1 SETS **

They say that mathematics is the language of science - it is certainly the lan-
guage of the theory of computation, the scientific discipline we shall be studying
*in this book. And the language of mathematics deals with sets, and the com-*
plex ways in which they overlap, intersect, and in fact take part themselves in
forming new sets.

A set is a collection of objects. For example, the collection of the four letters
*a, b, *c, and *d *is a set, which we may name *L; *we write *L *

### =

*{a, b,*c,

*d}.*The objects comprising a set are called its elements or members. For example,

*b*is an element of the set

*L;*in symbols,

*bEL.*Sometimes we simply say that

*b*is in

*L,*or that

*L*contains

*b.*On the other hand,

*z*is not an element of

*L,*and we write

*z*~

*L.*

In a set we do not distinguish repetitions of the elements. Thus the set {red, blue, red} is the same set as {red, blue}. Similarly, the order of the elements is immaterial; for example, {3, 1, 9}, {9, 3,1}, and {I, 3, 9} are the same set. To summarize: Two sets are equal (that is, the same) if and only if they have the same elements.

The elements of a set need not be related in any way (other than happening
to be all members of the same set); for example, {3, red, *{d, *blue}} is a set with
three clements, one of which is itself a set. A set may have only one element;

it is then called a singleton. For example, {I} is the set with 1 as its only element; thus {1} and 1 are quite different. There is also a set with no element at all. Naturally, there can be only one such set: it is called the empty set, and is denoted by

### 0.

Any set other than the empty set is said to be nonempty.So far we have specified sets by simply listing all their elements, separated by commas and included in braces. Some sets cannot be written in this way, 5

6 Chapter 1: SETS. RELATIONS. AND LANGUAGES because they are infinite. For example, the set N of natural numbers is infinite;

we may suggest its elements by writing N = *{D, *1, 2, ... }, using the three dots
and your intuition in place of an infinitely long list. A set that is not infinite is
finite.

Another way to specify a set is by referring to other sets and to properties
*that elements mayor may not have. Thus if I *= {l, *3, 9} and G *

### =

{3,9}, G may be described as the set of elements of I that are greater than 2. We write this fact as follows.*G *= *{x : x *E *I *and *x is greater than *2}.

In general, if a set *A *has been defined and *P *is a property that elements of *A *
mayor may not have, then we can define a new set

*B *= *{x: x *E *A *and *x has property *

*Pl. *

As another example, the set of odd natural numbers is
*0= {x : x *^{E }N and x is not divisible by 2}.

A set *A *is a subset of a set *B *--in symbols, *A *~ *B-* if each element of
*A *is also an element of *B. * Thus 0 ~ N, since each odd natural number is a
natural number. Note that any set is a subset of itself. If *A *is a subset of *B *
but *A *is not the same as *B, *we say that *A *is a proper subset of *B *and write
*A *C *B. *Also note that the empty set is a subset of every set. For if B is any set,
then

### 0

^{~ }

*B,*since each element of

### 0

(of which there are none) is also an element of*B.*

To prove that two sets *A *and *B *are equal, we may prove that *A *~ Band
*B *~ *A. * Every element of *A *must then be an element of *B *and vice versa, so
that *A *and *B *have the same elements and *A *

### =

*B.*

*Two sets can be combined to form a third by various set operations, just *
as numbers are eombined by arithmetic operations such as addition. One set
operation is union: the union of two sets is that set having as elements the
objects that are elements of at least one of the two given sets, and possibly of
both. We use the symbol U to denote union, so that

*A *U *B *= *{x : x *E *A *or *x *E *B}. *

For example,

{l, 3, 9}

### u

{3, 5, 7} = {I, 3, 5, 7, 9}.The intersection of two sets is the eollection of all elements the two sets have in common; that is,

An *B *= *{x : x *E *A *and *x *E *B}. *

**1.1: ** Sets 7
For example,

{I, 3, 9}

### n

{3, 5, 7} = {3}, and{1,3,9}

### n

*{a,b,c,d}*=

### 0.

**Finally, the difference of two sets ***A *and *B, *denoted by *A - B, *is the set of all
elements of *A *that are not elements of *B. *

*A - B *= *{x: x *E *A and x *~ *B}. *

For example,

{I, 3, 9} - {3, 5, 7} = {I, 9}.

Certain properties of the set operations follow easily from their definitions.

For example, if *A, B, *and C are sets, the following laws hold.

Idempotency Commutativity Associativity Distributivity Absorption DeMorgan's laws

*AUA=A *
*AnA=A *
*AUB=BUA *
*AnB=BnA *

*(A *U *B) *U C = AU *(B *U C)
*(A *

### n

*B)*

### n C

=### An

*(B*

### n C)

*(A *U *B) *

### n

C =*(A*

### n

C) U*(B*

### n

C)*(A*

### n

*B)*U C =

*(A*U C)

### n

*(B*U C)

*(A*U

*B)*

### nA =

^{A }*(AnB) *UA = *A *

*A - (B *U C) = *(A - B) *n *(A -* C)
*A - (B *n C) = *(A - B) *U *(A -* C)

**Example 1.1.1: **Let us prove the first of De Morgan's laws. Let
*L *= *A - (B *U C)

and

*R *= *(A - B) *

### n

*(A - C);*

we are to show that *L *

### =

*R.*We do this by showing (a)

*L*~

*R*and (b)

*R*~

*L.*

*(a) Let x be any element of L; then x *E *A, but x *~ *B and x *~ C. Hence
*x *is an element of both *A - B *and *A -* C, and is thus an element of *R. *

Therefore *L *~ *R. *

( b) Let *x *E *R; *then *x *is an element of both *A - B *and *A -* C, and is
therefore in *A *but in neither *B *nor C. Hence *x *E *A *but *x *~ *B *U C, so
*x *E *L. *

8 Chapter 1: SETS, RELATIONS, AND LANGUAGES
Therefore *R *~ *L, *and we have established that *L *= *R.O *

Two sets are disjoint if they have no element in common, that is, if their intersection is empty.

It is possible to form intersections and unions of more than two sets. If
*S *is any collection of sets, we write

### U

*S*for the set whose elements are the

*elements of all the sets in S. For example, if S*=

*{{a, b}, {b, c}, {c, d}}*then

*US*=

*{a, b,*c,

*d};*and if

*S*= {{

*n} : n*EN}, that is, the collection of all the singleton sets with natural numbers as elements, then

*US*= N. In general,

### u

^{S }^{= }

^{{x : x }^{E }

*for some set*

^{P }*PES}.*

Similarly,

### n

^{S }^{= }

^{{x : x }^{E }

*for each set*

^{P }*PES}.*

The collection of all subsets of a set *A *is itself a set, called the power set
of *A *and denoted *2A. * For example, the subsets of *{c, d} *are *{c, d} *itself, the
singletons *{c} *and *{d} *and the empty set

### 0,

so*2{c,d} *= *{{c,d},{c},{d},0}. *

A partition of a non empty set *A *is a subset II of *2A *such that

### 0

is not an element of II and such that each element of*A*is in one and only one set in II.

That is, II is a partition of *A *if II is a set of subsets of *A *such that
(1) each element of II is nonempty;

(2) distinct members of II are disjoint;

(3) UII = *A. *

For example, *{{a, b}, {c}, {d}} *is a partition of *{a, b, *c, *d}, *but *{{b, *c}, *{c, d} } *
is not. The sets of even and odd natural numbers form a partition of N.

Problems for Section 1.1

1.1.1. Determine whether each of the following is true or false.

(a)

### 0

~### 0

(b)

### 0

E### 0

(c) 0 E {0}

(d)

### 0

~### {0}

(e) *{a,b} *E *{a,b,c,{a,b}} *

(f) *{a, b} *~ *{a, b, {a, b}} *

(g) *{a,b} *~ *2{a,b,{a,b}} *

(h) *{{a, b}} *E *2{a,b,{a,b}} *

(i) *{a,b,{a,b}}-{a,b}={a,b} *

1.2: Relations and Functions 9 1.1.2. '\'hat are these sets? Write them using braces, commas, and numerals only.

(a) ({1,3,5}U{3,1})n{3,5,7}

(b) U{ {3}, {3, 5},

### n{

{5, 7}, {7, 9}}}(c) ({1,2,5} - {5, 7,9})U ({5, 7,9} - {l,2,5}) (d) 2{7,8,9} _ 2{7,9}

(e) 2^{0 }

1.1.3. Prove each of the following.

(a) *AU(BnC)=(AUB)n(AUC) *

(b) An *(B *U C) = *(A *n *B) *U *(A *n C)
(c) *A *n *(A *U *B) *= *A *

*(d) AU(AnB)=A *

(e) *A - (B *

### n

C) =*(A - B)*U

*(A - C)*1.1.4. Let

*S*=

*{a, b, c, d}.*

(a) What partition of *S *has the fewest members? The most members?

(b) List all partitions of *S *with exactly two members.

### liiJ RELATIONS AND FUNCTIONS

Mathematics deals with statements about objects and the relations between them. It is natural to say, for example, that "less than" is a relation between objects of a certain kind -namely, numbers- which holds between 4 and 7 but does not hold between 4 and 2, or between 4 and itself. But how can we express relations between objects in the only mathematical language we have available at this point -that is to say, the language of sets? We simply think of a relation as being itself a set. The objects that belong to the relation are, in essence, the combinations of individuals for which that relation holds in the intuitive sense.

So the less-than relation is the set of all *pairs *of numbers such that the first
number is less than the second.

But we have moved a bit quickly. In a pair that belongs to a relation, we need to be able to distinguish the two parts of the pair, and we have not explained how to do so. We cannot write these pairs as sets, since {4, 7} is the same thing as {7, 4}. It is easiest to introduce a new device for grouping objects called an ordered pair.

### t

We write the ordered pair of two objects *a *and *b *as *(a, b); a *and *b *are called
the components of the ordered pair *(a, b). * The ordered pair *(a, b) *is not the
same as the set *{a,b}. * First, the order matters: *(a,b) *is different from *(b,a), *

### t

True fundamentalists would see the ordered pair (a, b) not as a new kind of object,but as identical to {a, {a,b}}.

10 Chapter 1: SETS, RELATIONS, AND LANGUAGES
whereas *{a, b} *= *{b, a}. * Second, the two components of an ordered pair need
not be distinct; (7,7) is a valid ordered pair. Note that two ordered pairs *(a, b) *
and *(c, d) *are equal only when *a *

### =

^{c and }

^{b }### =

^{d. }The Cartesian product of two sets *A *and *B, *denoted by *A *x *B, *is the
set of all ordered pairs *(a, b) *with *a *E *A *and *b *E *B. *For example,

{ 1, 3, 9} x *{b, *c, *d} *= {( 1, *b), *(1, *c), *(1, *d), *(3, *b), *(3 , *c), *(3, *d), *(9, *b), *(9 , *c), *(9, *d)} . *
A binary relation on two sets *A *and *B *is a subset of *Ax B. *For example,
*{(1,b),(1,c),(3,d),(9,d)} *is a binary relation on {1,3,9} and *{b,c,d}. * And
*{(i,j) : i,j EN and i *

### <

j} is the less-than relation; it is a subset of N x N ---often the two sets related by a binary relation are identical.More generally, let *n *be any natural number. Then if *aI, ... , an *are any *n *
objects, not necessarily distinct, *(al"'" an) *is an ordered tuple; for each
i = *1, ... *

*,n, *

*ai*is the ith component of

*(al, ... ,an).*An ordered m-tuple

*(bl, ... ,bm),*where m is a natural number, is the same as

*(al, ... ,an)*

^{if and }only ifm = nand

*ai*=

*b*

*i ,*for i =

*1, ... ,n.*Thus (4,4), (4,4,4), ((4,4),4), and (4, (4,4)) are all distinct. Ordered 2-tuples are the same as the ordered pairs discussed above, and ordered 3-, 4-, 5-, and 6-tuples are called ordered triples, quadruples, quintuples, and sextuples, respectively. On the other hand, a sequence is an ordered n-tuple for some unspecified

*n*(the length of the sequence). If

*AI, ... , An*are any sets, then the n-fold Cartesian product Al x '" x

*An*is the set of all ordered n-tuples

*(al, ... , an),*with

*ai*E

*Ai,*for each i = 1, ... ,

*n.*In case all the

*Ai,*are the same set

*A,*the n-fold Cartesian product

*A*x ... x

*A*of

*A*with itself is also written

*An.*For example, N

^{2 }is the set of ordered pairs of natural numbers. An n-ary relation on sets

*AI, ... , An*is a subset of Al x ... x

*An;*1-, 2-, and 3-ary relations are called unary, binary, and ternary relations, respectively.

Another fundamental mathematical idea is that of a *function. *On the intu-
itive level, a function is an association of each object of one kind with a unique
object of another kind: of persons with their ages, dogs with their owners, num-
bers with their successors, and so on. But by using the idea of a binary relation
as a set of ordered pairs, we can replace this intuitive idea by a concrete defini-
tion. A function from a set *A *to a set *B *is a binary relation *R *on *A *and *B *
with the following special property: for each element *a *E *A, *there is *exactly one *
ordered pair in *R with first component a. To illustrate the definition, let C be *
the set of cities in the United States and let *51 *be the set of states; and let

*RI *= *{(x,y) : x *E *C,y *E *51, *and *x *is a city in state *y}, *
*R2 *= *{(x,y): x *E *S,y *E C, and *y *is a city in state *x}. *

1.2: Relations and Functions 11
Then Rl is a function, since each city is in one and only one state, but *R2 *is not
a function, since some states have more than one city.t

In general, we use letters such as

*I, *

*and*

^{g, }*h for functions and we write*

*I : A*I-t

*B*to indicate that

*I*is a function from

*A*to

*B.*We call

*A*

**the domain**of

*f. *

If *a*is any element of

*A*we write

*I(a)*for that element

*b*of

*B*such that

*(a, b) E*

*I; *

since *I *

is a function, there is exactly one *b E B with this property, so*

*I(a)*denotes a unique object. The object

*I(a)*

**is called the image of**

*a*under

*f. *

To specify a function *I : *

*A*I-t

*B,*it suffices to specify

*I *

*(a)*for each

*a*E

*A;*

for example, to specify the function *RI *above, it suffices to specify, for each city,
the state in which it is located. If *I : A *I-t *B *and *AI *is a subset of *A, *then we
define J[A'l

### =

*{/(a) : a*E

*A'}*(that is,

*{b: b*

### =

*I(a)*for some

*a*E

*A'}).*We call J[A'l

**the image of**

*AI*under

*f. *

**The range of**

*I *

is the image of its domain.
Ordinarily, if the domain of a function is a Cartesian product, one set of parentheses is dropped. For example, if

*I : *

N x N I-t N is defined so that the
image under *I *

of an ordered pair *(m, n)*is the sum of m and n, we would write

*I(m, n)*

### =

*m+n*rather than

*I((m, n))*

### =

*m+n,*simply as a matter of notational convenience.

If *I: Al *x *A2 *^{X ... }x *An *I-t *B *is a function, and *I(al, ... ,an) *= *b, *where
*ai * ^{E } *Ai *for i = 1, ... , nand *b *^{E } *B, *then we sometimes call *al,··., *an the
**arguments **of *I *and *b ***the corresponding value of ***I. *Thus *I *may be specified
by giving its value for each n-tuple of arguments.

Certain kinds of functions are of special interest. A function

*I : *

*A*I-t

*B*is one-to-one if for any two distinct elements

*a, a'*

^{E }

*A, I(a)*

^{::f:-}*I(a' ).*

^{For }example, if C is the set of cities in the United States, 5 is the set of states, and

*g :*5 I-t C is specified by

*g( *s)

### =

the capital of state*s*

for each s E 5, then *g *is one-to-one since no two states have the same capital.

A function

*I : *

*A*I-t

**B is onto B if each element of B is the image under***I *

of
some element of *A.*The function

*g*just specified is not onto C, but the function

*RI*defined above is onto 5 since each state contains at least one city. Finally a mapping

*I : *

*A*I-t

*B*

**is a bijection between**

*A*and

*B*if it is both one-to-one and onto

*B;*for example, if Co is the set of capital cities, then the function

*g: 5*I-t Co specified, as before, by

*g( *s) = the capital of state s
is a bijection between 5 and *Co. *

### t

We consider Cambridge, Massachusetts, and Cambridge, Maryland, not the same city, but different cities that happen to have the same name.12 Chapter 1: SETS, RELATIONS, AND LANGUAGES
The inverse of a binary relation *R *t:;;; *A *x *B, *denoted *R-** ^{1 }*t:;;;

*B*x

*A,*is simply the relation

*{(b, a) : (a, b)*E

*R}.*For example, the relation

*R2*defined above is the inverse of

*R*

*Thus, the inverse of a function need not be a function. In the case of*

_{1 • }*Rl*its inverse fails to be a function since some states have more than one city;

that is, there are distinct cities Cl and C2 such that *Rl (Cl) *= *Rl (C2). * A function

*I : *

*A*f-t

*B*may also fail to have an inverse if there is some element

*b*E

*B*such that

*I(a) ::f:-*

*b*for all

*a*E

*A.*If

*I: A*f-t

*B*is a bijection, however, neither of these eventualities can occur, and

*1-1*is a function -indeed, a bijection between

*B*and

*A.*Moreover

*I-l(f(a))*=

*a*for each

*a*E

*A,*and

*l(f-l(b))*=

*b*for each bE

*B.*

When a particularly simple bijection between two sets has been specified, it is sometimes possible to view an object in the domain and its image in the range as virtually indistinguishable: the one may be seen as a renaming or a way of rewriting the other. For example, singleton sets and ordered I-tuples are, strictly speaking, different, but not much harm is done if we occasionally blur the distinction, because of the obvious bijection

*I *

such that *I ( { *

*a})*=

*(a)*for

*any singleton {a}. Such a bijection is called a natural isomorphism; of course*this is not a formal definition since what is "natural" and what distinctions can be blurred depend on the context. Some slightly more complex examples should make the point more clearly.

Example 1.2.1: For any three sets *A, B, *and C, there is a natural isomorphism
of *Ax B *x C to *(A *x *B) *x C, namely

*I(a,b,c) *= *((a,b),c) *
*for any a *E *A, *bE *B, *and c E

### C.O

Example 1.2.2: For any sets *A *and *B, *there is a natural isomorphism ¢ from

that is, the set of all binary relations on *A *and *B, *to the set

*{I : I *

is a function from *A*to

*2B}.*

Namely, for any relation *R *t:;;; *A *x B, let *¢(R) *be that function

*I : *

*A*f-t

*2B*such that

*I(a) *= *{b: *bE Band *(a,b) *E *R}. *

For example, if S is the set of states and R t:;;; *S *x *S *contains any ordered
pair of states with a common border, then the naturally associated function

*I : *

*S*f-t 2

^{5 }is specified by

*I *

*(s)*=

*{s' : s'*E

*Sand s' shares a border with s}.*

### 0

1.3: Special Types of Binary Relations 13 Example 1.2.3: Sometimes we regard the inverse of a function

*I : *

*A*f--t

*B*as a function even when

*I *

is not a bijection. The idea is to regard 1-1 t::;; *B x A as a*function from

*B*to

*2A,*using the natural isomorphism described under Example 1.2.2. Thus

*1-1 (b) is, for any b*E

*B, the set of all a*E

*A such that I(a)*=

*b.*

For example, if *Rl *is as defined above -the function that assigns to each city
the state in which it is located- then *Rll (s), *where s is a state, is the set of
all cities in that state.

If *Q *and *R *are binary relations, then their composition *Q *^{0 }*R, *or simply
*QR, *is the relation *{(a, b) : *for some c, *(a, *c) E *Q *and *(c, b) *E *R}. *Note that the
composition of two functions

*I : *

*A*f--t Band

*g :*

*B*f--t C is a function

*h*from

*A*to C such that

*h(a) = g(f(a))*for each

*a*E

*A.*For example, if

*I *

is the function
that assigns to each dog its owner and *g*assigns to each person his or her age,

*then log assigns to each dog the age of its owner.<>*

Problems for Sectior 1.2

1.2.1. Write each of the following explicitly.

(a)

### {I}

x### {1,2}

x### {1,2,3}

(b)

### 0

^{x }

### {1,2}

(c) 2{1,2} x {I, 2}

1.2.2. Let *R *= *{(a, b), (a, c), (c, d), (a, a), (b, a)}. *What is *R *0 *R, *the composition
of *R *with itself? What is *R-*^{1}*, *the inverse of *R? * Is *R, R *^{0 }*R, *or *R-** ^{1 }*a
function?

1.2.3. Let

*I : *

*A*f--t Band

*g :*

*B*f--t C. Let

*h : A*f--t C be their composition. In each of the following cases state necessary and sufficient conditions on

*I *

and *g *for *h to be as specified. *

(a) Onto.

(b) One-to-one.

(c) A bijection.

1.2.4. If *A *and *B *are any sets, we write *BA *for the set of all functions from *A *to
*B. *Describe a natural isomorphism between {O, l}A and *2A. *

### liiJ SPECIAL TYPES OF BINARY RELATIONS

Binary relations will be found over and over again in these pages; it will be helpful to have convenient ways of representing them and some terminology for discussing their properties. A completely "random" binary relation has no significant internal structure; but many relations we shall encounter arise out

14 Chapter 1: SETS, RELATIONS, AND LANGUAGES of specific contexts and therefore have important regularities. For example, the relation that holds between two cities if they belong to the same state has certain "symmetries" and other properties that are worth noting, discussing, and exploiting.

In this section we study relattons that exhibit these and similar regularities.

We shall deal only with binary relations on a set and itself. Thus, let *A *be a
set, and R t::;: *A *x *A *be a relation

*bn *

*The relation*

^{A. }*R can be represented by a*directed graph. Each element of

*A*is represented by a small circle-what we call a node of the directed graph~

*and an arrow is drawn from a to b if and only*if

*(a, b)*E

*R.*The arrows are the edges of the directed graph. For example, the relation

*R*=

*{(a,b),(b,a), (a,d),(d,c),(c,c),(c,a)}*is represented by the graph in Figure 1-1. Note in particular the loop from c to itself, corresponding to the pair

*(c, c)*E

*R.*From a node of a graph to another there is either no edge, or one edge ~we do not allow "parallel arrows."

*a * *b *

~--~

*d *

Figure 1-1

There is no formal distinction between binary relations on a set *A *and di-
*rected graphs with nodes from A. We use the term directed graph when we want *
to emphasize that the set on which the relation is defined is of no independent
interest to us, outside the context of this particular relation. Directed graphs,
*as well as the undirected graphs soon to be introduced, are useful as models and *
abstractions of complex systems (traffic and communication networks, compu-
tational structures and processes, etc.). In Section 1.6, and in much more detail
*in Chapters 6 and 7, we shall discuss many interesting computational problems *
arising in connection with directed graphs.

For another example of a binary relation/directed graph, the less-than-or- equal-to relation::; defined on the natural numbers is illustrated in Figure 1-2.

Of course, the entire directed graph cannot be drawn, since it would be infinite.

A relation R t::;: *A x A is reflexive if (a, a) *E *R for each a *E *A. *The directed
graph representing a reflexive relation has a loop from each node to itself. For
example, the directed graph of Figure 1-2 represents a reflexive relation, but
that of Figure 1-1 does not.

A relation *R *t::;: *A *x *A *is symmetric if *(b, a) *E *R *whenever *(a, b) * E *R. *

1.3: Special Types of Binary Relations 15

Figure 1-2

In the corresponding directed graph, whenever there is an arrow between two
nodes, there are arrows between those nodes in both directions. For exam-
ple, the directed graph of Figure 1-3 represents a symmetric relation. This
directed graph might depict the relation of "friendship" among six people,
*since whenever x is a friend of y, y is also a friend of x. The relation of *
friendship is not reflexive, since we do not regard a person as his or her own
friend. Of course, a relation could be both symmetric and reflexive; for exam-
ple, *{(a, b) : a *and *b *are persons with the same father} is such a relation.

*a *

^{f } ~:

^{f }

Figure 1-3

A symmetric relation *without pairs of the form (a, a) *is represented as an
undirected graph, or simply a graph. Graphs are drawn without arrowheads,
combining pairs of arrows going back' and forth between the same nodes. For
example, the relation shown in Figure 1-3 could also be represented by the graph
in Figure 1-4.

A relation *R *is antisymmetric if whenever *(a, b) * E *R *and *a *and bare
distinct, then *(b, a) *~ *R. *For example, let *P *be the set of all persons. Then

*{(a,b) : a,b *E *P *and *a *is the father of *b} *

16 Chapter 1: SETS, RELATIONS, AND LANGUAGES

## ao-r ^{IC }

*10--1 *

^{d }Figure 1-4

is antisymmetric. A relation may be neither symmetric nor antisymmetric; for example, the relation

*{( a, b) ; a, b *E *P *and *a *is the brother of *b} *

and the relation represelrted in Figure 1-1 are neither.

A binary relation *R *is transitive if whenever *(a, b) *E Rand *(b, c) *E *R, *
then *(a, c) *E *R. *The relation

*{(a, b) ; a, b *E *P *and *a *is an ancestor of *b} *

*is transitive, since if a is an ancestor of band b is an ancestor of c, then a is *
an ancestor of c. So is the less-than-or-equal relation. In terms of the directed
graph representation, transitivity is equivalent to the requirement that whenever
*there is a sequence of arrows leading from an element a to an element z, there *
*is an arrow directly from a to z. For example, the relation illustrated in Figure *
1-5 is transitive.

*aK77( *

### b~C

Figure 1-5

A relation that is reflexive, symmetric, and transitive is called an equiva-
lence relation. The representation of an equivalence relation by an undirected
*graph consists of a number of clusters; within each cluster, each pair of nodes is *
connected by a line (see Figure 1-6). The "clusters" of an equivalence relation
are called its equivalence classes. We normally write *[aJ *for the equivalence
class containing an element *a, provided the equivalence relation R *is under-
stood by the context. That is, *[aJ * = *{b ; (a, b) *E *R}, or, since R *is symmetric,
*[aJ *= *{b ; (b, a) *E *R}. * For example, the equivalence relation in Figure 1-6 has
three equivalence classes, one with four elements, one with three elements, and
one with one element.

1.3: Special Types of Binary Relations 17

### Q

Figure 1-6

*Theorem 1.3.1: Let R be an equivalence relation on a nonempty set A. Then *
*the equivalence classes of R constitute a partition of A. *

*Proof: *Let II = *{raj : a E A}. *We must show that the sets in II are non empty,
disjoint, and together exhaust *A. * All equivalence classes are nonempty, since
*a E [aJ * for all *a *E *A, *by reflexivity. To show that they are disjoint, consider
any two distinct equivalence classes *[aJ *and *[bJ, * and suppose that *[aJ *

### n

[bJ*f::. * *0. *

Thus there is an element *c such that c E [aJ * and c E [bJ. Hence *(a, c) *E Rand
*(c, b) E R; since R is transitive, (a, b) E R; and since R is symmetric, (b, a) E R. *

But now take any element dE *raj; *then (d, *a) *E R and, by transitivity, (d, b) E *R. *

Hence *d *E *[b], *so that *[aJ * <:;;; [bJ. Likewise [bJ <:;;; *raj. * Therefore *[aJ *= [bJ. But this
contradicts the assumption that *[aJ *and [bJ are distinct.

To see that

### U

II =*A,*simply notice that each element

*a*of A is in some set in II -namely,

*a*E

*[aJ,*by reflexivity . •

Thus starting from an equivalence relation *R, *we can always construct a
corresponding partition II. For example, if

*R *= {( *a, b) : a and b are persons and a and b have the same parents}, *
then the equivalence classes of R are all groups of siblings. Note that the con-
struction of Theorem 1.3.1 can be reversed: from any partition, we can construct
a corresponding equivalence relation. Namely, if II is a partition of A, then

*R *

### = {(

*a, b) : a*and b belong in the same set of II}

is an equivalence relation. Thus there is a natural isomorphism between the set
of equivalence relations on a set *A *and the set of partitions of *A. *

A relation that is reflexive, antisymmetric, and transitive is called a partial order. For example,

*{ ( a, b) : a, b are persons and a is an ancestor of b} *

18 Chapter 1: SETS, RELATIONS, AND LANGUAGES
is a partial order (provided we consider eaeh person to be an ancestor of himself
or herself). If *R *~ *A x A *is a partial order, an element *a *E *A *is called minimal
if the following is true: *(b, a) *E *R *only if *a *

### =

*For example, in the ancestor relation defined above, Adam and Eve are the only minimal elements. A finite partial order must have at least one minimal element, but an infinite partial order need not have one.*

^{b. }A partial order *R *~ *A *x *A *is a total order if, for all *a, b *E *A, *either
*( a, b) *E *R or (b, a) *E *R. *Thus the ancestor relation is not a total order since not
any two people are ancestrally related (for example, siblings are not); but the
less-than-or-equal-to relation on numbers is a total order. A total order cannot
have two or more minimal elements.

A path in a binary relation *R *is a sequence *(al, ... , an) *for some *n ;::: 1 *
such that *(ai, ai+l) *E *R for *i = 1, ... , n - 1; this path is said to be from al to
*an. * The length of a path *(al, ... , an) *is *n. The path (al, ... , an) *is a cycle if
the *ai's *are all distinct and also *(an' al) *E *R. *

Problems for Section 1.3

1.3.1. Let *R *

### =

^{{(a, }^{c), }

*(c,e), (e, e), (e, b), (d, b), (d, d)}.*Draw directed graphs rep- resenting each of the following.

(a) *R *
(b) *R-*^{I }

(c) *RUR-** ^{I }*
(d)

*Rn R-*

^{I }1.3.2. Let Rand *S *be the binary relations on *A *

### =

{I, ... , 7} with the graphical representations shown in the next page.(a) Indicate whether each of Rand *S *is (i) symmetric, (ii) reflexive, and
(iii) transitive.

(b) Repeat (a) for the relation *R *U *S. *

1.3.3. Draw directed graphs representing relations of the following types.

(a) Reflexive, transitive, and antisymmetric.

(b) Reflexive, transitive, and neither symmetric nor antisymmetric.

1.3.4. Let *A *be a nonempty set and let *R *~ *A *x *A *be the empty set. Which
properties does R have?

( a) Reflexi vi ty.

(b) Symmetry.

(c) Antisymmetry.

(d) Transitivity.

1.3.5. Let

*f : *

*A*f--t

*B.*Show that the following relation

*R*is an equivalence relation on

*A: (a, b)*E

*R*if and only if

*f(a)*

### =

*f(b).*

1.3: Special Types of Binary Relations 19

>---{3

### ®

1.3.6. Let *R *<:;:; *A *x *A *be a binary relation as defined below. In which cases is *R *
a partial order'? a total order'?

(a) *A *= the positive integers; *(a, b) *E *R *if and only if b is divisible by *a. *

(b) *A *= N x N; *((a, b)(c, d)) *E *R if and only if a:::: *cor b:::: d.

(c) *A *

### =

N;*(a, b)*E

*R*if and only if b

### =

*a or b*

### =

*a*

### +

l.(d) *A is the set of all English words; ( a, b) *E *R if and only if a *is no longer
than *b. *

(e) *A is the set of all English words; (a, b) *E *R if and only if a *is the same
*as b or occurs more frequently than b in the present book. *

20 Chapter 1: SETS, RELATIONS, AND LANGUAGES
1.3.7. Let *Rl *and *R2 *be any two partial orders on the same set.4. Show that

*Rl *

### n

*R2*is a partial order.

1.3.8. (a) Prove that if 5 is any collection of sets, then *Rs *= {(.4, *B) : A, B *E
5 and *A *~ *B} *is a partial order.

(b) Let 5 = 2{1,2,3}. Draw a directed graph representing the partial order
*Rs *defined in (a). Which are the minimal elements of *Rs? *

1.3.9. Under what circumstances does a directed graph represent a function?

1.3.10. Show that any function from a finite set to itself contains a cycle.

1.3.11. Let 5 be any set, and let *P * be the set of all partitions of 5. Let *R *be
the binary relation on *P *such that (III, II2) E *R *if and only if for every
51 E III, there is an 52 E II2 such that 51 ~ 52; if (III, II2) E R we say that
III refines II2. Show that *R *is a partial order on *P. *What elements of *P *
are maximal and minimal? Suppose that *P *were an arbitrary collection of
subsets of 2*s , *which need not be partitions of 5. Would R necessarily be a
partial order?

### G

FINITE AND INFINITE SETSA basic property of a finite set is its *size, that is, the number of elements it *
contains. Some facts about the sizes of finite sets are so obvious they hardly
need proof. For example, if *A *~ *B, *then the size of *A *is less than or equal to
that of *B; *the size of *A *is zero if and only if *A *is the empty set.

However, an extension of the notion of "size" to infinite sets leads to dif- ficulties if we attempt to follow our intuition. Are there more multiples of 17 (0,17,34,51,68, ... ) than there are perfect squares (0,1,4,9,16, ... )? You are welcome to speculate on alternatives, but experience has shown that the only satisfactory convention is to regard these sets as having the same size.

We call two sets *A *and *B *equinumerous if there is a bijection

*I : *

*A*f-t

*B. * Recall that if there is a bijection

*I : *

*A*

^{f-t }

*B,*then there is a bijection

*1-1 : B*

^{f-t }

*A;*hence equinumerosity is a symmetrie relation. In fact, as is easily shown, it is an equivalence relation. For example, {8,red,{0,b}} and {1,2,3}

are equinumerous; let *f(8) *= 1, I(red) = 2, *f( *{0, *b}) *= 3. So are the multiples
of17 and the perfect squares; a bijection is given by *f(17n) *= *n** ^{2 }*for each n E N.

In general, we call a set finite if, intuitively, it is equinumerous with
{I, 2, ... , *n} for some natural number n. (For n *= 0, {I, ... , n} is the empty set,
so (I) is finite, being equinumerous with itself.) If *A and {1, ... ,n} are equinumer-*
ous, then we say that the cardinality of *A *(in symbols, IAI) *is n. The cardinality *
of a finite set is thus the number of elements in it.

1.4: Finite and Infinite Sets 21
A set is infinite if it is not finite. For example, the set N of natural numbers
is infinite; so are sets such as the set of integers, the set of reals, and the set of
perfect squares. However, *not all infinite sets are equinumerous. *

A set is said to be countably infinite if it is equinumerous with N, and
countable if it is finite or count ably infinite. A set that is not countable is
uncountable. To show that a set *A *is count ably infinite we must exhibit a
bijection / between *A. *and N; equivalently, we need only suggest a way in which
*A. *can be enumerated as

and so on, since such an enumeration immediately suggests a bijection -just
take /(0) = *ao, */(1) = *a1,'" *

For example, we can show that the union of any finite number of countably
infinite sets is count ably infinite. Let us only illustrate the proof for the case
of three pairwise disjoint, count ably infinite sets; a similar argument works in
general. Call the sets *A, B, *and *C. * The sets can be listed as above: *A *

*= *

*{ao, a1, ... }, B *

### =

*b1 , . . . },*

_{{bo, }*C*

### =

{co, C1, . . . }, Then their union can be listed as*Au B*

### u

*C*=

*{ao, bo,*

^{Co, }

*a1,*b1 , C1,

*a2, ... }.*This listing amounts to a way of

"visiting" all the elements in *A *U *B *

### u

*C*by alternating between different sets, as illustrated in Figure 1-7. The technique of interweaving the enumeration of several sets is called "dovetailing" (for reasons that any carpenter can give after looking at Figure 1-7).

*A *

*B *

*C *

Figure 1-7

The same idea can be used to show that the union of a countably infinite collection of countably infinite sets is count ably infinite. For example, let us show that N x N is count ably infinite; note that N x N is the union of {O} x N, {I} x N, {2} x N, and so on, that is, the union of a count ably infinite collection of countably infinite sets. Dovetailing must here be more subtle than in the