# Slides credited from Hsueh-I Lu, Hsu-Chun Hsiao, & Michael Tsai

## Full text

(1)

Slides credited from Hsueh-I Lu, Hsu-Chun Hsiao, & Michael Tsai

(2)

Mini-HW 7 released

Due on 11/30 (Thur) 17:20

Homework 3 released tonight

Due on 12/14 (Thur) 17:20 (three weeks)

Class change!!

Start from 12/07 (Thur) to forever (NOT next week!!)

Location: R103

Frequently check the website for the updated information!

(3)

3

(4)

(5)

### A graph G is defined as

V: a finite, nonempty set of vertices

E: a set of edges / pairs of vertices

5

(6)

### Graph type

Undirected: edge 𝑢, 𝑣 = 𝑣, 𝑢

Directed: edge 𝑢, 𝑣 goes from vertex 𝑢 to vertex 𝑣; 𝑢, 𝑣 ≠ 𝑣, 𝑢

Weighted: edges associate with weights

(7)

𝑢, 𝑣

7

(8)

## Degree

𝑢

### is the number of edges incident on

𝑢

In-degree of 𝑢: #edges 𝑥, 𝑢 in a directed graph

Out-degree of 𝑢: #edges 𝑢, 𝑥 in a directed graph

Degree = in-degree + out-degree

Isolated vertex: degree = 0

𝑖

𝑖

(9)

## Path

1

1

2

𝑘−1

𝑘

𝑘

### , 𝑣)

Reachable: 𝑣 is reachable from 𝑢 if there exists a path from 𝑢 to 𝑣

## Simple Path

All vertices except for 𝑢 and 𝑣 are all distinct

9

(10)

## Forest

(11)

### 𝐺 is acyclic, but if any edge is added to 𝐸, the resulting graph contains a cycle

11

Proofs in Textbook Appendix B.5

(12)

(13)

13

(14)

### Euler path

Can you traverse each edge in a connected graph exactly once without lifting the pen from the paper?

### Euler tour

Can you finish where you started?

14

### BD

Euler path Euler path Euler path

(15)

### 𝐺 has an Euler tour iff all vertices must be even vertices

15

Is it possible to determine whether a graph has an Euler path or an Euler tour, without necessarily having to find one explicitly?

Even vertices = vertices with even degrees Odd vertices = vertices with odd degrees

(16)

(17)

### Modeling applications using graph theory

What do the vertices represent?

What do the edges represent?

Undirected or directed?

Social Network Knowledge Graph 17

(18)

(19)

19

(20)

## Matrix

### Adjacency matrix = 𝑉 × 𝑉 matrix 𝐴 with 𝐴[𝑢][𝑣] = 1 if (𝑢, 𝑣) is an edge

1 2 3 4 5 6

1 1 1

2 1 1 1

3 1 1 1

4 1 1 1

5 1

6 1 1

1

2

3

5 4

6

• For undirected graphs, 𝐴 is symmetric; i.e., 𝐴 = 𝐴𝑇

• If weighted, store weights instead of bits in 𝐴

(21)

21

(22)

## List

### One list per vertex, where for 𝑢 ∈ 𝑉, 𝐴[𝑢] consists of all vertices adjacent to 𝑢

1 2 3 4 5 6

1 4

3

2 3

2

3 2

5

1 4 6

4

6 1

2

3

5 4

6

If weighted, store weights also in adjacency lists

(23)

## List

23

(24)

### Besides graph density, you may also choose a data structure based on the performance of other operations

Space Query an edge

Insert an edge

Delete an edge

List a vertex’s neighbors

Identify all edges

(25)

Textbook Chapter 22 – Elementary Graph Algorithms

(26)

### Standard graph-searching algorithms

Depth-First Search (DFS, 深度優先搜尋)

(27)

Textbook Chapter 22.2 – Breadth-first search

(28)

Source 𝒔

Layer 1

Layer 2

(29)

BFS

### 𝑣. 𝑑: distance from 𝑠 to 𝑣, for all 𝑣 ∈ 𝑉

Distance is the length of a shortest path in G

𝑣. 𝑑 = ∞ if 𝑣 is not reachable from 𝑠

𝑣. 𝑑 is also the depth of 𝑣 in 𝑇BFS

### 𝑣. 𝜋 = 𝑢 if (𝑢, 𝑣) is the last edge on shortest path to 𝑣

𝑢 is 𝑣’s predecessor in 𝑇BFS

29

(30)

BFS

### As 𝑣 is discovered from 𝑢, 𝑣 and (𝑢, 𝑣) are added to 𝑇

BFS

𝑇BFS is not explicitly stored; can be reconstructed from 𝑣. 𝜋

### Color the vertices to keep track of progress:

GRAY: discovered (first time encountered)

BLACK: finished (all adjacent vertices discovered)

WHITE: undiscovered

BFS(G, s)

for each vertex u in G.V-{s}

u.color = WHITE u.d = ∞

u.pi = NIL s.color = GRAY s.d = 0

s.pi = NIL Q = {}

ENQUEUE(Q, s) while Q! = {}

u = DEQUEUE(Q)

if v.color == WHITE v.color = GRAY v.d = u.d + 1 v.pi = u

ENQUEUE(Q,v) u.color = BLACK

(31)

𝑠 0

31

𝑤 𝑟

1 1

𝑟 𝑡 𝑥

1 2 2

𝑡 𝑥 𝑣

2 2 2

𝑥 𝑣 𝑢

2 2 3

𝑣 𝑢 𝑦

2 3 3

(32)

𝑢 𝑦

3 3

𝑦 3

(33)

Definition of 𝛿(𝑠, 𝑣): the shortest-path distance from 𝑠 to 𝑣 = the minimum number of edges in any path from 𝑠 to 𝑣

If there is no path from 𝑠 to 𝑣, then 𝛿 𝑠, 𝑣 = ∞

The BFS algorithm finds the shortest-path distance to each reachable vertex in a graph 𝐺 from a given source vertex 𝑠 ∈ 𝑉.

33

(34)

### Proof

Case 1: 𝑢 is reachable from 𝑠

𝑠- 𝑢- 𝑣 is a path from 𝑠 to 𝑣 with length 𝛿 𝑠, 𝑢 + 1

Hence, 𝛿 𝑠, 𝑣 ≤ 𝛿 𝑠, 𝑢 + 1

Case 2: 𝑢 is unreachable from 𝑠

Then 𝑣 must be unreachable too.

Hence, the inequality still holds.

Lemma 22.1

Let 𝐺 = 𝑉, 𝐸 be a directed or undirected graph, and let 𝑠 ∈ 𝑉 be an arbitrary vertex. Then, for any edge 𝑢, 𝑣 ∈ 𝐸, 𝛿 𝑠, 𝑣 ≤ 𝛿 𝑠, 𝑢 + 1.

𝑠-𝑣的最短路徑一定會小於等於𝑠-𝑢的最短路徑距離+1

s

v

𝛿 𝑠, 𝑢 u

(35)

### Proof by induction

Holds when 𝑛 = 1: 𝑠 is in the queue and 𝑣. 𝑑 = ∞ for all 𝑣 ∈ 𝑉 𝑠

After 𝑛 + 1 ENQUEUE ops, consider a white vertex 𝑣 that is discovered during the search from a vertex 𝑢

Vertex 𝑣 is never enqueued again, so 𝑣. 𝑑 never changes again

35

Lemma 22.2

Let 𝐺 = 𝑉, 𝐸 be a directed or undirected graph, and suppose BFS is run on 𝐺 from a given source vertex 𝑠 ∈ 𝑉. Then upon termination, for each vertex 𝑣 ∈ 𝑉, the value 𝑣. 𝑑 computed by BFS satisfies 𝑣. 𝑑 ≥ 𝛿 𝑠, 𝑣 .

BFS算出的d值必定大於等於真正距離

Inductive hypothesis: 𝑣. 𝑑 ≥ 𝛿 𝑠, 𝑣 after 𝑛 ENQUEUE ops

(by induction hypothesis) (by Lemma 22.1)

(36)

### Proof by induction

Holds when 𝑄 = 𝑠 .

Consider two operations for inductive step:

Dequeue op: when 𝑄 = 𝑣1, 𝑣2, … , 𝑣𝑟 and dequeue 𝑣1

Enqueue op: when 𝑄 = 𝑣1, 𝑣2, … , 𝑣𝑟 and enqueue 𝑣𝑟+1

Lemma 22.3

Suppose that during the execution of BFS on a graph 𝐺 = 𝑉, 𝐸 , the queue 𝑄 contains the vertices 𝑣1, 𝑣2, … , 𝑣𝑟 , where 𝑣1 is the head of 𝑄 and 𝑣𝑟 is the tail. Then, 𝑣𝑟. 𝑑 ≤ 𝑣1. 𝑑 + 1 and 𝑣𝑖. 𝑑 ≤ 𝑣𝑖+1. 𝑑 for 1 ≤ 𝑖 < 𝑟.

• Q中最後一個點的d值 ≤ Q中第一個點的d值+1

• Q中第i個點的d值 ≤ Q中第i+1點的d值

Inductive hypothesis:𝑣𝑟. 𝑑 ≤ 𝑣1. 𝑑 + 1 and 𝑣𝑖. 𝑑 ≤ 𝑣𝑖+1. 𝑑 after 𝑛 queue ops

(37)

### Enqueue op

37

Inductive hypothesis:

𝑣1 𝑣2 … 𝑣𝑟−1 𝑣𝑟

𝑣2 … 𝑣𝑟−1 𝑣𝑟 (induction hypothesis H2)

𝑣1 𝑣2 … 𝑣𝑟−1 𝑣𝑟

(induction hypothesis H2) 𝑣1 𝑣2 … 𝑣𝑟−1 𝑣𝑟 𝑣𝑟+1

𝑢

Let 𝑢 be 𝑣𝑟+1’s predecessor,

Since 𝑢 has been removed from 𝑄, the new head 𝑣1 satisfies

(induction hypothesis H1) H1

H2

 H1 holds

 H2 holds

 H1 holds 𝑢

(induction hypothesis H1)

 H2 holds

(Q中最後一個點的d值 ≤ Q中第一個點的d值+1)

(Q中第i個點的d值 ≤ Q中第i+1點的d值)

(38)

### Proof

Lemma 22.3 proves that 𝑣𝑖. 𝑑 ≤ 𝑣𝑖+1. 𝑑 for 1 ≤ 𝑖 < 𝑟

Each vertex receives a finite 𝑑 value at most once during the course of BFS

Hence, this is proved.

Corollary 22.4

Suppose that vertices 𝑣𝑖 and 𝑣𝑗 are enqueued during the execution of BFS, and that 𝑣𝑖 is enqueued before 𝑣𝑗. Then 𝑣𝑖. 𝑑 ≤ 𝑣𝑗. 𝑑 at the time that 𝑣𝑗 is enqueued.

(39)

### Proof of (1)

All vertices 𝑣 reachable from 𝑠 must be discovered; otherwise they would have 𝑣. 𝑑 = ∞ > 𝛿 𝑠, 𝑣 . (contradicting with Lemma 22.2)

39

Theorem 22.5 – BFS Correctness

Let 𝐺 = 𝑉, 𝐸 be a directed or undirected graph, and and suppose that BFS is run on 𝐺 from a given source vertex 𝑠 ∈ 𝑉.

1) BFS discovers every vertex 𝑣 ∈ 𝑉 that is reachable from the source 𝑠 2) Upon termination, 𝑣. 𝑑 = 𝛿 𝑠, 𝑣 for all 𝑣 ∈ 𝑉

3) For any vertex 𝑣 ≠ 𝑠 that is reachable from 𝑠, one of the shortest paths from 𝑠 to 𝑣 is a shortest path from 𝑠 to 𝑣. 𝜋 followed by the edge 𝑣. 𝜋, 𝑣

(40)

(2)

### Proof of (2) by contradiction

Assume some vertices receive 𝑑 values not equal to its shortest-path distance

Let 𝑣 be the vertex with minimum 𝛿 𝑠, 𝑣 that receives such an incorrect 𝑑 value; clearly 𝑣 ≠ 𝑠

By Lemma 22.2, 𝑣. 𝑑 ≥ 𝛿 𝑠, 𝑣 , thus 𝑣. 𝑑 > 𝛿 𝑠, 𝑣 (𝑣 must be reachable)

Let 𝑢 be the vertex immediately preceding 𝑣 on a shortest path from 𝑠 to 𝑣, so 𝛿 𝑠, 𝑣 = 𝛿 𝑠, 𝑢 + 1

Because 𝛿 𝑠, 𝑢 < 𝛿 𝑠, 𝑣 and 𝑣 is the minimum 𝛿 𝑠, 𝑣 , we have 𝑢. 𝑑 = 𝛿 𝑠, 𝑢

𝑣. 𝑑 > 𝛿 𝑠, 𝑣 = 𝛿 𝑠, 𝑢 + 1 = 𝑢. 𝑑 + 1

(41)

(2)

### Proof of (2) by contradiction (cont.)

𝑣. 𝑑 > 𝛿 𝑠, 𝑣 = 𝛿 𝑠, 𝑢 + 1 = 𝑢. 𝑑 + 1

When dequeuing 𝑢 from 𝑄, vertex 𝑣 is either WHITE, GRAY, or BLACK

WHITE: 𝑣. 𝑑 = 𝑢. 𝑑 + 1, contradiction

BLACK: it was already removed from the queue

By Corollary 22.4, we have 𝑣. 𝑑 ≤ 𝑢. 𝑑, contradiction

GRAY: it was painted GRAY upon dequeuing some vertex 𝑤

Thus 𝑣. 𝑑 = 𝑤. 𝑑 + 1 (by construction)

𝑤 was removed from 𝑄 earlier than 𝑢, so 𝑤. 𝑑 ≤ 𝑢. 𝑑 (by Corollary 22.4)

𝑣. 𝑑 = 𝑤. 𝑑 + 1 ≤ 𝑢. 𝑑 + 1, contradiction

Thus, (2) is proved.

41

(42)

(3) For any vertex 𝑣 ≠ 𝑠 that is reachable from 𝑠, one of the shortest paths from 𝑠 to 𝑣 is a shortest path from 𝑠 to 𝑣. 𝜋 followed by the edge 𝑣. 𝜋, 𝑣

### Proof of (3)

If 𝑣. 𝜋 = 𝑢, then 𝑣. 𝑑 = 𝑢. 𝑑 + 1. Thus, we can obtain a shortest path from 𝑠 to 𝑣 by taking a shortest path from 𝑠 to 𝑣. 𝜋 and then traversing the edge 𝑣. 𝜋, 𝑣 .

(43)

BFS(G, s) forms a BFS tree with all reachable 𝑣 from 𝑠

We can extend the algorithm to find a BFS forest that contains every vertex in 𝐺

43

BFS-Visit(G, s) s.color = GRAY s.d = 0

s.π = NIL Q = empty ENQUEUE(Q, s) while Q ≠ empty

u = DEQUEUE(Q) for v in G.adj[u]

if v.color == WHITE v.color = GRAY v.d = u.d + 1 v.π = u

ENQUEUE(Q, v) u.color = BLACK

//explore full graph and builds up a collection of BFS trees

BFS(G)

for u in G.V

u.color = WHITE u.d = ∞

u.π = NIL for s in G.V

if(s.color == WHITE) // build a BFS tree BFS-Visit(G, s)

(44)

Textbook Chapter 22.3 – Depth-first search

### 44

(45)

Search as deep as possible and then backtrack until finding a new path

Timestamps: discovery time / finishing time 45

1

2

3 4

8

9 12 13

14

5 6

7

10 11

(46)

### Color the vertices to keep track of progress:

GRAY: discovered (first time encountered)

BLACK: finished (all adjacent vertices discovered)

// Explore full graph and builds up a collection of DFS trees

DFS(G)

for each vertex u in G.V u.color = WHITE

u.pi = NIL

time = 0 // global timestamp for each vertex u in G.V

if u.color == WHITE DFS-VISIT(G, u)

DFS-Visit(G, u) time = time + 1

u.d = time // discover time u.color = GRAY

if v.color == WHITE v.pi = u

DFS-VISIT(G, v) u.color = BLACK

time = time + 1

u.f = time // finish time

(47)

### Parenthesis Theorem

Parenthesis structure: represent the discovery of vertex 𝑢 with a left

parenthesis “(𝑢” and represent its finishing by a right parenthesis “𝑢)”. In DFS, the parentheses are properly nested.

### White Path Theorem

In a DFS forest of a directed or undirected graph 𝐺 = 𝑉, 𝐸 ,

vertex 𝑣 is a descendant of vertex 𝑢 in the forest  at the time 𝑢. 𝑑 that the search discovers 𝑢, there is a path from 𝑢 to 𝑣 in 𝐺 consisting entirely of WHITE vertices

Tree Edge

Back Edge

Forward Edge

Cross Edge

47

(48)

### Parenthesis Theorem

Parenthesis structure: represent the discovery of vertex 𝑢 with a left parenthesis “(𝑢” and represent its finishing by a right parenthesis

“𝑢)”. In DFS, the parentheses are properly nested.

Properly nested: (x (y y) x) Not properly nested: (x (y x) y)

Proof in textbook p. 608

(49)

### White Path Theorem

In a DFS forest of a directed or undirected graph 𝐺 = 𝑉, 𝐸 ,

vertex 𝑣 is a descendant of vertex 𝑢 in the forest  at the time 𝑢. 𝑑 that the search discovers 𝑢, there is a path from 𝑢 to 𝑣 in 𝐺 consisting entirely of WHITE vertices

### Proof.

Since 𝑣 is a descendant of 𝑢, 𝑢. 𝑑 < 𝑣. 𝑑

Hence, 𝑣 is WHITE at time 𝑢. 𝑑

In fact, since 𝑣 can be any descendant of 𝑢, any vertex on the path from 𝑢 to 𝑣 are WHITE at time 𝑢. 𝑑

 (textbook p. 608)

49

(50)

### Classification of Edges in 𝐺

Tree Edge (GRAY to WHITE)

Edges in the DFS forest

Found when encountering a new vertex 𝑣 by exploring 𝑢, 𝑣

Back Edge (GRAY to GRAY)

𝑢, 𝑣 , from descendant 𝑢 to ancestor 𝑣 in a DFS tree

Forward Edge (GRAY to BLACK)

𝑢, 𝑣 , from ancestor 𝑢 to descendant 𝑣. Not a tree edge.

Cross Edge (GRAY to BLACK)

Any other edge between trees or subtrees. Can go between vertices in same DFS tree or in different DFS trees

In an undirected graph, back edge = forward edge.

To avoid ambiguity, classify edge as the first type in the list that applies.

(51)

### Edge classification by the color of 𝑣 when visiting 𝑢, 𝑣

WHITE: tree edge

GRAY: back edge

BLACK: forward edge or cross edge

𝑢. 𝑑 < 𝑣. 𝑑  forward edge

𝑢. 𝑑 > 𝑣. 𝑑  cross edge

51

Why?

Theorem 22.10

In DFS of an undirected graph, there are only tree edges and back edges without forward and cross edge.

(52)

(53)

(54)

### a maximal subset 𝑈 of 𝑉 s.t. any two nodes in 𝑈 are connected in 𝐺

(55)

55

10 1

2

5

3 4

6

7

8 9

Time Complexity:

BFS and DSF both find the connected components with the same complexity

(56)
(57)

Textbook Chapter 22.5 – Strongly connected components

(58)

### a maximal subset 𝑈 of 𝑉 s.t. any two nodes in 𝑈 are reachable in 𝐺

1

2

4

6 3

5

7

8

Why must the strongly connected components of a graph be disjoint?

(59)

59

### Step 3: output the vertex partition by the second DFS

(60)

1

2

4

6 3

5

1

2

4

6 3

5

(61)

61

1

3

2

6 5

4 1

2

4

5 3

6

(62)

Assume that 𝑣, 𝑤 is an incoming edge to 𝐶.

Since 𝐶 is a strongly connected component of 𝐺, there cannot be any path from any node of 𝐶 to 𝑣 in 𝐺.

Therefore, the finish time of 𝑣 has to be larger than any node in 𝐶, including 𝑢.  𝑣. 𝑓 > 𝑢. 𝑓, contradiction

𝑢

### C

𝑤

𝑣

Lemma

Let 𝐶 be the strongly connected component of 𝐺 (and 𝐺𝑇) that contains the node 𝑢 with the largest finish time 𝑢. 𝑓. Then 𝐶 cannot have any incoming edge from any node of 𝐺 not in 𝐶.

(63)

63

𝑢

### C

Theorem

By continuing the process from the vertex 𝑢 whose finish time 𝑢. 𝑓 is the largest excluding those in 𝐶, the algorithm returns the strongly connected components.

𝑢

T

(64)

1

3

2

6 5

4

(65)

65

1

3

2

6 5

4

(66)

### Time Complexity:

(67)

67

(68)

Textbook Chapter 22.4 – Topological sort

(69)

69

1

2

3

5 4

6 1

2

3

5 4

6

(70)

### a directed graph without any directed cycle

1

2

3

5 4

6

(71)

Taking courses should follow the specific order

How to find a course taking order?

71

(72)

a b d

f c e

a

b

d

f c

f b d e

a c e

(73)

73

### Output the nodes in decreasing order of their finish time.

DFS(G)

for each vertex u in G.V u.color = WHITE

u.pi = NIL time = 0

for each vertex u in G.V if u.color == WHITE

DFS-VISIT(G, u)

DFS-Visit(G, u) time = time + 1 u.d = time

u.color = GRAY

for each v in G.Adj[u] (outgoing) if v.color == WHITE

v.pi = u

DFS-VISIT(G, v) u.color = BLACK

time = time + 1

u.f = time // finish time

(74)

a

b

d

f c

e

a b d

f c e

1

2

3 4

5

6

(75)

75

a

b

d

f c

e

f b d

a c e

1

2

3 4

6

5

(76)

### Output the nodes in decreasing order of their finish time.

As each vertex is finished, insert it onto the front of a linked list

Return the linked list of vertices

DFS(G)

for each vertex u in G.V u.color = WHITE

u.pi = NIL time = 0

for each vertex u in G.V if u.color == WHITE

DFS-VISIT(G, u)

DFS-Visit(G, u) time = time + 1 u.d = time

u.color = GRAY

if v.color == WHITE v.pi = u

DFS-VISIT(G, v) u.color = BLACK

time = time + 1

u.f = time // finish time

Time Complexity:

(77)

### Proof

: suppose there is a back edge 𝑢, 𝑣

𝑣 is an ancestor of 𝑢 in DFS forest

There is a path from 𝑣 to 𝑢 in 𝐺 and 𝑢, 𝑣 completes the cycle

 : suppose there is a cycle 𝑐

Let 𝑣 be the first vertex in 𝑐 to be discovered and 𝑢 is a predecessor of 𝑣 in 𝑐

Upon discovering 𝑣 the whole cycle from 𝑣 to 𝑢 is WHITE

At time 𝑣. 𝑑, the vertices of 𝑐 form a path of white vertices from 𝑣 to 𝑢

By the white-path theorem, vertex 𝑢 becomes a descendant of 𝑣 in the DFS forest

Therefore, 𝑢, 𝑣 is a back edge

77

Lemma 22.11

A directed graph is acyclic  a DFS yields no back edges.

(78)

### Proof

When 𝑢, 𝑣 is being explored, 𝑢 is GRAY and there are three cases for 𝑣:

Case 1 – GRAY

𝑢, 𝑣 is a back edge (contradicting Lemma 22.11), so 𝑣 cannot be GRAY

Case 2 – WHITE

𝑣 becomes descendant of 𝑢

𝑣 will be finished before 𝑢

Case 3 – BLACK

Theorem 22.12

The algorithm produces a topological sort of the input DAG. That is, if 𝑢, 𝑣 is a directed edge (from 𝑢 to 𝑣) of 𝐺, then 𝑢. 𝑓 > 𝑣. 𝑓.

(79)

79

(80)

(81)

(82)

### 82

Important announcement will be sent to @ntu.edu.tw mailbox

& post to the course website

Updating...

## References

Related subjects :