Algorithm Design and AnalysisMidterm Review

Algorithm Design and Analysis Midterm Review

Yun-Nung (Vivian) Chen

http://ada.miulab.tw

Slides credited from Hsueh-I Lu, Hsu-Chun Hsiao, & Michael Tsai

• Mini-HW 6 released

• Due on 11/28 (Thur) 14:20

• Homework 3 released soon

• Due on 12/12 (Thur) 14:20 (three weeks)

Announcement

Frequently check the website for the updated information!

Mini-HW 6

3

Outline

• Graph Basics

• Graph Theory

• Graph Representations

• Graph Traversal

• Breadth-First Search (BFS)

• Depth-First Search (DFS)

• DFS Applications

• Connected Components

• Strongly Connected Components

• Topological Sorting

Graph Basics

• A graph G is defined as

• V: a finite, nonempty set of vertices

• E: a set of edges / pairs of vertices

5

3 5

1

4

2

Graph Basics

• Graph type

• Undirected: edge 𝑢, 𝑣 = 𝑣, 𝑢

• Directed: edge 𝑢, 𝑣 goes from vertex 𝑢 to vertex 𝑣; 𝑢, 𝑣 ≠ 𝑣, 𝑢

• Weighted: edges associate with weights

3 5

1 4

2

3 5

1 4

2

How many edges at most can an undirected (or directed) graph have?

Graph Basics

• Adjacent (相鄰)

• If there is an edge 𝑢, 𝑣 , then 𝑢 and 𝑣 are adjacent.

• Incident (作用)

• If there is an edge 𝑢, 𝑣 , the edge 𝑢, 𝑣 is incident from 𝑢 and is incident to 𝑣.

• Subgraph (子圖)

• If a graph 𝐺

= 𝑉

, 𝐸′ is a subgraph of 𝐺 = 𝑉, 𝐸 , then 𝑉

⊆ 𝑉 and 𝐸

⊆ 𝐸

7

Graph Basics

• Degree

• The degree of a vertex 𝑢 is the number of edges incident on 𝑢

• In-degree of 𝑢: #edges 𝑥, 𝑢 in a directed graph

• Out-degree of 𝑢: #edges 𝑢, 𝑥 in a directed graph

• Degree = in-degree + out-degree

• Isolated vertex: degree = 0

𝐸 = σ

𝑑

2

Graph Basics

• Path

• a sequence of edges that connect a sequence of vertices

• If there is a path from 𝑢 (source) to 𝑣 (target), there is a sequence of edges 𝑢, 𝑖₁ , 𝑖₁, 𝑖₂ , … , 𝑖_𝑘−1, 𝑖_𝑘 , (𝑖_𝑘, 𝑣)

• Reachable: 𝑣 is reachable from 𝑢 if there exists a path from 𝑢 to 𝑣

• Simple Path

• All vertices except for 𝑢 and 𝑣 are all distinct

• Cycle

• A simple path where 𝑢 and 𝑣 are the same

• Subpath

• A subsequence of the path

9

Graph Basics

• Connected

• Two vertices are connected if there is a path between them

• A connected graph has a path from every vertex to every other

• Tree

• a connected, acyclic, undirected graph

• Forest

• an acyclic, undirected but possibly disconnected graph

3 5

1 4

2

3 5

1 4

2

3 5

1 4

2

Graph Basics

• Theorem. Let 𝐺 be an undirected graph. The following statements are equivalent:

• 𝐺 is a tree

• Any two vertices in 𝐺 are connected by a unique simple path

• 𝐺 is connected, but if any edge is removed from 𝐸, the resulting graph is disconnected.

• 𝐺 is connected and 𝐸 = 𝑉 − 1

• 𝐺 is acyclic, and 𝐸 = 𝑉 − 1

• 𝐺 is acyclic, but if any edge is added to 𝐸, the resulting graph contains a cycle

11

Proofs in Textbook Appendix B.5

Graph Theory

Seven Bridges of Königsberg (七橋問題)

• How to traverse all bridges where each one can only be passed through once

13

A

B

D

C C

A

B D

Euler Path and Euler Tour (一筆畫問題)

• Euler path

• Can you traverse each edge in a connected graph exactly once without lifting the pen from the paper?

• Euler tour

• Can you finish where you started?

C A

B D

C A

B D

C A

B D

Euler path Euler tour

Euler path Euler tour

Euler path Euler tour

Euler Path and Euler Tour

• Solved by Leonhard Euler in 1736

• 𝐺 has a Euler path iff 𝐺 has exactly 0 or 2 odd vertices

• 𝐺 has a Euler tour iff all vertices must be even vertices

15

Is it possible to determine whether a graph has an Euler path or an Euler tour, without necessarily having to find one explicitly?

Even vertices = vertices with even degrees Odd vertices = vertices with odd degrees

Hamiltonian Path

• Hamiltonian Path

• A path that visits each vertex exactly once

• Hamiltonian Cycle

• A Hamiltonian path where the start and destination are the same

• Both are NP-complete

Real-World Applications

• Modeling applications using graph theory

• What do the vertices represent?

• What do the edges represent?

• Undirected or directed?

17

Social Network Knowledge Graph

Graph Representations

Graph Representations

• How to represent a graph in computer programs?

• Two standard ways to represent a graph 𝐺 = 𝑉, 𝐸

• Adjacency matrix

• Adjacency list

19

Adjacency Matrix

• Adjacency matrix = 𝑉 × 𝑉 matrix 𝐴 with 𝐴[𝑢][𝑣] = 1 if (𝑢, 𝑣) is an edge

1 2 3 4 5 6

1 1 1

2 1 1 1

3 1 1 1

4 1 1 1

5 1

6 1 1

1

2

3

5 4

6

• For undirected graphs, 𝐴 is symmetric; i.e., 𝐴 = 𝐴^𝑇

• If weighted, store weights instead of bits in 𝐴

Complexity of Adjacency Matrix

• Space:

• Time for querying an edge:

• Time for inserting an edge:

• Time for deleting an edge:

• Time for listing all neighbors of a vertex:

• Time for identifying all edges:

• Time for finding in-degree and out-degree of a vertex?

21

Adjacency List

• Adjacency lists = vertex indexed array of lists

• One list per vertex, where for 𝑢 ∈ 𝑉, 𝐴[𝑢] consists of all vertices adjacent to 𝑢

1 2 3 4 5 6

1 4

3

2 3

2 2 3

5

1 4 6

4

6 1

2

3

5 4

6

If weighted, weights are also stored in adjacency lists

Complexity of Adjacency List

• Space:

• Time for querying an edge:

• Time for inserting an edge:

• Time for deleting an edge:

• Time for listing all neighbors of a vertex:

• Time for identifying all edges:

• Time for finding in-degree and out-degree of a vertex?

23

Representation Comparison

• Matrix representation is suitable for dense graphs

• List representation is suitable for sparse graphs

• Besides graph density, you may also choose a data structure based on the performance of other operations

Space Query an edge

Insert an edge

Delete an edge

List a vertex’s neighbors

Identify all edges Adjacency Matrix

Adjacency List

Graph Traversal

25

Textbook Chapter 22 – Elementary Graph Algorithms

Graph Traversal

• From a source vertex, systematically follow the edges of a graph to visit all reachable vertices of the graph

• Useful to discover the structure of a graph

• Standard graph-searching algorithms

• Breadth-First Search (BFS, 廣度優先搜尋)

• Depth-First Search (DFS, 深度優先搜尋)

Breadth-First Search

27

Textbook Chapter 22.2 – Breadth-first search

Breadth-First Search (BFS)

Source 𝒔

Layer 1

Layer 2

Breadth-First Search (BFS)

• Input: directed/undirected graph 𝐺 = (𝑉, 𝐸) and source 𝑠

• Output: a breadth-first tree with root 𝑠 (𝑇

) that contains all reachable vertices

• 𝑣. 𝑑: distance from 𝑠 to 𝑣, for all 𝑣 ∈ 𝑉

• Distance is the length of a shortest path in G

• 𝑣. 𝑑 = ∞ if 𝑣 is not reachable from 𝑠

• 𝑣. 𝑑 is also the depth of 𝑣 in 𝑇_BFS

• 𝑣. 𝜋 = 𝑢 if (𝑢, 𝑣) is the last edge on shortest path to 𝑣

• 𝑢 is 𝑣’s predecessor in 𝑇_BFS

29

Breadth-First Tree

• Initially 𝑇

contains only 𝑠

• As 𝑣 is discovered from 𝑢, 𝑣 and (𝑢, 𝑣) are added to 𝑇

• 𝑇_BFS is not explicitly stored; can be reconstructed from 𝑣. 𝜋

• Implemented via a FIFO queue

• Color the vertices to keep track of progress:

• GRAY: discovered (first time encountered)

• BLACK: finished (all adjacent vertices discovered)

• WHITE: undiscovered

BFS(G, s)

for each vertex u in G.V-{s}

u.color = WHITE u.d = ∞

u.pi = NIL s.color = GRAY s.d = 0

s.pi = NIL Q = {}

ENQUEUE(Q, s) while Q! = {}

u = DEQUEUE(Q)

for each v in G.Adj[u]

if v.color == WHITE v.color = GRAY v.d = u.d + 1 v.pi = u

ENQUEUE(Q,v) u.color = BLACK

BFS Illustration

31

𝑠 0

𝑤 𝑟 1 1

𝑟 𝑡 𝑥

1 2 2

𝑡 𝑥 𝑣

2 2 2

𝑥 𝑣 𝑢

2 2 3

𝑣 𝑢 𝑦

2 3 3

BFS Illustration

𝑢 𝑦 3 3

𝑦 3

Shortest-Path Distance from BFS

• Definition of 𝛿(𝑠, 𝑣): the shortest-path distance from 𝑠 to 𝑣 = the minimum number of edges in any path from 𝑠 to 𝑣

• If there is no path from 𝑠 to 𝑣, then 𝛿 𝑠, 𝑣 = ∞

• The BFS algorithm finds the shortest-path distance to each

reachable vertex in a graph 𝐺 from a given source vertex 𝑠 ∈ 𝑉.

33

Shortest-Path Distance from BFS

• Proof

• Case 1: 𝑢 is reachable from 𝑠

• 𝑠- 𝑢- 𝑣 is a path from 𝑠 to 𝑣 with length 𝛿 𝑠, 𝑢 + 1

• Hence, 𝛿 𝑠, 𝑣 ≤ 𝛿 𝑠, 𝑢 + 1

• Case 2: 𝑢 is unreachable from 𝑠

• Then 𝑣 must be unreachable too.

• Hence, the inequality still holds.

Lemma 22.1

Let 𝐺 = 𝑉, 𝐸 be a directed or undirected graph, and let 𝑠 ∈ 𝑉 be an arbitrary vertex. Then, for any edge 𝑢, 𝑣 ∈ 𝐸, 𝛿 𝑠, 𝑣 ≤ 𝛿 𝑠, 𝑢 + 1.

𝑠-𝑣的最短路徑一定會小於等於_𝑠-𝑢的最短路徑距離₊₁

s

v

𝛿 𝑠, 𝑢 u

Shortest-Path Distance from BFS

• Proof by induction

• Holds when 𝑛 = 1: 𝑠 is in the queue and 𝑣. 𝑑 = ∞ for all 𝑣 ∈ 𝑉 𝑠

• After 𝑛 + 1 ENQUEUE ops, consider a white vertex 𝑣 that is discovered during the search from a vertex 𝑢

• Vertex 𝑣 is never enqueued again, so 𝑣. 𝑑 never changes again

35

Lemma 22.2

Let 𝐺 = 𝑉, 𝐸 be a directed or undirected graph, and suppose BFS is run on 𝐺 from a given source vertex 𝑠 ∈ 𝑉. Then upon termination, for each vertex 𝑣 ∈ 𝑉, the value 𝑣. 𝑑 computed by BFS satisfies 𝑣. 𝑑 ≥ 𝛿 𝑠, 𝑣 .

BFS算出的_d值必定大於等於真正距離

Inductive hypothesis: 𝑣. 𝑑 ≥ 𝛿 𝑠, 𝑣 after 𝑛 ENQUEUE ops

(by induction hypothesis) (by Lemma 22.1)

Shortest-Path Distance from BFS

• Proof by induction

• Holds when 𝑄 = 𝑠 .

• Consider two operations for inductive step:

• Dequeue op: when 𝑄 = 𝑣₁, 𝑣₂, … , 𝑣_𝑟 and dequeue 𝑣₁

• Enqueue op: when 𝑄 = 𝑣₁, 𝑣₂, … , 𝑣_𝑟 and enqueue 𝑣_𝑟+1

Lemma 22.3

Suppose that during the execution of BFS on a graph 𝐺 = 𝑉, 𝐸 , the queue 𝑄

contains the vertices 𝑣₁, 𝑣₂, … , 𝑣_𝑟 , where 𝑣₁ is the head of 𝑄 and 𝑣_𝑟 is the tail. Then, 𝑣_𝑟. 𝑑 ≤ 𝑣₁. 𝑑 + 1 and 𝑣_𝑖. 𝑑 ≤ 𝑣_𝑖+1. 𝑑 for 1 ≤ 𝑖 < 𝑟.

• Q中最後一個點的_d值 _{≤ Q}中第一個點的_d值₊₁

• Q中第_i個點的_d值 _{≤ Q}中第_i+1點的_d值

Inductive hypothesis:𝑣_𝑟. 𝑑 ≤ 𝑣₁. 𝑑 + 1 and 𝑣_𝑖. 𝑑 ≤ 𝑣_𝑖+1. 𝑑 after 𝑛 queue ops

Shortest-Path Distance from BFS

• Dequeue op

• Enqueue op

37

Inductive hypothesis:

𝑣₁ 𝑣₂ … 𝑣_𝑟−1 𝑣_𝑟 𝑣₂ … 𝑣_𝑟−1 𝑣_𝑟

(induction hypothesis H2)

𝑣₁ 𝑣₂ … 𝑣_𝑟−1 𝑣_𝑟

(induction hypothesis H2) 𝑣₁ 𝑣₂ … 𝑣_𝑟−1 𝑣_𝑟 𝑣_𝑟+1

𝑢

Let 𝑢 be 𝑣_𝑟+1’s predecessor,

Since 𝑢 has been removed from 𝑄, the new head 𝑣₁ satisfies

(induction hypothesis H1)

H1 H2

→ H1 holds

→ H2 holds

→ H1 holds 𝑢

(induction hypothesis H1)

→ H2 holds

(Q中最後一個點的_d值 _{≤ Q}中第一個點的_d值₊₁₎

(Q中第_i個點的_d值 _{≤ Q}中第_i+1點的_d值₎

Shortest-Path Distance from BFS

• Proof

• Lemma 22.3 proves that 𝑣_𝑖. 𝑑 ≤ 𝑣_𝑖+1. 𝑑 for 1 ≤ 𝑖 < 𝑟

• Each vertex receives a finite 𝑑 value at most once during the course of BFS

• Hence, this is proved.

Corollary 22.4

Suppose that vertices 𝑣_𝑖 and 𝑣_𝑗 are enqueued during the execution of BFS, and that 𝑣_𝑖 is enqueued before 𝑣_𝑗. Then 𝑣_𝑖. 𝑑 ≤ 𝑣_𝑗. 𝑑 at the time that 𝑣_𝑗 is enqueued.

若_𝑣𝑖比_𝑣𝑗早加入_queue → 𝑣_𝑖. 𝑑 ≤ 𝑣_𝑗. 𝑑

Shortest-Path Distance from BFS

• Proof of (1)

• All vertices 𝑣 reachable from 𝑠 must be discovered; otherwise they would have 𝑣. 𝑑 =

∞ > 𝛿 𝑠, 𝑣 . (contradicting with Lemma 22.2)

39

Theorem 22.5 – BFS Correctness

Let 𝐺 = 𝑉, 𝐸 be a directed or undirected graph, and suppose that BFS is run on 𝐺 from a given source vertex 𝑠 ∈ 𝑉.

1) BFS discovers every vertex 𝑣 ∈ 𝑉 that is reachable from the source 𝑠 2) Upon termination, 𝑣. 𝑑 = 𝛿 𝑠, 𝑣 for all 𝑣 ∈ 𝑉

3) For any vertex 𝑣 ≠ 𝑠 that is reachable from 𝑠, one of the shortest paths from 𝑠 to 𝑣 is a shortest path from 𝑠 to 𝑣. 𝜋 followed by the edge 𝑣. 𝜋, 𝑣

(2)

Shortest-Path Distance from BFS

• Proof of (2) by contradiction

• Assume some vertices receive 𝑑 values not equal to its shortest-path distance

• Let 𝑣 be the vertex with minimum 𝛿 𝑠, 𝑣 that receives such an incorrect 𝑑 value;

clearly 𝑣 ≠ 𝑠

• By Lemma 22.2, 𝑣. 𝑑 ≥ 𝛿 𝑠, 𝑣 , thus 𝑣. 𝑑 > 𝛿 𝑠, 𝑣 (𝑣 must be reachable)

• Let 𝑢 be the vertex immediately preceding 𝑣 on a shortest path from 𝑠 to 𝑣, so 𝛿 𝑠, 𝑣 = 𝛿 𝑠, 𝑢 + 1

• Because 𝛿 𝑠, 𝑢 < 𝛿 𝑠, 𝑣 and 𝑣 is the minimum 𝛿 𝑠, 𝑣 , we have 𝑢. 𝑑 = 𝛿 𝑠, 𝑢

• 𝑣. 𝑑 > 𝛿 𝑠, 𝑣 = 𝛿 𝑠, 𝑢 + 1 = 𝑢. 𝑑 + 1

Shortest-Path Distance from BFS

• Proof of (2) by contradiction (cont.)

• 𝑣. 𝑑 > 𝛿 𝑠, 𝑣 = 𝛿 𝑠, 𝑢 + 1 = 𝑢. 𝑑 + 1

• When dequeuing 𝑢 from 𝑄, vertex 𝑣 is either WHITE, GRAY, or BLACK

• WHITE: 𝑣. 𝑑 = 𝑢. 𝑑 + 1, contradiction

• BLACK: it was already removed from the queue

• By Corollary 22.4, we have 𝑣. 𝑑 ≤ 𝑢. 𝑑, contradiction

• GRAY: it was painted GRAY upon dequeuing some vertex 𝑤

• Thus 𝑣. 𝑑 = 𝑤. 𝑑 + 1 (by construction)

• 𝑤 was removed from 𝑄 earlier than 𝑢, so 𝑤. 𝑑 ≤ 𝑢. 𝑑 (by Corollary 22.4)

• 𝑣. 𝑑 = 𝑤. 𝑑 + 1 ≤ 𝑢. 𝑑 + 1, contradiction

• Thus, (2) is proved.

41

(2)

(3) For any vertex 𝑣 ≠ 𝑠 that is reachable from 𝑠, one of the shortest paths from 𝑠 to 𝑣 is a shortest path from 𝑠 to 𝑣. 𝜋 followed by the edge 𝑣. 𝜋, 𝑣

Shortest-Path Distance from BFS

• Proof of (3)

• If 𝑣. 𝜋 = 𝑢, then 𝑣. 𝑑 = 𝑢. 𝑑 + 1. Thus, we can obtain a shortest path from 𝑠 to 𝑣 by taking a shortest path from 𝑠 to 𝑣. 𝜋 and then traversing the edge 𝑣. 𝜋, 𝑣 .

BFS Forest

• BFS(G, s) forms a BFS tree with all reachable 𝑣 from 𝑠

• We can extend the algorithm to find a BFS forest containing every vertex in 𝐺

43 BFS-Visit(G, s)

s.color = GRAY s.d = 0

s.π = NIL Q = empty

ENQUEUE(Q, s) while Q ≠ empty

u = DEQUEUE(Q) for v in G.adj[u]

if v.color == WHITE v.color = GRAY v.d = u.d + 1 v.π = u

ENQUEUE(Q, v) u.color = BLACK

//explore full graph and builds up a collection of BFS trees

BFS(G)

for u in G.V

u.color = WHITE u.d = ∞

u.π = NIL for s in G.V

if(s.color == WHITE) // build a BFS tree BFS-Visit(G, s)

Depth-First Search

Textbook Chapter 22.3 – Depth-first search

Depth-First Search (DFS)

• Search as deep as possible and then backtrack until finding a new path

45

Timestamps: discovery time / finishing time 1

2 3

4 8

9 12 13

14

5 6

7

10 11

DFS Algorithm

• Implemented via recursion (stack)

• Color the vertices to keep track of progress:

• GRAY: discovered (first time encountered)

• BLACK: finished (all adjacent vertices discovered)

• WHITE: undiscovered

// Explore full graph and builds up a collection of DFS trees

DFS(G)

for each vertex u in G.V u.color = WHITE

u.pi = NIL

time = 0 // global timestamp for each vertex u in G.V

if u.color == WHITE DFS-VISIT(G, u)

DFS-Visit(G, u) time = time + 1

u.d = time // discover time u.color = GRAY

for each v in G.Adj[u]

if v.color == WHITE v.pi = u

DFS-VISIT(G, v) u.color = BLACK

time = time + 1

u.f = time // finish time

DFS Properties

• Parenthesis Theorem

• Parenthesis structure: represent the discovery of vertex 𝑢 with a left parenthesis “(𝑢”

and represent its finishing by a right parenthesis “𝑢)”. In DFS, the parentheses are properly nested.

• White Path Theorem

• In a DFS forest of a directed or undirected graph 𝐺 = 𝑉, 𝐸 ,

• vertex 𝑣 is a descendant of vertex 𝑢 in the forest  at the time 𝑢. 𝑑 that the search discovers 𝑢, there is a path from 𝑢 to 𝑣 in 𝐺 consisting entirely of WHITE vertices

• Classification of Edges in 𝐺

• Tree Edge

• Back Edge

• Forward Edge

• Cross Edge

47

DFS Properties

• Parenthesis Theorem

• Parenthesis structure: represent the discovery of vertex 𝑢 with a left parenthesis “(𝑢”

and represent its finishing by a right parenthesis “𝑢)”. In DFS, the parentheses are properly nested.

Properly nested: (x (y y) x)

Not properly nested: (x (y x) y)

Proof in textbook p. 608

DFS Properties

• White Path Theorem

• In a DFS forest of a directed or undirected graph 𝐺 = 𝑉, 𝐸 ,

• vertex 𝑣 is a descendant of vertex 𝑢 in the forest  at the time 𝑢. 𝑑 that the search discovers 𝑢, there is a path from 𝑢 to 𝑣 in 𝐺 consisting entirely of WHITE vertices

• Proof.

• →

• Since 𝑣 is a descendant of 𝑢, 𝑢. 𝑑 < 𝑣. 𝑑

• Hence, 𝑣 is WHITE at time 𝑢. 𝑑

• In fact, since 𝑣 can be any descendant of 𝑢, any vertex on the path from 𝑢 to 𝑣 are WHITE at time 𝑢. 𝑑

•  (textbook p. 608)

49

DFS Properties

• Classification of Edges in 𝐺

• Tree Edge (GRAY to WHITE)

• Edges in the DFS forest

• Found when encountering a new vertex 𝑣 by exploring 𝑢, 𝑣

• Back Edge (GRAY to GRAY)

• 𝑢, 𝑣 , from descendant 𝑢 to ancestor 𝑣 in a DFS tree

• Forward Edge (GRAY to BLACK)

• 𝑢, 𝑣 , from ancestor 𝑢 to descendant 𝑣. Not a tree edge.

• Cross Edge (GRAY to BLACK)

• Any other edge between trees or subtrees. Can go between vertices in same DFS tree or in different DFS trees

In an undirected graph, back edge = forward edge.

To avoid ambiguity, classify edge as the first type in the list that applies.

DFS Properties

• Edge classification by the color of 𝑣 when visiting 𝑢, 𝑣

• WHITE: tree edge

• GRAY: back edge

• BLACK: forward edge or cross edge

• 𝑢. 𝑑 < 𝑣. 𝑑 → forward edge

• 𝑢. 𝑑 > 𝑣. 𝑑 → cross edge

51

Why?

Theorem 22.10

In DFS of an undirected graph, there are only tree edges and back edges without forward and cross edge.

DFS Applications

• Connected Components

• Strongly Connected Components

• Topological Sort

Connected Components

53

Connected Components Problem

• Input: a graph 𝐺 = 𝑉, 𝐸

• Output: a connected component of 𝐺

• a maximal subset 𝑈 of 𝑉 s.t. any two nodes in 𝑈 are connected in 𝐺

Why must the connected components of a graph be disjoint?

Connected Components

55

10 1

2

5

3 4

6

7

8 9

Time Complexity:

BFS and DSF both find the connected components with the same complexity

Problem Complexity

Strongly Connected Components

57

Textbook Chapter 22.5 – Strongly connected components

Strongly Connected Components

• Input: a directed graph 𝐺 = 𝑉, 𝐸

• Output: a connected component of 𝐺

• a maximal subset 𝑈 of 𝑉 s.t. any two nodes in 𝑈 are reachable in 𝐺

1

2

4

6 3

5

7

8

Why must the strongly connected components of a graph be disjoint?

Algorithm

• Step 1: Run DFS on 𝐺 to obtain the finish time 𝑣. 𝑓 for 𝑣 ∈ 𝑉.

• Step 2: Run DFS on the transpose of 𝐺 where the vertices 𝑉 are processed in the decreasing order of their finish time.

• Step 3: output the vertex partition by the second DFS

59

Transpose of A Graph

1

2

4

6 3

5

1

2

4

6 3

5

Example Illustration

61

1

3

2

6 5

4 1

2

4

5 3

6

Algorithm Correctness

• Proof by contradiction

• Assume that 𝑣, 𝑤 is an incoming edge to 𝐶.

• Since 𝐶 is a strongly connected component of 𝐺, there cannot be any path from any node of 𝐶 to 𝑣 in 𝐺.

• Therefore, the finish time of 𝑣 has to be larger than any node in 𝐶, including 𝑢. → 𝑣. 𝑓 > 𝑢. 𝑓, contradiction

Lemma

Let 𝐶 be the strongly connected component of 𝐺 (and 𝐺^𝑇) that contains the node 𝑢 with the largest finish time 𝑢. 𝑓. Then 𝐶 cannot have any incoming edge from any node of 𝐺 not in 𝐶.

𝑢

G

C 𝑤

𝑣

Algorithm Correctness

• Practice to prove using induction

63

𝑢

G

C

𝑢

G^T

C Theorem

By continuing the process from the vertex 𝑢^∗ whose finish time 𝑢^∗. 𝑓 is the largest excluding those in 𝐶, the algorithm returns the strongly connected components.

Example

1

3

2

6 5

4

Example

65

1

3

2

6 5

4

Time Complexity

• Step 1: Run DFS on 𝐺 to obtain the finish time 𝑣. 𝑓 for 𝑣 ∈ 𝑉.

• Step 2: Run DFS on the transpose of 𝐺 where the vertices 𝑉 are processed in the decreasing order of their finish time.

• Step 3: output the vertex partition by the second DFS

Time Complexity:

Problem Complexity

67

Topological Sort

Textbook Chapter 22.4 – Topological sort

Directed Graph

69

1

2

3

5 4

6 1

2

3

5 4

6

Directed Acyclic Graph (DAG)

• Definition

• a directed graph without any directed cycle

1

2

3

5 4

6

Topological Sort Problem

• Taking courses should follow the specific order

• How to find a course taking order?

71

計程 資料結構 演算法

計概 作業系統

計算機網路

微積分上 微積分下 機率

計組

Topological Sort Problem

• Input: a directed acyclic graph 𝐺 = (𝑉, 𝐸)

• Output: a linear order of 𝑉 s.t. all edges of 𝐺 going from lower-indexed nodes to higher-indexed nodes (左→右)

a b d

f c e

a

b

d

f c

f b d e

a c e

Algorithm

• Run DFS on the input DAG G.

• Output the nodes in decreasing order of their finish time.

73

DFS(G)

for each vertex u in G.V u.color = WHITE

u.pi = NIL time = 0

for each vertex u in G.V if u.color == WHITE

DFS-VISIT(G, u)

DFS-Visit(G, u) time = time + 1 u.d = time

u.color = GRAY

for each v in G.Adj[u] (outgoing) if v.color == WHITE

v.pi = u

DFS-VISIT(G, v) u.color = BLACK

time = time + 1

u.f = time // finish time

Example Illustration

a

b

d

f c

e

a b d

f c e

1

2

3 4

5

6

Example Illustration

75

a

b

d

f c

e

f b d

a c e

1

2

3 4

6

5

Time Complexity

• Run DFS on the input DAG G.

• Output the nodes in decreasing order of their finish time.

• As each vertex is finished, insert it onto the front of a linked list

• Return the linked list of vertices

DFS(G)

for each vertex u in G.V u.color = WHITE

u.pi = NIL time = 0

for each vertex u in G.V if u.color == WHITE

DFS-VISIT(G, u)

DFS-Visit(G, u) time = time + 1 u.d = time

u.color = GRAY

for each v in G.Adj[u]

if v.color == WHITE v.pi = u

DFS-VISIT(G, v) u.color = BLACK

time = time + 1

u.f = time // finish time

Time Complexity:

Algorithm Correctness

• Proof

• →: suppose there is a back edge 𝑢, 𝑣

• 𝑣 is an ancestor of 𝑢 in DFS forest

• There is a path from 𝑣 to 𝑢 in 𝐺 and 𝑢, 𝑣 completes the cycle

•  : suppose there is a cycle 𝑐

• Let 𝑣 be the first vertex in 𝑐 to be discovered and 𝑢 is a predecessor of 𝑣 in 𝑐

• Upon discovering 𝑣 the whole cycle from 𝑣 to 𝑢 is WHITE

• At time 𝑣. 𝑑, the vertices of 𝑐 form a path of white vertices from 𝑣 to 𝑢

• By the white-path theorem, vertex 𝑢 becomes a descendant of 𝑣 in the DFS forest

• Therefore, 𝑢, 𝑣 is a back edge

77

Lemma 22.11

A directed graph is acyclic  a DFS yields no back edges.

Algorithm Correctness

• Proof

• When 𝑢, 𝑣 is being explored, 𝑢 is GRAY and there are three cases for 𝑣:

• Case 1 – GRAY

• 𝑢, 𝑣 is a back edge (contradicting Lemma 22.11), so 𝑣 cannot be GRAY

• Case 2 – WHITE

• 𝑣 becomes descendant of 𝑢

• 𝑣 will be finished before 𝑢

• Case 3 – BLACK

• 𝑣 is already finished

Theorem 22.12

The algorithm produces a topological sort of the input DAG. That is, if 𝑢, 𝑣 is a directed edge (from 𝑢 to 𝑣) of 𝐺, then 𝑢. 𝑓 > 𝑣. 𝑓.

Problem Complexity

79

Discussion

• Since cycle detection becomes back edge detection (Lemma 22.11), DFS can be used to test whether a graph is a DAG

• Is there a topological order for cyclic graphs?

• Given a topological order, is there always a DFS traversal that produces such

an order?

To Be Continued…

81

Question?

Important announcement will be sent to

@ntu.edu.tw mailbox & post to the course website

Course Website: http://ada.miulab.tw Email: ada-ta@csie.ntu.edu.tw