Mini-HW 8 released
Due on 12/07 (Thur) 17:20
Homework 3 released
Due on 12/14 (Thur) 17:20 (two weeks)
Next-week room changed!!
12/07 (Thur) to forever
Location: R103
Midterm discussion
Today 16:30-17:20
Location: R103
2
Frequently check the website for the updated information!
Minimal Spanning Trees (MST)
Boruvka’s Algorithm
Kruskal’s Algorithm
Prim’s Algorithm
Single-Source Shortest Paths
Bellman-Ford Algorithm
Lawler Algorithm (SSSP in DAG)
Dijkstra Algorithm
4
5
Definition
a subgraph that is a tree and connects all vertices
Exactly 𝑛 − 1 edges
Acyclic
There can be many spanning trees of a graph
BFS and DFS also generate spanning trees
BFS tree is typically “short and bushy”
DFS tree is typically “long and stringy”
6
2
1 1
2 2
3
1
Input: a connected 𝑛-node 𝑚-edge graph 𝐺 with edge weights 𝑤
Output: a spanning tree 𝑇 of 𝐺 with minimum 𝑤(𝑇)
2
1
1
1
2
2 2
3
1
Q: What if the graph is unweighted?
Q: What if the graph contains edges with negative weights?
8
Trivial
Add a large constant to every edge; a MST remains the same
Proof by contradiction
Suppose there are two MSTs 𝐴 and 𝐵
Let 𝑒 be the least-weight edge in 𝐴⋃𝐵 and 𝑒 is not in both
WLOG, assume 𝑒 is in 𝐴
Add 𝑒 to 𝐵; 𝑒 ⋃𝐵 contains a cycle 𝐶
B includes at least one edge 𝑒′ that is not in 𝐴 but on 𝐶
Replacing 𝑒′ with 𝑒 yields a MST with less cost
Theorem: MST is unique if all edge weights are distinct
If edge weights are not all distinct, then the (multi-)set of weights in MST is unique
10
Otakar Borůvka
Czech scientist
Introduced the problem
Gave an 𝑂 𝑚 log 𝑛 time algorithm
The original paper was written in Czech in 1926
The purpose was to efficiently provide electric coverage of Bohemia
Repeat the following procedure until the resulting graph becomes a single node
For each node 𝑢, mark its lightest incident edge
From the marked edges form a forest 𝐹, add the edges of 𝐹 into the set of edges to be reported
Contract each maximal subtree of 𝐹 into a single node
12
2.1
1.3
2.3
1.2 2.2
3.1
2.4 3
1
1.5
1.4
2.6
2.7 2.5
3.2
5
3.3 4
4.1 5.1
Proof via contradiction
An MST 𝑇 of 𝐺 that does not contain 𝑢, 𝑣
A cycle 𝐶 = 𝑇 ∪ 𝑢, 𝑣 contains an edge 𝑢, 𝑤 in 𝐶 that has larger weight than 𝑢, 𝑣
𝑇′ = 𝑇 ∪ 𝑢, 𝑣 \ 𝑢, 𝑤 must be a spanning tree of 𝐺 lighter than 𝑇
14
Claim: If 𝑢, 𝑣 is the lightest edge incident to 𝑢 in 𝐺, 𝑢, 𝑣 must belong to any MST of 𝐺
u
v w
The recurrence relation
We check all edges in each phase
After each contraction phase, the number of nodes is reduced by at least one half
Time complexity:
Proof by contradiction
Suppose 𝑒 is in the MST
Removing 𝑒 disconnects the MST into two components T1 and T2
There exists another edge 𝑒′ in 𝐶 that can reconnect T1 and T2
Since 𝑤 𝑒’ < 𝑤(𝑒), the new tree has a lower weight
Contradiction!
16
Let 𝐶 be any cycle in the graph 𝐺, and let 𝑒 be an edge with the maximum weight on 𝐶. Then the MST does not contain 𝑒.
• For simplicity, assume all edge weights are distinct
Proof by contradiction
Suppose 𝑒 is not in the current MST
Adding 𝑒 creates a cycle in the MST
There exists another edge 𝑒′ in 𝐶 that can break the cycle
Since 𝑤 𝑒’ > 𝑤(𝑒), the new tree has a lower weight
Contradiction!
Let 𝐶 be a cut in the graph, and let 𝑒 be the edge with the minimum cost in 𝐶. Then the MST contains 𝑒.
• Cut = a partition of the vertices
• For simplicity, assume all edge weights are distinct
Textbook Chapter 23.2 – The algorithms of Kruskal and Prim
18
For each node 𝑢
Make-set(𝑢): create a set consisting of 𝑢
For each edge 𝑢, 𝑣 , taken in non-decreasing order by weights
if Find-set(𝑢) ≠Find-set(𝑣) (i.e., 𝑢 and 𝑣 are not in the same set) then
Output edge 𝑢, 𝑣
Union(𝑢, 𝑣): union the sets containing 𝑢 and 𝑣 into a single set
20
2.1
1.3 1.2
2.2
1
1.5
1.4 2.7
2.5 3.2
4.1 3.1 2.3
2.4 3
5 2.6 3.3 4
5.1
The lightest edge incident to a vertex must be in the MST
Consider whether adding 𝑒 creates a cycle:
If adding 𝑒 to 𝑇 creates a cycle 𝐶
Then 𝑒 is the max weight edge in 𝐶
The cycle property ensures that 𝑒 is not in the MST
If adding 𝑒 = 𝑢, 𝑣 to 𝑇 does not create a cycle
Before adding 𝑒, the current MST can be divided into two trees T1 and T2 such that 𝑢 in T1 and 𝑉 in T2
𝑒 is the minimum-cost edge on the cut of T1 and T2
The cut property ensures that 𝑒 is in the MST
22
MST-KRUSKAL(G, w) // w = weights A = empty // edge set of MST for v in G.V
MAKE-SET(v)
sort edges of G.E into non-decreasing order by weight w for (u, v) in G.E, taken in non-decreasing order by weight
if FIND-SET(u) ≠ FIND-SET(v) A = A ∪ {u, v}
UNION(u, v) return A
Disjoint-set data structure with union-by-rank (Textbook Ch. 21)
MAKE-SET:
FIND-SET:
UNION:
The amortized cost of 𝑚 operations on 𝑛 elements (Exercise 21.4-4):
Textbook Chapter 23.2 – The algorithms of Kruskal and Prim
24
Let 𝑇 consist of an arbitrary node
For 𝑖 = 1 to 𝑛 − 1
add the least-weighted edge incident to the current subtree 𝑇 that does not incur a cycle
26
13
50
11
7
2
8 12
9
10 40 14
1 3 6
20
13
50
11
7
2
8 12
9
10 40 14
1 3 6
20
28
13
50
11
7
2
8 12
9
10 40 14
1 3 6
20
13
50
11
7
2
8 12
9
10 40 14
1 3 6
20
30
13
50
11
7
2
8 12
9
10 40 14
1 3 6
20
13
50
11
7
2
8 12
9
10 40 14
1 3 6
20
32
13
50
11
7
2
8 12
9
10 40 14
1 3 6
20
13
50
11
7
2
8 12
9
10 40 14
1 3 6
20
34
13
50
11
7
2
8 12
9
10 40 14
1 3 6
20
The lightest edge incident to a vertex must be in the MST
36
MST-PRIM(G, w, r) // w = weights, r = root for u in G.V
u.key = ∞ u.π = NIL r.key = 0 Q = G.V
while Q ≠ empty
u = EXTRACT-MIN(Q) for v in G.adj[u]
if v ∈ Q and w(u, v) < v.key v.π = u
v.key = w(u, v) // DECREASE-KEY
Binary min-heap (Textbook Ch. 6)
BUILD-MIN-HEAP:
EXTRACT-MIN:
DECREASE-KEY:
Total complexity:
Fibonacci heap (Textbook Ch. 19)
BUILD-MIN-HEAP:
EXTRACT-MIN: (amortized)
DECREASE-KEY: (amortized)
MST-PRIM(G, w, r) // w = weights, r = root for u in G.V
u.key = ∞ u.π = NIL r.key = 0 Q = G.V
while Q ≠ empty
u = EXTRACT-MIN(Q) for v in G.adj[u]
if v ∈ Q and w(u, v) < v.key v.π = u
v.key = w(u, v) // DECREASE-KEY
Textbook Chapter 24 – Single-Source Shortest Paths
38
Input: a weighted, directed graph 𝐺 = 𝑉, 𝐸
Weights can be arbitrary numbers, not necessarily distance
Weight function needs not satisfy triangle inequality
Output: a minimal-cost path from 𝑠 to 𝑡 s.t. 𝛿 𝑠, 𝑡 is the minimum weight from 𝑠 to 𝑡
Problem Variants
Single-source shortest-path problem
Single-destination shortest-path problem
Single-pair shortest-path problem
All-pair shortest path problem
Can a shortest path contain a negative-weight edge?
Can a shortest path contain a negative-weight cycle?
Can a shortest path contain a cycle?
40
Yes.
Doesn’t make sense.
No.
Input: a weighted, directed graph 𝐺 = 𝑉, 𝐸 and a source vertex 𝑠
Output: a minimal-cost path from 𝑠 to 𝑡, where 𝑡 ∈ 𝑉
Let 𝐺 = 𝑉, 𝐸 be a weighted, directed graph with no negative-weight cycles reachable from 𝑠
A shortest path tree 𝐺
′= 𝑉
′, 𝐸′ of 𝑠 is a subgraph of 𝐺 s.t.
𝑉′ is the set of vertices reachable from 𝑠 in 𝐺
𝐺′ forms a rooted tree with root 𝑠
For all 𝑣 ∈ 𝑉′, the unique simple path from 𝑠 to 𝑣 in 𝐺′ is a shortest path from 𝑠 to 𝑣 in 𝐺
42
Input: a weighted, directed graph 𝐺 = 𝑉, 𝐸 and a vertex 𝑠
Output: a tree 𝑇 rooted at 𝑠 s.t. the path from 𝑠 to 𝑢 of 𝑇 is
a shortest path from 𝑠 to 𝑢 in 𝐺
The shortest path tree problem is equivalent to finding the minimal cost 𝛿 𝑠, 𝑢 from 𝑠 to each node 𝑢 in 𝐺
The minimal cost from 𝑠 to 𝑢 in 𝐺 is the length of any shortest path from 𝑠 to 𝑢 in 𝐺
44
“equivalence”: a solution to either problem can be obtained from a solution to the other problem in linear time
Shortest Path Tree Problem
Single-Source Shortest Path Problem
=
45
Richard Bellman, 1920~1984
Norbert Wiener Prize in Applied Mathematics, 1970
Dickson Prize, Carnegie-Mellon University, 1970
John von Neumann Theory Award, 1976.
IEEE Medal of Honor, 1979,
Fellow of the American Academy of Arts and Sciences, 1975.
Membership in the National Academy of Engineering, 1977
Lester R. Ford, Jr. 1927~2017
A important contributor to the theory of network flow.
We will learn Ford and Fulkerson’s maximum flow algorithm in a couple of weeks.
46
Idea: estimate the value of 𝑑 𝑢 to approximate 𝛿 𝑠, 𝑢
Initialization
Let 𝑑 𝑢 = ∞ for 𝑢 ∈ 𝐺
Let 𝑑 𝑠 = 0
Repeat the following step for sufficient number of phases
For each edge 𝑢, 𝑣 ∈ 𝐸, relax edge 𝑢, 𝑣
Relaxing: If 𝑑 𝑣 > 𝑑 𝑢 + 𝑤 𝑢, 𝑣 , let 𝑑 𝑣 = 𝑑 𝑢 + 𝑤 𝑢, 𝑣
improve the estimation of 𝑑 𝑢
48
0
∞ ∞
6 3
2 1 4
2 7
3 6 5
∞ 5 ∞
0
∞ ∞
6 3
2 1 4
2 7
3 6 5
5 ∞
6
50
0
∞ 6
3
2 1 4
2 7
3 6 5
5 ∞
63
Observation: let 𝑃 be a shortest path from 𝑠 to 𝑟
For any vertex 𝑢 in 𝑃, the subpath of 𝑃 from 𝑠 to 𝑢 has to be a shortest path from 𝑠 to 𝑢 optimal substructure
For any edge 𝑢, 𝑣 in 𝑃, if 𝑑 𝑢 = 𝛿 𝑠, 𝑢 , then 𝑑 𝑣 = 𝛿 𝑠, 𝑣 also holds after relaxing edge 𝑢, 𝑣
If 𝐺 contains no negative cycles, then each node 𝑢 has a shortest path from 𝑠 to 𝑢 that has at most n – 1 edges
From observation, after the first 𝑖 phases of improvement via relaxation, the estimation of 𝑑 𝑢 for the first 𝑖 + 1 nodes 𝑢 in the path is precise (= 𝛿 𝑠, 𝑢 )
s u r
52
s u
Time complexity:
BELLMAN-FORD(G, w, s) INITIALIZATION(G, s) for i = 1 to |G.V| - 1
for (u, v) in G.E RELAX(u, v, w)
INITIALIZATION(G, s) for v in G.V
v.d = ∞ v.π = NIL s.d = 0
RELAX(u, v, w)
if v.d > u.d + w(u, v) // DECREASE-KEY
v.d = u.d + w(u, v) v.π = u
How to do if there is a
negative cycle in the graph?
Q: How do we know 𝐺 has negative cycles?
A: Using another phase of improvement via relaxation
Run another phase of improving the estimation of 𝑑 𝑢 for each vertex 𝑢 ∈ 𝑉 via relaxing all edges 𝐸
If in the 𝑛-th phase, there are still some 𝑑 𝑢 being modified, we know that 𝐺 has negative cycles
54
Proof by contradiction
Let 𝐶 be a negative cycle of 𝑘 nodes 𝑣1, 𝑣2, … , 𝑣𝑘 (𝑣𝑘+1 = 𝑣1)
Assume 𝑑 𝑣𝑖 for all 1 ≤ 𝑖 ≤ 𝑘 are not changed in a phase of improvement, then for 1 ≤ 𝑖 ≤ 𝑘
Summing all 𝑘 inequalities, the sum of edge weights of 𝐶 is nonnegative
If there exists a negative cycle in 𝐺, in the 𝑛-th phase, there are still some 𝑑 𝑢 being modified.
negative
Time complexity:
Finding a shortest-path tree of 𝐺:
56
BELLMAN-FORD(G, w, s) INITIALIZATION(G, s) for i = 1 to |G.V| - 1
for (u, v) in G.E RELAX(u, v, w) for (u, v) in G.E
if v.d > u.d + w(u, v) return FALSE
return TRUE
INITIALIZATION(G, s) for v in G.V
v.d = ∞ v.π = NIL s.d = 0
RELAX(u, v, w)
if v.d > u.d + w(u, v) // DECREASE-KEY
v.d = u.d + w(u, v) v.π = u
negative cycle detection
Textbook Chapter 24.2 – Single-source shortest paths in directed
57
Input: a weighted, directed, and acyclic graph 𝐺 = 𝑉, 𝐸 and a source vertex 𝑠
Output: a shortest-path distance from 𝑠 to 𝑡, where 𝑡 ∈ 𝑉
No negative cycle! 58
Idea: one phase relaxation
Perform a topological sort in linear time on the input DAG
For 𝑖 = 1 to 𝑛
Let 𝑣𝑖 be the 𝑖-th node in the above order
Relax each outgoing edge (𝑣𝑖, 𝑢) from 𝑣𝑖
Time complexity:
s
Assume this is a shortest path from 𝑠 to 𝑢
If we follow the order from topological sort to relax the
vertices’ edges, in this shortest path, the left edge must be relaxed before the right edge
One phase of improvement is enough
60
s u
61
Input: a non-negative weighted, directed, graph 𝐺 = 𝑉, 𝐸 and a source vertex 𝑠
Output: a shortest-path distance from 𝑠 to 𝑡, where 𝑡 ∈ 𝑉
No negative cycle! 62
Idea: BFS finds shortest paths on unweighted graph by expanding the search frontier
Initialization
Loops for 𝑛 iterations, where each iteration
relax outgoing edges of an unprocessed node 𝑢 with minimal 𝑑 𝑢
marks 𝑢 as processed
64
0
∞ ∞
∞ ∞
10
2
3 1
4 6
9
7
2 5
0
10 ∞
5 ∞
10
2
3 1
4 6
9
7
2 5
66
0
8 14
5 7
10
2
3 1
4 6
9
7
2 5
0
8 13
5 7
10
2
3 1
4 6
9
7
2 5
68
0
8 9
5 7
10
2
3 1
4 6
9
7
2 5
0
8 9
5 7
10
2
3 1
4 6
9
7
2 5
70
0
8 9
5 7
10
2
3 1
4 6
9
7
2 5
Prove by contradiction
Assume 𝑢 is the first vertex for being processed
Let a shortest path 𝑃 from 𝑠 to 𝑢,
𝑥 is the last vertex in 𝑃 from 𝑆
𝑦 is the first vertex in 𝑃 not from 𝑆
𝑑 𝑦 = 𝛿 𝑠, 𝑦 because 𝑥, 𝑦 is relaxed when putting 𝑥 into 𝑆
s
x y
u
processed nodes 𝑆
The vertex selected by Dijkstra’s algorithm into the processed set must precise estimation of its shortest path distance.
The first node
a shortest path from 𝑠 to 𝑢
Min-priority queue
INSERT:
EXTRACT-MIN:
DECREASE-KEY:
Total complexity:
72
DIJKSTRA(G, w, s)
INITIALIZATION(G, s) S = empty
Q = G.v // INSERT while Q ≠ empty
u = EXTRACT-MIN(Q) S = S∪{u}
for v in G.adj[u]
RELAX(u, v, w)
INITIALIZATION(G, s) for v in G.V
v.d = ∞ v.π = NIL s.d = 0
RELAX(u, v, w)
if v.d > u.d + w(u, v) // DECREASE-KEY
v.d = u.d + w(u, v) v.π = u
Fibonacci heap (Textbook Ch. 19)
BUILD-MIN-HEAP:
EXTRACT-MIN: (amortized)
DECREASE-KEY: (amortized)
Total complexity:
DIJKSTRA(G, w, s)
INITIALIZATION(G, s) S = empty
Q = G.v // INSERT while Q ≠ empty
u = EXTRACT-MIN(Q) S = S∪{u}
for v in G.adj[u]
RELAX(u, v, w)
INITIALIZATION(G, s) for v in G.V
v.d = ∞ v.π = NIL s.d = 0
RELAX(u, v, w)
if v.d > u.d + w(u, v) // DECREASE-KEY
v.d = u.d + w(u, v) v.π = u
Minimal Spanning Trees (MST)
Boruvka’s Algorithm:
Kruskal’s Algorithm:
Prim’s Algorithm: with Fabonacci heap
Single-Source Shortest Paths
Bellman-Ford Algorithm (general graph and weights)
and detecting negative cycles
Lawler Algorithm (acyclic graph)
Dijkstra Algorithm (non-negative weights)
with Fabonacci heap
74
Course Website: http://ada17.csie.org
75
Important announcement will be sent to @ntu.edu.tw mailbox
& post to the course website
