Mini-HW 8 released

Due on 12/07 (Thur) 17:20

Homework 3 released

Due on 12/14 (Thur) 17:20 (two weeks)

Next-week room changed!!

12/07 (Thur) to forever

Location: R103

Midterm discussion

Today 16:30-17:20

Location: R103

**2**

Frequently check the website for the updated information!

### Minimal Spanning Trees (MST)

Boruvka’s Algorithm

Kruskal’s Algorithm

Prim’s Algorithm

### Single-Source Shortest Paths

Bellman-Ford Algorithm

Lawler Algorithm (SSSP in DAG)

Dijkstra Algorithm

**4**

**5**

### Definition

a subgraph that is a tree and connects all vertices

Exactly 𝑛 − 1 edges

Acyclic

There can be many spanning trees of a graph

### BFS and DFS also generate spanning trees

BFS tree is typically “short and bushy”

DFS tree is typically “long and stringy”

**6**

2

1 1

2 2

3

1

### Input: a connected 𝑛-node 𝑚-edge graph 𝐺 with edge weights 𝑤

### Output: a spanning tree 𝑇 of 𝐺 with minimum 𝑤(𝑇)

2

1

1

1

2

2 2

3

1

### Q: What if the graph is unweighted?

### Q: What if the graph contains edges with negative weights?

**8**

### Trivial

### Add a large constant to every edge; a MST remains the same

### Proof by contradiction

Suppose there are two MSTs 𝐴 and 𝐵

Let 𝑒 be the least-weight edge in 𝐴⋃𝐵 and 𝑒 is not in both

WLOG, assume 𝑒 is in 𝐴

Add 𝑒 to 𝐵; 𝑒 ⋃𝐵 contains a cycle 𝐶

B includes at least one edge 𝑒′ that is not in 𝐴 but on 𝐶

Replacing 𝑒′ with 𝑒 yields a MST with less cost

Theorem: MST is unique if all edge weights are distinct

If edge weights are not all distinct, then the (multi-)set of weights in MST is unique

**10**

### Otakar Borůvka

### Czech scientist

### Introduced the problem

### Gave an 𝑂 𝑚 log 𝑛 time algorithm

The original paper was written in Czech in 1926

The purpose was to efficiently provide electric coverage of Bohemia

### Repeat the following procedure until the resulting graph becomes a single node

For each node 𝑢, mark its lightest incident edge

From the marked edges form a forest 𝐹, add the edges of 𝐹 into the set of edges to be reported

Contract each maximal subtree of 𝐹 into a single node

**12**

2.1

1.3

2.3

1.2 2.2

3.1

2.4 3

1

1.5

1.4

2.6

2.7 2.5

3.2

5

3.3 4

4.1 5.1

### Proof via contradiction

An MST 𝑇 of 𝐺 that does not contain 𝑢, 𝑣

A cycle 𝐶 = 𝑇 ∪ 𝑢, 𝑣 contains an edge 𝑢, 𝑤 in 𝐶 that has larger weight than 𝑢, 𝑣

𝑇^{′} = 𝑇 ∪ 𝑢, 𝑣 \ 𝑢, 𝑤 must be a spanning tree of 𝐺 lighter
than 𝑇

**14**

Claim: If 𝑢, 𝑣 is the lightest edge incident to 𝑢 in 𝐺, 𝑢, 𝑣 must belong to any MST of 𝐺

*u*

*v*
*w*

### The recurrence relation

We check all edges in each phase

After each contraction phase, the number of nodes is reduced by at least one half

### Time complexity:

### Proof by contradiction

Suppose 𝑒 is in the MST

Removing 𝑒 disconnects the MST into two components T1 and T2

There exists another edge 𝑒′ in 𝐶 that can reconnect T1 and T2

Since 𝑤 𝑒’ < 𝑤(𝑒), the new tree has a lower weight

Contradiction!

**16**

Let 𝐶 be any cycle in the graph 𝐺, and let 𝑒 be an edge with the maximum weight on 𝐶. Then the MST does not contain 𝑒.

• For simplicity, assume all edge weights are distinct

### Proof by contradiction

Suppose 𝑒 is not in the current MST

Adding 𝑒 creates a cycle in the MST

There exists another edge 𝑒′ in 𝐶 that can break the cycle

Since 𝑤 𝑒’ > 𝑤(𝑒), the new tree has a lower weight

Contradiction!

Let 𝐶 be a cut in the graph, and let 𝑒 be the edge with the minimum cost in 𝐶. Then the MST contains 𝑒.

• Cut = a partition of the vertices

• For simplicity, assume all edge weights are distinct

Textbook Chapter 23.2 – The algorithms of Kruskal and Prim

**18**

### For each node 𝑢

Make-set(𝑢): create a set consisting of 𝑢

### For each edge 𝑢, 𝑣 , taken in non-decreasing order by weights

### if Find-set(𝑢) ≠Find-set(𝑣) (i.e., 𝑢 and 𝑣 are not in the same set) then

Output edge 𝑢, 𝑣

Union(𝑢, 𝑣): union the sets containing 𝑢 and 𝑣 into a single set

**20**

2.1

1.3 1.2

2.2

1

1.5

1.4 2.7

2.5 3.2

4.1 3.1 2.3

2.4 3

5 2.6 3.3 4

5.1

The lightest edge incident to a vertex must be in the MST

### Consider whether adding 𝑒 creates a cycle:

If adding ^{𝑒} to 𝑇 creates a cycle 𝐶

Then 𝑒 is the max weight edge in 𝐶

The cycle property ensures that 𝑒 is not in the MST

If adding ^{𝑒 = 𝑢, 𝑣} to 𝑇 does not create a cycle

Before adding 𝑒, the current MST can be divided into two trees T1 and T2 such that 𝑢 in T1 and 𝑉 in T2

𝑒 is the minimum-cost edge on the cut of T1 and T2

The cut property ensures that 𝑒 is in the MST

**22**

MST-KRUSKAL(G, w) // w = weights A = empty // edge set of MST for v in G.V

MAKE-SET(v)

sort edges of G.E into non-decreasing order by weight w for (u, v) in G.E, taken in non-decreasing order by weight

if FIND-SET(u) ≠ FIND-SET(v) A = A ∪ {u, v}

UNION(u, v) return A

Disjoint-set data structure with union-by-rank (Textbook Ch. 21)

MAKE-SET:

FIND-SET:

UNION:

The amortized cost of 𝑚 operations on 𝑛 elements (Exercise 21.4-4):

Textbook Chapter 23.2 – The algorithms of Kruskal and Prim

**24**

### Let 𝑇 consist of an arbitrary node

### For 𝑖 = 1 to 𝑛 − 1

add the least-weighted edge incident to the current subtree 𝑇 that does not incur a cycle

**26**

13

50

11

7

2

8 12

9

10 40 14

1 3 6

20

13

50

11

7

2

8 12

9

10 40 14

1 3 6

20

**28**

13

50

11

7

2

8 12

9

10 40 14

1 3 6

20

13

50

11

7

2

8 12

9

10 40 14

1 3 6

20

**30**

13

50

11

7

2

8 12

9

10 40 14

1 3 6

20

13

50

11

7

2

8 12

9

10 40 14

1 3 6

20

**32**

13

50

11

7

2

8 12

9

10 40 14

1 3 6

20

13

50

11

7

2

8 12

9

10 40 14

1 3 6

20

**34**

13

50

11

7

2

8 12

9

10 40 14

1 3 6

20

The lightest edge incident to a vertex must be in the MST

**36**

MST-PRIM(G, w, r) // w = weights, r = root for u in G.V

u.key = ∞ u.π = NIL r.key = 0 Q = G.V

while Q ≠ empty

u = EXTRACT-MIN(Q) for v in G.adj[u]

if v ∈ Q and w(u, v) < v.key v.π = u

v.key = w(u, v) // DECREASE-KEY

Binary min-heap (Textbook Ch. 6)

BUILD-MIN-HEAP:

EXTRACT-MIN:

DECREASE-KEY:

Total complexity:

Fibonacci heap (Textbook Ch. 19)

BUILD-MIN-HEAP:

EXTRACT-MIN: (amortized)

DECREASE-KEY: (amortized)

MST-PRIM(G, w, r) // w = weights, r = root for u in G.V

u.key = ∞ u.π = NIL r.key = 0 Q = G.V

while Q ≠ empty

u = EXTRACT-MIN(Q) for v in G.adj[u]

if v ∈ Q and w(u, v) < v.key v.π = u

v.key = w(u, v) // DECREASE-KEY

Textbook Chapter 24 – Single-Source Shortest Paths

**38**

### Input: a weighted, directed graph 𝐺 = 𝑉, 𝐸

Weights can be arbitrary numbers, not necessarily distance

Weight function needs not satisfy triangle inequality

### Output: a minimal-cost path from 𝑠 to 𝑡 s.t. 𝛿 𝑠, 𝑡 is the minimum weight from 𝑠 to 𝑡

### Problem Variants

Single-source shortest-path problem

Single-destination shortest-path problem

Single-pair shortest-path problem

All-pair shortest path problem

### Can a shortest path contain a negative-weight edge?

### Can a shortest path contain a negative-weight cycle?

### Can a shortest path contain a cycle?

**40**

### Yes.

### Doesn’t make sense.

### No.

### Input: a weighted, directed graph 𝐺 = 𝑉, 𝐸 and a source vertex 𝑠

### Output: a minimal-cost path from 𝑠 to 𝑡, where 𝑡 ∈ 𝑉

### Let 𝐺 = 𝑉, 𝐸 be a weighted, directed graph with no negative-weight cycles reachable from 𝑠

### A shortest path tree 𝐺

^{′}

### = 𝑉

^{′}

### , 𝐸′ of 𝑠 is a subgraph of 𝐺 s.t.

### 𝑉′ is the set of vertices reachable from 𝑠 in 𝐺

### 𝐺′ forms a rooted tree with root 𝑠

### For all 𝑣 ∈ 𝑉′, the unique simple path from 𝑠 to 𝑣 in 𝐺′ is a shortest path from 𝑠 to 𝑣 in 𝐺

**42**

### Input: a weighted, directed graph 𝐺 = 𝑉, 𝐸 and a vertex 𝑠

### Output: a tree 𝑇 rooted at 𝑠 s.t. the path from 𝑠 to 𝑢 of 𝑇 is

### a shortest path from 𝑠 to 𝑢 in 𝐺

### The shortest path tree problem is equivalent to finding the minimal cost 𝛿 𝑠, 𝑢 from 𝑠 to each node 𝑢 in 𝐺

### The minimal cost from 𝑠 to 𝑢 in 𝐺 is the length of any shortest path from 𝑠 to 𝑢 in 𝐺

**44**

“equivalence”: a solution to either problem can be obtained from a solution to the other problem in linear time

### Shortest Path Tree Problem

### Single-Source Shortest Path Problem

### =

**45**

**Richard Bellman, 1920~1984**

Norbert Wiener Prize in Applied Mathematics, 1970

Dickson Prize, Carnegie-Mellon University, 1970

John von Neumann Theory Award, 1976.

IEEE Medal of Honor, 1979,

Fellow of the American Academy of Arts and Sciences, 1975.

Membership in the National Academy of Engineering, 1977

**Lester R. Ford, Jr. 1927~2017**

A important contributor to the theory of network flow.

We will learn Ford and Fulkerson’s maximum flow algorithm in a couple of weeks.

**46**

### Idea: estimate the value of 𝑑 𝑢 to approximate 𝛿 𝑠, 𝑢

### Initialization

Let 𝑑 𝑢 = ∞ for 𝑢 ∈ 𝐺

Let 𝑑 𝑠 = 0

### Repeat the following step for sufficient number of phases

For each edge 𝑢, 𝑣 ∈ 𝐸, relax edge 𝑢, 𝑣

Relaxing: If 𝑑 𝑣 > 𝑑 𝑢 + 𝑤 𝑢, 𝑣 , let 𝑑 𝑣 = 𝑑 𝑢 + 𝑤 𝑢, 𝑣

improve the estimation of 𝑑 𝑢

**48**

0

### ∞ ∞

### 6 3

### 2 1 4

### 2 7

### 3 6 5

### ∞ **5** ∞

0

### ∞ ∞

### 6 3

### 2 1 4

### 2 7

### 3 6 5

**5** ∞

**6**

**50**

0

### ∞ 6

### 3

### 2 1 4

### 2 7

### 3 6 5

**5** ∞

**63**

Observation: let 𝑃 be a shortest path from 𝑠 to 𝑟

For any vertex 𝑢 in 𝑃, the subpath of 𝑃 from 𝑠 to 𝑢 has to be a shortest path from 𝑠 to 𝑢 optimal substructure

For any edge 𝑢, 𝑣 in 𝑃, if 𝑑 𝑢 = 𝛿 𝑠, 𝑢 , then 𝑑 𝑣 = 𝛿 𝑠, 𝑣 also holds after relaxing edge 𝑢, 𝑣

If 𝐺 contains no negative cycles, then each node 𝑢 has a shortest
*path from 𝑠 to 𝑢 that has at most n – 1 edges*

From observation, after the first 𝑖 phases of improvement via relaxation, the estimation of 𝑑 𝑢 for the first 𝑖 + 1 nodes 𝑢 in the path is precise (= 𝛿 𝑠, 𝑢 )

*s* *u* *r*

**52**

*s* *u*

Time complexity:

BELLMAN-FORD(G, w, s) INITIALIZATION(G, s) for i = 1 to |G.V| - 1

for (u, v) in G.E RELAX(u, v, w)

INITIALIZATION(G, s) for v in G.V

v.d = ∞ v.π = NIL s.d = 0

RELAX(u, v, w)

if v.d > u.d + w(u, v) // DECREASE-KEY

v.d = u.d + w(u, v) v.π = u

How to do if there is a

negative cycle in the graph?

### Q: How do we know 𝐺 has negative cycles?

### A: Using another phase of improvement via relaxation

Run another phase of improving the estimation of 𝑑 𝑢 for each vertex 𝑢 ∈ 𝑉 via relaxing all edges 𝐸

If in the 𝑛-th phase, there are still some 𝑑 𝑢 being modified, we know that 𝐺 has negative cycles

**54**

### Proof by contradiction

Let 𝐶 be a negative cycle of 𝑘 nodes 𝑣_{1}, 𝑣_{2}, … , 𝑣_{𝑘} (𝑣_{𝑘+1} = 𝑣_{1})

Assume 𝑑 𝑣_{𝑖} for all 1 ≤ 𝑖 ≤ 𝑘 are not changed in a phase of
improvement, then for 1 ≤ 𝑖 ≤ 𝑘

Summing all 𝑘 inequalities, the sum of edge weights of 𝐶 is nonnegative

If there exists a negative cycle in 𝐺, in the 𝑛-th phase, there are still some 𝑑 𝑢 being modified.

negative

Time complexity:

Finding a shortest-path tree of 𝐺:

**56**

BELLMAN-FORD(G, w, s) INITIALIZATION(G, s) for i = 1 to |G.V| - 1

for (u, v) in G.E RELAX(u, v, w) for (u, v) in G.E

if v.d > u.d + w(u, v) return FALSE

return TRUE

INITIALIZATION(G, s) for v in G.V

v.d = ∞ v.π = NIL s.d = 0

RELAX(u, v, w)

if v.d > u.d + w(u, v) // DECREASE-KEY

v.d = u.d + w(u, v) v.π = u

negative cycle detection

Textbook Chapter 24.2 – Single-source shortest paths in directed

**57**

### Input: a weighted, directed, and acyclic graph 𝐺 = 𝑉, 𝐸 and a source vertex 𝑠

### Output: a shortest-path distance from 𝑠 to 𝑡, where 𝑡 ∈ 𝑉

**No negative cycle!** **58**

### Idea: one phase relaxation

### Perform a topological sort in linear time on the input DAG

### For 𝑖 = 1 to 𝑛

Let 𝑣_{𝑖} be the 𝑖-th node in the above order

Relax each outgoing edge (𝑣_{𝑖}, 𝑢) from 𝑣_{𝑖}

### Time complexity:

*s*

### Assume this is a shortest path from 𝑠 to 𝑢

### If we follow the order from topological sort to relax the

### vertices’ edges, in this shortest path, the left edge must be relaxed before the right edge

### One phase of improvement is enough

**60**

*s* *u*

**61**

### Input: a non-negative weighted, directed, graph 𝐺 = 𝑉, 𝐸 and a source vertex 𝑠

### Output: a shortest-path distance from 𝑠 to 𝑡, where 𝑡 ∈ 𝑉

**No negative cycle!** **62**

### Idea: BFS finds shortest paths on unweighted graph by expanding the search frontier

### Initialization

### Loops for 𝑛 iterations, where each iteration

relax outgoing edges of an unprocessed node 𝑢 with minimal 𝑑 𝑢

marks 𝑢 as processed

**64**

0

∞ ∞

∞ ∞

10

2

3 1

4 6

9

7

2 5

0

10 ∞

5 ∞

10

2

3 1

4 6

9

7

2 5

**66**

0

8 14

5 7

10

2

3 1

4 6

9

7

2 5

0

8 13

5 7

10

2

3 1

4 6

9

7

2 5

**68**

0

8 9

5 7

10

2

3 1

4 6

9

7

2 5

0

8 9

5 7

10

2

3 1

4 6

9

7

2 5

**70**

0

8 9

5 7

10

2

3 1

4 6

9

7

2 5

### Prove by contradiction

Assume 𝑢 is the first vertex for being processed

Let a shortest path 𝑃 from 𝑠 to 𝑢,

𝑥 is the last vertex in 𝑃 from 𝑆

𝑦 is the first vertex in 𝑃 not from 𝑆

𝑑 𝑦 = 𝛿 𝑠, 𝑦 because 𝑥, 𝑦 is relaxed when putting 𝑥 into 𝑆

*s*

*x* *y*

*u*

processed nodes 𝑆

The vertex selected by Dijkstra’s algorithm into the processed set must precise estimation of its shortest path distance.

The first node

a shortest path from 𝑠 to 𝑢

Min-priority queue

INSERT:

EXTRACT-MIN:

DECREASE-KEY:

Total complexity:

**72**

DIJKSTRA(G, w, s)

INITIALIZATION(G, s) S = empty

Q = G.v // INSERT while Q ≠ empty

u = EXTRACT-MIN(Q) S = S∪{u}

for v in G.adj[u]

RELAX(u, v, w)

INITIALIZATION(G, s) for v in G.V

v.d = ∞ v.π = NIL s.d = 0

RELAX(u, v, w)

if v.d > u.d + w(u, v) // DECREASE-KEY

v.d = u.d + w(u, v) v.π = u

Fibonacci heap (Textbook Ch. 19)

BUILD-MIN-HEAP:

EXTRACT-MIN: (amortized)

DECREASE-KEY: (amortized)

Total complexity:

DIJKSTRA(G, w, s)

INITIALIZATION(G, s) S = empty

Q = G.v // INSERT while Q ≠ empty

u = EXTRACT-MIN(Q) S = S∪{u}

for v in G.adj[u]

RELAX(u, v, w)

INITIALIZATION(G, s) for v in G.V

v.d = ∞ v.π = NIL s.d = 0

RELAX(u, v, w)

if v.d > u.d + w(u, v) // DECREASE-KEY

v.d = u.d + w(u, v) v.π = u

### Minimal Spanning Trees (MST)

Boruvka’s Algorithm:

Kruskal’s Algorithm:

Prim’s Algorithm: with Fabonacci heap

### Single-Source Shortest Paths

Bellman-Ford Algorithm (general graph and weights)

and detecting negative cycles

Lawler Algorithm (acyclic graph)

Dijkstra Algorithm (non-negative weights)

with Fabonacci heap

**74**

Course Website: http://ada17.csie.org

**75**

Important announcement will be sent to @ntu.edu.tw mailbox

& post to the course website