All-Pairs Shortest Paths

All-Pairs Shortest Paths

HONG-MING CHU 2013/10/31

Refernece

Lecture slides from Prof. Hsin-Mu Tsai’s course slides and Prof. Ya- Yuin Su course slides.

Today’s Goal

 Quick recap of single source shortest path

 Floyd-Warshall algorithm

 Johson algorithm

Things we have learned so far

 Single-source shortest paths problem

 Two algorithms

 Bellman Ford algorithm

 Dijakstra’s algorithm

 Several properties about shorthest path

Optimal Substructure

 Theorem: A subpath of a shortest path is a shortest path

 If we decompose a path from 𝑣₀ to 𝑣_𝑘 to the following, then 𝑤 𝑝 = 𝑤 𝑝_0𝑖 + 𝑤 𝑝_𝑖𝑗 + 𝑤(𝑝_𝑗𝑘)

 If a shorter path 𝑝_𝑖𝑗^′ exists, then 𝑤 𝑝 = 𝑤 𝑝_0𝑖 + 𝑤 𝑝_𝑖𝑗^′ + 𝑤 𝑝_𝑗𝑘 < 𝑤 𝑝 , contradiction.

V⁰ Vⁱ V^j V^k

Triangle inequality

 For all vertices 𝑢, 𝑣, 𝑤 ∈ 𝑉 , 𝛿 𝑢, 𝑣 ≤ 𝛿 𝑢, 𝑤 + 𝛿 𝑤, 𝑣

 Idea: among all paths from u to v, a shortest path 𝛿 𝑢, 𝑣 will be shorter (or equal to) the path going from u to v through an intermediate node w by taking shortest path 𝛿 𝑢, 𝑤 and 𝛿 𝑤, 𝑣 .

Algortihms we have learned

Graph type Algorithm Runnging Time

Unweighted graph BFS O(V+E)

Non-negative edge weight graph

Dijkstra O(E+VlgV)

General graph Bellman-Ford O(VE)

DAG Bellman-Ford O(V+E)

Algortihms we have learned

 But what happen when the graph is dense?

ex. When |𝐸| ≈ |𝑉|^𝟐?

 And what happen when the graph is relatively sparse?

ex. When |E| = 𝜃 𝑉 ?

Algorithms we have learned

Graph type Algorithm Runnging Time |𝐸| ≈ |𝑉|^𝟐 |𝐸| = 𝜃 |𝑉|

Unweighted graph

BFS O(V+E) O(V²) O(V)

Non-negative edge weight

graph

Dijsktra O(E+VlgV) O(V²) O(VlgV)

General graph Bellman-Ford O(VE) O(V³) O(V²)

DAG Bellman-Ford O(V+E) O(V²) O(V)

All-pair shortest paths

 How about using the previous algorithms to solve all-pair shortest path problem ?

 Unweighted graph: run BFS |V| times → O(VE)

 Non-negative graph: run Dijkstra |V| times → Ο(VE+V²lgV)

 General case: run Bellman-Ford |V| times → Ο(V²E)

 When handling general cases, the time complexity is at most O(V⁴)……….

 We can do better!

But how…….?

 Recall that shortest paths has some useful properties.

 One of them is that a shortest path has an optimal substructure.

 As soon as we realize this…….

 Dynamic programming may be a good choice to solve this problem!

Floyd-Warshall algorithm

 First we label all vertices from 1 to |V|.

 Then define 𝐷^(𝑘) (k from 0 to |V|) to be an |V| * |V| matrix, and define each of its entry 𝑑_𝑖𝑗 to be the shortest path from i to j with intermediate vertices in set {1, 2, …, k}, if such path doesn’t exist, 𝑑_𝑖𝑗 = ∞ .

 Then define 𝑐_𝑖𝑗^(𝑘) to be the 𝑑_𝑖𝑗 in matrix 𝐷^(𝑘).

 So 𝑐_𝑖𝑗⁽⁰⁾ = 𝑤_𝑖𝑗 and 𝛿 𝑖, 𝑗 = 𝑐_𝑖𝑗^(|𝑉|).

The transition function



 If not → 𝑐_𝑖𝑗^(𝑘) = 𝑐_𝑖𝑗^(𝑘−1)

 Else → 𝑐_𝑖𝑗^(𝑘) = 𝑐_𝑖𝑘^(𝑘−1) + 𝑐_𝑘𝑗^(𝑘−1)

The transition function(Cont.)

 Since we are not yet sure if the intermediate vertices of 𝑐_𝑖𝑗^(𝑘) contains k, so both circumstances are possible.

 But fortunately, we know how to determine it!

 THE SHORTER, THE BETTER!

 So 𝑐_𝑖𝑗^(𝑘) = min(𝑐_𝑖𝑘^(𝑘−1) + 𝑐_𝑘𝑗^(𝑘−1), 𝑐_𝑖𝑗^(𝑘−1))

Pseudo code

for k = 1 to n

for i = 1 to n

for j = 1 to n

if 𝑐_𝑖𝑗>𝑐_𝑖𝑘+𝑐_𝑘𝑗

𝑐_𝑖𝑗 = 𝑐_𝑖𝑘+𝑐_𝑘𝑗 Running time : 𝜃 𝑉³

An Alternative: Transitive closure of a directed graph

 Determine if a graph G contains a path from vertex i to j for all vertices pairs

 𝑡_𝑖𝑗 = 1, 𝑖𝑓 𝑡ℎ𝑒𝑟𝑒 𝑒𝑥𝑖𝑠𝑡𝑠 𝑎 𝑝𝑎𝑡ℎ 𝑓𝑟𝑜𝑚 𝑖 𝑡𝑜 𝑗 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

 Idea: The key concepts of Floyd-warshall algorithm can be used on this question, but some modifications are needed.

An Alternative: Transitive closure of a directed graph(Cont.)

 The modification is shown as follow:

 Replace min with ∨ (logical OR)

 Replace + with ∧ (logical AND)

 𝑡_𝑖𝑗^(𝑘) = 𝑡_𝑖𝑗^(𝑘−1) ∧ ^(𝑡_𝑖𝑘^{(𝑘−1) ∨ 𝑡}_𝑘𝑗^(𝑘−1))

 The running time is also 𝜃 𝑉³ .

Another idea

 Floyd-Warshall yields a great improvement on time complexity.

 But when |𝐸| is relatively smaller, i.e. when 𝐸 = 𝜃 𝑉 , the improvement is not that significant……..

 Using Dijakstra’s algorithm 𝑉 times is now a good idea, but negative- weighted edge is a critical issue.

 Can we fix this?

Johnson’s algorithm

 Idea:

 Try to make all edges posstive.

 Then run Dijkstra’s algorithm |V| times.

 Solution: Graph Reweighting

 Given function ℎ : 𝑉 → ℛ, reweight each edge (𝑢,𝑣) ∈ 𝐸 by 𝑤_ℎ 𝑢, 𝑣 =

𝑤 𝑢, 𝑣 + ℎ 𝑢 − ℎ(𝑣), 𝑣 ∈ 𝑉. Then, for any vertices 𝑢, 𝑣 ∈ 𝑉, all paths have reweighted by the same amount.

Theorem

 Given function ℎ : 𝑉 → ℛ, reweight each edge (𝑢,𝑣)∈𝐸 by 𝑤_ℎ 𝑢, 𝑣 =

𝑤 𝑢, 𝑣 + ℎ 𝑢 − ℎ(𝑣), 𝑣 ∈ 𝑉. Then, for any vertices 𝑢, 𝑣 ∈ 𝑉, all paths are equally reweighted.

 Proof:

 Let 𝑝 = 𝑣₁ → 𝑣₂ → 𝑣₃ … 𝑣_𝑘 be a path in G

Theorem(Cont.)

 Proof:

 Let 𝑝 = 𝑣₁ → 𝑣₂ → 𝑣₃ … 𝑣_𝑘 be a path in G

𝑖=1 𝑘

𝑤_ℎ(𝑣_𝑖−1, 𝑣_𝑖) =

𝑖=1 𝑘

(𝑤_ℎ 𝑣_𝑖−1, 𝑣_𝑖 + ℎ 𝑣_𝑖−1 − ℎ(𝑣_𝑖)) =

𝑖=1 𝑘

𝑤_ℎ(𝑣_𝑖−1, 𝑣_𝑖) +

𝑖=1 𝑘

ℎ 𝑣_𝑖−1 − ℎ 𝑣_𝑖 = 𝑤 𝑝 + ℎ 𝑣₁ − ℎ 𝑣_𝑘

The same for every path

Collorary

 𝛿_ℎ 𝑢, 𝑣 = 𝛿 𝑢, 𝑣 + ℎ 𝑢 − ℎ(𝑣)

 Now we try to find ℎ : 𝑉 → ℛ such that 𝑤_ℎ 𝑢, 𝑣 ≥ 0 for all edges (𝑢, 𝑣) ∈ 𝐸

 𝑤_ℎ 𝑢, 𝑣 = 𝑤 𝑢, 𝑣 + ℎ 𝑢 − ℎ 𝑣 ≥ 0

 ℎ 𝑣 − ℎ 𝑢 ≤ 𝑤 𝑢, 𝑣

 The equations given by ℎ 𝑣 − ℎ 𝑢 , for all 𝑢, 𝑣 ∈ 𝑉 can form a difference constraints system!

Difference constraints

 The difference constraints system problem is to find a solution to a difference constraint system, where each equation has the following form:

𝑥_𝑖 − 𝑥_𝑗 ≤ 𝑐_𝑘, where 𝑐_𝑘 is a constant that can be negative.

 An example of a difference constraints system is as followed:

𝑥₁ − 𝑥₂ ≤ 3 𝑥₂ − 𝑥₃ ≤ −2 𝑥₁ − 𝑥₃ ≤ 2

Difference constraints(Cont.)

 Observe the fact that shortest paths have triangular inequality.

 Then for each equation 𝑥_𝑖 − 𝑥_𝑗 ≤ 𝑐_𝑘, we can construct an edge 𝑣_𝑗 → 𝑣_𝑖, and 𝑤(𝑣_𝑗 → 𝑣_𝑖) = 𝑐_𝑘

 Finally, add a new node s, and add for all 𝑣 ∈ 𝑉, add edges 𝑠 → 𝑣_𝑖, and 𝑤 𝑠 → 𝑣_𝑖 = 0

 For the example of last page, we can construct the graph as followed using the rule above.

Difference constraints(Cont.)

 𝑥₁ − 𝑥₂ ≤ 3

 𝑥₂−𝑥₃ ≤ −2

 𝑥₁ − 𝑥₃ ≤ 2

V¹ V²

V³ 2

3

-2 S

0

0 0

Difference constraints(End)

 After the graph is constructed, we can realize that for all variables 𝑥_𝑖 in the difference equations system, 𝑥_𝑖 = 𝛿 𝑠, 𝑣_𝑖 is a set of 𝑥 that can sastisfy the constraint due to triangular inequality if no negative cycle exists.

 So using Bellman-Ford algorithm, we can either tell that the difference constraints system is unsolvable, or find a set of solution in O(VE).

Return to Johnson Algorthim

 So for all 𝑢, 𝑣 ∈ 𝑉, ℎ 𝑣 − ℎ 𝑢 ≤ 𝑤 𝑢, 𝑣 , and these equations form a

difference constraints system, so Bellman-Ford algorithm can be used to detect negative cycles and determine the function ℎ. → 𝑶(𝑽𝑬)

 Then reweight all edges by 𝑤_ℎ 𝑢, 𝑣 = 𝑤 𝑢, 𝑣 + ℎ 𝑢 − ℎ 𝑣 . → 𝑶(𝑽𝑬)

 Then for all 𝑢 ∈ 𝑉, run Dijkstra’s algorithm on the rewiehted graph to find 𝛿_ℎ 𝑢, 𝑣 for all 𝑣 ∈ 𝑉. → 𝑶(𝑽^𝟐𝒍𝒈𝑽)

𝛿 𝑢, 𝑣 = 𝛿_ℎ 𝑢, 𝑣 + ℎ 𝑣 − ℎ(𝑢) for all 𝑢, 𝑣 ∈ 𝑉. → 𝑶(𝑽^𝟐)

 The total running time is 𝑶(𝑽𝑬 + 𝑽^𝟐𝒍𝒈𝑽).