Do We Teach the Right Algo rithm Design Techniques ?
Anany Levitin
ACM SIGCSE 1999
Outline
Introduction
Four General Design Techniques
A Test of Generality
Further Refinements
Conclusion
Introduction-1
According to many textbooks, a consensus seems to have evolved as to which approaches qualify as major techniques for designing algorithms.
This is includes: divide-and-conquer , greedy a
pproach , dynamic programming , backtracking , an
d branch-and-bound.
Introduction-2
However, this widely accepted taxonomy has ser ious shortcoming.
First, it includes techniques of different level s of generality. For example, it seems obvious t hat divide-and-conquer is more general than gree dy approach and branch-and-bound.
Second, it fails to distinguish divide-and-conqu er and decrease-and-conquer.
Third, it fails to include brute force and tran sform-and-conquer.
Introduction-3
Fourth, its linear, as opposed to hierarchical, structure fails to reflect important special cas es of techniques.
Finally, it fails to classify many classical alg orithms (e.g., Euclid’s algorithm, heapsort, se arch trees, hashing, etc.)
This paper seeks to rectify these shortcomings
and presents new taxonomy.
Four General Design Techniques-1
Brute Force
It usually based on the problem’s statement and definitions of concepts involved
This technique can not be overlooked by the follo wing reasons
Applicable to a very wide variety of problems, e.g., c omputing the sum of n numbers, adding two matrices ...
Useful for solving small-size instances of a problem
Serving an important theoretical or educational purpos e, e.g., NP-hard problem, as a yardstick for more effi cient alternatives for solving a problem
Four General Design Techniques-2
Divide-and-conquer
It based on partitioning a problem into a number o f smaller subproblems, usually of the same kind an d ideally of about the same size
The subproblems are then solved and their solution s combined to get a solution to the original probl em.
Divide-before-processing: the bulk of the work is done while combining solutions to smaller subproblems, e.g., mergesort.
Process-before-dividing: processing before a partition into subproblems, e.g., quicksort.
Four General Design Techniques-3
Decrease-and-conquer
This technique is solving a problem by reducing its instance to a smaller one, solving the latte r, and then extending the obtained solution to g et a solution to the original instance
decrease by a constant: insertion sort
decrease by a constant factor(a.k.a. prune-and-searc h): binary search
variable size decrease: Euclid’s algorithm
Four General Design Techniques-4
Transform-and-conquer
This technique is based on the idea of transform ation
simplification: solves a problem by first transformin g its instance to another instance of the same proble m which makes the problem easier to solve, e.g., Gaus sian elimination, heapsort ...
representation change: it is based on a transformatio n of a problem’s input to a different representation , e.g., hashing, heapsort …
Four General Design Techniques-4 cont.
preprossing: The idea is to process a part of the inp ut or the entire input to get some auxiliary informat ion which speeds up solving the problem, e.g., KMP al gorithms.
reduction: An instance of a problem is transformed to an instance of a different problem altogether, e.g, N P-hard problems.
A Test of Generality-1
We partition design techniques into two catego ries: more general and less general techniques .
How to make such a determinate ? We would like
to suggest the following test. In order to qua
lify for inclusion in the category of most gen
eral approaches, a technique must yield reason
able algorithms for the two problems: sorting
and searching.
A Test of Generality-2
Sorting Searching
Brute force Selection sort Sequential search Divide-and-conquer Mergesort Applicable
Decrease-and- conquer
Insertion sort Applicable Transform-and-
conquer Heapsort Search trees, hashing
The others - greedy approach, dynamic
programming, backtracking, and branch-and-bound - fail to qualify as the most general design techniques.
Further Refinements-1
Local search techniques
Greedy methods: it builds solutions piece by pie ce … the choice selected is that which produces the largest immediate gain while maintaining fea sibility, e.g., Prim’s algorithm.
Iterative methods: it start with any feasible so lution and proceed to improve upon the solution by repeated applications of a simple step, e.g., Ford-Fulkerson algorithm.
Further Refinements-2
Dynamic programming
Bottom-up: a table of solutions to subproblems i s filled starting with the problem’s smallest s ubproblems. A solution to the original instance of the problem is then obtained from the table c onstructed.
Top-down: memory function
Further Refinements-3
State-space-tree techniques
Backtracking: take coloring problem as an exampl e:
1 2
4 3
Use three colors to color the
vertices of this graph. How many different ways ?
Further Refinements-3 cont.
S
1 2 3
2
1 3
2
1 3
V1
V2
V3
Further Refinements-3 cont.
Branch-and-Bound: take TSP problem as an example . If there are four cities, all feasible solutio ns are
Further Refinements-3 cont.
Consider the traveling costs between any two cit ies:
=3+3+5+4+1
Further Refinements-3 cont.
Start to branch
Choose 12
Otherwise
Further Refinements-3 cont.
The current branch:
Further Refinements-3 cont.
The difference between them lies in that backt
racking is not limited to optimization problem
s, while branch-and-bound is not restricted to
a specific way of traversing the problem’s sp
ace tree.
Conclusion
This paper gives a new taxonomy of algorithm design tech niques.
No matter how many general design techniques are recogni zed, there will always be algorithms that cannot be natu rally interpreted as an application of those techniques.
Some algorithms can be interpreted as an application of different techniques, e.g., selection sort, as a brute-f orce algorithm and as a decrease-and-conquer method.
Some algorithms may incorporate ideas of several techniq ues, e.g. Fourier transform takes advantage of both the transform and divide-and-conquer ideas.
