**Do We Teach the Right Algo** **rithm Design Techniques ? **

### Anany Levitin

### ACM SIGCSE 1999

### Outline

### Introduction

### Four General Design Techniques

### A Test of Generality

### Further Refinements

### Conclusion

### Introduction-1

### According to many textbooks, a consensus seems to have evolved as to which approaches qualify as major techniques for designing algorithms.

### This is includes: divide-and-conquer , greedy a

### pproach , dynamic programming , backtracking , an

### d branch-and-bound.

### Introduction-2

### However, this widely accepted taxonomy has ser ious shortcoming.

First, it includes techniques of different level s of generality. For example, it seems obvious t hat divide-and-conquer is more general than gree dy approach and branch-and-bound.

Second, it fails to distinguish divide-and-conqu er and decrease-and-conquer.

Third, it fails to include brute force and tran sform-and-conquer.

### Introduction-3

Fourth, its linear, as opposed to hierarchical, structure fails to reflect important special cas es of techniques.

Finally, it fails to classify many classical alg orithms (e.g., Euclid’s algorithm, heapsort, se arch trees, hashing, etc.)

### This paper seeks to rectify these shortcomings

### and presents new taxonomy.

### Four General Design Techniques-1

### Brute Force

It usually based on the problem’s statement and definitions of concepts involved

This technique can not be overlooked by the follo wing reasons

Applicable to a very wide variety of problems, e.g., c omputing the sum of n numbers, adding two matrices ...

Useful for solving small-size instances of a problem

Serving an important theoretical or educational purpos e, e.g., NP-hard problem, as a yardstick for more effi cient alternatives for solving a problem

### Four General Design Techniques-2

### Divide-and-conquer

It based on partitioning a problem into a number o f smaller subproblems, usually of the same kind an d ideally of about the same size

The subproblems are then solved and their solution s combined to get a solution to the original probl em.

Divide-before-processing: the bulk of the work is done while combining solutions to smaller subproblems, e.g., mergesort.

Process-before-dividing: processing before a partition into subproblems, e.g., quicksort.

### Four General Design Techniques-3

### Decrease-and-conquer

This technique is solving a problem by reducing its instance to a smaller one, solving the latte r, and then extending the obtained solution to g et a solution to the original instance

decrease by a constant: insertion sort

decrease by a constant factor(a.k.a. prune-and-searc h): binary search

variable size decrease: Euclid’s algorithm

### Four General Design Techniques-4

### Transform-and-conquer

This technique is based on the idea of transform ation

simplification: solves a problem by first transformin g its instance to another instance of the same proble m which makes the problem easier to solve, e.g., Gaus sian elimination, heapsort ...

representation change: it is based on a transformatio n of a problem’s input to a different representation , e.g., hashing, heapsort …

### Four General Design Techniques-4 cont.

preprossing: The idea is to process a part of the inp ut or the entire input to get some auxiliary informat ion which speeds up solving the problem, e.g., KMP al gorithms.

reduction: An instance of a problem is transformed to an instance of a different problem altogether, e.g, N P-hard problems.

### A Test of Generality-1

### We partition design techniques into two catego ries: more general and less general techniques .

### How to make such a determinate ? We would like

### to suggest the following test. In order to qua

### lify for inclusion in the category of most gen

### eral approaches, a technique must yield reason

### able algorithms for the two problems: sorting

### and searching.

### A Test of Generality-2

**Sorting ** **Searching**

Brute force Selection sort Sequential search Divide-and-conquer Mergesort Applicable

Decrease-and- conquer

Insertion sort Applicable Transform-and-

conquer Heapsort Search trees, hashing

*The others - greedy approach, dynamic *

*programming, backtracking, and branch-and-bound - *
fail to qualify as the most general design techniques.

### Further Refinements-1

### Local search techniques

Greedy methods: it builds solutions piece by pie ce … the choice selected is that which produces the largest immediate gain while maintaining fea sibility, e.g., Prim’s algorithm.

Iterative methods: it start with any feasible so lution and proceed to improve upon the solution by repeated applications of a simple step, e.g., Ford-Fulkerson algorithm.

### Further Refinements-2

### Dynamic programming

Bottom-up: a table of solutions to subproblems i s filled starting with the problem’s smallest s ubproblems. A solution to the original instance of the problem is then obtained from the table c onstructed.

Top-down: memory function

### Further Refinements-3

### State-space-tree techniques

Backtracking: take coloring problem as an exampl e:

1 2

4 3

Use three colors to color the

vertices of this graph. How many different ways ?

### Further Refinements-3 cont.

S

1 2 3

2

1 3

2

1 3

V_{1}

V_{2}

V_{3}

### Further Refinements-3 cont.

Branch-and-Bound: take TSP problem as an example . If there are four cities, all feasible solutio ns are

### Further Refinements-3 cont.

Consider the traveling costs between any two cit ies:

=3+3+5+4+1

### Further Refinements-3 cont.

Start to branch

Choose 12

### Otherwise

### Further Refinements-3 cont.

The current branch:

### Further Refinements-3 cont.

### The difference between them lies in that backt

### racking is not limited to optimization problem

### s, while branch-and-bound is not restricted to

### a specific way of traversing the problem’s sp

### ace tree.

### Conclusion

This paper gives a new taxonomy of algorithm design tech niques.

No matter how many general design techniques are recogni zed, there will always be algorithms that cannot be natu rally interpreted as an application of those techniques.

Some algorithms can be interpreted as an application of different techniques, e.g., selection sort, as a brute-f orce algorithm and as a decrease-and-conquer method.

Some algorithms may incorporate ideas of several techniq ues, e.g. Fourier transform takes advantage of both the transform and divide-and-conquer ideas.