Third Edition
6.5 Priority queues
Heapsort is an excellent algorithm, but a good implementation of quicksort, pre-sented in Chapter 7, usually beats it in practice. Nevertheless, the heap data struc-ture itself has many uses. In this section, we present one of the most popular ap-plications of a heap: as an efficient priority queue. As with heaps, priority queues come in two forms: max-priority queues and min-priority queues. We will focus here on how to implement priority queues, which are in turn based on max-heaps; Exercise 6.5-3 asks you to write the procedures for min-priority queues.
A priority queue is a data structure for maintaining a set S of elements, each with an associated value called a key. A max-priority queue supports the following operations:
INSERT.S; x/ inserts the element x into the set S , which is equivalent to the oper-ation S D S [fxg.
MAXIMUM.S / returns the element of S with the largest key.
EXTRACT-MAX.S / removes and returns the element of S with the largest key.
INCREASE-KEY.S; x; k/ increases the value of element x’s key to the new value k, which is assumed to be at least as large as x’s current key value.
Among their other applications, we can use max-priority queues to schedule jobs on a shared computer. The max-priority queue keeps track of the jobs to be performed and their relative priorities. When a job is finished or interrupted, the scheduler selects the highest-priority job from among those pending by calling EXTRACT-MAX. The scheduler can add a new job to the queue at any time by calling INSERT.
Alternatively, a min-priority queue supports the operations INSERT, MINIMUM, EXTRACT-MIN, and DECREASE-KEY. A min-priority queue can be used in an event-driven simulator. The items in the queue are events to be simulated, each with an associated time of occurrence that serves as its key. The events must be simulated in order of their time of occurrence, because the simulation of an event can cause other events to be simulated in the future. The simulation program calls EXTRACT-MINat each step to choose the next event to simulate. As new events are produced, the simulator inserts them into the min-priority queue by calling INSERT.
We shall see other uses for min-priority queues, highlighting the DECREASE-KEY
operation, in Chapters 23 and 24.
Not surprisingly, we can use a heap to implement a priority queue. In a given ap-plication, such as job scheduling or event-driven simulation, elements of a priority queue correspond to objects in the application. We often need to determine which application object corresponds to a given priority-queue element, and vice versa.
When we use a heap to implement a priority queue, therefore, we often need to store a handle to the corresponding application object in each heap element. The exact makeup of the handle (such as a pointer or an integer) depends on the ap-plication. Similarly, we need to store a handle to the corresponding heap element in each application object. Here, the handle would typically be an array index.
Because heap elements change locations within the array during heap operations, an actual implementation, upon relocating a heap element, would also have to up-date the array index in the corresponding application object. Because the details of accessing application objects depend heavily on the application and its imple-mentation, we shall not pursue them here, other than noting that in practice, these handles do need to be correctly maintained.
Now we discuss how to implement the operations of a max-priority queue. The procedure HEAP-MAXIMUMimplements the MAXIMUMoperation in ‚.1/ time.
HEAP-MAXIMUM.A/
1 returnAŒ1
The procedure HEAP-EXTRACT-MAX implements the EXTRACT-MAX opera-tion. It is similar to the for loop body (lines 3–5) of the HEAPSORTprocedure.
HEAP-EXTRACT-MAX.A/
1 ifA:heap-size < 1
2 error “heap underflow”
3 max D AŒ1
4 AŒ1 D AŒA:heap-size
5 A:heap-size D A:heap-size 1 6 MAX-HEAPIFY.A; 1/
7 return max
The running time of HEAP-EXTRACT-MAX is O.lg n/, since it performs only a constant amount of work on top of the O.lg n/ time for MAX-HEAPIFY.
The procedure HEAP-INCREASE-KEY implements the INCREASE-KEY opera-tion. An index i into the array identifies the priority-queue element whose key we wish to increase. The procedure first updates the key of element AŒi to its new value. Because increasing the key of AŒi might violate the max-heap property,
the procedure then, in a manner reminiscent of the insertion loop (lines 5–7) of INSERTION-SORT from Section 2.1, traverses a simple path from this node toward the root to find a proper place for the newly increased key. As HEAP-INCREASE -KEYtraverses this path, it repeatedly compares an element to its parent, exchang-ing their keys and continuexchang-ing if the element’s key is larger, and terminatexchang-ing if the el-ement’s key is smaller, since the max-heap property now holds. (See Exercise 6.5-5 for a precise loop invariant.)
HEAP-INCREASE-KEY.A; i; key/
1 if key< AŒi
2 error “new key is smaller than current key”
3 AŒi D key
4 whilei > 1 and AŒPARENT.i / < AŒi 5 exchange AŒi with AŒPARENT.i /
6 i D PARENT.i /
Figure 6.5 shows an example of a HEAP-INCREASE-KEY operation. The running time of HEAP-INCREASE-KEY on an n-element heap is O.lg n/, since the path traced from the node updated in line 3 to the root has length O.lg n/.
The procedure MAX-HEAP-INSERTimplements the INSERToperation. It takes as an input the key of the new element to be inserted into max-heap A. The proce-dure first expands the max-heap by adding to the tree a new leaf whose key is 1.
Then it calls HEAP-INCREASE-KEY to set the key of this new node to its correct value and maintain the max-heap property.
MAX-HEAP-INSERT.A; key/
1 A:heap-size D A:heap-size C 1 2 AŒA:heap-size D 1
3 HEAP-INCREASE-KEY.A; A:heap-size; key/
The running time of MAX-HEAP-INSERT on an n-element heap is O.lg n/.
In summary, a heap can support any priority-queue operation on a set of size n in O.lg n/ time.
Exercises
6.5-1
Illustrate the operation of HEAP-EXTRACT-MAX on the heap A D h15; 13; 9; 5;
12; 8; 7; 4; 0; 6; 2; 1i.
16
Figure 6.5 The operation of HEAP-INCREASE-KEY. (a) The max-heap of Figure 6.4(a) with a node whose index is i heavily shaded. (b) This node has its key increased to 15. (c) After one iteration of the while loop of lines 4–6, the node and its parent have exchanged keys, and the index i moves up to the parent. (d) The max-heap after one more iteration of the while loop. At this point, AŒPARENT.i / AŒi. The max-heap property now holds and the procedure terminates.
6.5-2
Illustrate the operation of MAX-HEAP-INSERT.A; 10/ on the heap A D h15; 13; 9;
5; 12; 8; 7; 4; 0; 6; 2; 1i.
6.5-3
Write pseudocode for the procedures HEAP-MINIMUM, HEAP-EXTRACT-MIN, HEAP-DECREASE-KEY, and MIN-HEAP-INSERT that implement a min-priority queue with a min-heap.
6.5-4
Why do we bother setting the key of the inserted node to 1 in line 2 of MAX -HEAP-INSERTwhen the next thing we do is increase its key to the desired value?
6.5-5
Argue the correctness of HEAP-INCREASE-KEY using the following loop invari-ant:
At the start of each iteration of the while loop of lines 4–6, the subarray AŒ1 : : A:heap-size satisfies the max-heap property, except that there may be one violation: AŒi may be larger than AŒPARENT.i /.
You may assume that the subarray AŒ1 : : A: heap-size satisfies the max-heap prop-erty at the time HEAP-INCREASE-KEYis called.
6.5-6
Each exchange operation on line 5 of HEAP-INCREASE-KEY typically requires three assignments. Show how to use the idea of the inner loop of INSERTION -SORTto reduce the three assignments down to just one assignment.
6.5-7
Show how to implement a first-in, first-out queue with a priority queue. Show how to implement a stack with a priority queue. (Queues and stacks are defined in Section 10.1.)
6.5-8
The operation HEAP-DELETE.A; i / deletes the item in node i from heap A. Give an implementation of HEAP-DELETE that runs in O.lg n/ time for an n-element max-heap.
6.5-9
Give an O.n lg k/-time algorithm to merge k sorted lists into one sorted list, where n is the total number of elements in all the input lists. (Hint: Use a min-heap for k-way merging.)
Problems
6-1 Building a heap using insertion
We can build a heap by repeatedly calling MAX-HEAP-INSERT to insert the ele-ments into the heap. Consider the following variation on the BUILD-MAX-HEAP
procedure:
BUILD-MAX-HEAP0.A/
1 A:heap-size D 1 2 fori D 2 to A:length
3 MAX-HEAP-INSERT.A; AŒi /
a. Do the procedures BUILD-MAX-HEAPand BUILD-MAX-HEAP0always create the same heap when run on the same input array? Prove that they do, or provide a counterexample.
b. Show that in the worst case, BUILD-MAX-HEAP0 requires ‚.n lg n/ time to build an n-element heap.
6-2 Analysis ofd-ary heaps
A d-ary heap is like a binary heap, but (with one possible exception) non-leaf nodes have d children instead of 2 children.
a. How would you represent a d -ary heap in an array?
b. What is the height of a d -ary heap of n elements in terms of n and d ?
c. Give an efficient implementation of EXTRACT-MAXin a d -ary max-heap. An-alyze its running time in terms of d and n.
d. Give an efficient implementation of INSERTin a d -ary max-heap. Analyze its running time in terms of d and n.
e. Give an efficient implementation of INCREASE-KEY.A; i; k/, which flags an error if k < AŒi , but otherwise sets AŒi D k and then updates the d -ary max-heap structure appropriately. Analyze its running time in terms of d and n.
6-3 Young tableaus
An m n Young tableau is an m n matrix such that the entries of each row are in sorted order from left to right and the entries of each column are in sorted order from top to bottom. Some of the entries of a Young tableau may be 1, which we treat as nonexistent elements. Thus, a Young tableau can be used to hold r mn finite numbers.
a. Draw a 4 4 Young tableau containing the elementsf9; 16; 3; 2; 4; 8; 5; 14; 12g.
b. Argue that an m n Young tableau Y is empty if Y Œ1; 1 D 1. Argue that Y is full (contains mn elements) if Y Œm; n < 1.
c. Give an algorithm to implement EXTRACT-MIN on a nonempty m n Young tableau that runs in O.m C n/ time. Your algorithm should use a recur-sive subroutine that solves an m n problem by recurrecur-sively solving either an .m 1/ n or an m .n 1/ subproblem. (Hint: Think about MAX -HEAPIFY.) Define T .p/, where p D m C n, to be the maximum running time of EXTRACT-MIN on any m n Young tableau. Give and solve a recurrence for T .p/ that yields the O.m C n/ time bound.
d. Show how to insert a new element into a nonfull m n Young tableau in O.m C n/ time.
e. Using no other sorting method as a subroutine, show how to use an n n Young tableau to sort n2numbers in O.n3/ time.
f. Give an O.m C n/-time algorithm to determine whether a given number is stored in a given m n Young tableau.
Chapter notes
The heapsort algorithm was invented by Williams [357], who also described how to implement a priority queue with a heap. The BUILD-MAX-HEAP procedure was suggested by Floyd [106].
We use min-heaps to implement min-priority queues in Chapters 16, 23, and 24.
We also give an implementation with improved time bounds for certain operations in Chapter 19 and, assuming that the keys are drawn from a bounded set of non-negative integers, Chapter 20.
If the data are b-bit integers, and the computer memory consists of addressable b-bit words, Fredman and Willard [115] showed how to implement MINIMUMin O.1/ time and INSERTand EXTRACT-MIN in O.p
lg n/ time. Thorup [337] has improved the O.p
lg n/ bound to O.lg lg n/ time. This bound uses an amount of space unbounded in n, but it can be implemented in linear space by using random-ized hashing.
An important special case of priority queues occurs when the sequence of EXTRACT-MIN operations is monotone, that is, the values returned by succes-sive EXTRACT-MINoperations are monotonically increasing over time. This case arises in several important applications, such as Dijkstra’s single-source shortest-paths algorithm, which we discuss in Chapter 24, and in discrete-event simula-tion. For Dijkstra’s algorithm it is particularly important that the DECREASE-KEY
operation be implemented efficiently. For the monotone case, if the data are in-tegers in the range 1; 2; : : : ; C , Ahuja, Mehlhorn, Orlin, and Tarjan [8] describe
how to implement EXTRACT-MIN and INSERT in O.lg C / amortized time (see Chapter 17 for more on amortized analysis) and DECREASE-KEY in O.1/ time, using a data structure called a radix heap. The O.lg C / bound can be improved to O.p
lg C / using Fibonacci heaps (see Chapter 19) in conjunction with radix heaps. Cherkassky, Goldberg, and Silverstein [65] further improved the bound to O.lg1=3CC / expected time by combining the multilevel bucketing structure of Denardo and Fox [85] with the heap of Thorup mentioned earlier. Raman [291]
further improved these results to obtain a bound of O.min.lg1=4CC; lg1=3Cn//, for any fixed > 0.
The quicksort algorithm has a worst-case running time of ‚.n2/ on an input array of n numbers. Despite this slow worst-case running time, quicksort is often the best practical choice for sorting because it is remarkably efficient on the average: its expected running time is ‚.n lg n/, and the constant factors hidden in the ‚.n lg n/
notation are quite small. It also has the advantage of sorting in place (see page 17), and it works well even in virtual-memory environments.
Section 7.1 describes the algorithm and an important subroutine used by quick-sort for partitioning. Because the behavior of quickquick-sort is complex, we start with an intuitive discussion of its performance in Section 7.2 and postpone its precise analysis to the end of the chapter. Section 7.3 presents a version of quicksort that uses random sampling. This algorithm has a good expected running time, and no particular input elicits its worst-case behavior. Section 7.4 analyzes the random-ized algorithm, showing that it runs in ‚.n2/ time in the worst case and, assuming distinct elements, in expected O.n lg n/ time.