RESEARCHCONlRlWlIONS

Algorithms and Data Structures Pairing Heaps: G. Scott Graham Editor Experiments and Analysis

JOHN T. STASKO and JEFFREY SCOTT VlllER

ABSTRACT: The pairing has recently been and practical importance from their use in solving a introduced as a new for priority queues. wide range of combinatorial problems, including job Pairing heaps are extremely simple to implement and scheduling, minimal spanning , shortest path, seem to be very efficient in practice, but they are difficult and graph traversal. to analyze theoretically, and open problems remain. It Priority queues support the operations insert, has been conjectured that they achieve the same find-min, and delete-min; additional operations often amortized time bounds as Fibonacci heaps, namely, include decrease-key and delete. The insert(t, v) opera- O(log n) time for delete and delete-min and O(1) for tion adds item t with key value v to the priority all other operations, where n is the size of the . The find-min operation returns the item queue at the time of the operation. We provide empirical with minimum key value. The delete-min operation evidence that supports this conjecture. The most returns the item with minimum key value and promising algorithm in our simulations is a new variant removes it from the priority queue. The decrease- of the twopass method, called auxiliary twopass. We key(t, d) operation reduces item t’s key value by d. prove that, assuming no decrease-key operations are The delete(t) operation removes item t from the performed, it achieves the same amortized time bounds as priority queue. The decrease-key and delete opera- Fibonacci heaps. tions require that a pointer to the location in the priority queue of item t be supplied explicitly, since 1. INTRODUCTION priority queues do not support searching for arbi- A priority queue is an abstract data type for main- trary items by value. Some priority queues also sup- taining and manipulating a set of items based on port the merge operation, which combines two item- priority [I]. Prio’rity queues derive great theoretical disjoint priority queues. We will concentrate on the insert, delete-min, and Support was provided in part by NSF research grant DCR-84-03613, an NSF Presidential Young Investigator Award, an IBM Faculty Development Award, decrease-key operations because they are the opera- and a Guggenheim Fellowship. tions that primarily distinguish priority queues from Part of this research was performed at Mathematical Sciences Research Insti- tute. Berkeley, Calif., and the Institut National de Recherche en Informatique other set manipulation algorithms and because they et en Automatique, Rocquencourt. France. are the critical operations as far as the time bounds 0 1987 ACM OOOl-0782/87/0300-0234 75a: are concerned.

Communications of the ACM March 1987 Volume 30 Number 3 Research Contributions

Several implementations of priority queues, such quences of commands to make the pairing heaps as implicit heaps [lo], leftist heaps [3, 71, and bino- perform as poorly as we could. The results were mial heaps [2, 91 have been shown to exhibit an positive in that the pairing heaps always performed O(log n) worst-case time bound for all operations, extremely well. This does not prove that the desired where n is the size of the priority queue at the time time bounds do hold, but it is reassuring and makes of the operation. Fibonacci heaps [4] provide a dra- us optimistic that the conjecture is true. matic improvement on the general logarthmic bound In this article we study the “twopass” and “multi- by achieving amortized time bounds of O(1) for pass” versions of pairing heaps; the names arise from insert, decrease-key, and find-min and O(log n) for the method used to do the delete-min in each version delete-min and delete. This greatly improves the [5]. We also introduce new variants called “auxiliary best known theoretical bounds for the time required twopass” and “auxiliary multipass.” All versions are to solve several combinatorial problems.* Follow- described in the next section. In Section 3, we dis- ing the approach of [8], a sequence of operations cuss our simulations and the empirical data. Auxil-

op,, opz,. . ., opk is said to have amortized time bounds iary twopass performed best in the simulations, b,, bz, . . . , bk if based on our measure of performance. In Section 4, we provide a partial theoretical analysis of pairing Esj ti 5 C bit for all 1 5 j I k, lsisj heaps by introducing the concept of “batched poten- tial.” We show, for example, that auxiliary twopass where ti is the actual time used by opi. Intuitively, if uses O(1) amortized time per insert and find-min and operation opi uses less time than its allotted bi units, O(log n) amortized time for the other operations. then the leftover time may be held in reserve to be Conjectures and open problems follow in Section 5. used by later operations. Fibonacci heaps achieve their time bounds by 2. PAIRING HEAP ALGORITHMS complicated invariants with significant overhead, so A comprehensive description of pairing heaps ap- they are not the method of choice in practice. Re- pears in [5]. A summary is given below. Our studies cently, a self-adjusting data structure called the pair- involve the twopass algorithm, which was the sub- ing heap was proposed [5]. Pairing heaps are much ject of most of the analysis in [5], and the multipass simpler than Fibonacci heaps, both conceptually and algorithm. in implementation; and they have less overhead per Pairing heaps are represented by heap-ordered operation. The best amortized bound proved so far trees and forests. The key value of each node in the for pairing heaps is O(log n) time per operation. It heap is less than or equal to those of its children. is conjectured that pairing heaps achieve the same Consequently, the node with minimum value (for amortized time bounds as Fibonacci heaps, namely, simplicity, we will stop referring to a key value, and O(1) per operation except O(log n) for delete-min and just associate the value directly with the heap node) delete. is the root of its tree. Groups of siblings, such as tree To test whether the conjecture is true, we per- roots in a forest, have no intrinsic ordering. formed several simulations of the pairing heap algo- In the general sense, pairing heaps are represented rithms. These simulations differed significantly from by multiway trees with no restriction on the number the ones independently done in [6], since the latter of children that a node may have. Because this mul- ones did not address the conjecture. In our simula- tiple child representation is difficult to implement tions, we tested several different pairing heaps and directly, the child-sibling representation used “greedy” heuristics and the appropriate se- of a multiway tree is used, as illustrated in Figure 1 (p. 236). In this representation, the left pointer of a ‘For example, a standard implementation of Dijkstra’s algorithm (which finds node accesses its first child, and the right pointer of the shortest path from a specified vertex x to all other vertices in a graph with nonnegative edge lengths) uses a priority queue as follows: Let us denote the a node accesses its next sibling. In terms of the bi- number of vertices in the graph by V and the number of edges by E. The key value of each item y in the priority queue represents the length of the shortest nary tree representation, it then follows that the path from vertex x to vertex y using only the edges in the graph already value of a node is less than or equal to all the values processed. Initially, no edges are processed, and the priority queue contains V items: the key value of item x is 0 and all other items have key value m. The of nodes in its left subtree. A third pointer, to the algorithm successively performs delete-mins until the priority queue is empty. previous sibling, is also included in each node in Each time a delehe-min is performed (say, the vertex y is deleted), the algo- rithm outputs the shortest path between x and y, and each unprocessed edge order to facilitate the decrease-key and delete opera- (y, z) incident toy in the graph is processed; this may require that a decrease- key be performed in order to lower the key value of z in the priority queue. tions. The number of pointers can be reduced from Thus. there are at most V inserfs, V delete-mins, and E decrense-keys during the three to two, as explained in [5] at the expense of a ccwrse of the algorithm. If a is used to implemenithe priority V V1. constant factor increase in running time. Unless aueue. the resultine- runnineI time is OfE + lee- ,. which is a sienificant improvement over O((E + V) log V) using the other heap representations. stated otherwise, the terms “child,” “parent,” and Other examples of how Fibonacci heaps can improve worst-case running times are given in 141. “subtree” will be used in the multiway tree sense;

March 1987 Volume 30 Number 3 Communications of the ACM 235 Research Contributions

tree representation, the root node always has a null right pointer. The insert(t, V) operation performs a comparison-link between t and the tree root; the node with smaller value becomes the root of the resulting tree. The decrease-key(t, d) operation begins by reducing t’s value by d. This means that t may now have a value smaller than its parent. Conse- quently, it must be removed from the tree (with its own subtree intact) and comparison-linked with the tree root. Again, the node with smaller value be- comes the root of the resulting tree. The delete-min operation gives the twopass algo- rithm its name. First, the tree root node is deleted and its value returned. This leaves a forest of former children and their subtrees. Next, two comparison- linking passes are made over the roots of this forest. Pass 1 is a left-to-right pass, in which a comparison- link is performed on successive pairs of root nodes. Pass z then proceeds from right-to-left, In each step, the two rightmost trees are replaced by the tree re- sulting from a comparison-link; the “cumulative” rightmost tree is continually updated in this manner until it is the only remaining tree. The root of this final tree is the minimum of all the nodes in the tree. Figure 3 illustrates the delete-min procedure in terms of the binary tree representation, The delete(t) operation works as follows: If the node t to be deleted is the root of the main tree, then a delete-min operation is performed. Otherwise, t is deleted from the tree. The former subtrees oft are 03 recombined into a single tree via the twopass linking procedure. This tree is then comparison-linked to the root of the main tree. FIGURE1. An Example of a Heap Ordered Tree (a) Multiway tree The multipass algorithm that we studied was also heap representation. (b) Corresponding binary tree representation. presented in [5]. Both the insert(t, d) and decrease- key(t, d) operations function exactly as those in the twopass algorithm. The delete-min operation, how- their corresponding meaning in the binary tree rep- ever, distinguishes multipass from twopass. The first resentation should be clear. operation in the multipass delete-min is the deletion The primary action performed in pairing heap op- of the root node; its value is returned. This leaves erations is a comparison-link, in which the values of the heap with some number of trees, say r. Next, we two nodes are compared. The node with larger value repeatedly perform pairwise linking passes on the is demoted in the sense that it becomes the first roots of these trees until the heap is left with only one child of the smaller-valued node. The previous first tree. Each comparison-link reduces the number of child of the smaller node becomes the second child, trees by one, and each pass cuts the number of trees the previous second child becomes the third child, roughly in half. For r trees, a total of Ilog rl' passes and so on. Ties can be brolken arbitrarily. The binary are made. A simpler heuristic is to comparison-link tree representation of the comparison-link is given the first two tree roots and place the “winning” root in Figure 2. This comparison-link action is per- (smaller value) at the tail of the forest list. Alterna- formed repeatedly during the delete-min operation of tively, a circular list could be used to store the sib- a priority queue. It is the primary action that we lings. Both ways, a round robin effect emerges, and seek to minimize to reduce execution times. we see that for r tree roots, exactly r - 1 link opera- The twopass algorithm that we examined was the tions are performed. Following the linking phase, variant that yielded the O(log n)-time amoritized the heap is again left with a single tree; its root is the bounds for inser,t, decrease-key, and delete-min in [5]. Only one tree is maintained. Hence, in the binary ‘All logarithms in this article are base 2.

236 Communications of the ACM March 1987 Volume 30 Number 3 Research Contributions

FIGURE2. Two Binary Tree Heap Configurations and the Resulting Structures From a Comparison-Link Between Nodes X and Y

(a)

(4

FIGURE3. Twopass Delete Min Procedure, Using the Binary Tree Representation. (a) Beginning heap configuration. (b) Root (W (minimum) deletion. (c) Pass one. (d) Pass two.

March 1987 Volume 30 Number 3 Communications of the ACM 237 Research Contributions

node with minimum key value. Figure 4 illustrates and auxilia y multipass. Auxiliary twopass works as the multipass delete-min in terms of the binary tree follows: In addition to the main tree in the heap, we representation. The delete(t) operation is the same as maintain an auxiliary area that consists of an or- in twopass except that multipass comparison-linking dered list of other trees. It is convenient in the im- is used on the children of the deleted node t. plementation to store the auxiliary area as the right While working with these algorithms, we designed subtree (in the binary tree sense) of the root. two new variations, that we call auxiliary twopass The insert(t, V) and decrease-key(t, d) operations

(W

(4

FIGURE4. Multipass Delete Min Procedure, Using the Binary Tree Representation. (a) Beginning heap configuration. (b) Root (minimum) deletion. (c) Pass one. (d) Pass two. (e) Pass three.

238 Communications of ,the ACM March 1987 Volume 30 Number 3 Research Contributions

function as in the regular twopass algorithm except must be maintained; each insert or decrease-key node for one major difference. Rather than comparison- must be checked against the minimum pointer so linking node t with the tree root, the node is added that the pointer can be updated if necessary. to the end of the list of auxiliary trees. (As usual, in The delete-min operation, which is illustrated in the case of a decrease-key operation, the subtree Figure 5, begins by “batching” the auxiliary area, rooted at t remains intact.) If the find-min operation that is, by running the multipass pairing procedure is to be implemented, a separate minimum pointer on the auxiliary area. (Note that although the

A insert

main tree auxiliary forest

(4

(4

(e)

FIGURE5. Auxiliary Twopass Delete Min Procedure, Using the Binary Tree Representation. (a) Initial heap configuration. (b) Multipass on auxiliary forest. (c) Link auxiliary root to main root. (d) Root (minimum) deletion. (e) Twopass back to one root.

March 1987 Volume 30 Number 3 Communications of the ACM 239 ResearchContributions

method described is called auxiliary twopass, the should run faster. Although all make roughly the auxiliary area is linked together using the multipass same number of comparison-link actions on a simi- method.) When this linking is complete, the auxil- lar heap configuration, the multipass variants tend iary area consists of a single tree. If the auxiliary to build a more “structured” forest configuration area originally consists of 2k singleton trees, for some over time. All the uppermost nodes that are directly k 2 0, the resulti.ng tree is a binomial tree [2, 91. The involved in link actions will be formed into a binomia l- next action is a comparison-link between the main like tree, which helps limit the number of links tree root and the new auxiliary root. After this during subsequent delete-min operations. The sim- comparison-link, the heap again contains only one ulation results described in the next section are root node, that of minimurn key value. From this somewhat surprising; auxiliary twopass consistently point on, the delete-min opleration proceeds exactly outperforms the multipass versions. as in the twopass algorithm. The root node is re- moved, and we link the remaining forest of trees via 3. SIMULATIONS the twopass proc.edure. Our test simulations of the pairing heap algorithms The rationale :for maintaining the auxiliary area is consisted of structured sets of insert, decrease-key, that it prevents many comparisons between single and delete-min operations. No key values were ever nodes from an insert and large trees already in the assigned to nodes. Instead, we used a “greedy” heu- forest. We shall prove in Section 4 that if there are ristic to determine the winners of comparisons, in no decrease-key operations, auxiliary twopass hopes of causing a worst-case scenario. Every time a achieves the amortized time bounds of O(1) for insert comparison-link operation was performed, the node and O(log n) for

240 Communications of the ACM March 1987 Volume30 Number 3 Research Contributions

(4) one insert, then x(log n) - 1 greedy decrease-keys, operation can be done in O(log n) amortized time. followed by one delete-min, for x = 0.25, with an Therefore, it was impossible in Jones’s [6] simula- initial binomial tree of size 2”. tions to distinguish between O(1) time and O(log n) (5) same for x = 1.0. time per insert. Our simulations, on the other hand, (6) same for x = 4.0. tested the time bound conjectured more effectively (7) one insert then x(log n) - 1 greedy decrease-keys, by performing O(log n) insert and decrease-key opera- followed by one delete-min, for x = 0.22, with an tions for each delete-min. initial binomial tree of size z’~. Simulation 1 allowed us to examine how the (8) same for x = 1.0. heaps performed when no decrease-key operations (9) same for x = 4.0. were used. Work ratios were calculated over incre- ments of 12,000 nodes of heap growth, with the heap Our measure of performance compared the actual size eventually reaching 1,200,OOOnodes. Four ini- work done by each algorithm with an allowance for tial inserts were used to “start-up” the simulation. the operations processed. The actuaI work done was The results are graphed in Figure 6 (p. 242). Only considered to be one unit for an insert, one unit for multipass showed a steady increase in work ratio; a decrease-key, and one unit for the delete plus one however, there was a marked decrease near the sim- unit for each comparison-link that occurred during ulation’s end. Both twopass and auxiliary twopass a delete-min. Allowances for the operations corre- remained mostly steady, and auxiliary multipass ex- sponded to the amortized time bounds sought for hibited a clear decrease. Auxiliary twopass was the them: the insert and decrease-key allowances were fastest algorithm, a fact that would continue through each one unit, and the delete-min allowance was most of the following simulations. It is interesting to log n, where n was the heap size at the time of the note that the auxiliary algorithms were not subject delete-min. to wide fluctuations in work ratio as were the regu- Phases were grouped into a smaller number of lar algorithms. increments to facilitate graphical display of the re- Simulation 2 utilized random decrease-key opera- sults; in the first three simulations the size of the tions primarily to see how random disruptions in the heap grew by a fixed amount in each increment, and heap structure would affect overall algorithm perfor- in Simulations 4-9 (Figures 9-14.) increments con- mances. Again work ratios were calculated in 12,000 sisted of a fixed number of phases. For each incre- node increments, and the total heap size grew to ment in a simulation, we calculated its work ratio. ~,ZOO,OOOnodes. Sixteen inserts were used to initial- The work ratio is defined as actual work performed ize the simulation, This simulation’s results, which divided by the operations’ allowances. By seeing are shown in Figure 7 (p. 242), were quite similar to how the work ratio changed over time across these those of Simulation 1. No steady work ratio in- increments, we were able to judge the performances creases were evident, nor was there an appreciable of the algorithms. If the work ratio increased over gain in work ratio values from Simulation 1. Both time, the O(l)-time insert and decrease-key allow- multipass algorithms exhibited work ratio decreases, ances and the O(log n)-time delete-min allowance with regular multipass remaining slightly superior. would not be bounding the actual work growth. If Auxiliary twopass was again the fastest algorithm, instead the work ratio stayed constant or decreased, and twopass was the slowest. Curiously, the total then the experiments would provide encouragement work ratio for auxiliary twopass over the entire that the sought for amortized time bounds are simulation was slightly less than its total in Simu- possible. lation 1. In essence, the random decrease-keys helped In our simulations, we chose the particular order the algorithm run faster. and frequency of operations so that if any of the Simulation 3 utilized greedy decrease-key opera- conjectured amortized time bounds of O(1) for insert tions in which the node with the most children was and decrease-key and O(log n) for delete-min did not chosen for the operation. Nodes such as the root and hold, we would detect a discrepancy in the results. children of the root, whose choice would have no The simulations of pairing heaps in [6], on the other effect on the heap structure, were excluded from hand, were limited for several reasons: First, no being candidates. This simulation’s decrease-key op- decrease-key operations were performed. We will see eration was intended to move nodes with many chil- in Section 4 that if no decrease-keys are done, we can dren from the central heap up to the top root level, prove the conjectured bounds analytically for auxil- thereby forcing the delete-min operations to do even iary twopass. More importantly, however, the simu- more work. Because of the extra storage required, lations performed the same number of inserts as work ratios were calculated in increments of 6,000 delete-mins. It is already known from [5] that each nodes. The final heap size was 546,000 nodes. A

March 1987 Volume 30 Number 3 Communications of the ACM 241 Research Contributions

SIMULATION 1

2.3

twopass 2.254 2.2

2.1 work multipass ratio 2.122 2.0

1.9

1 .a 30 40 50 60 70 80 90 100 heap size (in 12,000 nodes)

FIGURE6. log n inserts, 1 delete min per phase

SIMULATION 2

2.27

2.17

2.12

work 2.07 ratio 2.02

1.97

1.92

1 .a7 -+---im-u- w-- ~~~4yP= 1.82 8 0 10 20 30 40 50 60 70 80 90 100 heap size (in 12,000 nodes)

FIGURE7. 0.5 log 12(insert, random decrease key) pairs, 1 delete min per phase

242 Communications of thle ACM March 1987 Volume 30 Number 3 Research Contributions start-up set of sixteen insert operations was used. 100 increments. The respective results are shown in After showing small jumps in the work ratios, all Figures 9-11 [pp. 244-245). All three tests showed four algorithms maintained steady levels at the sim- very steady work ratio rates, which is encouraging. ulation’s end. The results are graphed in Figure 8. The actual values were quite different, however. Clearly, the greedy decrease-key operations did have The twopass variants had lower work ratios when an effect, as work ratio values were higher than there were a smaller number of decrease-key opera- those in the first two simulations. Auxiliary twopass tions per phase, whereas the multipass variants ex- was again the fastest algorithm, twopass was the hibited an opposite behavior; they performed better slowest, and the multipass algorithms were quite as the number of decrease-keys per phase increased. similar. The general multipass algorithm had a defi- Auxiliary twopass was overall the best algorithm, nite superiority at smaller heap sizes, however. but regular twopass exhibited a curious variation on Simulations 4-9 primarily examined how the its usual slowest behavior. In the two simulations decrease-key operation affected algorithm perfor- with more decrease-key operations, twopass was mances. An initial binomial tree of some size was clearly slowest. In fact, in Simulation 6 with built. This provided all four algorithms with the 48 decrease-key operations per phase, twopass was same starting point, so no initial bias was intro- blatantly behind the other three algorithms perfor- duced. Each phase contained only one insert opera- mances. But in Simulation 4 with the fewest (three) tion; hence, a constant heap size was maintained, decrease-keys per phase, twopass was the fastest algo- that of the initial binomial tree. By varying the num- rithm! It appears that twopass performs best when ber of decrease-key operations between the insert and there are relatively few insert and decrease-key the delete-min in a phase, we could see exactly how operations compared to the number of delete-min this number affected the work ratio. operations. In Simulations 4-6, we used an initial binomial In Simulations 7-9, we used a much larger initial tree size of n = 2” = 4096 nodes. Simulations 4, 5, binomial tree size of n = 2” = 262,144 nodes. Simu- and 6 used 0.25 log n = 3, log n = 12, and 4 log n = lations 7-9 used 0.22 log n = 4, log n = 18, and 4 log 48 decrease-key operations per phase, respectively. n = 72 decrease-key operations per phase, respec- We grouped 100 phases into an increment for work tively. We grouped 1000 phases into an increment ratio calculations, then we ran the simulation for for work ratio calculations, then we ran the simula-

SIMULATION 3

2.55

2.50

2.45

2.40

2.35 work ratio 2.30

2.25

2.15

I 30 40 50 60 70 80 90 heap size (in 6,000 nodes)

FIGURE8. 0.5 log n (insert, greedy decrease key) pairs, 1 delete min per phase

March 1987 Volume 30 Number 3 Communications of the ACM 243 Research Contributions

SIMULATION 4

.. . . .:, : : ..,...,.~...,,.. ../,..Y. ....;: .::“’ I,. ,.., ?j,;.+; :, t .Y.._ .5...... I i ..,.....i .. .j-; ‘.., ..’ ..,.,,.....: .,/.. , _I ..<. . . i .,,.i aux multipass ,~,...... ‘. 2.173

ratio 1.8

twopass 1.776

1.6

heap size = 4096 1.5 ,rrrr,rrlrl,r,,,,,l,.,,,,,,,,,,,,,,,,,,,,,,~,,,,,.,,,,,~,,,,~~,.,,,,,,~~~~.~~,~.~~~~~.,,,,.,,,, 0 10 20 30 40 50 60 70 80 90 100 phases (in hundreds)

FIGUHE9. 1 insert, (0.25 log n) - 1 greedy decrease keys, 1 delete min par phase

SIMULATION 5

2.10

2.05

2.032

work ratio

1.85

1.80

IO 20 30 40 50 60 70 80 90 100 phases (in hundreds)

FIGURE10. 1 insert, (log n) - 1 greedy decrease keys, 1 delete min per phase

244 Communications of the ACM March 1987 Volume 30 Number 3 ResearchContributions

SIMULATION 6

2.6 hvopass J 2.568

2.5

2.4

2.3

aux multipass 1.986 2.0 = ,,,,i .,.....,,...... ,.’ ,..I,... i l<2 ,./ aux twopass -.--ye...- --~~“+&“.“- 1.9 17 1.9 multipass -k- -= heap size = 4096 i ,908 1.8 m,,,,,~~~~~~~~~~,~~,,,,,,-,,,,,,,ll~ll,l,,,llrrrIllrlrrlll,,,,,,,,,,,,,1,,1111IIIIII~- 0 10 20 30 40 50 60 70 a0 90 100 phases (in hundreds)

FIGURE11. 1 insert, (4.0 log n) - 1 greedy decrease keys, 1 delete min per phase

tion for 100 increments. The results are shown in Both results were consistent with those above; work Figures 12-14 (pp. 246-247). The relative perfor- ratio values stayed steady or showed a small de- mances of the four algorithms were quite similar to crease, and auxiliary twopass again exhibited the those of Simulations 4-6. The curious behavior of lowest overall work ratios. In the random sequence the twopass algorithm was again evident, as it per- simulations, however, twopass and auxiliary two- formed poorly with many decrease-key operations per pass performed almost identically. phase, but improved dramatically with few. With The data from these simulations allowed us to the larger initial heap size, however, it never over- make the following conclusions: First, the O(l)-time took auxiliary twopass as the fastest algorithm, as it bounds for insert and decrease-key and the O(log n)- did in Simulation 4. time bound for delete-min appear to hold in the am- In the last six simulations with constant heap size, ortized sense. Our data provided no evidence to the the work ratio during the formation of the initial contrary. In fact, they provide some clue as to the binomial tree was just under 2.0; we performed n actual coefficients implicit in the big-oh terms. Let insert operations, followed by n - 1 comparison- us make the simplifying assumption that the amor- links. Therefore, actual work was 2n - 1 units, and tized running time for each insert and decrease-key the insert allowance was n units, giving a work ratio operation in the simulations is c time units and that of 2 - l/n. This amount was included in the total the amortized time per delete-min is d log n units. work ratio for the simulation. Solving a set of linear equations obtained from Sim- Aside from Simulations 1-9, we also performed ulations 4-9 gives c = 2.9 and d = 1.5 for twopass, two randomized simulations to verify that our data c = 1.8 and d = 2.2 for multipass, c = 2.0 and d = 1.7 were not dependent on the fixed structure of each for auxiliary twopass, and c = 1.9 and d = 2.3 for phase. In the first, we kept the heap size constant as auxiliary multipass. Second, the auxiliary twopass in Simulations 4-9, but the number of decrease-keys algorithm was clearly the best overall. It typically per phase was uniformly distributed between 0 and exhibited lower work ratios than the other three 2 log n. The second began by performing 64 initial algorithms. Third, adding the auxiliary area to inserts which were followed by a random sequence multipass caused no great improvements to the algo- of insert, greedy decrease-key, and delete-min opera- rithm. In our tests, the multipass algorithm was al- tions, all having the same probability of occurrence. most always superior to its auxiliary variant. Finally,

March 1987 Volume 30 Number 3 Communications of the ACM 24s Research Contributions

SIMULATION 7

..... aux multipass .I,__.wi..... 2.272 2.3 - ;s Y” multipass 2.147

ratio ,,9

hvopass - 1.759

1.702

1.6

heap size = 262,144

40 50 60 70 80 90 100 phases (in thousands)

FIGURE12. 1 insert, (0.22 log n) - 1 greedy decrease keys, 1 delete min per phase

SIMULATION 8

twopass 2.16 , 2.114

e--+b-+-..,-rle, multipass 2.004

------__c_* --- I___L_^ aux1.795 twopass

heap size = 262,144 1.66 Tl#i..i III,,,, rrlrrrrlrrl,,,,,,,,,, Mrlrri.,, -,,,,,,I-rn-l 0 10 20 30 40 50 60 70 80 90 100 phases (in thousands)

FIGURE13. 1 insert, (log n) - 1 greedy decrease keys, 1 delete min per phase

246 Communications of the ACM March 1987 Volume 30 Number 3 Research Contributions the regular twopass algorithm’s performance was time of the operation plus the change in potential, quite variable. It was often the worst, especially namely, ti + a(i) - @(i - I), where ti is the actual when many insert and decrease-key operations were time of the ith operation, @a(i)is the potential after processed. As the number of these operations de- the ith operation, and @(i - 1) is the potential before clined, however, its performance improved to rival the ith operation. If we start with potential 0 and that of auxiliary twopass. end up with positive potential, then the total run- ning time is bounded by the total amortized time, 4. BATCHED POTENTIAL via the telescoping effect of the potential changes. The best known,amortized time bounds of the insert, THEOREM 1. decrease-key, and delete-min operations for the two- The auxilia y twopass pairing heap algorithm achieves pass pairing heap are all O(log n), due to [?I]. To amortized time bounds of O(1) for insert and find-min equal the time bounds for Fibonacci heaps, the insert and O(log n) for delete-min and delete if no decrease-key and decrease-key time bounds must be shown to be operations are allowed. O(1). In an effort to prove the constant time bounds for pairing heaps, we will use the auxiliary twopass PROOF. algorithm and introduce batched potential. But before Without loss of generality, we can restrict ourselves that, let us briefly review the concept of potential as to insert and delete-min operations exclusively. Let it applies to amortized algorithmic analysis. us define the rank of a node to be the binary loga- The potential technique for amortized analysis is rithm of the number of nodes in the subtree rooted discussed in [a]. Each configuration of the pairing there (in the binary tree sense). We use a variant of heap is assigned some real value a, known as the the potential function used to analyze twopass in [5]. “potential” of that configuration. For example, one We define the potential + of a heap configuration to could define the potential of a pairing heap configu- be the sum of the ranks of all nodes in the main tree ration to be the number of trees it contains. For any (that is, not counting the auxiliary area) plus 5 times sequence of n operations, the amortized time of the the number of roots in the auxiliary area. ith operation is defined to be the actual running Each insert places a new node into the auxiliary

SIMULATION 9

twopass 2.593 2.6

2.5

2.4

aux multipass 2.016 multipass 1.976 -dh aux hvopass heap size = 262,144 1.914 ,,.,.,,,,,,,,,,,,,I,....l~rlllrII.~~II.IIII,,,-,,.,,,~ 20 30 40 50 60 70 80 90 100 phases (in thousands)

FIGURE14. 1 insert, (4.0 log n) - 1 greedy decrease keys, 1 delete min per phase

March 1987 Volume 30 Number 3 Communications of the ACM 247 area and increases the potential by 5; its amortized one unit of work, reducing the potential by log n, time is thus 6. There is no large single change in and the resulting subtrees are recombined via the potential until a delete-min is performed, when the twopass scheme. If there are k subtrees, a total of nodes in the auxiliary area are added to the main k - 1 units of work are used to recombine; the re- tree. We refer to this as “batched potential” because sulting potential increase is shown in [5] to be in effect changes in potential are not considered bounded by 2 log n - k + 3. The total amount until a delete-min is performed. of work spent during the delete-min and the net Let us consider the case in which the auxiliary change in potential can thus be bounded by i + k area is nonempty when a delete-min takes place. We and 2 log n - i - k - 1, respectively, which bounds let i > 0 denote the number of (root) nodes in the the amortized time by 2 log n - 1. auxiliary area. The multipass linking spends i - 1 The other case to consider is when the auxiliary units of work building these i nodes into a single area is empty at the time of a delete-min, that is, tree, reducing the potential by 5i - 5. We can show when i = 0. In that case, the reasoning in the last that the sum of the ranks of the nodes in the result- paragraph shows that the total amount of work and ing auxiliary tree is bounded by 4i - 4, as follows: If the net change in potential are bounded by k and log i is a power of 2, then the auxiliary tree in the n - k + 3, respectively. This completes the proof. Cl binary sense has a complete left subtree of size i - 1 The same approach combined with the analysis in and no right subtree. Since the number of nodes on [5] proves the following: descending levels of the binary tree doubles as sub- THEOREM 2. tree sizes are roughly halved, the sum of the ranks is The auxilia y multipass algorithm achieves amortized time bounds of O(2) for insert and find-min and O((log n log i + z 240g + - 1 . log log n)/log log log n) for delete-min and delete if no Osjjdogi (’ 1 decrease-key operations are allowed. Now let m = log i and simphfy. The sum is bounded Unfortunately, we cannot as yet extend either al- by gorithm’s analysis to include decrease-key nodes in m + m C 2j -. 1 j2j the auxiliary area. The problem lies in the subtrees Osjam Osj3n attached to nodes whose values are decreased. We = m + m(2”‘+l - 1) - wz2m+2+ (m + 1)2m+’ - 2 are hopeful that some variant of this batching tech- nique will prove that the O(l)-time amortized = 2 m+l -2 bounds for insert and decrease-key and the O(log n)- time amortized bound for delete-min do hold. Note =2i-2 that we can get a weaker result by a slight modifica- If i is not a power of 2, then we can append I i - 2 tion of our algorithms. If the auxiliary area is extra dummy root nodes to the auxiliary area so that batched whenever a decrease-key or delete-min is per- there are 2”Ogi’ root nodes. It is straightforward to formed, then get the desired bounds, except that show by induction that the tree resulting from the decrease-key and delete-min use O(log n) time for aux- multipass linking of the auxiliary area without the iliary twopass and O((log n log log n)/log log log n) dummy nodes can be “embedded” in the tree result- time for auxiliary multipass. ing from the multipass linking of the auxiliary area with the dummy nodes. By the analysis given above 5. CONCLUSIONS AND OPEN PROBLEMS for the case when i is a power of 2, the sum of the The experimental data gathered from our simula- ranks is bounded by 2(2i - 2) - 2 + 2&l = 4i - 6 + tions provide empirical evidence that the O(l)-time 26iz1 5 4i - 4. (The Kronecker delta 6i=1 denotes 1 bounds for insert, decrease-key, and find-min and the if i = 1 and 0 otherwise.) We can get a better bound O(log n)-time bounds for delete-min and delete do by considering the contribution of the dummy hold in the amortized sense, where n is the size of nodes, but for our purposes this bound is adequate, the priority queue at the time of the operation. All since we are ignoring cOIlsti3nt factors. the pairing heap methods performed well in our When the auxiliary tree is linked with the main simulations. The auxiliary twopass variant clearly tree (so that all the nodes are in the main tree and did the best. This result is satisfying because we the auxiliary area is empty), one unit of work is have shown that the auxiliary twopass algorithm expended, and the change in potential is bounded by achieves the above mentioned bounds, assuming log n + 4i - 4 - 5, where n is the number of nodes that no decrease-key operations occur. Or if the auxil- in the priority queue. Next, the root is deleted via iary area is batched and merged with the main tree

248 Communications of the ACM March 1987 Volume 30 Number 3 Research Contributions whenever a decrease-key or delete-min operation is 6. Jones, D.W. An empirical comparison of priority queue and event set implementations. Commun. ACM 29, 4 (Apr. 1986), 300-311. done, then insert and find-min have O(l)-time 7. Knuth, D.E. The Art of Computer Programming, Vol. 3: Sorting and bounds and all other operations have O(log n)-time Searching. Addison-Wesley, Reading, Mass., 1973. 8. Tarjan. R.E. Amortized computational complexity. SIAM J Algorithm bounds in the amortized sense. Disc. Mefh. 6 (Apr. 1985). 306-316. Proving amortized time bounds of O(1) for insert 9. Vuillemin, J. A data structure for manipulating priority queues. Commun. ACM 21,4 (Apr. 19781, 309-314. and decrease-key and O(log n) for delete-min for some 10. Williams, J.W.J. Algorithm 232: Heapsort. Commun. ACM 7, 6 (June pairing heap implementation remains the major 1964),347-348. open problem. [5] has shown that an O(log n)-time

amortized bound can be proven for all operations of CR Categories and Subject Descriptors: D.l [Programming Tech- the twopass variant. No multipass variant has been niques]; E.l [Data Structures]--trees; F.2.2 [Analysis of Algorithms and Problem Complexity]: Nonnumerical Algorithms and Problems-sorting proven to achieve the same O(log n)-time amortized and searching; G.2.2 [Discrete Mathematics]: Graph Theory--trees bound for all operations; delete-min remains the pri- General Terms: Algorithms, Design, Experimentation, Performance, Theory mary stumbling block. Demonstrating this logarith- Additional Key Words and Phrases: Amortized analysis, heap, pair- mic time bound for all multipass operations is an- ing heap, priority queue other interesting open problem.

REFERENCES Received 4/86; revised 11/86; accepted 1 l/86 1. Aho, A.V.. Hopcroft. J.E., and Ullman, J.D. The Design and Analysis of Compufer Algorithms. Addison-Wesley, Reading, M&s., 1974. - 2. Brown. M.R. Implementation and analysis of binomial queue algo- Author’s Present Address: John T. Stasko and Jeffrey Scott Vitter. rithms. SIAM J. Comput. 7 (Aug. 1978), 298-319. Department of Computer Science, Box 1910, Brown University, 3. Crane, C.A. Linear lists and priority queues as balanced binary Providence, R.I. 02912. trees. Tech. Rep. STAN-CS-72259, Dept. of Computer Science, Stanford University, Feb. 1972. 4. Fredman, M.L., and Tarjan. R.E. Fibonacci heaps and their uses in improved network optimization algorithms. In Proceedings of the 25th Permission to copy without fee all or part of this material is granted Annual Symposium on Foundations of Computer Science, (West Palm provided that the copies are not made or distributed for direct commer- Beach, Fla.. Oct. 1984), 338-344. cial advantage, the ACM copyright notice and the title of the publication 5. Fredman. M.L., Sedgewick, R., Sleator, D.D., and Tarjan, R.E. The and its date appear, and notice is given that copying is by permission of pairing heap: A new form of self-adjusting heap. Algorithmica I (Mar. the Association for Computing Machinery. To copy otherwise, or to 1986), 111429. republish. requires a fee and/or specific permission.

Q: I-800-342-6626 tit’s Fast,Convenient, Your credit card and our toll free number provide quick fulfillment of your orders.

l Journals l Conference Proceedings l SIG Newsletters l SIGGRAPH VIDEO REVIEW l “Computers in your Life” (An Introductory Film from ACM)

For Inquiries and other Customer Service call: (301) 528-4261 ACM’s“Order Express u ASSOCIATION FOR ServiceforACM Publications. acm COMPUTING MACHINERY

March 1987 Volume 30 Number 3 Communications of the ACM 249