Heaps
Binary (max-)heaps (Chapter 6)
Properties
Operations
make-heap, heapsort and its analysis Binomial heaps (Chapter 19)
Mergeable heaps
Binomial trees, binomial heaps
Analysis results Heaps
Data structures that support efficient implementation for (at least) three operations: insert(x), find-max(), and delete-max() There are two obvious implementation approaches:
Operations insert(x) find-max() delete-max() Data structures Array (or O(1) time, simply O(1) time, if O(n) time, to linked list) append x to the maintains a reset the max end max pointer pointer A sorted O(n) time, to O(1) time, O(1) time, array keep array return the last delete last sorted value value Implementation of Heaps
A binary tree satisfies the heap property if the value stored at a node ≥ the values stored at its child nodes.
Example: The figure below depicts a (left-)complete binary tree that satisfies the heap property:
A left-complete binary tree contains the full sets of nodes at each level, from top (the root) 10 to bottom except possibly missing nodes on 79 the right end of the bottom level (see figure); a “packed” implementation using an array is 47 52 given below:
missing nodes 2 1 6 at right end of 1 794752216 bottom level 0 Operations
The find-max operation can be done in O(1) time. insert and delete-max can be done in O(lg n) time procedures described below: 10 3 9 9
79Decrease a 79Swap the 7 3 Swap 75 value, heap decreased again, heap 5 2 property 5 2 value with 5 2 property 3 2 the larger restored A binary violated child max-heap sift-down(T[1..n], i) moves the value at index i violating the heap property to a proper place. Since a left-complete binary tree of d levels stored as array T[1..n] satisfies 2d–1 ≤ n ≤ 2d – 1, thus, d –1 ≤ lg n < d, i.e., d = ⎣lg n⎦ +1. Therefore, the time complexity of sift-down is O(lg n). Sift-down sift-down (T[1..n], i) /* array T[1..n] stores a left-complete binary tree satisfying the heap property except the value at index i maybe too small */ k = i repeat j = k /* find the larger child of j */ j if 2j ≤ n and T[2j] > T[k] then k = 2j if 2j < n and T[2j+1] > T[k] then k = 2j+1 swap T[j] and T[k] /* see illustration */ k until j = k /* node j has reached a proper place */
When a left-complete binary tree (e.g. a heap) is stored in array T[1..n], node j’s left and right children (if present) are at indexes 2j and 2j +1, resp. Also, node j’s parent is at index ⎣j/2⎦ if node j is not the root. Sift-up
sift-up (or percolate) procedure is used to restore the heap property, which also runs in O(lg n) time. 10 10 10 10 Increase a Swap the Swap 7 value, heap 7 increased 7 again, heap 8 property value with property 4 violated 4 the parent 8 restored 7
2 1 2 8 2 4 2 4 procedure sift-up (T[1..n], i) /* array T[1..n] stores a left-complete binary tree satisfying the heap property except the value at index i maybe too large */ k = i repeat j = k if j > 1 and T[j / 2] < T[k] then k = j / 2 k swap T[j] and T[k] /* see illustration */ j until j = k Heap Operations
store the values in an array T[1..n] representing a left- complete binary tree satisfying the heap property function find-max(T[1..n]) return T[1] /* return the largest value */ procedure delete-max(T[1..n]) T[1] = T[n] /* delete the max stored at T[1] */ sift-down (T[1..n –1], 1) /* restore the heap property */ procedure insert(T[1..n], x) T[n+1] = x /* insert x */ sift-up (T[1..n +1], n +1) /* restore the heap property */
Note that find-max() can be done in O(1) time, both insert() and delete-max() can be done in O(lg n) time. Make-heap
Convert an array into a binary heap: Given an arbitrary array T[1..n] of numbers, convert T into a binary max-heap:
procedure slow-make-heap(T[1..n]) /* repeatedly insert T[i] into heap T[1..i –1] */ for i = 2 to n do sift-up(T[1..i], i) Example: Convert the array T[1..6] ={1, 6, 9, 2, 7, 5} into a heap. insert 6: insert 9: insert 2: insert 7: insert 5: 1 6 9 9 9 sift-up sift-up sift-up sift-up 6 1 9 16 26 7 6 2 1 7 125 The procedure’s time complexity is θ(n lgn) since inserting the n ith number costs θ(lg i) time in the worst case, and ∑lgi = θ(nlgn) i=2 An alternative approach: start from the “bottom” of the binary tree, repeatedly merging two subtrees (heaps) into a single, larger heap, until there is one single heap covering the entire array. sift-down and merge Example: Convert the array T[1..10] two sub-heaps into a ={1, 6, 9, 2, 7, 5, 2, 7, 4, 10} into a single heap heap using a bottom-up approach. heap heap 1 Step 3 1 69 69sift-down op after another Step 2 Step 1 Step 4 275 2 after 3 sift- down ops 7105 2
7 4 10 2 4 7 1 10 10 9 79 sift-down op 775 2 475 2 merging two sub-heaps a heap 2 4 6 2 1 6 A converted heap a heap Make-heap
A bottom-up procedure to make heap. Note i = ⎣n / 2⎦ is the largest index of a non-leaf vertex. for i = ⎣n / 2⎦ downto 1 do sift-down(T[1..n], i)
Theorem. The time complexity is O(n). Let the root vertex be at level 1, its children at level 2, etc., then there are ≤ 2k–1 vertices at level k, for 1 ≤ k ≤⎣lg n⎦ +1. The ∑complexity of sift-down() for a vertex at level k is O(d – k), where d = the number of tree levels = ⎛ ⎣⎦lg n ⎞ ⎣lg n⎦ +1. Therefore, the time complexity of make-heap is ⎜ k − 1 ⎟ O⎜ ∑ (d − k)2 ⎟ ⎝ k=1 ⎠ ⎣lg n⎦ ⎣⎦lg n d − k ∞ i leading to the complexity: (d − k)2k−1 = 2d −1 ≤ 2d −1 ≤ O(2⎣⎦lg n ) = O(n) ∑ ∑ d −k ∑ i k=1 k=1 2 i=12 procedure heapsort(T[1..n]) make-heap(T[1..n]) for i = n downto 2 do swap T[1] and T[i] /* swap max of T[1..i] with T[i] */ sift-down(T[1..i –1], 1) /* restore the heap property */ Binomial Heaps
The binary (max-)heaps supports efficient find-max (O(1) time), and insert, delete-max (both O(lg n) time) operations, but not efficient merge operations. Binomial heaps are a mergeable (or meldable) heap supporting all four operations (insert, find-max, delete-max, and merge) efficiently.
First, we consider a sequence of rooted trees, B0, B1, …, known as binomial trees, where B0 consists of a single vertex and, for i ≥ 0, Bi+1 is the union of two copies of Bi
Bi B0 B1 Bi B2 B3 Bi+1 Binomial Heaps A binomial heap consists of a collection of binomial trees without duplicated sizes, each satisfying the heap property. The roots of the binomial trees are linked in order of the tree size.
max
325 23
12 914 20 A binomial heap of 7 A binomial heap 1 517 nodes, in which the 3 of n = 11 nodes binomial trees correspond 8 to the bits in 7 = 1+2+4
The binomial trees for a heap of n nodes correspond to the non- zero bits of the binary representation for n. As a result, the number of such trees is ≤⎣lg (n+1)⎦. An example of n = 7 = (111)2 nodes is shown above. Implementation
A node (a C struct) contains 4 pointer fields (see below): Up, to parent As a result, the sibling nodes form a Left, sibling Right, sibling doubly-linked list; there is also a on its left on its right circular linked list connecting a Down, to parent and the children (then back to leftmost child the parent)
Example. A binomial heap with 11 nodes showing all pointers:
Note that the NIL pointer
B0 value is used wherever is (1 node) appropriate. The root list
B1 forms a doubly-linked list via (2 nodes) the root nodes’ left/right
B3 pointers. (8 nodes) Operations
Heap operations (including merge) using binomial heaps: find-max(H): this can be done in O(1) time, assuming a pointer to the root node with the largest value is maintained merge(H1, H2): this can be done in O(lg n) time, where n = total number of nodes in the two heaps H1 and H2, explained below: linking two binomial trees of equal size in O(1) time
15 12 15
19 48 19 12
6 3 6 48 The merged tree, in which the smaller root was made 3 merge a child of the other root Operations
Generally, merging two binomial heaps uses two pointers H1, H2 to the respective root lists, and a pointer M to the merged list (binomial heap): (Case 1) if 0 < |H1| < |H2| (or 0 < |H2| < |H1|), copy H1 (or H2) to M, advance H1 (or H2), M
MM
If 0 < |H1| < |H2|
H2 H2
(Case 2) when 0 < |H1| = |H2|, link two trees H1, H2 into a single tree (called a “carry”) of twice the size, advance both pointers H1 and H2, then consider the sub-cases below: (2.a) |Carry| < min(|H1|, |H2|) or |Carry| = |H1| = |H2|: copy tree Carry to the merged heap and advance its pointer M, set Carry to NIL
Same size H2 (2.b) |Carry| = |H1| < |H2| (or |Carry| = |H2| < |H1|): link trees Carry and H1 (or H2) into a single tree Carry, advance H1 (or H2) (2.c) |H | = 0 (or |H | = 0), but not both |H | = Carry 1 2 1 |H2| = 0: set H1 (or H2) to Carry, set Carry to NIL, go to Case 1 or Case 2
(2.d) |H1| = |H2| = 0: copy Carry to M, then terminate the merge operation
If Carry ≠ NIL (from 2.b), repeat one of the sub-cases 2.a-2.d. (Case 3) When H1 (or H2) = NIL, copy remaining trees of list H2 (or H1) to M, then terminate the merge operation Thus, the merge operation terminates (in case 2.d or case 3) when H1 = H2 = NIL Merge
Time complexity of the merge operation. Linking two binomial trees takes O(1) time, and there are O(lg n) iterations in the merge loop which advances at least one of the pointers H1 or H2 in each iteration, the entire merge operation takes O(1)⋅O(lg n) = O(lg n) time, including the time to reset the pointer to the maximum value of the merged heap. Another way: the analogy between merge(H1, H2) and adding two binary numbers B1, B2 where the bit string of Bi corresponds to the sizes of Hi’s binomial trees. For example:
1 2 2 1
4 8 111 16 32 16 32 + 110010 ------111001 Heap Operations Using Binomial Heaps
insert(H, x): inserting item x to a binomial heap H is a special case of merge, by first constructing a single-node binomial heap containing x, then merging the heap into H; the worst-case time complexity is O(1) + O(lg n) = O(lg n). delete-max(H): this operation can be done in three steps: (1) delete the tree rooted at max (a pointer to the tree that contains the largest value of H); (2) construct a binomial heap, call it H1, that contains all subtrees of the tree rooted at max; (3) merge H1 into H; the time complexity is O(1) + O(lg n) + O(lg n) = O(lg n). max max te tree dele merge The final Heap H
A H1 to H construct Heap H A 1 B B A C C E D DB Heap H C D E