Heapsort 1 Heaps • The next sorting algorithm we will talk about is Heapsort. • Before we talk about Heapsort, we need to discuss the heap data structure. • A heap is a complete binary tree such that for every node, key(child) ≤ key(parent) • Recall that a complete binary tree is full at every level except possibly the last, which is left-filled. • Example: X T O GS M N A E R A I Heapsort 2 Height of a Heap • Theorem: A heap with n nodes has height Θ(log n) • Proof: – Let n be the number of nodes of a heap of height h. – Since a binary tree of height h has at most 2h − 1 nodes, it is not too difficult to see that h−1 h 2 ≤ n ≤ 2 − 1 – Taking logs on both sides of the first inequality, we get h − 1 ≤ log n. – Adding one and taking logs on both sides of the second inequality, we get log(n + 1) ≤ h. – Thus, log(n + 1) ≤ h ≤ log n + 1. Heapsort 3 Heap Examples • Example: Heaps with height h = 3 T n = 4 G S A T n = 7 GO A C M N • Example: Heap with height h = 4 15 n=11 9 14 7 4 11 13 6 5 2 1 Heapsort 4 Heap Representation • We have already seen how to represent binary trees, so we know how to represent heaps. • Using arrays, we find parents/children as follows (Assuming we start indexing at 1): – Left(i) = 2i – Right(i) = 2i + 1 – P arent(i)= i/2 • Example: X 1 TO2 3 n 2 GS4 5 M 6 N 7 log AIE R A 8 910 11 12 X T O G S M N A E R A I 1 2 3 4 5 6 7 8 9 10 11 12 Heapsort 5 The Heapify Routine • The Heapify routine is the basis of all other routines needed to use heaps. • Heapify details: – Input: An array A and index i into the array. – Assumption: The subtrees rooted at Left(i) and Right(i) are heaps. – Problem: A[i] may violate the heap property. – Output: The array A, where the tree rooted at i is a heap. • It is not hard to see that Heapify runs in O(log n) time. • Why is this useful? We will see. • For now, just imagine that if we change the value of some key in the heap, it may violate the heap property, and thus must be fixed. Heapsort 6 Heapify • Here is a C++ implementation of Heapify: void Heapify(int *A,int i,int n) { l=Left(i); r=Right(i); if(l<= n && A[l] > A[i]) largest=l; else largest=i; if(r<= n && A[r] > A[largest]) largest=r; if(largest !=i) { Swap(A[i],A[largest]) Heapify(A,largest,n); } } This is much simpler than it looks. Heapify simply: • Determines the largest of A[i], A[Left(i)], and A[Right(i)] • If A[i] is not the largest, then – Swap A[i] with the A[largest] (where largest is either Right(i) or Left(i)) – Calls heapify on node largest. Heapsort 7 Heapify Example 15 i 3 14 6 7 11 13 4 5 2 1 15 7 14 i 6 3 11 13 4 5 2 1 Heapsort 8 Build Heap • The routine BuildHeap converts a regular array into a heap. • Essentially, it runs Heapify on the nodes in reverse order. • Since we are going in reverse order, we are know that the subtrees rooted at the children are heaps. • The last half of the array corresponds to leaf nodes, so we don’t need to heapify them. • Here is the C++ code for Heapify: void BuildHeap(int *A,int n) { for(i=n/2;i>=1;i--) Heapify(A,i,n); } • It is clear that BuildHeap runs in O(n log n) time. • In fact BuildHeap runs in O(n) time. Heapsort 9 Build Heap Example 13 1 9 5 7 15 14 11 6 13 1 9 11 7 15 14 5 6 13 1 15 11 7 9 14 5 6 13 11 15 6 7 9 14 5 1 15 11 14 6 7 9 13 5 1 Heapsort 10 Other Heap Operations • There are other operations one may wish to perform on a heap: – Insert() – Extract Max() • We won’t discuss these here, but it is not too difficult to implement them given the Heapify procedure. • With these operations, we can use a heap to implement a priority queue. • A priority queue is a data structure which supports the operations insert, maximum, and extract maximum. • We will use the priority queue data structure later. • We are interested in the Heapsort algorithm, so that is what we will do next. Heapsort 11 Heapsort The idea behind Heapsort is simple. • Run BuildHeap to turn array A into a heap. • Since A is a heap, the largest element is A[1]. • Swap A[1] with A[n]. Now A[n] is in place. • Since A[n] is in place, we can ignore it. Thus, we decrease the heap size by 1. • Now A[1] might violate the heap property, so we run Heapify on the (now smaller) heap. • We repeat until only one item is left in the heap. • Our array is sorted. The C++ implementation: void Heapsort(int *A, int n) { BuildHeap(A,n); for(int i=n;i>1;i--) { Swap(A[1],A[i]); Heapify(A,1,i-1); } } Heapsort 12 Heapsort Example • Let A = [13, 1, 9, 5, 7, 15, 14, 11, 6]. • Heapify will produce A = [15, 11, 14, 6, 7, 9, 13, 5, 1]. 15 11 14 6 7 9 13 5 1 • Now, swap, “remove”, and heapify until done: 15 Start 11 14 6 7 9 13 5 1 Swap and Remove 1 11 14 6 7 9 13 5 15 14 Heapify 11 13 6 7 9 1 5 15 Heapsort 13 Heapsort Example Continued 5 Swap and Remove 11 13 6 7 9 1 14 15 13 Heapify 11 9 6 7 5 1 14 15 1 Swap and Remove 11 9 6 7 5 13 14 15 11 Heapify 7 9 6 1 5 13 14 15 Heapsort 14 Heapsort Example Continued 5 Swap and Remove 7 9 6 1 11 13 14 15 9 Heapify 7 5 6 1 11 13 14 15 1 Swap and Remove 7 5 6 9 11 13 14 15 7 Heapify 6 5 1 9 11 13 14 15 Heapsort 15 Heapsort Example Continued 1 Swap and Remove 6 5 7 9 11 13 14 15 6 Heapify 1 5 7 9 11 13 14 15 5 Swap, Remove and Heapify 1 6 7 9 11 13 14 15 1 Swap, Remove and Heapify 5 6 7 9 11 13 14 15 We now have A = [1, 5, 6, 7, 9, 11, 13, 14, 15]. Heapsort 16 Heapsort Complexity • BuildHeap takes O(n) time. • Heapify takes O(log n) time. • Heapsort makes one call to BuildHeap and n − 1 calls to Heapify. • Thus, the complexity of Heapsort is O(n +(n − 1) log n)= O(n log n). • It is usually not as good as Quicksort in practice..