Bits and Radix Sort Here's a Slightly More Detailed Explanation of the Running Time of Radix Sort

Bits and Radix Sort Here's a slightly more detailed explanation of the running time of radix sort. Computers internally represent numbers in base 2, i.e. as bits. Suppose you are sorting numeric keys, each having length b = 16 bits. In binary, they might look like this:

1011000011101001

0001111001101010

1110010100010000

How many dierent keys are possible? 16-bit numbers can represent 2^16 = 65,536 distinct values. We could create 65,536 buckets and sort the keys with bucket sort or counting sort. But if we're only sorting 1,000 keys or so, the cost of our algorithm is dominated by managing empty buckets. The idea of radix sort is to do multiple passes of bucket or counting sort. During each pass, we examine just a few bits of each key. Suppose we decide to examine d = 3 bits of each key per pass. Then we break each key into digits of 3 bits each:

1|011|000|011|101|001

0|001|111|001|101|010

1|110|010|100|010|000

Now, we do 6 passes of bucket/csounting sort, processing the digits from right (least signicant) to left (most signicant). There's no way we can do one-third of a pass, so the leftmost bit requires a pass all for itself. How many buckets do we need? We need one bucket for each possible "digit", i.e. combination of 3 bits. Decimal Binary 0 000 1 001 2 010 3 011 4 100 5 101 6 110 7 111

1 If we scan 3 bits each pass, we need 23 = 8 buckets, one for each possible digit. More generally, if we scan d bits per pass, we need:

q = 2d buckets

Therefore, if we have q buckets, the number of bits we scan in one pass is:

bits d = log2 q and the number of passes we have to do is: d = log2 q b b = d log2 q But not really, because we can't do a fractional pass; we need to round up to the nearest integer and do

b b = passes. d log2 q Each pass of bucket/counting sort takes O(n + q) time, so the running time of radix sort is in:

b b (n + q) O (n + q) = O n + q + log2 q log2 q The ceiling is never less than one, and never adds more than one pass, so we get the second expression. It would be stupid to have way more buckets than items to sort, so make sure that q ≤ 10n or so. Then the running time simplies to:

bn O n + log2q On the other hand, we don't want to have too many passes, so we shouldn't make too small. The best way to choose is to experiment with an implemen- q q √ tation of radix sort, and try dierent values of q, roughly in the range from n to 2n. We denitely want q to be of the form 2i where i is an integer, because computers can extract base-2i digits (i.e. groups of bits, illustrated above) much faster than they can extract digits of other bases. √ √ Note that , so if we choose , we only do twice as log2 n = 0.5 log2 n q = n many passes as if we choose q = n. So if memory is at an absolute premium, we can reduce the number of buckets sharply by doubling the run time.

Running Time Problems Note, these are pretty hard problems. Come to oce hours if you have questions.

2 1. Amortized Analysis: Given an array of length n, rigorously show that to add n additional elements, including the costs involved in doubling the size of the array, the total cost is still O(n). 2. Amortized Analysis: Say you have a table of size N, which is some unimag- inably huge number. This table has two operations, insert, and clear. Insert is constant time, and inserts some element into the table. Clear takes time proportional to the number of elements currently in the table. Given a sequence of some mix of insertions and clears in whatever order summing up to n total calls, how long does it take? 3. You are designing a game-tree search searching 3-ply, where at each move, there are always exactly 3 possible moves.

(a) What is the running time of minimax game-tree search, giving a branching factor of 3, and no alpha-beta pruning? (b) Assume that your alpha-beta pruning algorithm involves a killer move heurisitic, such that you are always pruning as much of the tree as possible. i. How many nodes are being pruned? How many nodes do you end up exploring? ii. (challenge) What is the running time, given depth d?

4. You have a set of x elements. Find the running time for the following operations.

(a) Create a minheap for all of them. (b) Remove the min, y times and insert into a queue. (c) Remove each element from the queue and insert each element into a normal binary search tree. (d) How long does it take overall?

5. You have a set of x elements, each of which is 32 bits or less. Analyze the following running times. They are done in order.

(a) Sort them in increasing order using MergeSort, and save the result as an array. (b) Sort the sorted array again with Insertion Sort. (c) We insert all of the elements one at a time from the front to end of the array into a minHeap. How long does this take? We are not using bottom-up heapify. (Surprisingly, not x lg x)

6. You have a tree of n elements. Each of the nodes is an integer, with range [1, n] and all of them are unique. Each node has a list of its children. Your job is to sort each list of children, so they are each in increasing order, so

3 that each parent sees a list in ascending order. Each node has its own list of children, and they don't aect one another, so I only care that each list is sorted.

(a) Analyze the runtime using MergeSort at every node to sort its children. (b) George proposes that instead of using a comparison sort, we should use a linear sort like bucketsort. He claims that by using n buckets at every group of siblings, the sort is linear to the number of siblings, s, and that the sum of all of this is simply n, since there are a total of n integers in the tree. This is because the numbers range from 1 to n, and regardless of what the tree looks like, there will be at most 1 thing in each bucket. Is he brilliant or...not so much?