On the Adaptiveness of Quicksort

Home , Integer sorting, Sorting algorithm

Gerth Stølting Brodal∗,† Rolf Fagerberg‡,§ Gabriel Moruz∗

Abstract by Quicksort for a sequence of n elements is essentially Quicksort was first introduced in 1961 by Hoare. Many 2n ln n ≈ 1.4n log2 n [10]. Many variants and analysis variants have been developed, the best of which are of the algorithm have later been given, including [1, 12, among the fastest generic sorting algorithms available, 19, 20, 21]. In practice, tuned versions of Quicksort as testified by the choice of Quicksort as the default have turned out to be very competitive, and are used as sorting algorithm in most programming libraries. Some standard sorting algorithms in many software libraries, sorting algorithms are adaptive, i.e. they have a com- e.g. C glibc, C++ STL-library, Java JDK, and the .NET plexity analysis which is better for inputs which are Framework. nearly sorted, according to some specified measure of A sorting algorithm is called adaptive with respect presortedness. Quicksort is not among these, as it uses to some measure of presortedness if, for some given Ω(n log n) comparisons even when the input is already input size, the running time of the algorithm is provably sorted. However, in this paper we demonstrate em- better for inputs with low value of the measure. Perhaps pirically that the actual running time of Quicksort is the most well-known measure is Inv, the number of adaptive with respect to the presortedness measure Inv. inversions (i.e. pairs of elements that are in the wrong Differences close to a factor of two are observed be- order) in the input. Other measures of presortedness tween instances with low and high Inv value. We then include Rem, the minimum number of elements that show that for the randomized version of Quicksort, the must be removed for the remaining elements to be number of element swaps performed is provably adap- sorted, and Runs, the number of consecutive ascending tive with respect to the measure Inv. More precisely, runs. More examples of measures can be found in [5]. we prove that randomized Quicksort performs expected An example of an adaptive sorting algorithm is insertion O(n(1 + log(1 + Inv/n))) element swaps, where Inv de- sort using level-linked B-trees with finger searches for notes the number of inversions in the input sequence. locating each new insertion point [13], which sorts in This result provides a theoretical explanation for the O(n(1 + log(1 + Inv/n))) time. In the comparison observed behavior, and gives new insights on the be- model, this is known to be optimal with respect to the havior of the Quicksort algorithm. We also give some measure Inv [5]. empirical results on the adaptive behavior of Heapsort Most classic sorting algorithms, such as Quicksort, and Mergesort. Heapsort [6, 23], and Mergesort [11], are not adaptive: their time complexity is Θ(n log n) irrespectively of 1 Introduction the input. However, a large body of adaptive sorting algorithms, such as the one in [13], has been developed Quicksort was introduced by Hoare in 1961 as a simple over the last three decades. For an overview of this area, randomized sorting algorithm [8, 9]. Hoare proved we refer the reader to the 1992 survey [5]. Later work that the expected number of comparisons performed on adaptive sorting includes [2, 3, 4, 15, 17]. Most of these results are of theoretical nature, ∗BRICS (Basic Research in Computer Science, www.brics.dk, and few practical gains in running time have been funded by the Danish National Research Foundation), Depart- demonstrated for adaptive sorting algorithms compared ment of Computer Science, University of Aarhus, IT Parken, to good non-adaptive algorithms. ˚ ˚ Abogade 34, DK-8200 Arhus N, Denmark. E-mail: {gerth, Our starting point is the converse observation: the gabi}@brics.dk. †Supported by the Carlsberg Foundation (contract number actual running time of a sorting algorithm could well ANS-0257/20) and the Danish Natural Science Research Council be adaptive even if no worst case adaptive analysis (SNF). (showing asymptotical improved time complexity for ‡Department of Mathematics and Computer Science, Univer- input instances with low presortedness) can be given. sity of Southern Denmark, Campusvej 55, DK-5230 Odense M, In this paper, we study such practical adaptability Denmark. E-mail: [email protected]. §Supported in part by the Danish Natural Science Research and demonstrate empirically that significant gains can Council (SNF). be found for the classic non-adaptive algorithms Quick- sort, Mergesort, and Heapsort, under the measure of #define Item int #define random(l,r) (l+rand() % (r-l+1)) presortedness Inv. Gains of more than a factor of three #define swap(A, B) { Item t = A; A = B; B = t; } are observed. Furthermore, in the case of Quicksort, we give theo- void quicksort(Item a[], int l, int r) retical backing for why this should be the case. Specifi- { int i; cally, we prove that randomized Quicksort performs ex- if (r <= l) return; i = partition(a, l, r); pected O(n(1 + log(1 + Inv/n))) element swaps. This quicksort(a, l, i-1); not only provides new insight on the Quicksort algo- quicksort(a, i+1, r); rithm, but it also gives a theoretical explanation for the } observed behavior of Quicksort. The reason that element swaps in Quicksort should int partition(Item a[], int l, int r) { int i = l-1, j = r+1, p = random(l,r); be correlated with running time is (at least) two-fold: Item v = a[p]; element swaps incur not only read accesses but also for (;;) { write accesses (thereby making them more expensive while (++i < j && a[i] <= v); than read-only operations like comparisons), and ele- while (--j > i && v <= a[j]); ment swaps in Quicksort are correlated with branch mis- if (j <= i) break; swap(a[i], a[j]); predictions during the partition procedure of the algo- } rithm. if (p < i) i--; For Quicksort and Mergesort we show empirically swap(a[i], a[p]); the strong influence of branch mispredictions on the return i; running time. This is in line with recent findings of } Sanders and Winkel [18], who demonstrate the practi- Figure 1: Randomized Quicksort. cal importance of avoiding branch mispredictions in the design of sorting algorithms for current CPU architectures. For Heapsort, our experiments indicate that data we prove Theorem 1.1. In Section 3 we describe our cache misses are the dominant factor for the running experimental setup, and in Section 4 we describe and time. discuss our experimental results. Parts of our proof of The observed behavior of Mergesort can be ex- Theorem 1.1 were inspired by the proof by Seidel [22, plained using existing results (see Section 4.2), while Section 5] of the expected number of comparisons we leave open the problem of a theoretical analysis of performed by randomized Quicksort. the observed behavior of Heapsort. Since our theoretical contributions regard Quicksort, we concentrate our 2 Expected number of swaps by randomized experiments on this algorithm, while mostly indicat- Quicksort ing that similar gains can be found empirically also for In this section we analyze the expected number of ele- Mergesort and Heapsort. ment swaps performed by the classic version of random- The main result of this paper is Theorem 1.1 below ized Quicksort where in each recursive call a random stating a dependence between the expected number pivot is selected. The C code for the specific algorithm of swaps performed by randomized Quicksort and the considered is given in Figure 1. The parameters l and number of inversions in the input. In Section 4, the r are the first and last element, respectively, of the seg- theorem is shown to correlate very well with empirical ment of the array a to be sorted. results. We assume that the n input elements are distinct. Theorem 1.1. The expected number of element swaps In the following, let (x1, . . . , xn) denote the input se- performed by randomized Quicksort is at most n + quence, and let πi be the rank of xi in the sorted se- 2Inv quence. The number of inversions in the input sequence n ln n + 1 . is denoted by Inv. The main observation used in the We note that the bound on the number of element proof of Theorem 1.1 is that an element xi that has swaps in Theorem 1.1 is not optimal for sorting al- not yet been moved from its input position i is swapped gorithms. Straightforward in-place selection sort uses during a partitioning step if and only if the selected 2 O(n ) comparisons but performs at most n − 1 element pivot xj satisfies i ≤ πj < πi or πi < πj ≤ i, or xi is swaps for any input. An optimal in-place sorting algo- itself the pivot element (this is seen by inspection of the rithm performing O(n) swaps and O(n log n) compar- code, noting that after a partitioning step, the pivot el- isons was recently presented in [7]. ement xj resides at its final position πj ). We shall only This paper is organized as follows: In Section 2 need the “only if” part. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 of xi from its correct position in the sorted output. The 20 18 12 9 8 16 2 14 6 1 4 21 10 19 7 5 correlation between Inv and the di values is captured by

π8 ¡ ¡ ¡ ¡ ¡ ¡ ¡

¢¡¢¡¢¡¢¡¢¡¢¡¢¡¢ the following lemma: ¡ ¡ ¡ ¡ ¡ ¡ ¡ 5 7 12 9 8 10 2 4 6 1 ¢¡¢¡¢¡¢¡¢¡¢¡¢¡¢14

π11

©¡©¡©¡© £¡£¡£¡£¡£¡£¡£¡£ ¡ ¡ ¡

¤¡¤¡¤¡¤¡¤¡¤¡¤¡¤ Lemma 2.1. n

©¡©¡©¡© £¡£¡£¡£¡£¡£¡£¡£ ¡ ¡ ¡ 4 9 8 10 7 12 6 5 ¤¡¤¡¤¡¤¡¤¡¤¡¤¡¤ Inv ≤ i=1 di ≤ 2Inv.

π15

§¡§¡§¡§¡§¡§¡§¡§ ¥¡¥¡¥¡¥¡¥¡¥¡¥¡¥ ¨¡¨¡¨¡¨¡¨¡¨¡¨¡¨

¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦ P n

§¡§¡§¡§¡§¡§¡§¡§ ¥¡¥¡¥¡¥¡¥¡¥¡¥¡¥ ¨¡¨¡¨¡¨¡¨¡¨¡¨¡¨ 7 10 12 8 9 ¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦ Proof. For the left inequality, Inv ≤ di, we con-

¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ i=1

π13

¡ ¡ ¡ ¡ ¡ ¡ ¡

¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡¡¡¡¡¡¡¡¡¡

¡ ¡ ¡ ¡ ¡ ¡ ¡ sider the following algorithm: If there is an element xi

¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡¡¡¡¡¡¡¡¡¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ 8 9 10 P

¡¡¡¡¡¡¡¡¡¡¡ not at its correct position, move xi to position πi, such

π5

¡¡¡¡¡¡¡¡

¡¡¡¡¡¡¡¡ ¡¡¡¡¡¡¡¡¡¡¡ ¡¡¡¡¡¡¡¡¡¡¡

that position πi temporarily contains both xi and xπi in

¡¡¡¡¡¡¡¡ ¡¡¡¡¡¡¡¡¡¡¡ ¡¡¡¡¡¡¡¡8 sorted order. Next move xπi to its correct position, and repeat moving an element from the position temporar-

Figure 2: The partitions involving element 8. ily containing two elements to its correct position, until

DD FFF @@@ we move an element to position i. Repeat until the se-

i j

HH BB ** && >>

II CC ++ DDEE ?? FFFGGG @@@AA

'' quence is sorted. By moving element xi from position i

HH BB ** && >>

II CC ++ EE ?? GGG AA Input xi '' xj to its correct position πi, we move xi over the di − 1 I

πj πk πi

elements at positions between i and πi and possibly the Sorted xj xi current element at position πi. This decreases the num-

πi − πj ber of inversions in the sequence by at most di, namely

""" $$$

i j any inversions between xi and each of the at most di

(((

!! """## $$$%%% )) | {z }

elements moved over. In the ﬁnal sorted sequence there

(((

!! ## %%% )) Input x x i j n II are no inversions, hence we have Inv ≤ i=1 di.

πj πi n

For the right inequality, i=1 di ≤ 2Inv, consider Sorted xj xi P some xi with πi ≥ i. In the input sequence there are at P

πi − πj + 1 least di inversions between xi and other input elements,

66 88222 000 since there are at least di elements less than xi with

i | {z } j

44 <<... ::

55 6677 == ;;000111 889922233

/// indices greater than i in the input sequence. A similar

44 <<... ::

55 77 == ;;111 99 33 Input xi /// xj argument holds for the case when πi < i. Taking into III

πi πk πj ,,,,,,,,,,,,,

------------- account that we may count the same inversion twice, we ,,,,,,,,,,,,, Sorted -------------xi xj n 2 obtain i=1 di ≤ 2Inv.

di πj − πi + 1 ThePconstants in Lemma 2.1 are the best possible. | {z }| {z } Figure 3: The three different cases of Lemma 2.2. For even n, the sequence (2, 1, 4, 3, 6, 5, . . . , n, n − 1) has n n Inv = n/2 and i=1 di = n, i.e. ( i=1 di)/Inv = 2, whereas the sequence (n, n − 1, n − 2, n − 3, . . . , 3, 2, 1) Fact 2.1. When xi is swapped the first time, the pivot P n P 2 has Inv = n(n − 1)/2 and i=1 di = n /2, i.e. xj of the current partitioning step satisfies i ≤ πj < πi n 1 ( i=1 di)/Inv = 1 + n−1 which converges to one for or πi < πj ≤ i, or xi is itself the pivot element. increasing n. P P For the proof of Theorem 1.1 we make the following Figure 2 illustrates how the element x = 8 is moved 5 definition: during the execution of randomized Quicksort. Circled elements are the selected pivots. The first two selected Definition 2.1. For i 6= j let Xij denote the indicator pivots 14 and 4 do not cause 8 to be swapped, since variable that is one if and only if there is a recursive 8 is already correctly located with respect to the final call to quicksort where xj is selected as the pivot in positions of of the pivots 14 and 4. The first pivot the partition step and xi is swapped during this partition causing 8 to be swapped is x15 = 7, since π5 = 7, step. π15 = 6, and 5 ≤ π15 < π5. In the succeeding recursive calls after the first swap Note that xj can at most once become a pivot, of an element xi, the positions of xi in the array are since after a partition with pivot xj the input to the unrelated to i and πi. Eventually, xi is either picked as recursive calls do not contain xj . Furthermore note that a pivot or becomes a single element input to a recursive the elements swapped in a partition step with pivot xj call (the base case is reached), after which xi does not are the elements in the input to the partition which move further. are placed incorrectly relatively to the final position πj In the following we let di = |πi − i|, i.e. the distance of xj . There are three cases where Xij = 0: (i) xj is never xi has not been moved yet when xj becomes the pivot, selected as a pivot, i.e. there exists a recursive call where and the partitioning with pivot xj does not swap xi. xj is the only element to be sorted; (ii) xj is selected as It follows that the probability that xi is in the input to a pivot in a recursive call and xi is not in the input to the recursive call with pivot xj and xi is not swapped, is 1 this recursive call; and (iii) xj is selected as a pivot in at least . Since the probability that xi is in |πi−πj |+1+di 1 a recursive call and xi is in the input to this recursive the input to the recursive call with pivot xj is , |πi−πj |+1 call, but xi is not swapped because it is placed correctly the lemma follows. 2 relatively to the final position πj of xj . Using Lemma 2.1 and Lemma 2.2 we now have the Lemma 2.2. Pr[X = 1] ≤ ij following proof of Theorem 1.1. 0 if πj < i ≤ πi or πi ≤ i < πj , 1 Proof (Theorem 1.1). The for-loop in the partitioning if i ≤ πj < πi or πi < πj ≤ i ,  |πi−πj |+1 procedure in Figure 1 only swaps non-pivot elements 1 − 1 otherwise.  |πi−πj |+1 |πi−πj |+1+di and each element is swapped at most once in the loop. The loop is followed by one swap involving the pivot. Proof. For the case (i) where xj is never selected as Since a swap of two elements xi and xk not involving the a pivot for a partition, we in the following adopt pivot xj are counted by the two indicator variables Xij the convention that xj is considered the pivot for the and Xkj , the expected number of swaps is at most recursive call where the input consists of xj only. This ensures that each element becomes a pivot exactly once. We first note that the probability that xi is in the n n 1 1 input to the recursive call with pivot xj is , E 1 + Xij |πi−πj |+1   2  j=1 i=1,i=j since this is the probability that xj is the first element X X6 n n chosen as a pivot among the |πi − πj | + 1 elements xk   1  with π ≤ π ≤ π or π ≤ π ≤ π (if the first pivot x = n + Pr(Xij = 1) i k j j k i k 2 i=1 j=1,i=j among the |πi − πj | + 1 elements is not xj , then the X X6 n d selected pivot xk will cause xi and xj to not appear 1 i 1 together in any input to succeeding recursive calls). (2.1) ≤ n + + 2 k + 1 To prove the lemma we consider the three different i=1 k=1 n X X cases depending on the relative order of i, πi, and πj . In 1 1 the following we assume i ≤ πi. The cases where πi < i − k + 1 k + 1 + di k=1 ! are symmetric. The three possible scenarios are shown X n d in Figure 3. 1 i 1 ≤ n + 2 First consider the case where πj < i ≤ πi, see 2 k + 1 i=1 k=1 ! Figure 3 (I). If a pivot xk is selected with πj < πk ≤ πi X X n d +1 before xj becomes a pivot, then xi and xj do not appear i 1 = together in any input to succeeding recursive calls, so k i=1 k=1 xi cannot be involved in the partition with pivot xj . Xn X The only other possibility is that xj is a pivot before (2.2) ≤ (1 + ln(di + 1)) any element xk with πj < πk ≤ πi becomes a pivot, i=1 but then by Fact 2.1 xi has not been moved when xj X n (di + 1) becomes a pivot, and the partitioning with pivot xj does (2.3) ≤ n + n ln i=1 n not swap xi. P 2Inv For the second case, where i ≤ πj < πi, see (2.4) ≤ n + n ln + 1 n Figure 3 (II), we bound the probability that Xij equals one by the probability that xi is in the input to the where (2.1) follows from Lemma 2.2, (2.2) follows from recursive call with pivot xj . As argued above, this n 1 probability is 1 . i=1 i ≤ 1+ln n, (2.3) follows from the concavity of the |πi−πj |+1 logarithm function, and (2.4) follows from Lemma 2.1. For the last case where i ≤ πi < πj , see Fig- P 2 ure 3 (III), we consider the probability that xi is in the input to the recursive call with pivot xj and xi is not It should be noted that the upper bound achieved swapped. This is at least the probability that xj is the in (2.3) using the concavity of the logarithm function first element chosen as a pivot among the |πi−πj |+1+di can be much larger than the value (2.2). As an example, elements xk with i ≤ πk ≤ πj , since then by Fact 2.1 if there are Θ(n/ log n) di values of size Θ(n) and the rest of the di values are zero, then the difference 4 Experimental results between (2.2) and (2.3) is a factor Θ(log n), i.e. the 4.1 Quicksort. We first analyze the dependence of upper bound on the expected number of swaps stated the version of Quicksort shown in Figure 1 on the in Theorem 1.1 can be a factor of log n from the actual number of inversions in the input. bound. Figure 4 shows our data for the AMD Athlon architecture. The number of comparisons is independent 3 Experimental setup of the number of inversions in the input, as expected. In the remainder of this paper, we investigate whether For the number of element swaps, the plot is very classic, theoretically non-adaptive sorting algorithms close to linear when considering the input sequence with can show adaptive behavior in practice. We find that small di’s. Since the x-axis shows log(Inv), this is in very this indeed is the case—the running times for Quick- good correspondence with the bound O(n log(Inv/n)) sort, Mergesort, and Heapsort are observed to improve of Theorem 1.1 (recall that n is fixed in the plot). For by factors between 1.5 and 3.5 when the Inv value of the input sequence with large di’s, the plot is different. the input goes from high to low. Furthermore, the im- This is a sign of the slack in the analysis (for this type provements for Quicksort are in very good concordance of input) noted after the proof of Theorem 1.1. We with Theorem 1.1, which shows this result to be a likely will demonstrate below that this curve is in very good explanation for the observed behavior. correspondence with the version of the bound given by In more detail, we study how the number of in- Equation (2.2). The plots for the number of branch mis- versions in the input sequence affects the number of predictions and for the running time clearly show that comparisons, the number of element swaps, the num- they are correlated with the number of element swaps. ber of branch mispredictions, the running time, and the For the number of branch mispredictions, this is ex- number of data cache misses of the version of Quicksort plained by the fact that an element swap is performed shown in Figure 1. We also study the behavior of two after the two while loops stop, and hence corresponds variants of Quicksort, namely the randomized version to two branch mispredictions. For the running time, that chooses the median of three random elements as it seems reasonable to infer that branch mispredictions a pivot, and the deterministic version that chooses the are a dominant part of the running time of Quicksort on middle element as a pivot. Finally, we study the same this type of architecture. Finally, the number of data questions for the classic sorting algorithms Heapsort and cache misses seems independent of the presortedness of Mergesort. the input sequence, in correspondence with the fact that The input elements are 4 byte integers. We generate for all element swaps, the data to be manipulated is al- two types of input, having small di’s and large di’s, ready in the cache and therefore the element swaps do respectively. We generate a sequence with small di’s by not generate additional cache misses. choosing each element xi randomly in [i−d, . . . , i+d] for Figure 5 show the same plots for the P4 architec- some parameter d, whereas the sequence with large di’s ture, except that we were not able to obtain data for is generated by letting xi = i with the exception of d L2 data cache misses. We note that the plots follow the random i’s for which xi is chosen randomly in [1, . . . , n]. same trends as in Figure 4. The number of comparisons We perform our experiments by varying the disorder (by and the number of element swaps are approximately the varying d) while keeping the size n of the input sequence same, but the running time is affected by up to a factor constant. For most experiments, the input size is 2×106, of 1.8 on the P4, while only by up to a factor of 1.42 on but we also investigate larger and smaller input sizes. the Athlon. One reason for this behavior is the number Our experiments are conducted on two different of branch mispredictions, which is slightly smaller for machines. One of the machines has an Intel P4 2.4 GHz the Athlon. Also, the length of the pipeline, shorter for CPU with 512 MB RAM, running linux 2.4.20, while Athlon, makes the branch mispredictions more costly the other has an AMD Athlon XP 2400+ 2.0 GHz on a P4 than on an Athlon. CPU with 256 MB RAM, running linux 2.4.22. On Similar observations on the resemblance between both machines the C source code was compiled using the data for the two architectures apply to all our gcc-3.3.2 with optimization level -O3. The number of experiments. For this reason, and because of the branch mispredictions and L2 data cache misses was extra data for L2 that we have for Athlon, we for obtained using the PAPI library [16] version 3.0. the remaining plots restrict ourselves to the Athlon Source code and the plotted data are available at architecture. ftp://ftp.brics.dk/RS/04/47/Experiments. We now turn to the variants of Quicksort. Figure 6 shows the number comparisons, the number of element swaps, the number of branch mispredictions, the running time, and the L2 data cache misses for the version sort is affected slightly, while the number of branch mis- of Quicksort that chooses as pivot the median of three predictions is affected somewhat more. However, the random elements in the input sequence. We note that number of L2 data cache misses is greatly affected, and the plots have a behavior similar to the ones for the ver- varies by more than a factor of ten. The running time sion of Quicksort shown in Figure 4. However, some im- shows a virtually identical behavior, except the increase provements are noticed. The three-median pivot Quick- is by a factor close to four. This suggests that data sort performs around 25% less comparisons, due to the cache misses are the dominant factor for the running better choice of the pivot. This immediately triggers a time for Heapsort on this architecture. We leave open slight improvement in the number of data cache misses. the question of a theoretical analysis of the number of However, the number of branch mispredictions increases cache misses of Heapsort as a function of Inv. due to the extra branches required to compute the me- For Mergesort, we focus on the binary merge pro- dian of three elements. The number of element swaps cess, and count the number of times there is an alterna- remains approximately the same. tion in which of the two input subsequences provides the Figure 7 shows the same plots for the determin- next element output. It is easy to verify that the num- istic version of Quicksort that chooses the middle el- ber of such alternations is dominated by the running ement as pivot. In this case we note that the num- time of the Mergesort algorithm by Moffat [14] based ber of comparisons does depend on the presortedness on merging by finger search trees, which was proved to Inv of the input. This is because for small disorder, the have a running time of O(n log n ), i.e. the number of middle element is very close to the median and there- Inv alternations by standard Mergesort is O(n log n ). The fore the number of comparisons is close to n log n, as plots in Figure 12 show a very similar behavior for the opposed to ≈ 1.4n log n expected for the randomized number of alternations, the number of branch mispre- Quicksort [10]. The good pivot choice for small disorder dictions, and the running time. The number of alterna- in the input also triggers a smaller number of compar- tions is clearly correlated to the number of branch mis- isons and branch mispredictions. However, for large dis- predictions, and these appear to be a dominant factor order, the number of comparisons is larger compared to for the running time of Mergesort. The number of data randomized median-of-three Quicksort due to bad pivot cache misses increases only slightly for large disorder in choices. Also, the running time is affected by up to a the input. factor of two by the disorder in the input. Figure 8 and Figure 9 show that when varying the References input size n, the behavior of the plots remains the same for randomized Quicksort. Hence, our findings do not [1] J. L. Bentley and M. D. McIlroy. Engineering a seem to be tied to the particular choice of n = 2 × 106. sort function. Software—Practice and Experience, Finally, in Figure 10 we demonstrate that the 23(11):1249–1265, Nov. 1993. number of element swaps is very closely related to n [2] A. Elmasry. Priority queues, pairing, and adaptive i=1 log di, cf. the comment after the proof of Theo- sorting. In ICALP: Annual International Colloquium rem 1.1. Hence the reason for the non-linear shape of on Automata, Languages and Programming, 2002. P the previous plots for input sequences with large di’s [3] A. Elmasry. Adaptive sorting with AVL trees. Techni- seems to be the slack introduced (for this type of input) cal Report 2003-46, DIMACS, Feb. 2004. after Equation (2.2) in the proof of Theorem 1.1. As [4] A. Elmasry and M. L. Fredman. Adaptive sorting and in the other cases, the running time and the number the information theoretic lower bound. In STACS: An- of branch mispredictions follow the same trend as the nual Symposium on Theoretical Aspects of Computer Science number of swaps. , 2003. [5] V. Estivill-Castro and D. Wood. A survey of adaptive sorting algorithms. Computing Surveys, 24:441–476, 4.2 Heapsort and Mergesort. We briefly demon- 1992. strate that also for Heapsort and Mergesort, the actual [6] R. W. Floyd. Algorithm 245: Treesort3. Communica- running time varies with the presortedness of the input. tions of the ACM, 7(12):701, 1964. For Heapsort, Figure 11 shows the way the number [7] G. Franceschini and V. Geffert. An In-Place Sorting of inversions in the input affects the number of com- with O(n log n) Comparisons and O(n) Moves. In parisons, the number of elements swaps, the number of Proc. 44th Annual IEEE Symposium on Foundations branch mispredictions, the running time, and the num- of Computer Science, pages 242–250, 2003. ber of L2 data cache misses for input sequences of con- [8] C. A. R. Hoare. Algorithm 63: Partition. Commun. stant length n = 2 × 106. The number of comparisons ACM, 4(7):321, 1961. Commun. and the number of element swaps performed by Heap- [9] C. A. R. Hoare. Algorithm 64: Quicksort. ACM, 4(7):321, 1961. The Computer Journal 6.2e+07 [10] C. A. R. Hoare. Quicksort. , Small di Large di 5(1):10–15, April 1962. 6e+07 [11] D. E. Knuth. The Art of Computer Programming, Volume III: Sorting and Searching. Addison-Wesley, 5.8e+07

Reading, MA, 1973. 5.6e+07 [12] C. Mart´ınez and S. Roura. Optimal sampling strate- gies in Quicksort and Quickselect. SIAM Journal on 5.4e+07

Computing, 31(3):683–705, June 2002. 5.2e+07 [13] K. Mehlhorn. Sorting and Searching. Springer Verlag, 15 20 25 30 35 40 Berlin, 1984. Comparisons [14] A. Moﬀat, O. Petersson, and N. C. Wormald. Sort- 1.2e+07 Small di Large di ing and/by merging ﬁnger trees. In Algorithms and 1e+07 Computation: Third International Symposium, ISAAC ’92, volume 650 of Lecture Notes in Computer Science, 8e+06

pages 499–508. Springer Verlag, Berlin, 1992. 6e+06 [15] A. Pagh, R. Pagh, and M. Thorup. On adaptive integer sorting. In 12th Annual European Symposium 4e+06

on Algorithms, ESA 2004, volume 3221 of Lecture 2e+06 Notes in Computer Science, pages 556–567. Springer 15 20 25 30 35 40 Verlag, Berlin, 2004. Element swaps

[16] PAPI (Performance Application Programming Inter- 2.5e+07 Small di face). Software library found at http://icl.cs.utk. Large di

edu/papi/, 2004. 2e+07 [17] O. Petersson and A. Moﬀat. A framework for adaptive sorting. DAMATH: Discrete Applied Mathematics 1.5e+07 and Combinatorial Operations Research and Computer Science, 59, 1995. 1e+07 [18] P. Sanders and S. Winkel. Super scalar sample sort. In 12th Annual European Symposium on Algorithms, 5e+06 15 20 25 30 35 40 ESA 2004, volume 3221 of Lecture Notes in Computer Branch mispredictions Science, pages 784–796. Springer Verlag, Berlin, 2004. 0.6 Small di [19] R. Sedgewick. Quicksort. PhD thesis, Stanford Uni- Large di 0.55 versity, Stanford, CA, May 1975. Stanford Computer Science Report STAN-CS-75-492. 0.5

[20] R. Sedgewick. The analysis of quicksort programs. 0.45 Acta Informatica, 7:327–355, 1977. 0.4 [21] R. Sedgewick. Implementing quicksort programs. Communications of the ACM, 21:847–857, 1978. 0.35

[22] R. Seidel. Backwards analysis of randomized geomet- 0.3 15 20 25 30 35 40 ric algorithms. Technical Report TR-92-014, Interna- Running time tional Computer Science Institute, Univeristy of Cal- 900000 Small di fornia at Berkeley, February 1992. Large di [23] J. W. J. Williams. Algorithm 232: Heapsort. Commu- 800000

nications of the ACM, 7(6):347–348, 1964. 700000

600000

500000

400000

300000 15 20 25 30 35 40 L2 data cache misses

Figure 4: The number of comparisons, the number of element swaps, the number of branch mispredictions, the running time, and the number of L2 data cache misses performed by randomized Quicksort on Athlon, for n = 2 × 106. The x-axis shows log(Inv). 6.2e+07 5.1e+07 Small di Small di Large di Large di 6e+07 5e+07

4.9e+07 5.8e+07

4.8e+07 5.6e+07

4.7e+07 5.4e+07 4.6e+07 5.2e+07 4.5e+07 15 20 25 30 35 40 15 20 25 30 35 40 Comparisons Comparisons 1.2e+07 1.2e+07 Small di Small di Large di Large di 1e+07 1e+07

8e+06 8e+06

6e+06 6e+06

4e+06 4e+06

2e+06 2e+06

15 20 25 30 35 40 15 20 25 30 35 40 Element swaps Element swaps

Small di 2.5e+07 Small di 2.5e+07 Large di Large di

2e+07 2e+07

1.5e+07 1.5e+07

1e+07 1e+07

5e+06 5e+06 15 20 25 30 35 40 15 20 25 30 35 40 Branch mispredictions Branch mispredictions 0.7 0.4 Small di Small di Large di Large di 0.65 0.35

0.6 0.3

0.55 0.25

0.5 0.2

15 20 25 30 35 40 15 20 25 30 35 40 Running time Running time 800000 Small di 750000 Large di Figure 5: The number of comparisons, the number of 700000 element swaps, the number of branch mispredictions, 650000 and the running time of randomized Quicksort on P4, 600000 6 550000 for n = 2 × 10 . The x-axis shows log(Inv). 500000 450000 400000 350000 300000 15 20 25 30 35 40 L2 data cache misses

Figure 6: The number of comparisons, the number of element swaps, the number of branch mispredictions, the running time, and the number of L2 data cache misses performed by randomized median-of-three Quicksort on Athlon, for n = 2 × 106. The x-axis shows log(Inv). Small di 2.5e+06 Small di Large di Large di 5.5e+07 2.4e+06

5e+07 2.3e+06

2.2e+06 4.5e+07 2.1e+06

4e+07 2e+06

15 20 25 30 35 40 14 16 18 20 22 24 26 28 30 32 Comparisons Comparisons 1.2e+07 450000 Small di Small di Large di 400000 Large di 1e+07 350000

8e+06 300000

250000 6e+06 200000

4e+06 150000

100000 2e+06 50000 15 20 25 30 35 40 14 16 18 20 22 24 26 28 30 32 Element swaps Element swaps

Small di 900000 Small di 2e+07 Large di Large di 800000

700000 1.5e+07 600000

500000 1e+07 400000

300000 5e+06 200000

100000 15 20 25 30 35 40 14 16 18 20 22 24 26 28 30 32 Branch mispredictions Branch mispredictions 0.5 0.02 Small di Small di Large di 0.019 Large di 0.45 0.018 0.4 0.017 0.016 0.35 0.015 0.3 0.014 0.013 0.25 0.012 0.2 0.011 15 20 25 30 35 40 14 16 18 20 22 24 26 28 30 32 Running time Running time 700000 25000 Small di Small di 650000 Large di Large di

600000 20000

550000 15000 500000

450000 10000

400000 5000 350000

300000 0 15 20 25 30 35 40 14 16 18 20 22 24 26 28 30 32 L2 data cache misses L2 data cache misses

Figure 7: The number of comparisons, the number of Figure 8: The number of comparisons, the number of element swaps, the number of branch mispredictions, element swaps, the number of branch mispredictions, the running time, and the number of L2 data cache the running time, and the number of L2 data cache misses performed by deterministic Quicksort on Athlon, misses performed by randomized Quicksort on Athlon, for n = 2 × 106. The x-axis shows log(Inv). for n = 6 × 104. The x-axis shows log(Inv). 6.2e+07 Small di Small di 3.15e+08 Large di Large di 6e+07

3.1e+08 5.8e+07

3.05e+08 5.6e+07

3e+08 5.4e+07

2.95e+08 5.2e+07

20 25 30 35 40 45 0 5e+06 1e+07 1.5e+07 2e+07 2.5e+07 3e+07 3.5e+07 4e+07 Comparisons Comparisons 1.2e+07 6e+07 Small di Small di Large di Large di 1e+07 5e+07

8e+06 4e+07

6e+06 3e+07

2e+07 4e+06

1e+07 2e+06

20 25 30 35 40 45 0 5e+06 1e+07 1.5e+07 2e+07 2.5e+07 3e+07 3.5e+07 4e+07 Element swaps Element swaps 1.4e+08 Small di Small di Large di Large di 1.2e+08 2e+07

1e+08 1.5e+07 8e+07

6e+07 1e+07

4e+07 5e+06 2e+07

20 25 30 35 40 45 0 5e+06 1e+07 1.5e+07 2e+07 2.5e+07 3e+07 3.5e+07 4e+07 Branch mispredictions Branch mispredictions 3.2 0.65 Small di Small di Large d Large d 3 i 0.6 i

2.8 0.55

0.5 2.6 0.45 2.4 0.4 2.2 0.35 2 0.3 20 25 30 35 40 45 0 5e+06 1e+07 1.5e+07 2e+07 2.5e+07 3e+07 3.5e+07 4e+07 Running time Running time 5e+06 Small di 900000 Small di Large di Large di

4.5e+06 800000

700000 4e+06 600000

500000 3.5e+06 400000

3e+06 300000 20 25 30 35 40 45 0 5e+06 1e+07 1.5e+07 2e+07 2.5e+07 3e+07 3.5e+07 4e+07 L2 data cache misses L2 data cache misses

Figure 9: The number of comparisons, the number of Figure 10: The number of comparisons, the number of element swaps, the number of branch mispredictions, element swaps, the number of branch mispredictions, the running time, and the number of L2 data cache the running time, and the number of L2 data cache misses performed by randomized Quicksort on Athlon, misses performed by randomized Quicksort on Athlon, for n = 107. The x-axis shows log(Inv). for the input size n = 2 × 106. The x-axis shows n i=1 log(di + 1). P 7.75e+07 Small di Small di Large di Large di 2e+07

7.7e+07 1.5e+07

7.65e+07 1e+07

7.6e+07 5e+06

15 20 25 30 35 40 15 20 25 30 35 40 Comparisons Alternations 3.9e+07 Small di Small di 3.88e+07 Large di 2.5e+07 Large di 3.86e+07 2e+07 3.84e+07

3.82e+07 1.5e+07

3.8e+07 1e+07 3.78e+07 5e+06 3.76e+07

3.74e+07 0 15 20 25 30 35 40 15 20 25 30 35 40 Element swaps Branch mispredictions 2.4e+07 Small di 0.6 Small di 2.3e+07 Large di Large di 2.2e+07 0.55 2.1e+07 2e+07 0.5 1.9e+07 1.8e+07 0.45 1.7e+07 1.6e+07 0.4 1.5e+07 15 20 25 30 35 40 15 20 25 30 35 40 Branch mispredictions Running time 1.47e+06 Small di Small di Large di 1.46e+06 Large di 2 1.45e+06

1.44e+06 1.5 1.43e+06

1.42e+06 1 1.41e+06

1.4e+06 0.5 1.39e+06 15 20 25 30 35 40 15 20 25 30 35 40 Running time L2 data cache misses 1.6e+07 Small di 1.4e+07 Large di Figure 12: The number of alternations, the number 1.2e+07 of branch mispredictions, the running time, and the 1e+07 number of L2 data cache misses performed by Mergesort 8e+06 on Athlon, for n = 2 × 106. The x-axis shows log(Inv). 6e+06

4e+06

2e+06

0 15 20 25 30 35 40 L2 data cache misses

Figure 11: The number of comparisons, the number of element swaps, the number of branch mispredictions, the running time, and the number of L2 data cache misses performed by Heapsort on Athlon, for n = 2 × 106. The x-axis shows log(Inv).