Algorithms ROBERT SEDGEWICK | KEVIN WAYNE Two classic sorting algorithms: mergesort and quicksort

Critical components in the world’s computational infrastructure. ・Full scientific understanding of their properties has enabled us to develop them into practical system sorts. ・Quicksort honored as one of top 10 algorithms of 20th century 2.3 QUICKSORT in science and engineering.

‣ quicksort Mergesort. [last lecture] ‣ selection ... Algorithms ‣ duplicate keys FOURTH EDITION ‣ system sorts Quicksort. [this lecture] ROBERT SEDGEWICK | KEVIN WAYNE http://algs4.cs.princeton.edu ...

2

Quicksort t-shirt

2.3 QUICKSORT

‣ quicksort ‣ selection Algorithms ‣ duplicate keys ‣ system sorts

ROBERT SEDGEWICK | KEVIN WAYNE

http://algs4.cs.princeton.edu

3 number). 9.9 X 10 45 is used to represent infinity. Imaginary conunent I and J are output variables, and A is the array (with Quicksort valuesTony of x may Hoarenot be negative and reM values of x may not be subscript bounds M:N) which is operated upon by this procedure. smaller than 1. Partition takes the value X of a random element of the array A, Values of Qd~'(x) may be calculated easily by hypergeometric and rearranges the values of the elements of the array in such a series if x is not too small nor (n - m) too large. Q~m(x) can be way that there exist integers I and J with the following properties : computed from an appropriate set of values of Pnm(X) if X is near M _-< J < I =< NprovidedM < N 1.0 or ix is near 0. Loss of significant digits occurs for x as small as A[R] =< XforM =< R _-< J Basic plan. 1.1 if n is largerInvented than 10. Loss of significant quicksort digits is a major diffi-to translateA[R] = RussianXforJ < R < I into English. culty・ in using finite polynomiM representations also if n is larger A[R] ~ Xfor I =< R ~ N than m. However, QLEG has been tested in regions of x and n The procedure uses an integer procedure random (M,N) which Shuffle the array. both large and[ butsmall; couldn't explain hischooses algorithmequiprobably a random integer or F implementbetween M and N, and it! ] ・ procedure・ QLEG(m, nmax, x, ri, R, Q); value In, nmax, x, ri; also a procedure exchange, which exchanges the values of its two real In, mnax, x, ri; real array R, Q; parameters ; begin real t, i, n, q0, s; begin real X; integer F; j n := 20; ・Partition so that, for some ・Learned Algol 60 (and recursion).F := random (M,N); X := A[F]; if nmax > 13 then I:=M; J:=N; n := nmax +7; up: for I : = I step 1 until N do – entry a[j] is in place if Implementedri = 0then quicksort. if X < A [I] then go to down; ・ begin ifm = 0then I:=N; Q[0] := 0.5 X 10g((x + 1)/(x - 1)) down: forJ := J step --1 until M do else if A[J]Turing Award begin s := (x+x)X(i-1)Xt end 2: begin 4 × Q[0]+ (3i-i× i-2)× q0; else if [ < F then begin exchange (A[IL A[F]) i if c >= 0 then Sort each subarray recursively. q0 := Q[0]; I:=I+l 3:begin ・ Q[0] := s end end; end e:= aXe;f := bXd;goto8 if x = 1 then else if F < J tllen begin exchange (A[F], A[J]) ; end 3 ; Q[0] := 9.9 I" 45; J:=J-1 e:=bXc; R[n + 1] := x - sqrt(x X x - 1); end ; ifd ~ 0then for i := n step --1 until 1 do end partition 4: begin R[i] := (i + m)/((i + i + 1) X x f:=bXd; goto8 +(m-i- 1) X R[i+l]); end 4; ALGORITHM 61 go to the end; f:=aXd; goto8 input Q U I C K S O R T E X A M P L E if m = 0 then ALGORITHMPROCEDURES 64FOR RANGE ARITHMETIC 5: end 2; begin if x < 0.5 tben QUICKSORTALLAN GIBB* ifb > 0 then Q[0] := arctan(x) - 1.5707963 else C.University A. R. HOARE of Alberta, Calgary, Alberta, Canada 6: begin shufe Q[0] := - aretan(1/x)end else K R A T E L E P U I M Q C X O S Elliottbegin Brothers Ltd., Borehamwood, Hertfordshire, Eng. if d > 0 then begin t := 1/sqrt(x X x + 1); begin procedure RANGESUM (a, b, c, d, e, f); q0 := 0; procedure quicksort (A,M,N); value M,N; e := MIN(a X d, b X c); partitioning item real a,b,c,d,e,f; q[0] := t; array A; integer M,N; f:= MAX(a X c,b X d); go to8 comment The term "range number" was used by P. S. Dwyer, for i : = 2 step 1 until m do comment Quicksort is a very fast and convenient method of end 6; Linear Computations (Wiley, 1951). Machine procedures for begin s := (x + x) X (i -- 1) X t X Q[0I sorting an array in the random-access store of a computer. The e:= bX c; f := aX c; go to8 partition E C A I E K L P U T M Q R X O S range arithmetic were developed about 1958 by Ramon Moore, +(3i+iX i -- 2) × q0; entire contents of the store may be sorted, since no extra space is end 5; "Automatic Error Analysis in Digital Computation," LMSD qO := Q[0]; required. The average number of comparisons made is 2(M--N) In f:=aXc; Report 48421, 28 Jan. 1959, Lockheed Missiles and Space Divi- not greater not less Q[0] := s end end; (N--M), and the average nmnber of exchanges is one sixth this if d _-< O then sion, Palo Alto, California, 59 pp. If a _< x -< b and c ~ y ~ d, R[n + 1] := x - sqrt(x × x + 1); amount. Suitable refinements of this method will be desirable for 7: begin then RANGESUM yields an interval [e, f] such that e =< (x + y) for i := n step - 1 until 1 do its implementation on any actual computer; e:=bXd; goto8 f. Because of machine operation (truncation or rounding) the sort left A C E E I K L P U T M Q R X O S R[i] := (i + m)/((i -- m + 1) × R[i + 1] begin integer 1,J ; end 7 ; machine sums a -4- c and b -4- d may not provide safe end-points --(i+i+ 1) X x); if M < N then begin partition (A,M,N,I,J); e:=aXd; of the output interval. Thus RANGESUM requires a non-local for i : = 1 step 2 until nmax do quicksort (A,M,J) ; 8: e := ADJUSTPROD (e, -1); sort right A C E E I K L M O P Q R S T U X real procedure ADJUSTSUM which will compensate for the Rill := -- Rill; quicksort (A, I, N) f := ADJUSTPROD (f, 1) machine arithmetic. The body of ADJUSTSUM will be de- the: for i : = 1 step 1 until nnmx do end end RANGEMPY; pendent upon the type of machine for which it is written and so Q[i] := Q[i - 1] X R[i] end quieksort procedure RANGEDVD (a, b, c, d, e, f) ; result A C E E I K L M O P Q R S T U X is not given here. (An example, however, appears below.) It end QLEG; real a, b, c, d, e, f; is assumed that ADJUSTSUM has as parameters real v and w, comment If the range divisor includes zero the program * This procedure was developed in part under the sponsorship and integer i, and is accompanied by a non-local real procedure exists to a non-local label "zerodvsr". RANGEDVD assumes a Quicksort overview of the Air Force Cambridge Research Center. CORRECTION which gives an upper bound to the magnitude CommunicationsALGORITHM 65 of the ACM (July 1961) non-local real procedure ADJUSTQUOT which is analogous of the error involved in the machine representation of a number. FIND (possibly identical) to ADJUSTPROD; 5 The output ADJUSTSUM provides the left end-point of the 6 begin C.output A. R. interval HOARE of RANGESUM when ADJUSTSUM is called ALGORITHM 63 if c =< 0 A d ~ 0 then go to zer0dvsr; Elliottwith i Brothers= --1, and Ltd., the right Borehamwood, end-point when Hertfordshire, called with i Eng.= 1 if c < 0 then PARTITION The procedures RANGESUB, RANGEMPY, and RANGEDVD procedure find (A,M,N,K); value M,N,K; C. A. R. HOARE provide for the remaining fundamental operations in range 1: begin array A; integer M,N,K; ifb > 0 then Elliott Brothers Ltd., Borehamwood, Hertfordshire, Eng. arithmetic. RANGESQR gives an interval within which the comment Find will assign to A [K] the value which it would 2: begin square of a range nmnber must lie. RNGSUMC, RNGSUBC, procedure partition (A,M,N,I,J); value M,N; have if the array A [M:N] had been sorted. The array A will be e := b/d; go to3 RNGMPYC and RNGDVDC provide for range arithmetic with array A; integer M,N,1,J; partly sorted, and subsequent entries will be faster than the first; end 2; complex range arguments, i.e. the real and imaginary parts Tony Hoare Bob Sedgewick are range numbers~ e := b/c; begin Communications of the ACM 321 3: ifa -->_ 0then e := ADJUSTSUM (a, c, -1); 4: begin f := a/c; go to 8 f := ADJUSTSUM (b, d, 1) end RANGESUM; end 4; f := a/d; go to 8 procedure RANGESUB (a, b, c, d, e, f) ; real a, b,c, d,e, f; end 1 ; Invented quicksort to translate Russian into English. Refined and popularized quicksort. if a < 0 then ・ ・ comment RANGESUM is a non-local procedure; begin 5: begin RANGESUM (a, b, -d, --c, e, f) e := a/c; go to6 ・ [ but couldn't explain his algorithm or implement it! ] Analyzed many versions of quicksort. end 5 ; ・ end RANGESUB ; procedure RANGEMPY (a, b, c, d, e, f); e := a/d; real a, b, c, d, e, f; 6: ifb > 0then Learned Algol 60 (and recursion). comment ADJUSTPROD, which appears at the end of this 7: begin ・ f := b/c; go to8 procedure, is analogous to ADJUSTSUM above and is a non- local real procedure. MAX and MIN find the maximum and end 7 ; Implemented quicksort. minimum of a set of real numbers and are non-local; f := b/d; ・ begin 8: e := ADJUSTQUOT (e, -1); f := ADJUSTQUOT (f,1) real v, w; end RANGEDVD ; if a < 0A c => 0then procedure RANGESQR (a, b, e, f); real a, b, e, f; rithms,1: begin and we can learn from that experience to separate Tony Hoare v:=c; c:=a; a:=v; w:=d; d:=b; b:=w comment ADJUSTPROD isBob a non-10cal Sedgewick procedure; goodend algorithms 1; from bad ones. Third, if the tile fits into begin 1980 Turing Award the memoryif a => O of then the computer, there is one algorithm, if a < 0 then “ There are two ways of constructing a software design: One way is called Quicksort, which has been shown to perform well Programming S. L. Graham, R. L. Rivest in a variety of situations. Not only is this algorithm Communications of the &CM 319 Techniques Editors simpler than many other sorting algorithms, but empir- to make it so simple that there are obviously no deficiencies, and ical [2, ll, 13, 21] and analytic [9] studies show that Quicksort can be expected to be up to twice as fast as its Implementing nearest competitors. The method is simple enough to be the other way is to make it so complicated that there are no obvious learned by programmersActa Informatica who have no7, 327--355previous experi-(1977) Quicksort Programs ence with sorting, and9 by those Springer-Verlag who do know 1977 other sorting Robert Sedgewick methods should also find it profitable to learn about deficiencies. The first method is far more difficult. ” Brown University Quicksort. Because of its prominence, it is appropriate to study how Quicksort might be improved.The This Analysis subject has of Quicksort Programs* This paper is a practical study of how to implement received considerable attention (see, for example, [1, 4, the Quicksort sorting algorithm and its best variants on 11, 13, 14, 18, 20]), but few real improvements have beenRobert Sedgewick suggested beyond those described by C.A.R. Hoare, the real computers, including how to apply various code Received January 19, t976 optimization techniques. A detailed implementation inventor of Quicksort, in his original papers [5, 6]. Hoare also showed how to analyze Quicksort and predict its combining the most effective improvements to Summary. The Quicksort sorting algorithm and its best variants are presented “ I call it my billion-dollar mistake. It was the invention of the null Quicksort is given, along with a discussion of how to running time. The analysis has since been extended to the improvements thatand heanalyzed. suggested, Results and used are derivedto indicate which make it possible to obtain exact formulas de- implement it in assembly language. Analytic results scribing the total expected running time of particular implementations on real com- describing the performance of the programs are how they may bestputers be implementedof Quick,sort [9, and 15, an 17]. improvement The called the median-of-three modification. reference in 1965… This has led to innumerable errors, summarized. A variety of special situations are subject of the carefulDetailed implementation analysis of ofthe Quicksort effect of hasan implementation technique called loop unwrapping considered from a practical standpoint to illustrate not been studied asis presented.widely as globalThe paper improvements is intended to not only to present results of direct practical utility, Quicksort's wide applicability as an internal sorting the algorithm, butbut the also savings to illustrate to be therealized intriguing are as mathematics which arises in the complete analysis vulnerabilities, and system crashes, which have probably caused of this important algorithm. method which requires negligible extra storage. significant. The history of Quicksort is quite complex, Key Words and Phrases: Quicksort, analysis of and [15] contains a full survey of the many variants a billion dollars of pain and damage in the last forty years. ” algorithms, code optimization, sorting which, have been proposed. 1. Introduction CR Categories: 4.0, 4.6, 5.25, 5.31, 5.5 The purpose of this paper is to describe in detail how Quicksort can best beIn implemented t96t-62 C.A.R. to handle Hoare actual presented a new algorithm called Quicksort [7, 8] applications on realwhich computers. is suitable A general for puttingdescription files of into order by computer. This method combines the algorithm is followedelegance by and descriptions efficiency, of andthe mostit remains today the most useful general-purpose effective improvementssorting that method have forbeen computers. proposed The(as practical utility of the algorithm has meant 7 demonstrated in [15]).not only Next, that an it implementationhas been sfibjected of to countless modifications (though few real 8 Quicksort in a typicalimprovements high level language have been is presented, suggested beyond those described by Hoare), but also Introduction and assembly languagethat implementationit has been used issues .in arecountless con- applications, often to sort very large, files. sidered. This discussionConsequently, should easily it is translateimportant to toreal be able to estimate how long an implementation One of the most widely studied practical problems in languages on real machines.of Quicksort Finally, can a benumber expected of special to run, in order to be able to compare variants or computer science is sorting: the use of a computer to put issues are consideredestimate which expenses.may be of Fortunately, importance inas we shall see, this is an algorithm which can be files in order. A person wishing to use a computer to sort particular sorting applications.analyzed. (Hoare recognized this, and gave some analytic results in [8].) It is is faced with the problem of determining which of the This paper is intendedpossible to beto a deriveself-contained exact overviewformulas describing the average performance of real many available algorithms is best suited for his purpose. of the properties ofimplementations Quicksort for use byof thosethe algorithm. who need This task is becoming less difficult than it once was for to actually implement andThe use history the algorithm. of Quicksort A compan- is quite complex, and a full survey of the many variants three reasons. First, sorting is an area in which the ion paper [17] provideswhich the have analytical been proposed results which is given su- in [t 7]. In addition, [t 7] gives analytic results mathematical analysis of algorithms has been particu- port much of the discussion presented here. describing many of the improvements which have been suggested for the purpose larly successful: we can predict the performance of many sorting methods and compare them intelligently. Second, of determining which are the most effective. There are many examples in [~ 7] we have a great deal of experience using sorting algo- The Algofithm which illustrate that the simplicity of Quicksort is deceiving. The algorithm has hidden subtleties which can have significant effects on performance. Furthermore, Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct Quicksort is a recursiveas we shall method see, simplefor sorting changes an array to the algorithm or its implementation can radically commercial advantage, the ACM copyright notice and the title of the change the analysis. In this paper, we shall consider in detail how practical publication and its date appear, and notice is given that copying is by A[1], A[2] ..... A[N] by first "partitioning" it so that the permission of the Association for Computing Machinery. To copy following conditionsimplementations hold: of the best versions of Quicksort may be analyzed. otherwise, or to republish, requires a fee and/or specific permission. In this paper, we will deal with the analysis of: (i) the basic Quicksort algo- (i) Some key v is in its final position in the array. (If it This work was supported in part by the Fannie and John Hertz rithm; (ii) an improvement called the "median-of-three" modification which Foundation and in part by NSF Grants. No. GJ-28074 and MCS75- is thejth smallest, it is in position A[j].) 23738. reduces the average number of comparisons required; and (iii) an implementation (ii) All elements to the left of A[j] are less than or equal Author's address: Division of Applied Mathematics and Computer technique called "loop unwrapping" which reduces the amount of overhead per Science Program, Brown University, Providence, RI 02912. to it. (These elements A [ 1], A [2] ..... A [j - 1] are © 1978 ACM 0001-0782/78/1000-0847 $00.75 called the "leftcomparison. subtile.") These particular methods not only represent the most effective vari-

847 Communications * This workOctober was 1978 supported in part by the Fannie and John Hertz Foundation, and of in part byVolume the National21 Science Foundation Grants No. GJ-28074 and MCS75-23738. the ACM 22 Acta Informatica,Number 10Vol. 7 Quicksort partitioning demo Quicksort partitioning demo

Repeat until i and j pointers cross. Repeat until i and j pointers cross. ・Scan i from left to right so long as (a[i] < a[lo]). ・Scan i from left to right so long as (a[i] < a[lo]). ・Scan j from right to left so long as (a[j] > a[lo]). ・Scan j from right to left so long as (a[j] > a[lo]). ・Exchange a[i] with a[j]. ・Exchange a[i] with a[j].

When pointers cross. ・Exchange a[lo] with a[j].

K R A T E L E P U I M Q C X O S E C A I E K L P U T M Q R X O S

lo i j lo j hi

partitioned! 9 10

The music of quicksort partitioning (by Brad Lyon) Quicksort: Java code for partitioning

private static int partition(Comparable[] a, int lo, int hi) { int i = lo, j = hi+1; while (true) {

while (less(a[++i], a[lo])) find item on left to swap if (i == hi) break;

while (less(a[lo], a[--j])) find item on right to swap if (j == lo) break;

if (i >= j) break; check if pointers cross exch(a, i, j); swap }

exch(a, lo, j); swap with partitioning item before v https://googledrive.com/host/0B2GQktu-wcTicjRaRjV1NmRFN1U/index.html return j; return index of item now known to be in place } lo hi

before v during v Յ v Ն v

lo hi i j

after before v during v Յ v Ն v Յ v v Ն v

lo hi i j lo j hi

during v Յ v Ն v after Յ v v Ն v Quicksort partitioning overview 11 12 i j lo j hi

after Յ v v Ն v Quicksort partitioning overview

lo j hi

Quicksort partitioning overview Quicksort quiz 1 Quicksort: Java implementation

Q. How many compares (in the worst case) to partition an array of length N ?

public class Quick A. ~ ¼ N { private static int partition(Comparable[] a, int lo, int hi) B. ~ ½ N { /* see previous slide */ }

C. ~ N public static void sort(Comparable[] a) { StdRandom.shuffle(a); shuffle needed for D. ~ N lg N performance guarantee sort(a, 0, a.length - 1); (stay tuned) } E. I don't know. private static void sort(Comparable[] a, int lo, int hi) { if (hi <= lo) return; int j = partition(a, lo, hi); sort(a, lo, j-1); sort(a, j+1, hi); } }

13 14

Quicksort trace Quicksort animation

50 random items

lo j hi 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 initial values Q U I C K S O R T E X A M P L E random shufe K R A T E L E P U I M Q C X O S 0 5 15 E C A I E K L P U T M Q R X O S 0 3 4 E C A E I K L P U T M Q R X O S 0 2 2 A C E E I K L P U T M Q R X O S 0 0 1 A C E E I K L P U T M Q R X O S 1 1 A C E E I K L P U T M Q R X O S 4 4 A C E E I K L P U T M Q R X O S 6 6 15 A C E E I K L P U T M Q R X O S no partition 7 9 15 A C E E I K L M O P T Q R X U S for subarrays of size 1 7 7 8 A C E E I K L M O P T Q R X U S 8 8 A C E E I K L M O P T Q R X U S 10 13 15 A C E E I K L M O P S Q R T U X 10 12 12 A C E E I K L M O P R Q S T U X 10 11 11 A C E E I K L M O P Q R S T U X 10 10 A C E E I K L M O P Q R S T U X 14 14 15 A C E E I K L M O P Q R S T U X 15 15 A C E E I K L M O P Q R S T U X algorithm position result A C E E I K L M O P Q R S T U X in order Quicksort trace (array contents after each partition) current subarray not in order http://www.sorting-algorithms.com/quick-sort

15 16 Quicksort: implementation details Quicksort: empirical analysis (1961)

Partitioning in-place. Using an extra array makes partitioning easier Running time estimates: (and stable), but is not worth the cost. ・Algol 60 implementation. ・National-Elliott 405 computer. Terminating the loop. Testing whether the pointers cross is trickier than it might seem.

Equal keys. When duplicates are present, it is (counter-intuitively) better to stop scans on keys equal to the partitioning item's key. stay tuned

Preserving randomness. Shuffling is needed for performance guarantee. Equivalent alternative. Pick a random partitioning item in each subarray.

Elliott 405 magnetic disc sorting N 6-word items with 1-word keys (16K words)

17 18

Quicksort: empirical analysis Quicksort: best-case analysis

Running time estimates: Best case. Number of compares is ~ N lg N. ・Home PC executes 108 compares/second. ・Supercomputer executes 1012 compares/second.

initial values

random shuffle insertion sort (N2) mergesort (N log N) quicksort (N log N) computer thousand million billion thousand million billion thousand million billion

home instant 2.8 hours 317 years instant 1 second 18 min instant 0.6 sec 12 min

super instant 1 second 1 week instant instant instant instant instant instant

Lesson 1. Good algorithms are better than supercomputers. Lesson 2. Great algorithms are better than good ones.

19 20 Quicksort: worst-case analysis Quicksort: average-case analysis

2 Worst case. Number of compares is ~ ½ N . Proposition. The average number of compares CN to quicksort an array of

N distinct keys is ~ 2N ln N (and the number of exchanges is ~ ⅓ N ln N ).

initial values Pf. CN satisfies the recurrence C0 = C1 = 0 and for N ≥ 2: left right random shuffle partitioning

C0 + CN 1 C1 + CN 2 CN 1 + C0 C =(N + 1) + + + ... + N N N N ・Multiply both sides by N and collect terms: partitioning probability

NCN = N(N + 1) + 2(C0 + C1 + ... + CN 1) ・Subtract from this equation the same equation for N - 1:

NCN (N 1)CN 1 =2N +2CN 1 ・Rearrange terms and divide by N (N + 1):

CN CN 1 2 = + N +1 N N +1

21 22

Quicksort: average-case analysis Quicksort: summary of performance characteristics

・Repeatedly apply previous equation: Quicksort is a (Las Vegas) randomized algorithm. CCNN CCNN 1 1 22 Guaranteed to be correct. = ++ ・ NN+1+1 NN NN+1+1 Running time depends on random shuffle. CN 2 2 2 ・ = + + substitute previous equation N 1 N N +1 CN 3 2 2 2 Average case. Expected number of compares is . = + + + ~ 1.39 N lg N N 2 N 1 N N +1 39% more compares than mergesort. 2 2 2 2 ・ = + + + ... + 3 4 5 N +1 ・Faster than mergesort in practice because of less data movement.

・Approximate sum by an integral: Best case. Number of compares is ~ N lg N. Worst case. Number of compares is 2. 1 1 1 1 ~ ½ N C = 2(N + 1) + + + ... N 3 4 5 N +1 [ but more likely that lightning bolt strikes computer during execution ] ✓ ◆ N+1 1 2(N + 1) dx ⇠ x Z3

・Finally, the desired result: C 2(N + 1) ln N 1.39N lg N N ⇥ 23 24 Quicksort properties Quicksort: practical improvements

Proposition. Quicksort is an in-place sorting algorithm. Insertion sort small subarrays. Pf. ・Even quicksort has too much overhead for tiny subarrays. ・Partitioning: constant extra space. ・Cutoff to insertion sort for ≈ 10 items. ・Depth of recursion: logarithmic extra space (with high probability).

can guarantee logarithmic depth by recurring on smaller subarray before larger subarray (requires using an explicit stack)

Proposition. Quicksort is not stable. private static void sort(Comparable[] a, int lo, int hi) Pf. [ by counterexample ] { if (hi <= lo + CUTOFF - 1) i j 0 1 2 3 {

B1 C1 C2 A1 Insertion.sort(a, lo, hi); return; 1 3 B1 C1 C2 A1 } int j = partition(a, lo, hi); 1 3 B1 A1 C2 C1 sort(a, lo, j-1); sort(a, j+1, hi); 0 1 A1 B1 C2 C1 }

25 26

Quicksort: practical improvements

Median of sample. ・Best choice of pivot item = median. ・Estimate true median by taking median of sample. ・Median-of-3 (random) items. UICKSORT ~ 12/7 N ln N compares (14% fewer) 2.3 Q ~ 12/35 N ln N exchanges (3% more) ‣ quicksort ‣ selection

private static void sort(Comparable[] a, int lo, int hi) ‣ duplicate keys { Algorithms if (hi <= lo) return; ‣ system sorts

int median = medianOf3(a, lo, lo + (hi - lo)/2, hi); ROBERT SEDGEWICK | KEVIN WAYNE swap(a, lo, median); http://algs4.cs.princeton.edu int j = partition(a, lo, hi); sort(a, lo, j-1); sort(a, j+1, hi); }

27 Selection Quick-select

Goal. Given an array of N items, find the kth smallest item. Partition array so that: Ex. Min (k = 0), max (k = N - 1), median (k = N / 2). ・Entry a[j] is in place. ・No larger entry to the left of j. Applications. ・No smaller entry to the right of j. ・Order statistics. ・Find the "top k." Repeat in one subarray, depending on j; finished when j equals k.

Use theory as a guide. public static Comparable select(Comparable[] a, intbefore k) v { Easy N log N upper bound. How? lo if a[k] is here if a[k] is herehi ・ StdRandom.shuffle(a); set hi to j-1 set lo to j+1 ・Easy N upper bound for k = 1, 2, 3. How? int lo = 0, hi = a.length - 1; during v Յ v Ն v while (hi > lo) ・Easy N lower bound. Why? { i j int j = partition(a, lo, hi); after Յ v v Ն v if (j < k) lo = j + 1; Which is true? else if (j > k) hi = j - 1; lo j hi else return a[k]; N log N lower bound? is selection as hard as sorting? Quicksort partitioning overview ・ } ・N upper bound? is there a linear-time algorithm? return a[k]; }

29 30

Quick-select: mathematical analysis Theoretical context for selection

Proposition. Quick-select takes linear time on average. Proposition. [Blum, Floyd, Pratt, Rivest, Tarjan, 1973] Compare-based selection algorithm whose worst-case running time is linear. Pf sketch. Time Bounds for Selection ・Intuitively, each partitioning step splits array approximately in half: bY . N + N / 2 + N / 4 + … + 1 ~ 2N compares. Manuel Blum, Robert W. Floyd, Vaughan Watt, Ronald L. Rive&, and Robert E. Tarjan ・Formal analysis similar to quicksort analysis yields:

CN = 2 N + 2 k ln (N / k) + 2 (N – k) ln (N / (N – k)) Abstract

L The number of comparisons required to select the i-th smallest of

n numbers is shown to be at most a linear function of n by analysis of Ex: (2 + 2 ln 2) N ≈ 3.38 N compares to find median k = N / 2. i ・ a new selection algorithm -- PICK. Specifically, no more than i L 5.4305 n comparisons are ever required. This bound is improved for extreme values of i , and a new lower bound on the requisite number L of comparisons is also proved. Remark. Constants are high ⇒ not used in practice.

Use theory as a guide.

L ・Still worthwhile to seek practical linear-time (worst-case) algorithm. ・Until one is discovered, use quick-select if you don’t need a full sort.

31 32

This work was supported by the National Science Foundation under grants GJ-992 and GJ-33170X.

1 Duplicate keys

Often, purpose of sort is to bring items with equal keys together. ・Sort population by age. ・Remove duplicates from mailing list. ・Sort job applicants by college attended. sorted by time sorted by city (unstable) sorted by city (stable) 2.3 QUICKSORT Chicago 09:00:00 Chicago 09:25:52 Chicago 09:00:00 Typical characteristics of such applications.Phoenix 09:00:03 Chicago 09:03:13 Chicago 09:00:59 Houston 09:00:13 Chicago 09:21:05 Chicago 09:03:13 ‣ quicksort ・Huge array. Chicago 09:00:59 Chicago 09:19:46 Chicago 09:19:32 Houston 09:01:10 Chicago 09:19:32 Chicago 09:19:46 ・Small number of key values. Chicago 09:03:13 Chicago 09:00:00 Chicago 09:21:05 ‣ selection Seattle 09:10:11 Chicago 09:35:21 Chicago 09:25:52 Seattle 09:10:25 Chicago 09:00:59 Chicago 09:35:21 ‣ duplicate keys Phoenix 09:14:25 Houston 09:01:10 Houston 09:00:13 NOT Algorithms Chicago 09:19:32 Houston 09:00:13 Houston 09:01:10 sorted ‣ system sorts Chicago 09:19:46 Phoenix 09:37:44 sorted Phoenix 09:00:03 Chicago 09:21:05 Phoenix 09:00:03 Phoenix 09:14:25 Seattle 09:22:43 Phoenix 09:14:25 Phoenix 09:37:44 Seattle 09:22:54 Seattle 09:10:25 Seattle 09:10:11 ROBERT SEDGEWICK | KEVIN WAYNE Chicago 09:25:52 Seattle 09:36:14 Seattle 09:10:25 http://algs4.cs.princeton.edu Chicago 09:35:21 Seattle 09:22:43 Seattle 09:22:43 Seattle 09:36:14 Seattle 09:10:11 Seattle 09:22:54 Phoenix 09:37:44 Seattle 09:22:54 Seattle 09:36:14

Stability when sorting on a second key key

34

Duplicate keys: stop on equal keys Quicksort quiz 2

Our partitioning subroutine stops both scans on equal keys. What is the result of partitioning the following array (skip over equal keys)?

scan until > A scan until < A scan until ≥ P scan until ≤ P

A A A A A A A A A A A A A A A A P G E P A Q B P Y C O U P Z S R 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

A. A A A A A A A A A A A A A A A A

Q. Why not continue scans on equal keys? B. A A A A A A A A A A A A A A A A scan until > P scan until < P

P G E P A Q B P Y C O U P Z S R C. A A A A A A A A A A A A A A A A

D. I don't know. 35 36 Quicksort quiz 3 Partitioning an array with all equal keys

What is the result of partitioning the following array (stop on equal keys)?

scan until ≥ A scan until ≤ A

A A A A A A A A A A A A A A A A

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

A. A A A A A A A A A A A A A A A A

B. A A A A A A A A A A A A A A A A

C. A A A A A A A A A A A A A A A A

D. I don't know. 37 38

Duplicate keys: partitioning strategies DUTCH NATIONAL FLAG PROBLEM

Bad. Don't stop scans on equal keys. [ ~ ½ N 2 compares when all keys equal ] Problem. [Edsger Dijkstra] Given an array of N buckets, each containing a red, white, or blue pebble, sort them by color. B A A B A B B B C C C A A A A A A A A A A A

input

Good. Stop scans on equal keys. sorted [ ~ N lg N compares when all keys equal ] Operations allowed. B A A B A B C C B C B A A A A A A A A A A A ・swap(i, j): swap the pebble in bucket i with the pebble in bucket j. ・color(i): color of pebble in bucket i.

Better. Put all equal keys in place. How? Requirements. [ ~ N compares when all keys equal ] ・Exactly N calls to color(). ・At most N calls to swap(). A A A B B B B B C C C A A A A A A A A A A A ・Constant extra space.

39 40 3-way partitioning Dijkstra 3-way partitioning demo

Goal. Partition array into three parts so that: ・Let v be partitioning item a[lo]. ・Entries between lt and gt equal to the partition item. ・Scan i from left to right. ・No larger entries to left of lt. – (a[i] < v): exchange a[lt] with a[i]; increment both lt and i ・No smaller entries to right of gt. – (a[i] > v): exchange a[gt] with a[i]; decrement gt before v – (a[i] == v): increment i

lo hi before v during v

lo hi lt i gt lt i gt during v after v

lt i gt lo lt gt hi P A B X W P P V P D P C Y Z after v 3-way partitioning lo lt gt hi lo hi

3-way partitioning before v invariant Dutch national flag problem. [Edsger Dijkstra] lo hi ・Conventional wisdom until mid 1990s: not worth doing. during v ・Now incorporated into C library qsort() and Java 6 system sort. lt i gt 41 after v 42

lo lt gt hi

Dijkstra 3-way partitioning demo Dijkstra's 3-way partitioning:3-way trace partitioning

・Let v be partitioning item a[lo]. ・Scan i from left to right. v a[] – (a[i] < v): exchange a[lt] with a[i]; increment both lt and i lt i gt 0 1 2 3 4 5 6 7 8 9 10 11 – (a[i] > v): exchange a[gt] with a[i]; decrement gt 0 0 11 R B W W R W B R R W B R – (a[i] == v): increment i 0 1 11 R B W W R W B R R W B R 1 2 11 B R W W R W B R R W B R 1 2 10 B R R W R W B R R W B W 1 3 10 B R R W R W B R R W B W

lt gt 1 3 9 B R R B R W B R R W W W 2 4 9 B B R R R W B R R W W W 2 5 9 B B R R R W B R R W W W A B C D P P P P P V W Y Z X 2 5 8 B B R R R W B R R W W W 2 5 7 B B R R R R B R W W W W

lo hi 2 6 7 B B R R R R B R W W W W 3 7 7 B B B R R R R R W W W W before v invariant 3 8 7 B B B R R R R R W W W W lo hi 3 8 7 B B B R R R R R W W W W during v 3-way partitioning trace (array contents after each loop iteration)

lt i gt

after v 43 44

lo lt gt hi

3-way partitioning 3-way quicksort: Java implementation 3-way quicksort: visual trace

private static void sort(Comparable[] a, int lo, int hi) { if (hi <= lo) return; int lt = lo, gt = hi; Comparable v = a[lo]; int i = lo; while (i <= gt) { int cmp = a[i].compareTo(v); if (cmp < 0) exch(a, lt++, i++); else if (cmp > 0) exch(a, i, gt--); equal to partitioning element else i++; } before v

sort(a, lo, lt - 1); lo hi

sort(a, gt + 1, hi); during v } lt i gt

after v Visual trace of quicksort with 3-way partitioning

lo lt gt hi

3-way partitioning 45 46

Duplicate keys: lower bound Sorting summary

Sorting lower bound. If there are n distinct keys and the ith one occurs xi times, any compare-based sorting algorithm must use at least inplace? stable? best average worst remarks n N! xi lg x lg N lg N when all distinct; x ! x ! x ! ⇤ i N selection ✔ ½ N 2 ½ N 2 ½ N 2 N exchanges 1 2 n i=1 linear when only a constant number of distinct keys ··· ⇥ ⇤ compares in the worst case. use for small N insertion ✔ ✔ N ¼ N 2 ½ N 2 or partially ordered

tight code; shell ✔ N log N ? c N 3/2 3 subquadratic

N log N guarantee; merge ✔ ½ N lg N N lg N N lg N Proposition. [Sedgewick-Bentley 1997] proportional to lower bound stable Quicksort with 3-way partitioning is entropy-optimal. improves mergesort timsort ✔ N N lg N N lg N when preexisting order Pf. [beyond scope of course] N log N probabilistic guarantee; quick ✔ N lg N 2 N ln N ½ N 2 fastest in practice

improves quicksort 3-way quick ✔ N 2 N ln N ½ N 2 Bottom line. Quicksort with 3-way partitioning reduces running time when duplicate keys from linearithmic to linear in broad class of applications. ? ✔ ✔ N N lg N N lg N holy sorting grail

47 48 Sorting applications

Sorting algorithms are essential in a broad variety of applications: ・Sort a list of names. ・Organize an MP3 library. obvious applications ・Display Google PageRank results. 2.3 QUICKSORT ・List RSS feed in reverse chronological order.

‣ quicksort ・Find the median. Identify statistical outliers. problems become easy once items ‣ selection ・ are in sorted order Binary search in a database. ‣ duplicate keys ・ Algorithms ・Find duplicates in a mailing list. ‣ system sorts ・Data compression. ROBERT SEDGEWICK | KEVIN WAYNE http://algs4.cs.princeton.edu ・Computer graphics. non-obvious applications ・Computational biology. ・Load balancing on a parallel computer. . . .

50

War story (system sort in C) War story (system sort in C)

A beautiful bug report. [Allan Wilks and Rick Becker, 1991] Bug. A qsort() call that should have taken seconds was taking minutes.

We found that qsort is unbearably slow on "organ-pipe" inputs like "01233210": Why is qsort() so slow? main (int argc, char**argv) { int n = atoi(argv[1]), i, x[100000]; for (i = 0; i < n; i++) x[i] = i; for ( ; i < 2*n; i++) x[i] = 2*n-i-1; qsort(x, 2*n, sizeof(int), intcmp); } At the time, almost all qsort() implementations based on those in: Here are the timings on our machine: Version 7 Unix (1979): quadratic time to sort organ-pipe arrays. $ time a.out 2000 ・ real 5.85s ・BSD Unix (1983): quadratic time to sort random arrays of 0s and 1s. $ time a.out 4000 real 21.64s $time a.out 8000 real 85.11s

51 52 Engineering a system sort (in 1993) A beautiful mailing list post (Yaroslavskiy, September 2009)

Bentley-McIlroy quicksort. samples 9 items Replacement of quicksort in java.util.Arrays with new dual-pivot quicksort ・Cutoff to insertion sort for small subarrays. ・Partitioning item: median of 3 or Tukey's ninther. Hello All, ・Partitioning scheme: Bentley-McIlroy 3-way partitioning. I'd like to share with you new Dual-Pivot Quicksort which is faster than the known implementations (theoretically and experimental). I'd like to propose to replace the JDK's Quicksort implementation by new one. similar to Dijkstra 3-way partitioning SOFTWARE—PRACTICE AND EXPERIENCE, VOL. 23(11), 1249–1265(but fewer (NOVEMBER exchanges 1993) when not many equal keys) ...

The new Dual-Pivot Quicksort uses *two* pivots elements in this manner: Engineering a Sort Function 1. Pick an elements P1, P2, called pivots from the array.

JON L. BENTLEY 2. Assume that P1 <= P2, otherwise swap it. M. DOUGLAS McILROY AT&T Bell Laboratories, 600 Mountain Avenue, Murray Hill, NJ 07974, U.S.A. 3. Reorder the array into three parts: those less than the smaller pivot, those larger than the larger pivot, and in between are those elements between (or equal to) the two pivots. SUMMARY 4. Recursively sort the sub-arrays. We recount the history of a new qsort function for a C library. Our function is clearer, faster and more robust than existing sorts. It chooses partitioning elements by a new sampling scheme; it partitions by a novel solution to Dijkstra’s Dutch National Flag problem; and it swaps efficiently. Its behavior was assessed with timing and debugging testbeds, and with a program to certify performance. The design The invariant of the Dual-Pivot Quicksort is: techniques apply in domains beyond sorting.

KEY WORDS Quicksort Sorting algorithms Performance tuning Algorithm design and implementation Testing [ < P1 | P1 <= & <= P2 } > P2 ]

INTRODUCTION C libraries have long included a qsort function to sort an array, usually implemented by ... Very widelyHoare’s used. Quicksort. C,1 BecauseC++, existing Javaqsort 6,s are…. flawed, we built a new one. This paper summarizes its evolution. Compared to existing library sorts, our new qsort is faster—typically about twice as http://mail.openjdk.java.net/pipermail/core-libs-dev/2009-September/002630.html fast—clearer, and more robust under nonrandom inputs. It uses some standard Quicksort 53 54 tricks, abandons others, and introduces some new tricks of its own. Our approach to build- ing a qsort is relevant to engineering other algorithms. The qsort on our home system, based on Scowen’s ‘Quickersort’,2 had served faith- fully since Lee McMahon wrote it almost two decades ago. Shipped with the landmark Sev- enth Edition Unix System,3 it became a model for other qsorts. Yet in the summer of 1991 our colleagues Allan Wilks and Rick Becker found that a qsort run that should have A beautifultaken mailing a few minutes list was chewingpost up (Yaroslavskiy-Bloch-Bentley, hours of CPU time. Had they not interrupted it, it October 2009) Dual-pivot quicksort would have gone on for weeks.4 They found that it took n 2 comparisons to sort an ‘organ- pipe’ array of 2n integers: 123..nn.. 321. Shopping around for a better qsort, we found that a qsort written at Berkeley in 1983 would consume quadratic time on arrays that contain a few elements repeated many times—in particular arrays of random zeros and ones.5 In fact, among a dozen different Use two partitioning items p1 and p2 and partition into three subarrays: ReplacementUnix librariesof quicksort we found no inqsort java.util.Arraysthat could not easily bewith driven new to quadratic dual-pivot behavior; all quicksort were derived from the Seventh Edition or from the 1983 Berkeley function. The Seventh ・Keys less than p1. Date: Thu, 29 Oct 2009 11:19:39 +0000 0038-0644/93/111249–17$13.50 Received 21 August 1992 ・Keys between p1 and p2. Subject: Replace 1993 by Johnquicksort Wiley & Sons, in Ltd. java.util.Arrays withRevised dual-pivot 10 May 1993 implementation ・Keys greater than p2. Changeset: b05abb410c52 Author: alanb Date: 2009-10-29 11:18 +0000 URL: http://hg.openjdk.java.net/jdk7/tl/jdk/rev/b05abb410c52 < p1 p1 ≥ p1 and ≤ p2 p2 > p2 6880672: Replace quicksort in java.util.Arrays with dual-pivot implementation Reviewed-by: jjb Contributed-by: vladimir.yaroslavskiy at sun.com, joshua.bloch at google.com, lo lt gt hi jbentley at avaya.com

! make/java/java/FILES_java.gmk ! src/share/classes/java/util/Arrays.java + src/share/classes/java/util/DualPivotQuicksort.java Recursively sort three subarrays.

http://mail.openjdk.java.net/pipermail/compiler-dev/2009-October.txt

degenerates to Dijkstra's 3-way partitioning

Note. Skip middle subarray if p1 = p2.

55 56 Dual-pivot partitioning demo Dual-pivot partitioning demo

Initialization. Main loop. Repeat until i and gt pointers cross. ・Choose a[lo] and a[hi] as partitioning items. ・If (a[i] < a[lo]), exchange a[i] with a[lt] and increment lt and i. ・Exchange if necessary to ensure a[lo] ≤ a[hi]. ・Else if (a[i] > a[hi]), exchange a[i] with a[gt] and decrement gt. ・Else, increment i.

< p1 p1 ≥ p1 and ≤ p2 ? p2 > p2

lo lt i gt hi

S E A Y R L F V Z Q T C M K K E A F R L M C Z Q T V Y S

lo hi lo lt i gt hi

exchange a[lo] and a[hi]

Dual-pivot partitioning demo Dual-pivot quicksort

Finalize. Use two partitioning items p1 and p2 and partition into three subarrays: ・Exchange a[lo] with a[--lt]. ・Keys less than p1. ・Exchange a[hi] with a[++gt]. ・Keys between p1 and p2. ・Keys greater than p2.

< p1 p1 ≥ p1 and ≤ p2 p2 > p2 < p1 p1 ≥ p1 and ≤ p2 p2 > p2

lo lt gt hi lo lt gt hi

C E A F K L M R Q S Z V Y T

lo lt gt hi Now widely used. Java 7, Python unstable sort, Android, …

3-way partitioned 60 Three-pivot quicksort Quicksort quiz 4

Use three partitioning items p1, p2, and p3 and partition into four subarrays: Why do 2-pivot (and 3-pivot) quicksort perform better than 1-pivot?

・Keys less than p1. A. Fewer compares. ・Keys between p1 and p2. B. Fewer exchanges. ・Keys between p2 and p3. ・Keys greater than p3. C. Fewer cache misses. D. I don't know. < p1 p1 ≥ p1 and ≤ p2 p2 ≥ p2 and ≤ p3 p3 > p3

lo a1 a2 a3 hi

Multi-Pivot Quicksort: Theory and Experiments

Shrinu Kushagra Alejandro L´opez-Ortiz J. Ian Munro [email protected] [email protected] [email protected] University of Waterloo University of Waterloo University of Waterloo Aurick Qiao [email protected] University of Waterloo November 7, 2013

Abstract In 2009, Vladimir Yaroslavskiy introduced a novel 61 62 The idea of multi-pivot quicksort has recently received dual-pivot partitioning algorithm. When run on a bat- the attention of researchers after Vladimir Yaroslavskiy tery of tests under the JVM, it outperformed the stan- proposed a dual pivot quicksort algorithm that, con- dard quicksort algorithm [10]. In the subsequent re- trary to prior intuition, outperforms standard quicksort lease of Java 7, the internal sorting algorithm was re- by a a significant margin under the Java JVM [10]. More placed by Yaroslavskiy’s variant. Three years later, recently, this algorithm has been analysed in terms of Wild and Nebel [9] published a rigorous average-case Quicksort comparisonsquiz 4 and swaps by Wild and Nebel [9]. Our con- analysis of the algorithm. They stated that the previ- Which sorting algorithm to use? tributions to the topic are as follows. First, we perform ous lower bound relied on assumptions that no longer the previous experiments using a native C implementa- hold in Yaroslavskiy’s implementation. The dual pivot tion thus removing potential extraneous e↵ects of the approach actually uses less comparisons (1.9n ln n vs JVM. Second, we provide analyses on cache behavior 2.0n ln n) on average. However, the di↵erence in run- Why do 2-pivotof these algorithms.(and We3-pivot) then provide strong quicksort evidence time isperform much greater than better the di↵erence inthan number of1-pivot? Many sorting algorithms to choose from: that cache behavior is causing most of the performance comparisons. We address this issue and provide an ex- planation in 5. di↵erences in these algorithms. Additionally, we build § upon prior work in multi-pivot quicksort and propose Aum¨uller and Dietzfelbinger [1] (ICALP2013) have A. Fewera 3-pivot compares. variant that performs very well in theory and recently addressed the following question: If the previ- practice. We show that it makes fewer comparisons and ous lower bound does not hold, what is really the best has better cache behavior than the dual pivot quicksort we can do with two pivots? They prove a 1.8n ln n lower in the expected case. We validate this with experimen-# entriesbound scanned on the number is ofa comparisonsgood proxy for all dual-pivot sorts algorithms B. Fewer exchanges. quicksort algorithms and introduced an algorithm that tal results, showing a 7-8% performance improvementfor cache performance when in our tests. actually achieves that bound. In their experimentation, comparingthe algorithm quicksort is outperformed variants by Yaroslavskiy’s quick- 1 Introduction sort when sorting integer data. However, their algo- C. Fewer cache misses. rithm does perform better with large data (eg. strings) elementary sorts insertion sort, selection sort, bubblesort, shaker sort, ... 1.1 Background Up until about a decade ago it was since comparisons incur high cost. thought that the classic quicksort algorithm [3] using one pivot is superior to any multi-pivot scheme. It was 1.2 The Processor-Memory Performance Gap previously believed that using more pivots introduces Both presently and historically, the performance of CPU quicksort, mergesort, heapsort, shellsort, samplesort, ... too much overhead for the advantages gained. In 2002, subquadratic sorts registers have far outpaced that of main memory. Ad- Sedgewickpartitioning and Bentley [7] recognised andcompares outlined some exchanges entries scanned

Downloaded 02/07/14 to 96.248.80.75. Redistribution subject SIAM license or copyright; see http://www.siam.org/journals/ojsa.php ditionally, this performance gap between the processor of the advantages to a dual-pivot quicksort. However, and memory has been increasing since their introduc- the implementation did not perform as well as the classic tion. Every year, the performance of memory improves quicksort algorithm [9] and this path was not explored system sorts dual-pivot quicksort, timsort, introsort, ... by about 10% while that of the processor improves by again until recent years. 1-pivot 2 N ln N 60% [5]. The performance0.333 N di ↵lnerence N grows so quickly 2 N ln N

external sorts Poly-phase mergesort, cascade-merge, psort, .… 47 Copyright © 2014. median-of-3 1.714 N ln N by the Society0.343 for Industrial N ln andN Applied Mathematics.1.714 N ln N

MSD, LSD, 3-way radix quicksort, … 2-pivot 1.9 N ln N 0.6 N ln N 1.6 N ln N radix sorts

3-pivot 1.846 N ln N 0.616 N ln N 1.385 N ln N parallel sorts bitonic sort, odd-even sort, smooth sort, GPUsort, ...

Reference: Analysis of Pivot Sampling in Dual-Pivot Quicksort by Wild-Nebel-Martínez

Bottom line. Caching can have a significant impact on performance.

beyond scope of this course 63 64 Which sorting algorithm to use? System sort in Java 7

Applications have diverse attributes. Arrays.sort(). ・Stable? ・Has method for objects that are Comparable. ・Parallel? ・Has overloaded method for each primitive type. ・In-place? ・Has overloaded method for use with a Comparator. ・Deterministic? ・Has overloaded methods for sorting subarrays. ・Duplicate keys? ・Multiple key types? Algorithms. ・Linked list or arrays? ・Dual-pivot quicksort for primitive types. Large or small items? Timsort for reference types. ・ many more combinations of ・ ・Randomly-ordered array? attributes than algorithms ・Guaranteed performance? Q. Why use different algorithms for primitive and reference types?

Q. Is the system sort good enough? A. Usually.

65 66

Ineffective sorts

http://xkcd.com/1185 67