h ec G T NL and csusthk g ON Science golin a of c aterlo o y W ersit rudolf f GEG A aterlo o uwaterloo tD W g Email of Kong Univ y imunro Hong ersit Kong The Univ y HKUST gran Hong eddemaine Science Science wlo on Ko tmchan y HKUST Theoretical Center Research Report HKUST-TCSC-2001-03 Ba rt of Computer Computer the t and tain biedl of of Most f para ater of rt t t h h W y as partially supp orted b ts increas a Shellso re n main rt tests an basic arra whic whic log Email her w Clear  notion Munro rt elemen they n on Bubbleso only few Demaine n Departmen Departmen page O y h a en the p erforms from one p oin GuessSo t of view Ian ev D Canada nology R Fleisc unsorted on rt eg  for sorts whic J time algorithms DumbSo n that But rt rt O Erik cells sortedness sorts term based y h Shellso y time initially merging sorting of be t far arra an to hange b etter Golin and us to approac rom another p oin InsertionSo a exc in F arian inplace v Chan of is runs in exp ected h reasonable this in t rt NotSoDumbSo y rt The y in to cycles or in time made th Abstract searc unSort y arian greedy selection n v F n Mordecai ork ys of increasingly greater length as the pro cess Shellso O w p ossible pairs of ebruary arra densit Timoth F DumbSo algorithms tend n to binary taining an all study ts of y progress the time y e log inplace b exceptions w  hanging of or her main n the greater t an sorting y exc happ en ersion of O b rt pap er Biedl ts on notable cycles Fleisc insertions erhaps t cells also sorts in segmen this unSo P are tains sorted subarra F arian op erate In v ed sorted segmen n insertion tro duction rt v ues longer p eated randomized v And sorts in time adjacen rep eatedly compares in miraculously Rudolf Therese In tin terlea con view main metho ds in ingly Quickso strong in Comparison based digms

is delightfully chaotic Atany stage one is presented with a subregion which

one hop es is reasonably close to sorted order and then is to complete the

task by using linear InsertionSort Magicallyitworks pretty well

Our starting point is the notion of sorting by rep eatedly moving values

from current to more likely lo cations by p erforming binary searches If an

array were in sorted order except for the lo cation of one value then this

approach would minimize b oth the number of comparisons and the number

of moves necessary to complete the task The approachwould certainly place

the set maximum and minimum in their prop er sp ots and never movethem

again If the array is close to b eing in sorted order the pro cess will generally

make further progress Our question is whether the general approach of

binary insertion into an initially unordered array can be harnassed to yield

a viable

We are able to adapt the metho d to correctly p erform a sort and b ound

the worstcase runtime reasonably accurately However we feel the greater

contribution of this pap er is to lay op en the approach b oth for purp oses

of algorithmic improvements and for improvements in the exp ected runtime

of our approach In particular we observe that the exp ected p osition of an

element binary inserted into an array is approximately its rank

Several variations on this theme are explored In a more greedy approach

we move values to more likely lo cations by comparing all pairs of array

cells and swapping their contents if they are not in the right order We show

that n iterations or cycles of this bruteforce approach suce to sort any

input indep endent of the order of the comparisons in a cycle Interestingly



it turns out that not all n pairs have to b e considered in each cycle It is

sucient to have n cycles that only compare the n pairs of adjacent

cells in any arbitrary order Bubblesort is a sp ecial case of such an algorithm

Previously it was only known that n cycles will sort correctly

Our results are also of interest b ecause all of our sorting algorithms are

purely inplace And with the exception of the unordered binary searches

algorithm they are also oblivious ie they could be realized by a sorting

network But in contrast to other work on sorting networks we are not

interested in exploiting the parallelism of our sequential algorithms to reduce

the numb er of rounds or the depth of the network for example

This pap er is organized as follows In Section weintro duce our mo del of

inversionxing algorithms and show that all inversionxing algorithms will

eventually terminate In Section we give a simple pro of based on the

Principle that DumbSort which p erforms n cycles of comparisons of all

p ossible cell pairs sorts correctly We think that this is a more natural n

time slowsort algorithm than Julstroms Slow Sort which is somehow

articially bloated We then show that NotSoDumbSort a streamlined

version of DumbSort which only considers adjacent cells also sorts correctly



in n cycles reducing the runtime to O n

In Section we intro duce GuessSort a randomized variant of Dumb



Sort which sorts in exp ected time O n log n In Section we show that

FunSort whichisInsertionSort with binarysearch insertions into an initially



unordered array sorts in time O n log n And we conclude with a few op en

problems in Section

To Swap or not to Swap Thats our Question

Assume we are given n elements from a totally ordered set stored in an array

of size n which we think of b eing arranged from left to right in increasing

order For simplicity we assume that all elements are dierent but the

results in this pap er also hold in the general case At any time we denote

the content of cell i by a for i n Our goal is to sort these elements

i

in nondecreasing order Intuitively the simplest way to sort an array is to

concentrate on just two elements at a time from which we can extract only

one bit of information A pair i j of cells i and j where i j isgood if the

cell contents are correctly ordered ie a a if i j or a a if i j

i j i j

Otherwise the pair is bad Bad pairs are also called inversions The number

of inversions in the array can be between zero if the array is correctly sorted

n

and if the array is sorted in reverse order



Following these ideas the most basic metho d of sorting is to search for and

swap bad pairs De Bruijns called the action of comparing the contents

of a pair of cells and leaving the smaller value in the left cell ie the cell

with lower index a swapora miniswap if the two cells are adjacent Note

that swapping a go o d pair has no eect whereas in a swap of a bad pair an

exchange swapthe cell contents are exchanged

The following simple observation seems to be folklore but we could not

nd it stated prop erly in the literature although it is used in It implies

that we can sort by rep eatedly doing exchange swaps

Lemma An exchange swap strictly decreases the number of inversions

Proof Consider an exchange swap i j where ij and a a see Fig

i j

Let k be a good pair b efore the swap where k Since the content

of cell i is decreasing and the content of cell j is increasing k can only

b ecome a bad pair if k j or i

i m j k

Figure If weswap the bad pair i j then the go o d pair k b ecomes a bad

pair However the bad pair i b ecomes a go o d pair We also accidentally turn

the bad pairs i mandm j into go o d pairs but that is ok

In the former case i was a bad pair b efore the swap b ecause a moves

i

to cell j but it is good afterwards and in the latter case k j was a bad

pair b efore the swap b ecause a moves to cell i but it is go o d afterwards

j

In any case the numb er of bad pairs decreases b ecause i j becomesagood

pair

We only consider sorting algorithms based on swaps called exchange sort

ing algorithms byKnuth page We prefer to call them inversionxing

algorithms Knuth also calls sorting algorithms purely based on miniswaps

primitive Exercise on page Lemma implies that inversion

xing algorithms cannot makean unb ounded number of exchange swaps

n



Corollary Any inversionxing algorithm makes at most O n



exchange swaps

n

Proof Initially the array can have at most inversions and each exchange



swap strictly decreases the number of inversions by Lemma



So if we knew the indices of the bad pairs we could easily sort in O n

time Unfortunately we cannot exp ect to get this information for free In

the next three sections we will show several simple but nevertheless correct

and even semiecient ways to nd bad pairs We only consider oblivious

algorithms where the sequence of comparisons is xed in advance In par

ticular it is indep endent of the input sequence or the outcome of individual

comparisons That makes the algorithms similar to sorting networks except

that we do not try to exploit any parallelism

DumbSort Test All Hit One

A not to o clever brute force way of nding an exchange swap is to try all

swaps This approach is similar to the BellmanFord algorithm for computing

shortest paths in a weighted graph where in eachstepwe know that there is

one no de of optimal value which we should relax next but since we cannot

identify that no de we simply relax all no des Section therefore the

bad runtime of that algorithm

n

A cycle is a sequence of all swaps in some arbitrary order the swap



sequence A minicycle is a sequence of all n miniswaps in some arbitrary

order the miniswap sequence Note that no miniswap sequence can sort

the input sequence On the other hand there is aswap sequence that

sorts all sequences namely the swap sequence

n n

However not all swap sequences can sort any input sequence For example

the sequence will not b e sorted by the swap sequence

the result would be

Since anyswap sequence contains at least one exchange swap if the input

n

is not sorted cycles of arbitrary swap sequences suce to sort any input





by Corollary resulting in a runtime of O n The same is true for mini

cycles resulting in a runtime of O n But this is overly p essimistic We will

show that any input sequence must already b e sorted after n minicycles

We use the generic name DumbSort for any algorithm which consists of

n cycles of p ossibly dierent swap sequences Similarly NotSoDumb

Sort denotes any algorithm doing n arbitrary minicycles Since we can

view the sequence of comparisons made by DumbSort or NotSoDumbSort

as an oblivious sorting network we can use the wellknown Principle

see Theorem Z on page for example to prove the correctness of the

algorithm

Theorem Principle If an oblivious sorting network sorts al l se

quences of s and s then it also sorts al l sequences of any numbers

Theorem DumbSort sorts n elements in n cycles or time O n

Proof Assume the input is a sequence of s and s Note that a can

only move to the left and a can only move to the right In particular

if the leftmost rightmost in the array is replaced by a the

content of this cell can never change again ie the cell contains the nal

correctly sorted numb er

If the input consists of n s or n s the theorem is clearly true Oth

erwise assume that at the b eginning of a cycle cell i contains the leftmost

i j

Figure i is the leftmost cell and j is the rightmost cell

and cell j the rightmost see Fig If the array is unsorted wehave

ij During the cycle none of the s to the left of cell i or s to the right

of cell j are swapp ed Also once a is placed in cell i or a in cell j it

is never moved out again Consider the p ointinthe cycle at which we swap

i j If a and a then they are exchange swapp ed Thus by the

i j

end of the cycle either a a or b oth

i j

The following example shows that the b ound of Theorem is tight As

sume the largest element is initially stored in cell and the swap sequence

ends n Then the largest element only pro ceeds to

cell after the rst cycle Iterating this construction we obtain a swap

sequence that can move the largest element in any cell only one cell to the

right so we need n of these cycles to move it from cell to cell n

The pro of of Theorem actually suggests that minicycles instead of

full cycles might sometimes be sucient We can for example similarly

prove that the miniswap sequence n n or the reverse

sequence sorts in n minicycles the rightmost left of the rightmost

will replace that in any minicycle And the runtime of this algorithm

which an attentive reader might have recognized as Bubblesort p



is only O n Since any oblivious sorting network must contain all mini

swaps Exercise on page Bubblesort is minimal among all

sorting algorithms based on iterating cycles Our pro of also shows that we

could arbitrarily augment the Bubblesort swap sequence by other swaps and

still sort in n cycles in general adding comparisons to an oblivious

sorting network mayprevent the network from sorting correctly Exercise

on page However this result is not new De Bruijn actually

showed the following stronger theorem Theorem

Theorem Any sequence of miniswaps that sorts correctly can safely be

augmented by arbitrary swaps

Some authors Fact Prop also attribute the following the

orem to de Bruijn even though his pap er do es not mention it However

it is an easy corollary of Theorem



Theorem Any sequenceof n minicycles sorts n elements in time O n

Proof It is known that n phases of the oddeven transposition sorting network

sort n numb ers Exercise on page Since the swap sequence

of the ith minicycle contains the ith phase of Transp osition Sort either

or as a subsequence Theorem implies that

the n minicycles sort correctly

Since no primitive sorting network like Transp osition Sort can have less

n

than swaps otherwise n n could not b e sorted see Exercise



on page n phases of Transp osition Sort do not suce to sort

correctly so this pro of cannot be adapted to prove that NotSoDumbSort

which p erforms only n minicycles sorts correctly We also note that de

Bruijn proved Theorem by the not universally accepted metho d of pro of

by convincing example so purists might ob ject that the pro of of Theorem

actually still contains a gap To put things straightwe give a more direct

pro of which how nice even shows a stronger statement

Theorem NotSoDumbSort sorts n elements in n minicycles or



time O n

Proof We make again use of the Principle Assume the input sequence

contains k s and n k s for some k n We denote the s from

left to rightby T T T We proveby induction on the minicycles that

  k

after the ith minicycle T has reached at least cell minfi j k n j k g

j

for j k Thus T reaches its nal destination n j k after at most

j

n j n minicycles

Since j k j for j k the claim is true initially ie for i

Now consider the ith minicycle for some i We use induction on j to

prove the claim Since T is the rightmost it will move at least one cell to

k

the right during the minicycle if it has not yet reached its nal cell n This

proves the inductive step for T

k

Now consider T for some j k By the inductive hyp othesis T was

j j 

at least in cell minfi j k n j k g minfi j

k n j k g b efore the ith minicycle If T was at that time in cell

j

minfi j k n j k g or further to the right then the claim is proved

Otherwise it must have b een in cell minfi j k n j k g by the

inductive hyp othesis If i j k n j k then it started the

ith minicycle left of a and therefore moved at least one cell to the right

during the minicycle which proves the claim

We note that the worst case in the pro of ab ove o ccurs if the input is

and then the b ound can be tight These inputs are the

maximal elements of de Bruijns partial order on input p ermutations see

for details and we implicitly proved an analogon to his Theorem which

states that it is sucient to prove correctness for maximal elements of the

partial order in his case there is exactly one maximum element

In view of Theorem Theorem is actually a corollary of Theorem

b ecause each full cycle contains a miniswap subsequence But we think our

simple direct pro of has its own merits

GuessSort Even a Blind Pig Finds an Acorn

As argued b efore we cannot build cycles from swap sequences with less than

n swaps so there is no hop e of getting a dumb or greedy O n log n

size sorting network like the AKS sorting network But there is a simple

randomized variant of DumbSort which do es better on the average Guess

Sort cho oses random pairs of cells to swap until the array is sorted



Theorem GuessSort sorts n elements in expected time O n log n

m

Proof If there are m bad pairs in the arraywe hit one with probability

n

Since the number of bad pairs decreases by Lemma after each exchange

swap the exp ected number T of swaps until the array is sorted is b ounded

n

by

n



X

n



A

T O O n log n

n

m

m

There is just a minor problem with this algorithm Howdoweknowwhen

we can stop ie when the array is sorted One solution is to check after

every n tests whether the elements are already sorted correctly This can

be done with n comparisons so we at most double the runtime

The b ound of the theorem is tight GuessSort needs exp ected time



n log n to sort the sequence nn This is an applica

tion of the coup on collectors problem page We need n steps to

guess a pair of adjacent cells and this must happ en n log n times until

we have found all the pairs Note that it do es not help us

to cho ose only random adjacent pairs we nd bad pairs faster but we make

less progress by swapping them than by swapping some farapart bad pair

FunSort Smart Search in Bad Arrays

Another way of nding bad pairs is by binary search While we are nor

mally taught to apply binary search only when an array is sorted we could

reb elingly apply the same pro cedure to an unsorted array To avoid sp ecial

treatment of the array b oundaries we assume a and a We

 n

keep two indices and h such that at any time a x a Initially

h

h

and h n In each step we compare x to a where m b cand set

m



h m if x a or m if xa We stop if h and return the

m m

index h as the result of the search Wesay the searchwas successful if x a

h

note that x a is imp ossible otherwise it failed Note that a search for x

l

can fail even though x is contained in the array

FunSort is an inplace variant of InsertionSort It p erforms rep eated

insertions by binary search into an initially unsorted array and surprisingly



sorts correctly after only O n of these unordered insertions

FunSort runs in n phases In phase i for i n we rep eatedly

binary search for the current content a of cell i until the search is successful

i

and nds the element in cell i If the search fails we nd an index h such

that a a a We then either swap the bad pair i h if ih

h i h

or the bad pair h i if h i The phase continues with a search for the

new content of cell i Note that FunSort is not an inversionxing algorithm

in the strict sense b ecause we do not repair wrongly ordered pairs during a

binary search but a failed search is always followed by an exchange swap



Theorem FunSort sorts n elements in time O n log n

Proof Weshowby induction that after phase i for i n the elements

stored in cells i are ordered correctly although they are not necessarily

the smallest i elements

The claim is clearly true for i If in phase i a search fails and

returns index h and h i then we swap h i But a a a so this

h i h

do es not destroy the sortedness of the rst i cells And if the search for

a is successful we have a a so the rst i cells are sorted at the end of

i i i

phase i

Each binary search takes time O log n There are n successful searches

n

one p er phase And there are at most unsuccessful searches by Lemma





Thus the total time is O n log n

The following example shows that the b ound of Theorem is tight As

sume n k for some k and the initial sequence is k k

k k kk In the rst phase we rst try to lo

cate k The binary search ends with h n so we swap n

Then cell contains k and we search for k The search ends with

h n etc After k unsuccessful searches the sequence of numbers is

k k k k k and the phase ends with a

successful search for Analogously the next phases will have k k

unsuccessful searches

Note that we actually have some lib erty in the denition of binary search

regarding how to break ties If x a we continue the search in the left

m

half but we could also continue in the right half or even stop immediately

and return index m These three variants always pro duce the same result

on sorted arrays but not on unsorted arrays FunSort works ne with all

variants just the correctness pro of b ecomes slightly more involved for the

other two variants

Conclusion

Quite a few interesting questions still wait for answers

There is one swap sequence that can sort in one cycle whereas arbitrary

swap sequences may need up to n cycles Is there a characterization

of swap sequences that need at most k cycles for k n And

which minicycle needs the fewest rep etitions to sort any input

Is there an inversionxing algorithm in the strict sense based on un

sorted binary searches In particular how ecient is the following

algorithm if it works at all binary search on a xed search path and

swap elements which are in the wrong order ie on each level of the

path havelower b ound searchkey and upp er b ound ordered suchthat

the binary search follows the xed path

Is it true that FunSort searches at most n times for each element of the



sequence This would also imply the O n log n runtime of FunSort

We implemented unordered binary search and observed the at the rst

view somewhat surprising fact that binary searches for i n

in a random of n end on the average very close to

cell i ie the correct lo cation for element i it seems to end exactly

k

at i if n ie if the binary search always divides the search

interval into equalsize halves We only have the following quick dirty

and wrong pro of but this can sometimes be a reasonably convincing

approach as for example demonstrated in Let Posi n denote

the exp ected index found by a binary search for i In the rst step of

ni

the binary search we go left with probability and we go rightwith

n

i i

probability In each half of the input i has exp ected rank Thus

n 

and this thus is really not correct even though it lo oks convincing

at the rst glance

i n i i n n i n i n i

Pos Pos Pos Posi n

n n

Since Posn we have Posi n i

We ask Where do es the binary search for i end on the average and

what would b e a correct pro of

Acknowledgements

We thank Otfried Cheong for providing a slightly smaller example of a swap

sequence that cannot sort all inputs We also thank the unknown referees

whose valuable comments will undoubtedly b enet this pap er And last not

least we thank the Second Cup Coee Shop at the University of Waterlo o

for providing a stimulating problem solving environment over the past year

References

M Ajtai J Komlos and E Szemeredi Sorting in c log n parallel steps

Combinatorica

A preliminary version was published with a dierent title An O n log n

sorting network In Proceedings of the th ACM Symposium on Theory

of Computation STOC pages

E A Bender and S G Williamson Perio dic sorting using minimum

delay recursively constructed merging networks The Electronic Journal

of Combinatorics R

N G de Bruijn Sorting by means of swapping Discrete Mathematics

R Fleischer and P Sanders Asymptotic complexity from exp eriments

A case study for randomized algorithms In Proceedings of the th Work

shop of Algorithms and Engineering WAE To app ear A full

version of this pap er will b e submitted as book chapter

B A Julstrom Slow sorting A whimsical inquiry ACM SIGCSE

Bul letin

M Kik Ecient comparator networks PhD thesis Institute of Com

puter Science University of Wro claw Poland

M Kik M Kutylowski and G Stachowiak Perio dic constant depth

sorting networks In Proceedings of the th Annual Symposium on the

Theoretical Aspects of Computer ScienceSTACS Springer Lecture

Notes in Computer Science pages

D E Knuth The Art of Computer Programming Volume Sorting

and Searching AddisonWesley Reading MA edition

C McGeogh P Sanders R Fleischer P R Cohen and D Precup

Searching for BigOh in the data Inferring asymptotic complexity from

exp eriments In preparation

K Mehlhorn and S Naher LEDA A Platform for Combinatorial and

Geometric Computing Cambridge University Press Cambridge Eng

land

R Motwani and P Raghavan Randomized algorithms Cambridge Uni

versity Press Cambridge England

M Kutylowski K Lorys B Oesterdiekho and R Wanka Perio d

ication scheme Constructing sorting networks with constant p erio d

Journal of the ACM

Y Rabani A Sinclair and R Wanka Lo cal divergence of Markov

chains and the analysis of iterative loadbalancing schemes In Pro

ceedings of the th Symposium on Foundations of Computer Science

FOCSpages