Sorting Networks

Uri Zwick Tel Aviv University

May 2015 Comparators

푥 min(푥, 푦) 푦 max(푥, 푦)

Can we use comparators to build efficient sorting networks? Comparator networks

A comparator network is a network composed of comparators. There are 푛 input wires, each feeding a single comparator. Each output of a comparator is either an output wire, or feeds a single comparator. The network must be acyclic. Number of output wires must also be 푛. “Standard form”

Exercise: Show that any comparator network is equivalent to a network in standard form. A simple sorting network

5 comparators 3 levels Insertion sort

푆표푟푡(푛) Selection/bubble sort

푆표푟푡(푛) Selection/bubble Sort

푛 푛−1 Size = Depth = 2푛 − 1 2 Exercise: Any sorting network that only compares 푛 푛−1 adjacent lines must be of size at least 2 Exercise: Prove that the network on the next slide, whose depth is 푛, is a sorting network Odd-Even Transposition Sort

푛 푛−1 Size = Depth = 푛 2 The 0-1 principle

Theorem: If a network sort all 0-1 inputs, then it sort all inputs The 0-1 principle

Lemma: Let 푓 be a monotone non-decreasing function. Then, if a network maps 푥1, 푥2, . . , 푥푛 to 푦1, 푦2, . . , 푦푛, then it maps 푓(푥1), 푓(푥2), . . , 푓(푥푛) to 푓(푦1), 푓(푦2), . . , 푓(푦푛) Proof: By induction on the number of comparisons using 푓 min 푎, 푏 = min(푓 푎 , 푓 푏 ) 푓 max 푎, 푏 = max(푓 푎 , 푓 푏 ) The 0-1 principle

Proof: Suppose that a network is not a sorting network.

It then maps some 푥1, 푥2, . . , 푥푛 to 푦1, 푦2, . . , 푦푛, where 푦푖 > 푦푖+1, for some 1 ≤ 푖 < 푛.

Let 푓 푥 = 1, iff 푥 ≥ 푦푖, 0 otherwise.

The network maps 푓(푥1), 푓(푥2), . . , 푓(푥푛) to 푓(푦1), … , 푓(푦푖) = 1, 푓 푦푖+1 = 0 , . . . , 푓(푦푛) Thus, the network does not sort all 0-1 inputs. Sorting by merging

푆표푟푡(푛)

푀푒푟푔푒(푛, 푚)

푆표푟푡(푚) Batcher’s odd-even merge

푛 푛 푚 푛 푚 푀( , ) 푀( , ) 2 2 2 2

푚 Batcher’s odd-even merge

푛 푛 푚 푛 푚 푀( , ) 푀( , ) 2 2 2 2

푚 Batcher’s odd-even merge

To merge 푎1, 푎2, . . , 푎푛 with 푏1, 푏2, . . , 푏푚:

Split 푎1, 푎2, . . , 푎푛 into 푎1, 푎3, … and 푎2, 푎4, …

Split 푏1, 푏2, . . , 푏푚 into 푏1, 푏3, … and 푏2, 푏4, …

Merge odd-indexed items to create 표1, 표2, …

Merge even-indexed items to create 푒1, 푒2, …

Compare/swap (푒1, 표2), (푒2, 표3), …

Does it really work? Batcher’s odd-even merge

푀(1,1)

푀(2,2) Batcher’s odd-even merge - 푀(4,4) Batcher’s odd-even merge - 푀(4,4) Odd-even merge → Odd-even sort Batcher’s odd-even merge Proof using the 0-1 principle.

Suppose that 푎1, 푎2, . . , 푎푛 starts with 푛0 0’s and that 푏1, 푏2, . . , 푏푚 starts with 푚0 0’s. 푛 푚 Then 표 , 표 , … starts with 0 + 0 0’s, 1 2 2 2 푛 푚 and 푒 , 푒 , … starts 0 + 0 0’s. 1 2 2 2 The difference is either 0,1 or 2! There is a problem only if difference is 2 and last level of comparators fixes it. Batcher’s odd-even merge

푒 0 0 0 0 0 1 1 1 표 0 0 0 0 0 1 1 1

푒 0 0 0 0 1 1 1 1 표 0 0 0 0 0 1 1 1

푒 0 0 0 1 1 1 1 1 표 0 0 0 0 0 1 1 1 Batcher’s odd-even merge

Exercise: Justify the use of the 0-1 principle. Exercise: Prove the correctness of the odd-even merging network directly without relying on the 0-1 principle. Batcher’s odd-even merge Size – number of comparators

푀 푛, 0 = 푀 0, 푚 = 0 푀 1,1 = 1 푛 푚 푛 푚 푛 + 푚 − 1 푀 푛, 푚 = 푀 , + 푀 , + 2 2 2 2 2 푛 푛 푀 푛, 푛 = 2푀 , + (푛 − 1) 2 2 푀 2푘, 2푘 = 푘2푘 + 1

푀 푛, 푛 = 푛 lg 푛 + 푂(푛) No better merging networks are known for any 푛, 푚! Are the odd-even merge networks optimal? Bitonic sequences A sequence is strict bitonic iff it is a concatenation of a decreasing sequence and an increasing sequence A sequence is bitonic iff it is a cyclic shift of a strict bitonic sequence 8 6 4 1 2 5 7 9 4 1 2 5 7 9 8 6 1 4 2 3 A Bitonic sorter

A (strict) bitonic sorter is a network that sorts every (strict) bitonic sequence.

If 푎1, 푎2, . . , 푎푛 and 푏1, 푏2, . . , 푏푚 are sorted, then 푎푛, 푎푛−1, . . , 푎1, 푏1, 푏2, . . , 푏푚, is bitonic.

Thus, a strict bitonic sorter can serve as a merging network Batcher’s bitonic sorter

푛 푛 퐵 푛 퐵 2 2

Is this different from odd-even merge? Batcher’s bitonic sorter

Simple proof using the 0-1 principle. A strict binary bitonic sequence – 1푘0ℓ1푛−푘−ℓ The odd and even subsequences are also strict binary bitonic sequences. By induction they are sorted correctly. The difference between the number of 0’s in the two sorted sequences is one of −1,0,1. Final level of comparators fixes the problem. Batcher’s bitonic sorter

Exercise: Show that the difference between the number of 0’s in the odd and even subsequences of 1푘0ℓ1푛−푘−ℓ is 푘 + ℓ 푘 + ℓ 푘 푘 − − − 2 2 2 2

Note: We do not need this exact formula in the proof given in the previous slide. Seeing that the difference is −1,0,1 is immediate. Batcher’s bitonic sorter for n=2푘

Very regular structure!

When 푛 = 2푘, there are 푘 levels with 푛 comparators each. 2 Lines 푖 and 푗 are compared at level ℓ iff they differ only in the ℓ-th most significant bit Batcher’s bitonic sorter for n=2푘

Alternative recursive definition for 푛 = 2푘

Sorts general bitonic sequences The AKS sorting networks [Ajtai-Komlós-Szemerédi (1983)]

There are sorting networks of 푂(log 푛) depth, and hence 푂(푛 log 푛) size. The construction is fairly complicated. The constant factors are very large.