I. Sorting Networks Thomas Sauerwald

Easter 2015 Outline

Outline of this Course

Introduction to Sorting Networks

Counting Networks

I. Sorting Networks Outline of this Course 2 Closely follow the book and use the same numberring of theorems/lemmas etc.

I. Sorting Networks (Sorting, Counting, Load Balancing) II. Matrix Multiplication (Serial and Parallel) III. Linear Programming (Formulating, Applying and Solving) IV. Approximation Algorithms: Covering Problems V. Approximation Algorithms via Exact Algorithms VI. Approximation Algorithms: Travelling Salesman Problem VII. Approximation Algorithms: Randomisation and Rounding VIII. Approximation Algorithms: MAX-CUT Problem

(Tentative) List of Topics

Algorithms (I, II) Complexity Theory Advanced Algorithms

I. Sorting Networks Outline of this Course 3 Closely follow the book and use the same numberring of theorems/lemmas etc.

(Tentative) List of Topics

Algorithms (I, II) Complexity Theory Advanced Algorithms

I. Sorting Networks Outline of this Course 3 (Tentative) List of Topics

Algorithms (I, II) Complexity Theory Advanced Algorithms

Closely follow the book and use the same numberring of theorems/lemmas etc.

I. Sorting Networks Outline of this Course 3 (Tentative) List of Topics

Algorithms (I, II) Complexity Theory Advanced Algorithms

Closely follow the book and use the same numberring of theorems/lemmas etc.

I. Sorting Networks Outline of this Course 3 Outline

Outline of this Course

Introduction to Sorting Networks

Batcher’s Sorting Network

Counting Networks

I. Sorting Networks Introduction to Sorting Networks 2 Allows to sort n numbers Comparisons can be performed in parallel in sublinear time!

Simple concept, but surprisingly deep and complex theory!

Sorting Networks only perform comparisons can only handle inputs of a ﬁxed size sequence of comparisons is set in advance

Overview: Sorting Networks

(Serial) Sorting Algorithms we already know several (comparison-based) sorting algorithms: Insertion sort, Bubble sort, Merge sort, Quick sort, Heap sort execute one operation at a time can handle arbitrarily large inputs sequence of comparisons is not set in advance

I. Sorting Networks Introduction to Sorting Networks 3 Allows to sort n numbers Comparisons can be performed in parallel in sublinear time!

Simple concept, but surprisingly deep and complex theory!

Overview: Sorting Networks

Sorting Networks only perform comparisons can only handle inputs of a ﬁxed size sequence of comparisons is set in advance

I. Sorting Networks Introduction to Sorting Networks 3 Simple concept, but surprisingly deep and complex theory!

Overview: Sorting Networks

Sorting Networks only perform comparisons can only handle inputs of a ﬁxed size sequence of comparisons is set in advance Allows to sort n numbers Comparisons can be performed in parallel in sublinear time!

I. Sorting Networks Introduction to Sorting Networks 3 Overview: Sorting Networks

Simple concept, but surprisingly deep and complex theory!

I. Sorting Networks Introduction to Sorting Networks 3 A sorting network is a comparison network which works correctly (that is, it sorts every input)

comparator is a device with, on given two inputs, x and y, returns two operates in O(1) outputs x0 and y 0 wire connect output of one comparator to the input of another special wires: n input wires a1, a2,..., an and n output wires b1, b2,..., bn

Convention:27.1 Comparison use networks the same name for both a wire and its value. 705

7 3 x x! min(x, y) x x ! min(x, y) comparator = = 3 7 y y! max(x, y) y y! max(x, y) = = (a) (b)

Figure 27.1 (a) Acomparatorwithinputsx and y and outputs x! and y!. (b) The same comparator, drawn as a single vertical line. Inputs x 7, y 3andoutputsx 3, y 7areshown. = = ! = ! =

Acomparisonnetworkiscomposedsolelyofwiresandcomparators. A comparator,showninFigure27.1(a),isadevicewithtwoinputs,x and y,andtwooutputs, x! and y!,thatperformsthefollowingfunction:

x! min(x, y), = y! max(x, y). = Because the pictorial representation of a comparator in Figure 27.1(a) is too bulky for our purposes, we shall adopt the convention of drawing comparators as single vertical lines, as shown in Figure 27.1(b). Inputs appear on the left and outputs on the right, with the smaller input value appearing on the top output and the larger input value appearing on the bottom output. We can thus think of a comparator as sorting its two inputs. We shall assume that each comparator operates in O(1) time. In other words, we assume that the time between the appearance of the input values x and y and the production of the output values x! and y! is a constant. A wire transmits a value from place to place. Wires can connect the output of one comparator to the input of another, but otherwise they are either network input wires or network output wires. Throughout this chapter, we shall assume that a comparison network contains n input wires a1, a2,...,an,throughwhich the values to be sorted enter the network, and n output wires b1, b2,...,bn,which produce the results computed by the network. Also, we shall speak of the input sequence a1, a2,...,an and the output sequence b1, b2,...,bn ,referringto the values" on the input and# output wires. That is, we use" the same name# for both a wire and the value it carries. Our intention will always be clear from the context. Figure 27.2 shows a comparison network,whichisasetofcomparatorsinter- connected by wires. We draw a comparison network on n inputs as a collection of n horizontal lines with comparators stretched vertically. Note that a line does not represent a single wire, but rather a sequence of distinct wires connecting vari- ous comparators. The top line in Figure 27.2, for example, represents three wires: input wire a1,whichconnectstoaninputofcomparatorA;awireconnectingthe top output of comparator A to an input of comparator C;andoutputwireb1,which comes from the top output of comparator C.Eachcomparatorinputisconnected

Comparison Networks

Comparison Network A comparison network consists solely of wires and comparators:

I. Sorting Networks Introduction to Sorting Networks 4 A sorting network is a comparison network which works correctly (that is, it sorts every input)

operates in O(1) wire connect output of one comparator to the input of another special wires: n input wires a1, a2,..., an and n output wires b1, b2,..., bn

Convention: use the same name for both a wire and its value.

Comparison Networks

Comparison Network A comparison network consists solely of wires and comparators: comparator is a device with, on given two inputs, x and y, returns two outputs x0 and y 0

27.1 Comparison networks 705

7 3 x x! min(x, y) x x ! min(x, y) comparator = = 3 7 y y! max(x, y) y y! max(x, y) = = (a) (b)

Figure 27.1 (a) Acomparatorwithinputsx and y and outputs x! and y!. (b) The same comparator, drawn as a single vertical line. Inputs x 7, y 3andoutputsx 3, y 7areshown. = = ! = ! =

Acomparisonnetworkiscomposedsolelyofwiresandcomparators. A comparator,showninFigure27.1(a),isadevicewithtwoinputs,I. Sorting Networks Introduction to Sorting Networks x and y,andtwooutputs,4 x! and y!,thatperformsthefollowingfunction:

wire connect output of one comparator to the input of another special wires: n input wires a1, a2,..., an and n output wires b1, b2,..., bn

Convention: use the same name for both a wire and its value.

Comparison Networks

Comparison Network A comparison network consists solely of wires and comparators: comparator is a device with, on given two inputs, x and y, returns two operates in O(1) outputs x0 and y 0

27.1 Comparison networks 705

7 3 x x! min(x, y) x x ! min(x, y) comparator = = 3 7 y y! max(x, y) y y! max(x, y) = = (a) (b)

Figure 27.1 (a) Acomparatorwithinputsx and y and outputs x! and y!. (b) The same comparator, drawn as a single vertical line. Inputs x 7, y 3andoutputsx 3, y 7areshown. = = ! = ! =

operates in O(1)

special wires: n input wires a1, a2,..., an and n output wires b1, b2,..., bn

Convention: use the same name for both a wire and its value.

Comparison Networks

Comparison Network A comparison network consists solely of wires and comparators: comparator is a device with, on given two inputs, x and y, returns two outputs x0 and y 0 wire connect output of one comparator to the input of another

27.1 Comparison networks 705

7 3 x x! min(x, y) x x ! min(x, y) comparator = = 3 7 y y! max(x, y) y y! max(x, y) = = (a) (b)

Figure 27.1 (a) Acomparatorwithinputsx and y and outputs x! and y!. (b) The same comparator, drawn as a single vertical line. Inputs x 7, y 3andoutputsx 3, y 7areshown. = = ! = ! =

operates in O(1)

Convention: use the same name for both a wire and its value.

Comparison Networks

27.1 Comparison networks 705

7 3 x x! min(x, y) x x ! min(x, y) comparator = = 3 7 y y! max(x, y) y y! max(x, y) = = (a) (b)

Figure 27.1 (a) Acomparatorwithinputsx and y and outputs x! and y!. (b) The same comparator, drawn as a single vertical line. Inputs x 7, y 3andoutputsx 3, y 7areshown. = = ! = ! =

operates in O(1)

Comparison Networks

Convention:27.1 Comparison use networks the same name for both a wire and its value. 705

7 3 x x! min(x, y) x x ! min(x, y) comparator = = 3 7 y y! max(x, y) y y! max(x, y) = = (a) (b)

Figure 27.1 (a) Acomparatorwithinputsx and y and outputs x! and y!. (b) The same comparator, drawn as a single vertical line. Inputs x 7, y 3andoutputsx 3, y 7areshown. = = ! = ! =

Convention: use the same name for both a wire and its value.

Comparison Networks

A sorting network is a comparison network which Comparison Network works correctly (that is, it sorts every input) A comparison network consists solely of wires and comparators: comparator is a device with, on given two inputs, x and y, returns two outputs x0 and y 0 wire connect output of one comparator to the input of another special wires: n input wires a1, a2,..., an and n output wires b1, b2,..., bn

27.1 Comparison networks 705

7 3 x x! min(x, y) x x ! min(x, y) comparator = = 3 7 y y! max(x, y) y y! max(x, y) = = (a) (b)

Figure 27.1 (a) Acomparatorwithinputsx and y and outputs x! and y!. (b) The same comparator, drawn as a single vertical line. Inputs x 7, y 3andoutputsx 3, y 7areshown. = = ! = ! =

Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires 9 5 2 2

5 9 6 5

F D F 2 2 F 5 6

D 6 6 9 9

depth 0 1 1 2 2 3 TracingThis backnetwork a path is in must fact a neverMaximum sorting cycle network! depth back ofon an output Depth of a wire: itself and go through the samewire comparator equals total twice. running time

Example of a Comparison Network (Figure 27.2)

a1 b1 A C

a2 b2 E

a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 X

Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

Interconnections between comparators must be acyclic 9 5 2 2

5 9 6 5

F D F 2 2 F 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

A horizontal line represents a sequence of distinct wires

a1 b1 A C

a2 b2 E

a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 X

Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

Interconnections between comparators must be acyclic 9 5 2 2

5 9 6 5

F D F 2 2 F 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

A horizontal line represents a sequence of distinct wires

a1 b1 A C

a2 b2 E

a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 X

Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

Interconnections between comparators must be acyclic 9 5 2 2

5 9 6 5

F D F 2 2 F 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

A horizontal line represents a sequence of distinct wires

a1 b1 A C

a2 b2 E

a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 X

Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

Interconnections between comparators must be acyclic 9 5 2 2

5 9 6 5

F D F 2 2 F 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

A horizontal line represents a sequence of distinct wires

a1 b1 A C

a2 b2 E

a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinctX wires 9 5 2 2

5 9 6 5

F D F 2 2 F 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic

a1 b1 A C

a2 b2 E

a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinctX wires 9 5 2 2

5 9 6 5

D F 2 2 F 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic

a1 b1 A C

a2 b2 F E

a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinctX wires 9 5 2 2

5 9 6 5

D F 2 2 F 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic

a1 b1 A C

a2 b2 F E

a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinctX wires 9 5 2 2

5 9 6 5

D F 2 2 F 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic

a1 b1 A C

a2 b2 F D E

a3 b3 B

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinctX wires 9 5 2 2

5 9 6 5

D F 2 2 F 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic

a1 b1 A C

a2 b2 F D E

a3 b3 B

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinctX wires 9 5 2 2

5 9 6 5

D F 2 2 F 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic

a1 b1 A C

a2 b2 F D E

a3 b3 B

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinctX wires 9 5 2 2

5 9 6 5

F D 2 2 F 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic

a1 b1 A C

a2 b2 D F E

a3 b3 B

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinct wires 9 5 2 2

5 9 6 5

D F 2 2 F 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic X

a1 b1 A C

a2 b2 F E

a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinctX wires 9 5 2 2

5 9 6 5

F D F 2 2 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic

a1 b1 A C

a2 b2 E F a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinctX wires 9 5 2 2

5 9 6 5

F D F 2 2 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic

a1 b1 A C

a2 b2 E F a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinctX wires 9 5 2 2

5 9 6 5

F D F 2 2 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic

a1 b1 A C

a2 b2 E F a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinctX wires 9 5 2 2

5 9 6 5

F D F 2 2 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic

a1 b1 A C

a2 b2 E F a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinctX wires 9 5 2 2

5 9 6 5

F D F 2 2 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic

a1 b1 A C

a2 b2 E F a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinctX wires 9 5 2 2

5 9 6 5

F D F 2 2 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic

a1 b1 A C

a2 b2 E F a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinctX wires 9 5 2 2

5 9 6 5

F D F 2 2 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic

a1 b1 A C

a2 b2 E F a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

A horizontal line represents a sequence of distinctX wires 9 5 2 2

5 9 6 5

F D F 2 2 5 6

D 6 6 9 9

depth 0 1 1 2 2 3 This network is in fact aMaximum sorting network! depth of an output Depth of a wire: wire equals total running time

Example of a Comparison Network (Figure 27.2)

Interconnections between comparators must be acyclic

a1 b1 A C

a2 b2 E F a3 b3 B D

a4 b4

Tracing back a path must never cycle back on itself and go through the same comparator twice.

I. Sorting Networks Introduction to Sorting Networks 5 X

Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires 9 5 2 2

5 9 6 5

F D F 2 2 F 5 6

D 6 6 9 9

Example of a Comparison Network (Figure 27.2)

a1 b1 A C

a2 b2 E

a3 b3 B D

a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 X

Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires 5 2 2

9 6 5

F D F 2 F 5 6

D 6 9 9

Example of a Comparison Network (Figure 27.2)

9 a1 b1 A C 5 a2 b2 E 2 a3 b3 B D 6 a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 X

Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires 2 2

6 5

F D F F 5 6

D 9 9

Example of a Comparison Network (Figure 27.2)

9 5 a1 b1 A C 5 9 a2 b2 E 2 2 a3 b3 B D 6 6 a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 X

Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires 2

F D F F 6

D 9

Example of a Comparison Network (Figure 27.2)

9 5 2 a1 b1 A C 5 9 6 a2 b2 E 2 2 5 a3 b3 B D 6 6 9 a4 b4

I. Sorting Networks Introduction to Sorting Networks 5 X

Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires

F D F F

depth 0 1 1 2 2 3 Tracing back a path must neverMaximum cycle depth back ofon an output Depth of a wire: itself and go through the samewire comparator equals total twice. running time

Example of a Comparison Network (Figure 27.2)

9 5 2 2 a1 b1 A C 5 9 6 5 a2 b2 E 2 2 5 6 a3 b3 B D 6 6 9 9 a4 b4

This network is in fact a sorting network!

I. Sorting Networks Introduction to Sorting Networks 5 X

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires

F D F F

depth 0 1 1 2 2 3 TracingThis backnetwork a path is in must fact a neverMaximum sorting cycle network! depth back ofon an output itself and go through the samewire comparator equals total twice. running time Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

Example of a Comparison Network (Figure 27.2)

9 5 2 2 a1 b1 A C 5 9 6 5 a2 b2 E 2 2 5 6 a3 b3 B D 6 6 9 9 a4 b4

Depth of a wire:

I. Sorting Networks Introduction to Sorting Networks 5 X

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires

F D F F

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

Example of a Comparison Network (Figure 27.2)

9 5 2 2 a1 b1 A C 5 9 6 5 a2 b2 E 2 2 5 6 a3 b3 B D 6 6 9 9 a4 b4

Depth of a wire: Input wire has depth 0

I. Sorting Networks Introduction to Sorting Networks 5 X

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires

F D F F

Example of a Comparison Network (Figure 27.2)

9 5 2 2 a1 b1 A C 5 9 6 5 a2 b2 E 2 2 5 6 a3 b3 B D 6 6 9 9 a4 b4

Depth of a wire: Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

I. Sorting Networks Introduction to Sorting Networks 5 X

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires

F D F F

0 1 1 2 2 3

TracingThis backnetwork a path is in must fact a neverMaximum sorting cycle network! depth back ofon an output itself and go through the samewire comparator equals total twice. running time

Example of a Comparison Network (Figure 27.2)

9 5 2 2 a1 b1 A C 5 9 6 5 a2 b2 E 2 2 5 6 a3 b3 B D 6 6 9 9 a4 b4 depth

Depth of a wire: Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

I. Sorting Networks Introduction to Sorting Networks 5 X

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires

F D F F

1 1 2 2 3

TracingThis backnetwork a path is in must fact a neverMaximum sorting cycle network! depth back ofon an output itself and go through the samewire comparator equals total twice. running time

Example of a Comparison Network (Figure 27.2)

9 5 2 2 a1 b1 A C 5 9 6 5 a2 b2 E 2 2 5 6 a3 b3 B D 6 6 9 9 a4 b4 depth 0

Depth of a wire: Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

I. Sorting Networks Introduction to Sorting Networks 5 X

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires

F D F F

1 2 2 3

TracingThis backnetwork a path is in must fact a neverMaximum sorting cycle network! depth back ofon an output itself and go through the samewire comparator equals total twice. running time

Example of a Comparison Network (Figure 27.2)

9 5 2 2 a1 b1 A C 5 9 6 5 a2 b2 E 2 2 5 6 a3 b3 B D 6 6 9 9 a4 b4 depth 0 1

Depth of a wire: Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

I. Sorting Networks Introduction to Sorting Networks 5 X

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires

F D F F

2 2 3

TracingThis backnetwork a path is in must fact a neverMaximum sorting cycle network! depth back ofon an output itself and go through the samewire comparator equals total twice. running time

Example of a Comparison Network (Figure 27.2)

9 5 2 2 a1 b1 A C 5 9 6 5 a2 b2 E 2 2 5 6 a3 b3 B D 6 6 9 9 a4 b4 depth 0 1 1

Depth of a wire: Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

I. Sorting Networks Introduction to Sorting Networks 5 X

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires

F D F F

2 3

TracingThis backnetwork a path is in must fact a neverMaximum sorting cycle network! depth back ofon an output itself and go through the samewire comparator equals total twice. running time

Example of a Comparison Network (Figure 27.2)

9 5 2 2 a1 b1 A C 5 9 6 5 a2 b2 E 2 2 5 6 a3 b3 B D 6 6 9 9 a4 b4 depth 0 1 1 2

Depth of a wire: Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

I. Sorting Networks Introduction to Sorting Networks 5 X

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires

F D F F

TracingThis backnetwork a path is in must fact a neverMaximum sorting cycle network! depth back ofon an output itself and go through the samewire comparator equals total twice. running time

Example of a Comparison Network (Figure 27.2)

9 5 2 2 a1 b1 A C 5 9 6 5 a2 b2 E 2 2 5 6 a3 b3 B D 6 6 9 9 a4 b4 depth 0 1 1 2 2

Depth of a wire: Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

I. Sorting Networks Introduction to Sorting Networks 5 X

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires

F D F F

TracingThis backnetwork a path is in must fact a neverMaximum sorting cycle network! depth back ofon an output itself and go through the samewire comparator equals total twice. running time

Example of a Comparison Network (Figure 27.2)

9 5 2 2 a1 b1 A C 5 9 6 5 a2 b2 E 2 2 5 6 a3 b3 B D 6 6 9 9 a4 b4 depth 0 1 1 2 2 3

Depth of a wire: Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

I. Sorting Networks Introduction to Sorting Networks 5 X

InterconnectionsA horizontal between line represents comparators a sequencemust be of acyclic distinct wires

F D F F

TracingThis backnetwork a path is in must fact a never sorting cycle network! back on itself and go through the same comparator twice.

Example of a Comparison Network (Figure 27.2)

9 5 2 2 a1 b1 A C 5 9 6 5 a2 b2 E 2 2 5 6 a3 b3 B D 6 6 9 9 a4 b4 depth 0 1 1 2 2 3 Maximum depth of an output Depth of a wire: wire equals total running time Input wire has depth 0

If a comparator has two inputs of depths dx and dy , then outputs have depth max{dx , dy } + 1

I. Sorting Networks Introduction to Sorting Networks 5 Lemma 27.1

If a comparison network transforms the input a = ha1, a2,..., ani into the output b = hb1, b2,..., bni, then for any monotonically increasing function f , the network transforms f (a) = hf (a1), f (a2),..., f (an)i into f (b) = hf (b1), f (b2),..., f (bn)i.

Zero-One Principle

Zero-One Principle: A sorting networks works correctly on arbitrary inputs if it works correctly on binary inputs.

I. Sorting Networks Introduction to Sorting Networks 6 Zero-One Principle

Zero-One Principle: A sorting networks works correctly on arbitrary inputs if it works correctly on binary inputs.

Lemma 27.1

I. Sorting Networks Introduction to Sorting Networks 6 Zero-One Principle

Zero-One Principle: A sorting networks works correctly on arbitrary inputs if it works correctly on binary inputs.

Lemma 27.1

710 Chapter 27 Sorting Networks

f (x) min( f (x), f (y)) f (min(x, y)) = f (y) max( f (x), f (y)) f (max(x, y)) =

Figure 27.4 The operation of the comparator in the proof of Lemma 27.1. Thefunction f is monotonically increasing.

To prove the claim, consider a comparator whose input values are x and y.The I. Sorting Networks Introduction to Sorting Networks 6 upper output of the comparator is min(x, y) and the lower output is max(x, y). Suppose we now apply f (x) and f (y) to the inputs of the comparator, as is shown in Figure 27.4. The operation of the comparator yields the value min( f (x), f (y)) on the upper output and the value max( f (x), f (y)) on the lower output. Since f is monotonically increasing, x y implies f (x) f (y).Consequently,wehave the identities ! ! min( f (x), f (y)) f (min(x, y)) , = max( f (x), f (y)) f (max(x, y)) . = Thus, the comparator produces the values f (min(x, y)) and f (max(x, y)) when f (x) and f (y) are its inputs, which completes the proof of the claim. We can use induction on the depth of each wire in a general comparison network to prove a stronger result than the statement of the lemma: if awireassumesthe value ai when the input sequence a is applied to the network, then it assumes the value f (ai ) when the input sequence f (a) is applied. Because the output wires are included in this statement, proving it will prove the lemma. For the basis, consider a wire at depth 0, that is, an input wire ai .Theresult follows trivially: when f (a) is applied to the network, the input wire carries f (ai ). For the inductive step, consider a wire at depth d,whered 1. The wire is the output of a comparator at depth d,andtheinputwirestothiscomparatorareata" depth strictly less than d.Bytheinductivehypothesis,therefore,iftheinputwires to the comparator carry values ai and a j when the input sequence a is applied, then they carry f (ai ) and f (a j ) when the input sequence f (a) is applied. By our earlier claim, the output wires of this comparator then carry f (min(ai , a j )) and f (max(ai , a j )).Sincetheycarrymin(ai , a j ) and max(ai , a j ) when the input sequence is a,thelemmaisproved.

As an example of the application of Lemma 27.1, Figure 27.5(b)showsthesort- ing network from Figure 27.2 (repeated in Figure 27.5(a)) with the monotonically increasing function f (x) x/2 applied to the inputs. The value on every wire is f applied to the value on= the# same↔ wire in Figure 27.2. When a comparison network is a sorting network, Lemma 27.1 allows us to prove the following remarkable result. Zero-One Principle

Zero-One Principle: A sorting networks works correctly on arbitrary inputs if it works correctly on binary inputs.

Lemma 27.1

Theorem 27.2 (Zero-One Principle) If a comparison network with n inputs sorts all 2n possible sequences of 0’s and 1’s correctly, then it sorts all sequences of arbitrary numbers correctly.

I. Sorting Networks Introduction to Sorting Networks 6 ( 0 if x ≤ a , f (x) = i 1 if x > ai .

⇒ f (aj ) is placed before f (ai )

For the sake of contradiction, suppose the network does not correctly sort.

Let a = ha1, a2,..., ani be the input with ai < aj , but the network places aj before ai in the output Deﬁne a monotonically increasing function f as:

Since the network places aj before ai , by the previous lemma

But f (aj ) = 1 and f (ai ) = 0, which contradicts the assumption that the network sorts all sequences of 0’s and 1’s correctly

Proof:

Proof of the Zero-One Principle

Theorem 27.2 (Zero-One Principle) If a comparison network with n inputs sorts all 2n possible sequences of 0’s and 1’s correctly, then it sorts all sequences of arbitrary numbers correctly.

I. Sorting Networks Introduction to Sorting Networks 7 ( 0 if x ≤ a , f (x) = i 1 if x > ai .

⇒ f (aj ) is placed before f (ai )

For the sake of contradiction, suppose the network does not correctly sort.

Let a = ha1, a2,..., ani be the input with ai < aj , but the network places aj before ai in the output Deﬁne a monotonically increasing function f as:

Since the network places aj before ai , by the previous lemma

But f (aj ) = 1 and f (ai ) = 0, which contradicts the assumption that the network sorts all sequences of 0’s and 1’s correctly

Proof of the Zero-One Principle

Theorem 27.2 (Zero-One Principle) If a comparison network with n inputs sorts all 2n possible sequences of 0’s and 1’s correctly, then it sorts all sequences of arbitrary numbers correctly.

Proof:

I. Sorting Networks Introduction to Sorting Networks 7 ( 0 if x ≤ a , f (x) = i 1 if x > ai .

⇒ f (aj ) is placed before f (ai )

Let a = ha1, a2,..., ani be the input with ai < aj , but the network places aj before ai in the output Deﬁne a monotonically increasing function f as:

Since the network places aj before ai , by the previous lemma

But f (aj ) = 1 and f (ai ) = 0, which contradicts the assumption that the network sorts all sequences of 0’s and 1’s correctly

Proof of the Zero-One Principle

Theorem 27.2 (Zero-One Principle) If a comparison network with n inputs sorts all 2n possible sequences of 0’s and 1’s correctly, then it sorts all sequences of arbitrary numbers correctly.

Proof: For the sake of contradiction, suppose the network does not correctly sort.

I. Sorting Networks Introduction to Sorting Networks 7 ( 0 if x ≤ a , f (x) = i 1 if x > ai .

⇒ f (aj ) is placed before f (ai )

Deﬁne a monotonically increasing function f as:

Since the network places aj before ai , by the previous lemma

But f (aj ) = 1 and f (ai ) = 0, which contradicts the assumption that the network sorts all sequences of 0’s and 1’s correctly

Proof of the Zero-One Principle

Theorem 27.2 (Zero-One Principle) If a comparison network with n inputs sorts all 2n possible sequences of 0’s and 1’s correctly, then it sorts all sequences of arbitrary numbers correctly.

Proof: For the sake of contradiction, suppose the network does not correctly sort.

Let a = ha1, a2,..., ani be the input with ai < aj , but the network places aj before ai in the output

I. Sorting Networks Introduction to Sorting Networks 7 ⇒ f (aj ) is placed before f (ai )

( 0 if x ≤ a , f (x) = i 1 if x > ai .

Since the network places aj before ai , by the previous lemma

But f (aj ) = 1 and f (ai ) = 0, which contradicts the assumption that the network sorts all sequences of 0’s and 1’s correctly

Proof of the Zero-One Principle

Theorem 27.2 (Zero-One Principle) If a comparison network with n inputs sorts all 2n possible sequences of 0’s and 1’s correctly, then it sorts all sequences of arbitrary numbers correctly.

Proof: For the sake of contradiction, suppose the network does not correctly sort.

Let a = ha1, a2,..., ani be the input with ai < aj , but the network places aj before ai in the output Deﬁne a monotonically increasing function f as:

I. Sorting Networks Introduction to Sorting Networks 7 ⇒ f (aj ) is placed before f (ai )

Since the network places aj before ai , by the previous lemma

But f (aj ) = 1 and f (ai ) = 0, which contradicts the assumption that the network sorts all sequences of 0’s and 1’s correctly

Proof of the Zero-One Principle

Theorem 27.2 (Zero-One Principle) If a comparison network with n inputs sorts all 2n possible sequences of 0’s and 1’s correctly, then it sorts all sequences of arbitrary numbers correctly.

Proof: For the sake of contradiction, suppose the network does not correctly sort.

Let a = ha1, a2,..., ani be the input with ai < aj , but the network places aj before ai in the output Deﬁne a monotonically increasing function f as: ( 0 if x ≤ a , f (x) = i 1 if x > ai .

I. Sorting Networks Introduction to Sorting Networks 7 ⇒ f (aj ) is placed before f (ai )

But f (aj ) = 1 and f (ai ) = 0, which contradicts the assumption that the network sorts all sequences of 0’s and 1’s correctly

Proof of the Zero-One Principle

Theorem 27.2 (Zero-One Principle) If a comparison network with n inputs sorts all 2n possible sequences of 0’s and 1’s correctly, then it sorts all sequences of arbitrary numbers correctly.

Proof: For the sake of contradiction, suppose the network does not correctly sort.

Let a = ha1, a2,..., ani be the input with ai < aj , but the network places aj before ai in the output Deﬁne a monotonically increasing function f as: ( 0 if x ≤ a , f (x) = i 1 if x > ai .

Since the network places aj before ai , by the previous lemma

I. Sorting Networks Introduction to Sorting Networks 7 But f (aj ) = 1 and f (ai ) = 0, which contradicts the assumption that the network sorts all sequences of 0’s and 1’s correctly

Proof of the Zero-One Principle

Theorem 27.2 (Zero-One Principle) If a comparison network with n inputs sorts all 2n possible sequences of 0’s and 1’s correctly, then it sorts all sequences of arbitrary numbers correctly.

Proof: For the sake of contradiction, suppose the network does not correctly sort.

Let a = ha1, a2,..., ani be the input with ai < aj , but the network places aj before ai in the output Deﬁne a monotonically increasing function f as: ( 0 if x ≤ a , f (x) = i 1 if x > ai .

Since the network places aj before ai , by the previous lemma ⇒ f (aj ) is placed before f (ai )

I. Sorting Networks Introduction to Sorting Networks 7 Proof of the Zero-One Principle

Theorem 27.2 (Zero-One Principle) If a comparison network with n inputs sorts all 2n possible sequences of 0’s and 1’s correctly, then it sorts all sequences of arbitrary numbers correctly.

Proof: For the sake of contradiction, suppose the network does not correctly sort.

Let a = ha1, a2,..., ani be the input with ai < aj , but the network places aj before ai in the output Deﬁne a monotonically increasing function f as: ( 0 if x ≤ a , f (x) = i 1 if x > ai .

Since the network places aj before ai , by the previous lemma ⇒ f (aj ) is placed before f (ai )

But f (aj ) = 1 and f (ai ) = 0, which contradicts the assumption that the network sorts all sequences of 0’s and 1’s correctly

I. Sorting Networks Introduction to Sorting Networks 7 Insertion??? Sort

Bubble Sort

These are Sorting Networks, but with depth Θ(n).

1 2 3 4 n-wire Sorting Network 5

n − 1 n n + 1

Some Basic (Recursive) Sorting Networks

1 2 3 4 n-wire Sorting Network ??? 5

n − 1 n n + 1

I. Sorting Networks Introduction to Sorting Networks 8 Insertion??? Sort

???

These are Sorting Networks, but with depth Θ(n).

1 2 3 4 n-wire Sorting Network 5

n − 1 n n + 1

Some Basic (Recursive) Sorting Networks

1 2 3 4 n-wire Sorting Network Bubble Sort 5

n − 1 n n + 1

I. Sorting Networks Introduction to Sorting Networks 8 ???

These are Sorting Networks, but with depth Θ(n).

Insertion Sort

Some Basic (Recursive) Sorting Networks

1 2 3 4 n-wire Sorting Network Bubble Sort 5

n − 1 n n + 1

1 2 3 4 n-wire Sorting Network ??? 5

n − 1 n n + 1

I. Sorting Networks Introduction to Sorting Networks 8 ???

These are Sorting Networks, but with depth Θ(n).

???

Some Basic (Recursive) Sorting Networks

1 2 3 4 n-wire Sorting Network Bubble Sort 5

n − 1 n n + 1

1 2 3 4 n-wire Sorting Network Insertion Sort 5

n − 1 n n + 1

I. Sorting Networks Introduction to Sorting Networks 8 ???

???

Some Basic (Recursive) Sorting Networks

1 2 3 4 n-wire Sorting Network Bubble Sort 5

n − 1 n n + 1 These are Sorting Networks, but with depth Θ(n).

1 2 3 4 n-wire Sorting Network Insertion Sort 5

n − 1 n n + 1

I. Sorting Networks Introduction to Sorting Networks 8 Outline

Outline of this Course

Introduction to Sorting Networks

Batcher’s Sorting Network

Counting Networks

I. Sorting Networks Batcher’s Sorting Network 9 binary sequences:

h1, 4, 6, 8, 3, 2i h6, 9, 4, 2, 3, 5i h9, 8, 3, 2, 4, 6i

Examples:

Bitonic Sequences

Bitonic Sequence A sequence is bitonic if it monotonically increases and then monotonically decreases, or can be circularly shifted to become monotonically increasing and then monotonically decreasing.

Sequences of one or two numbers are deﬁned to be bitonic.

I. Sorting Networks Batcher’s Sorting Network 10 binary sequences:

h1, 4, 6, 8, 3, 2i h6, 9, 4, 2, 3, 5i h9, 8, 3, 2, 4, 6i

Sequences of one or two numbers are deﬁned to be bitonic.

Examples:

Bitonic Sequences

I. Sorting Networks Batcher’s Sorting Network 10 binary sequences:

Sequences of one or two numbers are deﬁned to be bitonic.

h1, 4, 6, 8, 3, 2i h6, 9, 4, 2, 3, 5i h9, 8, 3, 2, 4, 6i

Bitonic Sequences

Examples:

I. Sorting Networks Batcher’s Sorting Network 10 binary sequences:

Sequences of one or two numbers are deﬁned to be bitonic.

h6, 9, 4, 2, 3, 5i h9, 8, 3, 2, 4, 6i

Bitonic Sequences

Examples: h1, 4, 6, 8, 3, 2i ?

I. Sorting Networks Batcher’s Sorting Network 10 binary sequences:

Sequences of one or two numbers are deﬁned to be bitonic.

h6, 9, 4, 2, 3, 5i h9, 8, 3, 2, 4, 6i

Bitonic Sequences

Examples: h1, 4, 6, 8, 3, 2i X

I. Sorting Networks Batcher’s Sorting Network 10 binary sequences:

Sequences of one or two numbers are deﬁned to be bitonic.

h9, 8, 3, 2, 4, 6i

Bitonic Sequences

Examples: h1, 4, 6, 8, 3, 2i X h6, 9, 4, 2, 3, 5i ?

I. Sorting Networks Batcher’s Sorting Network 10 binary sequences:

Sequences of one or two numbers are deﬁned to be bitonic.

h9, 8, 3, 2, 4, 6i

Bitonic Sequences

Examples: h1, 4, 6, 8, 3, 2i X h6, 9, 4, 2, 3, 5i X

I. Sorting Networks Batcher’s Sorting Network 10 binary sequences:

Sequences of one or two numbers are deﬁned to be bitonic.

Bitonic Sequences

Examples: h1, 4, 6, 8, 3, 2i X h6, 9, 4, 2, 3, 5i X h9, 8, 3, 2, 4, 6i ?

I. Sorting Networks Batcher’s Sorting Network 10 binary sequences:

Sequences of one or two numbers are deﬁned to be bitonic.

Bitonic Sequences

Examples: h1, 4, 6, 8, 3, 2i X h6, 9, 4, 2, 3, 5i X h9, 8, 3, 2, 4, 6i X

I. Sorting Networks Batcher’s Sorting Network 10 binary sequences:

Sequences of one or two numbers are deﬁned to be bitonic.

Bitonic Sequences

Examples: h1, 4, 6, 8, 3, 2i X h6, 9, 4, 2, 3, 5i X h9, 8, 3, 2, 4, 6i X h4, 5, 7, 1, 2, 6i ?

I. Sorting Networks Batcher’s Sorting Network 10 binary sequences:

Sequences of one or two numbers are deﬁned to be bitonic.

Bitonic Sequences

Examples: h1, 4, 6, 8, 3, 2i X h6, 9, 4, 2, 3, 5i X h9, 8, 3, 2, 4, 6i X ((( (h4,(5,(7, 1, 2, 6i

I. Sorting Networks Batcher’s Sorting Network 10 Sequences of one or two numbers are deﬁned to be bitonic.

Bitonic Sequences

Examples: h1, 4, 6, 8, 3, 2i X h6, 9, 4, 2, 3, 5i X h9, 8, 3, 2, 4, 6i X ((( (h4,(5,(7, 1, 2, 6i binary sequences: ?

I. Sorting Networks Batcher’s Sorting Network 10 Sequences of one or two numbers are deﬁned to be bitonic.

Bitonic Sequences

Examples: h1, 4, 6, 8, 3, 2i X h6, 9, 4, 2, 3, 5i X h9, 8, 3, 2, 4, 6i X ((( (h4,(5,(7, 1, 2, 6i binary sequences: 0i 1j 0k , or, 1i 0j 1k , for i, j, k ≥ 0.

I. Sorting Networks Batcher’s Sorting Network 10 both the top half and the bottom half are bitonic, every element in the top is not larger than any element in the bottom, at least one half is clean.

We always assume that n is even. Lemma 27.3 If the input to a half-cleaner is a bitonic sequence of 0’s and 1’s, then the output satisﬁes the following properties:

27.3 A bitonic sorting network 713

0 0 0 0 0 0 bitonic, 0 0 bitonic 1 0 clean 1 1 1 0 1 0 bitonic bitonic 1 1 1 1 0 0 1 1 bitonic, bitonic 0 1 1 1 clean 0 1 0 1

Figure 27.7 The comparison network HALF-CLEANER[8]. Two different sample zero-one input and output values are shown. The input is assumed to be bitonic. A half-cleaner ensures that every output element of the top half is at least as small as every output element of the bottom half. Moreover, both halves are bitonic, and at least one half is clean.

even.) Figure 27.7 shows HALF-CLEANER[8], the half-cleaner with 8 inputs and 8outputs. When a bitonic sequence of 0’s and 1’s is applied as input to a half-cleaner, the half-cleaner produces an output sequence in which smaller values are in the top half, larger values are in the bottom half, and both halves arebitonic.Infact,at least one of the halves is clean—consisting of either all 0’s or all 1’s—and it is from this property that we derive the name “half-cleaner.” (Note that all clean sequences are bitonic.) The next lemma proves these properties of half-cleaners.

Lemma 27.3 If the input to a half-cleaner is a bitonic sequence of 0’s and 1’s, then the output satisﬁes the following properties: both the top half and the bottom half are bitonic, every element in the top half is at least as small as every element of the bottom half, and at least one half is clean.

Proof The comparison network HALF-CLEANER[n]comparesinputsi and i n/2fori 1, 2,...,n/2. Without loss of generality, suppose that the input+ is of the form= 00 ...011 ...100 ...0. (The situation in which the input is of the form 11 ...100 ...011 ...1issymmetric.)Therearethreepossiblecasesde- pending upon the block of consecutive 0’s or 1’s in which the midpoint n/2falls, and one of these cases (the one in which the midpoint occurs in the block of 1’s) is further split into two cases. The four cases are shown in Figure 27.8. In each case shown, the lemma holds.

Towards Bitonic Sorting Networks

Half-Cleaner A half-cleaner is a comparison network of depth 1 in which input wire i is compared with wire i + n/2 for i = 1, 2,..., n/2.

I. Sorting Networks Batcher’s Sorting Network 11 both the top half and the bottom half are bitonic, every element in the top is not larger than any element in the bottom, at least one half is clean.

Lemma 27.3 If the input to a half-cleaner is a bitonic sequence of 0’s and 1’s, then the output satisﬁes the following properties:

27.3 A bitonic sorting network 713

0 0 0 0 0 0 bitonic, 0 0 bitonic 1 0 clean 1 1 1 0 1 0 bitonic bitonic 1 1 1 1 0 0 1 1 bitonic, bitonic 0 1 1 1 clean 0 1 0 1

Towards Bitonic Sorting Networks

Half-Cleaner A half-cleaner is a comparison network of depth 1 in which input wire i is compared with wire i + n/2 for i = 1, 2,..., n/2.

We always assume that n is even.

We always assume that n is even. Lemma 27.3 If the input to a half-cleaner is a bitonic sequence of 0’s and 1’s, then the output satisﬁes the following properties:

Towards Bitonic Sorting Networks

Half-Cleaner A half-cleaner is a comparison network of depth 1 in which input wire i is compared with wire i + n/2 for i = 1, 2,..., n/2.

27.3 A bitonic sorting network 713

0 0 0 0 0 0 bitonic, 0 0 bitonic 1 0 clean 1 1 1 0 1 0 bitonic bitonic 1 1 1 1 0 0 1 1 bitonic, bitonic 0 1 1 1 clean 0 1 0 1

I. Sorting Networks Batcher’s Sorting Network 11 Figure 27.7 The comparison network HALF-CLEANER[8]. Two different sample zero-one input and output values are shown. The input is assumed to be bitonic. A half-cleaner ensures that every output element of the top half is at least as small as every output element of the bottom half. Moreover, both halves are bitonic, and at least one half is clean.

We always assume that n is even. Lemma 27.3 If the input to a half-cleaner is a bitonic sequence of 0’s and 1’s, then the output satisﬁes the following properties:

Towards Bitonic Sorting Networks

Half-Cleaner A half-cleaner is a comparison network of depth 1 in which input wire i is compared with wire i + n/2 for i = 1, 2,..., n/2.

27.3 A bitonic sorting network 713

0 0 0 0 0 0 bitonic, 0 0 bitonic 1 0 clean 1 1 1 0 1 0 bitonic bitonic 1 1 1 1 0 0 1 1 bitonic, bitonic 0 1 1 1 clean 0 1 0 1

We always assume that n is even. Lemma 27.3 If the input to a half-cleaner is a bitonic sequence of 0’s and 1’s, then the output satisﬁes the following properties:

Towards Bitonic Sorting Networks

Half-Cleaner A half-cleaner is a comparison network of depth 1 in which input wire i is compared with wire i + n/2 for i = 1, 2,..., n/2.

27.3 A bitonic sorting network 713

0 0 0 0 0 0 bitonic, 0 0 bitonic 1 0 clean 1 1 1 0 1 0 bitonic bitonic 1 1 1 1 0 0 1 1 bitonic, bitonic 0 1 1 1 clean 0 1 0 1

We always assume that n is even. Lemma 27.3 If the input to a half-cleaner is a bitonic sequence of 0’s and 1’s, then the output satisﬁes the following properties:

Towards Bitonic Sorting Networks

Half-Cleaner A half-cleaner is a comparison network of depth 1 in which input wire i is compared with wire i + n/2 for i = 1, 2,..., n/2.

27.3 A bitonic sorting network 713

0 0 0 0 0 0 bitonic, 0 0 bitonic 1 0 clean 1 1 1 0 1 0 bitonic bitonic 1 1 1 1 0 0 1 1 bitonic, bitonic 0 1 1 1 clean 0 1 0 1

Towards Bitonic Sorting Networks

Half-Cleaner A half-cleaner is a comparison network of depth 1 in which input wire i is compared with wire i + n/2 for i = 1, 2,..., n/2.

Lemma 27.3 If the input to a half-cleaner is a bitonic sequence of 0’s and 1’s, then the output satisﬁes the following properties: both the top half and the bottom half are bitonic, every element in the top is not larger than any element in the bottom, at least one half is clean. 27.3 A bitonic sorting network 713

0 0 0 0 0 0 bitonic, 0 0 bitonic 1 0 clean 1 1 1 0 1 0 bitonic bitonic 1 1 1 1 0 0 1 1 bitonic, bitonic 0 1 1 1 clean 0 1 0 1

Towards Bitonic Sorting Networks

Half-Cleaner A half-cleaner is a comparison network of depth 1 in which input wire i is compared with wire i + n/2 for i = 1, 2,..., n/2.

Lemma 27.3 If the input to a half-cleaner is a bitonic sequence of 0’s and 1’s, then the output satisﬁes the following properties: both the top half and the bottom half are bitonic, every element in the top is not larger than any element in the bottom, at least one half is clean. 27.3 A bitonic sorting network 713

0 0 0 0 0 0 bitonic, 0 0 bitonic 1 0 clean 1 1 1 0 1 0 bitonic bitonic 1 1 1 1 0 0 1 1 bitonic, bitonic 0 1 1 1 clean 0 1 0 1

Proof of Lemma 27.3

W.l.o.g. assume that the input is of the form 0i 1j 0k , for some i, j, k ≥ 0.

I. Sorting Networks Batcher’s Sorting Network 12 This suggests a recursive approach, since it now sufﬁces to sort the top and bottom half separately.

Proof of Lemma 27.3 714 Chapter 27 Sorting Networks W.l.o.g. assume that the input is of the form 0i 1j 0k , for some i, j, k ≥ 0.

divide compare combine

0 top top bitonic, 0 1 1 clean 0 bitonic 0 0 1 0 1 1 1 0 bitonic 0 bottom bottom 1 (a)

0 0 top top 1 bitonic 0 0 1 0 bitonic 1 1 1 1 0 0 1 bitonic, bottom bottom clean 0 (b)

top top bitonic, 0 0 0 0 clean bitonic I. Sorting0 Networks1 Batcher’s0 Sorting1 Network 12 0 0 0 1 1 bitonic bottom bottom 0 0 (c)

0 top top bitonic, 1 0 0 0 clean bitonic 1 0 0 1 0 0 0 0 1 bitonic bottom bottom 0 (d)

Figure 27.8 The possible comparisons in HALF-CLEANER[n]. The input sequence is assumed to be a bitonic sequence of 0’s and 1’s, and without loss of generality, we assume that it is of the form 00 ...011 ...100 ...0. Subsequences of 0’s are white, and subsequences of 1’s are gray. We can think of the n inputs as being divided into two halves such that for i 1, 2,...,n/2, inputs i = and i n/2arecompared. (a)–(b) Cases in which the division occurs in the middle subsequence + of 1’s. (c)–(d) Cases in which the division occurs in a subsequence of 0’s. Forallcases,every element in the top half of the output is at least as small as every element in the bottom half, both halves are bitonic, and at least one half is clean. This suggests a recursive approach, since it now sufﬁces to sort the top and bottom half separately.

Proof of Lemma 27.3 714 Chapter 27 Sorting Networks W.l.o.g. assume that the input is of the form 0i 1j 0k , for some i, j, k ≥ 0.

divide compare combine

0 top top bitonic, 0 1 1 clean 0 bitonic 0 0 1 0 1 1 1 0 bitonic 0 bottom bottom 1 (a)

0 0 top top 1 bitonic 0 0 1 0 bitonic 1 1 1 1 0 0 1 bitonic, bottom bottom clean 0 (b)

top top bitonic, 0 0 0 0 clean bitonic I. Sorting0 Networks1 Batcher’s0 Sorting1 Network 12 0 0 0 1 1 bitonic bottom bottom 0 0 (c)

0 top top bitonic, 1 0 0 0 clean bitonic 1 0 0 1 0 0 0 0 1 bitonic bottom bottom 0 (d)

714 Chapter 27 Sorting Networks

divide compare combine

0 top top bitonic, 0 1 1 clean 0 bitonic 0 0 1 0 1 1 1 0 bitonic 0 bottom bottom 1 (a)

0 0 top top 1 bitonic 0 0 1 0 bitonic 1 1 1 1 Proof of Lemma 27.3 0 0 1 bitonic, bottom bottom clean 0 i j k W.l.o.g. assume that the input(b) is of the form 0 1 0 , for some i, j, k ≥ 0.

top top bitonic, 0 0 0 0 clean bitonic 0 1 0 1 0 0 0 1 1 bitonic bottom bottom 0 0 (c)

0 top top bitonic, 1 0 0 0 clean bitonic 1 0 0 1 0 0 0 0 1 bitonic bottom bottom 0 (d)

Figure 27.8 The possible comparisons in HALF-CLEANER[n]. The input sequence is assumed to be a bitonic sequence of 0’s and 1’s, and without loss of generality, we assume that it is of the form 00 ...011 ...100 ...0. Subsequences of 0’s are white, and subsequences of 1’s are gray. We can think of the n inputsI. Sorting as Networks being divided into Batcher’s two halvesSorting Network such that for i 1, 2,...,n/2, inputs12 i = and i n/2arecompared. (a)–(b) Cases in which the division occurs in the middle subsequence + of 1’s. (c)–(d) Cases in which the division occurs in a subsequence of 0’s. Forallcases,every element in the top half of the output is at least as small as every element in the bottom half, both halves are bitonic, and at least one half is clean. 714 Chapter 27 Sorting Networks

divide compare combine

0 top top bitonic, 0 1 1 clean 0 bitonic 0 0 1 0 1 1 1 0 bitonic 0 bottom bottom 1 (a)

0 0 top top 1 bitonic 0 0 1 0 bitonic 1 1 1 1 Proof of Lemma 27.3 0 0 1 bitonic, bottom bottom clean 0 i j k W.l.o.g. assume that the input(b) is of the form 0 1 0 , for some i, j, k ≥ 0.

top top bitonic, 0 0 0 0 clean bitonic 0 1 0 1 0 0 0 1 1 bitonic bottom bottom 0 0 (c)

0 top top bitonic, 1 0 0 0 clean bitonic 1 0 0 1 0 0 0 0 1 bitonic bottom bottom 0 (d)

This suggests a recursive approach, since it now Figure 27.8 The possible comparisonssufﬁces in HALF to-CLEANER sort the[n]. top The and input bottom sequence halfis assumed separately. to be a bitonic sequence of 0’s and 1’s, and without loss of generality, we assume that it is of the form 00 ...011 ...100 ...0. Subsequences of 0’s are white, and subsequences of 1’s are gray. We can think of the n inputsI. Sorting as Networks being divided into Batcher’s two halvesSorting Network such that for i 1, 2,...,n/2, inputs12 i = and i n/2arecompared. (a)–(b) Cases in which the division occurs in the middle subsequence + of 1’s. (c)–(d) Cases in which the division occurs in a subsequence of 0’s. Forallcases,every element in the top half of the output is at least as small as every element in the bottom half, both halves are bitonic, and at least one half is clean. Henceforth we will always assume that n is a power of 2. Recursive Formula for depth D(n): ( 0 if n = 1, D(n) = D(n/2) + 1 if n = 2k .

BITONIC-SORTER[n] has depth log n and sorts any zero-one bitonic sequence.

The Bitonic27.3 Sorter A bitonic sorting network 715

0 0 0 0 0 0 BITONIC- 0 0 SORTER[n/2] 1 0 0 0 0 0 HALF- 1 0 bitonic sorted CLEANER[n] 1 1 1 0 0 0 BITONIC- 0 1 SORTER[n/2] 0 1 1 1 0 1 1 1 (a) (b)

The bitonic sorter By recursively combining half-cleaners, as shown in Figure 27.9, we can build a bitonic sorter,whichisanetworkthatsortsbitonicsequences.Theﬁrststage of BITONIC-SORTER[n]consistsofHALF-CLEANER[n], which, by Lemma 27.3, produces two bitonic sequences of half the size such that every element in the top half is at least as small as every element in the bottom half. Thus, we can complete the sort by using two copies of BITONIC-SORTER[n/2] to sort the two halves recursively. In Figure 27.9(a), the recursion has been shown explicitly, and in Figure 27.9(b), the recursion has been unrolled to show theprogressivelysmaller half-cleaners that make up the remainder of the bitonic sorter. The depth D(n) of BITONIC-SORTER[n]isgivenbytherecurrence 0ifn 1 , D(nI.) Sorting Networks= Batcher’s Sorting Network 13 = D(n/2) 1ifn 2k and k 1 , ! + = ! whose solution is D(n) lg n. = Thus, a zero-one bitonic sequence can be sorted by BITONIC-SORTER,which has a depth of lg n.Itfollowsbytheanalogofthezero-oneprinciplegivenas Exercise 27.3-6 that any bitonic sequence of arbitrary numbers can be sorted by this network.

Exercises

27.3-1 How many zero-one bitonic sequences of length n are there? Henceforth we will always assume that n is a power of 2.

BITONIC-SORTER[n] has depth log n and sorts any zero-one bitonic sequence.

The Bitonic27.3 Sorter A bitonic sorting network 715

0 0 0 0 0 0 BITONIC- 0 0 SORTER[n/2] 1 0 0 0 0 0 HALF- 1 0 bitonic sorted CLEANER[n] 1 1 1 0 0 0 BITONIC- 0 1 SORTER[n/2] 0 1 1 1 0 1 1 1 (a) (b)

The bitonic sorter Recursive FormulaBy recursively for depth combiningD( half-cleaners,n): as shown in Figure 27.9, we can build a bitonic sorter,whichisanetworkthatsortsbitonicsequences.Theﬁrstst( age of BITONIC-SORTER[n]consistsofH0ALF-CLEANER if n =[n],1, which, by Lemma 27.3, produces twoD bitonic(n) = sequences of half the size such thatk every element in the top half is at least as smallD as(n every/2) element + 1 in if then = bottom2 . half. Thus, we can complete the sort by using two copies of BITONIC-SORTER[n/2] to sort the two halves recursively. In Figure 27.9(a), the recursion has been shown explicitly, and in Figure 27.9(b), the recursion has been unrolled to show theprogressivelysmaller half-cleaners that make up the remainder of the bitonic sorter. The depth D(n) of BITONIC-SORTER[n]isgivenbytherecurrence 0ifn 1 , D(nI.) Sorting Networks= Batcher’s Sorting Network 13 = D(n/2) 1ifn 2k and k 1 , ! + = ! whose solution is D(n) lg n. = Thus, a zero-one bitonic sequence can be sorted by BITONIC-SORTER,which has a depth of lg n.Itfollowsbytheanalogofthezero-oneprinciplegivenas Exercise 27.3-6 that any bitonic sequence of arbitrary numbers can be sorted by this network.

Exercises

27.3-1 How many zero-one bitonic sequences of length n are there? BITONIC-SORTER[n] has depth log n and sorts any zero-one bitonic sequence.

The Bitonic27.3 Sorter A bitonic sorting network 715

0 0 0 0 0 0 BITONIC- 0 0 SORTER[n/2] 1 0 0 0 0 0 HALF- 1 0 bitonic sorted CLEANER[n] 1 1 1 0 0 0 BITONIC- 0 1 SORTER[n/2] 0 1 1 1 0 1 1 1 (a) (b)

Figure 27.9 The comparison network BITONIC-SORTER[n], shown here for n 8. (a) The re- = cursive construction: HALF-CLEANER[n]followedbytwocopiesofBITONIC-SORTER[n/2] that operate in parallel. (b) The network after unrolling the recursion. Each half-cleaner is shaded. Sam- ple zero-one values are shown on the wires. Henceforth we will always The bitonic sorter assume that n is a power of 2. Recursive FormulaBy recursively for depth combiningD( half-cleaners,n): as shown in Figure 27.9, we can build a bitonic sorter,whichisanetworkthatsortsbitonicsequences.Theﬁrstst( age of BITONIC-SORTER[n]consistsofH0ALF-CLEANER if n =[n],1, which, by Lemma 27.3, produces twoD bitonic(n) = sequences of half the size such thatk every element in the top half is at least as smallD as(n every/2) element + 1 in if then = bottom2 . half. Thus, we can complete the sort by using two copies of BITONIC-SORTER[n/2] to sort the two halves recursively. In Figure 27.9(a), the recursion has been shown explicitly, and in Figure 27.9(b), the recursion has been unrolled to show theprogressivelysmaller half-cleaners that make up the remainder of the bitonic sorter. The depth D(n) of BITONIC-SORTER[n]isgivenbytherecurrence 0ifn 1 , D(nI.) Sorting Networks= Batcher’s Sorting Network 13 = D(n/2) 1ifn 2k and k 1 , ! + = ! whose solution is D(n) lg n. = Thus, a zero-one bitonic sequence can be sorted by BITONIC-SORTER,which has a depth of lg n.Itfollowsbytheanalogofthezero-oneprinciplegivenas Exercise 27.3-6 that any bitonic sequence of arbitrary numbers can be sorted by this network.

Exercises

27.3-1 How many zero-one bitonic sequences of length n are there? The Bitonic27.3 Sorter A bitonic sorting network 715

0 0 0 0 0 0 BITONIC- 0 0 SORTER[n/2] 1 0 0 0 0 0 HALF- 1 0 bitonic sorted CLEANER[n] 1 1 1 0 0 0 BITONIC- 0 1 SORTER[n/2] 0 1 1 1 0 1 1 1 (a) (b)

Figure 27.9 The comparison network BITONIC-SORTER[n], shown here for n 8. (a) The re- = cursive construction: HALF-CLEANER[n]followedbytwocopiesofBITONIC-SORTER[n/2] that operate in parallel. (b) The network after unrolling the recursion. Each half-cleaner is shaded. Sam- ple zero-one values are shown on the wires. Henceforth we will always The bitonic sorter assume that n is a power of 2. Recursive FormulaBy recursively for depth combiningD( half-cleaners,n): as shown in Figure 27.9, we can build a bitonic sorter,whichisanetworkthatsortsbitonicsequences.Theﬁrstst( age of BITONIC-SORTER[n]consistsofH0ALF-CLEANER if n =[n],1, which, by Lemma 27.3, produces twoD bitonic(n) = sequences of half the size such thatk every element in the top half is at least as smallD as(n every/2) element + 1 in if then = bottom2 . half. Thus, we can complete the sort by using two copies of BITONIC-SORTER[n/2] to sort the two halves recursively. In Figure 27.9(a), the recursion has been shown explicitly, and BITONIC-SORTERin Figure[n] 27.9(b),has depth the recursion log hasn and been unrolled sorts toany show zero-one theprogressivelysmaller bitonic sequence. half-cleaners that make up the remainder of the bitonic sorter. The depth D(n) of BITONIC-SORTER[n]isgivenbytherecurrence 0ifn 1 , D(nI.) Sorting Networks= Batcher’s Sorting Network 13 = D(n/2) 1ifn 2k and k 1 , ! + = ! whose solution is D(n) lg n. = Thus, a zero-one bitonic sequence can be sorted by BITONIC-SORTER,which has a depth of lg n.Itfollowsbytheanalogofthezero-oneprinciplegivenas Exercise 27.3-6 that any bitonic sequence of arbitrary numbers can be sorted by this network.

Exercises

27.3-1 How many zero-one bitonic sequences of length n are there? consider two given sequences X = 00000111, Y = 00001111 concatenating X with Y R (the reversal of Y ) ⇒ 0000011111110000

Basic Idea:

This sequence is bitonic!

Hence in order to merge the sequences X and Y , it suf- ﬁces to perform a bitonic sort on X concatenated with Y R .

Merging Networks

Merging Networks can merge two sorted input sequences into one sorted output sequences will be based on a modiﬁcation of BITONIC-SORTER[n]

I. Sorting Networks Batcher’s Sorting Network 14 consider two given sequences X = 00000111, Y = 00001111 concatenating X with Y R (the reversal of Y ) ⇒ 0000011111110000

This sequence is bitonic!

Hence in order to merge the sequences X and Y , it suf- ﬁces to perform a bitonic sort on X concatenated with Y R .

Merging Networks

Merging Networks can merge two sorted input sequences into one sorted output sequences will be based on a modiﬁcation of BITONIC-SORTER[n]

Basic Idea:

I. Sorting Networks Batcher’s Sorting Network 14 concatenating X with Y R (the reversal of Y ) ⇒ 0000011111110000

This sequence is bitonic!

Hence in order to merge the sequences X and Y , it suf- ﬁces to perform a bitonic sort on X concatenated with Y R .

Merging Networks

Merging Networks can merge two sorted input sequences into one sorted output sequences will be based on a modiﬁcation of BITONIC-SORTER[n]

Basic Idea: consider two given sequences X = 00000111, Y = 00001111

I. Sorting Networks Batcher’s Sorting Network 14 This sequence is bitonic!

Hence in order to merge the sequences X and Y , it suf- ﬁces to perform a bitonic sort on X concatenated with Y R .

Merging Networks

Merging Networks can merge two sorted input sequences into one sorted output sequences will be based on a modiﬁcation of BITONIC-SORTER[n]

Basic Idea: consider two given sequences X = 00000111, Y = 00001111 concatenating X with Y R (the reversal of Y ) ⇒ 0000011111110000

I. Sorting Networks Batcher’s Sorting Network 14 Hence in order to merge the sequences X and Y , it suf- ﬁces to perform a bitonic sort on X concatenated with Y R .

Merging Networks

Merging Networks can merge two sorted input sequences into one sorted output sequences will be based on a modiﬁcation of BITONIC-SORTER[n]

Basic Idea: consider two given sequences X = 00000111, Y = 00001111 concatenating X with Y R (the reversal of Y ) ⇒ 0000011111110000

This sequence is bitonic!

I. Sorting Networks Batcher’s Sorting Network 14 Merging Networks

Merging Networks can merge two sorted input sequences into one sorted output sequences will be based on a modiﬁcation of BITONIC-SORTER[n]

Basic Idea: consider two given sequences X = 00000111, Y = 00001111 concatenating X with Y R (the reversal of Y ) ⇒ 0000011111110000

This sequence is bitonic!

Hence in order to merge the sequences X and Y , it suf- ﬁces to perform a bitonic sort on X concatenated with Y R .

I. Sorting Networks Batcher’s Sorting Network 14 We know it sufﬁces to bitonically sort ha1, a2,..., an/2, an, an−1,..., an/2+1i Recall: ﬁrst half-cleaner of BITONIC-SORTER[n] compares i and n/2 + i ⇒ First part of MERGER[n] compares inputs i and n − i for i = 1, 2,..., n/2 Remaining27.4 A part merging is network identical to BITONIC-SORTER[n] 717

0 0 0 0 a1 b1 a1 b1 0 0 0 0 a2 b2 a2 b2 sorted 1 0 bitonic 1 0 bitonic a3 b3 a3 b3 1 0 1 0 a4 b4 a4 b4 0 1 bitonic 1 1 a5 b5 a8 b8 0 1 0 0 a6 b6 a7 b7 sorted 0 0 bitonic 0 1 bitonic a7 b7 a6 b6 1 1 0 1 a8 b8 a5 b5 (a) (b) Lemma 27.3 still applies, since the reversal of a bitonic sequence is bitonic.

Figure 27.10 Comparing the ﬁrst stage of MERGER[n]withHALF-CLEANER[n], for n 8. = (a) The ﬁrst stage of MERGER[n]transformsthetwomonotonicinputsequences a , a ,...,a ! 1 2 n/2" and an/2 1, an/2 2,...,an into two bitonic sequences b1, b2,...,bn/2 and bn/2 1, bn/2 2, ! + + " ! " ! + + ..., bn . (b) The equivalent operation for HALF-CLEANER[n]. The bitonic input sequence " a1, a2,...,an/2 1, an/2, an, an 1,...,an/2 2, an/2 1 is transformed into the two bitonic se- ! # # + + " quences b1, b2,...,bn/2 and bn, bn 1,...,bn/2 1 . ! " ! # + "

We can construct M ERGER[n]bymodifyingthefirsthalf-cleanerofBITONIC- SORTER[n]. The key is to perform the reversal of the second half of the inputs implicitly. Given two sorted sequences a1, a2,...,an/2 and an/2 1, an/2 2, ..., a to be merged, we want the effect! of bitonically" sorting! the+ sequence+ n" a1, a2,...,an/2, an, an 1,...,an/2 1 .Sincethefirsthalf-cleanerofBITONIC- ! # + " SORTER[n]comparesinputsi and n/2 i,fori 1, 2,...,n/2, we make the first stage of the merging network compare+ inputs =i and n i 1. Figure 27.10 shows the correspondence. The only subtlety is that the orde#roftheoutputsfrom+ the bottom of the first stage of MERGER[n]arereversedcomparedwiththeorder of outputs from an ordinary half-cleaner. Since the reversalofabitonicsequence is bitonic, however, the top and bottom outputs of the first stage of the merging network satisfy the properties in Lemma 27.3, and thus the topandbottomcanbe bitonically sorted in parallel to produce the sorted output of the merging network. The resulting merging network is shown in Figure 27.11. Only the first stage of MERGER[n]isdifferentfromBITONIC-SORTER[n]. Consequently, the depth of MERGER[n]islgn,thesameasthatofBITONIC-SORTER[n].

Exercises

27.4-1 Prove an analog of the zero-one principle for merging networks. Speciﬁcally, show that a comparison network that can merge any two monotonically increasing se-

Construction of a Merging Network (1/2)

Given two sorted sequences ha1, a2,..., an/2i and han/2+1, an/2+2,..., ani

I. Sorting Networks Batcher’s Sorting Network 15 Recall: ﬁrst half-cleaner of BITONIC-SORTER[n] compares i and n/2 + i ⇒ First part of MERGER[n] compares inputs i and n − i for i = 1, 2,..., n/2 Remaining27.4 A part merging is network identical to BITONIC-SORTER[n] 717

Exercises

27.4-1 Prove an analog of the zero-one principle for merging networks. Speciﬁcally, show that a comparison network that can merge any two monotonically increasing se-

Construction of a Merging Network (1/2)

Given two sorted sequences ha1, a2,..., an/2i and han/2+1, an/2+2,..., ani We know it sufﬁces to bitonically sort ha1, a2,..., an/2, an, an−1,..., an/2+1i

I. Sorting Networks Batcher’s Sorting Network 15 ⇒ First part of MERGER[n] compares inputs i and n − i for i = 1, 2,..., n/2 Remaining27.4 A part merging is network identical to BITONIC-SORTER[n] 717

Exercises

27.4-1 Prove an analog of the zero-one principle for merging networks. Speciﬁcally, show that a comparison network that can merge any two monotonically increasing se-

Construction of a Merging Network (1/2)

I. Sorting Networks Batcher’s Sorting Network 15 Remaining27.4 A part merging is network identical to BITONIC-SORTER[n] 717

Exercises

27.4-1 Prove an analog of the zero-one principle for merging networks. Speciﬁcally, show that a comparison network that can merge any two monotonically increasing se-

Construction of a Merging Network (1/2)

Given two sorted sequences ha1, a2,..., an/2i and han/2+1, an/2+2,..., ani We know it sufﬁces to bitonically sort ha1, a2,..., an/2, an, an−1,..., an/2+1i Recall: ﬁrst half-cleaner of BITONIC-SORTER[n] compares i and n/2 + i ⇒ First part of MERGER[n] compares inputs i and n − i for i = 1, 2,..., n/2

I. Sorting Networks Batcher’s Sorting Network 15 Remaining part is identical to BITONIC-SORTER[n]

Lemma 27.3 still applies, since the reversal of a bitonic sequence is bitonic.

Construction of a Merging Network (1/2)

0 0 0 0 a1 b1 a1 b1 0 0 0 0 a2 b2 a2 b2 sorted 1 0 bitonic 1 0 bitonic a3 b3 a3 b3 1 0 1 0 a4 b4 a4 b4 0 1 bitonic 1 1 a5 b5 a8 b8 0 1 0 0 a6 b6 a7 b7 sorted 0 0 bitonic 0 1 bitonic a7 b7 a6 b6 1 1 0 1 a8 b8 a5 b5 (a) (b)

We canI. Sorting construct Networks M ERGER[n Batcher’s]bymodifyingthefirsthalf-cleanerofB Sorting Network ITONIC15 - SORTER[n]. The key is to perform the reversal of the second half of the inputs implicitly. Given two sorted sequences a1, a2,...,an/2 and an/2 1, an/2 2, ..., a to be merged, we want the effect! of bitonically" sorting! the+ sequence+ n" a1, a2,...,an/2, an, an 1,...,an/2 1 .Sincethefirsthalf-cleanerofBITONIC- ! # + " SORTER[n]comparesinputsi and n/2 i,fori 1, 2,...,n/2, we make the first stage of the merging network compare+ inputs =i and n i 1. Figure 27.10 shows the correspondence. The only subtlety is that the orde#roftheoutputsfrom+ the bottom of the first stage of MERGER[n]arereversedcomparedwiththeorder of outputs from an ordinary half-cleaner. Since the reversalofabitonicsequence is bitonic, however, the top and bottom outputs of the first stage of the merging network satisfy the properties in Lemma 27.3, and thus the topandbottomcanbe bitonically sorted in parallel to produce the sorted output of the merging network. The resulting merging network is shown in Figure 27.11. Only the first stage of MERGER[n]isdifferentfromBITONIC-SORTER[n]. Consequently, the depth of MERGER[n]islgn,thesameasthatofBITONIC-SORTER[n].

Exercises

27.4-1 Prove an analog of the zero-one principle for merging networks. Speciﬁcally, show that a comparison network that can merge any two monotonically increasing se- Remaining part is identical to BITONIC-SORTER[n]

Construction of a Merging Network (1/2)

Exercises

Lemma 27.3 still applies, since the reversal of a bitonic sequence is bitonic.

Construction of a Merging Network (1/2)

Exercises

27.4-1 Prove an analog of the zero-one principle for merging networks. Speciﬁcally, show that a comparison network that can merge any two monotonically increasing se- Lemma 27.3 still applies, since the reversal of a bitonic sequence is bitonic.

Construction of a Merging Network (1/2)

Exercises

27.4-1 Prove an analog of the zero-one principle for merging networks. Speciﬁcally, show that a comparison network that can merge any two monotonically increasing se- Construction of a Merging Network (2/2)

718 Chapter 27 Sorting Networks

0 0 0 0 0 0 BITONIC- 0 0 sorted SORTER[n/2] 1 1 1 0 1 0 0 1 sorted 0 1 1 1 1 1 BITONIC- 1 1 sorted SORTER[n/2] 1 1 1 1 1 1 1 1 (a) (b)

Figure 27.11 Anetworkthatmergestwosortedinputsequencesintoonesorted output sequence. The network MERGER[n]canbeviewedasBITONIC-SORTER[n]withtheﬁrsthalf-cleaneralteredto compare inputs i and n i 1fori 1, 2,...,n/2. Here, n 8. (a) The network decomposed into ! + = = the ﬁrst stage followed by two parallel copies of BITONIC-SORTER[n/2]. (b) The same network with the recursion unrolled. Sample zero-one values are shown on the wires, and the stages are shaded.

quences of 0’s and 1’s can merge any two monotonically increasing sequences of arbitrary numbers. I. Sorting Networks Batcher’s Sorting Network 16 27.4-2 How many different zero-one input sequences must be applied to the input of a comparison network to verify that it is a merging network?

27.4-3 Show that any network that can merge 1 item with n 1sorteditemstoproducea sorted sequence of length n must have depth at least lg! n.

27.4-4 ! Consider a merging network with inputs a1, a2,...,an ,forn an exact power of 2, in which the two monotonic sequences to be merged are a1, a3,...,an 1 and " ! # a2, a4,...,an .Provethatthenumberofcomparatorsinthiskindofmerging network" is "(n#lg n).Whyisthisaninterestinglowerbound?(Hint: Partition the comparators into three sets.)

27.4-5 ! Prove that any merging network, regardless of the order of inputs, requires "(n lg n) comparators. merges two sorted input sequences depth log n

718 Chapter 27 Sorting Networks 2.M ERGER[n] 0 0 0 0 0 0 BITONIC- 0 0 sorted SORTER[n/2] 1 1 1 0 1 0 0 1 sorted 2. 0 1 1 1 1 1 BITONIC- 1 1 sorted SORTER[n/2] 1 1 1 1 1 1 1 1 (a) (b)

720 Chapter 27 Sorting Networks Figure 27.11 Anetworkthatmergestwosortedinputsequencesintoonesorted output sequence. Batcher’s Sorting Network The network MERGER[n]canbeviewedasBITONIC-SORTER[n]withthefirsthalf-cleaneralteredto compare inputs i and n i 1fori 1, 2,...,n/2. Here, n 8. (a) The network decomposed into ! + = MERGER[2] = ORTER[ ] the first stage followed by two parallel copies of BITONIC-SORTER[n/2]. (b) The same network with S n is defined recursively: SORTER[n/2] MERGER[4] the recursion unrolled. Sample zero-one values are shown on the wires, and the stages are shaded. If n = 2k , use two copies of SORTER[n/2] to MERGER[2] MERGER[n] MERGER[8] sort two subsequences of length n/2 each. quences of 0’s and 1’s can merge anyMERGER two[2] monotonically increasing sequences of arbitrary numbers. Then merge them using MERGER[n]. SORTER[n/2] MERGER[4] MERGER[2] If n = 1, network consists of a single wire. 27.4-2 How many(a) different zero-one input sequences must be(b) applied to the input of a comparison network to verify that it is a merging network?

can be seen as a parallel version of merge sort 1 0 27.4-30 0 0 1 Show0 that any network that can0 merge 1 item with n 1sorteditemstoproducea ! 1 0 sorted1 sequence of length n must0 have depth at least lg n. 0 1 1 0 1 0 27.4-40 ! 0 0 1 Consider0 a merging network with1 inputs a1, a2,...,an ,forn an exact power of 2, 0 0 in which0 the two monotonic1 sequences to be merged are a1, a3,...,an 1 and " ! # 0 0 a , a1 ,...,a .Provethatthenumberofcomparatorsinthiskindofmerging1 " 2 4 n# depth 1 2 2network3 4 is4"(4n lg4 n5).Whyisthisaninterestinglowerbound?(5 6 Hint: Partition the comparators(c) into three sets.)

27.4-5 ! FigureProve 27.12 thatThe any sorting merging network network, SORTER[n regardless]constructedbyrecursivelycombiningmergingnet- of the order of inputs, requires works."(n(a)lg nThe) comparators. recursive construction. (b) Unrolling the recursion. (c) Replacing the MERGER boxes with the actual merging networks. The depth of each comparator is indicated, and sample zero-one values are shown on the wires.

27.5-2 Show that the depth of SORTER[n]isexactly(lg n)(lg n 1)/2. + 27.5-3 Suppose that we have 2n elements a1, a2,...,a2n and wish to partition them into the n smallest and the n largest. Prove! that we can" do this in constant additional depth after separately sorting a1, a2,...,an and an 1, an 2,...,a2n . ! " ! + + " 27.5-4 ! Let S(k) be the depth of a sorting network with k inputs, and let M(k) be the depth of a merging network with 2k inputs. Suppose that we have a sequence of n numbers to be sorted and we know that every number is within k positions of its correct position in the sorted order. Show that we can sort the n numbers in depth S(k) 2M(k). +

Construction of a Sorting Network 27.3 A bitonic sorting network 715

0 0 0 0 0 0 Main Components BITONIC- 0 0 SORTER[n/2] 1 0 0 0 0 0 HALF- 1 0 1.B ITONIC-SORTER[n] 1. bitonic sorted CLEANER[n] 1 1 1 0 0 0 sorts any bitonic sequence BITONIC- 0 1 depth log n SORTER[n/2] 0 1 1 1 0 1 1 1 (a) (b)

The bitonic sorter By recursively combining half-cleaners, as shown in Figure 27.9, we can build a bitonic sorter,whichisanetworkthatsortsbitonicsequences.Theﬁrststage of BITONIC-SORTER[n]consistsofHALF-CLEANER[n], which, by Lemma 27.3, produces two bitonic sequences of half the size such that every element in the top half is at least as small as every element in the bottom half. Thus, we can complete the sort by using two copies of BITONIC-SORTER[n/2] to sort the two halves recursively. In Figure 27.9(a), the recursion has been shown explicitly, and in Figure 27.9(b), the recursion has been unrolled to show theprogressivelysmaller half-cleaners that make up the remainder of the bitonic sorter. The depth D(n) of BITONIC-SORTER[n]isgivenbytherecurrence 0ifn 1 , D(n) = = D(n/2) 1ifn 2k and k 1 , ! + = ! whose solution is D(n) lg n. = Thus, a zero-one bitonic sequence can be sorted by BITONIC-SORTER,which has a depth of lg n.Itfollowsbytheanalogofthezero-oneprinciplegivenas Exercise 27.3-6 that any bitonic sequence of arbitrary numbers can be sorted by this network. I. Sorting Networks Batcher’s Sorting Network 17 Exercises

27.3-1 How many zero-one bitonic sequences of length n are there? 720 Chapter 27 Sorting Networks Batcher’s Sorting Network ORTER[ ] MERGER[2] S n is deﬁned recursively: SORTER[n/2] MERGER[4] If n = 2k , use two copies of SORTER[n/2] to MERGER[2] MERGER[n] MERGER[8] sort two subsequences of length n/2 each. MERGER[2] Then merge them using MERGER[n]. SORTER[n/2] MERGER[4] MERGER[2] If n = 1, network consists of a single wire. (a) (b)

can be seen as a parallel version of merge sort 1 0 0 0 0 1 0 0 1 0 1 0 0 1 1 0 1 0 0 0 0 1 0 1 0 0 0 1 0 0 1 1 depth 1 2 2 3 4 4 4 4 5 5 6 (c)

Figure 27.12 The sorting network SORTER[n]constructedbyrecursivelycombiningmergingnet- works. (a) The recursive construction. (b) Unrolling the recursion. (c) Replacing the MERGER boxes with the actual merging networks. The depth of each comparator is indicated, and sample zero-one values are shown on the wires.

Construction of a Sorting Network 27.3 A bitonic sorting network 715

0 0 0 0 0 0 Main Components BITONIC- 0 0 SORTER[n/2] 1 0 0 0 0 0 HALF- 1 0 1.B ITONIC-SORTER[n] 1. bitonic sorted CLEANER[n] 1 1 1 0 0 0 sorts any bitonic sequence BITONIC- 0 1 SORTER[n/2] 1 1 depth log n 718 Chapter 27 Sorting Networks 0 1 0 1 1 1 2.M ERGER[n] (a) (b) 0 0 0 0 0 0 merges two sorted input sequences BITONIC- 0 0 sorted Figure 27.9SORTERThe[n/2] comparison network BITONIC-SORTER[n], shown1 here for1n 8. (a) The re- 1 = 0 depth log n cursive construction: HALF-CLEANER[n]followedbytwocopiesofBITONIC-SORTER[n/2] that 1 0 0 1 operate in parallel. (b) The network after unrolling the recursion. Each half-cleaner is shaded. Sam- 2. 1 1 sorted ple zero-one values are shown on the wires. 0 1 1 1 BITONIC- 1 1 sorted SORTER[n/2] 1 1 The bitonic sorter 1 1 1 1 1 1 By(a) recursively combining half-cleaners, as shown in Figure(b) 27.9, we can build a bitonic sorter,whichisanetworkthatsortsbitonicsequences.Thefirststage of BITONIC-SORTER[n]consistsofHALF-CLEANER[n], which, by Lemma 27.3, producesFigure 27.11 twoAnetworkthatmergestwosortedinputsequencesintoonesor bitonic sequences of half the size such that everyted element output sequence. in the topThenetwork half is M atERGER least[n as]canbeviewedasB small as everyITONIC element-SORTER in the[n]withthefirsthalf-cleaneralteredto bottom half. Thus, we can compare inputs i and n i 1fori 1, 2,...,n/2. Here, n 8. (a) The network decomposed into ! + = ITONIC= ORTER / completethe first stage the followed sort by by usingtwo parallel two copies copies of ofBITONIC B -SORTER-S [n/2]. (b)[n The2] same to sort network the two with halvesthe recursion recursively. unrolled.In Sample Figure zero-one 27.9(a), values the are recursion shown on the has wires, been and shown the stages explicitly, are shaded. and in Figure 27.9(b), the recursion has been unrolled to show theprogressivelysmaller half-cleaners that make up the remainder of the bitonic sorter. The depth D(n) of quences of 0’s and 1’s can merge any two monotonically increasing sequences of BITONIC-SORTER[n]isgivenbytherecurrence arbitrary numbers. 0ifn 1 , D27.4-2(n) = k = D(n/2) 1ifn 2 and k 1 , How many! different+ zero-one= input sequences! must be applied to the input of a whosecomparison solution network is D(n to) verifylg n. that it is a merging network? = Thus, a zero-one bitonic sequence can be sorted by BITONIC-SORTER,which has27.4-3 a depth of lg n.Itfollowsbytheanalogofthezero-oneprinciplegivenas ExerciseShow that 27.3-6 any network that any that bitonic can merge sequence 1 item of witharbitraryn 1sorteditemstoproducea numbers can be sorted by ! thissorted network. sequence of length n must have depth at least lg n. I. Sorting Networks Batcher’s Sorting Network 27.4-4 !17 Exercises Consider a merging network with inputs a1, a2,...,an ,forn an exact power of 2, in which the two monotonic sequences to be merged are a1, a3,...,an 1 and 27.3-1 " ! # Howa2, a many4,..., zero-onean .Provethatthenumberofcomparatorsinthiskindofmerging bitonic sequences of length n are there? network" is "(n#lg n).Whyisthisaninterestinglowerbound?(Hint: Partition the comparators into three sets.)

27.4-5 ! Prove that any merging network, regardless of the order of inputs, requires "(n lg n) comparators. can be seen as a parallel version of merge sort

Construction of a Sorting Network 27.3 A bitonic sorting network 715

0 0 0 0 0 0 Main Components BITONIC- 0 0 SORTER[n/2] 1 0 0 0 0 0 HALF- 1 0 1.B ITONIC-SORTER[n] 1. bitonic sorted CLEANER[n] 1 1 1 0 0 0 sorts any bitonic sequence BITONIC- 0 1 SORTER[n/2] 1 1 depth log n 718 Chapter 27 Sorting Networks 0 1 0 1 1 1 2.M ERGER[n] (a) (b) 0 0 0 0 0 0 merges two sorted input sequences BITONIC- 0 0 sorted Figure 27.9SORTERThe[n/2] comparison network BITONIC-SORTER[n], shown1 here for1n 8. (a) The re- 1 = 0 depth log n cursive construction: HALF-CLEANER[n]followedbytwocopiesofBITONIC-SORTER[n/2] that 1 0 0 1 operate in parallel. (b) The network after unrolling the recursion. Each half-cleaner is shaded. Sam- 2. 1 1 sorted ple zero-one values are shown on the wires. 0 1 1 1 BITONIC- 1 1 sorted SORTER[n/2] 1 1 The bitonic sorter 1 1 1 1 1 1 By(a) recursively combining half-cleaners, as shown in Figure(b) 27.9, we can build a bitonic sorter,whichisanetworkthatsortsbitonicsequences.Thefirststage 720 Chapterof BITONIC 27 Sorting-SORTER Networks[n]consistsofHALF-CLEANER[n], which, by Lemma 27.3, Batcher’s Sorting Network producesFigure 27.11 twoAnetworkthatmergestwosortedinputsequencesintoonesor bitonic sequences of half the size such that everyted element output sequence. in the topThenetwork half is M atERGER least[n as]canbeviewedasB small as everyITONIC element-SORTER in the[n]withthefirsthalf-cleaneralteredto bottom half. Thus, we can compare inputs i and n i 1fori 1, 2,...,n/2. Here, n 8. (a) The network decomposed into complete the sort by! using+ two= copiesMERGER of B[2]ITONIC=-SORTER[n/2] to sort the two ORTER[ ] the first stage followed by two parallel copies of BITONIC-SORTER[n/2]. (b) The same network with S n is defined recursively: SORTER[n/2] MERGER[4] halvesthe recursion recursively. unrolled.In Sample Figure zero-one 27.9(a), values the are recursion shown on the has wires, been and shown the stages explicitly, are shaded. and If n = 2k , use two copies of SORTER[n/2] to in Figure 27.9(b), the recursion has beenMERGER unrolled[2] to show theprogressivelysmaller half-cleanersM thatERGER make[n] up the remainder of the bitonic sorter. TheMERGER depth[8] D(n) of quences of 0’s and 1’s can merge any two monotonically increasing sequences of sort two subsequences of length n/2 each. BITONIC-SORTER[n]isgivenbytherecurrenceMERGER[2] arbitrary numbers. Then merge them using MERGER[n]. SORTER[n/2] MERGER[4] 0ifn 1 , MERGER[2] D27.4-2(n) = k If n = 1, network consists of a single wire. = D(n/2) 1ifn 2 and k 1 , How many(a)! different+ zero-one= input sequences! must be(b) applied to the input of a whosecomparison solution network is D(n to) verifylg n. that it is a merging network? = Thus, a zero-one bitonic sequence can be sorted by BITONIC-SORTER,which 1 0 has27.4-3 a0 depth of lg n.Itfollowsbytheanalogofthezero-oneprinciplegivenas0 0 1 ExerciseShow0 that 27.3-6 any network that any that bitonic can0 merge sequence 1 item of witharbitraryn 1sorteditemstoproducea numbers can be sorted by ! 1 0 thissorted network.1 sequence of length n must0 have depth at least lg n. 1 1 I. Sorting Networks Batcher’s Sorting Network 0 0 0 27.4-40 !17 1 Exercises 0 0 1 Consider0 a merging network with1 inputs a1, a2,...,an ,forn an exact power of 2, 0 0 in which0 the two monotonic1 sequences to be merged are a1, a3,...,an 1 and 27.3-1 " ! # 0 0 a , a1 ,...,a .Provethatthenumberofcomparatorsinthiskindofmerging1 How" 2 many4 zero-onen# bitonic sequences of length n are there? depth 1 2 2network3 4 is4"(4n lg4 n5).Whyisthisaninterestinglowerbound?(5 6 Hint: Partition the comparators(c) into three sets.)

0 0 0 0 0 0 Main Components BITONIC- 0 0 SORTER[n/2] 1 0 0 0 0 0 HALF- 1 0 1.B ITONIC-SORTER[n] 1. bitonic sorted CLEANER[n] 1 1 1 0 0 0 sorts any bitonic sequence BITONIC- 0 1 SORTER[n/2] 1 1 depth log n 718 Chapter 27 Sorting Networks 0 1 0 1 1 1 2.M ERGER[n] (a) (b) 0 0 0 0 0 0 merges two sorted input sequences BITONIC- 0 0 sorted Figure 27.9SORTERThe[n/2] comparison network BITONIC-SORTER[n], shown1 here for1n 8. (a) The re- 1 = 0 depth log n cursive construction: HALF-CLEANER[n]followedbytwocopiesofBITONIC-SORTER[n/2] that 1 0 0 1 operate in parallel. (b) The network after unrolling the recursion. Each half-cleaner is shaded. Sam- 2. 1 1 sorted ple zero-one values are shown on the wires. 0 1 1 1 BITONIC- 1 1 sorted SORTER[n/2] 1 1 The bitonic sorter 1 1 1 1 1 1 By(a) recursively combining half-cleaners, as shown in Figure(b) 27.9, we can build a bitonic sorter,whichisanetworkthatsortsbitonicsequences.Thefirststage 720 Chapterof BITONIC 27 Sorting-SORTER Networks[n]consistsofHALF-CLEANER[n], which, by Lemma 27.3, Batcher’s Sorting Network producesFigure 27.11 twoAnetworkthatmergestwosortedinputsequencesintoonesor bitonic sequences of half the size such that everyted element output sequence. in the topThenetwork half is M atERGER least[n as]canbeviewedasB small as everyITONIC element-SORTER in the[n]withthefirsthalf-cleaneralteredto bottom half. Thus, we can compare inputs i and n i 1fori 1, 2,...,n/2. Here, n 8. (a) The network decomposed into complete the sort by! using+ two= copiesMERGER of B[2]ITONIC=-SORTER[n/2] to sort the two ORTER[ ] the first stage followed by two parallel copies of BITONIC-SORTER[n/2]. (b) The same network with S n is defined recursively: SORTER[n/2] MERGER[4] halvesthe recursion recursively. unrolled.In Sample Figure zero-one 27.9(a), values the are recursion shown on the has wires, been and shown the stages explicitly, are shaded. and If n = 2k , use two copies of SORTER[n/2] to in Figure 27.9(b), the recursion has beenMERGER unrolled[2] to show theprogressivelysmaller half-cleanersM thatERGER make[n] up the remainder of the bitonic sorter. TheMERGER depth[8] D(n) of quences of 0’s and 1’s can merge any two monotonically increasing sequences of sort two subsequences of length n/2 each. BITONIC-SORTER[n]isgivenbytherecurrenceMERGER[2] arbitrary numbers. Then merge them using MERGER[n]. SORTER[n/2] MERGER[4] 0ifn 1 , MERGER[2] D27.4-2(n) = k If n = 1, network consists of a single wire. = D(n/2) 1ifn 2 and k 1 , How many(a)! different+ zero-one= input sequences! must be(b) applied to the input of a whosecomparison solution network is D(n to) verifylg n. that it is a merging network? = Thus, a zero-one bitonic sequence can be sorted by BITONIC-SORTER,which can be seen as a parallel version of merge sort 1 0 has27.4-3 a0 depth of lg n.Itfollowsbytheanalogofthezero-oneprinciplegivenas0 0 1 ExerciseShow0 that 27.3-6 any network that any that bitonic can0 merge sequence 1 item of witharbitraryn 1sorteditemstoproducea numbers can be sorted by ! 1 0 thissorted network.1 sequence of length n must0 have depth at least lg n. 1 1 I. Sorting Networks Batcher’s Sorting Network 0 0 0 27.4-40 !17 1 Exercises 0 0 1 Consider0 a merging network with1 inputs a1, a2,...,an ,forn an exact power of 2, 0 0 in which0 the two monotonic1 sequences to be merged are a1, a3,...,an 1 and 27.3-1 " ! # 0 0 a , a1 ,...,a .Provethatthenumberofcomparatorsinthiskindofmerging1 How" 2 many4 zero-onen# bitonic sequences of length n are there? depth 1 2 2network3 4 is4"(4n lg4 n5).Whyisthisaninterestinglowerbound?(5 6 Hint: Partition the comparators(c) into three sets.)

720 Chapter 27 Sorting Networks MERGER[2] SORTER[n/2] MERGER[4] MERGER[2]MERGER[2] SORTER[n/2] MERGER[n] MERGER[4] MERGER[8] MERGER[2] MERGER[2] MERGER[n] MERGER[8] SORTER[n/2] MERGER[4] MERGER[2] MERGER[2] SORTER[n/2] MERGER[4] MERGER[2] (a) (b) (a) (b)

1 0 0 0 1 0 0 0 0 1 0 0 Recursion for D(n): 0 1 0 0 1 0 1 0 1 0 1 0 1 1 ( 00 1 1 0 0 0 if n = 1, 0 0 D(n) = 11 0 0 0 0 D(n/2) + log n if n = 2k . 00 11 0 0 1 1 00 0 0 00 1 1 2 0 1 Solution: D(n) = Θ(log n). 00 0 1 1 1 depth 1 2 2 3 4 4 4 4 5 5 6 depth 1 2 2 3 4 4 4 4 5 5 6 (c) (c) SORTER[n] has depth Θ(log2 n) and sorts any input. Figure 27.12 The sorting network SORTER[n]constructedbyrecursivelycombiningmergingnet- works.Figure(a) 27.12The recursiveThe sortingconstruction. network(b) Unrolling SORTER the[n]constructedbyrecursivelycombiningmergingnet- recursion. (c) Replacing the MERGER boxes withworks. the actual(a) The merging recursive networks. construction. The depth(b) of eachUnrolling comparat theor recursion. is indicated,(c) andReplacing sample zero-one the MERGER boxes valueswith arethe shown actual on merging the wires. networks. The depth of each comparator is indicated, and sample zero-one values are shown on the wires. 27.5-2 Show that the depth of SORTER[n]isexactly(lg n)(lg n 1)/2. 27.5-2 + 27.5-3Show that the depth of SORTER[n]isexactly(lg n)(lg n 1)/2. + Suppose that we have 2n elements a1, a2,...,a2n and wish to partition them into the27.5-3n smallest and the n largest. Prove! that we can" do this in constant additional depthSuppose after separatelythat we have sorting 2n elementsa1, a2,...,an, aand,...,an 1a, an and2,..., wisha2n to. partition them into ! 1" 2 ! + 2n + " the n smallest and the n largest. Prove! that we can" do this in constant additional 27.5-4 ! depth after separately sorting a1, a2,...,an and an 1, an 2,...,a2n . Let S(k) be the depth of a sorting! network with k" inputs,! and+ let +M(k) be the" depth of a merging network with 2k inputs. Suppose that we have a sequence of n numbers27.5-4 to! be sorted and we know that every number is within k positions of its correctLet S position(k) be thein the depth sorted of order. a sorting Show that network we can with sort thkeinputs,n numbers and in letdepthM(k) be the Sdepth(k) 2 ofM( ak merging). network with 2k inputs. Suppose that we have a sequence of n numbers+ to be sorted and we know that every number is within k positions of its correct position in the sorted order. Show that we can sort the n numbers in depth S(k) 2M(k). +

720Unrolling Chapter the 27 Recursion Sorting Networks (Figure 27.12)

MERGER[2] SORTER[n/2] MERGER[4] MERGER[2] MERGER[n] MERGER[8] MERGER[2] SORTER[n/2] MERGER[4] MERGER[2]

(a) (b)

1 0 0 0 0 1 0 0 1 0 1 0 0 1 1 0 1 0 0 0 0 1 0 1 0 0 0 1 0 0 1 1 depth 1 2 2 3 4 4 4 4 5 5 6 (c)

MERGER[2] SORTER[n/2] MERGER[4] MERGER[2] MERGER[n] MERGER[8] MERGER[2] SORTER[n/2] MERGER[4] MERGER[2]

(a) (b)

1 0 0 0 0 1 0 0 Recursion for D(n): 1 0 1 0 ( 0 1 1 0 0 if n = 1, 0 0 D(n) = 1 0 D(n/2) + log n if n = 2k . 0 1 0 1 0 0 0 1 Solution: D(n) = Θ(log2 n). 0 0 1 1 depth 1 2 2 3 4 4 4 4 5 5 6 (c) SORTER[n] has depth Θ(log2 n) and sorts any input.

720720Unrolling Chapter Chapterthe 27 Recursion 27 Sorting Sorting Networks Networks (Figure 27.12)

MERGERMERGER[2] [2] SORTERSORTER[n/2][n/2] MERGERMERGER[4] [4] MERGERMERGER[2] [2] MERGERMERGER[n] [n] MERGERMERGER[8] [8] MERGERMERGER[2] [2] SORTERSORTER[n/2][n/2] MERGERMERGER[4] [4] MERGERMERGER[2] [2]

(a) (a) (b) (b)

1 1 0 0 0 0 0 0 0 0 1 1 0 0 0 0 1 1 0 0 1 1 0 0 0 0 1 1 1 1 0 0 1 1 0 0 0 0 0 0 0 0 1 1 0 0 1 1 0 0 0 0 0 0 1 1 0 0 0 0 1 1 1 1 depthdepth1 1 2 22 32 3 4 44 44 44 54 55 65 6 (c) (c)

27.5-227.5-2 ShowShow that that the thedepth depth of S ofORTER SORTER[n]isexactly[n]isexactly(lg n(lg)(lgn)(nlg n1)/2.1)/2. + + 27.5-327.5-3 SupposeSuppose that that we havewe have 2n elements 2n elementsa , a ,...,, a ,...,a aandand wish wish to partition to partition them them into into 1 ! 21 2 2n 2n" the then smallestn smallest and and the then largest.n largest. Prove! Prove that that we wecan" can do this do this in constant in constant additional additional depthdepth after after separately separately sorting sortinga1, a21,...,, a2,...,an aandn andan 1a,nan1, 2a,...,n 2,...,a2n a.2n . ! ! " " ! +! + + + " " 27.5-427.5-4! ! LetLetS(k)S(bek) thebe the depth depth of a of sorting a sorting network network with withk inputs,k inputs, and and let letM(kM) (bek) thebe the depthdepth of a of merging a merging network network with with 2k inputs. 2k inputs. Suppose Suppose that that we havewe have a sequence a sequence of n of n numbersnumbers to be to sorted be sorted and and we weknow know that that every every number number is within is withink positionsk positions of its of its correctcorrect position position in the in thesorted sorted order. order. Show Show that that we canwe can sort sort the thn enumbersn numbers in depth in depth S(k)S(k)2M2(kM).(k). + + Recursion for D(n): ( 0 if n = 1, D(n) = D(n/2) + log n if n = 2k .

Solution: D(n) = Θ(log2 n).

SORTER[n] has depth Θ(log2 n) and sorts any input.

720 Chapter 27 Sorting Networks

720720Unrolling Chapter Chapterthe 27 Recursion 27 Sorting Sorting Networks Networks (Figure 27.12) MERGER[2] SORTER[n/2] MERGER[4]

MERGERMERGER[2] [2]MERGER[2] SORTERSORTER[n/2][n/2] MERGER[n] MERGERMERGER[4] [4] MERGER[8] MERGERMERGER[2] [2] MERGER[2] MERGERMERGER[n] [n] MERGERMERGER[8] [8] SORTER[n/2] MERGER[4] MERGERMERGER[2] [2] MERGER[2] SORTERSORTER[n/2][n/2] MERGERMERGER[4] [4] MERGER[2] (a) MERGER[2] (b) (a) (a) (b) (b)

1 0 0 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 1 0 1 0 0 1 1 0 1 1 1 0 0 00 1 1 1 1 0 0 0 0 0 1 11 0 0 0 0 0 0 0 1 0 0 00 1 1 0 0 1 1 1 0 00 0 00 0 0 0 1 1 1 0 0 1 1 0 00 0 1 1 1 1 depth 1 2 2 3 4 4 4 4 5 5 6 depthdepth1 1 2 2 23 2 34 4 44 44 54 5 46 5 5 6 (c) (c) (c)

FigureFigure 27.12 27.12TheThe sorting sorting network network SORTER SORTER[n]constructedbyrecursivelycombiningmergingnet-[n]constructedbyrecursivelycombiningmergingnet- works.works.Figure(a) The(a) 27.12The recursive recursiveThe construction. sortingconstruction. network(b) Unrolling(b) Unrolling SORTER the recursion. the[n]constructedbyrecursivelycombiningmergingnet- recursion.(c) Replacing(c) Replacing the M theERGER MERGERboxesboxes withwith theworks. actual the actual(a) mergingThe merging recursive networks. networks. construction. The Thedepth depth of each(b) of eachUnrolling comparat comparator the isor indicated, recursion. is indicated, and(c) sampleandReplacing sample zero-one zero-one the MERGER boxes valuesvalueswith areI. shown arethe Sorting shown actual on Networks the on merging wires. the wires. networks. Batcher’s The Sorting depth Network of each comparator is indicated, and18 sample zero-one values are shown on the wires. 27.5-227.5-2 ShowShow that that the thedepth depth of S ofORTER SORTER[n]isexactly[n]isexactly(lg n(lg)(lgn)(nlg n1)/2.1)/2. 27.5-2 + + 27.5-327.5-3Show that the depth of SORTER[n]isexactly(lg n)(lg n 1)/2. SupposeSuppose that that we havewe have 2n elements 2n elementsa , a ,...,, a ,...,a aandand wish wish to partition to partition+ them them into into 1 ! 21 2 2n 2n" the then27.5-3smallestn smallest and and the then largest.n largest. Prove! Prove that that we wecan" can do this do this in constant in constant additional additional depthdepthSuppose after after separately separatelythat we sorting have sorting 2an1,elementsa21,...,, a2,...,an aand1n, aand2a,...,n 1a,nan1a,22an,...,n and2,...,a2 wishn a.2n to. partition them into ! ! " " ! +! + + + " " the n smallest and the n largest. Prove! that we can" do this in constant additional 27.5-427.5-4! ! depth after separately sorting a1, a2,...,an and an 1, an 2,...,a2n . LetLetS(k)S(bek) thebe the depth depth of a of sorting a sorting network! network with withk inputs,k" inputs, and! and+ let letM(+kM) (bek) thebe the" depthdepth of a of merging a merging network network with with 2k inputs. 2k inputs. Suppose Suppose that that we havewe have a sequence a sequence of n of n numbersnumbers27.5-4 to be to! sorted be sorted and and we weknow know that that every every number number is within is withink positionsk positions of its of its correctcorrectLet positionS position(k) be in the thein thesorted depth sorted order. of order. a Show sorting Show that that networkwe canwe can sort with sort the thnkenumbersinputs,n numbers in and depth in letdepthM(k) be the S(k)Sdepth(k)2M2( ofkM).( ak merging). network with 2k inputs. Suppose that we have a sequence of n numbers+ + to be sorted and we know that every number is within k positions of its correct position in the sorted order. Show that we can sort the n numbers in depth S(k) 2M(k). + Solution: D(n) = Θ(log2 n).

SORTER[n] has depth Θ(log2 n) and sorts any input.

720 Chapter 27 Sorting Networks

720720Unrolling Chapter Chapterthe 27 Recursion 27 Sorting Sorting Networks Networks (Figure 27.12) MERGER[2] SORTER[n/2] MERGER[4]

1 0 0 0 1 0 0 0 0 0 1 0 1 0 0 0 Recursion for D(n): 0 0 1 1 0 0 0 0 1 0 1 0 1 0 0 1 1 0 1 1 1 0 ( 0 00 1 1 1 1 0 0 0 0 if n = 1, 0 0 D(n) = 1 11 0 0 0 0 0 0 0 k 1 0 D(n/2) + log n if n = 2 . 0 00 1 1 0 0 1 1 1 0 00 0 00 0 0 0 1 1 1 0 0 1 1 0 00 0 1 1 1 1 depth 1 2 2 3 4 4 4 4 5 5 6 depthdepth1 1 2 2 23 2 34 4 44 44 54 5 46 5 5 6 (c) (c) (c)

720 Chapter 27 Sorting Networks

720720Unrolling Chapter Chapterthe 27 Recursion 27 Sorting Sorting Networks Networks (Figure 27.12) MERGER[2] SORTER[n/2] MERGER[4]

1 0 0 0 1 0 0 0 0 0 1 0 1 0 0 0 Recursion for D(n): 0 0 1 1 0 0 0 0 1 0 1 0 1 0 0 1 1 0 1 1 1 0 ( 0 00 1 1 1 1 0 0 0 0 if n = 1, 0 0 D(n) = 1 11 0 0 0 0 0 0 0 k 1 0 D(n/2) + log n if n = 2 . 0 00 1 1 0 0 1 1 1 0 00 0 0 0 0 00 1 1 1 2 0 0 1 1 Solution: D(n) = Θ(log n). 0 00 0 1 1 1 1 depth 1 2 2 3 4 4 4 4 5 5 6 depthdepth1 1 2 2 23 2 34 4 44 44 54 5 46 5 5 6 (c) (c) (c) SORTER[n] has depth Θ(log2 n) and sorts any input. FigureFigure 27.12 27.12TheThe sorting sorting network network SORTER SORTER[n]constructedbyrecursivelycombiningmergingnet-[n]constructedbyrecursivelycombiningmergingnet- works.works.Figure(a) The(a) 27.12The recursive recursiveThe construction. sortingconstruction. network(b) Unrolling(b) Unrolling SORTER the recursion. the[n]constructedbyrecursivelycombiningmergingnet- recursion.(c) Replacing(c) Replacing the M theERGER MERGERboxesboxes withwith theworks. actual the actual(a) mergingThe merging recursive networks. networks. construction. The Thedepth depth of each(b) of eachUnrolling comparat comparator the isor indicated, recursion. is indicated, and(c) sampleandReplacing sample zero-one zero-one the MERGER boxes valuesvalueswith areI. shown arethe Sorting shown actual on Networks the on merging wires. the wires. networks. Batcher’s The Sorting depth Network of each comparator is indicated, and18 sample zero-one values are shown on the wires. 27.5-227.5-2 ShowShow that that the thedepth depth of S ofORTER SORTER[n]isexactly[n]isexactly(lg n(lg)(lgn)(nlg n1)/2.1)/2. 27.5-2 + + 27.5-327.5-3Show that the depth of SORTER[n]isexactly(lg n)(lg n 1)/2. SupposeSuppose that that we havewe have 2n elements 2n elementsa , a ,...,, a ,...,a aandand wish wish to partition to partition+ them them into into 1 ! 21 2 2n 2n" the then27.5-3smallestn smallest and and the then largest.n largest. Prove! Prove that that we wecan" can do this do this in constant in constant additional additional depthdepthSuppose after after separately separatelythat we sorting have sorting 2an1,elementsa21,...,, a2,...,an aand1n, aand2a,...,n 1a,nan1a,22an,...,n and2,...,a2 wishn a.2n to. partition them into ! ! " " ! +! + + + " " the n smallest and the n largest. Prove! that we can" do this in constant additional 27.5-427.5-4! ! depth after separately sorting a1, a2,...,an and an 1, an 2,...,a2n . LetLetS(k)S(bek) thebe the depth depth of a of sorting a sorting network! network with withk inputs,k" inputs, and! and+ let letM(+kM) (bek) thebe the" depthdepth of a of merging a merging network network with with 2k inputs. 2k inputs. Suppose Suppose that that we havewe have a sequence a sequence of n of n numbersnumbers27.5-4 to be to! sorted be sorted and and we weknow know that that every every number number is within is withink positionsk positions of its of its correctcorrectLet positionS position(k) be in the thein thesorted depth sorted order. of order. a Show sorting Show that that networkwe canwe can sort with sort the thnkenumbersinputs,n numbers in and depth in letdepthM(k) be the S(k)Sdepth(k)2M2( ofkM).( ak merging). network with 2k inputs. Suppose that we have a sequence of n numbers+ + to be sorted and we know that every number is within k positions of its correct position in the sorted order. Show that we can sort the n numbers in depth S(k) 2M(k). + Quite elaborate construction, and involves huges constants.

Perfect Halver A perfect halver is a comparator network that, given any input, places the n/2 smaller keys in b1,..., bn/2 and the n/2 larger keys in bn/2+1,..., bn.

2 Perfect halver of depth log2 n exist yields sorting networks of depth Θ((log n) ).

Approximate Halver An (n, )-approximate halver, < 1, is a comparator network that for every k = 1, 2,..., n/2 places at most k of its k smallest keys in bn/2+1,..., bn and at most k of its k largest keys in b1,..., bn/2.

We will prove that such networks can be constructed in constant depth!

A Glimpse at the AKS Network

Ajtai, Komlós, Szemerédi (1983) There exists a sorting network with depth O(log n).

I. Sorting Networks Batcher’s Sorting Network 19 Perfect Halver A perfect halver is a comparator network that, given any input, places the n/2 smaller keys in b1,..., bn/2 and the n/2 larger keys in bn/2+1,..., bn.

2 Perfect halver of depth log2 n exist yields sorting networks of depth Θ((log n) ).

We will prove that such networks can be constructed in constant depth!

A Glimpse at the AKS Network

Ajtai, Komlós, Szemerédi (1983) There exists a sorting network with depth O(log n).

Quite elaborate construction, and involves huges constants.

I. Sorting Networks Batcher’s Sorting Network 19 Quite elaborate construction, and involves huges constants.

2 Perfect halver of depth log2 n exist yields sorting networks of depth Θ((log n) ).

We will prove that such networks can be constructed in constant depth!

A Glimpse at the AKS Network

Ajtai, Komlós, Szemerédi (1983) There exists a sorting network with depth O(log n).

Perfect Halver A perfect halver is a comparator network that, given any input, places the n/2 smaller keys in b1,..., bn/2 and the n/2 larger keys in bn/2+1,..., bn.

I. Sorting Networks Batcher’s Sorting Network 19 Quite elaborate construction, and involves huges constants.

We will prove that such networks can be constructed in constant depth!

A Glimpse at the AKS Network

Ajtai, Komlós, Szemerédi (1983) There exists a sorting network with depth O(log n).

Perfect Halver A perfect halver is a comparator network that, given any input, places the n/2 smaller keys in b1,..., bn/2 and the n/2 larger keys in bn/2+1,..., bn.

2 Perfect halver of depth log2 n exist yields sorting networks of depth Θ((log n) ).

I. Sorting Networks Batcher’s Sorting Network 19 Quite elaborate construction, and involves huges constants.

2 Perfect halver of depth log2 n exist yields sorting networks of depth Θ((log n) ).

We will prove that such networks can be constructed in constant depth!

A Glimpse at the AKS Network

Ajtai, Komlós, Szemerédi (1983) There exists a sorting network with depth O(log n).

Perfect Halver A perfect halver is a comparator network that, given any input, places the n/2 smaller keys in b1,..., bn/2 and the n/2 larger keys in bn/2+1,..., bn.

I. Sorting Networks Batcher’s Sorting Network 19 Quite elaborate construction, and involves huges constants.

2 Perfect halver of depth log2 n exist yields sorting networks of depth Θ((log n) ).

A Glimpse at the AKS Network

Ajtai, Komlós, Szemerédi (1983) There exists a sorting network with depth O(log n).

Perfect Halver A perfect halver is a comparator network that, given any input, places the n/2 smaller keys in b1,..., bn/2 and the n/2 larger keys in bn/2+1,..., bn.

We will prove that such networks can be constructed in constant depth!

I. Sorting Networks Batcher’s Sorting Network 19