Sorting Networks A Lecture in CE Freshman Seminar Series: Ten Puzzling Problems in Computer Engineering

May 2020 Sorting Networks Slide 1 About This Presentation

This presentation belongs to the lecture series entitled “Ten Puzzling Problems in Computer Engineering,” devised for a ten-week, one-unit, freshman seminar course by Behrooz Parhami, Professor of Computer Engineering at University of California, Santa Barbara. The material can be used freely in teaching and other educational settings. Unauthorized uses, including any use for financial gain, are prohibited. © Behrooz Parhami

Edition Released Revised Revised Revised Revised

First May 2007 May 2008 May 2009 May 2010 May 2011

May 2012 May 2015 May 2016 May 2020

May 2020 Sorting Networks Slide 2 Railroad Tracks and Switches

May 2020 Sorting Networks Slide 3 Coupling and Decoupling of Train Cars

Train cars and engines can be coupled and decoupled quickly

An engine can push a string of cars, or pull a desired subset by decoupling them from the rest

May 2020 Sorting Networks Slide 4 Railroad Yards Have Many Tracks and Switches

May 2020 Sorting Networks Slide 5 Rearranging Trains Train cars Engine Track B A D C Sorted order D C B A Stub or lead

B A D C Stack or LIFO data structure in CE : Assemble train in stub, beginning with the last car Repeat: If the next car is X, decouple train after X, push X into stub

Stub or lead D C

B A

B A D C Stub or lead C A B D Queue or FIFO Siding

May 2020 Sorting Networks Slide 6 Goleta Amtrak Station and Its Siding

May 2020 Sorting Networks Slide 7 Model Railroad Yard

May 2020 Sorting Networks Slide 8 Rearrangement with Change of Direction

A B C D Turnaround loop B A D C

C A B D

What types of sorting are possible with a turnaround loop? A wye? Wye

C A

Configuration after the first few steps Q1: Complete the sorting process, in sorting the train B assuming that it is okay for the engine to end up behind the train D

May 2020 Sorting Networks Slide 9 Model Train Turntable

May 2020 Sorting Networks Slide 10 Delivering Train Cars in a Specific Order Cars in the train below have been sorted according to their delivery points. However, it is still nontrivial to deposit car A in stub 1, car B in stub 2, and car C in siding 3. Cars can be pulled or pushed by the engine.

2 C B A D

1 3 2 C B A D

1 3 2 C B A

1 3 Is there a better initial ordering of the cars for the deliveries in this puzzle?

May 2020 Sorting Networks Slide 11 Train Passing Puzzle The trains below must pass each other using a siding that can hold only one car or one engine. Show how this can be done. B A 1 2

B A 1 2

B 1 2 A B 1 2 A 2 A

1 2 A B Q2: If the left and right trains have L and R cars, respectively, how many times will the siding be used for the trains to pass?

May 2020 Sorting Networks Slide 12 Fast Combining or Reordering of Train Cars

C A B D

Forming multiple trains from incoming cars

C A B D Train reordering via a set of stubs Cars are pushed or pulled by an engine Alternatively, movement in one direction may be achieved via sloped rails Switches used to be adjusted manually, but nowadays, electronic control is used

Q3: Sort the two trains shown above with the three sidings and five stubs

May 2020 Sorting Networks Slide 13 Sorting Train Cars in Parallel Track 1 4 2 2 1 Track 2 2 4 1 2 Track 3 1 1 4 3 Track 4 3 3 3 4

Target track Unconditional exchange Compare-exchange Track 1 4 2 1 Track 2 2 4 3 Track 3 1 1 2 Track 4 3 3 4

Is adding this compare-exchange The net in stick diagram element sufficient for producing a schematic valid sorting network?

May 2020 Sorting Networks Slide 14 Validating a Sorting Network Track 1 a min(a, b) min(a, b, c, d) b max(a, b) ???? Track 2 Track 3 c min(c, d) ???? d max(c, d) max(a, b, c, d) Track 4 In the example above, it was fairly easy to show the validity of the sorting network. Generally, it is much more difficult Stick diagram schematic

How would one establish the validity of this 16-input sorting network? More importantly, how does one come up with this design in the first place?

May 2020 Sorting Networks Slide 15 The Zero-One Principle A sorting net built of comparators is valid if it correctly sorts all 0-1 sequences 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 1 0 0 1 1 0 1 1 1

0 0 0 0 0 0 0 0 1 0 1 0 1 0 1 1 0 0 0 1 1 1 1 1 0 1 1 1 0 1 1 1

1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 1 1 1 1 0 1 1 1 0 1 1 1

1 0 1 0 1 0 1 1 1 0 1 1 1 1 1 1 0 1 0 1 1 1 1 1 0 1 1 1 0 1 1 1 So, we can validate a sorting network using 2n rather than n! input patterns n = 12: 2n = 4096, n! = 479,001,600 (thousands vs. half a billion)

May 2020 Sorting Networks Slide 16 A 16-Input Sorting Network Use 4-input sorters, follow by (4, 4)-mergers, and end with an (8, 8)-merger 5 5 2 0 10 8 5 1 8 10 6 2 12 12 7 3 6 2 8 4 14 6 10 5 2 7 12 6 7 14 14 7 4 1 0 8 15 4 1 9 9 9 3 10 1 15 4 11 11 0 9 12 13 3 11 13 3 11 13 14 0 13 15 15

Using the 0-1 principle, we can validate this network via 16 + 25 + 81 tests

4-sorter tests (4, 4)-merger tests (8, 8)-merger tests

May 2020 Sorting Networks Slide 17 and

x0 y0 x0 y0 x1 y1 x 1 y1 x2 y2 x 2 y2 . (n–1)-sorter . . . (n–1)-sorter ......

xn–2 yn–2 xn–2 yn–2 xn–1 yn–1 x n–1 yn–1 Insertion sort Selection sort Insertion sort Selection sort Parallel 3insertion3 sort3 = Parallel0 0 selection0 0 sort0 = 0Parallel0 ! 4 4 0 3 3 3 2 2 1 1 C(n) = n(n – 1)/2 0 0 4 4 4 2 3 1 2 2 D(n ) = 2n – 3 5 5 5 2 4 1 3 3 Cost  Delay = Q(n3) 2 2 5 1 4 4 1 1 5 5 Fig. 7.8 Sorting network based on insertion sort or selection sort.

May 2020 Sorting Networks Slide 18 The Best Sorting Networks

Which 10-input sorting n = 6, 12 modules, 5 levels network is better? n = 9, 25 modules, 8 levels Costn = = 10, 29 29 modules,Delay 9 levels = 9 Costn = 10, = 3131 modules,Delay 7 levels = 7 Cost  Delay = 29  9 = 261 Cost  Delay = 31  7 = 217

Criterion 1: The number of sticks or compare-exchange blocks (cost)

Criterion 2: The number of compare-exchanges in sequence (delay)

n = Criterion12, 40 modules, 3: The 8 levels product of cost and delay (cost-effectiveness) The most cost-effective n-input sorting network may be n = 16, 61 modules, 9 levels n = 16, 60 modules,neither 10 levelsthe fastest design, nor the lowest-cost design

May 2020 Sorting Networks Slide 19 Electronic Sorting Networks Electronic sorting networks are built of 2-sorters building blocks a min(a, b)

b max(a, b) y 3 2 1 1 x0 0 2 3 3 2 x1 y1 5 1 2 3 x2 y2 1 5 5 5 x3 y3 Applications of sorting networks: Directing information packets to their destinations in a network router Connecting n processors to n memory modules in a parallel computer

Q4: In the stick diagram of a 4-sorter on the top right, show that removing the top or bottom line and its comparators yields a valid 3-sorter but that removing one of the two middle lines does not

May 2020 Sorting Networks Slide 20