DOCSLIB.ORG

Register Level Sort Algorithm on Multi-Core SIMD Processors

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Register Level Sort Algorithm on Multi-Core SIMD Processors Register Level Sort Algorithm on Multi-Core SIMD Processors Tian Xiaochen, Kamil Rocki and Reiji Suda Graduate School of Information Science and Technology The University of Tokyo & CREST, JST {xchen, kamil.rocki, reiji}@is.s.u-tokyo.ac.jp ABSTRACT simultaneously. GPU employs a similar architecture: single- State-of-the-art hardware increasingly utilizes SIMD paral- instruction-multiple-threads(SIMT). On K20, each SMX has lelism, where multiple processing elements execute the same 192 CUDA cores which act as individual threads. Even a instruction on multiple data points simultaneously. How- desktop type multi-core x86 platform such as i7 Haswell ever, irregular and data intensive algorithms are not well CPU family supports AVX2 instruction set (256 bits). De- suited for such architectures. Due to their importance, it veloping algorithms that support multi-core SIMD architec- is crucial to obtain efficient implementations. One example ture is the precondition to unleash the performance of these of such a task is sort, a fundamental problem in computer processors. Without exploiting this kind of parallelism, only science. In this paper we analyze distinct memory accessing a small fraction of computational power can be utilized. models and propose two methods to employ highly efficient Parallel sort as well as sort in general is fundamental and bitonic merge sort using SIMD instructions as register level well studied problem in computer science. Algorithms such sort. We achieve nearly 270x speedup (525M integers/s) on as Bitonic-Merge Sort or Odd Even Merge Sort are widely a 4M integer set using Xeon Phi coprocessor, where SIMD used in practice. However, implementing sort algorithm level parallelism accelerates the algorithm over 3 times. Our on SIMD processors efficiently, especially on Xeon Phi or method can be applied to any device supporting similar x86 CPU remains a challenging job. Until now, many algo- SIMD instructions. rithms have been proposed for GPU or CPU. GPU version of the algorithm cannot be directly transplanted a proces- sor supporting 512-bit wide vector instructions such as Xeon Categories and Subject Descriptors Phi. Nvidia GPUs have fast, explicitly manageable on-chip C.1.2 [Computer Systems Organization]: PROCESSOR shared memory and their cores are more functional and ex- ARCHITECTURES|Single-instruction-stream, multiple-data- ecute individual threads concurrently with SIMT synchro- stream processors (SIMD) nization on hardware level. Programming Xeon Phi poses a different challenge - the instruction restrictions of memory accesses (discussed in Section 3.1) needed during SIMD sort General Terms or merge. Previous algorithms designed for SSE or CELL Algorithms, Performance, Design processor require too many registers or not strong scalable enough for long SIMD instructions. On Xeon Phi, proces- Keywords sor's resources can easily run out with previous methods. A general, strong scaling algorithm for Xeon Phi or future SIMD, Parallel, Sort, Xeon Phi, Register, Irregular SIMD instruction capable processors is necessary. In this paper, we focus on developing parallel sort and 1. INTRODUCTION merge algorithms only in register, under the constraints of Multi-core architecture has been widely used in state-of- memory accessing model within registers. We propose two art processors. For example, NVIDIA's Tesla K20 GPU in-register sort/merge algorithms which only take a constant has 14 Stream Multiprocessors(SMX) and Intel's Xeon Phi number of registers (2 or 3) no matter how long SIMD in- has 60 cores. Xeon Phi supports 512 bits SIMD instruc- struction is. Then, we present theoretical and empirical tion i.e. AVX-512 which can process 16 integers (4 bytes) analysis of the execution time. We chose AVX-512 and Xeon Phi as our main experiment environment as a extreme case of SIMD parallelism combined with restricted access pat- Permission to make digital or hard copies of all or part of this work for per- terns and limited bandwidth. We also implemented the al- sonal or classroom use is granted without fee provided that copies are not gorithm on Intel E5-2670 with AVX instructions. The same made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components algorithm can be derived for other SIMD processors. of this work owned by others than the author(s) must be honored. Abstract- ing with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a 2. RELATED WORK fee. Request permissions from [email protected]. First algorithm that needs to be mentioned is parallel SC13 November 17-21, 2013, Denver CO, USA Radix sort[5] as it is often used. The main advantage of the Copyright is held Copyright is held by the owner/author(s). ACM 978-1-4503-2503-5/13/11 method is that if the size of digits is fixed, it only needs O(n) http://dx.doi.org/10.1145/2535753.2535762. time complexity, where n is the length of data. The idea is to sort data digit by digit. Intel Laboratories' report[2] is the othy Furtak[7], it generates the code with search method. first published result presenting performance of radix sort on The main idea is to search a set of SSE instructions which Xeon Phi. Later in the paper, we are presenting our results follow exactly the same logic as Odd-Even Merge Sort at compared to those shown in this paper. Its main disadvan- each step. The advantage of this method is that the gener- tage on the other hand is that it is limited to the integer type ated code can be locally optimal. However they ignore the of data and exploits its decimal character. Contrary to more usage of the number of registers and the speed of genera- general, comparative sort for example, float numbers need tion. Finally, AA sort[10] is a different approach which does extra transformation[13] using radix sort. Furthermore, if not employ odd-even or any other sorting network. Comb the data type size is longer, the algorithm becomes slower. Sort[11] is their choice. However, Comb Sort needs O(n2) Nadathur Satish et al.[6] studied how radix sort and merge in worst case, also this algorithm requires the same number sort performs when size of data type varies. Though both of registers as the aforementioned Jatin's algorithm. of them get slower when data type changes from 32bit to We propose two strong scaling register-level-sort algorithms 128bit, radix sort get higher deceleration. which only need constant number of registers. The two algo- Comparison based sort is a more general sort algorithm. rithms are both derived from the code generation approach Ω(nlog(n)) is the lower bound of its time complexity. Many which is based on constructing methods. It can be finished theoretical results have been published in the field of parallel in polynomial time (the same as bitonic merge sort). sorting. Bitonic-Merge Sort[3] or Odd-Even Merge Sort[8] are just two algorithms usually used in practice. They are 2.2 Multi-Core Level Sort also sorting networks traversed in O(log2(n)) steps. Though In principle, Multi-Core Level Sort focuses on utilizing log(n)-step algorithm has been invented[9], due to the large multiple cores. MIMD (Multiple Instruction, Multiple Data) overhead it is hardly used in practice. Our algorithm keeps and PRAM are the appropriate hardware models. On GPU, the generality of a comparative sort while achieving speedup Odd-Even Merge Sort is still a widely chosen algorithm [12, only slightly worse than with radix sort. 14]. It has high parallelism, but suffers from high work There is a large number of research papers which describe complexity. Theoretically Bitonic-Merge Sort can be per- implementing parallel sort. We classify the algorithms into formed in O(nlogn) time by using a subtree(i.e. an address two types: Register Level Sort and Multi-Core Level Sort. pointer) exchanging algorithm[16] utilizing a tree data struc- This classification is made by the way of synchronization in ture. Contrary to merge, divide-and-conquer is another idea processors. In register level, SIMD instructions play the role of sorting elements. For example, Bucket Sort, the quick of cores. Synchronization is not necessary(GPU may need sort's parallel version can divide the data into a numbers of synchronization explicitly) since SIMD computation is natu- bucket by selecting several pivots. The data in each bucket rally synchronized after each instruction. In the situation of can be sorted by other sorting algorithms, since the buck- Multi-Core level, communication happens in slow cache or ets are ordered by pivots. An GPU implementation[15] uses RAM. The cost of synchronization becomes more expensive. hybrid algorithm of bucket sort and merge sort. However the algorithm may not be well balanced since buckets' sizes 2.1 Register Level Sort depend on the chosen of pivots. Some buckets may be too Register level sort means sorting a part of a small amount large or too small. A typical x86-family processor, such of data only using SIMD instructions. Usually the data as Intel i7 does not have as many core as common GPUs. should be loaded into a register from the main memory and Multi-way merge is an optional algorithm in such a CPU then sorted in registers in parallel. Finally the data should implementation[1]. Although, multi-way merge can be as be copied back to the main memory. Some implementations efficient as merge sort from time complexity aspect. If there may use Odd-Even Merge Sort at this point, for example an are too many cores, the communication between cores will algorithm using SSE instructions[1] by Jatin Chhugani et al.
Load more

Recommended publications
  • Radix Sort Comparison Sort Runtime of O(N*Log(N)) Is Optimal
    Radix Sort Comparison sort runtime of O(n*log(n)) is optimal • The problem of sorting cannot be solved using comparisons with less than n*log(n) time complexity • See Proposition I in Chapter 2.2 of the text How can we sort without comparison? • Consider the following approach: • Look at the least-significant digit • Group numbers with the same digit • Maintain relative order • Place groups back in array together • I.e., all the 0’s, all the 1’s, all the 2’s, etc. • Repeat for increasingly significant digits The characteristics of Radix sort • Least significant digit (LSD) Radix sort • a fast stable sorting algorithm • begins at the least significant digit (e.g. the rightmost digit) • proceeds to the most significant digit (e.g. the leftmost digit) • lexicographic orderings Generally Speaking • 1. Take the least significant digit (or group of bits) of each key • 2. Group the keys based on that digit, but otherwise keep the original order of keys. • This is what makes the LSD radix sort a stable sort. • 3. Repeat the grouping process with each more significant digit Generally Speaking public static void sort(String [] a, int W) { int N = a.length; int R = 256; String [] aux = new String[N]; for (int d = W - 1; d >= 0; --d) { aux = sorted array a by the dth character a = aux } } A Radix sort example A Radix sort example A Radix sort example • Problem: How to ? • Group the keys based on that digit, • but otherwise keep the original order of keys. Key-indexed counting • 1. Take the least significant digit (or group of bits) of each key • 2.
    [Show full text]
  • Secure Multi-Party Sorting and Applications
    Secure Multi-Party Sorting and Applications Kristján Valur Jónsson1, Gunnar Kreitz2, and Misbah Uddin2 1 Reykjavik University 2 KTH—Royal Institute of Technology Abstract. Sorting is among the most fundamental and well-studied problems within computer science and a core step of many algorithms. In this article, we consider the problem of constructing a secure multi-party computing (MPC) protocol for sorting, building on previous results in the field of sorting networks. Apart from the immediate uses for sorting, our protocol can be used as a building-block in more complex algorithms. We present a weighted set intersection algorithm, where each party inputs a set of weighted ele- ments and the output consists of the input elements with their weights summed. As a practical example, we apply our protocols in a network security setting for aggregation of security incident reports from multi- ple reporters, specifically to detect stealthy port scans in a distributed but privacy preserving manner. Both sorting and weighted set inter- section use O`n log2 n´ comparisons in O`log2 n´ rounds with practical constants. Our protocols can be built upon any secret sharing scheme supporting multiplication and addition. We have implemented and evaluated the performance of sorting on the Sharemind secure multi-party computa- tion platform, demonstrating the real-world performance of our proposed protocols. Keywords. Secure multi-party computation; Sorting; Aggregation; Co- operative anomaly detection 1 Introduction Intrusion Detection Systems (IDS) [16] are commonly used to detect anoma- lous and possibly malicious network traffic. Incidence reports and alerts from such systems are generally kept private, although much could be gained by co- operative sharing [30].
    [Show full text]
  • RISC-V Vector Extension Webinar II
    RISC-V Vector Extension Webinar II August 3th, 2021 Thang Tran, Ph.D. Principal Engineer Webinar II - Agenda • Andes overview • Vector technology background – SIMD/vector concept – Vector processor basic • RISC-V V extension ISA – Basic – CSR • RISC-V V extension ISA – Memory operations – Compute instructions • Sample codes – Matrix multiplication – Loads with RVV versions 0.8 and 1.0 • AndesCore™ NX27V • Summary Copyright© 2020 Andes Technology Corp. 2 Terminology • ACE: Andes Custom Extension • ISA: Instruction Set Architecture • CSR: Control and Status Register • GOPS: Giga Operations Per Second • SEW: Element Width (8-64) • GFLOPS: Giga Floating-Point OPS • ELEN: Largest Element Width (32 or 64) • XRF: Integer register file • XLEN: Scalar register length in bits (64) • FRF: Floating-point register file • FLEN: FP register length in bits (16-64) • VRF: Vector register file • VLEN: Vector register length in bits (128-512) • SIMD: Single Instruction Multiple Data • LMUL: Register grouping multiple (1/8-8) • MMX: Multi Media Extension • EMUL: Effective LMUL • SSE: Streaming SIMD Extension • VLMAX/MVL: Vector Length Max • AVX: Advanced Vector Extension • AVL/VL: Application Vector Length • Configurable: parameters are fixed at built time, i.e. cache size • Extensible: added instructions to ISA includes custom instructions to be added by customer • Standard extension: the reserved codes in the ISA for special purposes, i.e. FP, DSP, … • Programmable: parameters can be dynamically changed in the program Copyright© 2020 Andes Technology Corp. 3 RISC-V V Extension ISA Basic Copyright© 2020 Andes Technology Corp. 4 Vector Register ISA • Vector-Register ISA Definition: − All vector operations are between vector registers (except for load and store).
    [Show full text]
  • Coursenotes 4 Non-Adaptive Sorting Batcher's Algorithm
    4. Non Adaptive Sorting Batcher’s Algorithm 4.1 Introduction to Batcher’s Algorithm Sorting has many important applications in daily life and in particular, computer science. Within computer science several sorting algorithms exist such as the “bubble sort,” “shell sort,” “comb sort,” “heap sort,” “bucket sort,” “merge sort,” etc. We have actually encountered these before in section 2 of the course notes. Many different sorting algorithms exist and are implemented differently in each programming language. These sorting algorithms are relevant because they are used in every single computer program you run ranging from the unimportant computer solitaire you play to the online checking account you own. Indeed without sorting algorithms, many of the services we enjoy today would simply not be available. Batcher’s algorithm is a way to sort individual disorganized elements called “keys” into some desired order using a set number of comparisons. The keys that will be sorted for our purposes will be numbers and the order that we will want them to be in will be from greatest to least or vice a versa (whichever way you want it). The Batcher algorithm is non-adaptive in that it takes a fixed set of comparisons in order to sort the unsorted keys. In other words, there is no change made in the process during the sorting. Unlike other methods of sorting where you need to remember comparisons (tournament sort), Batcher’s algorithm is useful because it requires less thinking. Non-adaptive sorting is easy because outcomes of comparisons made at one point of the process do not affect which comparisons will be made in the future.
    [Show full text]
  • 1 Lower Bound on Runtime of Comparison Sorts
    CS 161 Lecture 7 { Sorting in Linear Time Jessica Su (some parts copied from CLRS) 1 Lower bound on runtime of comparison sorts So far we've looked at several sorting algorithms { insertion sort, merge sort, heapsort, and quicksort. Merge sort and heapsort run in worst-case O(n log n) time, and quicksort runs in expected O(n log n) time. One wonders if there's something special about O(n log n) that causes no sorting algorithm to surpass it. As a matter of fact, there is! We can prove that any comparison-based sorting algorithm must run in at least Ω(n log n) time. By comparison-based, I mean that the algorithm makes all its decisions based on how the elements of the input array compare to each other, and not on their actual values. One example of a comparison-based sorting algorithm is insertion sort. Recall that insertion sort builds a sorted array by looking at the next element and comparing it with each of the elements in the sorted array in order to determine its proper position. Another example is merge sort. When we merge two arrays, we compare the first elements of each array and place the smaller of the two in the sorted array, and then we make other comparisons to populate the rest of the array. In fact, all of the sorting algorithms we've seen so far are comparison sorts. But today we will cover counting sort, which relies on the values of elements in the array, and does not base all its decisions on comparisons.
    [Show full text]
  • Quicksort Z Merge Sort Z T(N) = Θ(N Lg(N)) Z Not In-Place CSE 680 Z Selection Sort (From Homework) Prof
    Sorting Review z Insertion Sort Introduction to Algorithms z T(n) = Θ(n2) z In-place Quicksort z Merge Sort z T(n) = Θ(n lg(n)) z Not in-place CSE 680 z Selection Sort (from homework) Prof. Roger Crawfis z T(n) = Θ(n2) z In-place Seems pretty good. z Heap Sort Can we do better? z T(n) = Θ(n lg(n)) z In-place Sorting Comppgarison Sorting z Assumptions z Given a set of n values, there can be n! 1. No knowledge of the keys or numbers we permutations of these values. are sorting on. z So if we look at the behavior of the 2. Each key supports a comparison interface sorting algorithm over all possible n! or operator. inputs we can determine the worst-case 3. Sorting entire records, as opposed to complexity of the algorithm. numbers,,p is an implementation detail. 4. Each key is unique (just for convenience). Comparison Sorting Decision Tree Decision Tree Model z Decision tree model ≤ 1:2 > z Full binary tree 2:3 1:3 z A full binary tree (sometimes proper binary tree or 2- ≤≤> > tree) is a tree in which every node other than the leaves has two c hildren <1,2,3> ≤ 1:3 >><2,1,3> ≤ 2:3 z Internal node represents a comparison. z Ignore control, movement, and all other operations, just <1,3,2> <3,1,2> <2,3,1> <3,2,1> see comparison z Each leaf represents one possible result (a permutation of the elements in sorted order).
    [Show full text]
  • Optimizing Packed String Matching on AVX2 Platform
    Optimizing Packed String Matching on AVX2 Platform M. Akif Aydo˘gmu¸s1,2 and M.O˘guzhan Külekci1 1 Informatics Institute, Istanbul Technical University, Istanbul, Turkey [email protected], [email protected] 2 TUBITAK UME, Gebze-Kocaeli, Turkey Abstract. Exact string matching, searching for all occurrences of given pattern P on a text T , is a fundamental issue in computer science with many applica- tions in natural language processing, speech processing, computational biology, information retrieval, intrusion detection systems, data compression, and etc. Speeding up the pattern matching operations benefiting from the SIMD par- allelism has received attention in the recent literature, where the empirical results on previous studies revealed that SIMD parallelism significantly helps, while the performance may even be expected to get automatically enhanced with the ever increasing size of the SIMD registers. In this paper, we provide variants of the previously proposed EPSM and SSEF algorithms, which are orig- inally implemented on Intel SSE4.2 (Streaming SIMD Extensions 4.2 version with 128-bit registers). We tune the new algorithms according to Intel AVX2 platform (Advanced Vector Extensions 2 with 256-bit registers) and analyze the gain in performance with respect to the increasing length of the SIMD registers. Profiling the new algorithms by using the Intel Vtune Amplifier for detecting performance bottlenecks led us to consider the cache friendliness and shared-memory access issues in the AVX2 platform. We applied cache op- timization techniques to overcome the problems particularly addressing the search algorithms based on filtering. Experimental comparison of the new solutions with the previously known-to- be-fast algorithms on small, medium, and large alphabet text files with diverse pattern lengths showed that the algorithms on AVX2 platform optimized cache obliviously outperforms the previous solutions.
    [Show full text]
  • An Evolutionary Approach for Sorting Algorithms
    ORIENTAL JOURNAL OF ISSN: 0974-6471 COMPUTER SCIENCE & TECHNOLOGY December 2014, An International Open Free Access, Peer Reviewed Research Journal Vol. 7, No. (3): Published By: Oriental Scientific Publishing Co., India. Pgs. 369-376 www.computerscijournal.org Root to Fruit (2): An Evolutionary Approach for Sorting Algorithms PRAMOD KADAM AND Sachin KADAM BVDU, IMED, Pune, India. (Received: November 10, 2014; Accepted: December 20, 2014) ABstract This paper continues the earlier thought of evolutionary study of sorting problem and sorting algorithms (Root to Fruit (1): An Evolutionary Study of Sorting Problem) [1]and concluded with the chronological list of early pioneers of sorting problem or algorithms. Latter in the study graphical method has been used to present an evolution of sorting problem and sorting algorithm on the time line. Key words: Evolutionary study of sorting, History of sorting Early Sorting algorithms, list of inventors for sorting. IntroDUCTION name and their contribution may skipped from the study. Therefore readers have all the rights to In spite of plentiful literature and research extent this study with the valid proofs. Ultimately in sorting algorithmic domain there is mess our objective behind this research is very much found in documentation as far as credential clear, that to provide strength to the evolutionary concern2. Perhaps this problem found due to lack study of sorting algorithms and shift towards a good of coordination and unavailability of common knowledge base to preserve work of our forebear platform or knowledge base in the same domain. for upcoming generation. Otherwise coming Evolutionary study of sorting algorithm or sorting generation could receive hardly information about problem is foundation of futuristic knowledge sorting problems and syllabi may restrict with some base for sorting problem domain1.
    [Show full text]
  • Improving the Performance of Bubble Sort Using a Modified Diminishing Increment Sorting
    Scientific Research and Essay Vol. 4 (8), pp. 740-744, August, 2009 Available online at http://www.academicjournals.org/SRE ISSN 1992-2248 © 2009 Academic Journals Full Length Research Paper Improving the performance of bubble sort using a modified diminishing increment sorting Oyelami Olufemi Moses Department of Computer and Information Sciences, Covenant University, P. M. B. 1023, Ota, Ogun State, Nigeria. E- mail: [email protected] or [email protected]. Tel.: +234-8055344658. Accepted 17 February, 2009 Sorting involves rearranging information into either ascending or descending order. There are many sorting algorithms, among which is Bubble Sort. Bubble Sort is not known to be a very good sorting algorithm because it is beset with redundant comparisons. However, efforts have been made to improve the performance of the algorithm. With Bidirectional Bubble Sort, the average number of comparisons is slightly reduced and Batcher’s Sort similar to Shellsort also performs significantly better than Bidirectional Bubble Sort by carrying out comparisons in a novel way so that no propagation of exchanges is necessary. Bitonic Sort was also presented by Batcher and the strong point of this sorting procedure is that it is very suitable for a hard-wired implementation using a sorting network. This paper presents a meta algorithm called Oyelami’s Sort that combines the technique of Bidirectional Bubble Sort with a modified diminishing increment sorting. The results from the implementation of the algorithm compared with Batcher’s Odd-Even Sort and Batcher’s Bitonic Sort showed that the algorithm performed better than the two in the worst case scenario. The implication is that the algorithm is faster.
    [Show full text]
  • Investigating the Effect of Implementation Languages and Large Problem Sizes on the Tractability and Efficiency of Sorting Algorithms
    International Journal of Engineering Research and Technology. ISSN 0974-3154, Volume 12, Number 2 (2019), pp. 196-203 © International Research Publication House. http://www.irphouse.com Investigating the Effect of Implementation Languages and Large Problem Sizes on the Tractability and Efficiency of Sorting Algorithms Temitayo Matthew Fagbola and Surendra Colin Thakur Department of Information Technology, Durban University of Technology, Durban 4000, South Africa. ORCID: 0000-0001-6631-1002 (Temitayo Fagbola) Abstract [1],[2]. Theoretically, effective sorting of data allows for an efficient and simple searching process to take place. This is Sorting is a data structure operation involving a re-arrangement particularly important as most merge and search algorithms of an unordered set of elements with witnessed real life strictly depend on the correctness and efficiency of sorting applications for load balancing and energy conservation in algorithms [4]. It is important to note that, the rearrangement distributed, grid and cloud computing environments. However, procedure of each sorting algorithm differs and directly impacts the rearrangement procedure often used by sorting algorithms on the execution time and complexity of such algorithm for differs and significantly impacts on their computational varying problem sizes and type emanating from real-world efficiencies and tractability for varying problem sizes. situations [5]. For instance, the operational sorting procedures Currently, which combination of sorting algorithm and of Merge Sort, Quick Sort and Heap Sort follow a divide-and- implementation language is highly tractable and efficient for conquer approach characterized by key comparison, recursion solving large sized-problems remains an open challenge. In this and binary heap’ key reordering requirements, respectively [9].
    [Show full text]
  • Sorting Partnership Unless You Sign Up! Brian Curless • Homework #5 Will Be Ready After Class, Spring 2008 Due in a Week
    Announcements (5/9/08) • Project 3 is now assigned. CSE 326: Data Structures • Partnerships due by 3pm – We will not assume you are in a Sorting partnership unless you sign up! Brian Curless • Homework #5 will be ready after class, Spring 2008 due in a week. • Reading for this lecture: Chapter 7. 2 Sorting Consistent Ordering • Input – an array A of data records • The comparison function must provide a – a key value in each data record consistent ordering on the set of possible keys – You can compare any two keys and get back an – a comparison function which imposes a indication of a < b, a > b, or a = b (trichotomy) consistent ordering on the keys – The comparison functions must be consistent • Output • If compare(a,b) says a<b, then compare(b,a) must say b>a • If says a=b, then must say b=a – reorganize the elements of A such that compare(a,b) compare(b,a) • If compare(a,b) says a=b, then equals(a,b) and equals(b,a) • For any i and j, if i < j then A[i] ≤ A[j] must say a=b 3 4 Why Sort? Space • How much space does the sorting • Allows binary search of an N-element algorithm require in order to sort the array in O(log N) time collection of items? • Allows O(1) time access to kth largest – Is copying needed? element in the array for any k • In-place sorting algorithms: no copying or • Sorting algorithms are among the most at most O(1) additional temp space.
    [Show full text]
  • Selected Sorting Algorithms
    Selected Sorting Algorithms CS 165: Project in Algorithms and Data Structures Michael T. Goodrich Some slides are from J. Miller, CSE 373, U. Washington Why Sorting? • Practical application – People by last name – Countries by population – Search engine results by relevance • Fundamental to other algorithms • Different algorithms have different asymptotic and constant-factor trade-offs – No single ‘best’ sort for all scenarios – Knowing one way to sort just isn’t enough • Many to approaches to sorting which can be used for other problems 2 Problem statement There are n comparable elements in an array and we want to rearrange them to be in increasing order Pre: – An array A of data records – A value in each data record – A comparison function • <, =, >, compareTo Post: – For each distinct position i and j of A, if i < j then A[i] ≤ A[j] – A has all the same data it started with 3 Insertion sort • insertion sort: orders a list of values by repetitively inserting a particular value into a sorted subset of the list • more specifically: – consider the first item to be a sorted sublist of length 1 – insert the second item into the sorted sublist, shifting the first item if needed – insert the third item into the sorted sublist, shifting the other items as needed – repeat until all values have been inserted into their proper positions 4 Insertion sort • Simple sorting algorithm. – n-1 passes over the array – At the end of pass i, the elements that occupied A[0]…A[i] originally are still in those spots and in sorted order.
    [Show full text]