Cache-Oblivious Algorithms by Harald Prokop
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Cache-Oblivious Algorithms EXTENDED ABSTRACT Matteo Frigo Charles E
Cache-Oblivious Algorithms EXTENDED ABSTRACT Matteo Frigo Charles E. Leiserson Harald Prokop Sridhar Ramachandran MIT Laboratory for Computer Science, 545 Technology Square, Cambridge, MA 02139 ¢¡¤£¦¥¨§¢© ¡ § ¨¨ ¥¨¡ ! "#¨§#£$§ %¥'&( # &*)+%£,&-§" Main Abstract This paper presents asymptotically optimal algo- organized by Memory rithms for rectangular matrix transpose, FFT, and sorting on optimal replacement computers with multiple levels of caching. Unlike previous strategy optimal algorithms, these algorithms are cache oblivious: no Cache variables dependent on hardware parameters, such as cache CPU size and cache-line length, need to be tuned to achieve opti- mality. Nevertheless, these algorithms use an optimal amount W of work and move data optimally among multiple levels of work Q cache. For a cache with size Z and cache-line length L where Z 3 L Cache lines 2 0 cache / 1 . Ω Z L the number of cache misses for an m n ma- misses Θ 0 Lines 2 3 trix transpose is / 1 mn L . The number of cache misses of length L for either an n-point FFT or the sorting of n numbers is 050 0 Θ 0 Θ 24/ 3 / 2 / / 1 n L 1 logZ n . We also give an mnp -work al- 1 gorithm to multiply an m 1 n matrix by an n p matrix that Figure 1: The ideal-cache model 0 0 26/ 2 2 3 2 3 7 incurs Θ / 1 mn np mp L mnp L Z cache faults. We introduce an “ideal-cache” model to analyze our algo- shall assume that word size is constant; the particular rithms. We prove that an optimal cache-oblivious algorithm constant does not affect our asymptotic analyses. -
Engineering Cache-Oblivious Sorting Algorithms
Engineering Cache-Oblivious Sorting Algorithms Master’s Thesis by Kristoffer Vinther June 2003 Abstract Modern computers are far more sophisticated than simple sequential programs can lead one to believe; instructions are not executed sequentially and in constant time. In particular, the memory of a modern computer is structured in a hierarchy of increasingly slower, cheaper, and larger storage. Accessing words in the lower, faster levels of this hierarchy can be done virtually immediately, but accessing the upper levels may cause delays of millions of processor cycles. Consequently, recent developments in algorithm design have had a focus on developing algorithms that sought to minimize accesses to the higher levels of the hierarchy. Much experimental work has been done showing that using these algorithms can lead to higher performing algorithms. However, these algorithms are designed and implemented with a very specific level in mind, making it infeasible to adapt them to multiple levels or use them efficiently on different architectures. To alleviate this, the notion of cache-oblivious algorithms was developed. The goal of a cache-oblivious algorithm is to be optimal in the use of the memory hierarchy, but without using specific knowledge of its structure. This automatically makes the algorithm efficient on all levels of the hierarchy and on all implementations of such hierarchies. The experimental work done with these types of algorithms remain sparse, however. In this thesis, we present a thorough theoretical and experimental analysis of known optimal cache-oblivious sorting algorithms. We develop our own simpler variants and present the first optimal sub-linear working space cache-oblivious sorting algorithm. -
Design Implement Analyze Experiment
Felix Putze and Peter Sanders design experiment Algorithmics analyze implement Course Notes Algorithm Engineering TU Karlsruhe October 19, 2009 Preface These course notes cover a lecture on algorithm engineering for the basic toolbox that Peter Sanders is reading at Universitat¨ Karlsruhe since 2004. The text is compiled from slides, scientific papers and other manuscripts. Most of this material is in English so that this language was adopted as the main language. However, some parts are in German. The primal sources of our material are given at the beginning of each chapter. Please refer to the original publications for further references. This document is still work in progress. Please report bugs of any type (content, language, layout, . ) to [email protected]. Thank you! 1 Contents 1 Was ist Algorithm Engineering? 6 1.1 Einfuhrung¨ . .6 1.2 Stand der Forschung und offene Probleme . .8 2 Data Structures 16 2.1 Arrays & Lists . 16 2.2 External Lists . 18 2.3 Stacks, Queues & Variants . 20 3 Sorting 24 3.1 Quicksort Basics . 24 3.2 Refined Quicksort . 25 3.3 Lessons from experiments . 27 3.4 Super Scalar Sample Sort . 30 3.5 Multiway Merge Sort . 34 3.6 Sorting with parallel disks . 35 3.7 Internal work of Multiway Mergesort . 38 3.8 Experiments . 41 4 Priority Queues 44 4.1 Introduction . 44 4.2 Binary Heaps . 45 4.3 External Priority Queues . 47 4.4 Adressable Priority Queues . 54 5 External Memory Algorithms 57 5.1 Introduction . 57 5.2 The external memory model and things we already saw . -
Chapter 3 Cache-Oblivious Algorithms
Portable High-Performance Programs by Matteo Frigo Laurea, Universita di Padova (1992) Dottorato di Ricerca, Universita di Padova (1996) Submitted to the Department of Electrical Engineering and Computer Science in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY J ne 1999 -. @ Matteo Frigo, MCMXCIX. All rights reserved. The author hereby grants to MIT permission to reproduce and distribute publicly paper and electronic copies of this thesis document in whole or in part, and to grant others the right to do so. Author........ ......... ...................... Department of Electrical Engineering and Computer Science June 22, 1999 Certified by ............ ................... -V Charles E. Leiserson Professor of Computer Science and Engineering Thesis Supervisor Accepted by .................. & .. ..-.-.-. -..... ......-- Arthur C. Smith . Denartmental Commnsie on Graduate Students Copyright @ 1999 Matteo Frigo. Permission is granted to make and distribute verbatim copies of this thesis provided the copy- right notice and this permission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this thesis under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this thesis into another language, under the above conditions for modified versions, except -
Cache-Oblivious and Data-Oblivious Sorting and Applications
Cache-Oblivious and Data-Oblivious Sorting and Applications T-H. Hubert Chan∗ Yue Guoy Wei-Kai Liny Elaine Shiy Abstract Although external-memory sorting has been a classical algorithms abstraction and has been heavily studied in the literature, perhaps somewhat surprisingly, when data-obliviousness is a requirement, even very rudimentary questions remain open. Prior to our work, it is not even known how to construct a comparison-based, external-memory oblivious sorting algorithm that is optimal in IO-cost. We make a significant step forward in our understanding of external-memory, oblivious sorting algorithms. Not only do we construct a comparison-based, external-memory oblivious sorting algorithm that is optimal in IO-cost, our algorithm is also cache-agnostic in that the algorithm need not know the storage hierarchy's internal parameters such as the cache and cache-line sizes. Our result immediately implies a cache-agnostic ORAM construction whose asymptotical IO-cost matches the best known cache-aware scheme. Last but not the least, we propose and adopt a new and stronger security notion for external- memory, oblivious algorithms and argue that this new notion is desirable for resisting possible cache-timing attacks. Thus our work also lays a foundation for the study of oblivious algorithms in the cache-agnostic model. ∗Department of Computer Science, The University of Hong Kong. [email protected] yComputer Science Department, Cornell University. [email protected], [email protected], [email protected] 1 Introduction In data-oblivious algorithms, a client (also known as a CPU) performs some computation over sensitive data, such that a possibly computationally bounded adversary, who can observe data access patterns, gains no information about the input or the data.