DOCSLIB.ORG

Cache-Oblivious Priority Queue with Decrease-Keys )

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Cache-Oblivious Priority Queue with Decrease-Keys ) CSE 638: Advanced Algorithms Lectures 20 & 21 ( Cache-oblivious Priority Queue with Decrease-Keys ) Rezaul A. Chowdhury Department of Computer Science SUNY Stony Brook Spring 2013 Cache-Oblivious Buffer Heap Amortized I/O Bounds Priority Queue Delete / Delete-Min Decrease-Key Cache- Buffer Heap oblivious 1 N O log B2 B Cache- Tournament Tree aware Binary Heap O(((log 2 N ))) ( worst-case ) Internal Memory O 1 Fibonacci Heap O(((log 2 N ))) ((( ))) Cache-Oblivious Buffer Heap: Structure Consists of r = 1 + log 2N levels, where N = total number of elements. For 0 ≤≤≤ i ≤≤≤ r - 1, level i contains two buffers: Element Buffers Update Buffers element buffer Bi B U contains elements of the form 0 0 B U (x, kx), where x is the element 1 1 id, and kx is its key B2 U2 update buffer Ui B U contains updates ( Delete , i i Decrease-Key and Sink ), each B U augmented with a time-stamp. r - 1 r - 1 Fig: The Buffer Heap Cache-Oblivious Buffer Heap: Invariants i Element Buffers Update Buffers Invariant 1 : |Bi|≤≤≤ 2 B0 U0 U Invariant 2 : B1 1 B2 U2 (a) No key in Bi is larger than any key in Bi+1 (b) For each element x in Bi, all updates yet Bi Ui to be applied on x reside in U0, U1, … , Ui Br - 1 Ur - 1 Invariant 3 : Fig: The Buffer Heap (a) Each Bi is kept sorted by element id (b) Each Ui ( except U0 ) is kept ( coarsely ) sorted by element id and time-stamp Cache-Oblivious Buffer Heap: Operations The following operations are supported: − Delete (x): Deletes the element x from the queue. − Delete-Min ( ) : Extracts an element with minimum key from queue. − Decrease-Key (x, kx): ( weak Decrease-Key ) ′′′ ′′′ If x already exists in the queue, replaces key k x of x with min( kx, k x), otherwise inserts x with key kx into the queue. A new element x with key kx can be inserted into queue by Decrease-Key (x, kx ). Cache-Oblivious Buffer Heap: Operations Decrease-Key (x, kx) : Insert the operation into U0 augmented with current time-stamp. Delete (x) : Insert the operation into U0 augmented with current time-stamp. Delete-Min ( ) : Two phases: − Descending Phase ( Apply Updates ) − Ascending Phase ( Redistribute Elements ) Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Descending Phase ( Apply Updates ) : B0 U0 B1 U1 B2 U2 Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Descending Phase ( Apply Updates ) : 1. sort updates B0 U0 B1 U1 B2 U2 Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Descending Phase ( Apply Updates ) : B0 U0 B1 2. apply updates: U1 − Delete (x) B2 − Decrease-Key (x, kx) U2 Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Descending Phase ( Apply Updates ) : B0 U0 B1 U1 B2 U2 3. carry updates: − all updates that did not apply − applied Decrease-Keys as Deletes Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Descending Phase ( Apply Updates ) : 1. sort updates: − merge segments B0 U0 B1 U1 B2 U2 Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Descending Phase ( Apply Updates ) : B0 U0 B1 U1 B2 2. apply updates: U2 − Delete (x) − Decrease-Key (x, kx) − Sink (x, kx) Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Descending Phase ( Apply Updates ) : B0 U0 B1 U1 B2 U2 3. carry updates Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Descending Phase ( Apply Updates ) : B0 U0 B1 U1 B2 U2 Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Descending Phase ( Apply Updates ) : B0 U0 B1 U1 B2 U2 Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Descending Phase ( Apply Updates ) : B0 U0 B1 U1 B2 U2 Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Ascending Phase ( Redistribute Elements ) : B0 U0 B1 U1 B2 U2 Bk-1 Uk-1 Bk Uk − find 2k elements with 2k smallest keys Uk+1 − convert each remaining element x with key kx to Sink (x, kx) Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Ascending Phase ( Redistribute Elements ) : B0 U0 B1 U1 B2 U2 Bk-1 Uk-1 Bk Uk Uk+1 push the Sinks to Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Ascending Phase ( Redistribute Elements ) : B0 U0 B1 U1 B2 U2 Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Ascending Phase ( Redistribute Elements ) : B0 U0 B1 U1 B2 U2 move all elements from Bk to shallower levels leaving Bk empty Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Ascending Phase ( Redistribute Elements ) : B0 U0 B1 U1 B2 U2 Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Ascending Phase ( Redistribute Elements ) : B0 U0 B1 U1 B2 U2 Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Ascending Phase ( Redistribute Elements ) : B0 U0 B1 U1 B2 U2 Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Ascending Phase ( Redistribute Elements ) : B0 U0 B1 U1 B2 U2 Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: Delete-Min Delete-Min ( ) - Ascending Phase ( Redistribute Elements ) : element with minimum key B0 U0 B1 U1 B2 U2 Bk-1 Uk-1 Bk Uk Uk+1 Cache-Oblivious Buffer Heap: I/O Complexity 1 r−1 r − 1 Potential Function: 4 3 1 Φ ((()(H))) = rU0 +∑((()( riU −++))) i ∑ ((()( iB))) i B i=1 i = 0 Element Buffers Update Buffers direction of direction of B0 U0 elements updates B1 U1 B2 U2 Bi Ui updates generate elements Br - 1 Ur - 1 Fig: Major Data Flow Paths in the Buffer Heap Lemma: A Buffer Heap on N elements supports Delete , Delete-Min and 1 Decrease-Key operations cache-obliviously in amortized Ο log 2 N B I/Os each using Ο ((( N ))) space. Cache-Oblivious Buffer Heap: I/O Complexity 1 r−1 r − 1 Potential Function: 4 3 1 Φ ((()(H))) = rU0 +∑((()( riU −++))) i ∑ ((()( iB))) i B i=1 i = 0 Element Buffers Update Buffers direction of direction of B0 U0 elements updates B1 U1 B2 U2 Bi Ui updates generate elements Br - 1 Ur - 1 Fig: Major Data Flow Paths in the Buffer Heap Lemma: A Buffer Heap on N elements supports Delete , Delete-Min and 1 Decrease-Key operations cache-obliviously in amortized Ο log 2 N B I/Os each using Ο ((( N ))) space. Cache-Oblivious Buffer Heap: I/O Complexity 1 r−1 r − 1 Potential Function: 4 3 1 Φ ((()(H))) = rU0 +∑((()( riU −++))) i ∑ ((()( iB))) i B i=1 i = 0 Element Buffers Update Buffers direction of direction of B0 U0 elements updates B1 U1 B2 U2 Bi Ui updates generate elements Br - 1 Ur - 1 Fig: Major Data Flow Paths in the Buffer Heap Lemma: A Buffer Heap on N elements supports Delete , Delete-Min and 1 Decrease-Key operations cache-obliviously in amortized Ο log 2 N B I/Os each using Ο ((( N ))) space. Cache-Oblivious Buffer Heap: I/O Complexity 1 r−1 r − 1 Potential Function: 4 3 1 Φ ((()(H))) = rU0 +∑((()( riU −++))) i ∑ ((()( iB))) i B i=1 i = 0 Element Buffers Update Buffers direction of direction of B0 U0 elements updates B1 U1 B2 U2 Bi Ui updates generate elements Br - 1 Ur - 1 Fig: Major Data Flow Paths in the Buffer Heap Lemma: A Buffer Heap on N elements supports Delete , Delete-Min and 1 Decrease-Key operations cache-obliviously in amortized Ο log 2 N B I/Os each using Ο ((( N ))) space. Cache-Oblivious Buffer Heap: I/O Complexity 1 r−1 r − 1 Potential Function: 4 3 1 Φ ((()(H))) = rU0 +∑((()( riU −++))) i ∑ ((()( iB))) i B i=1 i = 0 Element Buffers Update Buffers direction of direction of B0 U0 elements updates B1 U1 B2 U2 Bi Ui updates generate elements Br - 1 Ur - 1 Fig: Major Data Flow Paths in the Buffer Heap Lemma: A Buffer Heap on N elements supports Delete , Delete-Min and 1 Decrease-Key operations cache-obliviously in amortized Ο log 2 N B I/Os each using Ο ((( N ))) space. Cache-Oblivious Buffer Heap: Layout • All Bi’s are kept in a stack SB. • All Ui’s are kept in a stack SU. • An array As is maintained in a stack SA. for 0 ≤≤≤ i ≤≤≤ r – 1 As[i] contains: − |Bi| − number of segments in Ui − number of updates in each segment of Ui In both stacks lower level buffers are placed above higher level buffers. The left to right order of the elements in any buffer are maintained top to bottom in the stack. Cache-Oblivious Buffer Heap: Reconstruction After each operation check whether ∑∑∑|Ui| ≥≥≥ ∑∑∑|Bi|, and if so, Step 1: Sort the elements in SB by element id and level number. Step 2: Sort the updates in SU by element id and time-stamp. Step 3: Scan SB and Su simultaneously, and apply the updates in Su on the elements of SB. Step 4: Reconstruct the data structure by filling the shallowest levels with the current elements in SB, and emptying Su.
Load more

Recommended publications
  • Amortized Analysis
    Amortized Analysis Outline for Today ● Euler Tour Trees ● A quick bug fix from last time. ● Cartesian Trees Revisited ● Why could we construct them in time O(n)? ● Amortized Analysis ● Analyzing data structures over the long term. ● 2-3-4 Trees ● A better analysis of 2-3-4 tree insertions and deletions. Review from Last Time Dynamic Connectivity in Forests ● Consider the following special-case of the dynamic connectivity problem: Maintain an undirected forest G so that edges may be inserted an deleted and connectivity queries may be answered efficiently. ● Each deleted edge splits a tree in two; each added edge joins two trees and never closes a cycle. Dynamic Connectivity in Forests ● Goal: Support these three operations: ● link(u, v): Add in edge {u, v}. The assumption is that u and v are in separate trees. ● cut(u, v): Cut the edge {u, v}. The assumption is that the edge exists in the tree. ● is-connected(u, v): Return whether u and v are connected. ● The data structure we'll develop can perform these operations time O(log n) each. Euler Tours on Trees ● In general, trees do not have Euler tours. a b c d e f a c d b d f d c e c a ● Technique: replace each edge {u, v} with two edges (u, v) and (v, u). ● Resulting graph has an Euler tour. A Correction from Last Time The Bug ● The previous representation of Euler tour trees required us to store pointers to the first and last instance of each node in the tours.
    [Show full text]
  • Lecture Notes of CSCI5610 Advanced Data Structures
    Lecture Notes of CSCI5610 Advanced Data Structures Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong July 17, 2020 Contents 1 Course Overview and Computation Models 4 2 The Binary Search Tree and the 2-3 Tree 7 2.1 The binary search tree . .7 2.2 The 2-3 tree . .9 2.3 Remarks . 13 3 Structures for Intervals 15 3.1 The interval tree . 15 3.2 The segment tree . 17 3.3 Remarks . 18 4 Structures for Points 20 4.1 The kd-tree . 20 4.2 A bootstrapping lemma . 22 4.3 The priority search tree . 24 4.4 The range tree . 27 4.5 Another range tree with better query time . 29 4.6 Pointer-machine structures . 30 4.7 Remarks . 31 5 Logarithmic Method and Global Rebuilding 33 5.1 Amortized update cost . 33 5.2 Decomposable problems . 34 5.3 The logarithmic method . 34 5.4 Fully dynamic kd-trees with global rebuilding . 37 5.5 Remarks . 39 6 Weight Balancing 41 6.1 BB[α]-trees . 41 6.2 Insertion . 42 6.3 Deletion . 42 6.4 Amortized analysis . 42 6.5 Dynamization with weight balancing . 43 6.6 Remarks . 44 1 CONTENTS 2 7 Partial Persistence 47 7.1 The potential method . 47 7.2 Partially persistent BST . 48 7.3 General pointer-machine structures . 52 7.4 Remarks . 52 8 Dynamic Perfect Hashing 54 8.1 Two random graph results . 54 8.2 Cuckoo hashing . 55 8.3 Analysis . 58 8.4 Remarks . 59 9 Binomial and Fibonacci Heaps 61 9.1 The binomial heap .
    [Show full text]
  • Fundamental Data Structures Contents
    Fundamental Data Structures Contents 1 Introduction 1 1.1 Abstract data type ........................................... 1 1.1.1 Examples ........................................... 1 1.1.2 Introduction .......................................... 2 1.1.3 Defining an abstract data type ................................. 2 1.1.4 Advantages of abstract data typing .............................. 4 1.1.5 Typical operations ...................................... 4 1.1.6 Examples ........................................... 5 1.1.7 Implementation ........................................ 5 1.1.8 See also ............................................ 6 1.1.9 Notes ............................................. 6 1.1.10 References .......................................... 6 1.1.11 Further ............................................ 7 1.1.12 External links ......................................... 7 1.2 Data structure ............................................. 7 1.2.1 Overview ........................................... 7 1.2.2 Examples ........................................... 7 1.2.3 Language support ....................................... 8 1.2.4 See also ............................................ 8 1.2.5 References .......................................... 8 1.2.6 Further reading ........................................ 8 1.2.7 External links ......................................... 9 1.3 Analysis of algorithms ......................................... 9 1.3.1 Cost models ......................................... 9 1.3.2 Run-time analysis
    [Show full text]
  • Introduction to Algorithms, 3Rd
    Thomas H. Cormen Charles E. Leiserson Ronald L. Rivest Clifford Stein Introduction to Algorithms Third Edition The MIT Press Cambridge, Massachusetts London, England c 2009 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form or by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher. For information about special quantity discounts, please email special [email protected]. This book was set in Times Roman and Mathtime Pro 2 by the authors. Printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Introduction to algorithms / Thomas H. Cormen ...[etal.].—3rded. p. cm. Includes bibliographical references and index. ISBN 978-0-262-03384-8 (hardcover : alk. paper)—ISBN 978-0-262-53305-8 (pbk. : alk. paper) 1. Computer programming. 2. Computer algorithms. I. Cormen, Thomas H. QA76.6.I5858 2009 005.1—dc22 2009008593 10987654321 Index This index uses the following conventions. Numbers are alphabetized as if spelled out; for example, “2-3-4 tree” is indexed as if it were “two-three-four tree.” When an entry refers to a place other than the main text, the page number is followed by a tag: ex. for exercise, pr. for problem, fig. for figure, and n. for footnote. A tagged page number often indicates the first page of an exercise or problem, which is not necessarily the page on which the reference actually appears. ˛.n/, 574 (set difference), 1159 (golden ratio), 59, 108 pr. jj y (conjugate of the golden ratio), 59 (flow value), 710 .n/ (Euler’s phi function), 943 (length of a string), 986 .n/-approximation algorithm, 1106, 1123 (set cardinality), 1161 o-notation, 50–51, 64 O-notation, 45 fig., 47–48, 64 (Cartesian product), 1162 O0-notation, 62 pr.
    [Show full text]
  • Draft Version 2020-10-11
    Basic Data Structures for an Advanced Audience Patrick Lin Very Incomplete Draft Version 2020-10-11 Abstract I starting writing these notes for my fiancée when I was giving her an improvised course on Data Structures. The intended audience here has somewhat non-standard background for people learning this material: a number of advanced mathematics courses, courses in Automata Theory and Computational Complexity, but comparatively little exposure to Algorithms and Data Structures. Because of the heavy math background that included some advanced theoretical computer science, I felt comfortable with using more advanced mathematics and abstraction than one might find in a standard undergraduate course on Data Structures; on the other hand there are also some exercises which are purely exercises in implementational details. As such, these notes would require heavy revision in order to be used for more general audiences. I am heavily indebted to Pat Morin’s Open Data Structures and Jeff Erickson’s Algorithms, in particular the parts of the “Director’s Cut” of the latter that deal with randomization. Anyone familiar with these two sources will undoubtedly recognize their influences. I apologize for any strange notation, butchered analyses, bad analogies, misleading and/or factually incorrect statements, lack of references, etc. Due to the specific nature of these notes it can be hard to find time/reason to do large rewrites. Contents 1 Models of Computation for Data Structures 2 2 APIs (ADTs) vs Data Structures 4 3 Array-Based Data Structures 6 3.1 ArrayLists and Amortized Analysis . 6 3.2 Array{Stack,Queue,Deque}s ..................................... 9 3.3 Graphs and AdjacencyMatrices .
    [Show full text]
  • Amortizedanalysis-2X
    Data structures Static problems. Given an input, produce an output. DATA STRUCTURES I, II, III, AND IV Ex. Sorting, FFT, edit distance, shortest paths, MST, max-flow, ... I. Amortized Analysis Dynamic problems. Given a sequence of operations (given one at a time), II. Binary and Binomial Heaps produce a sequence of outputs. Ex. Stack, queue, priority queue, symbol table, union–find, …. III. Fibonacci Heaps IV. Union–Find Algorithm. Step-by-step procedure to solve a problem. Data structure. Way to store and organize data. Ex. Array, linked list, binary heap, binary search tree, hash table, … 1 2 3 4 5 6 7 8 7 Lecture slides by Kevin Wayne 33 22 55 23 16 63 86 9 http://www.cs.princeton.edu/~wayne/kleinberg-tardos 33 10 44 86 33 99 1 ● 4 ● 1 ● 3 ● 46 83 90 93 47 60 Last updated on 11/13/19 5:37 AM 2 Appetizer Appetizer Goal. Design a data structure to support all operations in O(1) time. Data structure. Three arrays A[1.. n], B[1.. n], and C[1.. n], and an integer k. ・INIT(n): create and return an initialized array (all zero) of length n. ・A[i] stores the current value for READ (if initialized). ・READ(A, i): return element i in array. ・k = number of initialized entries. ・WRITE(A, i, value): set element i in array to value. ・C[j] = index of j th initialized element for j = 1, …, k. ・If C[j] = i, then B[i] = j for j = 1, …, k. true in C or C++, but not Java Assumptions.
    [Show full text]
  • Amortized Analysis & Splay Trees
    Lecture 2: Amortized Analysis & Splay Trees Rafael Oliveira University of Waterloo Cheriton School of Computer Science [email protected] September 14, 2020 1 / 66 Overview Introduction Meet your TAs! Types of amortized analyses Splay Trees Implementing Splay-Trees Setup Splay Rotations Analysis Acknowledgements 2 / 66 Meet your TAs Thi Xuan Vu Office hours: Wed morning (put exact time here) Anubhav Srivastava Office hours: Thursday afternoon (put exact time here) 3 / 66 With regards to the final project: I highly encourage you all to explore an open problem (but survey is also completely fine! :) ). The reason I wanted to mention is that this may be a unique opportunity for many of you to explore! So be bold! :) If you solve an open problem, you also automatically get 100 in this course and get to publish a paper! (and also get to experience the exhilarating feeling of solving a cool problem!) Twenty years from now you will be more disappointed by the things you didn't do than by the ones you did do. So throw off the bowlines. Sail away from the safe harbor. Catch the trade winds in your sails. Explore. Dream. Discover. - Mark Twain Admin notes Late homework policy: I updated the late homework policy to be more flexible. Now each student has 10 late days without penalty for the entire term. 4 / 66 Twenty years from now you will be more disappointed by the things you didn't do than by the ones you did do. So throw off the bowlines. Sail away from the safe harbor.
    [Show full text]
  • Purely Functional Data Structures
    Purely Functional Data Structures Kristjan Vedel November 18, 2012 Abstract This paper gives an introduction and short overview of various topics related to purely functional data structures. First the concept of persistent and purely functional data structures is introduced, followed by some examples of basic data structures, relevant in purely functional setting, such as lists, queues and trees. Various techniques for designing more efficient purely functional data structures based on lazy evaluation are then described. Two notable general-purpose functional data structures, zippers and finger trees, are presented next. Finally, a brief overview is given of verifying the correctness of purely functional data structures and the time and space complexity calculations. 1 Data Structures and Persistence A data structure is a particular way of organizing data, usually stored in memory and designed for better algorithm efficiency[27]. The term data structure covers different distinct but related meanings, like: • An abstract data type, that is a set of values and associated operations, specified independent of any particular implementation. • A concrete realization or implementation of an abstract data type. • An instance of a data type, also referred to as an object or version. Initial instance of a data structure can be thought of as the version zero and every update operation then generates a new version of the data structure. Data structure is called persistent if it supports access to all versions. [7] It's called partially persistent if all versions can be accessed, but only last version can be modified and fully persistent if every version can be both accessed and mod- ified.
    [Show full text]
  • Mechanized Verification of the Correctness and Asymptotic Complexity of Programs
    Universit´ede Paris Ecole´ doctorale 386 | Sciences Math´ematiquesde Paris Centre Doctorat d'Informatique Mechanized Verification of the Correctness and Asymptotic Complexity of Programs The Right Answer at the Right Time Arma¨elGu´eneau Th`ese dirig´eepar Fran¸coisPottier et Arthur Chargu´eraud et soutenue le 16 d´ecembre 2019 devant le jury compos´ede : Jan Hoffmann Assistant Professor, Carnegie Mellon University Rapporteur Yves Bertot Directeur de Recherche, Inria Rapporteur Georges Gonthier Directeur de Recherche, Inria Examinateur Sylvie Boldo Directrice de Recherche, Inria Examinatrice Mihaela Sighireanu Ma^ıtrede Conf´erences, Universit´ede Paris Examinatrice Fran¸coisPottier Directeur de Recherche, Inria Directeur Arthur Chargu´eraud Charg´ede Recherche, Inria Co-directeur Abstract This dissertation is concerned with the question of formally verifying that the imple- mentation of an algorithm is not only functionally correct (it always returns the right result), but also has the right asymptotic complexity (it reliably computes the result in the expected amount of time). In the algorithms literature, it is standard practice to characterize the performance of an algorithm by indicating its asymptotic time complexity, typically using Landau's \big-O" notation. We first argue informally that asymptotic complexity bounds are equally useful as formal specifications, because they enable modular reasoning: a O bound abstracts over the concrete cost expression of a program, and therefore abstracts over the specifics of its implementation. We illustrate|with the help of small illustrative examples|a number of challenges with the use of the O notation, in particular in the multivariate case, that might be overlooked when reasoning informally. We put these considerations into practice by formalizing the O notation in the Coq proof assistant, and by extending an existing program verification framework, CFML, with support for a methodology enabling robust and modular proofs of asymptotic complexity bounds.
    [Show full text]
  • Amortized Analysis
    DESIGN AND ANALYSIS OF ALGORITHMS (DAA 2018) Juha Kärkkäinen Based on slides by Veli Mäkinen Master’s Programme in Computer Science 06/09/2018 1 ANALYSIS OF RECURRENCES & AMORTIZED ANALYSIS Master’s Programme in Computer Science DAA 2018 week 1 / Juha Kärkkäinen 06/09/2018 2 ANALYSIS OF RECURRENCES • Analysing recursive, divide-and-conquer algorithms • Step 1: Divide problem into subproblems • Step 2: Solve subproblems recursively • Step 3: Combine subproblem results • Three methods • Substitution method (Section 4.3 in book) • Recursion-tree method (Section 4.4) • Master method (Section 4.5) • Quicksort (Chapter 7) • We will continue on Week II with this topic with advanced recursive algorithms Master’s Programme in Computer Science DAA 2018 lecture 1 / Juha Kärkkäinen 06/09/2018 3 QUICKSORT pivot 4 7 8 1 3 6 5 2 9 1 3 2 4 7 8 6 5 9 1 3 2 4 6 5 7 8 9 2 3 4 5 6 7 8 9 2 3 5 6 Bad pivot causes recursion tree to be skewed O(n2) worst case. We learn next week how to select a perfect pivot (median) in linear time! Master’s Programme in Computer Science DAA 2018 lecture 1 / Juha Kärkkäinen 06/09/2018 4 QUICKSORT WITH PERFECT PIVOT … … … … … … … log n levels … … O(n) work on each level O(n log n) time This is called the recursion tree method. Master’s Programme in Computer Science DAA 2018 lecture 1 / Juha Kärkkäinen 06/09/2018 5 QUICKSORT WITH PERFECT PIVOT • Running time can also be stated as a recurrence (recursively defined equation): recursive calls • T(n) = 2T(n/2) + O(n) divide and combine • T(1) = O(1) base case • Assumes n=2k for some integer k>0 (why is this fine to assume?).
    [Show full text]
  • Analysis of Algorithms
    Analysis of Algorithms CSci 653, TTh 9:30-10:50, ISC 0248 Professor Mao, [email protected], 1-3472, McGl 114 General Information I Office Hours: TTh 11:00 – 12:00 and W 3:00 – 4:00 I Textbook: Intro to Algorithms (any edition), CLRS, McGraw Hill or MIT Press. I Prerequisites/background: Linear algebra, Data structures and algorithms, and Discrete math Use of Blackboard I Announcements I Problem sets (aka assignments or homework) I Lecture notes I Grades I Check at least weekly Lecture Basics I Lectures: Slides + board (mostly) I Lecture slides ⊂ lecture notes posted on BB I Not all taught in class are in the lecture notes/slides, e.g., some example problems. So take your own notes in class Course Organization I Mathematical foundation I Methods of analyzing algorithms I Methods of designing algorithms I Additional topics chosen from the lower bound theory, amortization, probabilistic analysis, competitive analysis, NP-completeness, and approximation algorithms Grading I About 12 problem sets: 60% I Final: 40% (in-class) Grading Policy (may be curved) I [90;100]: A or A- I [80;90): B+, B, or B- I [70;80): C+, C, or C- I [60;70): D+, D, or D- I [0;60):F Homework Submission Policy I Hardcopy (stapled and typeset in LaTex) to be submitted at the beginning of class on the due date I Extensions may be permitted for illness, family emergency, and travel to interviews/conferences. Requests must be made prior to the due date Homework Policy I Homework must be typeset in LaTex.
    [Show full text]
  • 8 Priority Queues 8 Priority Queues Dijkstra's Shortest Path Algorithm
    8 Priority Queues 8 Priority Queues A Priority Queue S is a dynamic set data structure that supports the following operations: ñ S: build(x1; : : : ; xn): Creates a data-structure that contains An addressable Priority Queue also supports: just the elements x1; : : : ; xn. ñ handle S: insert(x): Adds element x to the data-structure, ñ S: insert(x): Adds element x to the data-structure. and returns a handle to the object for future reference. ñ element S: minimum(): Returns an element x S with 2 ñ S: delete(h): Deletes element specified through handle h. minimum key-value key[x]. ñ S: decrease-key(h; k): Decreases the key of the element ñ element delete-min : Deletes the element with S: () specified by handle h to k. Assumes that the key is at least minimum key-value from and returns it. S k before the operation. ñ boolean S: is-empty(): Returns true if the data-structure is empty and false otherwise. Sometimes we also have ñ S: merge(S ): S : S S ; S : . 0 = [ 0 0 = 8 Priority Queues © Harald Räcke 303 © Harald Räcke 304 Dijkstra’s Shortest Path Algorithm Prim’s Minimum Spanning Tree Algorithm Algorithm 19 Prim-MST(G (V , E, d), s V) Algorithm 18 Shortest-Path(G (V , E, d), s V) = ∈ = ∈ 1: Input: weighted graph G (V , E, d); start vertex s; 1: Input: weighted graph G (V , E, d); start vertex s; = = 2: Output: pred-fields encode MST; 2: Output: key-field of every node contains distance from s; 3: S.build(); // build empty priority queue 3: S.build(); // build empty priority queue 4: for all v V s do 4: for all v V s do ∈ \{ } ∈ \{ } 5: v.
    [Show full text]