Advanced Algorithmics (6EAP) MTAT.03.238 Linear structures, sor.ng, searching, etc Jaak Vilo 2014 Fall Jaak Vilo 1 Big-Oh notaon classes Class Informal Intuion Analogy f(n) ∈ ο ( g(n) ) f is dominated by g Strictly below < f(n) ∈ O( g(n) ) Bounded from above Upper bound ≤ f(n) ∈ Θ( g(n) ) Bounded from “equal to” = above and below f(n) ∈ Ω( g(n) ) Bounded from below Lower bound ≥ f(n) ∈ ω( g(n) ) f dominates g Strictly above > Conclusions • Algorithm complexity deals with the behavior in the long-term – worst case -- typical – average case -- quite hard – best case -- bogus, “chea9ng” • In pracGce, long-term someGmes not necessary – E.g. for sorGng 20 elements, you don’t need fancy algorithms… Linear, sequenGal, ordered, list … Memory, disk, tape etc – is an ordered sequentially addressed media. Physical ordered list ~ array • Memory /address/ – Garbage collecGon • Files (character/byte list/lines in text file,…) • Disk – Disk fragmentaon Linear data structures: Arrays • Array • Hashed array tree • BidirecGonal map • Heightmap • Bit array • Lookup table • Bit field • Matrix • Bitboard • Parallel array • Bitmap • Sorted array • Circular buffer • Sparse array • Control table • Sparse matrix • Image • Iliffe vector • Dynamic array • Variable-length array • Gap buffer Linear data structures: Lists • Doubly linked list • Array list • Xor linked list • Linked list • Zipper • Self-organizing list • Doubly connected edge • Skip list list • Unrolled linked list • Difference list • VList Lists: Array 0 1 size MAX_SIZE-1 3 6 7 5 2 L = int[MAX_SIZE] L[2]=7 Lists: Array 0 1 size MAX_SIZE-1 3 6 7 5 2 L = int[MAX_SIZE] L[2]=7 1 2 size MAX_SIZE 3 6 7 5 2 L[3]=7 2D array & A[i,j] = A + i*(nr_el_in_row*el_size) + j*el_size MulGple lists, 2-D-arrays, etc… Linear Lists • Operaons which one may want to perform on a linear list of n elements include: – gain access to the kth element of the list to examine and/or change the contents – insert a new element before or aer the kth element – delete the kth element of the list Reference: Knuth, The Art of Comptuer Programming, Vol 1, Fundamental Algorithms, 3rd Ed., p.238-9 Abstract Data Type (ADT) • High-level definiGon of data types • An ADT specifies – A collecon of data – A set of operaons on the data or subsets of the data • ADT does not specify how the operaons should be implemented • Examples – vector, list, stack, queue, deque, priority queue, table (map), associave array, set, graph, digraph ADT • A datatype is a set of values and an associated set of operaons • A datatype is abstract iff it is completely described by its set of operaons regradless of its implementaon • This means that it is possible to change the implementaon of the datatype without changing its use • The datatype is thus described by a set of procedures • These operaons are the only thing that a user of the abstracGon can assume Abstract data types: • DicGonary (key,value) • Stack (LIFO) • Queue (FIFO) • Queue (double-ended) • Priority queue (fetch highest-priority object) • ... DicGonary • Container of key-element (k,e) pairs • Required operaons: – insert( k,e ), – remove( k ), – find( k ), – isEmpty() • May also support (when an order is provided): – closestKeyBefore( k ), – closestElemAoer( k ) • Note: No duplicate keys Abstract data types • Container • Diconary/Map/ • Stack Associave array • Queue • MulGmap • Deque • Set • Priority queue • MulGset • String • Tree • Graph • Hash Some data structures for Diconary ADT • Unordered – Array – Sequence/list • Ordered – Array – Sequence (Skip Lists) – Binary Search Tree (BST) – AVL trees, red-black trees – (2; 4) Trees – B-Trees • Valued – Hash Tables – Extendible Hashing PrimiGve & composite types Primive types Composite types • Boolean (for boolean values True/False) • Array • Char (for character values) • Record (also called tuple • int (for integral or fixed-precision or struct) values) • Union • Float (for storing real number values) • Tagged union (also called • Double (a larger size of type float) a variant, variant record, • String (for string of chars) discriminated union, or • Enumerated type disjoint union) • Plain old data structure Linear data structures Arrays • Gap buffer Lists • • Array Hashed array tree • Doubly linked list • BidirecGonal • Heightmap • Linked list map • • Bit array Lookup table • Self-organizing list • • Bit field Matrix • Skip list • Parallel array • Bitboard • Unrolled linked list • • Bitmap Sorted array • • VList • Circular buffer Sparse array • • Xor linked list • Control table Sparse matrix • • Image Iliffe vector • Zipper • • Dynamic array Variable-length • Doubly connected edge list array Trees … Binary trees • Treap • 2-3 heap • X-fast trie Space-par..oning • Cover tree • AA tree • Weight-balanced • Soo heap • Y-fast trie trees • M-tree • AVL tree tree • Pairing heap • Ctrie • Segment tree • VP-tree • Binary search • Leoist heap • Interval tree • BK-tree tree B-trees • Treap Mul.way trees • Range tree • Bounding • Binary tree • B-tree • Beap • Ternary search • Bin interval hierarchy • Cartesian tree • B+ tree • Skew heap tree • Kd-tree • BSP tree • Pagoda • B*-tree • Ternary heap • And–or tree • Implicit kd-tree • Rapidly-exploring random tree • Randomized • B sharp tree • D-ary heap • (a,b)-tree • Min/max kd-tree binary search • Dancing tree • • Link/cut tree • AdapGve k-d tree tree • 2-3 tree • Tries • SPQR-tree • Kdb tree • Red-black tree Applica.on-specific • 2-3-4 tree • Trie • Spaghe stack • Quadtree • trees Rope • Queap • Disjoint-set data • Octree • Radix tree • Syntax tree • Scapegoat tree • structure • Fusion tree • Suffix tree Linear octree • Abstract syntax • Self-balancing • Fusion tree • Bx-tree • • Z-order tree binary search Suffix array • Enfilade • tree • Compressed UB-tree • Parse tree Heaps suffix array • ExponenGal tree • • Splay tree R-tree • Decision tree • Heap • FM-index • Fenwick tree • R+ tree • T-tree • Alternang • Binary heap • Generalised • Van Emde Boas • R* tree decision tree • Tango tree • Binomial heap suffix tree tree • • • Threaded binary Hilbert R-tree Minimax tree • • tree Fibonacci heap B-trie • X-tree • ExpecGminimax • AF-heap • Judy array tree • Top tree • Metric tree • Finger tree Hashes, Graphs, Other • Hashes Graphs • Directed acyclic graph • Bloom filter • Adjacency list • ProposiGonal directed • Distributed hash table • Adjacency matrix acyclic graph • Hash array mapped • Graph-structured • MulGgraph trie stack • Hypergraph • Hash list • Scene graph • Hash table • Binary decision Other • Hash tree diagram • Lightmap • Hash trie • Zero suppressed • Winged edge • Koorde decision diagram • Quad-edge • • Prefix hash tree And-inverter graph • RouGng table • • Directed graph • Symbol table *array (memory address) Lists: Array size MAX_SIZE 0 1 size MAX_SIZE-1 3 6 7 5 2 0 1 size 3 6 7 8 25 2 Insert 8 after L[2] 0 1 size 3 6 7 8 25 2 Delete last Lists: Array • Access i O(1) • Insert to end O(1) • Delete from end O(1) • Insert O(n) • Delete O(n) • Search O(n) 0 1 size 3 6 7 8 25 2 Insert 8 after L[2] 0 1 size 3 6 7 8 25 2 Delete last Linear Lists • Other operaons on a linear list may include: – determine the number of elements – search the list – sort a list – combine two or more linear lists – split a linear list into two or more lists – make a copy of a list Stack • push(x) -- add to end (add to top) • pop() -- fetch from end (top) • O(1) in all reasonable cases J • LIFO – Last In, First Out Linked lists head tail Singly linked head tail Doubly linked Linked lists: add head tail head tail size Linked lists: delete head tail (+ garbage collecGon?) head tail size Operaons • Array indexed from 0 to n – 1: k = 1 1 < k < n k = n access/change the kth element O(1) O(1) O(1) insert before or after the kth element O(n) O(n) O(1) delete the kth element O(n) O(n) O(1) • Singly-linked list with head and tail pointers k = 1 1 < k < n k = n access/change the kth element O(1) O(n) O(1) insert before or after the kth element O(1) O(n) O(1)1 O(n) O(1) delete the kth element O(1) O(n) O(n) 1 under the assumpGon we have a pointer to the kth node, O(n) otherwise Improving Run-Time Efficiency • We can improve the run-Gme efficiency of a linked list by using a doubly-linked list: Singly-linked list: Doubly-linked list: – Improvements at operaons requiring access to the previous node – Increases memory requirements... Improving Efficiency Singly-linked list: k = 1 1 < k < n k = n access/change the kth element O(1) O(n) O(1) insert before or after the kth element O(1) O(n) O(1)1 O(n) O(1) delete the kth element O(1) O(n) O(n) Doubly-linked list: k = 1 1 < k < n k = n access/change the kth element O(1) O(n) O(1) insert before or after the kth element O(1) O(1)1 O(1) delete the kth element O(1) O(1)1 O(1) 1 under the assumpGon we have a pointer to the kth node, O(n) otherwise • Array indexed from 0 to n – 1: k = 1 1 < k < n k = n access/change the kth element O(1) O(1) O(1) insert before or after the kth element O(n) O(n) O(1) delete the kth element O(n) O(n) O(1) • Singly-linked list with head and tail pointers k = 1 1 < k < n k = n access/change the kth element O(1) O(n) O(1) insert before or after the kth element O(1) O(n) O(1)1 O(n) O(1) delete the kth element O(1) O(n) O(n) • Doubly linked list k = 1 1 < k < n k = n access/change the kth element O(1) O(n) O(1) insert before or after the kth element O(1) O(1)1 O(1) delete the kth element O(1) O(1)1 O(1) IntroducGon to linked lists • Consider the following struct definiGon struct node { string word; int num; node *next; //pointer for the next node }; node *p = new node; p ? ? ? num word next IntroducGon