20.09.19 Big-Oh notation classes Algorithmics (6EAP) Class Informal Intuition Analogy MTAT.03.238 f(n) ∈ ο ( g(n) ) f is dominated by g Strictly below < Linear structures, sorting, f(n) ∈ O( g(n) ) Bounded from above Upper bound ≤ searching, Recurrences, f(n) ∈ Θ( g(n) ) Bounded from “equal to” = Master Theorem... above and below f(n) ∈ Ω( g(n) ) Bounded from below Lower bound ≥ Jaak Vilo f(n) ∈ ω( g(n) ) f dominates g Strictly above > 2019 Fall Jaak Vilo 1 Linear, sequential, ordered, list … Physical ordered list ~ array • Memory /address/ – Garbage collection • Files (character/byte list/lines in text file,…) • Disk – Memory, disk, tape etc – is an ordered sequentially addressed media. Disk fragmentation Linear data structures: Arrays Linear data structures: Lists • Array • Hashed array tree • Doubly linked list • • Bidirectional map Heightmap • Array list • Xor linked list • Bit array • Lookup table • Linked list • Zipper • Bit field • Matrix • Self-organizing list • Doubly connected edge • Bitboard • Parallel array list • Bitmap • Sorted array • Skip list • Circular buffer • Sparse array • Unrolled linked list • Difference list • Control table • Sparse matrix • VList • Image • Iliffe vector • Dynamic array • Variable-length array • Gap buffer 1 20.09.19 Lists: Array Lists: Array 0 1 size MAX_SIZE-1 0 1 size MAX_SIZE-1 3 6 7 5 2 3 6 7 5 2 L = int[MAX_SIZE] L = int[MAX_SIZE] L[2]=7 L[size++] = new L[2]=7 1 2 size MAX_SIZE 3 6 7 5 2 L[3]=7 L[++size] = new Multiple lists, 2-D-arrays, etc… 2D array & A[i,j] = A + i*(nr_el_in_row*el_size) + j*el_size Linear Lists Abstract Data Type (ADT) • Operations which one may want to perform • High-level definition of data types on a linear list of n elements include: • An ADT specifies – A collection of data – gain access to the kth element of the list to – A set of operations on the data or subsets of the data examine and/or change the contents • ADT does not specify how the operations should be – insert a new element before or after the kth implemented element • Examples – delete the kth element of the list – vector, list, stack, queue, deque, priority queue, table (map), associative array, set, graph, digraph Reference: Knuth, The Art of Comptuer Programming, Vol 1, Fundamental Algorithms, 3rd Ed., p.238-9 2 20.09.19 ADT Primitive & composite types • A datatype is a set of values and an associated set of Primitive types Composite types operations • Boolean (for boolean values True/False) • Array • A datatype is abstract iff it is completely described by its set • Char (for character values) • Record (also called tuple of operations regradless of its implementation • int (for integral or fixed-precision or struct) • This means that it is possible to change the implementation values) • Union of the datatype without changing its use • Float (for storing real number • Tagged union (also called a • The datatype is thus described by a set of procedures values) • Double (a larger size of type float) variant, variant record, • These operations are the only thing that a user of the discriminated union, or abstraction can assume • String (for string of chars) • Enumerated type disjoint union) • Plain old data structure Abstract Data Types (ADT) Abstract data types: • Some common ADTs, which have proved useful in a great variety of applications, are • Dictionary (key,value) Container Stack • Stack (LIFO) List Graph Set Queue • Queue (FIFO) Multiset Priority queue • Queue (double-ended) Map Double-ended queue Multimap Double-ended priority queue • Priority queue (fetch highest-priority object) • ... • Each of these ADTs may be defined in many ways and variants, not necessarily equivalent. Dictionary Some data structures for Dictionary ADT • Unordered • Container of key-element (k,e) pairs – Array • Required operations: – Sequence/list • – insert( k,e ), Ordered – Array – remove( k ), – Sequence (Skip Lists) – find( k ), – Binary Search Tree (BST) – isEmpty() – AVL trees, red-black trees • May also support (when an order is provided): – (2; 4) Trees – B-Trees – closestKeyBefore( k ), • Valued – closestElemAfter( k ) – Hash Tables • Note: No duplicate keys – Extendible Hashing 3 20.09.19 Linear data structures Trees … • Hashed array Binary trees • Treap • Soft heap • Ctrie • Interval tree • BK-tree Arrays Lists • AA tree • Weight-balanced • Pairing heap • Range tree • Bounding tree • AVL tree tree • Leftist heap Multiway trees • Bin interval hierarchy • Array • Binary search • Treap • Ternary search • Kd-tree • Doubly linked list • BSP tree • Heightmap tree B-trees • Beap tree • Implicit kd-tree • Bidirectional • Binary tree • B-tree • Skew heap • And–or tree • Min/max kd-tree• Rapidly-exploring • Linked list random tree • Lookup table • Cartesian tree • B+ tree • • (a,b)-tree • map Ternary heap Adaptive k-d tree • • Pagoda • B*-tree • D-ary heap • Link/cut tree • Kdb tree Matrix • Self-organizing list • Randomized • B sharp tree • SPQR-tree • Bit array • • Quadtree Application-specific binary search • Dancing tree • Tries • Spaghetti stack • Octree trees • Parallel array tree • Bit field • Skip list • 2-3 tree • Trie • Disjoint-set data • Linear octree • Syntax tree • Red-black tree • 2-3-4 tree • Radix tree structure • Z-order • Abstract syntax • Sorted array • Rope • Bitboard • Unrolled linked list • Queap • Suffix tree • Fusion tree • UB-tree tree • Scapegoat tree • Enfilade • Sparse array • Fusion tree • Suffix array • R-tree • Parse tree • Self-balancing • Exponential tree • Bitmap • Bx-tree • Compressed • R+ tree • Decision tree • VList binary search suffix array • Fenwick tree • Alternating • Sparse matrix tree • R* tree • Circular buffer Heaps • Van Emde Boas decision tree • Splay tree • FM-index • Hilbert R-tree • Iliffe vector • Xor linked list • Heap tree • Minimax tree • Control table • T-tree • Generalised • X-tree • Binary heap suffix tree • Expectiminimax • Tango tree • Metric tree • Variable-length • Zipper • Binomial heap • B-trie tree • Image • Threaded binary • Cover tree • Fibonacci heap • Judy array Space-partitioning • Finger tree array tree • M-tree • Doubly connected edge list • AF-heap • X-fast trie trees • Dynamic array • Top tree • Segment tree • VP-tree • 2-3 heap • Y-fast trie • Gap buffer Hashes, Graphs, Other *array (memory address) Lists: Array size • Hashes Graphs • Propositional directed MAX_SIZE 0 1 size MAX_SIZE-1 • Bloom filter • Adjacency list acyclic graph 3 6 7 5 2 • Distributed hash table • Adjacency matrix • Multigraph • Hash array mapped • Graph-structured • Hypergraph trie stack • Hash list • Scene graph Other 0 1 size • Hash table • Binary decision • Lightmap 3 6 7 8 25 2 Insert 8 after L[2] • Hash tree diagram • Winged edge • Zero suppressed • Hash trie • Quad-edge 0 1 size • Koorde decision diagram • Routing table • And-inverter graph 3 6 7 8 25 2 Delete last • Prefix hash tree • Symbol table • • Directed graph • Directed acyclic graph Lists: Array Linear Lists • Access i O(1) • Insert to end O(1) • Other operations on a linear list may include: • Delete from end O(1) • Insert O(n) – determine the number of elements • Delete O(n) • Search O(n) – search the list 0 1 size – sort a list 3 6 7 8 25 2 Insert 8 after L[2] – combine two or more linear lists – split a linear list into two or more lists 0 1 size – 3 6 7 8 25 2 Delete last make a copy of a list 4 20.09.19 Linked lists Linked lists: add head tail head tail Singly linked head tail head tail Doubly linked size Linked lists: delete • Array indexed from 0 to n – 1: k = 1 1 < k < n k = n head tail (+ garbage collection?) access/change the kth element O(1) O(1) O(1) insert before or after the kth element O(n) O(n) O(1) delete the kth element O(n) O(n) O(1) • Singly-linked list with head and tail pointers k = 1 1 < k < n k = n head tail access/change the kth element O(1) O(n) O(1) insert before or after the kth element O(1) O(n) O(1)1 O(n) O(1) delete the kth element O(1) O(n) O(n) • Doubly linked list k = 1 1 < k < n k = n access/change the kth element O(1) O(n) O(1) insert before or after the kth element O(1) O(1)1 O(1) size delete the kth element O(1) O(1)1 O(1) Introduction to linked lists Introduction to linked lists: inserting a node • Consider the following struct definition • node *p; struct node { • p = new node; string word; • p->num = 5; int num; • p->word = "Ali"; node *next; //pointer for the next node • p->next = NULL }; node *p = new node; • p 5 Ali p ? ? ? num word next num word next 5 20.09.19 Introduction to linked lists: adding a new Introduction to linked lists node node *p; • How can you add another node that is pointed by p->link? p = new node; • node *p; • p = new node; p->num = 5; • p->num = 5; p->word = "Ali"; • p->word = "Ali"; • p->next = NULL; p->next = NULL; • node *q; node *q; q q • q = new node; p 5 Ali ? num word link p 5 Ali ? ? ? ? num word link num word link Introduction to linked lists Introduction to linked lists node *p, *q; node *p, *q; p = new node; p = new node; p->num = 5; p->num = 5; p->word = "Ali"; p->word = "Ali"; p->next = NULL; p->next = NULL; q = new node; q = new node; q->num=8; q->num=8; q->word = "Veli"; p->next = q; q->word = "Veli"; q->next = NULL; q q p 5 Ali ? 8 Veli ? p 5 Ali ? 8 Veli num word next num word next num word link num word link Pointers in C/C++ Book-keeping p = new node ; delete p ; • malloc, new – remember what has been p = new node[20] ; created free(p), delete (C/C++) • When you need many small areas to be p = malloc( sizeof( node ) ) ; free p ; allocated, reserve a big chunk (array) and maintain your own set of free objects • Elements of array of objects can be pointed by p = malloc( sizeof( node ) * 20 ) ; the pointer to an object.