08.09.20 Big-Oh notation classes Algorithmics (6EAP) Class Informal Intuition Analogy MTAT.03.238 f(n) ∈ ο ( g(n) ) f is dominated by g Strictly below < Bounded from above Upper bound Linear structures, sorting, f(n) ∈ O( g(n) ) ≤ Bounded from “equal to” searching, Recurrences, f(n) ∈ Θ( g(n) ) above and below = Master Theorem... Bounded from below Lower bound f(n) ∈ Ω( g(n) ) ≥ Jaak Vilo f(n) ∈ ω( g(n) ) f dominates g Strictly above > 2020 Fall Jaak Vilo 1 Linear, sequential, ordered, list … Physical ordered list ~ array • Memory /address/ – Garbage collection • Files (character/byte list/lines in text file,…) • Disk – Disk fragmentation Memory, disk, tape etc – is an ordered sequentially addressed media. Data structures Linear data structures: Arrays • https://en.wikipedia.org/wiki/List_of_data_structures • Array • Hashed array tree • Bidirectional map • Heightmap • Bit array • Lookup table • https://en.wikipedia.org/wiki/List_of_terms_relating_to_algo • Bit field • Matrix rithms_and_data_structures • Bitboard • Parallel array • Bitmap • Sorted array • Circular buffer • Sparse array • Control table • Sparse matrix • Image • Iliffe vector • Dynamic array • Variable-length array • Gap buffer 1 08.09.20 Linear data structures: Lists Lists: Array • Doubly linked list 0 1 size MAX_SIZE-1 • Array list • Xor linked list 3 6 7 5 2 • Linked list • Zipper • Self-organizing list • Doubly connected edge L = int[MAX_SIZE] • Skip list list L[2]=7 • Unrolled linked list • Difference list • VList Lists: Array Multiple lists, 2-D-arrays, etc… 0 1 size MAX_SIZE-1 3 6 7 5 2 L = int[MAX_SIZE] L[2]=7 L[size++] = new 1 2 size MAX_SIZE 3 6 7 5 2 L[3]=7 L[++size] = new 2D array Linear Lists • Operations which one may want to perform on a linear list of n elements include: – gain access to the kth element of the list to examine and/or change the contents & A[i,j] = A + i*(nr_el_in_row*el_size) + j*el_size – insert a new element before or after the kth element – delete the kth element of the list Reference: Knuth, The Art of Comptuer Programming, Vol 1, Fundamental Algorithms, 3rd Ed., p.238-9 2 08.09.20 Abstract Data Type (ADT) ADT • High-level definition of data types • A datatype is a set of values and an associated set of operations • An ADT specifies • A datatype is abstract iff it is completely described by its set – A collection of data – A set of operations on the data or subsets of the data of operations regradless of its implementation • This means that it is possible to change the implementation • ADT does not specify how the operations should be of the datatype without changing its use implemented • The datatype is thus described by a set of procedures • These operations are the only thing that a user of the • Examples abstraction can assume – vector, list, stack, queue, deque, priority queue, table (map), associative array, set, graph, digraph Primitive & composite types Abstract data types: Primitive types Composite types • Boolean (for boolean values True/False) • Array • Dictionary (key,value) • Char (for character values) • Record (also called tuple • int (for integral or fixed-precision or struct) • Stack (LIFO) values) • Union • • Float (for storing real number Queue (FIFO) values) • Tagged union (also called a • Queue (double-ended) • Double (a larger size of type float) variant, variant record, • String (for string of chars) discriminated union, or • Priority queue (fetch highest-priority object) disjoint union) • Enumerated type • ... • Plain old data structure Abstract Data Types (ADT) Dictionary • Some common ADTs, which have proved useful • Container of key-element (k,e) pairs in a great variety of applications, are • Required operations: Container Stack – insert( k,e ), List Graph – remove( k ), Set Queue – find( k ), Multiset Priority queue – isEmpty() Map Double-ended queue • May also support (when an order is provided): Multimap Double-ended priority queue – closestKeyBefore( k ), • Each of these ADTs may be defined in many – closestElemAfter( k ) ways and variants, not necessarily equivalent. • Note: No duplicate keys 3 08.09.20 Some data structures for Dictionary ADT Linear data structures • Unordered • Hashed array – Array Arrays Lists tree – Sequence/list • Array • Heightmap • Doubly linked list • Bidirectional • Ordered • Lookup table • Linked list map – Array • • Bit array Matrix • Self-organizing list – Sequence (Skip Lists) • Parallel array • Bit field • Skip list – Binary Search Tree (BST) • Sorted array • Bitboard • Unrolled linked list – AVL trees, red-black trees • • Bitmap Sparse array – (2; 4) Trees • Sparse matrix • VList • Circular buffer – B-Trees • Iliffe vector • Xor linked list • Control table • Valued • Variable-length • Image • Zipper – Hash Tables array • Dynamic array • Doubly connected edge list – Extendible Hashing • Gap buffer Trees … Hashes, Graphs, Other Binary trees • Treap • Soft heap • Ctrie • Interval tree • BK-tree • AA tree • Weight-balanced• Pairing heap • Range tree • Bounding • Hashes Graphs • Propositional directed • AVL tree tree • Leftist heap Multiway trees • Bin interval • Bloom filter • Adjacency list acyclic graph • Binary search • Treap • Ternary search • Kd-tree hierarchy • BSP tree tree B-trees • Beap tree • Implicit kd-tree • Distributed hash table• Adjacency matrix • Multigraph • • Binary tree • B-tree • • And–or tree • Rapidly-exploring Skew heap Min/max kd-tree random tree • • Cartesian tree • B+ tree • Ternary heap • (a,b)-tree • Adaptive k-d tree • Hash array mapped • Graph-structured Hypergraph • Pagoda • B*-tree • D-ary heap • Link/cut tree • Kdb tree • • • trie stack Randomized B sharp tree • SPQR-tree • Quadtree Application-specific binary search • Dancing tree • • Spaghetti stack • tree Tries Octree trees • Hash list • Scene graph Other • 2-3 tree • Trie • Disjoint-set data • Linear octree • Syntax tree • Red-black tree structure • 2-3-4 tree • Radix tree • Z-order • Abstract syntax • Rope • Fusion tree • Hash table • Binary decision • Lightmap • Queap • Suffix tree • UB-tree tree • • Scapegoat tree • Fusion tree • Suffix array Enfilade • R-tree • Parse tree • Hash tree diagram • Winged edge • • Self-balancing • Bx-tree • Compressed Exponential tree • R+ tree • Decision tree binary search • • • Zero suppressed tree suffix array Fenwick tree • R* tree Alternating • Hash trie • Quad-edge • • • Splay tree Heaps FM-index Van Emde Boas • Hilbert R-tree decision tree decision diagram • • tree • • Koorde • Routing table • T-tree Heap Generalised • X-tree Minimax tree • • Binary heap suffix tree • Metric tree • Expectiminimax • And-inverter graph Tango tree tree • Prefix hash tree • Symbol table • Binomial heap • B-trie • Cover tree • Threaded binary • tree • Fibonacci heap • Judy array Space-partitioning • M-tree Finger tree • Directed graph • AF-heap • X-fast trie trees • • Top tree • • VP-tree • 2-3 heap • Y-fast trie Segment tree • Directed acyclic graph *array (memory address) Lists: Array Lists: Array size MAX_SIZE • Access i O(1) 0 1 size MAX_SIZE-1 • Insert to end O(1) • Delete from end O(1) 3 6 7 5 2 • Insert O(n) • Delete O(n) • Search O(n) 0 1 size 0 1 size 3 6 7 8 25 2 Insert 8 after L[2] 3 6 7 8 25 2 Insert 8 after L[2] 0 1 size 0 1 size 3 6 7 8 25 2 Delete last 3 6 7 8 25 2 Delete last 4 08.09.20 Linear Lists Linked lists head tail • Other operations on a linear list may include: Singly linked – determine the number of elements – search the list – sort a list head tail – combine two or more linear lists – split a linear list into two or more lists Doubly linked – make a copy of a list Linked lists: add Linked lists: delete (+ garbage collection?) head tail head tail head tail head tail size size • Array indexed from 0 to n – 1: Introduction to linked lists k = 1 1 < k < n k = n access/change the kth element O(1) O(1) O(1) • Consider the following struct definition insert before or after the kth element O(n) O(n) O(1) delete the kth element O(n) O(n) O(1) struct node { string word; • Singly-linked list with head and tail pointers k = 1 1 < k < n k = n int num; access/change the kth element O(1) O(n) O(1) node *next; //pointer for the next node insert before or after the kth element O(1) O(n) O(1)1 O(n) O(1) delete the kth element O(1) O(n) O(n) }; • Doubly linked list node *p = new node; k = 1 1 < k < n k = n access/change the kth element O(1) O(n) O(1) p ? ? ? insert before or after the kth element O(1) O(1)1 O(1) num word next delete the kth element O(1) O(1)1 O(1) 5 08.09.20 Introduction to linked lists: inserting a Introduction to linked lists: adding a new node node • node *p; • How can you add another node that is pointed by p->link? • p = new node; • node *p; • p->num = 5; • p = new node; • p->num = 5; • p->word = "Ali"; • p->word = "Ali"; • p->next = NULL; • p->next = NULL • node *q; q • • p 5 Ali p 5 Ali ? num word next num word link Introduction to linked lists Introduction to linked lists node *p; node *p, *q; p = new node; p = new node; p->num = 5; p->num = 5; p->word = "Ali"; p->word = "Ali"; p->next = NULL; p->next = NULL; q = new node; node *q; q->num=8; q q = new node; q->word = "Veli"; q ? p 5 Ali ? ? ? p 5 Ali ? 8 Veli ? num word link num word link num word next num word next Introduction to linked lists Pointers in C/C++ node *p, *q; p = new node; p = new node ; delete p ; p->num = 5; p->word = "Ali"; p = new node[20] ; p->next = NULL; q = new node; q->num=8; p = malloc( sizeof( node ) ) ; free p ; q->word = "Veli"; p->next = q; q->next = NULL; p = malloc( sizeof( node ) * 20 ) ; q (p+10)->next = NULL ; /* 11th elements */ p 5 Ali ? 8 Veli num word link num word link 6 08.09.20 Book-keeping Object • malloc, new – “remember” what has been • Object = new object_type ; created free(p), delete (C/C++) • When you need many small areas to be • Equals to creating a new object with necessary allocated, reserve a big chunk (array) and size of allocated memory (delete can free it) maintain your own set of free objects • Elements of array of objects can be pointed by the pointer to an object.