Ordered Dictionary Sorted Tables Dictionary is a set of elements, the If a dictionary D is ordered, we can keys, supported by the dictionary store its items in a vector S by non- operations, such as: decreasing order of the keys. findElement (k) -- k position, e element This allows for faster searching than insertItem(k,e) would be possible had S been, e.g., a removeElement (k) linked list. Ordered dictionary maintains an We refer to this ordered vector order relation for the elements in it. implementation of a dictionary D as a lookup table. week 3 Complexity of Algorithms 1 week 3 Complexity of Algorithms 2 Binary Search Binary Search Accessing an element of S (in array- Looking for k in S, the range in S is based representation of size n) by its defined as a pair of ranks: low and high, rank takes O(1) time s.t., all elements in S with ranks < low The item at rank i has a key no (> high) are smaller (larger) then k smaller than keys of the items of ranks 0,…,i-1, and no larger than keys Initially, low = 0 and high = n-1 of the items of ranks i+1,…,n-1. key(i) denotes the key at rank i, and Searching is done by decreasing the elem(i) denotes an element in key(i) range of the elements in S week 3 Complexity of Algorithms 3 week 3 Complexity of Algorithms 4 1 Binary Search Binary Search - Algorithm In order to decrease a size of the range we compare k to the key of the median mid of the range, i.e., mid = ë(low+high)/2û 3 cases are possible k = key(mid), the search is completed successfully k < key(mid), search continued with high = mid-1 k > key(mid), search continued with low = mid+1 week 3 Complexity of Algorithms 5 week 3 Complexity of Algorithms 6 Binary Search - Illustration Binary Search - Complexity Let function f(n) represent the running time of the binary search method We can characterize the running time of the recursive binary search algorithm as follows: Binary search runs in time O(log n) week 3 Complexity of Algorithms 7 week 3 Complexity of Algorithms 8 2 Binary Search - Complexity Binary Search Tree (BST) Binary search vs. linear file Binary Search Tree (BST) applies the motivation of the binary search procedure to a tree-based data structure. In BST each internal node v stores an element e, s.t., the elements stored in the left subtree of v are less than or equal to e, and the elements stored in the right subtree of v are greater than or equal to e . week 3 Complexity of Algorithms 9 week 3 Complexity of Algorithms 10 BST Searching in BST An inorder traversal of BST visits the elements stored in such a tree in non- decreasing order. week 3 Complexity of Algorithms 11 week 3 Complexity of Algorithms 12 3 Searching in BST - Analysis Insertion in BST Insertion of 78 implemented in time O(h) week 3 Complexity of Algorithms 13 week 3 Complexity of Algorithms 14 Removal in BST Removal in BST Removal of a node (32) with one external child Removal of a node (65) with two internal is done in time O(h) children is done in time O(h) week 3 Complexity of Algorithms 15 week 3 Complexity of Algorithms 16 4 Inefficiency of general BSTs AVL Trees Height-Balance Property: for every All operations in BST are performed in internal node v of T, the heights of the time O(h), where h is the height of BST children of v can differ by at most 1 Unfortunately h might be as large as n, e.g., after n consecutive insertions of elements with keys in increasing order The advantages of the binary search (O(log n) time update) might be lost if BST is not balanced week 3 Complexity of Algorithms 17 week 3 Complexity of Algorithms 18 AVL Trees Insertion in AVL Theorem: the height of an AVL tree An insertion in an AVL tree begins as T storing n items is O(log n) insertion in a general BST, i.e., we attach a new external node (leaf) to BST Consequence 1: the search in AVL can This action may violate the height-balance be performed in time O(log n) property, i.e., for some nodes the action Consequence 2: The insertion and the may increase their heights by one removal in AVL need more careful The bottom-up mechanism (based on implementation (rotations) rotations) is applied to fix the “unbalance’’ of AVL subtrees week 3 Complexity of Algorithms 19 week 3 Complexity of Algorithms 20 5 Fixing AVL after Insertion Single rotations in AVL Fixing of an AVL after insertion of 54 week 3 Complexity of Algorithms 21 week 3 Complexity of Algorithms 22 Double rotations in AVL Removal in AVL An removal in an AVL tree begins as removal in a general BST This action may violate the height-balance property, i.e., for some nodes the action may decrease their heights by one The bottom-up mechanism (based on rotations) is applied to fix the “unbalance’’ of AVL subtrees week 3 Complexity of Algorithms 23 week 3 Complexity of Algorithms 24 6 Fixing AVL after Removal AVL Performance All operations (search, insertion and removal) can be implemented in AVL in O(log n) time Fixing of an AVL after removal of 32 week 3 Complexity of Algorithms 25 week 3 Complexity of Algorithms 26 (2,4) Trees (2,4) Trees Every node in (2,4) tree has at least 2 Each internal node v in (2,4) tree contains 1, and at most 4 children 2 or 3 keys that define the ranges of keys stored in 2, 3 or 4 (respectively) consecutive All external nodes (leaves) in (2,4) subtrees of v tree have the same depth Theorem: The height of a (2,4) tree storing n items is Q(log n) week 3 Complexity of Algorithms 27 week 3 Complexity of Algorithms 28 7 (2,4) Trees - Search (2,4) Tree - Insertion Search for an key k in (2,4) tree T is An insertion of k in an (2,4) tree T begins done via tracing the path in T starting with a search for an internal node on the lowest level that could accommodate k at the root in a top-down manner without violation of the range rule Visiting a node v we compare k with This action may overflow the node-size of keys (k ) stored at v: a node v acquiring new key k, i.e., the i number of keys in v can grow up to 4 If k = k the search is completed i The bottom-up mechanism (based on split th If ki £ k £ ki+1, the i+1 subtree of v is operation) is applied to fix the “overflown’’ searched recursively nodes of (2,4) tree T week 3 Complexity of Algorithms 29 week 3 Complexity of Algorithms 30 (2,4) Trees – Split Operation Sequence of Insertions - 1 Example of a split operation week 3 Complexity of Algorithms 31 week 3 Complexity of Algorithms 32 8 Sequence of Insertions - 2 (2,4) Tree - Removal Removal of key k from (2,4) tree T begins with search for a node v possessing key ki = k Key ki is replaced by the largest key in the ith consecutive subtree of v The bottom-up mechanism (based on transfer and fusion operation) is applied to fix the “underflown’’ nodes of (2,4) tree week 3 Complexity of Algorithms 33 week 3 Complexity of Algorithms 34 Sequence of Removals - 1 Sequence of Removals - 2 week 3 Complexity of Algorithms 35 week 3 Complexity of Algorithms 36 9 Sequence of Removals - 3 (2,4) Tree Performance The height of a (2,4) tree storing n elements is O(log n) A split, transfer and fusion operations take O(1) time A search, insertion, and removal of an element in a tree visits O(log n) nodes week 3 Complexity of Algorithms 37 week 3 Complexity of Algorithms 38 10.