CS 270 CS 270 Algorithms General remarks Algorithms Oliver Oliver Week 7 Kullmann Kullmann

Binary Binary search search Arrays, lists, pointers and rooted trees Lists Lists Pointers Pointers

Trees We conclude elementary data structures by discussing and Trees 1 Binary search Implementing implementing arrays, lists, pointers and trees. Implementing rooted trees rooted trees We also consider binary search. Tutorial Tutorial 2 Lists Reading from CLRS for week 7 3 Pointers 1 Chapter 10, Sections 10.2, 10.3, 10.4. 4 Trees

5 Implementing rooted trees

6 Tutorial

CS 270 CS 270 Arrays Algorithms Vectors Algorithms Oliver Oliver Kullmann Kullmann Arrays are the most fundamental : The dynamic form of an array (i.e., it can grow) can be called a Binary Binary search vector (as for ++; or “dynamic array”): search

An array A is a static data-structure, with a fixed length Lists Lists The growth of the vector happens by internally holding an n N0, holding n objects of the same type. Pointers Pointers ∈ array, and when the need arises, to allocate a new, bigger Access to elements happens via A[i] for indices i, typically Trees Trees array, copy the old content, and delete the old array. 0-based (C-based languages), that is, i 0,..., n 1 , or Implementing Implementing ∈ { − } rooted trees When done “infrequently”, insertions (and deletions) at the rooted trees 1-based, that is, i 1,..., n . Tutorial Tutorial ∈ { } end of the vector require only amortised constant time; see This access, called , happens in constant the tutorial. time, and can be used for reading and writing. However insertions and deletions at the beginning of the Due to the fixed length of arrays, one cannot really speak of vector (or somewhere else) needs time linear in the current “insertion” and “deletion” for arrays. size of the vector, since the elements need to be shifted. Search in general is slow (one has to run through all A vector with additional structure, where also insertions and elements in the worst case), however fast in sorted arrays, deletions at the beginning happens in amortised constant via “binary search”. time, is typically called a deque (a “double-ended queue”). CS 270 CS 270 Searching in sorted vectors Algorithms Binary search Algorithms Searching in general vectors takes linear time (running through Oliver Oliver Kullmann class BinarySearch Kullmann all elements): { Binary public static int binary search( final int [] Binary search search 1 However, if the vector is sorted (we assume, as it is the A, int begin , int end , final int x) Lists { Lists default, ascending order), then it can be done in i f (A == null ) return 1; Pointers − Pointers logarithmic time (in the length n of the vector). i f (begin == end) return 1; Trees − Trees 2 We present the Java-function binary search, which while ( true ) Implementing { Implementing searches for an element x in an array A. rooted trees final int mid = (begin+end)/2; rooted trees Tutorial Tutorial 3 Instead of just returning true or false (for found or not), i f (A[mid] == x) return mid; i f (begin+1 == end) return 1; it is more informative to return an index i with A[i]= x, if − i f (A[mid] < x) found, and to return 1 otherwise. { − begin = mid+1; 4 Since it might not be so easy to (efficiently) form i f (begin == end) return 1; sub-arrays, our version of binary search allows to specify − a sub-array by its indices begin and end. } else end = mid; 5 As it is usually best, this so-called “range” is right-open, i.e., the beginning is included, but the ending excluded. } 6 The role model for that is begin = 0 and end = n. }

CS 270 CS 270 Binary search (cont.) Algorithms Binary search with assertions Algorithms Oliver Oliver Kullmann public static int binary search( final int [] A Kullmann , int begin , int end , final int x) Binary { Binary search i f (A == null ) return 1; search − Lists assert(0 <= begin <= end <= A.length); Lists public static int binary search( final int [] Pointers i f (begin == end) return 1; Pointers − A, final int x) Trees while ( true ) Trees { { i f (A == null ) return 1; Implementing assert(0 <= begin < end <= A.length); Implementing − rooted trees rooted trees return binary search(A, 0, A.length, x); Tutorial final int mid = (begin+end)/2; Tutorial assert(begin <= mid < end); } i f (A[mid] == x) return mid; } i f (begin+1 == end) return 1; − assert(begin < mid) ; i f (A[mid] < x) begin = mid+1; { i f (begin == end) return 1; − } else end = mid; } } CS 270 CS 270 Analysing binary search Algorithms Analysing binary search (cont.) Algorithms Oliver Oliver Kullmann Kullmann

Binary We obtain the second case of the Master Theorem Binary We have a divide-and-conquer algorithm, with the characteristic search search (log (1) = 0), whence recurrence Lists 2 Lists T (n)= T (n/2) + 1. Pointers Pointers Trees T (n) = Θ(lg n). Trees

Implementing Implementing rooted trees Recall that this actually only implies an upper bound for the rooted trees That’s because we divide the array into two (nearly) equal Tutorial run-time of binary search — the lower bound implied by the Tutorial parts, i.e., b = 2 in the standard form of the recurrence for implicit Ω holds only for the recurrence, but not necessarily for the Master Theorem. the run-time. While we only need to investigate one of the two parts (due However, it is not too hard to see that also for the algorithm, to the sorting!), i.e., a = 1 for the Master Theorem. and actually for every possible search algorithm, we need at least Finally the work done for splitting happens in constant lg(n) comparisons. time, and thus c = 0 for the Master Theorem.

CS 270 CS 270 Removing random access from vectors, gaining fast Algorithms Pointers to next and previous elements Algorithms Oliver Like a vector, the elements of a list are arranged in a linear order. Oliver general insertion and deletion: Linked lists Kullmann Kullmann

Binary Binary search The basic idea here is that search With vectors we obtain random access — via indices, which are Lists each elements contains a pointer Lists just natural numbers, and thus arbitrary arithmetic can be Pointers to the next and the previous element of the list. Pointers performed with them — due to the contiguous and uniform Trees Trees

storage scheme: underlying is an array, which is stored as one Implementing Implementing So a list-object x is a triple: contiguous block of memory cells, all of the same size. rooted trees rooted trees Tutorial Tutorial But to maintain contiguity, only deletions and insertions at the x.prev is a pointer to the previous element in the list; end of the vector are efficient (amortised constant-time) — if we x.next is a pointer to the next element in the list; give up contiguity, then we loose random access, but we gain x.key contains the key (or the data, if there is no “key”). efficient arbitrary deletions and insertions: (linked) lists. For the first element of the list, x.prev is NIL, and for the last Lists formally implement a dictionary (search, insertion, element, x.next is NIL. deletion), but, different from “real” dictionaries, search is slow, while insertion and deletion is very fast, i.e., constant-time. The whole list is represented by a pointer L to the first element (as usual, NIL if the list is empty). CS 270 CS 270 Searching Algorithms Excursion: Searching, in C++ Algorithms Oliver Oliver Kullmann For comparison, the same code in C++: Kullmann Binary Binary search const List search( const List L, const Key k) search ∗ ∗ { The SEARCH-function in Java-like code, using List as the Lists while (L != nullptr and L.key != k) Lists L = L.next; ∗ pointer-type (recall — nearly everything in Java is a pointer!): Pointers ∗ Pointers Trees return L; Trees static List search(List L, final Key k) { Implementing } Implementing while (L != null && L.key != k) rooted trees rooted trees L = L.next; Tutorial We see that in C/C++ we not only have pointers, but also Tutorial return L; values (as the ints!), and thus one can distinguish between pointers and values: The *-operator makes pointer-types from } value-types, and dereferences pointers (to values). Further Note that if x is not found, then L will automatically finally remarks: become NIL (that is, null for Java). “const List ” means that we do not change the values. ∗ More idiomatic would be the use of the > operator, − which makes for example L >key instead of L.key. − ∗

CS 270 CS 270 Insertion Algorithms Deletion Algorithms Oliver Oliver Kullmann Kullmann

Binary Binary Inserting a list-object x into list L, at the beginning, again as search Deleting the list-element x from list L: search Java-code: Lists Lists Pointers static List delete(List L, final List x) Pointers static List insert(List L, final List x) { { Trees assert(x != null ); Trees assert(x != null ); Implementing assert(L != null ); Implementing x.next = L; rooted trees rooted trees Tutorial i f (x.prev != null ) x.prev.next = x.next; Tutorial x.prev = null ; else L = x.next; i f (L != null ) L.prev = x; i f (x.next != null ) x.next.prev = x.prev; L = x; return L; return L; } } Again the return-value is the new list. Note that the return-value is the new list. CS 270 CS 270 Other forms of linked lists Algorithms Final remarks on lists Algorithms Oliver Oliver Kullmann Kullmann

Binary Binary search What we have outlined as the class List (see the lab session for search Our from of a (the standard form) is more precisely Lists Lists the full implementation) would typically be considered as a type called doubly linked list, since we have back- and forth-pointers Pointers Pointers ListNode, while the List-class itself would be kind of a for every node. Trees Trees Implementing manager-class, with class ListNode as private type hidden in it. Implementing In a singly linked list we only have the next-pointer; see the rooted trees rooted trees tutorial for a discussion. Tutorial This yields finally more user-convenience. Tutorial The usage of the field L.head in the book is a faint move A list can be sorted or unsorted, if a linear order on the keys is in that direction. given. Our approach is more rough, kind of swiss-army-knife Finally we can also have a circular list, if we link first and last approach (as one would do it in a simple-minded element appropriately. C-implementation). But this has also its advantages ...

CS 270 CS 270 The concept of “pointer” Algorithms The concept of “pointer” (cont.) Algorithms Oliver Oliver A “pointer” as a value is typically a memory address, and thus Kullmann Kullmann

“points” to some value (at that memory address). Pointers are Binary Binary the basic means for indirection. search search Lists Lists

In Java nearly everything (that is, besides the primitive types like Pointers Variables for objects are always pointers. Pointers int) is a pointer — so you can’t see them! Trees If you for example compare two such variables via x == y, Trees Implementing Implementing The situation is much better in C/C++, where we have full rooted trees the compiler assumes you mean the pointers x, y rooted trees duality between pointers and values: Tutorial themselves. Tutorial If one the other hand you write something like x.prev, then For a type T we construct the pointer type T . ∗ the value pointed to by x is taken. For a pointer p of type T , via the dereference operator ∗ p we get a value of type T (pointed to by p). ∗ For a value v of type T , via the reference operator &v we get a pointer of type T (pointing to v). ∗ So in C/C++ we have full control on values versus pointers. On the contrary, in Java it is the compiler which decides for you: CS 270 CS 270 Simulating pointers Algorithms Allocation and deletion of objects Algorithms Oliver Oliver Kullmann In Java, allocation of new objects (on the so-called “”) Kullmann The basic technique to simulate pointers of type T is to use ∗ Binary happens via the operator new, however deallocation is not under Binary arrays, with indices replacing the pointers: search the control of the programmer, but happens via the garbage search Lists Lists 1 collection. The objects (for example the list-objects) can be distributed Pointers Pointers over several arrays (in this case three arrays, two for the Trees In C++, one has symmetry between allocation (operator new) Trees pointers (i.e., indices), one for the key), each array holding Implementing and deallocation (operator delete), and furthermore new objects Implementing rooted trees rooted trees some part of the data, and with data belonging to the same don’t have to be created on the (program-)heap, but can also be Tutorial Tutorial object using the same index. created on the (program-)stack, namely using values (the slogan 2 Or the objects can be put into a single array, of type T is “when in C++, do like the ints do”). (which might have advantages for caching). Accordingly, Java only knows constructors, but no 3 Going to a more basic level, the single array might hold destructors, while C++ has both. basic memory units (like real memory), where then an object of type T uses several units, and via an offset (stored C++ allows much more fine-grained memory-control. in an offset-table) one accesses the parts (data members) Especially the time of deletion is under full control of the of an object. programmer (which is important for time-critical applications, where garbage collection must not kick-in).

CS 270 CS 270 The notion of a “”: origins Algorithms The notion of a “tree”: roots Algorithms Oliver Oliver Kullmann To obtain a rooted tree, one of the vertices of the tree has to be Kullmann In Week 4 we defined the notion of a “tree” (see Sections B.5.1, Binary distinguished as a “root” (recall BFS and DFS), and then the Binary search search B.5.2 and B.5.3 in CLRS for more information on this topic). tree is typically drawn growing downwards (in the direction of Lists Lists

This original notion of a tree (in our context) comes from Pointers reading). For our example T we can make seven rooted trees out Pointers mathematics, where a tree is a “connected undirected graph Trees of T , for example choosing vertex 1 resp. vertex 6 as the root: Trees without cycles”, e.g. Implementing Implementing rooted trees (T , 1) = 1 (T , 6) = 6 rooted trees

Tutorial Tutorial T = 1 2 6 7 2 5 ❃❃ ✁ ❂❂ ❃❃ ✁ ❂ ❃ ✁✁ ❂ ❃❃ 3 7 3 ❃ ❂ ❂ 4 3 5 ✁✁ ❂❂ ✁✁ ❂❂ ✁✁ ❂ ✁✁ ❂ 4 5 4 2 ❂ We have vertices 1,..., 7 and edges between them. From any ✁✁ ❂❂ ✁✁ ❂ vertices we can reach every other vertex, and this essentially in a 6 7 1 unique way (this is the special property of trees). The leaves of (T , 1) are 4, 6, 7, the leaves of (T , 6) are 7, 4, 1. CS 270 CS 270 The height of rooted trees Algorithms The notion of a “tree”: order Algorithms Oliver Oliver Kullmann Kullmann The same two rooted trees as before can be drawn differently, Binary Binary Given a rooted tree T , its height ht(T ) search since there is no order on the children of a node, for example: search Lists (T , 1) = 1 (T , 6) = 6 Lists is the length of the longest path from the root to some leaf, Pointers Pointers Trees Trees where the length of a path is the number of edges on it. 2 5 Implementing ❂ Implementing rooted trees ✁✁ ❂❂ rooted trees The recursive definition is as follows: ✁✁ ❂ Tutorial 3 7 3 Tutorial ❂ ❂ ✁✁ ❂❂ ✁✁ ❂❂ the trivial tree, just consisting of the root, has height 0; ✁✁ ❂ ✁✁ ❂ 5 4 2 4 if a rooted tree T has subtrees T ,..., T for m 1 ❂ 1 m ≥ ✁✁ ❂❂ (always at the root), then ✁✁ ❂ 6 7 1 m ht(T )=1+ max ht(Ti ). i=1 However, if we consider ordered rooted trees, then the order of the children of a node is part of the tree, and the above re-drawings are different ordered rooted trees.

CS 270 CS 270 The notion of a “tree”: binary trees Algorithms Nodes and pointers Algorithms Oliver Oliver Kullmann Kullmann To begin with, a binary tree is a rooted ordered tree, where every A node of a binary tree has the following information naturally Binary Binary node which is not a leaf has either one or two children. But search associated with it: search

that’s not quite complete: In case a node has one child, then for Lists Lists its parent node this child it must be said whether it is a left child or a right child. Pointers Pointers

For example, the ordered rooted tree (T , 1) as given on the Trees its left and its right child Trees 4 previous slide has 2 = 16 different versions as binary tree, e.g. Implementing its label. Implementing rooted trees rooted trees

1 1 Tutorial Tutorial ❂ There are four special cases: ✁✁ ❂❂ ✁✁ ❂ 2 2 ❂ 1 the root has no parent node ❂❂ ✁✁ ❂ ✁✁ 2 a node may have only one child, left or right 3 3 ❂ ❂ ✁✁ ❂❂ ✁✁ ❂❂ 3 a leaf has no children. ✁✁ ❂ ✁✁ ❂ 5 4 5 4 ❂ ❂ The links to parents and children can be represented by pointers ✁✁ ❂❂ ✁✁ ❂❂ ✁✁ ❂ ✁✁ ❂ (possibly the null pointer), while the label is just an “attribute” 6 7 6 7 (a “data member” or “instance variable” of the class). CS 270 CS 270 Representing the example Algorithms Remark on pointers Algorithms Oliver Oliver Using as label just the key, we arrive at a class with four data Kullmann Since in a Java-environment students typically don’t get a good Kullmann members: Binary understanding on pointers, here some remarks we’ve already Binary search search seen: “p” is the pointer to the parent Lists Lists Pointers Pointers “left” and “right” are pointers to left resp. right child Note that dereferenciation of pointers v is denoted by “*v” Trees (the object to which it points). Trees “key” is the key. Implementing Implementing rooted trees Since Java is “purely pointer-based”, one can not get at the rooted trees Considering the first binary tree from two slides before, we can Tutorial address of an object, or to the object of a pointer: Every Tutorial represent the tree by 7 nodes v1,..., v7, where each node is a variable is a pointer, and using it in most cases(!) means to pointer and where the data members are given in order p, left, dereference it (an exception is for example when using ==). right, key: Another exception in Java is the handling of primitive types, where we can not have pointers. v =(NIL, v , NIL, 1), v =(v , NIL, v , 2), v =(v , v , v , 3), ∗ 1 2 ∗ 2 1 3 ∗ 3 2 5 4 C and C++ have both values (objects) and pointers, and v =(v , NIL, NIL, 4), v =(v , v , v , 5), ∗ 4 3 ∗ 5 3 6 7 we have full symmetry (and full control): v6 =(v5, NIL, NIL, 6), v7 =(v5, NIL, NIL, 7). For an object x the address is “&x”. ∗ ∗ For a pointer p the object is “*p”.

CS 270 CS 270 Rooted trees with unbounded branching Algorithms How to implement vectors (dynamic arrays) Algorithms Oliver Oliver Kullmann Kullmann

Binary Binary A few remarks in case there are more than two children: search search Lists Lists For k-ary trees (generalised binary trees) we just need to Pointers We already said that we could implement vectors Pointers add further data members to the class, with appropriate Trees Trees Implementing by holding arrays Implementing names. rooted trees rooted trees which are replaced by bigger arrays when needed (copying This works best for small k. For large k or unbounded k Tutorial Tutorial the old values over). (not known in advance) an easy way is to store the children in a list. We also said we could do this in a such a way that insertion Section 10.4 of CLRS contains a variation, where each becomes amortised constant time — how is this possible?? element of the (singly-linked) list of children also contains a pointer to its parent. CS 270 CS 270 On deletion in linked lists Algorithms A more powerful insertion Algorithms Oliver Oliver Kullmann Kullmann

Binary Binary search search

Lists Lists

Pointers Pointers

Trees Trees

Implementing Implementing Our implementation of delete returns the new value of the list L rooted trees Our insert-function inserts at the beginning — how to insert rooted trees — when is this different from the original value? Tutorial anywhere? Tutorial

CS 270 CS 270 Binary search in linked lists Algorithms Singly linked lists Algorithms Oliver Oliver Kullmann Kullmann

Binary Binary search search

Lists Lists

Pointers Pointers

Trees Trees

Implementing Implementing Can we perform binary search for a sorted linked list? If yes, rooted trees A singly linked list has no prev-pointer (only the next-pointer): rooted trees with what complexity? Tutorial What are the advantages and disadvantages? Tutorial