Introduction to asymptotic complexity Search Organization You are responsible for: Weiss, chapter 5, as follows: •5.1 What is algorithmic analysis? • Searching in arrays •5.2 Examples of running time – Linear search •5.3 NO, not responsible for this No. – Binary search •5.4 Definition of Big-oh and Big-theta. • Asymptotic complexity of algorithms •5.5 Everything except harmonic numbers. This includes the repeated doubling and repeated having stuff. •5.6. No, not responsible, No. Instead, you should know the following algorithms as presented in the handout on correctness of algorithms and elsewhere and be able to determine their worst-case order of execution time: linear search, finding the min, binary search, partition, , , , quick sort. •5.7 Checking an analysis. •5.8 Limitations of big-oh analysis 0 1

/** = index of first occ. of v in b (b.length is v not in b) */ Binary search public static boolean linearSearch(Comparable[] a, Object v) { • Input: int i = 0; – A[0..n-1] of Comparable // invariant: v is not in b[0..i-1] – Value v of type Comparable while (i < a.length) { • Output: returns true if v is in the array; false otherwise if (a[i].compareTo(v) == 0) return true; • Algorithm: similar to looking up telephone directory i= i+1; – Let m be the middle element of the array } – If (m ==v) return true return false; – If (m < v) search right half of array } – If (m > v) search left half of array 2 1 Linear search: 7 4 6 19 3 7 8 10 32 54 67 98 Search for 6 -2 0 6 8 9 1113 22 34 45 56 78

Search for 94 2 1 2 3 4 3

/**b is sorted. Return a value k such that b[0..k] ! v < b[k+1..] */ Comparison of linear and binary search public static boolean binarySearch(Comparable[] b, Object v) { int k= -1; int j= b.length; • binary search runs much faster than linear search. // invariant: b[0..k] ! v < b[j..]; • Stating this precisely can be quite subtle. while (j != k+1) { • One approach: asymptotic complexity of programs int e= (k+j)/ 2; – big-O notation // { -1 <= k < e < j <= b.length } if (b[k].compareTo(v) <= 0) k= e; • Two steps: else j= e; – Compute running time of program } Each iteration performs one – Running time ! asymptotic running time return k; comparison and cuts b[k+1..j-1] is half } Asymptotic running time gives you a formula that 0 k e j tells you something about the running time binary search: <= v > v on LARGE arrays.

4 5 Running time of algorithms Defining running time of programs • In general, running time of a program such as 1. Machine on which programs are executed. linear search depends on many factors: – Random-access Memory (RAM) model of computing 1. machine on which program is executed • Measure of running time: number of operations • laptop vs. supercomputer executed 2. size of input (array A) – Other models used in CS: Turing machine, Parallel • big array vs. small array RAM model, … 3. values of input • v is first element in array vs. v is not in array – Simplified RAM model for now: • To talk precisely about running times of • Each data comparison is one operation. programs, we must specify all three factors • All other operations are free. above. • Evaluate searching/sorting algorithms by estimating number of comparisons they make It can be shown that for searching and sorting algorithms, total number of operations executed on RAM model is

6 proportional to number of data comparisons executed. 7

Defining running time (contd.) Define running time (contd.) 2. Dependence on size of input 3. Dependence of running time on input values

– Rather than compute a single number, we • Consider set In of possible compute a from problem size to inputs of size n. number of comparisons. ([3,6], 2) • Find number of comparisons ([-4,5], -9) • (e.g.) f(n) = 32n2 – 2n + 23 for each possible input in this set. • Compute ([3,6], 3) ……. where is problem size •Average: harder to compute – Each program has its own measure of •Worst-case: easier to compute problem size. •We will use worst-case complexity. – For searching/sorting, natural measure is size Possible inputs of size 2 of array being searched/sorted. for linear/binary search

8 9

Computing running times Base-2 Linear search: 7 4 6 19 3 7 8 10 32 54 67 98 k 2**k log(2**k) binary rep of 2**k 0 1 0 00000000000000001 Assume array is of size n. 1 2 1 00000000000000010 Worst-case number of comparisons: v is not in array. 2 4 2 00000000000000100 Number of comparisons = n. 3 8 3 00000000000001000 Running time of linear search: T (n) = n L 4 16 4 00000000000010000 5 32 5 00000000000100000 Binary search: sorted array of size n 6 64 6 00000000001000000 -2 0 6 8 9 1113 22 34 45 56 78 15 32768 15 01000000000000000 If n = 2**k, then k is called the (to the base 2) of n Worst-case number of comparisons: v is not in array. If n is a power of 2, then log(n) is number of 0’s following the 1. TB(n) = log2(n) + 1 10 11 Running time ! Rules for computing Asymptotic running time asymptotic running time

• Compute running time as a function of input size. • Drop lower order terms. Linear search: TL(n) = n • From the term that remains, drop floors/ceilings as well as

Binary search: TB(n) = log2(n) + 1 any constant multipliers. • Result: usually something like O(n), O(n2), O(nlog(n)), We are really interested in comparing running times only O(2n) for large problem sizes. For small problem sizes, running time is small enough that we may not care which algorithm we use.

For large values of n, we can drop the “+1” term and the floor operation, and keep only the leading term, and say that

TB(n) ! log2(n) as n gets larger.

Formally, TB(n) = O(log2(n)) and TL(n) = O(n) 12 13

Summary of informal introduction Finding the minimum

• Asymptotic running time of a program Function min(b, h, k) returns the index of the minimum value 1. Running time: compute worst-case number of of b[h..k]. operations required to execute program on RAM It performs a linear search, comparing each element of model as a function of input size. b[h+1..k] to the minimum of the previous segment. – for searching/sorting algorithms, we will compute only the number of comparisons h i k 2. Running time ! asymptotic running time: keep only b b[j] is min of these the leading term(s) in this function.

To find the minimum of a segment of size n takes n-1 comparisons: O(n).

14 15

Selection sort: O(n**2) running time

/** Sort b */ public static void selectionsort(Comparable b[]) { // inv: b[0..k-1] sorted, b[0..k-1] ! b[k..] k # comp. for (int k= 0; k != b.length; k= k+1) { 0 n-1 int t= min(b,k,b.length); 1 n-2 Swap b[k] and b[t]; 2 n-3 k= k+1; 3 n-4 } … … } n-1 0 0 k n (n-1)*n/2 b sorted, <= >= = n**2 - n / 2

16