Analyzing Algorithms
Goal: “Runs fast on typical real problem instances”
Randomized Algorithms How do we evaluate this?
Example: Binary search Given a sorted array, determine if the array contains the number 157?
2 3
Complexity Measuring efficiency T! analysis n! Time ≈ # of instructions executed in a simple Problem size n programming language Best-case complexity: min # steps algorithm only simple operations (+,*,-,=,if,call,…) takes on any input of size n each operation takes one time step Average-case complexity: avg # steps algorithm each memory access takes one time step takes on inputs of size n
Worst-case complexity: max # steps
algorithm takes on any input of size n
4 5
1 Complexity
The complexity of an algorithm associates a number Complexity T(n), the worst-case time the algorithm takes on problems of size n, with each problem size n. T(n)!
Mathematically,
T: N+ → R+ ! I.e., T is a function that maps positive integers (problem Time sizes) to positive real numbers (number of steps).
Problem size ! 6 7
Simple Example Complexity
Array of bits. The complexity of an algorithm associates a number I promise you that either they are all 1’s or ½ 0’s T(n), the worst-case time the algorithm takes on and ½ 1’s. problems of size n, with each problem size n.
Give me a program that will tell me which it is. For randomized algorithms, look at worst-case value of E(T), where the Best case? expectation is taken over randomness in Worst case? algorithm.
Neat idea: use randomization to reduce the worst case 8 9
2 Quicksort Analysis of Quicksort
Worst case number of comparisons: n Sorting algorithm (assume for now all elements 2 distinct) ✓ ◆ How can we use randomization to improve running Given array of some length n time? If n = 0 or 1, halt Pick random element as a pivot each step Else pick element p of array as “pivot” Split array into subarrays
10 11
Analysis of Randomized Quicksort Analysis of Randomized Quicksort Quicksort with random pivots Fix pair i,j. Compute E(Xij)
Define A k indicator r.v. that is 1 if elt in [ei, ej] X = # of comparisons. first selected at kth level of tree
X = Xij E(Xij)=Pr(Xij = 1) 1 i 3 Analysis of Randomized Quicksort E(X)= E(Xij) 1 i 4