sign in sign up
<<
Home , Time complexity

CompSci 230 Announcements Discrete Math • Read for next time Chap. 4.1-4.3 for • Finish Chapter 3 first, then start Chapter 4, number theory October 10, 2013

Prof. Rodger

Slides modified from Rosen

1 2

Chap 3.3 - The Complexity of The Complexity of Algorithms

• Given an , how efficient is this algorithm for • In this course, focus on time complexity. solving a problem given input of a particular size? • Measure time complexity in terms of the – How much time does this algorithm use to solve a problem? number of operations an algorithm uses – How much computer memory does this algorithm use to solve • Use big-O and big-Theta notation to estimate a problem? the time complexity • time complexity - analyze the time the algorithm uses to • Is it practical to use this algorithm to solve solve the problem given input of a particular size problems with input of a particular size? • - analyze the computer memory the • Compare the efficiency of different algorithms algorithm uses to solve the problem, given input of a for solving the same problem. particular size

3 4 Time Complexity Complexity Example: Describe the time complexity of the algorithm • For time complexity, determine the number of for finding the maximum element in a finite sequence. operations, such as comparisons and arithmetic procedure max(a1, a2, …., an: integers) operations (addition, multiplication, etc.). max := a1 • Ignore minor details, such as the “house for i := 2 to n if max < ai then max := ai keeping” aspects of the algorithm. return max{max is the largest element} • Focus on the worst-case time complexity of an algorithm. Provides an upper bound. Solution: Count the number of comparisons. • Compare max < ai n 1 times. • More difficult to determine the average case • when i incremented, compare if i n. n 1 times • One last comparison for i> n. time complexity of an algorithm (average • 2(n 1 1 2n 1 comparisons are made. number of operations over all inputs of a particular size) Hence, the time complexity of the algorithm is Θ(n). 5 6

Complexity Analysis of Algorithms Worst-Case Complexity of procedure linear search(x:integer, Example: Describe the time complexity of the algorithm a , a , …,a : distinct integers) for finding the maximum element in a finite sequence. 1 2 n i := 1 procedure max(a1, a2, …., an: integers) while (i n and x ≠ a ) max := a i 1 i := i + 1 for i := 2 to n if i n then location := i if max < ai then max := ai return max{max is the largest element} else location := 0 return location{location is the subscript of the term that equals x, or is 0 if x is not found} Solution: Count the number of comparisons. Solution: Count the number of comparisons. • Compare max < a n 1 times. i • At each step two comparisons are made; i n and x ≠ a . • when i incremented, compare if i n. n 1 times i • end of loop, one comparison i n is made. • One last comparison for i> n. • After loop, one more i n comparison is made. • 2(n 1 1 2n 1 comparisons are made. If x = ai , 2i + 1 comparisons are used. If x is not on the list, 2n + 1 comparisons are made. One comparison to exit loop. Hence, the time complexity of the algorithm is Θ(n). Worst case 2n + 2 comparisons, complexity is Θ(n). 7 8 Worst-Case Complexity of Linear Search Average-Case Complexity of Linear Search procedure linear search(x:integer,

a1, a2, …,an: distinct integers) i := 1 Example: average case performance of linear search while (i n and x ≠ ai) i := i + 1 Solution: Assume the element is in the list and that if i n then location := i the possible positions are equally likely. else location := 0 By the argument on the previous slide, if x = a , the return location{location is the subscript of the term that i equals x, or is 0 if x is not found} number of comparisons is 2i + 1.

Solution: Count the number of comparisons.

• At each step two comparisons are made; i n and x ≠ ai . • end of loop, one comparison i n is made. • After loop, one more i n comparison is made. If x = a , 2i + 1 comparisons are used. If x is not on the list, 2n + 1 i Hence, the average‐case complexity of linear search is Θn. comparisons are made. One comparison to exit loop. Worst case 2n + 2 comparisons, complexity is Θ(n). 9 10

Average-Case Complexity of Linear Search Worst-Case Complexity of Binary Search procedure binary search(x: integer, a1,a2,…, an: increasing integers) i := 1 {i is the left endpoint of interval} j := n {j is right endpoint of interval} Example: average case performance of linear search while i < j Solution: Assume the element is in the list and that m := (i + j)/2 if x > am then i := m + 1 the possible positions are equally likely. else j := m if x = ai then location := i By the argument on the previous slide, if x = ai , the else location := 0 number of comparisons is 2i + 1. return location{location is the subscript i of the term ai equal to x, or 0 if x is not found}

Solution: Assume n = 2k elements. Note that k = log n.

• Two comparisons are made at each stage; i < j, and x > am . • Size of list is 2k , then 2k-1. then 2k-2 , … then 21 = 2. • At the last step, list size is 20 = 1 and single last element compared. Hence, the average‐case complexity of linear search is Θn. • Hence, at most 2k + 2 = 2 log n + 2 comparisons are made. • Therefore, the time complexity is Θ (log n), better than linear search. 11 12 Worst-Case Complexity of Binary Search Worst-Case Complexity of Bubble procedure binary search(x: integer, a1,a2,…, an: increasing integers) i := 1 {i is the left endpoint of interval} procedure bubblesort(a1,…,an: real numbers j := n {j is right endpoint of interval} with n ≥ ) while i < j m := (i + j)/2 for i := to n if x > am then i := m + 1 for j := to n i else j := m

if x = ai then location := i if aj >aj+1 then interchange aj and aj+1 else location := 0 {a1,…, an is now in increasing order} return location{location is the subscript i of the term ai equal to x, or 0 if x is not found}

Solution: Assume n = 2k elements. Note that k = log n. Solution: n passes through list. pass n i comparisons

• Two comparisons are made at each stage; i < j, and x > am . • Size of list is 2k , then 2k-1. then 2k-2 , … then 21 = 2. • At the last step, list size is 20 = 1 and single last element compared. • Hence, at most 2k + 2 = 2 log n + 2 comparisons are made. Θ(n2) since • Therefore, the time complexity is Θ (log n), better than linear search. . 13 14

Worst-Case Complexity of Worst-Case Complexity of procedure insertion sort(a ,…,a : procedure bubblesort(a1,…,an: real numbers 1 n real numbers with n ≥ 2) with n ≥ ) for j := 2 to n for i := to n i := 1 while aj > ai for j := to n i i := i + 1 m := a if a >a then interchange a and a j j j+1 j j+1 for k := 0 to j i 1 {a1,…, an is now in increasing order} aj-k := aj-k-1 ai := m Solution: n passes through list. pass n i comparisons Solution: The total number of comparisons are: 1 2 23 … 1 Θ(n ) since 2 . 2 15 Therefore the complexity is Θ(n ). 16 Worst-Case Complexity of Insertion Sort procedure insertion sort(a1,…,an: real numbers with n ≥ 2) for j := 2 to n • n elements are in an array i := 1 • If the value at the end is smaller than the first while aj > ai i := i + 1 element, swap them m := a j • If there are three of more elements then: for k := 0 to j i 1

aj-k := aj-k-1 – Stooge sort the first 2/3 of the array a := m i – Stooge sort the last 2/3 of the array – Stooge sort the first 2/3 of the array again Solution: The total number of comparisons are: •Else 1 23 … 1 – Done 2 2 Therefore the complexity is Θ(n ). 17 • Worst case time is O( ) 18

Stooge Sort Worst case of • n elements are in an array • If the value at the end is smaller than the first • Also known as “StupidSort” element, swap them • If there are three of more elements then: – Stooge sort the first 2/3 of the array • n elements in an array. – Stooge sort the last 2/3 of the array • While (not in order) – Stooge sort the first 2/3 of the array again – Shuffle array •Else – Done • Worst case: • Worst case time is O( .) 19 • Average Case: 20 Matrix Multiplication Algorithm Worst case of BogoSort • matrix multiplication algorithm; C = A B where C is an mnmatrix that is the product of the mkmatrix A and • Also known as “StupidSort” the knmatrix B.

procedure matrix multiplication(A,B: matrices) • n elements in an array. for i := to m • While (not in order) for j := to n

– Shuffle array cij := for q := to k c := c + a b • Worst case: unbounded ij ij iq qj return C{C = [cij] is the product of A and B} • Average Case: O( 21 22

Complexity of Matrix Multiplication Complexity of Matrix Multiplication

Example: How many additions of integers and Example: How many additions of integers and multiplications of integers are used by the multiplications of integers are used by the matrix multiplication algorithm to multiply matrix multiplication algorithm to multiply two n n matrices. two n n matrices. Solution: There are n2 Solution: There are n2 n n n n n3 n2 n n3 n2 n O(n3). O(n3).

23 24 Matrix-Chain Multiplication Matrix-Chain Multiplication

• Compute matrix-chain A1A2∙ ∙ ∙An with fewest • Compute matrix-chain A1A2∙ ∙ ∙An with fewest multiplications, where A1 , A2 , ∙ ∙ ∙ , An are , multiplications, where A1 , A2 , ∙ ∙ ∙ , An are , , ∙ ∙ ∙ integer matrices. Matrix , ∙ ∙ ∙ integer matrices. Matrix multiplication is associative. multiplication is associative. Example: In which order should the integer matrices Example: In which order should the integer matrices A1A2A3 - where A1 is 3020 , A2 2040, A3 4010 -be A1A2A3 - where A1 is 3020 , A2 2040, A3 4010 -be multiplied? Solution: two possible ways for A1A2A3. multiplied? Solution: two possible ways for A1A2A3. – A1(A2A3): A2A3 takes 20 ∙ 40 ∙ 10 8000 mults.. A1 – A1(A2A3): A2A3 takes 20 ∙ 40 ∙ 10 8000 mults.. A1 by the 2010 matrix A2A3 takes 30 ∙ 20 ∙ 10 6000 by the 2010 matrix A2A3 takes 30 ∙ 20 ∙ 10 6000 mults. Total number is 8000 6000 14,000. mults. Total number is 8000 6000 14,000.

–(A1A2)A3: A1A2 takes 30 ∙ 20 ∙ 40 24,000 mults. –(A1A2)A3: A1A2 takes 30 ∙ 20 ∙ 40 24,000 mults. A1A2 by A3 takes 30 ∙ 40 ∙ 10 12,000 mults. Total A1A2 by A3 takes 30 ∙ 40 ∙ 10 12,000 mults. Total is 24,000 12,000 36,000. is 24,000 12,000 36,000. So the first method is best. So the first method is best.

25 26

Understanding the Complexity of Understanding the Complexity of Algorithms Algorithms

Times of more than 10100 years are indicated

27 with an *. 28 P Versus NP Problem

Complexity of Problems (Born 1939) • The P versus NP problem asks whether the class P = NP? Are there problems • Tractable Problem: There exists a time algorithm to whose solutions can be checked in polynomial time, but can not be solved in solve this problem. These problems are said to belong to the polynomial time? Class P. – Note that just because no one has found a polynomial time algorithm is • Intractable Problem: There does not exist a polynomial time different from showing that the problem can not be solved by a algorithm to solve this problem polynomial time algorithm. • Unsolvable Problem : No algorithm exists to solve this problem, • If a polynomial time algorithm for any of the problems in the NP complete .g., halting problem. class were found, then that algorithm could be used to obtain a polynomial • Class NP: Solution can be checked in polynomial time. But no time algorithm for every problem in the NP complete class. polynomial time algorithm has been found for finding a solution – Satisfiability (in Section 1.3) is an NP complete problem. to problems in this class. • It is generally believed that PNP since no one has been able to ind a • NP Complete Class: If you find a polynomial time algorithm for polynomial time algorithm for any of the problems in the NP complete one member of the class, it can be used to solve all the problems class. in the class. • The problem of P versus NP remains one of the most famous unsolved problems in mathematics including theoretical computer science. The

29 Clay Mathematics Institute has offered a prize of $1,000,000 for a 30 solution.