CS483-07 Divide and Conquer Announcements
➣ October 9: no class. Oct. 8th is Columbus Day recess.
➣ Review class: October 11.
➣ Midterm is scheduled on October 16, 2007
Instructor: Fei Li ➣ Today’s lecture: Divide and Conquer (cont’)
Room 443 ST II 1. Quicksort
Office hours: Tue. & Thur. 1:30pm - 2:30pm or by appointments 2. Binary search 3. Binary tree traversal [email protected] with subject: CS483 4. Strassen’s matrix multiplication http://www.cs.gmu.edu/∼ lifei/teaching/cs483_fall07/
This lecture note is based on notes by Anany Levitin and Jyh-Ming Lian.
CS483 Design and Analysis of Algorithms 1 Lecture 07, September 18, 2007 CS483 Design and Analysis of Algorithms 2 Lecture 07, September 18, 2007
General Divide-and-Conquer Recurrence
➣ Problem size: n. Divide the problems into b smaller instances; a of them need to be solved. f(n) is the time spent on dividing and merging. Sorting Problem
T (n)=aT (n/b)+f(n). ➣ Given an array of n numbers, sort the elements in non-decreasing order.
➣ Master Theorem:Iff(n) ∈ Θ(nd), where d ≥ 0, then ➣ Input: An array A[1,...,n] of numbers ⎧ ⎪ d d ➣ Output: An array A[1,...,n] of sorted numbers ⎨⎪ Θ(n )ifabd
CS483 Design and Analysis of Algorithms 3 Lecture 07, September 18, 2007 CS483 Design and Analysis of Algorithms 4 Lecture 07, September 18, 2007 Mergesort - Algorithm Mergesort - Algorithm ➣ n Given an array of numbers, sort the elements in non-decreasing order. ➣ Merge two sorted arrays, B and C and put the result in A
Algorithm 0.1: MERGESORT(A[1,...n]) Algorithm 0.2: MERGE(B[1,...p],C[1,...q],A[1, ···p + q]) if n =1 i ← j ← then return (A) 1; 1 8 > ← [1 ···n ] k ∈{ , ,...p q − } >B A 2 for ⎧ 1 2 + 1 > n >C ← A[ ···n] ⎪ < 2 ⎨⎪if B[i]
CS483 Design and Analysis of Algorithms 5 Lecture 07, September 18, 2007 CS483 Design and Analysis of Algorithms 6 Lecture 07, September 18, 2007
Quicksort - Algorithm Mergesort - Analysis ➣ Given an array of n numbers, sort the element in non-decreasing order. ➣ n n All cases have same time efficiency: Θ( log2 ) Algorithm 0.3: QUICKSORT(A[1 ···n]) Tmerge(n)=n − 1. if n =1 ( ) T (n)=2T (n/2) + n − 1, ∀n>1,T(1) = 0 then return8 A > >Create two arrays B,C > ➣ Number of comparisons in the worst case is close to theoretical minimum for > ∈{2 3 } >for i 8 , ,...n > comparison-based sorting: n ≈n n − . n > > [ ] [1] log2 ! log2 1 44 >
CS483 Design and Analysis of Algorithms 7 Lecture 07, September 18, 2007 CS483 Design and Analysis of Algorithms 8 Lecture 07, September 18, 2007 Quicksort - Algorithm
➣ Quicksort allows fast “in-place partition”. Consider large files (n ≥ 10000).
Algorithm 0.4: PARTITION(A[a ···b])
p ← A[a] i ← a +1;j ← b
➣ Example: 24, 11, 91, 10, 22, 32, 22, 3, 7, 99 repeat8 > [ ] >while A i
Is this algorithm Quicksort stable? > ← +1 > do i i <> while A[j] >p > > do j ← j − 1 > > >if i
CS483 Design and Analysis of Algorithms 9 Lecture 07, September 18, 2007 CS483 Design and Analysis of Algorithms 10 Lecture 07, September 18, 2007
Quicksort - Analysis
➣ Best case: split in the middle Ð Θ(n log n).
T (n)=2T (n/2) + Θ(n).
➣ Worst case: sorted array! Ð Θ(n2).
Example: 5, 7, 3, 2, 8, 3, 6. T (n)=T (n − 1) + Θ(n).
➣ Average case: random arrays Ð Θ(n log n)
➣ Improvements (these combine to 20 − 25% improvement): 1. Better pivot selection: median of three partitioning
2. Switch to insertion sort on small subfiles.
3. Elimination of recursion.
CS483 Design and Analysis of Algorithms 11 Lecture 07, September 18, 2007 CS483 Design and Analysis of Algorithms 12 Lecture 07, September 18, 2007 Binary Search
➣ Imagine that you are placed in an unknown building and you are given a room number (say STII, 443), you need to find your CS 483 instructor. What will you do?
➣ Binary Search:
Very efficient algorithm for searching in sorted array Binary Search - Algorithm Example: find 70 in {3, 14, 27, 31, 39, 42, 55, 70, 74, 81, 85, 93, 98} ➣ Given a sorted array A of n numbers, find a key K in A
CS483 Design and Analysis of Algorithms 13 Lecture 07, September 18, 2007 CS483 Design and Analysis of Algorithms 14 Lecture 07, September 18, 2007
Algorithm 0.5: BINARYSEARCH(A[1 ···n],K) Binary Search - Analysis a ← 1; b ← n ➣ T n while8a ←a+b >m 2 > >if K = A[m] > T n T n/ n , n> ,T . <> worst( )= worst( 2 )+1 = Θ(log2 ) for 1 worst(1) = 1 return (m) do > [ ] ➣ T n > else if K
CS483 Design and Analysis of Algorithms 15 Lecture 07, September 18, 2007 CS483 Design and Analysis of Algorithms 16 Lecture 07, September 18, 2007 Binary Tree
➣ In a binary tree, each node has zero or two nodes.
Algorithm 0.6: HEIGHT(T )
if T = ∅ return (−1)
else return (max{Height(TL), Height(TR)} +1)
➣ Compute the height of a given binary tree T
CS483 Design and Analysis of Algorithms 17 Lecture 07, September 18, 2007 CS483 Design and Analysis of Algorithms 18 Lecture 07, September 18, 2007
Integer Multiplication
➣ What is the time complexity of multiplying two integers using the algorithms we Binary Tree Traversals learned in elementary schools? 12345 × 67890 ➣ 3 classical traversals Example: how do you compute this: ? 2 → → n digit multiplication + n addition Preorder traversals: root left subtree right subtree → → ➣ Is there a better way of multiplying two integers, in terms of reducing the number of Inorder traversals: left subtree root right subtree → → multiplication? Postorder traversals: left subtree right subtree root Carl Friedrich Gauss (1777-1855) discovered that ➣ n n n n Example: page 142. AB =(a10 2 + b)(c10 2 + d)=K210 + K110 2 + K0, where K2 = ac, K0 = bd, K1 =(a + b)(c + d) − (K0 + K2). Example: how do you compute this: 12345 × 67890?
CS483 Design and Analysis of Algorithms 19 Lecture 07, September 18, 2007 CS483 Design and Analysis of Algorithms 20 Lecture 07, September 18, 2007 Integer Multiplication
➣ Divide-and-conquer integer multiplication
Algorithm 0.7: M(A[1 ···n],B[1 ···n]) Integer Multiplication if n =1 ➣ What is the time complexity? then return (A[1]B[1]) 8 > ← [1 ··· n ] ← [ n +1··· ] First we formulate the time complexity as: T n T n O n . >a A 2 ,b A 2 n ( )=3 ( 2 )+ ( ) > > ← [1 ··· n ] ← [ n +1··· ] >c B 2 ,d B 2 n Using Master Theorem, we have a =3, b =2and d =1.So, <> K2 ← M(a, c) T n nlog2 3 ≈ n1.6 else ( )=Θ( ) Θ( ) > ← ( ) >K0 M b, d > >K1 ← M(a + b, c + d) − (K0 + K2) > : n n return (K210 + K110 2 + K0)
CS483 Design and Analysis of Algorithms 21 Lecture 07, September 18, 2007 CS483 Design and Analysis of Algorithms 22 Lecture 07, September 18, 2007
Matrix Multiplication
Strassen’s Matrix Multiplication: 2 3 2 3 2 3 Matrix Multiplication C11 C12 A11 A12 B11 A12 4 5 = 4 5 4 5 C21 C22 A21 A22 B21 B22 ➣ 2 3 What is the time complexity? m1 + m4 − m5 + m7 m3 + m5 T n T n O n = 4 5 ( )=7 ( 2 )+ ( ) m2 + m4 m1 + m3 − m2 + m6 Using Master Theorem, we have a =7, b =4and d =1. log 7 2.8 =( + )( + ) So, T (n)=Θ(n 2 ) ≈ Θ(n ) m1 A11 A22 B11 B22 =( + ) m2 A21 A22 B11 ➣ Do you still remember what the time complexity of the brute-force algorithm = ( − ) m3 A11 B12 B22 is? = ( − ) 3 m4 A22 B21 B11 O(n ) =( + ) m5 A11 A12 B22 =( − )( + ) m6 A21 A11 B11 B12 =( − )( + ) m7 A12 A22 B21 B22
CS483 Design and Analysis of Algorithms 23 Lecture 07, September 18, 2007 CS483 Design and Analysis of Algorithms 24 Lecture 07, September 18, 2007 Summary of Sorting Algorithms
Algorithm Time Notes
selection-sort O(n2) in-place. slow (good for small inputs)
insertion-sort O(n2) in-place. slow (good for small inputs)
quick-sort expected O(n log n) in-place, randomized, fastest (good for large inputs)
merge-sort O(n log n) sequential data access. fast (good for huge inputs)
CS483 Design and Analysis of Algorithms 25 Lecture 07, September 18, 2007