CSE 373: Analysis of Algorithms Lectures 5 ‒ 8 ( Correctness of Algorithms ) Rezaul A. Chowdhury Department of Computer Science SUNY Stony Brook Fall 2014 Insertion Sort Input: An array of numbers. Output: Elements̻ʞ 1 of ∶ ͢ ʟ ͢ rearranged in non-decreasing order of value. ̻ʞ 1 ∶ ͢ ʟ INSERTION -SORT ( A ) 1. for to 2. ͞ Ɣ 2 ̻. ͙͛ͨ͢͜͠ 3. //͙ͭ͟ insert Ɣ ̻ ͞ into the sorted sequence 4. ̻ ͞ ̻ 1. ͞ Ǝ 1 5. while͝ Ɣ ͞ Ǝ 1 and 6. ͝ Ƙ 0 ̻ ͝ Ƙ ͙ͭ͟ 7. ̻ ͝ ƍ 1 Ɣ ̻ ͝ 8. ͝ Ɣ ͝ Ǝ 1 ̻ ͝ƍ1 Ɣ ͙ͭ͟ Loop Invariants We use loop invariants to prove correctness of iterative algorithms A loop invariant is associated with a given loop of an algorithm, and it is a formal statement about the relationship among variables of the algorithm such that ― [ Initialization ] It is true prior to the first iteration of the loop ― [ Maintenance ] If it is true before an iteration of the loop, it remains true before the next iteration ― [ Termination ] When the loop terminates, the invariant gives us a useful property that helps show that the algorithm is correct Loop Invariants for Insertion Sort INSERTION -SORT ( A ) 1. for to 2. ͞ Ɣ 2 ̻. ͙͛ͨ͢͜͠ 3. //͙ͭ͟ insert Ɣ ̻ ͞ into the sorted sequence 4. ̻ ͞ ̻ 1. ͞ Ǝ 1 5. while͝ Ɣ ͞ Ǝ 1 and 6. ͝ Ƙ 0 ̻ ͝ Ƙ ͙ͭ͟ 7. ̻ ͝ ƍ 1 Ɣ ̻ ͝ 8. ͝ Ɣ ͝ Ǝ 1 ̻ ͝ƍ1 Ɣ ͙ͭ͟ Loop Invariants for Insertion Sort INSERTION -SORT ( A ) 1. for to Invariant͞ Ɣ 2 ̻. 1: ͙͛ͨ͢͜͠ consists of the elements ̻originally1. ͞ Ǝ 1 in , but in sorted order 2. ̻ 1. ͞ Ǝ 1 3. //͙ͭ͟ insert Ɣ ̻ ͞ into the sorted sequence 4. ̻ ͞ ̻ 1. ͞ Ǝ 1 5. while͝ Ɣ ͞ Ǝ 1 and 6. ͝ Ƙ 0 ̻ ͝ Ƙ ͙ͭ͟ 7. ̻ ͝ ƍ 1 Ɣ ̻ ͝ 8. ͝ Ɣ ͝ Ǝ 1 ̻ ͝ƍ1 Ɣ ͙ͭ͟ Loop Invariants for Insertion Sort INSERTION -SORT ( A ) 1. for to Invariant͞ Ɣ 2 ̻. 1: ͙͛ͨ͢͜͠ consists of the elements ̻originally1. ͞ Ǝ 1 in , but in sorted order 2. ̻ 1. ͞ Ǝ 1 3. //͙ͭ͟ insert Ɣ ̻ ͞ into the sorted sequence 4. ̻ ͞ ̻ 1. ͞ Ǝ 1 5. while͝ Ɣ ͞ Ǝ 1 and Invariant͝ Ƙ 0 2: ̻ ͝ Ƙare ͙ͭ͟ each 6. ̻ ͝. ͞ ƚ ͙ͭ͟ 7. ̻ ͝ ƍ 1 Ɣ ̻ ͝ 8. ͝ Ɣ ͝ Ǝ 1 ̻ ͝ƍ1 Ɣ ͙ͭ͟ Loop Invariant 1: Initialization At the start of the first iteration of the loop ( in lines ): Hence, subarray consists of a single element1 Ǝ 8 ͞, Ɣ which 2 is in fact the original̻ element1. ͞ Ǝ 1 in . ̻ 1 The subarray consisting of a single̻ 1 element is trivially sorted. Hence, the invariant holds initially. Loop Invariant 1: Maintenance We assume that invariant 1 holds before the start of the current iteration. Hence, the following holds: consists of the elements originally ̻in 1. ͞ Ǝ 1 , but in sorted order. For invariant 1 to hold before the̻ 1. start . ͞ Ǝ of 1 the next iteration, the following must hold at the end of the current iteration: consists of the elements originally in , but in sorted order. We̻ 1. use . ͞ invariant 2 to prove this. ̻ 1. ͞ Loop Invariant 1: Maintenance Loop Invariant 2: Initialization At the start of the first iteration of the loop ( in lines ): Hence, subarray consists of only two entries: 5 Ǝ 7and͝ Ɣ ͞. Ǝ 1 We know the following:̻ ͝. ͞ ̻ ͝ ̻ ͞ ― ( explicitly tested in line ) ― ̻ ͝ Ƙ ͙ͭ͟ ( from line ) 5 Hence, invariant̻ ͞ Ɣ ͙ͭ͟holds initially. 2 2 Loop Invariant 1: Maintenance Loop Invariant 2: Maintenance We assume that invariant 2 holds before the start of the current iteration. Hence, the following holds: are each . Since line copies which̻ is͝. .known ͞ to beƚ ͙ͭ͟ to which also held a value6 ̻, the͝ following holds at theƘ end ͙ͭ͟ of the̻ current͝ ƍ 1 iteration: areƚ ͙ͭ͟each . Before̻ ͝ ƍ 1. the . ͞ start of theƚ next ͙ͭ͟ iteration the check in line 5 ensures that invariant 2 continues to hold. ̻ ͝ Ƙ ͙ͭ͟ Loop Invariant 1: Maintenance Loop Invariant 2: Maintenance Observe that the inner loop ( in lines ) does not destroy any data because though the first iteration overwrites5 Ǝ 7 , that has already been saved in in line . ̻ ͞ ̻ ͞ As long as ͙ͭ͟is copied back2 into a location in without destroying any other element͙ͭ͟ in that subarray, we maintain̻ the1. ͞invariant that contains the first elements of the original list. ̻ 1. ͞ ͞ Loop Invariant 1: Maintenance Loop Invariant 2: Termination When the inner loop terminates we know the following. ― is sorted with each element ̻ 1. ͝ if , true by default ƙ ͙ͭ͟ if ͝ Ɣ 0, true because is sorted and ― ͝ Ƙis sorted 0 with each̻ element1. ͝ because̻ ͝ the ƙ ͙ͭ͟ following ̻held͝ ƍ before 1. ͞ was decremented: isƚ sorted ͙ͭ͟ with each item ― ͝ if the loop was̻ ͝. executed . ͞ at least once, and ƚ ͙ͭ͟ ̻ ͝ ƍ 1 Ɣ ̻ ͝ ƍotherwise2 ̻ ͝ ƍ 1 Ɣ ͙ͭ͟ Loop Invariant 1: Maintenance Loop Invariant 2: Termination When the inner loop terminates we know the following. ― is sorted with each element ― ̻ 1. ͝ is sorted with each elementƙ ͙ͭ͟ ― ̻ ͝ ƍ 1. ͞ or ƚ ͙ͭ͟ ̻ ͝ ƍ 1 Ɣ ̻ ͝ ƍ 2 ̻ ͝ ƍ 1 Ɣ ͙ͭ͟ Given the facts above, line 8 does not destroy any data, and gives us as the sorted permutation of the original data in . ̻ 1. ͞ ̻ 1. ͞ Loop Invariant 1: Termination When the outer loop terminates we know that . Hence, is the entire array ͞ Ɣ ̻.͙͛ͨ͢͜͠ƍ1, which is sorted and contains̻ the1. .original ͞ Ǝ 1 elements of ̻ 1. ̻. ͙͛ͨ͢͜͠. ̻ 1. ̻. ͙͛ͨ͢͜͠ Worst Case Runtime of Insertion Sort ( Upper Bound ) INSERTION -SORT ( A ) cost times 1. for to ͗ͥ ͢ 2. ͞ Ɣ 2 ̻. ͙͛ͨ͢͜͠ ͗ͦ 3. //͙ͭ͟ insert Ɣ ̻ ͞ into the sorted sequence 0 ͢ Ǝ 1 4. ̻ ͞ ̻ 1. ͞ Ǝ 1 ͗ͨ 5. while͝ Ɣ ͞ Ǝ 1 and ͝ Ƙ 0 ̻ ͝ Ƙ ͙ͭ͟ ͗ͩ ȕ ͞ 6. ͦ%) ̻ ͝ ƍ 1 Ɣ ̻ ͝ ͗ͪ 7. ȕ ͞ Ǝ 1 ͗ͫ ͦ%) 8. ͝ Ɣ ͝ Ǝ 1 ̻ ͝ƍ1 Ɣ ͙ͭ͟ ͗ͬ ͢ Ǝ 1 Running time, ͎ ͢ ƙ ͗ͥ͢ ƍ ͗ͦ ͢ Ǝ 1 ƍ ͗ͨ ͢ Ǝ 1 ) ) ) ƍ͗ͩ ∑%Ͱͦ ͞ ƍ ͗ͪ ∑%Ͱͦ ͞ Ǝ 1 ƍ ͗ͫ ∑%Ͱͦ ͞ Ǝ 1 ƍ ͗ͬ ͢ Ǝ 1 ͦ Ɣ 0.5 ͗ͩ ƍ ͗ͪ ƍ ͗ͫ ͢ ƍ 0.5 2͗ͥ ƍ 2͗ͦ ƍ 2͗ͨ ƍ ͗ͩ Ǝ ͗ͪ Ǝ ͗ͫ ƍ 2͗ͬ ͢ Ǝ ͗ͦ ƍ ͗ͨ ƍ ͗ͩ ƍ ͗ͬ ͦ ⇒ ͎ ͢ Ɣ ͉ ͢ Best Case Runtime of Insertion Sort ( Lower Bound ) INSERTION -SORT ( A ) cost times 1. for to ͗ͥ ͢ 2. ͞ Ɣ 2 ̻. ͙͛ͨ͢͜͠ ͗ͦ 3. //͙ͭ͟ insert Ɣ ̻ ͞ into the sorted sequence ̻ ͞ ̻ 1. ͞ Ǝ 1 0 4. ͢ Ǝ 1 ͗ͨ 5. while͝ Ɣ ͞ Ǝ 1 and ͗ͩ 6. ͝ Ƙ 0 ̻ ͝ Ƙ ͙ͭ͟ ̻ ͝ ƍ 1 Ɣ ̻ ͝ ͗ͪ 7. 0 ͗ͫ 8. ͝ Ɣ ͝ Ǝ 1 ̻ ͝ƍ1 Ɣ ͙ͭ͟ ͗ͬ ͢ Ǝ 1 Running time, ͎ ͢ ƚ ͗ͥ͢ ƍ ͗ͦ ͢ Ǝ 1 ƍ ͗ͨ ͢ Ǝ 1 ƍ͗ͩ ͢ Ǝ 1 ƍ ͗ͬ ͢ Ǝ 1 Ɣ ͗ͥ ƍ ͗ͦ ƍ ͗ͨ ƍ ͗ͩ ƍ ͗ͬ ͢ Ǝ ͗ͦ ƍ ͗ͨ ƍ ͗ͩ ƍ ͗ͬ ⇒ ͎ ͢ Ɣ Ω ͢ Selection Sort Input: An array of numbers. Output: Elements̻ʞ 1 of ∶ ͢ ʟ ͢ rearranged in non-decreasing order of value. ̻ʞ 1 ∶ ͢ ʟ SELECTION -SORT ( A ) 1. for to 2. //͞ Ɣfind 1 the̻. index ͙͛ͨ͢͜͠ of an entry with the smallest value in 3. ̻ ͞. ̻. ͙͛ͨ͢͜͠ 4. for͢͡͝ Ɣ ͞ to 5. if͝ Ɣ ͞ ƍ 1 ̻. ͙͛ͨ͢͜͠ 6. ̻ ͝ Ɨ ̻ ͢͡͝ 7. // swap͢͡͝ Ɣand ͝ 8. ̻ ͞ ̻ ͢͡͝ ̻ ͞ ↔ ̻ ͢͡͝ Selection Sort Input: An array of numbers. Output: Elements̻ʞ 1 of ∶ ͢ ʟ ͢ rearranged in non-decreasing order of value. ̻ʞ 1 ∶ ͢ ʟ SELECTION -SORT ( A ) 1. for to Invariant͞ Ɣ 1 ̻. 1: ͙͛ͨ͢͜͠ ? 2. // find the index of an entry with the smallest value in 3. ̻ ͞. ̻. ͙͛ͨ͢͜͠ 4. for͢͡͝ Ɣ ͞ to Invariant͝ Ɣ ͞ ƍ 1 2: ̻.? ͙͛ͨ͢͜͠ 5. if 6. ̻ ͝ Ɨ ̻ ͢͡͝ 7. // swap͢͡͝ Ɣand ͝ 8. ̻ ͞ ̻ ͢͡͝ ̻ ͞ ↔ ̻ ͢͡͝ Merging Two Sorted Subarrays Input: Two subarrays and in sorted order ( ). Output: A single sorted̻ʞ ͤ subarray ∶ ͥ ʟ ̻ ͥ ƍ 1:by ͦ merging the input subarrays.ͤ ƙ ͥ Ɨ ͦ ̻ʞ ͤ ∶ ͦ ʟ Merging Two Sorted Subarrays Input: Two subarrays and in sorted order ( ). Output: A single sorted̻ʞ ͤ subarray ∶ ͥ ʟ ̻ ͥ ƍ 1:by ͦ merging the input subarrays.ͤ ƙ ͥ Ɨ ͦ ̻ʞ ͤ ∶ ͦ ʟ Loop Invariant At the start of each iteration of the for loop of lines 12‒17 the following invariant holds: The subarray contains the smallest̻ ͤ: ͟elements Ǝ 1 of ͟ Ǝ ͤ and , ͆in 1:sorted ͥ͢ ƍ order.