Python Programming: Objectives
An Introduction to ¢ To understand the basic techniques for Computer Science analyzing the efficiency of algorithms. ¢ To know what searching is and understand the algorithms for linear and binary Chapter 13 search.
Algorithm Design and Recursion ¢ To understand the basic principles of recursive definitions and functions and be able to write simple recursive functions.
Python Programming, 1/e 1 Python Programming, 1/e 2
Objectives Searching
¢ To understand sorting in depth and ¢ Searching is the process of looking for a know the algorithms for selection sort particular value in a collection.
and merge sort. ¢ For example, a program that maintains ¢ To appreciate how the analysis of a membership list for a club might need algorithms can demonstrate that some to look up information for a particular problems are intractable and others are member œ this involves some sort of unsolvable. search process.
Python Programming, 1/e 3 Python Programming, 1/e 4
A simple Searching Problem A Simple Searching Problem
¢ Here is the specification of a simple ¢ In the first example, the function searching function: returns the index where 4 appears in def search(x, nums): the list. # nums is a list of numbers and x is a number # Returns the position in the list where x occurs ¢ # or -1 if x is not in the list. In the second example, the return value -1 indicates that 7 is not in the list. ¢ Here are some sample interactions: >>> search(4, [3, 1, 4, 2, 5]) 2 ¢ Python includes a number of built-in >>> search(7, [3, 1, 4, 2, 5]) -1 search-related methods!
Python Programming, 1/e 5 Python Programming, 1/e 6
Python Programming, 1/e 1 A Simple Searching Problem A Simple Searching Problem
¢ We can test to see if a value appears in ¢ The only difference between our a sequence using in. search function and index is that
if x in nums: index raises an exception if the target # do something value does not appear in the list. ¢ If we want to know the position of x in ¢ We could implement search using a list, the index method can be used. index by simply catching the exception >>> nums = [3, 1, 4, 2, 5] >>> nums.index(4) and returning -1 for that case. 2
Python Programming, 1/e 7 Python Programming, 1/e 8
A Simple Searching Problem Strategy 1: Linear Search
¢ def search(x, nums): ¢ Pretend you‘re the computer, and you were try: given a page full of randomly ordered return nums.index(x) numbers and were asked whether 13 was in except: return -1 the list. ¢ How would you do it? ¢ Sure, this will work, but we are really ¢ Would you start at the top of the list, interested in the algorithm used to scanning downward, comparing each number actually search the list in Python! to 13? If you saw it, you could tell me it was in the list. If you had scanned the whole list and not seen it, you could tell me it wasn‘t there.
Python Programming, 1/e 9 Python Programming, 1/e 10
Strategy 1: Linear Search Strategy 1: Linear Search
¢ This strategy is called a linear search, ¢ The Python in and index operations where you search through the list of both implement linear searching items one by one until the target value algorithms. is found. ¢ def search(x, nums): ¢ If the collection of data is very large, it for i in range(len(nums)): if nums[i] == x: # item found, return the index value return i makes sense to organize the data return -1 # loop finished, item was not in list somehow so that each data value ¢ This algorithm wasn‘t hard to develop, doesn‘t need to be examined. and works well for modest-sized lists.
Python Programming, 1/e 11 Python Programming, 1/e 12
Python Programming, 1/e 2 Strategy 1: Linear Search Strategy 2: Binary Search
¢ If the data is sorted in ascending order ¢ If the data is sorted, there is an even better (lowest to highest), we can skip checking searching strategy œ one you probably some of the data. already know!
¢ As soon as a value is encountered that is ¢ Have you ever played the number guessing greater than the target value, the linear game, where I pick a number between 1 and search can be stopped without looking at the 100 and you try to guess it? Each time you rest of the data. guess, I‘ll tell you whether your guess is
¢ On average, this will save us about half the correct, too high, or too low. What strategy work. do you use?
Python Programming, 1/e 13 Python Programming, 1/e 14
Strategy 2: Binary Search Strategy 2: Binary Search
¢ Young children might simply guess ¢ Each time we guess the middle of the numbers at random. remaining numbers to try to narrow ¢ Older children may be more systematic, down the range. using a linear search of 1, 2, 3, 4, … ¢ This strategy is called binary search. until the value is found. ¢ Binary means two, and at each step we ¢ Most adults will first guess 50. If told the value is higher, it is in the range 51- are diving the remaining group of 100. The next logical guess is 75. numbers into two parts.
Python Programming, 1/e 15 Python Programming, 1/e 16
Strategy 2: Binary Search Strategy 2: Binary Search
¢ We can use the same approach in our ¢ The heart of the algorithm is a loop that looks binary search algorithm! We can use two at the middle element of the range, variables to keep track of the endpoints of comparing it to the value x. the range in the sorted list where the ¢ If x is smaller than the middle item, high is number could be. moved so that the search is confined to the lower half. ¢ Since the target could be anywhere in the list, initially low is set to the first location ¢ If x is larger than the middle item, low is in the list, and high is set to the last. moved to narrow the search to the upper half.
Python Programming, 1/e 17 Python Programming, 1/e 18
Python Programming, 1/e 3 Strategy 2: Binary Search Strategy 2: Binary Search
def search(x, nums): ¢ The loop terminates when either low = 0 high = len(nums) - 1 ¢ x is found while low <= high: # There is still a range to search mid = (low + high)/2 # Position of middle item item = nums[mid] ¢ There are no more places to look if x == item: # Found it! Return the index ( > ) return mid low high elif x < item: # x is in lower half of range high = mid - 1 # move top marker down else: # x is in upper half of range low = mid + 1 # move bottom marker up return -1 # No range left to search, # x is not there
Python Programming, 1/e 19 Python Programming, 1/e 20
Comparing Algorithms Comparing Algorithms
¢ Which search algorithm is better, linear or ¢ One way to conduct the test would be to binary? code up the algorithms and try them on ¢ The linear search is easier to understand and varying sized lists, noting the runtime. implement ¢ Linear search is generally faster for lists of length ¢ The binary search is more efficient since it doesn‘t need to look at each element in the list 10 or less ¢ There was little difference for lists of 10-1000 ¢ Intuitively, we might expect the linear search to work better for small lists, and binary ¢ Binary search is best for 1000+ (for one million list search for longer lists. But how can we be elements, binary search averaged .0003 seconds sure? while linear search averaged 2.5 second)
Python Programming, 1/e 21 Python Programming, 1/e 22
Comparing Algorithms Comparing Algorithms
¢ While interesting, can we guarantee that ¢ How do we count the number of these empirical results are not dependent on —steps“? the type of computer they were conducted on, the amount of memory in the computer, ¢ Computer scientists attack these the speed of the computer, etc.? problems by analyzing the number of ¢ We could abstractly reason about the steps that an algorithm will take relative algorithms to determine how efficient they are. We can assume that the algorithm with to the size or difficulty of the specific the fewest number of —steps“ is more problem instance being solved. efficient.
Python Programming, 1/e 23 Python Programming, 1/e 24
Python Programming, 1/e 4 Comparing Algorithms Comparing Algorithms
¢ For searching, the difficulty is determined by ¢ Let‘s consider linear search. the size of the collection œ it takes more steps ¢ For a list of 10 items, the most work we might have to do is to look at each item in turn œ looping at most 10 to find a number in a collection of a million times.
numbers than it does in a collection of 10 ¢ For a list twice as large, we would loop at most 20 numbers. times. ¢ For a list three times as large, we would loop at most ¢ How many steps are needed to find a value in 30 times! a list of size n? ¢ The amount of time required is linearly ¢ In particular, what happens as n gets very related to the size of the list, n. This is what large? computer scientists call a linear time algorithm. Python Programming, 1/e 25 Python Programming, 1/e 26
Comparing Algorithms Comparing Algorithms
¢ Now, let‘s consider binary search. ¢ To determine how many items are ¢ Suppose the list has 16 items. Each time through examined in a list of size n, we need to the loop, half the items are removed. After one i n = 2 i = log n loop, 8 items remain. solve for i, or 2 .
¢ After two loops, 4 items remain. ¢ Binary search is an example of a log ¢ After three loops, 2 items remain time algorithm œ the amount of time it ¢ After four loops, 1 item remains. takes to solve one of these problems ¢ If a binary search loops i times, it can find a grows as the log of the problem size. single value in a list of size 2i.
Python Programming, 1/e 27 Python Programming, 1/e 28
Comparing Algorithms Comparing Algorithms
¢ This logarithmic property can be very ¢ Our analysis shows us the answer to powerful! this question is l o g 1 2 0 0 0 0 0 0 . ¢ Suppose you have the New York City phone 2 million book with 12 names. You could walk ¢ We can guess the name of the New up to a New Yorker and, assuming they are listed in the phone book, make them this Yorker in 24 guesses! By comparison, proposition: —I‘m going to try guessing your using the linear search we would need name. Each time I guess a name, you tell me if your name comes alphabetically before or to make, on average, 6,000,000 after the name I guess.“ How many guesses guesses! will you need?
Python Programming, 1/e 29 Python Programming, 1/e 30
Python Programming, 1/e 5 Comparing Algorithms Recursive Problem-Solving
¢ Earlier, we mentioned that Python uses ¢ The basic idea between the binary linear search in its built-in searching search algorithm was to successfully methods. We doesn‘t it use binary divide the problem in half. search? ¢ This technique is known as a divide and conquer approach. ¢ Binary search requires the data to be sorted ¢ Divide and conquer divides the original problem into subproblems that are ¢ If the data is unsorted, it must be sorted first! smaller versions of the original problem.
Python Programming, 1/e 31 Python Programming, 1/e 32
Recursive Problem-Solving Recursive Problem-Solving
Algorithm: binarySearch – search for x in nums[low]…nums[high] ¢ In the binary search, the initial range is mid = (low + high) /2 the entire list. We look at the middle if low > high element… if it is the target, we‘re done. x is not in nums elsif x < nums[mid] Otherwise, we continue by performing a perform binary search for x in nums[low]…nums[mid-1] else binary search on either the top half or perform binary search for x in nums[mid+1]…nums[high] bottom half of the list. ¢ This version has no loop, and seems to refer to itself! What‘s going on??
Python Programming, 1/e 33 Python Programming, 1/e 34
Recursive Definitions Recursive Definitions
¢ A description of something that refers ¢ Have you had a teacher tell you that to itself is called a recursive definition. you can‘t use a word in its own definition? This is a circular definition. ¢ In the last example, the binary search algorithm uses its own description œ a ¢ In mathematics, recursion is frequently used. The most common example is the —call“ to binary search —recurs“ inside of factorial: n!= n(n −1)(n − 2)...(1) the definition œ hence the label ¢ For example, 5! = 5(4)(3)(2)(1), or —recursive definition.“ 5! = 5(4!)
Python Programming, 1/e 35 Python Programming, 1/e 36
Python Programming, 1/e 6 Recursive Definitions Recursive Definitions
¢ In other words, n!= n(n −1)! ¢ Our definition is recursive, but definitely À 1 if n = 0 not circular. Consider 4! ¢ Or n!= Ã ¢ 4! = 4(4-1)! = 4(3!) Õn(n −1)! otherwise ¢ What is 3!? We apply the definition again ¢ This definition says that 0! is 1, while 4! = 4(3!) = 4[3(3-1)!] = 4(3)(2!)
the factorial of any other number is that ¢ And so on… number times the factorial of one less 4! = 4(3!) = 4(3)(2!) = 4(3)(2)(1!) = than that number. 4(3)(2)(1)(0!) = 4(3)(2)(1)(1) = 24
Python Programming, 1/e 37 Python Programming, 1/e 38
Recursive Definitions Recursive Definitions
¢ Factorial is not circular because we ¢ All good recursive definitions have these two key characteristics: eventually get to 0!, whose definition ¢ There are one or more base cases for which no does not rely on the definition of recursion is applied. ¢ All chains of recursion eventually end up at one of factorial and is just 1. This is called a the base cases. base case for the recursion. ¢ The simplest way for these two conditions to occur is for each recursion to act on a smaller ¢ When the base case is encountered, we version of the original problem. A very small get a closed expression that can be version of the original problem that can be solved without recursion becomes the base directly computed. case.
Python Programming, 1/e 39 Python Programming, 1/e 40
Recursive Functions Recursive Functions
¢ We‘ve seen previously that factorial can ¢ We‘ve written a function that calls itself, be calculated using a loop accumulator. a recursive function.
¢ If factorial is written as a separate ¢ The function first checks to see if we‘re function: at the base case (n==0). If so, return 1. def fact(n): Otherwise, return the result of if n == 0: return 1 multiplying n by the factorial of n-1, else: fact(n-1). return n * fact(n-1)
Python Programming, 1/e 41 Python Programming, 1/e 42
Python Programming, 1/e 7 Recursive Functions Recursive Functions
>>> fact(4) 24 >>> fact(10) 3628800 >>> fact(100) 93326215443944152681699238856266700490715968264381621468592963 89521759999322991560894146397615651828625369792082722375825 1185210916864000000000000000000000000L >>> ¢ Remember that each call to a function starts that function anew, with its own copies of local variables and parameters.
Python Programming, 1/e 43 Python Programming, 1/e 44
Example: String Reversal Example: String Reversal
¢ Python lists have a built-in method that ¢ Using recursion, we can calculate the can be used to reverse the list. What if reverse of a string without the you wanted to reverse a string? intermediate list step.
¢ If you wanted to program this yourself, ¢ Think of a string as a recursive object:
one way to do it would be to convert ¢ Divide it up into a first character and —all the string into a list of characters, the rest“
reverse the list, and then convert it ¢ Reverse the —rest“ and append the first back into a string. character to the end of it
Python Programming, 1/e 45 Python Programming, 1/e 46
Example: String Reversal Example: String Reversal
¢ >>> reverse("Hello") ¢ def reverse(s): Traceback (most recent call last): return reverse(s[1:]) + s[0] File "
Python Programming, 1/e 47 Python Programming, 1/e 48
Python Programming, 1/e 8 Example: String Reversal Example: String Reversal
¢ Remember: To build a correct recursive ¢ Each time a function is called it takes some function, we need a base case that memory. Python stops it at 1000 calls, the doesn‘t use recursion. default —maximum recursion depth.“ ¢ What should we use for our base case? ¢ We forgot to include a base case, so our program is an infinite recursion. ¢ Following our algorithm, we know we will eventually try to reverse the empty string. Each call to reverse contains another Since the empty string is its own reverse, we call to reverse, so none of them can use it as the base case. return.
Python Programming, 1/e 49 Python Programming, 1/e 50
Example: String Reversal Example: Anagrams
¢ def reverse(s): if s == "": ¢ An anagram is formed by rearranging return s else: the letters of a word. return reverse(s[1:]) + s[0] ¢ Anagram formation is a special case of ¢ >>> reverse("Hello") 'olleH' generating all permutations (rearrangements) of a sequence, a problem that is seen frequently in mathematics and computer science.
Python Programming, 1/e 51 Python Programming, 1/e 52
Example: Anagrams Example: Anagrams
¢ Let‘s apply the same approach from the ¢ Suppose the original string is —abc“. Stripping previous example. off the —a“ leaves us with —bc“. ¢ Generating all anagrams of —bc“ gives us —bc“ ¢ Slice the first character off the string. and —cb“. ¢ Place the first character in all possible locations within the anagrams formed from ¢ To form the anagram of the original string, the —rest“ of the original string. we place —a“ in all possible locations within these two smaller anagrams: [—abc“, —bac“, —bca“, —acb“, —cab“, —cba“]
Python Programming, 1/e 53 Python Programming, 1/e 54
Python Programming, 1/e 9 Example: Anagrams Example: Anagrams
¢ As in the previous example, we can use ¢ A list is used to accumulate results. ¢ The outer for loop iterates through each the empty string as our base case. anagram of the tail of s. ¢ def anagrams(s): if s == "": ¢ The inner loop goes through each position in return [s] the anagram and creates a new string with else: the original first character inserted into that ans = [] for w in anagrams(s[1:]): position. for pos in range(len(w)+1): ¢ The inner loop goes up to len(w)+1 so the ans.append(w[:pos]+s[0]+w[pos:]) new character can be added at the end of the return ans anagram.
Python Programming, 1/e 55 Python Programming, 1/e 56
Example: Anagrams Example: Anagrams
¢ w[:pos]+s[0]+w[pos:] ¢ The number of anagrams of a word is
¢ w[:pos] gives the part of w up to, but not the factorial of the length of the word. including, pos. ¢ >>> anagrams("abc") ['abc', 'bac', 'bca', 'acb', 'cab', 'cba'] ¢ w[pos:] gives everything from pos to the end.
¢ Inserting s[0] between them effectively inserts it into w at pos.
Python Programming, 1/e 57 Python Programming, 1/e 58
Example: Fast Exponentiation Example: Fast Exponentiation
n ¢ One way to compute a for an integer n ¢ We can also solve this problem using divide is to multiply a by itself n times. and conquer. 8 ¢ Using the laws of exponents, we know that 2 ¢ This can be done with a simple = 24(24). If we know 24, we can calculate 28 accumulator loop: using one multiplication. 4 4 2 2 2 def loopPower(a, n): ¢ What‘s 2 ? 2 = 2 (2 ), and 2 = 2(2). ans = 1 8 ¢ 2(2) = 4, 4(4) = 16, 16(16) = 256 = 2 for i in range(n): 8 ans = ans * a ¢ We‘ve calculated 2 using only three return ans multiplications!
Python Programming, 1/e 59 Python Programming, 1/e 60
Python Programming, 1/e 10 Example: Fast Exponentiation Example: Fast Exponentiation
¢ We can take advantage of the fact that ¢ This method relies on integer division (if an = an/2(an/2) n is 9, then n/2 = 4). ¢ To express this algorithm recursively, ¢ This algorithm only works when n is even. How can we extend it to work we need a suitable base case. when n is odd? ¢ If we keep using smaller and smaller 9 4 4 1 values for n, n will eventually be equal ¢ 2 = 2 (2 )(2 ) to 0 (1/2 = 0 in integer division), and Àan/ 2 (an/ 2 ) if n is even a0 = 1 for any value except a = 0. an = Ã Õ an/ 2 (an/ 2 )(a) if n is odd Python Programming, 1/e 61 Python Programming, 1/e 62
Example: Fast Exponentiation Example: Binary Search
¢ def recPower(a, n): # raises a to the int power n ¢ Now that you‘ve seen some recursion if n == 0: return 1 examples, you‘re ready to look at doing else: factor = recPower(a, n/2) binary searches recursively. if n%2 == 0: # n is even return factor * factor ¢ Remember: we look at the middle value first, else: # n is odd return factor * factor * a then we either search the lower half or upper ¢ Here, a temporary variable called factor half of the array. is introduced so that we don‘t need to ¢ The base cases are when we can stop n/2 calculate a more than once, simply searching,namely, when the target is found for efficiency. or when we‘ve run out of places to look.
Python Programming, 1/e 63 Python Programming, 1/e 64
Example: Binary Search Example: Binary Search
¢ def recBinSearch(x, nums, low, high): if low > high: # No place left to look, return -1 ¢ The recursive calls will cut the search in return -1 mid = (low + high)/2 half each time by specifying the range item = nums[mid] if item == x: return mid of locations that are —still in play“, i.e. elif x < item: # Look in lower half return recBinSearch(x, nums, low, mid-1) else: # Look in upper half have not been searched and may return recBinSearch(x, nums, mid+1, high) contain the target value. ¢ We can then call the binary search with ¢ Each invocation of the search routine a generic search wrapping function:
will search the list between the given def search(x, nums): lowand high parameters. return recBinSearch(x, nums, 0, len(nums)-1)
Python Programming, 1/e 65 Python Programming, 1/e 66
Python Programming, 1/e 11 Recursion vs. Iteration Recursion vs. Iteration
¢ There are similarities between iteration ¢ In the factorial and binary search problems, (looping) and recursion. the looping and recursive solutions use roughly the same algorithms, and their ¢ In fact, anything that can be done with a loop efficiency is nearly the same. can be done with a simple recursive function! ¢ In the exponentiation problem, two different Some programming languages use recursion algorithms are used. The looping version exclusively. takes linear time to complete, while the ¢ Some problems that are simple to solve with recursive version executes in log time. The recursion are quite difficult to solve with difference between them is like the difference loops. between a linear and binary search.
Python Programming, 1/e 67 Python Programming, 1/e 68
Recursion vs. Iteration Recursion vs. Iteration
¢ So… will recursive solutions always be ¢ Loop version:
as efficient or more efficient than their ¢ Let‘s use two variables, curr and prev, to iterative counterpart? calculate the next number in the sequence.
¢ The Fibonacci sequence is the sequence ¢ Once this is done, we set prev equal to of numbers 1,1,2,3,5,8,… curr, and set curr equal to the just- calculated number. ¢ The sequence starts with two 1‘s ¢ All we need to do is to put this into a loop ¢ Successive numbers are calculated by to execute the right number of times! finding the sum of the previous two
Python Programming, 1/e 69 Python Programming, 1/e 70
Recursion vs. Iteration Recursion vs. Iteration
¢ def loopfib(n): ¢ # returns the nth Fibonacci number The Fibonacci sequence also has a recursive definition: curr = 1 prev = 1 À 1 if n < 3 for i in range(n-2): fib(n) = Ã curr, prev = curr+prev, curr Õ fib(n −1) + fib(n − 2) otherwise return curr ¢ This recursive definition can be directly ¢ Note the use of simultaneous assignment to turned into a recursive function! calculate the new values of curr and prev. ¢ def fib(n): if n < 3: ¢ The loop executes only n-2 since the first two return 1 else: values have already been —determined“. return fib(n-1)+fib(n-2)
Python Programming, 1/e 71 Python Programming, 1/e 72
Python Programming, 1/e 12 Recursion vs. Iteration Recursion vs. Iteration
¢ This function obeys the rules that we‘ve ¢ The recursive solution is extremely set out. inefficient, since it performs many duplicate calculations! ¢ The recursion is always based on smaller values.
¢ There is a non-recursive base case.
¢ So, this function will work great, won‘t it? œ Sort of…
Python Programming, 1/e 73 Python Programming, 1/e 74
Recursion vs. Iteration Recursion vs. Iteration
¢ Recursion is another tool in your problem- solving toolbox. ¢ Sometimes recursion provides a good solution because it is more elegant or efficient than a looping version. ¢ At other times, when both algorithms are quite similar, the edge goes to the looping ¢ To calculate fib(6), fib(4)is calculated twice, solution on the basis of speed. fib(3)is calculated three times, fib(2)is ¢ Avoid the recursive solution if it is terribly calculated four times… For large numbers, this inefficient, unless you can‘t come up with an adds up! iterative solution (which sometimes happens!)
Python Programming, 1/e 75 Python Programming, 1/e 76
Sorting Algorithms Naive Sorting: Selection Sort
¢ The basic sorting problem is to take a ¢ To start out, pretend you‘re the list and rearrange it so that the values computer, and you‘re given a shuffled are in increasing (or nondecreasing) stack of index cards, each with a order. number. How would you put the cards back in order?
Python Programming, 1/e 77 Python Programming, 1/e 78
Python Programming, 1/e 13 Naive Sorting: Selection Sort Naive Sorting: Selection Sort
¢ One simple method is to look through the ¢ We already have an algorithm to find deck to find the smallest value and place the smallest item in a list (Chapter 7). that value at the front of the stack. As you go through the list, keep track ¢ Then go through, find the next smallest of the smallest one seen so far, number in the remaining cards, place it updating it when you find a smaller behind the smallest card at the front. one.
¢ Rinse, lather, repeat, until the stack is in ¢ This sorting algorithm is known as a sorted order! selection sort.
Python Programming, 1/e 79 Python Programming, 1/e 80
Naive Sorting: Selection Sort Naive Sorting: Selection Sort
¢ The algorithm has a loop, and each time ¢ When we place a value into its proper through the loop the smallest remaining position, we need to be sure we don‘t element is selected and moved into its proper position. accidentally lose the value originally stored in ¢ For n elements, we find the smallest value and put that position. it in the 0th position. ¢ If the smallest item is in position 10, moving ¢ Then we find the smallest remaining value from position 1 œ (n-1) and put it into position 1. it into position 0 involves the assignment: ¢ The smallest value from position 2 œ (n-1) goes in nums[0] = nums[10] position 2. ¢ This wipes out the original value in nums[0]! ¢ Etc.
Python Programming, 1/e 81 Python Programming, 1/e 82
Naive Sorting: Selection Sort Naive Sorting: Selection Sort
def selSort(nums): ¢ We can use simultaneous assignment to # sort nums into ascending order swap the values between nums[0] and n = len(nums) # For each position in the list (except the very last) nums[10]: for bottom in range(n-1): nums[0],nums[10] = nums[10],nums[0] # find the smallest item in nums[bottom]..nums[n-1] mp = bottom # bottom is smallest initially for i in range(bottom+1, n): # look at each position ¢ Using these ideas, we can implement if nums[i] < nums[mp]: # this one is smaller our algorithm, using variable bottom mp = i # remember its index # swap smallest item to the bottom for the currently filled position, and mp nums[bottom], nums[mp] = nums[mp], nums[bottom] is the location of the smallest remaining value.
Python Programming, 1/e 83 Python Programming, 1/e 84
Python Programming, 1/e 14 Naive Sorting: Selection Sort Naive Sorting: Selection Sort
¢ Rather than remembering the minimum value ¢ The selection sort is easy to write and scanned so far, we store its position in the list works well for moderate-sized lists, but in the variable mp. is not terribly efficient. We‘ll analyze ¢ New values are tested by comparing the item this algorithm in a little bit. in position i with the item in position mp.
¢ bottom stops at the second to last item in the list. Why? Once all items up to the last are in order, the last item must be the largest!
Python Programming, 1/e 85 Python Programming, 1/e 86
Divide and Conquer: Divide and Conquer: Merge Sort Merge Sort
¢ We‘ve seen how divide and conquer ¢ This process of combining two sorted works in other types of problems. How lists into a single sorted list is called could we apply it to sorting? merging.
¢ Say you and your friend have a deck of ¢ Our merge sort algorithm looks like: split nums into two halves shuffled cards you‘d like to sort. Each of sort the first half you could take half the cards and sort sort the second half them. Then all you‘d need is a way to merge the two sorted halves back into nums recombine the two sorted stacks!
Python Programming, 1/e 87 Python Programming, 1/e 88
Divide and Conquer: Divide and Conquer: Merge Sort Merge Sort
¢ Step 1: split nums into two halves ¢ Once the smaller value is removed, examine both top cards. Whichever is smaller will be the next ¢ Simple! Just use list slicing! item in the list. ¢ Step 4: merge the two sorted halves ¢ Continue this process of placing the smaller of the top two cards until one of the stacks runs out, in back into nums which case the list is finished with the cards from ¢ This is simple if you think of how you‘d do it the remaining stack. yourself… ¢ In the following code, lst1 and lst2 are the
¢ You have two sortedstacks, each with the smaller lists and lst3 is the larger list for the smallest value on top. Whichever of these two is results. The length of lst3 must be equal to the sum of the lengths of lst1 and lst2. smaller will be the first item in the list.
Python Programming, 1/e 89 Python Programming, 1/e 90
Python Programming, 1/e 15 Divide and Conquer: Divide and Conquer: Merge Sort Merge Sort
def merge(lst1, lst2, lst3): # Here either lst1 or lst2 is done. One of the following loops # merge sorted lists lst1 and lst2 into lst3 # will execute to finish up the merge.
# these indexes keep track of current position in each list # Copy remaining items (if any) from lst1 i1, i2, i3 = 0, 0, 0 # all start at the front while i1 < n1: n1, n2 = len(lst1), len(lst2) lst3[i3] = lst1[i1] i1 = i1 + 1 # Loop while both lst1 and lst2 have more items i3 = i3 + 1
while i1 < n1 and i2 < n2: # Copy remaining items (if any) from lst2 if lst1[i1] < lst2[i2]: # top of lst1 is smaller while i2 < n2: lst3[i3] = lst1[i1] # copy it into current spot in lst3 lst3[i3] = lst2[i2] i1 = i1 + 1 i2 = i2 + 1 else: # top of lst2 is smaller i3 = i3 + 1 lst3[i3] = lst2[i2] # copy itinto current spot in lst3 i2 = i2 + 1 i3 = i3 + 1 # item added to lst3, update position
Python Programming, 1/e 91 Python Programming, 1/e 92
Divide and Conquer: Divide and Conquer: Merge Sort Merge Sort
¢ We can slice a list in two, and we can ¢ We need to find at least one base case merge these new sorted lists back into that does not require a recursive call, a single list. How are we going to sort and we also need to ensure that the smaller lists? recursive calls are always made on
¢ We are trying to sort a list, and the smaller versions of the original problem. algorithm requires two smaller sorted ¢ For the latter, we know this is true since lists… this sounds like a job for each time we are working on halves of recursion! the previous list.
Python Programming, 1/e 93 Python Programming, 1/e 94
Divide and Conquer: Divide and Conquer: Merge Sort Merge Sort
if len(nums) > 1: ¢ Eventually, the lists will be halved into lists with a single element each. What do we split nums into two halves mergeSort the first half know about a list with a single item? mergeSort the seoncd half ¢ It‘s already sorted!! We have our base case! mergeSort the second half merge the two sorted halves back into nums ¢ When the length of the list is less than 2, we do nothing.
¢ We update the mergeSort algorithm to make it properly recursive…
Python Programming, 1/e 95 Python Programming, 1/e 96
Python Programming, 1/e 16 Divide and Conquer: Divide and Conquer: Merge Sort Merge Sort
def mergeSort(nums): # Put items of nums into ascending order ¢ Recursion is closely related to the idea n = len(nums) # Do nothing if nums contains 0 or 1 items of mathematical induction, and it if n > 1: # split the two sublists requires practice before it becomes m = n/2 nums1, nums2 = nums[:m], nums[m:] comfortable. # recursively sort each piece mergeSort(nums1) ¢ Follow the rules and make sure the mergeSort(nums2) # merge the sorted pieces back into original list merge(nums1, nums2, nums) recursive chain of calls reaches a base case, and your algorithms will work!
Python Programming, 1/e 97 Python Programming, 1/e 98
Comparing Sorts Comparing Sorts
¢ We now have two sorting algorithms. ¢ Let‘s start with selection sort. Which one should we use? ¢ In this algorithm we start by finding the smallest item, then finding the smallest of the ¢ The difficulty of sorting a list depends remaining items, and so on. on the size of the list. We need to ¢ Suppose we start with a list of size n. To find figure out how many steps each of our the smallest element, the algorithm inspects sorting algorithms requires as a all n items. The next time through the loop, it inspects the remaining n-1 items. The total function of the size of the list to be number of iterations is: sorted. n + (n-1) + (n-2) + (n-3) + … + 1
Python Programming, 1/e 99 Python Programming, 1/e 100
Comparing Sorts Comparing Sorts
¢ The time required by selection sort to ¢ If the size of a list doubles, it will take sort a list of n items is proportional to four times as long to sort. Tripling the the sum of the first n whole numbers, size will take nine times longer to sort! n(n +1) or 2 . ¢ Computer scientists call this a quadratic 2 2 ¢ This formula contains an n term, or n algorithm. meaning that the number of steps in the algorithm is proportional to the square of the size of the list.
Python Programming, 1/e 101 Python Programming, 1/e 102
Python Programming, 1/e 17 Comparing Sorts Comparing Sorts
¢ In the case of the merge sort, a list is divided into two pieces and each piece is sorted before merging them back together. The real place where the
sorting occurs is in the merge function. ¢ This diagram shows how [3,1,4,1,5,9,2,6] is sorted. ¢ Starting at the bottom, we have to copy the n values into the second level.
Python Programming, 1/e 103 Python Programming, 1/e 104
Comparing Sorts Comparing Sorts
¢ We know from the analysis of binary
search that this is just log2n. ¢ Therefore, the total work required to
sort n items is nlog2n. ¢ From the second to third levels the n values ¢ Computer scientists call this an n log n need to be copied again. algorithm. ¢ Each level of merging involves copying n values. The only remaining question is how many levels are there? Python Programming, 1/e 105 Python Programming, 1/e 106
Comparing Sorts Comparing Sorts
2 ¢ So, which is going to be better, the n selection sort, or the n logn merge sort?
¢ If the input size is small, the selection sort might be a little faster because the code is simpler and there is less overhead.
¢ What happens as n gets large? We saw in our discussion of binary search that the log function grows very slowly, so nlogn will grow much slower than n2.
Python Programming, 1/e 107 Python Programming, 1/e 108
Python Programming, 1/e 18 Hard Problems Towers of Hanoi
¢ Using divide-and-conquer we could ¢ One elegant application of recursion is to the design efficient algorithms for searching Towers of Hanoi or Towers of Brahma puzzle and sorting problems. attributed to ÉdouardLucas. ¢ Divide and conquer and recursion are ¢ There are three posts and sixty-four very powerful techniques for algorithm concentric disks shaped like a pyramid. design. ¢ The goal is to move the disks from one post ¢ Not all problems have efficient to another, following these three rules: solutions!
Python Programming, 1/e 109 Python Programming, 1/e 110
Towers of Hanoi Towers of Hanoi
¢ Only one disk may be moved at a time.
¢ A disk may not be —set aside“. It may only be stacked on one of the three posts.
¢ A larger disk may never be placed on top of a smaller one.
¢ If we label the posts as A, B, and C, we could express an algorithm to move a pile of disks from A to C, using B as temporary storage, as: Move disk from A to C Move disk from A to B Move disk from C to B
Python Programming, 1/e 111 Python Programming, 1/e 112
Towers of Hanoi Towers of Hanoi
¢ Let‘s consider some easy cases œ ¢ 3 disks To move the largest disk to C, we first ¢ 1 disk Move disk from A to C need to move the two smaller disks out of the way. These two smaller disks form a ¢ 2 disks pyramid of size 2, which we know how to Move disk from A to B solve. Move disk from A to C Move a tower of two from A to B Move disk from B to C Move one disk from A to C Move a tower of two from B to C
Python Programming, 1/e 113 Python Programming, 1/e 114
Python Programming, 1/e 19 Towers of Hanoi Towers of Hanoi
¢ Algorithm: move n-disk tower from source to destination via resting place ¢ In moveTower, n is the size of the tower (integer), and source, dest, and temp are move n-1 disk tower from source to resting place move 1 disk tower from source to destination the three posts, represented by —A“, —B“, and move n-1 disk tower from resting place to destination —C“.
¢ What should the base case be? ¢ def moveTower(n, source, dest, temp): Eventually we will be moving a tower of if n == 1: print "Move disk from", source, "to", dest+".“ size 1, which can be moved directly else: moveTower(n-1, source, temp, dest) without needing a recursive call. moveTower(1, source, dest, temp) moveTower(n-1, temp, dest, source)
Python Programming, 1/e 115 Python Programming, 1/e 116
Towers of Hanoi Towers of Hanoi
¢ To get things started, we need to supply ¢ Why is this a —hard Number of Steps in parameters for the four parameters: problem“? Disks Solution def hanoi(n): moveTower(n, "A", "C", "B") ¢ How many steps in 1 1 our program are ¢ >>> hanoi(3) 2 3 Move disk from A to C. required to move a Move disk from A to B. tower of size n? Move disk from C to B. 3 7 Move disk from A to C. Move disk from B to A. 4 15 Move disk from B to C. Move disk from A to C. 5 31
Python Programming, 1/e 117 Python Programming, 1/e 118
Towers of Hanoi Towers of Hanoi
n ¢ To solve a puzzle of size n will require 2 -1 ¢ Even though the algorithm for Towers of steps. Hanoi is easy to express, it belongs to a ¢ Computer scientists refer to this as an class of problems known as intractable exponential time algorithm. problems œ those that require too many ¢ Exponential algorithms grow very fast. computing resources (either time or ¢ For 64 disks, moving one a second, round the memory) to be solved except for the clock, would require 580 billion years to complete. The current age of the universe is simplest of cases. estimated to be about 15 billion years. ¢ There are problems that are even harder than the class of intractable problems.
Python Programming, 1/e 119 Python Programming, 1/e 120
Python Programming, 1/e 20 The Halting Problem The Halting Problem
¢ Let‘s say you want to write a program that ¢ Program Specification: looks at other programs to determine ¢ Program: Halting Analyzer whether they have an infinite loop or not. ¢ Inputs: A Python program file. The input for the program. ¢ We‘ll assume that we need to also know the ¢ Outputs: —OK“ if the program will eventually stop. input to be given to the program in order to —FAULTY“ if the program has an infinite loop. make sure it‘s not some combination of input ¢ You‘ve seen programs that look at programs and the program itself that causes it to before œ like the Python interpreter! infinitely loop. ¢ The program and its inputs can both be represented by strings.
Python Programming, 1/e 121 Python Programming, 1/e 122
The Halting Problem The Halting Problem
¢ There is no possible algorithm that can ¢ To do a proof by contradiction, we meet this specification! assume the opposite of what we‘re
¢ This is different than saying no one‘s trying to prove, and show this leads to been able to write such a program… we a contradiction. can prove that this is the case using a ¢ First, let‘s assume there is an algorithm mathematical technique known as proof that can determine if a program by contradiction. terminates for a particular set of inputs. If it does, we could put it in a function:
Python Programming, 1/e 123 Python Programming, 1/e 124
The Halting Problem The Halting Problem
¢ def terminates(program, inputData): def main(): # program and inputData are both strings # Read a program from standard input # Returns true if program would halt when run lines = [] # with inputData as its input print "Type in a program (type 'done' to quit)." ¢ If we had a function like this, we could write line = raw_input("") while line != "done": the following program: lines.append(line) ¢ # turing.py line = raw_input("") testProg = string.join(lines, "\n") import string
def terminates(program, inputData): # If program halts on itself as input, go into # program and inputData are both strings # an inifinite loop # Returns true if program would halt when run if terminates(testProg, testProg): # with inputData as its input while True: pass # a pass statement does nothing
Python Programming, 1/e 125 Python Programming, 1/e 126
Python Programming, 1/e 21 The Halting Problem The Halting Problem
¢ The program is called —turing.py“ in ¢ turing.py first reads in a program typed by honor of Alan Turing, the British the user, using a sentinel loop. mathematician who is considered to be ¢ The string.join function then the —father of Computer Science“. concatenates the accumulated lines together, putting a newline (\n) character between ¢ Let‘s look at the program step-by-step them. to see what it does… ¢ This creates a multi-line string representing the program that was entered.
Python Programming, 1/e 127 Python Programming, 1/e 128
The Halting Problem The Halting Problem
¢ turing.py next uses this program as not ¢ This was all just a set-up for the big only the program to test, but also as the question: What happens when we run input to test. turing.py, and use turing.py as ¢ In other words, we‘re seeing if the program the input? you typed in terminates when given itself as input. ¢ Does turing.py halt when given itself ¢ If the input program terminates, the turing as input? program will go into an infinite loop.
Python Programming, 1/e 129 Python Programming, 1/e 130
The Halting Problem The Halting Problem
¢ In the terminates function, turing.py will ¢ Turing.py does not halt be evaluated to see if it halts or not. ¢ terminates returns false ¢ When terminates returns false, the program ¢ We have two possible cases: quits ¢ turing.py halts when given itself as input ¢ When the program quits, it has halted, a ¢ Terminates returns true contradiction
¢ So, turing.py goes into an infinite loop ¢ The existence of the function terminates ¢ Therefore turing.py doesn‘t halt, a contradiction would lead to a logical impossibility, so we can conclude that no such function exists.
Python Programming, 1/e 131 Python Programming, 1/e 132
Python Programming, 1/e 22 Conclusion
¢ Computer Science is more than programming!
¢ The most important computer for any computing professional is between their ears.
Python Programming, 1/e 133
Python Programming, 1/e 23