CS 251 SpringFall 2019 2020 Principles of of Programming Programming Languages Languages Ben Wood Topics λ Ben Wood Recursion is an elegant and natural match for many computations and data structures. Tail Recursion • Natural recursion with immutable data can be space- inefficient compared to loop iteration with mutable data. • Tail recursion eliminates the space inefficiency with a +tail.rkt simple, general pattern. • Recursion over immutable data expresses iteration more clearly than loop iteration with mutable state. • More higher-order patterns: fold https://cs.wellesley.edu/~cs251/s20/ Tail Recursion 1 Tail Recursion 2 Naturally recursive factorial CS 240-style machine model Registers Code Stack (define (fact n) Call frame (if (= n 0) 1 Call frame (* n (fact (- n 1))))) Call frame Heap arguments, variables, fixed size, general purpose general size, fixed return address per function call Space: O( ) Program How efficient is this implementation? Counter cons cells, Time: O( ) Stack data structures, … Pointer Tail Recursion 3 Tail Recursion 4 Evaluation (define (fact n) (if (= n 0) Naturally recursive factorial example 1 (* n (fact (- n 1))))) Call stacks at each step (fact 3) (fact 3): 3*_ (fact 3): 3*_ (fact 3): 3*_ (define (fact n) (fact 2) (fact 2): 2*_ (fact 2): 2*_ (if (= n 0) Compute result so far Base case returns 1 after/from recursive call. (fact 1) (fact 1): 1*_ base result. Remember: n ↦ 2; and (* n (fact (- n 1))))) “rest of function” for this call. (fact 0) Recursive case returns result so far. Compute remaining argument before/for recursive call. (fact 3): 3*_ (fact 3): 3*_ (fact 3): 3*_ (fact 3): 3*2 (fact 2): 2*_ (fact 2): 2*_ (fact 2): 2*1 Space: O( ) (fact 1): 1*_ (fact 1): 1*1 Time: O( ) (fact 0): 1 Tail Recursion 5 Tail Recursion 6 Tail recursive factorial Common patterns of work Accumulator parameter Natural recursion: Tail recursion: provides result so far. (define (fact n) Argument Full result Argument Base result (define (fact-tail n acc) (if (= n 0) Compute result so far before/for recursive call. Base case returns acc Reduce Reduce full result. (fact-tail (- n 1) (* n acc)))) argument argument Recursive case returns full result. Compute remaining argument Accumulate Accumulate before/for recursive call. result result (fact-tail n 1)) so far so far Initial accumulator calls Deeper recursive provides base result. Base case Base result Base case Full result Tail Recursion 7 Tail Recursion 8 argument full result Natural Tail argument base result (fact 4): 24 (fact-tail 4 1): 24 recursion -1 * recursion -1 * reduce reduce (fact 3): 6 (fact-tail 3 4): 24 Recursive case: -1 * Recursive case: -1 * accumulate Compute result Compute recursive argument accumulate (fact 2): 2 (fact-tail 2 12): 24 in terms of argument and in terms of argument and accumulated recursive result. -1 * accumulator. -1 * (fact 1): 1 (fact-tail 1 24): 24 -1 * -1 * (fact 0): 1 (fact-tail 0 24): 24 (define (fact n) (define (fact n) base case base result base case full result (if (= n 0) (define (fact-tail n acc) 1 (if (= n 0) (* n (fact (- n 1))))) acc (fact-tail (- n 1) (* n acc)))) (fact-tail n 1)) Tail Recursion 9 Tail Recursion 10 Evaluation example Nothing useful Tail-call optimization Call stacks at each step remembered here. (fact 3) (fact 3): _ (fact 3): _ (fact 3): _ (define (fact n) Space: O( ) (define (fact-tail n acc) (ft 3 1) (ft 3 1):_ (ft 3 1):_ (if (= n 0) Time: O( ) acc ft = fact-tail (ft 2 3) (ft 2 3):_ (fact-tail (- n 1) (* n acc)))) (fact-tail n 1)) (ft 1 6) (fact 3) (ft 3 1) (ft 2 3) (ft 1 6) (ft 0 6) (fact 3): _ (fact 3): _ (fact 3): _ (fact 3): _ (ft 3 1):_ (ft 3 1):_ (ft 3 1):_ (ft 3 1):_ Language implementation recognizes tail calls. • Caller frame never needed again. (ft 2 3):_ (ft 2 3):_ (ft 2 3):_ (ft 2 3):6 • Reuse same space for every recursive tail call. (ft 1 6):_ (ft 1 6):_ (ft 6 1):6 • Low-level: acts just like a loop. etc. (ft 0 6) (ft 0 6):6 Racket, ML, most “functional” languages, but not Java, C, etc. Tail Recursion 11 Tail Recursion 12 Tail call intuition: “nothing left for caller to do after call”, “callee result is immediate caller result” Tail recursion transformation Tail position Common pattern for transforming naturally recursive functions to tail-recursive form. Works for functions that do commutative operations (order of steps doesn't matter). Recursive definition of tail position: (define (fact n) natural recursion – In (lambda (x1 … xn) e), the body e is in tail position. (if (= n 0) – If (if e1 e2 e3) is in tail position, 1 then e2 and e3 are in tail position (but e1 is not). (* n (fact (- n 1)) ) )) – If (let ([x1 e1] … [xn en]) e) is in tail position, then e is in tail position (but the binding expressions are not). (define (fact n) tail recursion Note: (define (fact-tail n acc ) • If a non-lambda expression is not in tail position, then no subexpressions are. • Critically, in a function call expression(e1 e2), (if (= n 0) Accumulator subexpressions e1 and e2 are not in tail position. becomes acc base result. (fact-tail (- n 1) (* n acc ) ))) A tail call is a function call in tail position. (fact-tail n 1 )) A function is tail-recursive if it uses a recursive tail call. Recursive step applied to accumulator Base result becomes instead of recursive result. Tail Recursion 13 initial accumulator. Tail Recursion 14 (order matters) Practice: use the transformation Transforming non-commutative steps (define (reverse-natural-slow xs) ;; Naturally recursive sum (if (null? xs) (define (sum-natural xs) null (if (null? xs) (append (reverse-natural-slow (cdr xs)) 0 (list (car xs))))) (+ (car xs) (sum-natural (cdr xs))))) (define (reverse-tail-just-kidding xs) (define (rev xs acc) ;; Tail-recursive sum (if (null? xs) acc ✘ (define (sum-tail xs) (define (sum-onto xs acc) (rev (cdr xs) (append acc (list (car xs)))))) (if (null? xs) (rev xs null)) acc (define (reverse-tail-slow xs) (sum-onto (cdr xs) (+ (car xs) acc))) (define (rev xs acc) (if (null? xs) acc ✓ (sum-onto xs 0)) (rev (cdr xs) (append (list (car xs)) acc)))) Tail Recursion 15 (rev xs null)) Tail Recursion 16 The transformation is not always ideal. Tail recursion ≠ accumulator pattern (define (reverse-tail-slow xs) (define (rev xs acc) ; mutually tail recursive (if (null? xs) O( ) (define (even n) acc (or (zero? n) (odd (- n 1)))) (rev (cdr xs) (append (list (car xs)) acc)))) (define (odd n) (rev xs null)) (or (not (zero? n)) (even (- n 1)))) (define (reverse-tail-good xs) ; tail recursive (define (rev xs acc) (define (even2 n) (if (null? xs) O(n) (cond [(= 0 n) #t] acc [(= 1 n) #f] (rev (cdr xs) (cons (car xs) acc)))) [#t (even2 (- n 2))])) (rev xs null)) • Append-based recursive reverse is O(n2): each recursive call must • Tail recursion and the accumulator pattern are commonly traverse to end of list and build a fully new list. used together. They are not synonyms. – 1+2+…+(n-1) is almost n*n/2 – Natural recursion may use an accumulator. – Moral: beware append, especially within recursion – Tail recursion does not necessarily involve an accumulator. • Tail-recursive reverse can avoid append in O(n). What about map, filter? – Cons is O(1), done n times. Tail Recursion 17 Tail Recursion 18 Why tail recursion instead of Identify dependences between ________. loops with mutation? (define (fib n) Racket: immutable natural recursion (if (< n 2) recursive n calls 1. Simpler language, but just as efficient. (+ (fib (- n 1)) (fib (- n 2))))) 2. Explicit dependences for easier reasoning. (define (fib n) Racket: immutable tail recursion (define (fib-tail n fibi fibi+1) – Especially with HOFs like fold! (if (= 0 n) fibi (fib-tail (- n 1) fibi+1 (+ fibi fibi+1)))) (fib n 0 1)) def fib(n): Python: loop iteration with mutation fib_i = 0 fib_i_plus_1 = 1 for i in range(n): loop fib_i_prev = fib_i fib_i = fib_i_plus_1 iterations fib_i_plus_1 = fib_i_prev + fib_i_plus_1 return fib_i Tail Recursion 19 Tail Recursion 20 What must we inspect to Identify dependences between ________. HOF HOF Fold: iterator over recursive structures Racket: immutable natural recursion (define (fib n) (a.k.a. reduce, inject, …) (if (< n 2) recursive n calls (fold_ combine init list) (+ (fib (- n 1)) (fib (- n 2))))) accumulates result by iteratively applying (define (fib n) Racket: immutable tail recursion (combine element accumulator) (define (fib-tail n fibi fibi+1) (if (= 0 n) to each element of the list and accumulator so far fibi (starting from init) to produce the next accumulator. (fib-tail (- n 1) fibi+1 (+ fibi fibi+1)))) (fib n 0 1)) – (foldr f init (list 1 2 3)) def fib(n): Python: loop iteration with mutation fib_i = 0 computes (f 1 (f 2 (f 3 init))) fib_i_plus_1 = 1 for i in range(n): loop fib_i_prev = fib_i – (foldl f init (list 1 2 3)) fib_i = fib_i_plus_1 iterations computes (f 3 (f 2 (f 1 init))) fib_i_plus_1 = fib_i_prev + fib_i_plus_1 return fib_i Tail Recursion 21 Tail Recursion 22 Folding geometry Fold code: tail.rkt Natural recursion (foldr combine init L) • foldr implementation • foldl implementation result combine combine ⋯ combine combine init • using foldr/foldl • bonus mystery folding puzzle L ⟼ V V V V 1 2 ⋯ n-1 n init combine combine ⋯ combine combine result (foldl combine init L) Tail recursion Tail Recursion 23 Tail Recursion 24 Super-iterators! • Not built into the language – Just a programming pattern – Many languages have built-in support, often allow stopping early without resorting to exceptions • Pattern separates recursive traversal from data processing – Reuse same traversal, different folding functions – Reuse same folding functions, different data structures – Common vocabulary concisely communicates intent • map, filter, fold + closures/lexical scope = superpower – Later: argument function can use any “private” data in its environment.