tally [] y0 tally [] y0 tally [8] y8+0 tally [8] y8+0 tally [6;8] y6+8 tally [6;8] y6+8 tally [2;6;8] y2+14 y2+14 y4+16 y4+16 20 20

Forward recursion

CS 321 Programming Languages In forward recursion, you first call the function recursively on all Intro to OCaml – Recursion (tail vs forward) the recursive components, and then build the final result from the partial results.

Baris Aktemur Wait until the whole structure has been traversed to start building Ozye˘ginUniversity¨ the answer.

th # let rec tally lst= Last update made on Thursday 12 October, 2017 at 11:25. match lst with |[] ->0 |x::xs ->x+ tally xs;;

Much of the contents here are taken from Elsa Gunter and Sam # let rec squareUp lst= Kamin’s OCaml notes available at match lst with http://courses.engr.illinois.edu/cs421 |[] ->[] |x::xs ->(x*x)::squareUp xs;;

Ozye˘gin¨ University — CS 321 Programming Languages 1 Ozye˘gin¨ University — CS 321 Programming Languages 2

Functions calls and the stack Functions calls and the stack

tally [2;6;8] tally [4;2;6;8] tally [4;2;6;8]

Ozye˘gin¨ University — CS 321 Programming Languages 3 Ozye˘gin¨ University — CS 321 Programming Languages 3 tally [] y0 tally [] y0 tally [8] y8+0 y8+0 y6+8 y6+8 y2+14 y2+14 y4+16 y4+16 20 20

y0 y8+0 y8+0 y6+8 y6+8 y2+14 y2+14 y4+16 y4+16 20 20

Functions calls and the stack Functions calls and the stack

tally [8] tally [6;8] tally [6;8] tally [2;6;8] tally [2;6;8] tally [4;2;6;8] tally [4;2;6;8]

Ozye˘gin¨ University — CS 321 Programming Languages 3 Ozye˘gin¨ University — CS 321 Programming Languages 3

Functions calls and the stack Functions calls and the stack

tally [] tally [] y0 tally [8] tally [8] tally [6;8] tally [6;8] tally [2;6;8] tally [2;6;8] tally [4;2;6;8] tally [4;2;6;8]

Ozye˘gin¨ University — CS 321 Programming Languages 3 Ozye˘gin¨ University — CS 321 Programming Languages 3 tally [] y0 tally [] y0 tally [8] y8+0 y6+8 y2+14 y2+14 y4+16 y4+16 20 20

tally [] y0 tally [] y0 tally [8] y8+0 tally [8] y8+0 tally [6;8] y6+8 tally [6;8] y6+8 tally [2;6;8] y2+14 y4+16 20 20

Functions calls and the stack Functions calls and the stack

tally [8] y8+0 tally [6;8] tally [6;8] y6+8 tally [2;6;8] tally [2;6;8] tally [4;2;6;8] tally [4;2;6;8]

Ozye˘gin¨ University — CS 321 Programming Languages 3 Ozye˘gin¨ University — CS 321 Programming Languages 3

Functions calls and the stack Functions calls and the stack

tally [2;6;8] y2+14 tally [4;2;6;8] tally [4;2;6;8] y4+16

Ozye˘gin¨ University — CS 321 Programming Languages 3 Ozye˘gin¨ University — CS 321 Programming Languages 3 tally [] y0 tally [8] y8+0 tally [6;8] y6+8 tally [2;6;8] y2+14 tally [4;2;6;8] y4+16

tally [] 20 y20 tally [] 20 y20 tally [8] 12 y20 tally [8] 12 y20 tally [6;8] 6 y20 tally [6;8] 6 y20 tally [2;6;8] 4 y20 y20 y20 y20 20 20

Functions calls and the stack Tail recursion

A recursive function is tail-recursive if all recursive calls are the last thing that the function does.

Tail recursion generally requires extra “accumulator” arguments to pass partial results.

I May require an auxiliary function.

# let rec tally lst acc= match lst with |[] -> acc 20 |x::xs -> tally xs(x+acc);;

val tally:’a list -> int -> int

# tally[1;2;3;4;5]0;; (* Have to give an initial accumulated value *) val it: int= 15

Ozye˘gin¨ University — CS 321 Programming Languages 3 Ozye˘gin¨ University — CS 321 Programming Languages 4

Functions calls and the stack Functions calls and the stack

tally [2;6;8] 4 tally [4;2;6;8] 0 tally [4;2;6;8] 0

Ozye˘gin¨ University — CS 321 Programming Languages 5 Ozye˘gin¨ University — CS 321 Programming Languages 5 tally [] 20 y20 tally [] 20 y20 tally [8] 12 y20 y20 y20 y20 y20 y20 y20 y20 20 20

y20 y20 y20 y20 y20 y20 y20 y20 y20 20 20

Functions calls and the stack Functions calls and the stack

tally [8] 12 tally [6;8] 6 tally [6;8] 6 tally [2;6;8] 4 tally [2;6;8] 4 tally [4;2;6;8] 0 tally [4;2;6;8] 0

Ozye˘gin¨ University — CS 321 Programming Languages 5 Ozye˘gin¨ University — CS 321 Programming Languages 5

Functions calls and the stack Functions calls and the stack

tally [] 20 tally [] 20 y20 tally [8] 12 tally [8] 12 tally [6;8] 6 tally [6;8] 6 tally [2;6;8] 4 tally [2;6;8] 4 tally [4;2;6;8] 0 tally [4;2;6;8] 0

Ozye˘gin¨ University — CS 321 Programming Languages 5 Ozye˘gin¨ University — CS 321 Programming Languages 5 tally [] 20 y20 tally [] 20 y20 tally [8] 12 y20 y20 y20 y20 y20 y20 20 20

tally [] 20 y20 tally [] 20 y20 tally [8] 12 y20 tally [8] 12 y20 tally [6;8] 6 y20 tally [6;8] 6 y20 tally [2;6;8] 4 y20 y20 20 20

Functions calls and the stack Functions calls and the stack

tally [8] 12 y20 tally [6;8] 6 tally [6;8] 6 y20 tally [2;6;8] 4 tally [2;6;8] 4 tally [4;2;6;8] 0 tally [4;2;6;8] 0

Ozye˘gin¨ University — CS 321 Programming Languages 5 Ozye˘gin¨ University — CS 321 Programming Languages 5

Functions calls and the stack Functions calls and the stack

tally [2;6;8] 4 y20 tally [4;2;6;8] 0 tally [4;2;6;8] 0 y20

Ozye˘gin¨ University — CS 321 Programming Languages 5 Ozye˘gin¨ University — CS 321 Programming Languages 5 tally [] 20 y20 tally [8] 12 y20 tally [6;8] 6 y20 tally [2;6;8] 4 y20 tally [4;2;6;8] 0 y20

Functions calls and the stack An optimization idea

Observation When we have tail recursion, a function that is waiting for the called function to return a value simply propagates the return value to its caller.

Idea We don’t need to keep the frame of a tail-recursive function on the 20 stack. Simply replace the tail-recursive function’s stack frame with the called function.

Ozye˘gin¨ University — CS 321 Programming Languages 5 Ozye˘ginUniversity¨ — CS 321 Programming Languages 6

Functions calls and the stack Functions calls and the stack

tally [4;2;6;8] 0 tally [2;6;8] 4

Ozye˘gin¨ University — CS 321 Programming Languages 7 Ozye˘gin¨ University — CS 321 Programming Languages 8 y20

Functions calls and the stack Functions calls and the stack

tally [6;8] 6 tally [8] 12

Ozye˘gin¨ University — CS 321 Programming Languages 9 Ozye˘ginUniversity¨ — CS 321 Programming Languages 10

Functions calls and the stack Functions calls and the stack

tally [] 20 tally [] 20 y20

Ozye˘ginUniversity¨ — CS 321 Programming Languages 11 Ozye˘ginUniversity¨ — CS 321 Programming Languages 11 ifn=0 then[] elsex:: makeList x(n-1);;

Functions calls and the stack Why do we care?

Reusing the stack frame of the tail-recursive function is known as 20 the tail call optimization. It is an automatic optimization applied by the compilers and interpreters.

Ozye˘ginUniversity¨ — CS 321 Programming Languages 12 Ozye˘ginUniversity¨ — CS 321 Programming Languages 13

Experiment Experiment

Write a function that takes a value x and an integer n, and returns Write a function that takes a value x and an integer n, and returns a list of length n whose elements are all x. a list of length n whose elements are all x. # let rec makeList x n= # let rec makeList x n=

ifn=0 then[] elsex:: makeList x(n-1);;

val makeList:’a -> int ->’a list= val makeList:’a -> int ->’a list=

# makeList "a"5;; # makeList "a"5;; -: string list=["a"; "a"; "a"; "a"; "a"] -: string list=["a"; "a"; "a"; "a"; "a"] # makeList3 40;; # makeList3 40;; -: int list= -: int list= [3;3;3;3;3;3;3;3;3;3;3;3;3;3;3; [3;3;3;3;3;3;3;3;3;3;3;3;3;3;3; 3;3;3;3;3;3;3;3;3;3;3;3;3;3;3; 3;3;3;3;3;3;3;3;3;3;3;3;3;3;3; 3;3;3;3;3;3;3;3;3;3] 3;3;3;3;3;3;3;3;3;3]

Ozye˘ginUniversity¨ — CS 321 Programming Languages 14 Ozye˘ginUniversity¨ — CS 321 Programming Languages 14 Experiment Experiment

# let rec makeList x n acc= ifn=0 then acc else makeList x(n-1)(x::acc);; # makeList3 99999;; -: int list= val makeList:’a -> int ->’a list ->’a list= [3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3; 3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3; # makeList3 10[];; 3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3; -: int list=[3;3;3;3;3;3;3;3;3;3] 3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3; ...] # makeList3 12345678[];; # makeList3 1234567;; -: int list= [3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3; Stack overflow during evaluation (looping recursion?). 3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3; 3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3; 3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3;3; ...]

Observation Hmm... So there really is a difference between tail vs. forward recursion.

Ozye˘ginUniversity¨ — CS 321 Programming Languages 15 Ozye˘gin¨ University — CS 321 Programming Languages 16

How to write tail-recursive functions? Tail recursion

# let rec squareUp lst= match lst with |[] ->[] |x::xs ->(x*x)::squareUp xs;; To write functions in tail-recursive form, answer the following val squareUp: int list -> int list= question: What information do I need to pass from the caller to the callee # let rec squareUp lst acc= (i.e. from a lower stack frame to the upper stack frame) so that I match lst with won’t need the caller again, and can simply throw it away? |[] -> acc |x::xs -> squareUp xs(acc@[x*x]);; val squareUp: int list -> int list -> int list=

# squareUp[1;2;3;4;5][];; -: int list=[1;4;9; 16; 25]

Ozye˘ginUniversity¨ — CS 321 Programming Languages 17 Ozye˘ginUniversity¨ — CS 321 Programming Languages 18 Exercise Exercise solutions

Convert the following functions to tail-recursive form. # let rec factorial n= # let rec factorial n acc= ifn=0 then acc ifn=0 then1 else factorial(n-1)(n*acc);; elsen* factorial(n-1);; val factorial: int -> int -> int # let rec power x n= # factorial01;; ifn=0 then1 -: int=1 elsex* power x(n-1);; # factorial11;; -: int=1 # let rec fib n= # factorial51;; -: int= 120 ifn=0 then1 else # factorial61;; ifn=1 then1 else -: int= 720 fib(n-1)+ fib(n-2);;

Ozye˘ginUniversity¨ — CS 321 Programming Languages 19 Ozye˘ginUniversity¨ — CS 321 Programming Languages 20

Exercise solutions Better Programming

# let rec fib n nm1 nm2= ifn=0 then nm2 else ifn=1 then nm1 Use an auxiliary function to hide the accumulator from the user. else fib(n-1)(nm1+nm2) nm1;; # let tally numbers= val fib: int -> int -> int -> int= let rec sum lst acc= match lst with |[] -> acc # fib011;; (* Initial values for nm1 and nm2 are 1 and 1 *) |x::xs -> sum xs(x+acc) -: int=1 in sum numbers0;; # fib111;; val tally: int list -> int= -: int=1 # fib211;; # let squareUp numbers= let rec aux lst acc= -: int=2 match lst with # fib311;; |[] -> acc -: int=3 |x::xs -> aux xs(acc@[x*x]) # fib411;; in aux numbers[];; -: int=5 val squareUp: int list -> int list= # fib511;; -: int=8 # fib611;; -: int= 13 Ozye˘ginUniversity¨ — CS 321 Programming Languages 21 Ozye˘ginUniversity¨ — CS 321 Programming Languages 22 # let rev aList= let rec aux lst acc= match lst with |[] -> acc |x::xs -> rev xs(x::acc) in aux aList[];; val rev:’a list ->’a list=

Better Programming Exercise

Convert the following function to tail-recursive form. Use an auxiliary function to hide the accumulator from the user. # let rec rev aList= # let factorial n= match aList with let rec fact m acc= |[] ->[] ifm=0 then acc |x::xs -> rev xs@[x];; else fact(m-1)(m*acc) in fact n1;; val rev:’a list ->’a list= val factorial: int -> int= # rev[1;2;3;4;5;6];; -: int list=[6;5;4;3;2;1] # let fib m= let rec aux n nm1 nm2= ifn=0 then nm2 else ifn=1 then nm1 else aux(n-1)(nm1+ nm2) nm1 in aux m11;; val fib: int -> int=

Ozye˘ginUniversity¨ — CS 321 Programming Languages 23 Ozye˘ginUniversity¨ — CS 321 Programming Languages 24

Exercise Tail vs. Forward

Convert the following function to tail-recursive form. # let rec rev aList= match aList with |[] ->[] |x::xs -> rev xs@[x];; Note: val rev:’a list ->’a list= In general, it is possible for a recursion to be neither forward nor # rev[1;2;3;4;5;6];; tail. This is the case when a function does some stuff, makes the -: int list=[6;5;4;3;2;1] recursive call, then further processes the return value of the recursive call. For simplicity and without loss of the point we are # let rev aList= let rec aux lst acc= making in this lecture, we consider only forward vs. tail recursion. match lst with |[] -> acc |x::xs -> rev xs(x::acc) in aux aList[];; val rev:’a list ->’a list=

Ozye˘ginUniversity¨ — CS 321 Programming Languages 24 Ozye˘ginUniversity¨ — CS 321 Programming Languages 25 Exercise How long will it take?

Convert the following function to tail-recursive form. # let rec numZeros lst= I Remember the big-oh notation from CS201? match lst with I Question: given input of size n, how long to generate output? |[] ->0 I Express output time in terms of input size, omit constants and |0::xs ->1+ numZeros xs take the biggest power. |x::xs -> numZeros xs;;

Ozye˘ginUniversity¨ — CS 321 Programming Languages 26 Ozye˘ginUniversity¨ — CS 321 Programming Languages 27

Common big-O times Linear time

I Expect most list operations to take linear time O(n). I Constant time O(1) I Each step of the recursion can be done in constant time. I input size doesn’t matter I Each step makes only one recursive call. I Linear time O(n)

I double input = double time I List example: length, squareUp, append ⇒2 I Quadratic time O(n ) I Integer example: factorial I double input = quadruple time # let rec length lst= ⇒ n I Exponential time O(2 ) match lst with I increment input = double time ⇒ |[] ->0 |x::xs ->1+ length xs;;

Ozye˘ginUniversity¨ — CS 321 Programming Languages 28 Ozye˘ginUniversity¨ — CS 321 Programming Languages 29 Quadratic time Comparison

I Each step of the recursion takes time proportional to input I Each step of the recursion makes only one recursive call. poorRev[1;2;3]= I List example: (poorRev[2;3])@[1]= let rec poorRev lst= ((poorRev[3])@[2])@[1]= match lst with (((poorRev[])@[3])@[2])@[1]= |[] ->[] (([]@[3])@[2])@[1]= |x::xs -> poorRev xs@[x];; ([3]@[2])@[1]= (* append is linear *) (* Compare poorRev to the function below *) (3::([]@[2]))@[1]= let rec rev_aux lst acc= [3;2]@[1]= match lst with 3::([2]@[1])= |[] -> acc 3::(2::([]@[1]))= |x::xs -> rev_aux xs(x::acc);; [3;2;1]

let rev lst= rev_aux lst[];;

Ozye˘ginUniversity¨ — CS 321 Programming Languages 30 Ozye˘ginUniversity¨ — CS 321 Programming Languages 31

Comparison Exponential time

rev[1;2;3]= I Hideous running times on input of any size rev_aux[1;2;3][]= I Each step of recursion takes constant time rev_aux[2;3][1]= rev_aux[3][2;1]= I Each recursion makes two recursive calls rev_aux[][3;2;1]= I Easy to write naive code that is exponential for functions that [3;2;1] can be linear

Ozye˘ginUniversity¨ — CS 321 Programming Languages 32 Ozye˘ginUniversity¨ — CS 321 Programming Languages 33 Exponential time

let rec naiveFib n= matchn with |0 ->1 |1 ->1 |_ -> naiveFib(n-1)+ naiveFib(n-2);;

let fib n= let rec tailFib n nm1 nm2= ifn=0 then nm2 else ifn=1 then nm1 else tailFib(n-1)(nm1+nm2) nm1 in tailFib n11;; Experiment Run the two versions with various big inputs. (e.g. 40) What difference do you see? Ozye˘gin¨ University — CS 321 Programming Languages 34