Foundations of Computer Science Lecture #9: Sequences, or Lazy Lists
Dr Amanda Prorok & Dr Anil Madhavapeddy 2019-2020 Slides
https://proroklab.org/teaching/FCS_LectureX.pdf
Posted online immediately after lecture. Warm-Up Warm-Up
Question 1: What is the type of this function? let cf y x = y;; Out: val cf : 'a -> 'b -> 'a =
Question 2: What does cf y return? It returns a constant function.
Question 3: We have the following: let add a b = a + b;; Use a partial application of add to define an increment function: In : let increment = ???
In : let increment = add 1;; Warm-Up Warm-Up What is the type of f? let f x y z = x z (y z) Step 1: analyze the right-hand side expression
function type (z) : 'a Step 2: what are the unknown types? return-type (y) : 'b return-type (x) : 'c Step 3: set those types. input-type (y) : 'a Step 4: infer the input types. input-type (x) : 'a -> 'b type (y) : 'a -> 'b Step 5: infer all types. type (x) : 'a -> 'b -> 'c type (z) : 'a let f x y z = x z (y z);; Step 6: infer function type. val f : ('a -> 'b -> 'c) -> ('a -> 'b) -> 'a -> 'c Warm-Up Warm-Up
Question 4: Is this function tail-recursive? Why?
let rec exists p = function | [] -> false | x::xs -> (p x) || exists p xs
It is…
let rec exists p = function | [] -> false | x::xs -> (p x) || ((exists[@ocaml.tailcall]) p xs) DataData Streams Streams - - Intro Intro
An example: perception-action loops (basic building block of autonomy)
perception
decision-making and control action interaction with the world while(true) get sensor data act upon sensor data repeat DataData Streams Streams - - Intro Intro
Sequential programs - examples include: “fully-defined” • Exhaustive search • search a book for a keyword • search a graph for the optimal path • Data processing • image processing (enhance / compress) • outlier removal / de-noise
Reactive programs - examples include: “event-triggered” • Control tasks “interactive” • flying a plane “closed-loop” • robot navigation (obstacle avoidance) • Resource allocation • computer processor • Mobility-on-Demand (e.g. Uber) IX FoundationsofComputerScience 97
AA Pipeline Pipeline APipeline
Producer Filter Filter Consumer → → ···→ →
Slide 901 Produce sequence of items
Filter sequence in stages
Consume results as needed
Lazy lists join the stages together
Two types of program can be distinguished. A sequential program accepts a problem to solve, processes for a while, and finally terminates with its result. A typical example is the huge numerical simulations that are run on supercomputers. Most of our ML functions also fit this model. At the other extreme are reactive programs, whose job is to interact with the environment. They communicate constantly during their operation and run for as long as is necessary. A typical example is the software that controls many modern aircraft. Reactive programs often consist of concurrent processes running at the same time and communicating with one another. Concurrency is too difficult to consider in this course, but wecanmodelsimple pipelines such as that shown above. The Producer represents one or more sources of data, which it outputs as a stream. The Filter stages convert the input stream to an output stream, perhaps consuming several input items to yield a single output item. The Consumer takes as many elements as necessary. The Consumer drives the pipeline: nothing is computed exceptinresponseto its demand for an additional datum. Execution of the Filter stages is interleaved as required for the computation to go through. The programmersetsupthedata dependencies but has no clear idea of what happens when. We have the illusion of concurrent computation. The Unix operating system provides similar ideas through its pipes that link processes together. In ML, we can model pipelines using lazy lists. IX FoundationsofComputerScience 98
LazyLazy Lists Lists — — or or Streams Streams Lazy Lists — or Streams
Lists of possibly INFINITE length
Slide 902 • elements computed upon demand
• avoids waste if there are many solutions
• infinite objects are a useful abstraction
In MLOCaml::implementlazinessby implement lazinessdelaying by delaying evaluation evaluationof the of the tail tail
In OCaml: ‘streams’ reserved for input/output channels, so we use term ‘sequences’
Lazy lists have practical uses. Some algorithms, like makingchange,canyield many solutions when only a few are required. Sometimes the original problem concerns infinite series: with lazy lists, we can pretend theyreallyexist! We are now dealing with infinite,oratleastunbounded,computations.Apo- tentially infinite source of data is processed one element at atime,upondemand. Such programs are harder to understand than terminating onesandhavemoreways of going wrong. Some purely functional languages, such as Haskell, use lazy evaluation every- where. Even the if-then-else construct can be a function, andalllistsarelazy.In ML, we can declare a type of lists such that evaluation of the tail does not occur until demanded. Delayed evaluation is weaker than lazy evaluation, but it is good enough for our purposes. The traditional word stream is reserved in ML parlance for input/output chan- nels. Let us call lazy lists sequences. LazyLazy Lists Lists in OCaml in ML
The type unit has one element: empty tuple ()
Uses: • Can appear in data-structures (e.g., unit-valued dictionary) • Can be the argument of a function • Can be the argument or result of a procedure (seen later in course)
Behaves as a tuple, is a constructor, and allowed in pattern matching:
let f () = … let f = function | () ->
Expression E not evaluated until the function is applied: fun () -> E
fun notation enables delayed evaluation! IX FoundationsofComputerScience 99
LazyLazy Lists Lists in OCamlin ML
Lazy Lists in ML
The emptytype tuple () 'aand seq its type= unit | Nil Delayed version| Cons of E ofis fn()=> 'a * (unitE -> 'a seq)
Slide 903 datatypelet'aseq=Nil head (Cons (x, _)) = x sequences |Consof 'a * (unit -> 'aseq); let tail (Cons (_, xf)) = xf ()
fun head (Cons(x,_)) = x; fun tail (Cons(_,xf)) = xf(); apply xf to () to evaluate
Cons(x,xf) has head x and tail function xf
The primitive ML type unit has one element, which is written ().Thiselement may be regarded as a 0-tuple, and unit as the nullary Cartesian product. (Think of the connection between multiplication and the number 1.) The empty tuple serves as a placeholder in situations where noinformationis required. It has several uses:
•Itmayappearinadatastructure.Forexample,aunit-valued dictionary represents a set of keys. •Itmaybetheargumentofafunction,whereitseffectistodelay evaluation.
•Itmaybetheargumentorresultofaprocedure.(SeeLect.12.) The empty tuple, like all tuples, is a constructor and is allowed in patterns:
fun f () = . . .
In particular, fn() => E is the function that takes an argument of type unit and returns the value of E as its result. Expression E is not evaluated until the function is called, even though the only possible argument is ().Thefunctionsimplydelays the evaluation of E. TheThe Infinite Infinite Sequence, Sequence, k, k, k+1, k+1, k+2, k+2, … …
let rec from k = Cons (k, fun () -> from (k + 1));; val from : int -> int seq =
let it = from 1;; val it : int seq = Cons (1,
let it = tail it;; val it : int seq = Cons (2,
tail it;; - : int seq = Cons (3,
Recall: let tail (Cons(_, xf)) = xf ();; force the evaluation > val tail : 'a seq -> 'a seq IX FoundationsofComputerScience 101
ConsumingConsuming a a Sequence Sequence
Consuming a Sequence
funlet get(0,xq)rec get n s = = [] if n = 0 then [] |get(n,Nil) =[] Slide 905 else force the list |get(n,Cons(x,xf))=x::get(n-1,xf()); match s with | Nil -> [] >valget=fn:int | Cons (x, xf) -> x *::' aseq->get (n - 1)'alist (xf ())
Get the first n elements as a list
xf() forces evaluation
The function get converts a sequence to a list. It takes the first n elements; it takes all of them if n < 0, which can terminate only if the sequence is finite. In the third line of get,theexpressionxf() calls the tail function, demanding evaluation of the next element. This operation is called forcing the list. SampleSample Evaluation Evaluation
get 2 (from 6) ⇒ get 2 (Cons (6, fun () -> from (6 + 1))) ⇒ 6 :: get 1 (from (6 + 1)) ⇒ 6 :: get 1 (Cons (7, fun () -> from (7 + 1)))
⇒ 6 :: 7 :: get 0 (from (7 + 1))
⇒ 6 :: 7 :: get 0 (Cons (8, fun () -> from (8 + 1))) ⇒ 6 :: 7 :: [] ⇒ [6; 7] IX FoundationsofComputerScience 103
JoiningJoining Two Two Sequences Sequences
let rec appendq xq yq = match xq withJoining Two Sequences | Nil -> yq | Cons (x, xf) -> fun appendq Cons (x, (Nil, fun () yq) -> appendq = yq (xf ()) yq) |appendq(Cons(x,xf),yq)= Slide 907 Cons(x, fn()=> appendq(xf(), yq));
A fair alternative. . . let rec interleave xq yq = fun interleavematch xq with (Nil, yq) = yq |interleave(Cons(x,xf),yq)= | Nil -> yq | ConsCons(x, (x, xf)fn()=> -> interleave(yq, xf())); Cons (x, fun () -> interleave yq (xf ()))
Most list functions and functionals have analogues on sequences, but strange things can happen. Can an infinite list be reversed? Function appendq is precisely the same idea as append (Lect. 3): it concate- nates two sequences. If the first argument is infinite, then appendq never gets to its second argument, which is lost. Concatenation of infinite sequences is not terribly interesting. The function interleave avoids this problem by exchanging the two arguments in each recursive call. It combines the two lazy lists, losingnoelements.Interleav- ing is the right way to combine two potentially infinite information sources into one. In both function declarations, observe that each xf() is enclosed within a fn()=>. . . .Eachforce is enclosed within a delay.Thispracticemakesthe functions lazy. A force not enclosed in a delay, as in get above, runs the risk of evaluating the sequence in full. IX FoundationsofComputerScience 104 IX FoundationsofComputerScience 104
Functionals for Lazy Lists FunctionalsFunctionals for for Lazy Lazy Lists Lists Functionals for Lazy Lists filtering let rec filterq p = function fun filterq p Nil = Nil | Nil -> Nil filtering |filterqp(Cons(x,xf))=| Cons (x, xf) -> Slide 908 fun filterq p Nil = Nil ifif p x x then |filterqp(Cons(x,xf))= Cons (x, fun () -> filterq p (xf ())) Slide 908 then Cons(x, fn()=>filterq p (xf())) elseif p x else filterq filterq p (xf p ()) (xf()); then Cons(x, fn()=>filterqWhat p (xf())) happens here? else filterq p (xf()); The infinite sequence x, f (x), f ( f (x)),. . .
fun iteratesThe infinite f x sequence= x, f (x), f ( f (x)),. . . Cons(x, fn()=> iterates f (f x)); fun iterateslet rec iterates f x = f x = Cons(x,Cons (x,fn()=> fun () iterates-> iterates f (ff (f x)); x))
val filterq : ('a -> bool) -> 'a seq -> 'a seq =
Example: val filterq : ('a -> bool) -> 'a seq -> 'a seq val iterates : ('a -> 'a) -> 'a -> 'a seq
> let myseq = iterates (fun x -> x + 1) 1;; val myseq : int seq = Cons (1,
> filterq (fun x -> x = 0) myseq;; …… ReusingFunctionals Functionals for Lazyfor Lazy Lists Lists
Same Examples, but with no new functions:
> succ;; - : int -> int =
> let myseq = iterates succ 1;; val myseq : int seq = Cons (1,
Example: val filterq : ('a -> bool) -> 'a seq -> 'a seq val iterates : ('a -> 'a) -> 'a -> 'a seq val get : int -> 'a seq -> 'a list
> val myseq = iterates (fun x -> x + 1) 1;; val myseq : int seq Cons (1,
NumericalNumerical Computations Computations on on Infinite Infinite Sequences Sequences NumericalNumerical Computations Computations on on Infinite Infinite Sequences Sequences find sqrt(a) xn
funfunlet nextnext next a a a x x x= = =(a (a/x (a/x /. x + ++. x) x) x) / / 2.0;/. 2.0; 2.0
CloseClose enough? enough? funlet within rec within (eps:real) eps = (Cons(x,xf))function = SlideSlide 909 909 fun| withinCons (x, (eps:real) xf) -> (Cons(x,xf)) = let val Cons(y,yf) = xf() let matchval xf Cons(y,yf) () with = xf() in if abs(x-y) <= eps then y in | Consif abs(x-y) (y, yf) -> <= eps then y else else if withinabs_float within eps eps (x (Cons(y,yf)) (Cons(y,yf))-. y) <= eps then y end; end; else within eps (Cons (y, yf)) x0 : initial guess SquareSquare Roots! Roots!
funfunlet root root root a a = =a within= within within 1E~6 1E~6 1e-6 (iterates (iterates(iterates (next(next (next a) a) a) 1.0) 1.0) 1.0)
epsilon sequence
> root 3.0;; - : float = 1.73205080756887719 TheTheNewton-RaphsonNewton-Raphson method methodisis widely widely used used for for computing computing square square roots. roots. The The infinite series x , (a/x x )/2, . . . converges rapidly to √a.Theinitialap- infinite series x00, (a/x00 x00)/2, . . . converges rapidly to √a.Theinitialap- proximation, x ,istypicallyretrievedfromatable,andisaccurateenough++ that proximation, x00,istypicallyretrievedfromatable,andisaccurateenoughthat onlyonly a a few few iterations iterations of of the the method method are are necessary. necessary. Calling Callingiteratesiterates (next (next a) a) x0x0generatesgenerates the theinfiniteinfinite series seriesofof approximations approximations to to the the square square root root of ofaausingusing thethe Newton-Raphson Newton-Raphson method. method. To To compute compute√√2,2, the the resulting resulting series series begins begins 1, 1, 1.5, 1.5, 1.41667,1.41667, 1.4142157, 1.4142157, 1.414213562 1.414213562 ...... , , and and this this last last figure figure is isalreadyalready accurate accurate to to 10 10 significantsignificant digits! digits! FunctionFunctionwithinwithinsearchessearches down down the the lazy lazy list list for for two two points points whose whose difference difference isis less less than thanepseps.Itteststheirabsolutedifference.Relativedifferencea.Itteststheirabsolutedifference.Relativedifferenceandnd other other ‘close‘close enough’ enough’ tests tests can can be be coded. coded. Such Such components components can can be be use usedtoimplementdtoimplement otherother numerical numerical functions functions directly directly as as functions functions over over seque sequences.nces. The The point point is is to to buildbuild programs programs from from small, small, interchangeable interchangeable parts. parts. FunctionFunctionrootrootusesuseswithinwithin,,iteratesiteratesandandnextnexttoto apply apply Newton-Raphson Newton-Raphson with a tolerance of 10 66and a (poor) initial approximation of 1.0. with a tolerance of 10−− and a (poor) initial approximation of 1.0. ThisThis treatment treatment of of numerical numerical computation computation has has received received some some a attentionttention in in the the researchresearch literature; literature; a a recurring recurring example example is isRichardsonRichardson extrapolation extrapolation[3,[3, 4]. 4]. NumericalNumerical Computations Computations on on Infinite Infinite Sequences Sequences
Aside: Newton-Raphson Method
Series is: So if we want to find sqrt(k) we use:
x2 = k f(x) = x2 − k
f′( x) = 2x