Polymorphism and Type Inference

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Liam O’Connor CSE, UNSW (and data61) Term 3 2019

1 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Where we’re at

Syntax Foundations X Concrete/Abstract Syntax, Ambiguity, HOAS, Binding, Variables, Substitution Semantics Foundations X Static Semantics, Dynamic Semantics (Small-Step/Big-Step), Abstract Machines, Environments (Assignment 1) Features Algebraic Data Types X Polymorphism Polymorphic Type Inference( Assignment 2) Overloading Subtyping Modules Concurrency

2 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

A Swap Function

Consider the humble swap function in Haskell:

swap :: (t1, t2) → (t2, t1) swap (a, b) = (b, a)

In our MinHS with algebraic data types from last lecture, we can’t deﬁne this function.

3 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Monomorphic In MinHS, we’re stuck copy-pasting our function over and over for every diﬀerent type we want to use it with:

recfun swap1 :: ((Int × Bool) → (Bool × Int)) p = (snd p, fst p)

recfun swap2 :: ((Bool × Int) → (Int × Bool)) p = (snd p, fst p)

recfun swap3 :: ((Bool × Bool) → (Bool × Bool)) p = (snd p, fst p) ···

This is an acceptable state of aﬀairs for some domain-speciﬁc languages, but not for general purpose programming.

4 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Solutions

We want some way to specify that we don’t care what the types of the tuple elements are.

swap :: (∀a b. (a × b) → (b × a))

This is called parametric polymorphism (or just polymorphism in functional programming circles). In Java and some other languages, this is called generics and polymorphism refers to something else. Don’t be confused.

5 In MinHS, we will write using type expressions like so: (the literature usesΛ)

swap = type a. type b. recfun swap :: (a × b) → (b × a) p = (snd p, fst p)

2 Type application is the ability to instantiate polymorphic functions to speciﬁc types. In MinHS, we use @ signs.

swap@Int@Bool (3, True)

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

How it works There are two main components to parametric polymorphism: 1 Type abstraction is the ability to deﬁne functions regardless of speciﬁc types (like the swap example before).

6 2 Type application is the ability to instantiate polymorphic functions to speciﬁc types. In MinHS, we use @ signs.

swap@Int@Bool (3, True)

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

How it works There are two main components to parametric polymorphism: 1 Type abstraction is the ability to deﬁne functions regardless of speciﬁc types (like the swap example before).In MinHS, we will write using type expressions like so: (the literature usesΛ)

swap = type a. type b. recfun swap :: (a × b) → (b × a) p = (snd p, fst p)

7 In MinHS, we use @ signs.

swap@Int@Bool (3, True)

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

swap = type a. type b. recfun swap :: (a × b) → (b × a) p = (snd p, fst p)

2 Type application is the ability to instantiate polymorphic functions to speciﬁc types.

8 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

swap = type a. type b. recfun swap :: (a × b) → (b × a) p = (snd p, fst p)

2 Type application is the ability to instantiate polymorphic functions to speciﬁc types. In MinHS, we use @ signs.

swap@Int@Bool (3, True)

9 Example (Identity Function) (type a. recfun f :: (a → a) x = x)@Int 3 7→ (recfun f :: (Int → Int) x = x) 3 7→ 3

This means that type expressions can be thought of as functions from types to values.

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Analogies

The reason they’re called type abstraction and application is that they behave analogously to λ-calculus. We have a β-reduction principle, but for types:

(type a. e)@τ 7→β (e[a := τ])

10 7→ (recfun f :: (Int → Int) x = x) 3 7→ 3

This means that type expressions can be thought of as functions from types to values.

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Analogies

The reason they’re called type abstraction and application is that they behave analogously to λ-calculus. We have a β-reduction principle, but for types:

(type a. e)@τ 7→β (e[a := τ])

Example (Identity Function) (type a. recfun f :: (a → a) x = x)@Int 3

11 7→ 3

This means that type expressions can be thought of as functions from types to values.

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Analogies

The reason they’re called type abstraction and application is that they behave analogously to λ-calculus. We have a β-reduction principle, but for types:

(type a. e)@τ 7→β (e[a := τ])

Example (Identity Function) (type a. recfun f :: (a → a) x = x)@Int 3 7→ (recfun f :: (Int → Int) x = x) 3

12 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Analogies

The reason they’re called type abstraction and application is that they behave analogously to λ-calculus. We have a β-reduction principle, but for types:

(type a. e)@τ 7→β (e[a := τ])

Example (Identity Function) (type a. recfun f :: (a → a) x = x)@Int 3 7→ (recfun f :: (Int → Int) x = x) 3 7→ 3

This means that type expressions can be thought of as functions from types to values.

13 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Type Variables

What is the type of this?

(type a. recfun f :: (a → a) x = x)

14 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Type Variables

What is the type of this?

(type a. recfun f :: (a → a) x = x)

∀a. a → a

Types can mention type variables now1.

1Technically, they already could with recursive types. 15 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Type Variables

What is the type of this?

(type a. recfun f :: (a → a) x = x)

∀a. a → a

Types can mention type variables now1.

If id : ∀a.a → a, what is the type of id@Int?

1Technically, they already could with recursive types. 16 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Type Variables

What is the type of this?

(type a. recfun f :: (a → a) x = x)

∀a. a → a

Types can mention type variables now1.

If id : ∀a.a → a, what is the type of id@Int?

(a → a)[a := Int] = (Int → Int)

1Technically, they already could with recursive types. 17 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Typing Rules Sketch

We would like rules that look something like this:

Γ ` e : τ Γ ` type a. e : ∀a. τ

Γ ` e : ∀a. τ Γ ` e@ρ : τ[a := ρ] But these rules don’t account for what type variables are available or in scope.

18 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Type Wellformedness

With variables in the picture, we need to check our types to make sure that they only refer to well-scoped variables.

t bound ∈ ∆ ∆ ` t ok ∆ ` Int ok ∆ ` Bool ok

∆ ` τ1 ok ∆ ` τ2 ok ∆ ` τ1 ok ∆ ` τ2 ok

∆ ` τ1 → τ2 ok ∆ ` τ1 × τ2 ok (etc.)

∆, a bound ` τ ok ∆ ` ∀a. τ ok

19 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Typing Rules, Properly

We add a second context of type variables that are bound.

a bound, ∆;Γ ` e : τ ∆;Γ ` type a. e : ∀a. τ

∆;Γ ` e : ∀a. τ ∆ ` ρ ok ∆;Γ ` e@ρ : τ[a := ρ]

(the other typing rules just pass∆ through)

20 Then we apply our β-reduction principle:

(type a. e)@τ 7→M e[a := τ]

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Dynamic Semantics

First we evaluate the LHS of a type application as much as possible: 0 e 7→M e 0 e@τ 7→M e @τ

21 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Dynamic Semantics

First we evaluate the LHS of a type application as much as possible: 0 e 7→M e 0 e@τ 7→M e @τ Then we apply our β-reduction principle:

(type a. e)@τ 7→M e[a := τ]

22 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Curry-Howard

Previously we noted the correspondence between types and logic: ×∧ + ∨ →⇒ 1 > 0 ⊥ ∀ ?

23 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Curry-Howard

Previously we noted the correspondence between types and logic: ×∧ + ∨ →⇒ 1 > 0 ⊥ ∀∀

24 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Curry-Howard

The type quantifier ∀ corresponds to a universal quantifier ∀, but it is not the same as the ∀ from first-order logic. What’s the difference?

25 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Curry-Howard The type quantifier ∀ corresponds to a universal quantifier ∀, but it is not the same as the ∀ from first-order logic. What’s the difference? First-order logic quantifiers range over a set of individuals or values, for example the natural numbers:

∀x. x + 1 > x

These quantiﬁers range over propositions (types) themselves. It is analogous to second-order logic, not ﬁrst-order:

∀A. ∀B. A ∧ B ⇒ B ∧ A ∀A. ∀B. A × B → B × A

The ﬁrst-order quantiﬁer has a type-theoretic analogue too (type indices), but this is not nearly as common as polymorphism.

26 Generality A type τ is more general than a type ρ, often written ρ v τ, if type variables in τ can be instantiated to give the type ρ.

Example (Functions)

Int → Int v ∀z. z → z v ∀x y. x → y v ∀a. a

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Generality

27 Example (Functions)

Int → Int v ∀z. z → z v ∀x y. x → y v ∀a. a

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Generality

If we need a function of type Int → Int, a polymorphic function of type ∀a. a → a will do just ﬁne, we can just instantiate the type variable to Int. But the reverse is not true. This gives rise to an ordering. Generality A type τ is more general than a type ρ, often written ρ v τ, if type variables in τ can be instantiated to give the type ρ.

28 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Generality

Example (Functions)

Int → Int v ∀z. z → z v ∀x y. x → y v ∀a. a

29 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Implementation Strategies

Our simple dynamic semantics belies a complex implementation headache. While we can easily deﬁne functions that operate uniformly on multiple types, when this is compiled to machine code the results may diﬀer depending on the size of the type in question. There are two main approaches to solve this problem.

30 For example, if we deﬁned our polymorphic swap function:

swap = type a. type b. recfun swap :: (a × b) → (b × a) p = (snd p, fst p)

Then a type application like swap@Int@Bool would be replaced statically by the compiler with the monomorphic version:

swapIB = recfun swap :: (Int × Bool) → (Bool × Int) p = (snd p, fst p)

A new copy is made for each unique type application.

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Template Instantiation Key Idea Automatically generate a monomorphic copy of each polymorphic functions based on the types applied to it.

31 Then a type application like swap@Int@Bool would be replaced statically by the compiler with the monomorphic version:

swapIB = recfun swap :: (Int × Bool) → (Bool × Int) p = (snd p, fst p)

A new copy is made for each unique type application.

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Template Instantiation Key Idea Automatically generate a monomorphic copy of each polymorphic functions based on the types applied to it.

For example, if we deﬁned our polymorphic swap function:

swap = type a. type b. recfun swap :: (a × b) → (b × a) p = (snd p, fst p)

32 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Template Instantiation Key Idea Automatically generate a monomorphic copy of each polymorphic functions based on the types applied to it.

For example, if we deﬁned our polymorphic swap function:

swap = type a. type b. recfun swap :: (a × b) → (b × a) p = (snd p, fst p)

Then a type application like swap@Int@Bool would be replaced statically by the compiler with the monomorphic version:

swapIB = recfun swap :: (Int × Bool) → (Bool × Int) p = (snd p, fst p)

A new copy is made for each unique type application. 33 This approach has a number of advantages: 1 Little to no run-time cost 2 Simple mental model 3 Allows for custom specialisations (e.g. list of booleans into bit-vectors) 4 Easy to implement However the downsides are just as numerous: 1 Large binary size if many instantiations are used 2 This can lead to long compilation times 3 Restricts the type system to statically instantiated type variables. Languages that use Template Instantiation: Rust, C++, Cogent, some ML dialects

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Evaluating Template Instatiation

34 However the downsides are just as numerous: 1 Large binary size if many instantiations are used 2 This can lead to long compilation times 3 Restricts the type system to statically instantiated type variables. Languages that use Template Instantiation: Rust, C++, Cogent, some ML dialects

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

35 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Evaluating Template Instatiation This approach has a number of advantages: 1 Little to no run-time cost 2 Simple mental model 3 Allows for custom specialisations (e.g. list of booleans into bit-vectors) 4 Easy to implement However the downsides are just as numerous: 1 Large binary size if many instantiations are used 2 This can lead to long compilation times 3 Restricts the type system to statically instantiated type variables. Languages that use Template Instantiation: Rust, C++, Cogent, some ML dialects

36 This describes a list of matrices of increasing dimensionality, e.g:

Step 1 (Step [1, 2] (Step [[1, 2], [3, 4]] Epsilon)) :: Dims Int

We can write a sum function like this: sumDims :: ∀a. (a → Int) → Dims a → Int sumDims f Epsilon = 0 sumDims f (Step a t) = (f a) + sumDims (sum f ) t

How many diﬀerent instantiations of the type variable a are there? We’d have to run the program to ﬁnd out.

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Polymorphic Recursion

Consider the following Haskell data type:

data Dims a = Step a (Dims [a]) | Epsilon

37 We can write a sum function like this: sumDims :: ∀a. (a → Int) → Dims a → Int sumDims f Epsilon = 0 sumDims f (Step a t) = (f a) + sumDims (sum f ) t

How many diﬀerent instantiations of the type variable a are there? We’d have to run the program to ﬁnd out.

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Polymorphic Recursion

Consider the following Haskell data type:

data Dims a = Step a (Dims [a]) | Epsilon

This describes a list of matrices of increasing dimensionality, e.g:

Step 1 (Step [1, 2] (Step [[1, 2], [3, 4]] Epsilon)) :: Dims Int

38 How many diﬀerent instantiations of the type variable a are there? We’d have to run the program to ﬁnd out.

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Polymorphic Recursion

Consider the following Haskell data type:

data Dims a = Step a (Dims [a]) | Epsilon

This describes a list of matrices of increasing dimensionality, e.g:

Step 1 (Step [1, 2] (Step [[1, 2], [3, 4]] Epsilon)) :: Dims Int

We can write a sum function like this: sumDims :: ∀a. (a → Int) → Dims a → Int sumDims f Epsilon = 0 sumDims f (Step a t) = (f a) + sumDims (sum f ) t

39 We’d have to run the program to ﬁnd out.

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Polymorphic Recursion

Consider the following Haskell data type:

data Dims a = Step a (Dims [a]) | Epsilon

This describes a list of matrices of increasing dimensionality, e.g:

Step 1 (Step [1, 2] (Step [[1, 2], [3, 4]] Epsilon)) :: Dims Int

We can write a sum function like this: sumDims :: ∀a. (a → Int) → Dims a → Int sumDims f Epsilon = 0 sumDims f (Step a t) = (f a) + sumDims (sum f ) t

How many diﬀerent instantiations of the type variable a are there?

40 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Polymorphic Recursion

Consider the following Haskell data type:

data Dims a = Step a (Dims [a]) | Epsilon

This describes a list of matrices of increasing dimensionality, e.g:

Step 1 (Step [1, 2] (Step [[1, 2], [3, 4]] Epsilon)) :: Dims Int

We can write a sum function like this: sumDims :: ∀a. (a → Int) → Dims a → Int sumDims f Epsilon = 0 sumDims f (Step a t) = (f a) + sumDims (sum f ) t

How many diﬀerent instantiations of the type variable a are there? We’d have to run the program to ﬁnd out.

41 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

HM Types

Automatically generating a copy for each instantiation is great but can’t handle all polymorphic programs. In practice a statically determined subset can be carved out by restricting what sort of programs can be written: 1 Only allow ∀ quantiﬁers on the outermost part of a type declaration (not inside functions or type constructors). 2 Recursive functions cannot call themselves with diﬀerent type parameters. This restriction is sometimes called Hindley-Milner polymorphism. This is also the subset for which type inference is both complete and tractable.

42 Typically this is done by boxing each type. That is, all data types are represented as a pointer to a data structure on the heap. If everything is a pointer, then all values use exactly 32 (or 64) bits of stack space. The extra indirection has a run-time penalty, and it can make garbage collection more necessary, but it results in smaller binaries and unrestricted polymorphism. Languages that use boxing: Haskell, Java, C], OCaml

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Boxing

An alternative to our copy-paste-heavy template instantiation approach is to make all types represented the same way. Thus, a polymorphic function only requires one function in the generated code.

43 The extra indirection has a run-time penalty, and it can make garbage collection more necessary, but it results in smaller binaries and unrestricted polymorphism. Languages that use boxing: Haskell, Java, C], OCaml

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Boxing

44 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Boxing

An alternative to our copy-paste-heavy template instantiation approach is to make all types represented the same way. Thus, a polymorphic function only requires one function in the generated code. Typically this is done by boxing each type. That is, all data types are represented as a pointer to a data structure on the heap. If everything is a pointer, then all values use exactly 32 (or 64) bits of stack space. The extra indirection has a run-time penalty, and it can make garbage collection more necessary, but it results in smaller binaries and unrestricted polymorphism. Languages that use boxing: Haskell, Java, C], OCaml

45 How about this type?

∀a. a → a Polymorphic type signatures constrain implementations.

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Constraining Implementations

How many possible implementations are there of a function of the following type? Int → Int

46 Polymorphic type signatures constrain implementations.

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Constraining Implementations

How many possible implementations are there of a function of the following type? Int → Int How about this type?

∀a. a → a

47 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Constraining Implementations

How many possible implementations are there of a function of the following type? Int → Int How about this type?

∀a. a → a Polymorphic type signatures constrain implementations.

48 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Parametricity

Deﬁnition The principle of parametricity states that the result of polymorphic functions cannot depend on values of an abstracted type. More formally, suppose I have a polymorphic function g that takes a type parameter. If run any arbitrary function f : τ → τ on some values of type τ, then run the function g@τ on the result, that will give the same results as running g@τ ﬁrst, then f .

Example

foo :: ∀a. [a] → [a]

49 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Parametricity Deﬁnition The principle of parametricity states that the result of polymorphic functions cannot depend on values of an abstracted type. More formally, suppose I have a polymorphic function g that takes a type parameter. If run any arbitrary function f : τ → τ on some values of type τ, then run the function g@τ on the result, that will give the same results as running g@τ ﬁrst, then f .

Example

foo :: ∀a. [a] → [a] We know that every element of the output occurs in the input. The parametricity theorem we get is, for all f :

foo ◦ (map f ) = (map f ) ◦ foo

50 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

More Examples

head :: ∀a. [a] → a

What’s the parametricity theorems?

51 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

More Examples

head :: ∀a. [a] → a

What’s the parametricity theorems? Example (Answer) For any f :

f (head `) = head (map f `)

52 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

More Examples

(++) :: ∀a. [a] → [a] → [a]

What’s the parametricity theorem?

53 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

More Examples

(++) :: ∀a. [a] → [a] → [a]

What’s the parametricity theorem? Example (Answer)

map f (a ++ b) = map f a ++ map f b

54 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

More Examples

concat :: ∀a. [[a]] → [a]

What’s the parametricity theorem?

55 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

More Examples

concat :: ∀a. [[a]] → [a]

What’s the parametricity theorem? Example (Answer)

map f (concat ls) = concat (map (map f ) ls)

56 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Higher Order Functions

ﬁlter :: ∀a. (a → Bool) → [a] → [a] What’s the parametricity theorem?

57 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Higher Order Functions

ﬁlter :: ∀a. (a → Bool) → [a] → [a] What’s the parametricity theorem? Example (Answer)

ﬁlter p (map f ls) = map f (ﬁlter (p ◦ f ) ls)

58 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Parametricity Theorems

Follow a similar structure. In fact it can be mechanically derived, using the relational parametricity framework invented by John C. Reynolds, and popularised by Wadler in the famous paper, “Theorems for Free!”2. Upshot: We can ask lambdabot on the Haskell IRC channel for these theorems.

2 https://people.mpi-sws.org/~dreyer/tor/papers/wadler.pdf 59 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Implicitly Typed MinHS

Explicitly typed languages are awkward to use3. Ideally, we’d like the compiler to determine the types for us. Example What is the type of this function?

recfun f x = fst x + 1

We want the compiler to infer the most general type.

3See Java 60 Remove recursive types, as we can’t infer types for them.

See whiteboard for why.

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Implicitly Typed MinHS

Start with our polymorphic MinHS, then: Remove type signatures from recfun, let, etc. Remove explicit type abstractions, and type applications (the @ operator). Keep ∀-quantiﬁed types.

61 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Implicitly Typed MinHS

Start with our polymorphic MinHS, then: Remove type signatures from recfun, let, etc. Remove explicit type abstractions, and type applications (the @ operator). Keep ∀-quantiﬁed types. Remove recursive types, as we can’t infer types for them.

See whiteboard for why.

62 Γ ` e1 : τ1 → τ2 Γ ` e2 : τ1 App Γ ` e1 e2 : τ2

Γ ` e1 : τ1 Γ ` e2 : τ2 ConjI Γ ` (Pair e1 e2): τ1 × τ2

Γ ` e1 : Bool Γ ` e2 : τ Γ ` e3 : τ If Γ ` (If e1 e2 e3): τ

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Typing Rules

x : τ ∈ Γ Var Γ ` x : τ

63 Γ ` e1 : τ1 Γ ` e2 : τ2 ConjI Γ ` (Pair e1 e2): τ1 × τ2

Γ ` e1 : Bool Γ ` e2 : τ Γ ` e3 : τ If Γ ` (If e1 e2 e3): τ

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Typing Rules

x : τ ∈ Γ Var Γ ` x : τ

Γ ` e1 : τ1 → τ2 Γ ` e2 : τ1 App Γ ` e1 e2 : τ2

64 Γ ` e1 : Bool Γ ` e2 : τ Γ ` e3 : τ If Γ ` (If e1 e2 e3): τ

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Typing Rules

x : τ ∈ Γ Var Γ ` x : τ

Γ ` e1 : τ1 → τ2 Γ ` e2 : τ1 App Γ ` e1 e2 : τ2

Γ ` e1 : τ1 Γ ` e2 : τ2 ConjI Γ ` (Pair e1 e2): τ1 × τ2

65 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Typing Rules

x : τ ∈ Γ Var Γ ` x : τ

Γ ` e1 : τ1 → τ2 Γ ` e2 : τ1 App Γ ` e1 e2 : τ2

Γ ` e1 : τ1 Γ ` e2 : τ2 ConjI Γ ` (Pair e1 e2): τ1 × τ2

Γ ` e1 : Bool Γ ` e2 : τ Γ ` e3 : τ If Γ ` (If e1 e2 e3): τ

66 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Primitive Operators

For convenience, we treat prim ops as functions, and place their types in the environment.

(+) : Int → Int → Int, Γ ` (App (App (+) (Num 2)) (Num 1)) : Int

67 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Functions

x : τ1, f : τ1 → τ2, Γ ` e : τ2 Func Γ ` (Recfun (f .x. e)) : τ1 → τ2

68 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Sum Types

Γ ` e : τ1 DisjI1 Γ ` InL e : τ1 + τ2

Γ ` e : τ2 DisjI2 Γ ` InR e : τ1 + τ2 Note that we allow the other side of the sum to be any type.

69 We can quantify over any variable that has not already been used.

Γ ` e : τ a ∈/ TV (Γ) All Γ ` e : ∀a. τ I

(Where TV (Γ) here is all type variables occurring free in the types of variables inΓ)

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Polymorphism

If we have a polymorphic type, we can instantiate it to any type:

Γ ` e : ∀a.τ All Γ ` e : τ[a := ρ] E

70 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Polymorphism

If we have a polymorphic type, we can instantiate it to any type:

Γ ` e : ∀a.τ All Γ ` e : τ[a := ρ] E

We can quantify over any variable that has not already been used.

Γ ` e : τ a ∈/ TV (Γ) All Γ ` e : ∀a. τ I

(Where TV (Γ) here is all type variables occurring free in the types of variables inΓ)

71 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

The Goal

We want an algorithm for type inference: With a clear input and output. Which terminates. Which is fully deterministic.

72 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Typing Rules

Γ ` e1 : τ1 Γ ` e2 : τ2

Γ ` (Pair e1 e2): τ1 × τ2 Can we use the existing typing rules as our algorithm?

infer :: Context → Expr → Type

73 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Typing Rules

Γ ` e1 : τ1 Γ ` e2 : τ2

Γ ` (Pair e1 e2): τ1 × τ2 Can we use the existing typing rules as our algorithm?

infer :: Context → Expr → Type

This approach can work for monomorphic types, but not polymorphic ones. Why not?

74 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

First Problem

Γ ` e : ∀a.τ All Γ ` e : τ[a := ρ] E The rule to add a ∀-quantiﬁer can always be applied: . . Γ ` (Num 5) : ∀a. ∀b. Int AllE Γ ` (Num 5) : ∀a. Int AllE Γ ` (Num 5) : Int

This makes the rules give rise to a non-deterministic algorithm – there are many possible rules for a given input. Furthermore, as it can always be applied, a depth-ﬁrst search strategy may end up attempting inﬁnite derivations.

75 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Another Problem

Γ ` e : ∀a.τ All Γ ` e : τ[a := ρ] E The above rule can be applied at any time to a polymorphic type, even if it would break later typing derivations:

Γ ` fst : ∀a. ∀b. (a × b) → a ··· Γ ` fst : (Bool × Bool) → Bool Γ ` (Pair 1 True) : (Int × Bool) Γ ` (Apply fst (Pair 1 True)) : ???

76 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Yet Another Problem

The rule for recfun mentions τ2 in both input and output positions.

x : τ1, f : τ1 → τ2, Γ ` e : τ2 Func Γ ` (Recfun (f .x. e)) : τ1 → τ2

In order to infer τ2 we must provide a context that includes τ2 — this is circular. Any guess we make for τ2 could be wrong.

77 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Solution

We allow types to include unknowns, also known as uniﬁcation variables or schematic variables. These are placeholders for types that we haven’t worked out yet. We shall use α, β etc. for these variables. Example (Int × α) → β is the type of a function from tuples where the left side of the tuple is Int, but no other details of the type have been determined yet.

As we encounter situations where two types should be equal, we unify the two types to determine what the unknown variables should be.

78 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Example

Γ ` fst : ∀a. ∀b. (a × b) → a ··· Γ ` fst : (α × β) → α Γ ` (Pair 1 True) : (Int × Bool) Γ ` (Apply fst (Pair 1 True)) : γ

79 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Example

Γ ` fst : ∀a. ∀b. (a × b) → a ··· Γ ` fst : (α × β) → α Γ ` (Pair 1 True) : (Int × Bool) Γ ` (Apply fst (Pair 1 True)) : γ

(α × β) → α ∼ (Int × Bool) → γ

80 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Example

Γ ` fst : ∀a. ∀b. (a × b) → a ··· Γ ` fst : (α × β) → α Γ ` (Pair 1 True) : (Int × Bool) Γ ` (Apply fst (Pair 1 True)) : γ

(α × β) → α ∼ (Int × Bool) → γ

[α := Int, β := Bool, γ := Int]

81 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Unification We call this substitution a unifier. Definition A substitution S to unification variables is a unifier of two types τ and ρ iff Sτ = Sρ. Furthermore, it is the most general unifier, or mgu, of τ and ρ if there is no other unifier S0 where Sτ v S0τ. We write τ ∼U ρ if U is the mgu of τ and ρ.

Example (Whiteboard) α × (α × α) ∼ β × γ (α × α) × β ∼ β × γ Int + α ∼ α + Bool (α × α) × α ∼ α × (α × α)

82 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Back to Type Inference

We will decompose the typing judgement to allow for an additional output — a substitution that contains all the uniﬁers we have found about unknowns so far. Inputs Expression, Context Outputs Type, Substitution We will write this as SΓ ` e : τ, to make clear how the original typing judgement may be reconstructed.

83 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Application, Elimination

U S1Γ ` e1 : τ1 S2S1Γ ` e2 : τ2 S2τ1 ∼ (τ2 → α) (α fresh) US2S1Γ ` (Apply e1 e2): Uα

(x : ∀a1. ∀a2.... ∀an. τ) ∈ Γ (α1 . . . αn fresh) Γ ` x : τ[a1 := α1, a2 := α2,..., an = αn]

Example (Whiteboard)

(fst : ∀a b. (a × b) → a) ` (Apply fst (Pair 1 2))

84 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Functions

U S(Γ, x : α1, f : α2) ` e : τ Sα2 ∼ (Sα1 → τ) (α1, α2 fresh) USΓ ` (Recfun (f .x. e)): U(Sα1 → τ)

Example (Whiteboard)

(Recfun (f .x. (Pair x x)))

(Recfun (f .x. (Apply f x)))

85 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Generalisation

In our typing rules, we could generalise a type to a polymorphic type by introducing a ∀ at any point in the typing derivation. We want to be able to restrict this to only occur in a syntax-directed way. Consider this example:

let f = (recfun f x = (x, x)) in (fst (f 4), fst (f True))

Where should generalisation happen?

86 This means that let expressions are now not just sugar for a function application, they actually play a vital role in the language’s syntax, as a place for generalisation to occur.

We deﬁne Gen(Γ, τ) = ∀(TV (τ) \ TV (Γ)). τ Then we have:

0 S1Γ ` e1 : τ S2(S1Γ, x : Gen(S1Γ, τ)) ` e2 : τ 0 S2S1Γ ` (Let e1(x. e2)): τ

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Let-generalisation

To make type inference tractable, we restrict generalisation to only occur when a binding is added to the context via a let expression.

87 We deﬁne Gen(Γ, τ) = ∀(TV (τ) \ TV (Γ)). τ Then we have:

0 S1Γ ` e1 : τ S2(S1Γ, x : Gen(S1Γ, τ)) ` e2 : τ 0 S2S1Γ ` (Let e1(x. e2)): τ

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Let-generalisation

To make type inference tractable, we restrict generalisation to only occur when a binding is added to the context via a let expression.

This means that let expressions are now not just sugar for a function application, they actually play a vital role in the language’s syntax, as a place for generalisation to occur.

88 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Let-generalisation

To make type inference tractable, we restrict generalisation to only occur when a binding is added to the context via a let expression.

This means that let expressions are now not just sugar for a function application, they actually play a vital role in the language’s syntax, as a place for generalisation to occur.

We deﬁne Gen(Γ, τ) = ∀(TV (τ) \ TV (Γ)). τ Then we have:

0 S1Γ ` e1 : τ S2(S1Γ, x : Gen(S1Γ, τ)) ` e2 : τ 0 S2S1Γ ` (Let e1(x. e2)): τ

89 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Summary

The rest of the rules are straightforward from their typing rule. We’ve specified Robin Milner’s algorithm W for type inference. Many other algorithms exist, for other kinds of type systems, including explicit constraint-based systems. This algorithm is restricted to the Hindley-Milner subset of decidable polymorphic instantiations, and requires that polymorphism is top-level — polymorphic functions are not first class. We still need an algorithm to compute the unifiers.

90 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Uniﬁcation

unify :: Type → Type → Maybe Unifier (where the Type arguments do not include any ∀ quantiﬁers and the Unifier returned is the mgu) We shall discuss cases for unify τ1 τ2

91 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Cases

Both type variables: τ1 = v1 and τ2 = v2:

v1 = v2 ⇒ empty uniﬁer

v1 6= v2 ⇒ [v1 := v2]

92 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Cases

Both primitive type constructors: τ1 = C1 and τ2 = C2:

C1 = C2 ⇒ empty uniﬁer

C1 6= C2 ⇒ no uniﬁer

93 1 Compute the mgu S of τ11 and τ21. 0 2 Compute the mgu S of Sτ12 and Sτ22. 3 Return S ∪ S0 (same for sum, function types)

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Cases

Both are product types τ1 = τ11 × τ12 and τ2 = τ21 × τ22.

94 0 2 Compute the mgu S of Sτ12 and Sτ22. 3 Return S ∪ S0 (same for sum, function types)

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Cases

Both are product types τ1 = τ11 × τ12 and τ2 = τ21 × τ22.

1 Compute the mgu S of τ11 and τ21.

95 3 Return S ∪ S0 (same for sum, function types)

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Cases

Both are product types τ1 = τ11 × τ12 and τ2 = τ21 × τ22.

1 Compute the mgu S of τ11 and τ21. 0 2 Compute the mgu S of Sτ12 and Sτ22.

96 (same for sum, function types)

Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Cases

Both are product types τ1 = τ11 × τ12 and τ2 = τ21 × τ22.

1 Compute the mgu S of τ11 and τ21. 0 2 Compute the mgu S of Sτ12 and Sτ22. 3 Return S ∪ S0

97 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Cases

Both are product types τ1 = τ11 × τ12 and τ2 = τ21 × τ22.

1 Compute the mgu S of τ11 and τ21. 0 2 Compute the mgu S of Sτ12 and Sτ22. 3 Return S ∪ S0 (same for sum, function types)

98 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Cases

One is a type variable v, the other is just any term t. v occurs in t ⇒ no uniﬁer otherwise ⇒ [v := t]

99 Motivation Polymorphism Implementation Parametricity Implicitly Typed MinHS Inference Algorithm Uniﬁcation

Done

Implementing this algorithm will be the focus of Assignment 2. Assignment 2 will be released tonight. You should allow plenty of time to tackle it. You will be given a generous deadline but it requires time to complete. Haskell-wise, this code will use a monad to track errors and the state needed to generate fresh uniﬁcation variables.

100