Towards Size-Dependent Types for Array Programming

Troels Henriksen Martin Elsman DIKU DIKU University of Copenhagen University of Copenhagen Copenhagen, Denmark Copenhagen, Denmark [email protected] [email protected] Abstract 1 Introduction We present a type system for expressing size constraints on Array programming often involves functions whose argu- array types in an ML-style type system. The goal is to detect ments must satisfy certain shape constraints. For example, shape mismatches at compile-time, while being simpler than when multiplying two matrices, the column count of the full dependent types. The main restrictions is that the only former must match the row count of the latter. Most of to- terms that can occur in types are array sizes, and syntactically day’s programming languages do not support compile-time they must be variables or constants. For those programs checking of such constraints, instead deferring them to run- where this is not sufficient, we support a form of existential time. As with all dynamic type checking, this can result in types, with the type system automatically managing the frustrating crashes. requisite book-keeping. We formalise a large subset of the Dependently typed languages allow us to encode the sizes type system in a small core language, which we prove sound. of arrays and have the type checker enforce the shape con- We also present an integration of the type system in the straints. From the perspective of array programming, current high-performance parallel functional language Futhark, and dependently typed languages have two problems: show on a collection of 44 representative programs that the 1. They are typically not very fast, where array programs restrictions in the type system are not too problematic in are often performance-critical. practice. 2. Full dependent types are complicated, and array pro- grammers may not need or want all of their capabili- CCS Concepts: • Computing methodologies → Parallel ties. In particular, working with existential sizes, as pro- programming languages. duced by operations such as filtering, requires deeper understanding of type theory. Keywords: type systems, parallel programming, functional programming In this paper, we investigate the foundations of a type system that supports size-dependent types. Our goal is to ACM Reference Format: construct a programming language in the ML family that Troels Henriksen and Martin Elsman. 2021. Towards Size-Dependent allows a “conventional” style of programming, while still Types for Array Programming. In Proceedings of the 7th ACM SIG- verifying shape constraints statically. We have the following PLAN International Workshop on Libraries, Languages and Compilers concrete design goals for size-dependent types: for Array Programming (ARRAY ’21), June 21, 2021, Virtual, Canada. 1. It must be possible to write functions that operate ACM, New York, NY, USA, 14 pages. https://doi.org/10.1145/3460944. 3464310 on multi-dimensional arrays, where we can express equality constraints on the dimensions of these arrays. When a function is applied, the type checker must verify that the arguments satisfy the constraints. Permission to make digital or hard copies of all or part of this work for 2. Any book-keeping involved in handling existential personal or classroom use is granted without fee provided that copies sizes should be done automatically. are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights 3. Multidimensional arrays should be handled as “ar- for components of this work owned by others than the author(s) must rays of arrays”, while still guaranteeing that all arrays be honored. Abstracting with credit is permitted. To copy otherwise, or are regular.1. This permits parametric polymorphism, republish, to post on servers or to redistribute to lists, requires prior specific which allows functions such as map to have their con- permission and/or a fee. Request permissions from [email protected]. ventional behaviour. ARRAY ’21, June 21, 2021, Virtual, Canada © 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM. 1A regular multidimensional array is one where all rows have the same ACM ISBN 978-1-4503-8466-7/21/06...$15.00 shape. These are sometimes also called rectangular arrays, and are in oppo- https://doi.org/10.1145/3460944.3464310 sition to jagged arrays, which we do not support. ARRAY ’21, June 21, 2021, Virtual, Canada Troels Henriksen and Martin Elsman

4. The type system should not impede a high-performance 푑 ::= Size sorts implementation of the language. In particular it must | 푛 constant not require complicated value representations or main- | 푥 variable tenance of complex runtime structures. 휏 ::= Basic types We value simplicity over completeness. As we shall see, | int integer we place limitations on the form of sizes which means some | bool boolean programs will require dynamic shape checking, although | (휏, 휏 ′) pair such checks will always be syntactically explicit size coer- | [푑]휏 array cions that are inserted by the programmer when needed. An | (푥 : 휏) → 휇 function intentional non-goal is that we are not trying to statically eliminate out-of-bounds indexing. 휇 ::= Return types We begin in Section2 by defining 퐹, a small core language | ∃푥.휇 existential size that we use to illustrate the mechanics of the type system. | 휏 basic type In Section3 we prove soundness for 퐹, which implies the absence of shape errors at runtime. 퐹 is a simplified lan- 휎 ::= Type schemes guage, and does not yet fulfill all our goals—in particular, | 휏 basic type it is monomorphic. However, Section5 demonstrates how | ★ abstract size type we have added size types to the data parallel programming language Futhark [13], which has full support for paramet- 푒 ::= Expressions ric polymorphism. This not only shows that the type sys- | 푛 constant integer tem, despite its simplicity, is expressive enough to handle | true | false constant boolean real programs, but also that it does not provide obstacles to | 푥 variable high-performance implementation. Section6 discusses our | 휆(푥 : 휏).푒 function experience with using size types in practice, in particular | 푒 푒 application whether they are flexible enough to handle a wide variety | [푒, ··· , 푒] array of parallel benchmark problems. Finally, Section7 discusses | iota 푒 index array how our approach relates to prior work, with a particular | 푒 [푒] array index focus on how it fits into the array language tradition. | (푒, 푒) pair | fst 푒 | snd 푒 projection 2 A Language with Size-Dependent Types | if 푒 then 푒 else 푒 conditional In this section, we present a language 퐹 with support for size | 푒 ⊲ 휏 dynamic type coercion types. The language is simplistic in a variety of ways. First of | let 푝 = 푒 in 푒 let all, it does not support type polymorphism, which we shall 푝 ::= 푥 treat later in Section4. Second, the language assumes that | [푥¯] 푥 : 휏 expressions are flattened so that expressions that allocate ex- istentially sized arrays are bound by explicit let-constructs. Figure 1. Grammar for 퐹. Thus, if we assume the availability of a type-monomorphic (but size-polymorphic) map-function and a function iota that takes an integer 푛 and returns an index-array of size 푛, we also assume that an expression Whereas the type of this entire expression will be ∃푥.[푥]int, 푥 map (휆푥.푥 + 1)(iota 5) where is an existentially bound size variable, inside the scope of the let-construct, it will be known that 푎 and the has been converted to the form result of the map application have the same size. let 푎 = iota 5 in map (휆푥.푥 + 1) 푎. 2.1 Types and Expressions This conversion simplifies the framework and can easily We assume a denumerably infinite set of program variables, be implemented in the frontend of a compiler. Although, ranged over by 푥, 푦, and 푓 . The grammars for size sorts (푑), the type system supports an implicit let-binding construct, basic types (휏), and return types (휇) are defined in Figure1. as used in the map example, an explicit version is available, Size sorts appear in array types of the form [푑]휏, which which eases type inference and which brings existential sizes denote arrays of size 푑 containing elements of type 휏.A into scope at expression level as int variables. In the explicit function type (푥 : 휏) → 휇 allows for 푥 to appear in 휇, which version of the let construct, the expression becomes is useful for expressing a dependency between a parameter let [푛] 푎 : [푛]int = iota 5 in map (휆푥.푥 + 푛) 푎. of type int and the size of an array in the result. Towards Size-Dependent Types for Array Programming ARRAY ’21, June 21, 2021, Virtual, Canada

For basic types on the form (푥 : 휏) → 휇, we consider 푥 Size sorts Γ ⊢ 푑 ok bound in 휇. For return types on the form ∃푥.휇, we consider 푥 bound in 휇. Respectively, basic types and return types Γ(푥) = int Γ(푥) = ★ are considered identical up to renaming of bound variables. Γ ⊢ 푛 ok Γ ⊢ 푥 ok Γ ⊢ 푥 ok Return types are also considered identical up to removal of 휏 bound existential variables that do not occur free in the body. Types Γ ⊢ ok Contexts, ranged over by Γ, map program variables to type schemes, which, in the simple setting that lacks parametric Γ ⊢ int ok Γ ⊢ bool ok size- and type-polymorphism, amounts to being a type 휏 or a type scheme of the form ★, which denote an abstract size. Γ ⊢ 푑 ok Γ ⊢ 휏 ok Γ ⊢ 휏 ok Γ ⊢ 휏 ′ ok 푥 푥 , , 푥 휎 휎 , , 휎 푥 휎 ′ When ¯ = 1 ··· 푛 and ¯ = 1 ··· 푛, we write ¯ : ¯ Γ ⊢ [푑]휏 ok Γ ⊢ (휏, 휏 ) ok for specifying the type assumption 푥1 : 휎1, ··· , 푥푛 : 휎푛. We shall also sometimes write 푥¯ : 푐¯, where 푐 is some basic type Γ ⊢ 휏 ok Γ ⊢ 휇 ok Γ, 푥 : ★ ⊢ 휇 ok scheme such as ★ or int, to denote the type assumption Γ ⊢ (푥 : 휏) → 휇 ok Γ ⊢ (푥 : int) → 휇 ok 푥1 : 푐, ··· , 푥푛 : 푐. We shall use similar notation for size assumptions and dynamic environments (defined later). Return types Γ ⊢ 휇 ok Notice that we assume that whenever we have a context , 푥 휏 푥 ∉ ( ) on the form Γ : , we have fv Γ . In the rules that Γ, 푥 : ★ ⊢ 휇 ok Γ ⊢ 휏 ok 휇 = 휏 follow, this property can most often be ensured by renaming Γ ⊢ ∃푥.휇 ok Γ ⊢ 휇 ok of bound variables (i.e., alpha renaming). 푥 푥 , , 푥 푥.휇 Whenever ¯ = 1 ··· 푛, we often write ∃¯ to denote Type schemes Γ ⊢ 휎 ok the return type ∃푥1. · · · ∃푥푛.휇 (similarly for universal size quantification.) Γ ⊢ 휏 ok 휎 = 휏 Type schemes of the form ★ play a special role. Vari- Γ ⊢ ★ ok Γ ⊢ 휎 ok ables bound to such type schemes have no dynamic rep- resentation. In fact, in the simple setting of 퐹, programmers have no way of referring to such abstract sizes even though Figure 2. Well-formedness of size sorts, basic types, return they form the basis for establishing static shape properties types, and type schemes. and static shape equalities of runtime values. For instance, type schemes of the form ★ are used for establishing, for instance, that a certain expression returns a value of return one that implicitly “opens” existentially bound size variables type ∃푥.([푥]int, [푥](bool, int)) for which inhabitants are (making them available only to meta-level reasoning), and pairs of equally sized arrays, where the first array contains one that explicitly extracts sizes for existentially bound size integers and the second array contains pairs of a boolean variables (bringing them into scope as variables of type int). and an integer. Finally, the language features a type coercion construct of Basic types (휏), return types (휇), and size sorts (푑) are well- the form 푒 ⊲ 휏, which allows the programmer to coerce an formed under assumptions Γ, written Γ ⊢ 휏 ok, Γ ⊢ 휇 ok, array value to be of a specific shape, checked dynamically. and Γ ⊢ 푑 ok, respectively, if the particular judgement can For simplicity, the rules for the dynamic semantics shall be derived using the rules in Figure2. A context Γ′ is well- deal only with succeeding evaluations, which means that formed under assumptions Γ, written Γ ⊢ Γ′ ok, if Γ ⊢ 휎 ok the type safety property that we shall prove only relates to for all 휎 in the range of Γ′. We shall write ⊢ Γ ok to mean programs for which a dynamic derivation exists. We consider Γ ⊢ Γ ok. When 휇 is some return type (or type), we write it future work to further distinguish between the possible Γ ⊢ 휇 ok to mean Γ ⊢ 휇 ok and ∀푥 ∈ fv(휇), Γ(푥) ≠ ★. dynamic effects and to establish a guarantee for termination. Expressions (푒) and patterns (푝) are defined according to the grammar in Figure1. 2.2 Type System The expression language has support for basic constants, Typing rules for expressions allow inferences among sen- including integers and booleans, lambda abstraction and tences on the form Γ ⊢ 푒 : 휇, which are read “under assump- function application, pairs and projections, and conditional tions Γ, the expression 푒 has return type 휇.” expressions. The language also has support for immediate The typing rules are shown in Figure3. In general, exis- arrays and it features an explicit index-space operation iota, tential quantification in return types is used for modeling which, given an integer argument 푛, creates an array with existential sizes. In rule t-let, existentially bound sizes are 푛 integers ranging from 0 to 푛 − 1. 퐹 also features array implicitly opened in the scope of the let construct and closed indexing, which fails dynamically if an index is out of bounds. again when forming the return type of the entire expression. Further, the language features two kinds of let constructs: In rule t-iotad and rule t-appd, the type system supports ARRAY ’21, June 21, 2021, Virtual, Canada Troels Henriksen and Martin Elsman

Expressions Γ ⊢ 푒 : 휇

Γ(푥) = 휏 [t-true] [t-false] [t-int] [t-var] Γ ⊢ true : bool Γ ⊢ false : bool Γ ⊢ 푛 : int Γ ⊢ 푥 : 휏

Γ ⊢ 푒 : ∃푥¯ .휏 Γ ⊢ 푒 : ∃푥¯ .휏 Γ ⊢ 푒 : 휏 ··· Γ ⊢ 푒 : 휏 1 1 1 2 2 2 [t-pair] 1 푛 [t-array] Γ ⊢ (푒1, 푒2) : ∃푥¯1푥¯2.(휏1, 휏2) Γ ⊢ [푒1, ··· , 푒푛] : [푛]휏

Γ ⊢ 푒 : ∃푥.¯ (휏 , 휏 ) Γ ⊢ 푒 : ∃푥.¯ (휏 , 휏 ) Γ ⊢ 푒 : ∃푥.¯ [푑]휏 Γ ⊢ 푒 : int 1 2 [t-fst] 1 2 [t-snd] 1 2 [t-index] Γ ⊢ fst 푒 : ∃푥.휏¯ 1 Γ ⊢ snd 푒 : ∃푥.휏¯ 2 Γ ⊢ 푒1 [푒2] : ∃푥.휏¯

Γ ⊢ 푒 : bool Γ ⊢ 푒 : 휇 Γ ⊢ 푒 : 휇 Γ ⊢ 푒 : int Γ ⊢ 푑 : int 1 2 [t-if] [t-iota] [t-iotad] Γ ⊢ if 푒 then 푒1 else 푒2 : 휇 Γ ⊢ iota 푒 : ∃푥.[푥]int Γ ⊢ iota 푑 : [푑]int

Γ ⊢ 푒 : 휏 ′ Γ ⊢ 휏 ok 휏 = 휏 ′{푑¯/푥¯} Γ, 푥 : 휏 ⊢ 푒 : 휇 Γ ⊢ 휏 ok [t-coerce] [t-lam] Γ ⊢ 푒 ⊲ 휏 : 휏 Γ ⊢ 휆(푥 : 휏).푒 : (푥 : 휏) → 휇

Γ ⊢ 푒 : (푥 : 휏) → 휇 Γ ⊢ 푒 ′ : 휏 Γ ⊢ 푒 : (푥 : int) → 휇 Γ ⊢ 푑 : int [t-app] [t-appd] Γ ⊢ 푒 푒 ′ : ∃푥.휇 Γ ⊢ 푒 푑 : 휇{푑/푥}

Γ ⊢ 푒 : ∃푥.휏¯ Γ, 푥¯ : ★¯, 푥 : 휏 ⊢ 푒 ′ : 휇 Γ ⊢ 푒 : ∃푥.휏¯ Γ ⊢ ∃푥.휏¯ ok Γ, 푥¯ : int, 푥 : 휏 ⊢ 푒 ′ : 휇 [t-let] [t-let-sz] Γ ⊢ let 푥 = 푒 in 푒 ′ : ∃푥푥.휇¯ Γ ⊢ let [푥¯] 푥 : 휏 = 푒 in 푒 ′ : ∃푥푥.휇¯

Figure 3. Typing rules for expressions. explicit size applications, which allows the programmer to 푣 ::= Values establish a connection between a computed value and the | 푛 | true | false basic constant size of an array. A dual feature of the type system is provided | ⟨푥, 푒, 휌⟩ closure by rule t-let-sz, which allows the programmer to extract | [푣, ··· , 푣] array the size of an existentially sized array and use the size in an | (푣, 푣) pair expression. Finally, rule t-coerce allows the programmer to specify a type for an expression, which will be checked 퐹 dynamically against the actual value. Notice that we allow Figure 4. Grammar for values in . only type coercions that change the static dimensions of an otherwise unchanged array type. Statically, this restriction 휌 ⊢ 푒 { 푣, which are read “The expression 푒 evaluates to 푣 is modelled by the existence of a substitution that maps size in the dynamic environment 휌”. variables 푥¯ in the type 휏 ′ into type descriptors 푑¯ in the type The rules make use of a set of dynamic size-matching rules 휏. A set of dynamic rules (as we shall see in the next section) that allow for extracting sizes from array values at runtime. ensures that the involved sizes are identical at runtime. These rules take the form 휏 ⊢푥 푣 { 푟, where 휏 is a type, 푥 is a variable, 푣 is a value, and 푟 is either • (denoting that 푥 2.3 Dynamic Semantics does not appear appropriately in 휏) or a size 푛 extracted from 푣 Values ( ) are defined according to Figure4, where dynamic an array value in 푣, guided by the first preorder location of 휌 environments ( ) are maps from program variables to values: 푥 in 휏.2 A rule for extracting multiple sizes takes the form Notice here that the goal of our type system is not to rule 휏 ⊢푥¯ 푣 { 푛¯. The size-matching rules are given in Figure5. out index and type coercion errors which means that even The rules for the dynamic semantics also make use of a well-typed expressions, may fail to evaluate to a value. set of dynamic size-checking rules that allow for dynamic The dynamic semantics for the language evaluates an checking of array sizes according to a type. These rules take expression 푒 to a result value 푣 in a dynamic environment 휌. The dynamic semantics is specified by a set of inference 2All found occurrences of a size variable should result in the same value, so rules, which allow inferences among sentences of the form we just use the first one we find. Towards Size-Dependent Types for Array Programming ARRAY ’21, June 21, 2021, Virtual, Canada

Single variable 휏 ⊢푥 푣 { 푛/• [m-int] [m-bool] [m-fun] int ⊢푥 푣 { • bool ⊢푥 푣 { • (푥 : 휏) → 휇 ⊢푥 푣 { •

휏 ⊢푥 푣 { 푛 휏 ⊢푥 푣 { • 휏 ⊢푥 푣 { 푟 1 1 [m-pair1] 1 1 2 2 [m-pair2] (휏1, 휏2) ⊢푥 (푣1, 푣2) { 푛 (휏1, 휏2) ⊢푥 (푣1, 푣2) { 푟

푥 ≠ 푦 휏 ⊢푥 푣 { 푟 [m-arr1] 1 [m-arr2] [푥]휏 ⊢푥 [푣1, ··· , 푣푛] { 푛 [푦]휏 ⊢푥 [푣1, ··· , 푣푛] { 푟

Multiple variables 휏 ⊢푥¯ 푣 { 푛¯

휏 푣 푛 휏 푣 푛 ⊢푥1 { 1 ··· ⊢푥푚 { 푚 휏 푣 푛 푛 [m-multi] ⊢(푥1 ···푥푚) { ( 1 ··· 푚)

Figure 5. Dynamic size-matching for the language.

Values 휌 ⊢ 푣 ⊲ 휏

푣 = true ∨ 푣 = false 휌 ⊢ 푣 ⊲ 휏 휌 ⊢ 푣 ⊲ 휏 [c-int] [c-bool] 1 1 2 2 [c-pair] 휌 ⊢ 푛 ⊲ int 휌 ⊢ 푣 ⊲ bool 휌 ⊢ (푣1, 푣2) ⊲ (휏1, 휏2)

휌(푥) = 푛 휌 ⊢ 푣 ⊲ 휏 ··· 휌 ⊢ 푣 ⊲ 휏 휌 ⊢ 푣 ⊲ 휏 ··· 휌 ⊢ 푣 ⊲ 휏 1 푛 [m-arr-x] 1 푛 [m-arr-n] 휌 ⊢ [푣1, ··· , 푣푛] ⊲ [푥]휏 휌 ⊢ [푣1, ··· , 푣푛] ⊲ [푛]휏

Figure 6. Dynamic size-checking for the language. the form 휌 ⊢ 푣 ⊲ 휏, where 휌 is a dynamic environment, 푣 is arrays that are constructed during evaluation are regular a value, and 휏 is a type. The size-checking rules are given in (all rows have the same shape). Moreover, the soundness Figure6. The dynamic semantics rules are given in Figure7. result allows the compiler to omit a large number of size compatibility checks (which are established statically by the 3 Metatheory for 퐹 type system). This section builds up a metatheory for 퐹, culminating in a The remainder of this section is quite dense, and readers soundness proof. are free to skip the detailed proofs, as later sections do not As mentioned earlier, for simplicity, the soundness result directly build on the actual proofs (only on the properties concerns well-typed and well-terminating programs. By re- they concern). stricting the result to well-terminating programs, we have pushed aside the technicalities involved in propagating er- 3.1 Properties of the Type System rors in the rules for the dynamic semantics. Notice that 퐹 The type system possesses a number of properties that are im- does not contain any form of unbounded recursion (all the portant for ensuring well-formedness of types. The following second-order array combinators can be defined inductively), two propositions are established by straightforward induc- thus, in priciple, by propagating out-of-bound array-index tion on the structure of the type-well-formedness derivation errors and failed type-matching errors, explicitly, in the rules and the typing derivation, respectively. for the dynamic semantics, termination can be established formally; we will leave it up to future work to consider this possibility. Proposition 3.1 (Type well-formedness preserved under 휏 휏 , 푥 휏 휇 The soundness result that we establish here gives a guaran- size substitution). Let = ★ or = int. If Γ : ⊢ ok and 푑 휇 푑 푥 tee that when an expression evaluates to a value containing Γ ⊢ ok then Γ ⊢ { / } ok. arrays, the arrays are described by the type of the expression. As a consequence, because size quantifiers cannot appear Proposition 3.2 (Typing yields well-formed types). If Γ ⊢ underneath array type constructors, all multi-dimensional Γ ok and Γ ⊢ 푒 : 휇 then Γ ⊢ 휇 ok. ARRAY ’21, June 21, 2021, Virtual, Canada Troels Henriksen and Martin Elsman

Expressions 휌 ⊢ 푒 { 푣 [d-int] [d-true] [d-false] 휌 ⊢ 푛 { 푛 휌 ⊢ true { true 휌 ⊢ false { false

휌(푥) = 푣 휌 ⊢ 푒 { 푣 ··· 휌 ⊢ 푒푛 { 푣푛 휌 ⊢ 푒 { 푣 휌 ⊢ 푒 { 푣 [d-var] 1 1 [d-array] 1 1 2 2 [d-pair] 휌 ⊢ 푥 { 푣 휌 ⊢ [푒1, ··· , 푒푛] { [푣1, ··· , 푣푛] 휌 ⊢ (푒1, 푒2) { (푣1, 푣2)

휌 ⊢ 푒1 { [푣0, . . . , 푣푚−1] 휌 ⊢ 푒 { (푣 , 푣 ) 휌 ⊢ 푒 { (푣 , 푣 ) 휌 ⊢ 푒 { 푛 0 ≤ 푛 < 푚 1 2 [d-fst] 1 2 [d-snd] 2 [d-index] 휌 ⊢ fst 푒 { 푣1 휌 ⊢ snd 푒 { 푣2 휌 ⊢ 푒1 [푒2] { 푣푛

′ 휌 ⊢ 푒1 { ⟨푥, 푒0, 휌 ⟩ 휌 ⊢ 푒 { 푛 휌 ⊢ 푒 { 푣 ′ 휌 ′, 푥 : 푣 ′ ⊢ 푒 { 푣 푣 = [0, ··· , 푛 − 1] [d-lam] 2 0 [d-app] [d-iota] 휌 ⊢ 휆푥.푒 { ⟨푥, 푒, 휌⟩ 휌 ⊢ 푒1 푒2 { 푣 휌 ⊢ iota 푒 { 푣

휌 ⊢ 푒 { true 휌 ⊢ 푒 { 푣 휌 ⊢ 푒 { false 휌 ⊢ 푒 { 푣 1 [d-if-t] 2 [d-if-f] 휌 ⊢ if 푒 then 푒1 else 푒2 { 푣 휌 ⊢ if 푒 then 푒1 else 푒2 { 푣

휌 ⊢ 푒 { 푣 휌 ⊢ 푒 { 푣 휏 ⊢푥¯ 푣 { 푛¯ 휌 ⊢ 푒 { 푣 휌, 푥 : 푣 ⊢ 푒 ′ { 푣 ′ 휌, 푥¯ : 푛,¯ 푥 : 푣 ⊢ 푒 ′ { 푣 ′ 휌 ⊢ 푣 ⊲ 휏 [d-let] [d-let-m] [d-coerce] 휌 ⊢ let 푥 = 푒 in 푒 ′ { 푣 ′ 휌 ⊢ let [푥¯] 푥 : 휏 = 푒 in 푒 ′ { 푣 ′ 휌 ⊢ 푒 ⊲ 휏 { 푣

Figure 7. Dynamic semantics for the language.

3.2 Relating Values and Types Types and Return Types 훿 |= 푣 : 휇 . In this section, we define a logical relation between values l-int 훿 |= 푛 : int and types. To do so, we use 훿 to range over size assignments, l-true 훿 |= true : bool which are mappings from program variables to sizes (i.e., l-false 훿 |= false : bool integers). The logical relation is specified using a series of l-pair 훿 |= (푣1, 푣2) : (휏1, 휏2) iff 훿 |= 푣1 : 휏1 and 훿 |= 푣2 : 훿 푣 휎 훿 푣 휇 inductive definitions on the forms |= : and |= : , 휏2 푣 휎 which are read “the value has type scheme (or return l-arr-n 훿 |= [푣1, ··· , 푣푛] : [푛]휏 iff 훿 |= 푣푖 : 휏, ∀푖 ∈ type 휇) under the size assignment 훿.” {1..푛} We define a notion of agreement between size assignments l-arr-x 훿 |= [푣1, ··· , 푣푛] : [푥]휏 iff 훿 (푥) = 푛 and 훿 |= 푋 on a set of size variables. When is a set of size variables and 푣푖 : 휏, ∀푖 ∈ {1..푛} 훿 and 훿 ′ are size assignments, we say that 훿 agrees with 훿 ′ on l-clos 훿 |= ⟨푥, 푒 ′, 휌⟩ : (푥 : 휏 ′) → 휇 iff 푋 훿 ′ 훿 푋 훿 푋 훿 ′ ′ ′ the set , written ≈푋 , if ⊆ Dom( ) and ⊆ Dom( ) ∀푣1, 푣2, (훿 |= 푣1 : 휏 and 휌, 푥 : 푣1 ⊢ 푒 { 푣2) and 훿 (푥) = 훿 ′(푥), for all 푥 ∈ 푋. =⇒ if 휏 ′ = int The notion of agreement, on a given set, is reflexive, tran- then 훿, 푥 : 푣1 |= 푣2 : 휇 훿 ′ 훿 푌 푋 sitive, and symmetric. Further, if ≈푋 and ⊆ then else 훿 |= 푣2 : 휇 ′ ′ ′ 훿 ≈푌 훿. l-rt 훿 |= 푣 : ∃푥.휇 iff ∃푛 s.t. 훿, 푥 : 푛 |= 푣 : 휇 The logical relation is first defined inductively on types The logical relation extends to type schemes as follows: and return types: Type Schemes 훿 |= 푣 : 휎 . l-ts-type 훿 |= 푣 : 휎 iff 훿 |= 푣 : 휏 (휎 = 휏) l-ts-★ 훿 |= 푛 : ★

The logical relation extends to environments point-wise, but we take care to ensure that abstract size variables (type scheme ★) or variables of type int, appear as size assump- tions in 훿 and appear, identically, as values in 휌. Towards Size-Dependent Types for Array Programming ARRAY ’21, June 21, 2021, Virtual, Canada

We shall also make use of the following definitions of 3.4 Size Matching and Size Checking Properties TyDom(Γ) and StarDom(Γ), which capture the set of vari- In the proof of soundness of the size-typing rules, we will ables in Γ that are associated with proper type, and the set make use of a property of the dynamic size-matching rules. of variables that are associated with abstract sizes (★): The following proposition holds: TyDom(Γ) = {푥 ∈ Dom(Γ) | Γ(푥) ≠ ★} Proposition 3.4 (Dynamic Size Matching). 훿 푣 푥.휏 휏 푣 푛 훿, 푥 푛 푣 휏 StarDom(Γ) = {푥 ∈ Dom(Γ) | Γ(푥) = ★} 1. If |= : ∃ and ⊢푥 { then : |= : . 2. If 훿 |= 푣 : ∃푥.휏¯ and 휏 ⊢푥¯ 푣 { 푛¯ then 훿, 푥¯ : 푛¯ |= 푣 : 휏.

Thus, for any Γ, we have Dom(Γ) = TyDom(Γ)∪StarDom(Γ) Proof. Property 1 follows from simple induction over the 휏 and TyDom(Γ) ∩ StarDom(Γ) = ∅. structure of . Property 2 follows by repeated application of property 1. □ Environments 훿 |= 휌 :Γ . We will make use also of the following property of the 훿 | 휌 l-env = :Γ iff size checking rules: 1. Dom(휌) = TyDom(Γ) and ⊢ Γ ok 2. ∀푥 ∈ TyDom(Γ), 훿 |= 휌(푥) :Γ(푥) Proposition 3.5 (Dynamic Size Checking). If 훿 |= 푣 : 휏1 and ¯ 3. ∀푥 ∈ TyDom(Γ),(Γ(푥) = int =⇒ 훿 (푥) = 휏 = 휏1{푑/푥¯} and 휌 ⊢ 푣 ⊲ 휏 and 휌(푥) = 훿 (푥), ∀푥 ∈ fv(휏), then 휌(푥)) 훿 |= 푣 : 휏. 4. StarDom(Γ) ⊆ Dom(훿) Proof. By induction on the structure of 휏 . □ The free size variables of an environment Γ, written fsv(Γ) 1 is defined as follows: 3.5 Soundness of Size Typing We can now state and prove a soundness property for the fsv(Γ) = {푥 ∈ Dom(Γ) | Γ(푥) = int ∨ Γ(푥) = ★} type system. The property gives us no guarantee that evalu- ∪ {푥 ∈ fv(Γ(푦)) | 푦 ∈ Dom(Γ)} ation is strongly normalising. We shall return to this issue later in Section 3.6. Notice also that 훿 |= 휌 :Γ entails ⊢ Γ ok, which is useful for ensuring well-formedness of type environments throughout Proposition 3.6 (Soundness). If Γ ⊢ 푒 : 휇 and 훿 |= 휌 :Γ and the inductive proofs. Well-formed type environments are 휌 ⊢ 푒 { 푣 then 훿 |= 푣 : 휇. important for establishing, for instance, well-formedness of Proof. By induction on the structure of the typing derivation. types, as we have seen earlier, which, allows us to reason See AppendixA for details. □ about the identity of types. For instance, we can establish 푥 휇 휇 푥.휇 that if ∉ Dom(Γ) and Γ ⊢ ok then = ∃ (because 3.6 Distinguishing Effects return types are considered identical up to removal of bound By considering only value-terminating expressions, we have variables that do not occur in the body). avoided specifying dynamic evaluation rules for propagating 3.3 Logical Relation Extensibility dynamic errors. There are a number of ways in which well- typed expressions may fail to evaluate. First, rule d-index Before we state a soundness property for the type system, requires the index to be in-bound. Second, rule d-coerce may we first demonstrate an important property of the logical hit an array size mismatch. Third, rule d-coerce assumes relation, namely that it supports an extension property say- that the result type does not contain any function types. ing that if 훿 |= 휌 :Γ and 훿 ′ ≈ 훿 then 훿 ′ |= 휌 :Γ. This fsv(Γ) Fourth, rule d-let-sz may fail either due to the presence of property is important for giving a soundness proof for the an empty array, due to the lack of a size name in the type, or type system, based on simple induction principles.3 due to a size name being present only below a function type. Proposition 3.3 (Logical Relation Extensibility). Dealing properly with inner sizes of empty arrays requires ′ ′ that shape information is available dynamically in cases 1. If 훿 |= 푣 : 휏 and 훿 ≈fv(휏) 훿 then 훿 |= 푣 : 휏. ′ ′ where an array can be empty. Whereas our Futhark imple- 2. If 훿 |= 푣 : 휇 and 훿 ≈fv(휇) 훿 then 훿 |= 푣 : 휇. ′ ′ mentation treats empty arrays properly (by annotating array 3. If 훿 |= 푣 : 휎 and 훿 ≈fv(휎) 훿 then 훿 |= 푣 : 휎. ′ ′ values with shape information at runtime), we leave it to 4. If 훿 |= 휌 :Γ and 훿 ≈fsv(Γ) 훿 then 훿 |= 휌 :Γ. future work to provide a formal treatment of empty arrays. Proof. The main interesting case is the case for function Technically, it would not be difficult to establish a termi- types. The remaining cases either follow immediately or nation result based on a logical relation argument similar to can be shown by straightforward induction. For details, see the one we have presented here but changed to demand that AppendixA. □ there exists an evaluation derivation resulting in a value (or 3In particular, the property is used in the proof cases for lambda abstraction a dynamic error) that relates to the result type of the typing and let-binding. derivation. Such an argument would follow closely Tait’s ARRAY ’21, June 21, 2021, Virtual, Canada Troels Henriksen and Martin Elsman

휏 ::= ··· Basic types which is ill-typed because | 훼 type variable Γ ⊢ (휆푥.iota 푥) : (푥 : 푖푛푡) → [푥]int and we cannot instantiate 훽 in the type scheme for map with 휎 ::= ··· Type schemes [푥]int because Γ ⊢ [푥]int ok does not hold (푥 ∉ Dom(Γ)). | ∀푥.휎 universal size quantification Similarly, the expression | ∀훼.휎 universal type quantification map (휆푥.filter 푓 푥) 푦 푒 ::= Expressions is ill-typed because t-inst only allows instantiation of type 푓 푥 훼 푒 푒 | let [ ¯] ¯ = in polymorphic let variables with basic types (휏). We postpone the development of a soundness result for the Figure 8. Grammar components for parametric polymor- language featuring size- and type-polymorphism. However, phism, extending those of Figure1. for the Hindley-Damas-Milner style let-polymorphism we support, extending the proof to cover these features, we believe can be built on standard techniques [18]. strong-normalisation argument for the simply-typed lambda calculus, which dates back to 1967 [25] and is later estab- 5 Size Types in Futhark lished as a fundamental proof technique [10] and adapted Apart from the formal treatment of 퐹, we have also added for a variety of uses [5]. size types to the Futhark programming language. The core For proving strong-normalisation in cases of inductively design is the same—in particular, sizes must still be constants defined data-structures, logical relation techniques have been or variable names—but the system has been extended and augmented with step-indexing techniques [2, 6], which serve augmented to handle the requirements of a full-featured pro- to establish a well-defined logical relation. Such techniques gramming language. In this section we will discuss the sig- could be useful also in the setting of array programming, nificant extensions that we found necessary, as well as reflect where array sizes could serve as a means for controlling array on their usability for writing real functional array programs. slicing and looping in an inductively defined way, which Our exposition of size types in Futhark is still informal—a could be used for providing strong-normalisation guarantees full formalisation remains future work. In Futhark, sizes are for a large class of array programs. of type i64, for 64-bit integers. Generally, the addition of size types did not complicate the compiler’s many front-end 4 Universal Size- and Type-Quantification transformations, such as defunctorisation [8], monomorphi- For supporting universal size- and type-quantification (para- sation, lambda lifting, and defunctionalisation [14]. metric polymorphism), the grammar for types, type schemes, and expressions is extended as shown in Figure8. Note that Nontrivial size expressions. The soundness proof in Sec- the polymorphic let binding binds only a single name, and tion3 assumes that all subexpressions are variables or integer requires explicit quantification of both type- and size param- constants. This assumption is awkward when writing real eters. The type rules are given in Figure9, where ftv(Γ, 휏) programs, and so the Futhark compiler morally rewrites denotes the free type variables in Γ and 휏. expressions of the form With this extension, we can express the types of common iota (f x) parallel building blocks: to 푣 val length : ∀푥.[푥]훼 → int let = f x in iota v val map : ∀푥.(훼 → 훽) → [푥]훼 → [푥]훽 for some compiler-generated name 푣. Further, it also uses the val zip : ∀푥.[푥]훼 → [푥]훽 → [푥](훼, 훽) explicit version of the let construct to assign internal names val reduce : ∀푥.(훼 → 훼 → 훼) → 훼 → [푥]훼 → 훼 to existential sizes. To keep type errors comprehensible, the val scan : ∀푥.(훼 → 훼 → 훼) → 훼 → [푥]훼 → [푥]훼 compiler remembers the original expression for 푣, so that val filter : ∀푥.(훼 → bool) → [푥]훼 → ∃푦.[푦]훼 whenever 푣 occurs as a size in a type error, the compiler will be able to point to the originating expression. Similarly, Notice, however, that 퐹 is not sufficiently expressive to when existential sizes arise due to names going out of scope, actually define all of these functions, as reduce, scan, and or application of a function that returns an existential size, filter require recursion or some other flexible construct the compiler tracks the origin of the existential size, so it for expressing recurrences. can explain where it came from. This requires some extra It is critical that parametric polymorphism does not allow book-keeping in the type checker, but has not complicated us to construct irregular arrays. To see how the type rules later stages or optimisations in the compiler, as merely main- prevent irregular arrays, consider the expression taining the program in Administrative Normal Form [21], map (휆푥.iota 푥) 푦 provides these properties. Towards Size-Dependent Types for Array Programming ARRAY ’21, June 21, 2021, Virtual, Canada

Types Γ ⊢ 휏 ok

Γ ⊢ 훼 ok

Expressions Γ ⊢ 푒 : 휇

Γ(푓 ) = ∀훼.¯ ∀푥.휏¯ Γ ⊢ 휏 ok ··· Γ ⊢ 휏 ok Γ ⊢ 푦 : int ··· Γ ⊢ 푦 : int 1 푛 1 푛 [t-inst] Γ ⊢ 푓 : 휏{휏¯/훼,¯ 푦¯/푥¯}

{훼¯} ∩ ftv(Γ, 휏 ) = ∅ {푥¯} ∩ fv(휏 ) = ∅ Γ, 푥¯ : int ⊢ 푒 : 휏 Γ, 푓 : ∀훼.¯ ∀푥.휏¯ ⊢ 푒 : 휏 2 2 1 1 1 2 2 [t-gen] Γ ⊢ let 푓 [푥¯] 훼¯ = 푒1 in 푒2 : 휏2

Figure 9. Type rules for polymorphism, extending those of Figures2 and3.

Type inference. While 퐹 is explicitly typed, the Futhark But as discussed in Section4, apply (filter p) xs is not type checker is built around a conventional Hindley-Damas- well typed. To address this, we introduce the notion of size- Milner type inference implementation. We have extended lifted type parameters, written '~a, which may be instantiated this implementation to also support size inference. When (in negative position) with return types with existential sizes: instantiating a type scheme, each size parameter is turned let apply 'a '~b (f: a → b) (x: a) : b = f x into a nonrigid size variable, which is then unified with other size variables and term-level variables. This allows inference To avoid irregular arrays, size-lifted types may never be array of a (contrived) function such as elements, which is enforced by the type checker. let f n xs = zip (iota n) xs 6 Experience Report which is assigned the type When designing a type system that limits flexibility to obtain val f 't : (n:i64) → (xs :[n]t) → [n](i64 ,t) convenience, we must evaluate whether the loss of flexibility is an obstacle in practice, which is necessarily a subjective Similarly to type variables, unconstrained size variables are question. Our prototype implementation of size types was turned into size parameters during let-generalization. Our added to Futhark in version 0.15.1, released in March 2020. type inference algorithm is currently merely a best-effort The Futhark benchmark suite 4 is the largest single collection implementation, and will sometimes need type annotations of Futhark programs, and serves as a case study for the usabil- in the source program to assign sizes that would otherwise ity of the type system. The suite comprises about 12,000 lines be ambiguous. of code, spread over ports of benchmarks from Rodinia [4] (13 programs), FinPar [1] (3 programs), Parboil [24] (5 pro- Implicit size parameters. Futhark supports size param- grams), the Accelerate [3] examples (13 programs), and an eters, as in Section4, which are also the main mechanism ad-hoc collection of further benchmarks (10 programs). Most we use to express shape constraints in function types. For of these programs were written before size types were added example:: to Futhark and had to be modified. val dotprod [n] : The benchmark suite contains 66 instances of dynami- (xs:[n]i64) → (ys:[n]i64) → i64 cally checked size coercions, including calls to library func- tions that include such size coercions. Most of these are in During compilation, these implicit parameters are turned initialisation code, where inputs must be packed in some into ordinary explicit parameters. For each application dotprod benchmark-specific format. For example, backprop from Ro- xs ys, the compiler will then use the types of xs and ys to dinia accepts and returns a variety of arrays of sizes 푛 and decide what to pass for the implicit arguments. 푛 + 1, where the program then splits the latter into arrays of size 1 and 푛 respectively. Since this relationship cannot be Extensions to parametric polymorphism. There are directly expressed in our size type system, we are forced to some cases where we wish to allow a functional argument use type coercions. The main computational core of back- to a higher-order function to have an existential size in its prop does not contain any type coercions. While dynamic result type checks are never desirable, they are relatively innocent in let apply 'a 'b (f: a → b) (x: a) : b = f x 4https://github.com/diku-dk/futhark-benchmarks/ ARRAY ’21, June 21, 2021, Virtual, Canada Troels Henriksen and Martin Elsman initialisation code, where they function as a form of input 7 Related Work validation. Size inference. A theory of shapes was developed by Jay In some cases we had to change our programming style. over a series of papers [15, 16]. Jay’s specific work focused For example, a relatively commonly occurring pattern was on “shapely” programs, where the shapes could be deter- map (휆i → map (휆j → ...) (iota (f x))) mined in advance (possibly by using partial evaluation to ( iota (g y)) construct an inspector [17]), while our size types also sup- port statically undeterminable existential sizes. The Futhark Since f x is not a constant or variable, the innermost iota compiler previously used a size inference technique based (and hence the entire inner map) returns an array of exis- on slicing [12] to e.g. compute in advance the return size tential size, which makes the outermost map type-incorrect, of the function passed to map, but this has been completely as its function must not return an array of existential size. replaced by the size type system, where the type checker While f x is invariant to the map, we did not want the type provides all the information needed by the compiler. checker to make such subtle transformations on behalf of the programmer. The immediate solution is to hoist out the Array languages. In the APL tradition, 푘-dimensional size as an outer binding: arrays are represented as a pair of a 푘-vector of sizes and an- other vector of values. This permits a trivial implementation let n = f x of operations such as reshaping by changing the shape vec- in map ( i → map ( j → ...) (iota n)) 휆 휆 tor, and of shape polymorphic functions that operate solely ( iota (g y)) on the value vector. This idea is given a deeper theoretical This solution is arguably ugly. But if we define a function treatment in the mathematics of arrays [20], and also used in typed functional languages such as SAC [22] and TAIL val tabulate_2d 'a : (n: i64) → (m: i64) [7, 11]. In contrast, our type system views multidimensional → (f: i64 → i64 → a) → [n][m]a arrays inductively as as “arrays of arrays”, and while it sup- then we can write the original expression as: ports a form of size polymorphism, it does not support rank polymorphism. The advantage is that this is a natural fit with tabulate_2d (g y) (f x) (휆i j → ...) conventional ML-style parametric polymorphism, where an array type [푛]훼 can have the element type 훼 substituted with Similarly, a common pattern was to write any type, including another array type. This also makes it zip xs (iota (length xs)) straightforward to, for example, map a function across the rows of a two-dimensional array. to associate each element of an array with its index. This is invalid under size typing, since as far as the type checker is Type systems. Our type system is heavily inspired by de- concerned, length just returns an arbitrary integer without pendent types, where tracking the sizes of vectors is one of any relation to the size of xs. A solution is to define another the classic examples. For example, Dependent ML uses sin- helper function gleton types to eliminate bounds checking in programs [27]. val indices [n] 'a : [n]a → [n]i64 While we do not address bounds checking in our type system and fall back to run-time checks, we assume that most pro- such that we can write zip xs (indices xs). Both of these grams will be written “index-free” using combinators such functions are ordinary library code, not compiler builtins, as map and reduce. The main limitation of our type system and do not contain any size coercions. In this way, we were versus full-spectrum dependent types is that we only allow usually able to both satisfy the type checker and write con- one specific term-level type (integers) to appear in types, cise code, although we did have to change old habits. only for array shapes, and only in a very restricted syntactic We also used size-typed Futhark to teach parallel program- form (variables and constants). A notable implementation ming to students at the University of Copenhagen. Most had advantage of this simplification is that it permits straight- no experience with advanced type systems, and many had forward type erasure, as arrays nevertheless have to carry little experience with functional programming at all, yet they around their shape at run-time, which is usually a negligible were able to write nontrivial Futhark programs without trip- amount of data compared to the array elements. ping over type restrictions too often. We attribute this to the Qube [26] is an interesting combination of APL-style shapes simplicity of the rules, as well as the ease with which they and dependent types, including support for shape polymor- can be broken when necessary. For programmers who are phic programming. Qube verifies all size constraints and not experts in type theory, it is useful to have a rigid type index operations at compile-time, using an SMT solver to system that covers only the simple cases and contains escape handle complex constraints. However, Qube still requires hatches for the rest, rather than a complex system that can the array element type to be monomorphic, which in par- handle all cases. ticular means that it does not support ML-style parametric Towards Size-Dependent Types for Array Programming ARRAY ’21, June 21, 2021, Virtual, Canada polymorphism. In contrast to both Futhark and 퐹, Qube sup- Proof. For proving property 1, there is one interesting case. ports irregular arrays via dependent products—we believe The remaining cases either follow immediately or can be the same approach could also be used with our size types. shown by straightforward induction. Remora [23] is an array language that is similarly depen- Case 휏 = (푥 : 휏 ′) → 휇: From assumption and l-clos, we ′ ′ dently typed, and focuses on capturing APL-style implicit ag- have ⌈1⌉ 푣 = ⟨푥, 푒 , 휌⟩ and ⌈2⌉ ∀푣1, 푣2,(훿 |= 푣1 : 휏 and ′ ′ gregate operations in the type system. In particular, Remora 휌, 푥 : 푣1 ⊢ 푒 { 푣2) =⇒ (if 휏 = int then 훿, 푥 : 푣1 |= 푣2 : 휇 supports arrays of functions, allowing a programming style else 훿 |= 푣2 : 휇). ′ that encodes control flow as data. It has been shown that To establish 훿 |= 푣 : 휏, we must establish ⌈3⌉ ∀푣1, 푣2, ′ ′ ′ ′ parts of this type discipline can be encoded using the frag- (훿 |= 푣1 : 휏 and 휌, 푥 : 푣1 ⊢ 푒 { 푣2) =⇒ (if 휏 = int then ′ ′ ment of dependent types available in Haskell [9] 훿 , 푥 : 푣1 |= 푣2 : 휇 else 훿 |= 푣2 : 휇). ′ ′ ′ Finally, Dex [19] is an array language developed at Google We first assume ⌈4⌉ 훿 |= 푣1 : 휏 and ⌈5⌉ 휌, 푥 : 푣1 ⊢ 푒 { ′ Research. In contrast to most such languages, it embraces 푣2. From assumption, we have ⌈6⌉ 훿 ≈fv(휏) 훿. From the explicit indexing, making it safe by making assigning distinct definition of agreement and because fv(휏 ′) ⊆ fv(휏), we have ′ types (and types of index values) to each distinct dimension ⌈7⌉ 훿 ≈fv(휏′) 훿. By induction on ⌈4⌉ and ⌈7⌉, we have ⌈8⌉ ′ of an array. Dex uses conventional existential types to handle 훿 |= 푣1 : 휏 . We can now apply ⌈2⌉ to ⌈8⌉ and ⌈5⌉ to get ⌈9⌉ ′ operations such as filter. (if 휏 = int then 훿, 푥 : 푣1 |= 푣2 : 휇 else 훿 |= 푣2 : 휇). There are now two cases. 8 Conclusions and Future Work Subcase 휏 ′ = int: From the definition of agreement and 훿 ′, 푥 푣 훿, 푥 푣 In this paper we have presented a type system for array pro- from ⌈6⌉, we have ⌈10⌉( : 1) ≈fv(휇) ( : 1). By 휏 ′ 훿 ′, 푥 푣 gramming that is fundamentally similar to existing ML-style induction on ⌈9⌉ (case = int) and ⌈10⌉, we get : 1 |= 푣 휇 type systems, yet allows the checking of array size invariants 2 : , as required. in function applications. We formalised the core of the type Subcase 휏 ≠ int: In this case 푥 ∉ fv(휇), thus, from ⌈6⌉ 훿 ′ 훿 system and explained informally how we have added it to and the definition of agreement, we have ⌈11⌉ ≈fv(휇) . It ′ the Futhark language. The type system has the significant now follows directly using induction on ⌈9⌉ (case 휏 ≠ int) 훿 ′ 푣 휇 restriction that only variables or constants can be used in and ⌈11⌉ that |= 2 : , as required. array sizes. Despite this limitation, it is flexible enough to For proving property 2, there is one case to consider: ′ ′ handle the programs found in the Futhark benchmark suite, Case 휇 = ∃푥.휇 : By renaming, we can assume 푥 ∉ Dom(훿 ). most of which are ported from other languages, while need- From l-rt, we have there exists 푛 such that 훿, 푥 : 푛 |= ing few dynamic checks. This suggests that the type system 푣 : 휇′. From the definition of agreement, we have (훿 ′, 푥 : ′ is a good fit for regular nested data parallel algorithms. 푛) ≈fv(휇′)∪{푥 } (훿, 푥 : 푛) and, thus, (훿 , 푥 : 푛) ≈fv(휇′) (훿, 푥 : 푛). However, there is more work to do. It is awkward that the We can now apply induction to get 훿 ′, 푥 : 푛 |= 푣 : 휇′. From type system is not strong enough to express the shapes that l-rt, we now have 훿 ′ |= 푣 : 휇, as required. arise from catenating or flattening arrays. It is likely that Property 3 follows immediately using property 2. such extensions will require the compiler to solve integer For proving property 4, there is again only one case to problems at compile-time, as in Dependent ML or Qube. consider. We are given 훿 |= 휌 :Γ and we need to establish Finally, our formal treatment of size types is still incom- 훿 ′ |= 휌 :Γ. The second property in the expanded definition of plete. It lacks important features such parametric polymor- |= on environments (i.e., l-env) are established directly using phism, does not describe a technique for type inference, and property 3 above and by the definition of fsv(Γ). The first contains some simplifying assumptions—most critically, that property is independent of 훿 ′ and thus follows immediately. arrays are never empty. The third and fourth properties follow immediately from the definition of agreement and fsv(Γ). □ Acknowledgments This research has been partially supported by a grant from Proposition 3.6 (Soundness) If Γ ⊢ 푒 : 휇 and 훿 |= 휌 :Γ and the Independent Research Fund Denmark, under the re- 휌 ⊢ 푒 { 푣 then 훿 |= 푣 : 휇. search project FUTHARK: Functional Technology for High- performance Architectures. Proof. By induction on the structure of the typing derivation. We proceed by case analysis and show most of the cases, A Proofs of Propositions 3.3 and 3.6 leaving out the case for conditions (which follow by straight- Proposition 3.3 (Logical Relation Extensibility) forward induction), the cases for booleans (which follow ′ ′ 1. If 훿 |= 푣 : 휏 and 훿 ≈fv(휏) 훿 then 훿 |= 푣 : 휏. the case for integers), and some cases that can be treated ′ ′ 2. If 훿 |= 푣 : 휇 and 훿 ≈fv(휇) 훿 then 훿 |= 푣 : 휇. similarly to other cases. ′ ′ 푒 푛 휇 3. If 훿 |= 푣 : 휎 and 훿 ≈fv(휎) 훿 then 훿 |= 푣 : 휎. Case = , = int: Follows immediately from rule t-int, ′ ′ 4. If 훿 |= 휌 :Γ and 훿 ≈fsv(Γ) 훿 then 훿 |= 휌 :Γ. rule d-int, and l-int. ARRAY ’21, June 21, 2021, Virtual, Canada Troels Henriksen and Martin Elsman

′ Case 푒 = 푥, 휇 = 휏: From assumptions, rule t-var, and rule d- by induction on ⌈0⌉, ⌈3⌉, and ⌈5⌉, we get 훿, 푥 : 푣1 |= 푣2 : 휇 , var, we have Γ(푥) = 휏 and 휌(푥) = 푣 and 훿 |= 휌 :Γ. From as required. l-env, we have 훿 |= 휌(푥) :Γ(푥), for all 푥 ∈ Dom(Γ). Thus, Subcase 휏 ′ ≠ int: From ⌈2⌉, assumptions, and from l-env, we have 훿 |= 푣 : 휇, as required. we have ⌈7⌉ 훿 |= (휌, 푥 : 푣 ′) : (Γ, 푥 : 휏 ′). By induction on ⌈0⌉, 훿 푣 휇′ Case 푒 = (푒1, 푒2), 휇 = ∃푥¯1푥¯2.(휏1, 휏2): From rule t-pair, we ⌈3⌉, and ⌈7⌉, we have |= 2 : , as required. 푒 = 푒 푒 휇 = ∃푥.휇′ have ⌈1⌉ Γ ⊢ 푒1 : ∃푥¯1.휏1 and ⌈2⌉ Γ ⊢ 푒2 : ∃푥¯2.휏2. From rule d- Case 1 2, : From assumptions and rule t- 훿 휌 푒 pair, we have ⌈3⌉ 휌 ⊢ 푒1 { 푣1 and ⌈4⌉ 휌 ⊢ 푒2 { 푣2 and app and rule d-app, we have ⌈0⌉ |= :Γ, ⌈1⌉ Γ ⊢ 1 : 푥 휏 휇′ 푒 휏 휌 푒 푥, 푒 ′, 휌 ′ 푣 = (푣1, 푣2). By induction on ⌈1⌉, ⌈2⌉, ⌈3⌉, and ⌈4⌉, we have ( : ) → , ⌈2⌉ Γ ⊢ 2 : , ⌈4⌉ ⊢ 1 { ⟨ ⟩, ⌈5⌉ 휌 ′, 푥 푣 ′ 푒 ′ 푣 휌 푒 푣 ′ 훿 |= 푣1 : ∃푥¯1.휏1 and 훿 |= 푣2 : ∃푥¯2.휏2. Now, let 휇1 = ∃푥¯1푥¯2.휏1 : ⊢ { , and ⌈6⌉ ⊢ 2 { . 훿 and 휇2 = ∃푥¯1푥¯2.휏2. By appropriate renaming, we have 훿 |= By induction applied to ⌈1⌉, ⌈0⌉, and ⌈4⌉, we get ⌈7⌉ |= 푥, 푒 ′, 휌 ′ 푥 휏 휇′ 푣1 : 휇1 and 훿 |= 푣2 : 휇2. From l-rt, we know there exists 푛¯1 ⟨ ⟩ : ( : ) → . From l-clos and ⌈7⌉, we have ⌈9⌉ ′ ′ 푣 , 푣 훿 푣 휏 휌 ′, 푥 푣 푒 ′ 푣 휏 and 푛¯2 such that 훿 = 훿, 푥¯1 : 푛¯1, 푥¯2 : 푛¯2 and 훿 |= 푣1 : 휏1 and ∀ 1 2,( |= 1 : and : 1 ⊢ { 2) =⇒ (if = int ′ ′ 훿, 푥 푣 푣 휇′ 훿 푣 휇′ 훿 |= 푣2 : 휏2. From l-pair, we now have 훿 |= (푣1, 푣2) : (휏1, 휏2). then : 1 |= 2 : else |= 2 : ). We can now use l-rt again to get 훿 |= 푣 : 휇, as required. By induction on ⌈2⌉, ⌈0⌉, and ⌈6⌉, we have ⌈10⌉ 훿 |= 푣 ′ : 휏. ′ ′ ′ Case 푒 = fst 푒 , 휇 = ∃푥.휏¯ : This case is proven by induc- We can now apply ⌈9⌉ to ⌈10⌉ and ⌈5⌉, choosing 푣1 = 푣 and 푣 푣 휏 훿, 푥 푣 ′ 푣 휇′ tion, l-pair, rule t-fst, rule d-fst, multiple uses of l-rt, and 2 = , to get ⌈11⌉ (if = int then : |= : else ′ from return types being considered identical up to removal 훿 |= 푣 : 휇 ). of variables that do not appear in the body. There are now two subcases. Case 푒 = snd 푒 ′, 휇 = ∃푥.휏¯ ′: Similar to the case for 푒 = Subcase 휏 = int: From l-rt and ⌈11⌉, and because 휇 = 푥.휇′ 훿 푣 휇 fst 푒 ′. ∃ , we have |= : , as required. Subcase 휏 ≠ int: In this case, we know that 푥 ∉ fv(휇′), Case 푒 = [푒1, ··· , 푒푛], 휇 = [푛]휏: From rule t-array, we thus, 휇 = 휇′ and it follows directly from ⌈11⌉ that 훿 |= 푣 : 휇, have Γ ⊢ 푒푖 : 휏, 푖 ∈ {1..푛}. From rule d-array, we have as required. 휌 ⊢ 푒푖 { 푣푖 , 푖 ∈ {1..푛}. By induction, we have 훿 |= 푣푖 : 휏, Case 푒 = 푒1 ⊲ 휏, 휇 = 휏: From rule t-coerce, we have ⌈1⌉ 푖 = {1..푛}. From l-arr-n, we have 훿 |= [푣1, ··· , 푣푛] : [푛]휏, 푒 휏 휏 휏 휏 푑¯ 푥 as required. Γ ⊢ 1 : 1 and ⌈2⌉ Γ ⊢ ok and ⌈3⌉ = 1{ /¯}. From 휌 푒 푣 휌 푣 휏 Case 푒 = iota 푑, 휇 = [푑]int: From rule t-iotad, we have rule d-coerce, we have ⌈4⌉ ⊢ 1 { and ⊢ ⊲ . From 훿 휌 ⌈0⌉ Γ ⊢ 푑 : int. From rule d-iota, we have 푣 = [0, ··· , 푛 − 1] assumption, we have ⌈6⌉ |= :Γ. By induction on ⌈6⌉, ⌈1⌉, and ⌈4⌉, we have ⌈7⌉ 훿 |= 푣 : 휏 . From ⌈6⌉ and l-env, we have and ⌈1⌉ 휌 ⊢ 푑 { 푛. From rule t-int, we have Γ ⊢ 푣푖 : int, 1 푥 푥 휌 푥 훿 푥 where 푣 = 푖 − 1, 푖 = {1..푛}. From l-int, we have ⌈2⌉ 훿 |= ⌈8⌉ ∀ ∈ TyDom(Γ),(Γ( ) = int =⇒ ( ) = ( )). From 푖 푥 휏 푥 푣 : int. There are two subcases corresponding to whether ⌈2⌉ and ⌈8⌉, we have ⌈9⌉ ∀ ∈ fv( ), ∈ TyDom(Γ) and 푖 푥 휌 푥 훿 푥 푥 휏 푑 = 푛 (for some 푛) or 푑 = 푥 (for some 푥). Γ( ) = int. Thus, we have ⌈10⌉ ( ) = ( ), ∀ ∈ fv( ). If 푑 = 푛, for some integer 푛, we immediately have from From Proposition 3.5, ⌈7⌉, ⌈3⌉, ⌈5⌉, and ⌈10⌉, we have 훿 푣 휏 훿 푣 휇 l-arr-n that 훿 ⊢ [푣 , ··· , 푣 ] : [푛]int, as required. |= : and, thus, |= : , as required. 1 푛 푒 푥 푒 푒 휇 푥푥.휇′ If 푑 = 푥, from ⌈1⌉ and rule d-var, we have ⌈3⌉ 휌(푥) = 푛. Case = let = 1 in 2, = ∃¯ : From assumption 훿 휌 푒 푥.휏 From ⌈0⌉ and rule t-var, we have ⌈4⌉ Γ(푥) = int. From and rule t-let, we have ⌈1⌉ |= :Γ, ⌈2⌉ Γ ⊢ 1 : ∃¯ , and , 푥 , 푥 휏 푒 휇′ assumption, we have 훿 |= 휌 :Γ, thus, from l-env and ⌈4⌉, Γ ¯ : ★¯ : ⊢ 2 : . From assumption and rule d-let, we 휌 푒 푣 휌, 푥 푣 푒 푣 we have 푥 ∈ TyDom(훿) and ⌈5⌉ 훿 (푥) = 휌(푥). From ⌈5⌉ and have ⌈4⌉ ⊢ 1 { 1 and : 1 ⊢ 2 { . By induction on 훿 푣 푥.휏 ⌈3⌉ it thus follows that ⌈6⌉ 훿 (푥) = 푛. From, l-arr-x, ⌈2⌉, and ⌈1⌉, ⌈2⌉, and ⌈4⌉, we get ⌈6⌉ |= 1 : ∃¯ . From l-rt and ⌈6⌉, 푛 훿, 푥 푛 푣 휏 ⌈6⌉, we now have 휌 |= [0, ··· , 푛 − 1] : [푑]int, as required. we have there exists ¯ such that ⌈7⌉ ¯ : ¯ |= 1 : . There ′ Case 푒 = 휆(푥 : 휏 ′).푒 ′, 휇 = (푥 : 휏 ′) → 휇′: From rule t-lam, are now two cases. We first assume 휏 ≠ int. Let 훿 = 훿, 푥¯ : 푛¯ ′ , 푥 , 푥 휏 휌 ′ 휌, 푥 푣 we have ⌈0⌉ Γ, 푥 : 휏 ′ ⊢ 푒 ′ : 휇′ and Γ: 휏 ′ : ok. From rule d- and Γ = Γ ¯ : ★¯ : , and = : 1. From ⌈1⌉ and 훿 ′ 훿 훿 ′ 휌 lam, we have 푣 = ⟨푥, 푒 ′, 휌⟩. From l-clos, we have that to because ≈fsv(Γ) , we have |= :Γ. Now, from l-env, establish 훿 |= 푣 : 휇, we need to show we have ⌈8⌉ Dom(휌) = TyDom(Γ) and ⊢ Γ ok ⌈ ⌉∀ 푣 , 푣 훿 |= 푣 휏 ′ 휌, 푥 푣 ⊢ 푒 ′ 푣 =⇒ 1 1 2,( 1 : and : 1 { 2) (if ⌈9⌉∀ 푥 ∈ TyDom(Γ), 훿 ′ |= 휌(푥) :Γ(푥) 휏 ′ = int 훿, 푥 푣 ′ |= 푣 휇′ 훿 |= 푣 휇′ then : 2 : else 2 : ) ⌈10⌉∀ 푥 ∈ TyDom(Γ), Γ(푥) = int =⇒ 훿 ′(푥) = 휌(푥) ′ ′ ⌈11⌉ StarDom(Γ) ⊆ Dom(훿 ′) We first assume ⌈2⌉ 훿 |= 푣1 : 휏 and ⌈3⌉ 휌, 푥 : 푣1 ⊢ 푒 { 푣2. There are now two subcases. From ⌈8⌉, the definitions of Γ′ and 휌 ′, and properties about ′ ′ Subcase 휏 = int: Because 푣1 = 푛 for some 푛, we have well-formedness of types, we have that ⌈16⌉ Dom(휌 ) = ′ ′ ′ from l-int that ⌈4⌉ 훿, 푥 : 푛 |= 푣1 : 휏 . From assumption and TyDom(Γ ) and ⊢ Γ ok. From ⌈9⌉, ⌈7⌉, and the definitions Property 3.3(4), we have ⌈4푎⌉ 훿, 푥 : 푛 |= 휌 :Γ. From ⌈4⌉ and of 훿 ′ and Γ′, we have ⌈17⌉ ∀푥 ∈ TyDom(Γ′), 훿 ′ |= 휌 ′(푥) : ⌈4푎⌉ and from l-env (and because we can demonstrate ⊢ Γ, 푥 : Γ′(푥). Because 휏 ≠ int (separate case), we have from ⌈10⌉ 휏 ′ ok), we have ⌈5⌉(훿, 푥 : 푛) |= (휌, 푥 : 푛) : (Γ, 푥 : int). Now, that ⌈18⌉ ∀푥 ∈ TyDom(Γ′),(Γ′(푥) = int =⇒ 훿 ′(푥) = Towards Size-Dependent Types for Array Programming ARRAY ’21, June 21, 2021, Virtual, Canada

′ ′ ′ 휌 (푥)). From ⌈11⌉ and the definitions of Γ and 훿 , we have [5] Kevin Donnelly and Hongwei Xi. 2007. A Formalization of Strong Nor- StarDom(Γ′) ⊆ Dom(훿 ′). From ⌈16⌉, ⌈17⌉, ⌈18⌉, and ⌈19⌉, malization for Simply-Typed Lambda-Calculus and System F. Elec- we have ⌈20⌉ 훿 ′ |= 휌 ′ :Γ′. By induction on ⌈20⌉, ⌈3⌉, and tronic Notes in Theoretical Computer Science 174, 5 (2007), 109–125. https://doi.org/10.1016/j.entcs.2007.01.021 Proceedings of the First Inter- ⌈5⌉, using the definitions of 훿 ′, 휌 ′, and Γ′, we have ⌈21⌉ ′ ′ ′ national Workshop on Logical Frameworks and Meta-Languages: Theory 훿 |= 푣 : 휇 . From l-rt and ⌈21⌉, we have ⌈22⌉ 훿 |= 푣 : ∃푥.휇¯ . and Practice (LFMTP 2006). ′ ′ ′ From ⌈1⌉, we have Γ ⊢ 휇 ok and because Γ (푥) ≠ int and [6] D. Dreyer, A. Ahmed, and L. Birkedal. 2009. Logical Step-Indexed Logical Γ′(푥) ≠ ★, we know that 푥 ∉ fv(휇′). Thus, ∃푥.휇′ = 휇′, which, Relations. In 2009 24th Annual IEEE Symposium on Logic In Computer together with ⌈22⌉ gives us 훿 |= 푣 : 휇, as required. If, instead, Science. 71–80. https://doi.org/10.1109/LICS.2009.34 휏 = int 푥 [7] Martin Elsman and Martin Dybdal. 2014. Compiling a Subset of APL we assume , we know ¯ is empty, but, instead, we Into a Typed Intermediate Language. In Proceedings of ACM SIG- have from ⌈7⌉ and l-int, that 푣1 = 푛 for some 푛. Now, let PLAN International Workshop on Libraries, Languages, and Compilers ′ ′ ′ 훿 = 훿, 푥 : 푛, Γ = Γ, 푥 : int, and 휌 = 휌, 푥 : 푛. From ⌈1⌉ and for Array Programming (Edinburgh, United Kingdom) (ARRAY’14). from l-env, we have Association for Computing Machinery, New York, NY, USA, 101–106. https://doi.org/10.1145/2627373.2627390 ⌈23⌉ Dom(휌) = TyDom(Γ) and ⊢ Γ ok [8] Martin Elsman, Troels Henriksen, Danil Annenkov, and Cosmin E. Oancea. ⌈24⌉∀ 푥 ∈ TyDom(Γ), 훿 |= 휌(푥) :Γ(푥) 2018. Static Interpretation of Higher-Order Modules in Futhark: Func- 푥 푥 훿 푥 휌 푥 tional GPU Programming in the Large. Proc. ACM Program. Lang. 2, ⌈25⌉∀ ∈ TyDom(Γ), Γ( ) = int =⇒ ( ) = ( ) ICFP, Article 97 (July 2018), 30 pages. https://doi.org/10.1145/3236792 ⌈26⌉ StarDom(Γ) ⊆ Dom(훿) [9] Jeremy Gibbons. 2016. APLicative Programming with Naperian Functors (Extended Abstract). In Proceedings of the 1st International Workshop From ⌈23⌉, the definition of well-formed environments, and on Type-Driven Development (Nara, Japan) (TyDe 2016). Association the definitions of 휌 ′ and Γ′, we have that ⌈27⌉ Dom(휌 ′) = for Computing Machinery, New York, NY, USA, 13–14. https://doi.org/ ′ ′ ′ 10.1145/2976022.2976023 TyDom(Γ ) and ⊢ Γ ok. From ⌈6⌉ and because 훿 ≈fv(Γ′ (푥)) 훿, forall 푥 ∈ TyDom(Γ′), and from the definitions of 휌 ′ and [10] Jean Yves Girard. 1971. Interpretation Fonctionnelle et Elimination des ′ 푥 ′ Coupures de l’Arithmetique d’Ordre Superieur. In Proceedings of the Γ , we have from Proposition 3.3 that ⌈28⌉ ∀ ∈ TyDom(Γ ), Second Scandinavian Logic Symposium. North-Holland, 63–92. ′ ′ ′ ′ 훿 |= 휌 (푥) :Γ (푥). From ⌈25⌉ and from the definitions of Γ , [11] Troels Henriksen, Martin Dybdal, Henrik Urms, Anna Sofie Kiehn, Daniel 훿 ′, and 휌 ′, we have ⌈29⌉ ∀푥 ∈ TyDom(Γ′), Γ′(푥) = int =⇒ Gavin, Hjalte Abelskov, Martin Elsman, and Cosmin Oancea. 2016. APL 훿 ′(푥) = 휌 ′(푥). From ⌈26⌉ and the definitions of Γ′ and 훿 ′, we on GPUs: A TAIL from the Past, Scribbled in Futhark. In Proceedings have ⌈30⌉ StarDom(Γ′) ⊆ Dom(훿 ′). Now, from ⌈27⌉, ⌈28⌉, of the 5th International Workshop on Functional High-Performance 훿 ′ 휌 ′ ′ Computing (Nara, Japan) (FHPC 2016). Association for Computing ⌈29⌉, and ⌈30⌉, we have ⌈31⌉ |= :Γ . By induction Machinery, New York, NY, USA, 38–43. https://doi.org/10.1145/2975991. ′ ′ on ⌈31⌉, ⌈3⌉, and ⌈5⌉, using the definitions of 훿 , 휌 , and 2975997 Γ′, we have ⌈32⌉ 훿 ′ |= 푣 : 휇′. From l-rt and ⌈32⌉, we have [12] Troels Henriksen, Martin Elsman, and Cosmin E. Oancea. 2014. Size 훿 |= 푣 : ∃푥.휇′ and because 휇 = ∃푥.휇′ (푥¯ is empty), we have Slicing: A Hybrid Approach to Size Inference in Futhark. In Proceedings 훿 |= 푣 : 휇, as required. of the 3rd ACM SIGPLAN Workshop on Functional High-performance 푥 푥 휏 푒 푒 휇 푥푥.휇′ Computing (Gothenburg, Sweden) (FHPC ’14). ACM, New York, NY, USA, Case let [ ¯] : = 1 in 2, = ∃¯ : The proof for 31–42. https://doi.org/10.1145/2636228.2636238 this case is similar to the proof for rule t-let, although size [13] Troels Henriksen, Niels G. W. Serup, Martin Elsman, Fritz Henglein, and variables are bound to int types in type environments. The Cosmin E. Oancea. 2017. Futhark: Purely Functional GPU-programming proof case also makes essential use of Proposition 3.4. □ with Nested Parallelism and In-place Array Updates. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation (Barcelona, Spain) (PLDI 2017). ACM, New References York, NY, USA, 556–571. https://doi.org/10.1145/3062341.3062354 [14] Anders Kiel Hovgaard, Troels Henriksen, and Martin Elsman. 2018. High- [1] Christian Andreetta, Vivien Bégot, Jost Berthold, Martin Elsman, Fritz performance defunctionalization in Futhark. In Symposium on Trends Henglein, Troels Henriksen, Maj-Britt Nordfang, and Cosmin E. Oancea. in Functional Programming (TFP’18). Springer-Verlag. 2016. FinPar: A Parallel Financial Benchmark. ACM Trans. Archit. Code [15] C.B. Jay. 1999. Denotational Semantics of Shape:: Past, Present and Future. 13, 2, Article 18 (June 2016), 27 pages. https://doi.org/10.1145/ Optim. Electronic Notes in Theoretical Computer Science 20 (1999), 320–333. 2898354 https://doi.org/10.1016/S1571-0661(04)80081-1 MFPS XV, Mathematical [2] Andrew W. Appel and David McAllester. 2001. An Indexed Model of Foundations of Progamming Semantics, Fifteenth Conference. Recursive Types for Foundational Proof-Carrying Code. ACM Trans. [16] C. Barry Jay. 1995. A Semantics for Shape. Science of Computer Pro- 23, 5 (Sept. 2001), 657–683. https://doi.org/10. Program. Lang. Syst. gramming 25 (1995), 25–251. 1145/504709.504712 [17] C. Barry Jay and Milan Sekanina. 1997. Shape Checking of Array [3] Manuel M.T. Chakravarty, Gabriele Keller, Sean Lee, Trevor L. McDonell, Programs. Technical Report. In Computing: the Australasian Theory and Vinod Grover. 2011. Accelerating Haskell Array Codes with Multicore Seminar, Proceedings. GPUs. In Proceedings of the Sixth Workshop on Declarative Aspects of [18] Xavier Leroy. 1992. Unboxed Objects and Polymorphic Typing. In Pro- (Austin, Texas, USA) . Association Multicore Programming (DAMP ’11) ceedings of the 19th ACM SIGPLAN-SIGACT Symposium on Principles for Computing Machinery, New York, NY, USA, 3–14. https://doi.org/ of Programming Languages (Albuquerque, New Mexico, USA) (POPL 10.1145/1926354.1926358 ’92). Association for Computing Machinery, New York, NY, USA, 177–188. [4] S. Che, M. Boyer, J. Meng, D. Tarjan, J. W. Sheaffer, S. H. Lee, and K. https://doi.org/10.1145/143165.143205 Skadron. 2009. Rodinia: A benchmark suite for heterogeneous computing. [19] Dougal Maclaurin, Alexey Radul, Matthew J. Johnson, and Dimitrios In Workload Characterization, 2009. IISWC 2009. IEEE International Vytiniotis. 2019. Dex: array programming with typed indices. In Poster Symposium on. 44–54. https://doi.org/10.1109/IISWC.2009.5306797 ARRAY ’21, June 21, 2021, Virtual, Canada Troels Henriksen and Martin Elsman

session at Workshop on Program Transformations for Machine Learn- [24] John A Stratton, Christopher Rodrigues, I-Jui Sung, Nady Obeid, Li- ing. Associated with NeurIPS ’2019. Wen Chang, Nasser Anssari, Geng Daniel Liu, and Wen-mei W Hwu. [20] Lenore Mullin. 1988. A mathematics of arrays. School of Computer and 2012. Parboil: A revised benchmark suite for scientific and commercial Information Science, Syracuse University. throughput computing. Center for Reliable and High-Performance [21] Amr Sabry and Matthias Felleisen. 1993. Reasoning about Programs in Computing 127 (2012). Continuation-Passing Style. In LISP AND SYMBOLIC COMPUTATION. [25] William W. Tait. 1967. Intensional interpretations of functionals of finite 288–298. type. Journal of symbolic logic 32 (1967), 198–212. [22] Sven-Bodo Scholz. 1994. Single Assignment C - Functional Program- [26] Kai Trojahner and Clemens Grelck. 2009. Dependently typed array ming Using Imperative Style. In In John Glauert (Ed.): Proceedings of programs don’t go wrong. J. Log. Algebr. Program. 78 (08 2009), 643–664. the 6th International Workshop on the Implementation of Functional https://doi.org/10.1016/j.jlap.2009.03.002 Languages. University of East Anglia. [27] Hongwei Xi and Frank Pfenning. 1998. Eliminating Array Bound Check- [23] Justin Slepak, Olin Shivers, and Panagiotis Manolios. 2014. An Array- ing through Dependent Types. In Proceedings of the ACM SIGPLAN 1998 Oriented Language with Static Rank Polymorphism. In Programming Conference on Programming Language Design and Implementation Languages and Systems, Zhong Shao (Ed.). Springer Berlin Heidelberg, (Montreal, Quebec, Canada) (PLDI ’98). Association for Computing Ma- Berlin, Heidelberg, 27–46. chinery, New York, NY, USA, 249–257. https://doi.org/10.1145/277650. 277732