Functional Pearl: Zero-Cost, Meta-Programmed, Dependently-Typed Stateful Functors in F★ JONATHAN PROTZENKO, Microsoft Research, USA SON HO, Inria, France Writing code is hard; proving it correct is even harder. As the scale of verified software projects reaches new heights, the problem of efficiently verifying large amounts of software becomes more and more salient. Nowhere is this issue more evident than in the context of verified cryptographic libraries. To achieve feature- parity and be competitive with unverified cryptographic libraries, a very large number of algorithms and APIs need to be verified. However, the task is oftentimes repetitive, and factoring out commonality between algorithms is fraught with difficulties, requiring until now a significant amount of manual effort. This paper shows how a judicious combination of known functional programming techniques leads to an ★ order-of-magnitude improvement in the amount of verified code produced by the popular HACL crypto- graphic library, without compromising performance. We review three techniques that build upon each other, in order of increasing sophistication. First, we use dependent types to crisply capture the specification and state machine of a block algorithm, a cryptographic notion that was until now only informally and impre- cisely specified. Next, we rely on partial evaluation to author a higher-order, stateful functor that transforms any unsafe block API into a safe counterpart. Finally, we rely on elaborator reflection to automate the very process of authoring a functor, using a code-rewriting tactic. This culminates in a style akin to templatized C++ code, but relying on a userland tactic and partial evaluation, rather than built-in compiler support. ★ We demonstrate our techniques in the F programming language, and show that they require no change in ★ the F compiler whatsoever. Our techniques require no user intervention either; the proofs are carried once and for all, and users can instantiate our higher-order functors without incurring any proof obligations. A distinguishing feature of our approach is that rather than introducing a novel extraction pipeline or incorpo- rating program synthesis techniques, we instead rely on the established Letouzey-style erasure and extraction ★ pipeline of F , with an extra early meta-programming stage added. Our encoding of higher-order functors ★ in F thus provides the programmer with a very high level of abstraction while incurring no run-time costs thanks to careful partial evaluation. Our approach is in no way specific to cryptography, and as such, this paper requires no cryptographic ★ knowledge. It simply turns out that the existing HACL library provides a grand challenge in terms of com- plexity; an ideal playground to evaluate the effectiveness of our techniques; and a concrete setting to identify ★ patterns of verifying in the large. We have contributed six higher-order functors to the HACL library, for arXiv:2102.01644v2 [cs.PL] 9 Jun 2021 a total of nearly 40 specialized functor applications. Individually verifying each one of these 40 algorithms would have been impossible; our method thus significantly improves programmer productivity while provid- ing a much higher level of assurance. 1 CRYPTOGRAPHIC VERIFICATION, IN THE LARGE Recent years have witnessed a flurry of activity in the field of formally verifying cryptography [11]. There are multiple reasons for this enthusiasm: by virtue of being input-output functions, many Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. ,, © 2018 Association for Computing Machinery. ACM ISBN 978-1-4503-XXXX-X/18/06...$15.00 https://doi.org/10.1145/1122445.1122456 1 ,, Jonathan Protzenko and Son Ho core cryptographic primitives lend themselves well to verification; implementors can hardly af- ford to ship incorrect cryptography anymore, leading to an above-average level of enthusiasm for verified artifacts in industry; and practitioners no longer need to fear functional languages: the output of recent verification pipelines directly generates C or assembly, which are known, famil- iar languages that can be readily integrated into the existing software ecosystem. Projects such as FiatCrypto [24], Jasmin [4, 5], CryptoLine [26], Vale-Crypto [16, 25] or HACL★ [37, 46] em- body this new reality. They all have demonstrated that is it now feasible to verify cryptographic algorithms whose performance matches or exceeds state-of-the-art, unverified code from projects like OpenSSL [35] or libsodium [1]. These projects all produce verified C or assembly code. Build- ing upon such efforts, projects like EverCrypt [38] offer entire verified cryptographic providers, which expose a wide range of modern algorithms, offering an agile and multiplexing API via CPU auto-detection. But beyond the initial successes and flagship verified algorithms lies a long tail of oftentimes tedious work required to meet the feature set of existing cryptographic libraries. For instance, authors of a verified library might be required to provide several nearly-identical, specialized vari- ants of the same algorithm to deal with different processor instruction sets. As another example, an industrial-grade cryptographic library might be required to provide several copies of a high-level API for algorithms that only differ in minor ways. Such tasks are repetitive; provide few novel insights each time a new piece of verified code is added; and make maintenance of the verified code base harder as it grows bigger. In our view, this highlights the stark need for language-based techniques to perform verifica- tion in the large [22]. That is, we envision for the programmer to operate at a very high-level of abstraction, composing high-level building blocks with a minimal amount of effort. We wish for all commonality to be factored out as much as possible, resulting in a minimal amount of verified code for a maximal amount of generated code. But more importantly, we want to make verification engineers’ lives better, while still producing the same optimized, low-level, first-order C code that practitioners have come to embrace. This paper presents our answer to the challenge of scaling up cryptographic verification, in the context of the HACL★ cryptographic library. With dependent types, we characterize previously fuzzy notions such as that of a block algorithm. With meta-programming, we encode higher-order stateful functors in F★. With elaborator reflection [19], we automate the process of writing a func- tor, thanks to a tactic that automatically generates a functor via syntax inspection and term injec- tion. We require no modifications to the F★ compiler, meaning that the trusted computing base remains unchanged. The technical ingredients are not new; but the key contribution of this paper is to show how their judicious combination allows scaling up verification, while bringing greater modularity and clarity to the verified codebase. This work was performed using the F★ programming language, and its Low★ toolchain that compiles a subset of F★ to C. Our techniques were contributed and integrated to HACL★. They form the language-based foundation of the most recent iteration of HACL★ [37], which represents a complete overhaul of the previous version [46]. Until now, these techniques had never been described in detail. All throughout the paper, we use cryptography as the main driving force. There are two reasons. First, cryptographic APIs offer the highest level of complexity, combining mutable state, state ma- chines, abstract representations and abstract predicates. While our techniques have been success- fully applied to simpler use-cases, e.g. parameterized data structures and associative lists, we wish to showcase the full power of our approach. Second, we benefit from the large existing codebase of cryptographic code in HACL★, meaning we can perform a real-world quantitative evaluation of how our techniques make verification more scalable. 2 Functional Pearl: Zero-Cost, Meta-Programmed, Dependently-Typed Stateful Functors in F★ ,, Our work is motivated by two very concrete issues previously faced by the HACL★ codebase. We now succinctly describe both problems, and highlight how (automated) higher-order stateful functors provide a technical answer. Unsafe block APIs. Verified cryptography has been successfully integrated in flagship software projects, such as Google Chrome, LibreSSL, Firefox or Linux. Unfortunately, most of the integra- tions so far have been piecewise, wherein a fragment of an algorithm, or a single algorithm at a time gets replaced with a verified version. We have yet to see an entire cryptographic library, such as Mozilla’s NSS, being swapped out for a verified library. One of the reasons is, when authoring verified code, it does not suffice to merely implement an algorithm as specified by the RFC; the author of verified code must strive for elegant, intuitive, easy-to-use APIs that minimize the risk of programmer error. Designing such a safe API requires more layers of abstraction, which increases the verification burden. As