Processing an Intermediate Representation Written in Lisp

Total Page:16

File Type:pdf, Size:1020Kb

Processing an Intermediate Representation Written in Lisp Processing an Intermediate Representation Written in Lisp Ricardo Pena,˜ Santiago Saavedra, Jaime Sanchez-Hern´ andez´ Complutense University of Madrid, Spain∗ [email protected],[email protected],[email protected] In the context of designing the verification platform CAVI-ART, we arrived to the need of deciding a textual format for our intermediate representation of programs. After considering several options, we finally decided to use S-expressions for that textual representation, and Common Lisp for processing it in order to obtain the verification conditions. In this paper, we discuss the benefits of this decision. S-expressions are homoiconic, i.e. they can be both considered as data and as code. We exploit this duality, and extensively use the facilities of the Common Lisp environment to make different processing with these textual representations. In particular, using a common compilation scheme we show that program execution, and verification condition generation, can be seen as two instantiations of the same generic process. 1 Introduction Building a verification platform such as CAVI-ART [7] is a challenging endeavour from many points of view. Some of the tasks require intensive research which can make advance the state of the art, as it is for instance the case of automatic invariant synthesis. Some others are not so challenging, but require a lot of pondering and discussion before arriving at a decision, in order to ensure that all the requirements have been taken into account. A bad or a too quick decision may have a profound influence in the rest of the activities, by doing them either more difficult, or longer, or more inefficient. This is the case of the problem presented in this paper: deciding a textual representation for an otherwise internal representation of the platform. The CAVI-ART platform accepts programs written in a variety of languages, both imperative and functional ones, and potentially it could be extended to more. A carefully chosen intermediate represen- tation common to all of them ensures that most of the platform activities can be performed in a language independent way. The CAVI-ART Intermediate Representation (in what follows IR), introduced in [7], is an internal programming language to which all the source languages supported by CAVI-ART are transformed. The language independent activities of the platform include termination analysis, invari- ant synthesis, verification condition generation, verification condition proving, test case generation, and some other. As we will show, the IR is the confluence point of all the tools. These tools either produce, consume, or transform the IR in different ways. As the tools are implemented in a variety of languages, we decided to use files to communicate the tools between them. This makes the tools more independent of each other, and also gives visibility to the intermediate results. So, we arrived at the point of deciding the format of these files containing IR-transformed programs. As we will see, it has not been an easy decision, given the somewhat contradictory requirements we have for these files. The final decision has been to represent the IR programs as Common Lisp S-expressions [5]. This satisfies all our requirements and also gives us for free a new possibility: that of being able to execute the ∗Work partially funded by the Spanish Ministry of Economy and Competitiveness, under the grant TIN2013-44742-C4-3-R Contribution to: © R. Pena,˜ S. Saavedra, J. Sanchez-Hern´ andez´ PROLE 2016 R. Pena,˜ S. Saavedra, J. Sanchez-Hern´ andez´ 149 a ::= c f constant g j x f variable g be ::= a f atomic expression g j f ai f function/primitive operator application g j haii f tuple construction g j C ai f constructor application g e ::= be f binding expression g j let hxi :: tii = be in e f sequential let. Left part of the binding can be a tuple g j letfun defi in e f recursive let for function definitions g j case a of alti[; ! e] f case distinction with optional default branch g def ::= f (xi :: ti) :: yi :: ti = e f function definition. Output results are named g alt ::= C xi :: ti ! e f case branch g t ::= a f type variable g j T ti f type constructor application g Figure 1: CAVI-ART IR syntax IR. Additionally, the Common Lisp environment provides a set of facilities, such as macro-expansion, evaluation while compiling, and some other, that can make the processing of the IR files a very systematic and uniform task, independently of whether we want to execute it, or to do any other processing. The plan of the paper is as follows: In Sec. 2 we explain the different uses of the IR and all the requirements that these uses pose on the IR-files; then, in Sec. 3 we justify the decision of using Lisp S- expressions; in Sec. 4 we introduce the facilities of the Common Lisp environment and present a common scheme for doing all kind of processing one may wish to do with IR files; as two instantiations of this scheme, we show how to execute an IR file, and how to extract the verification conditions for proving the program correct. A running example is used all across the paper, and in Sec. 5 we show the resulting goals of the second instantiation of the scheme. Finally, Sec. 6 concludes. 2 Contents and Uses of the Intermediate Representation The IR abstract syntax is reproduced in Fig. 1, where the vector notation z is an abbreviation for z1;:::;zn. One of its aims was to achieve minimality, as it was clear for us that the bigger the number of the differ- ent IR constructions, the longer, more difficult, and more error-prone would be the above cited tasks. In this sense, iteration and recursion are unified into a single construction letfun which allows to define sets of mutually recursive functions. The imperative assignments are translated into sequential let expres- sions, and an SSA1 transformation ensures that no variable is assigned more than once. All conditional statements, both of imperative and functional languages, are transformed into case expressions. The rest of the IR consists of function and constructor applications, tuples and atoms. The IR is a functional language, so there is nothing like destructive update, output function arguments, or arguments passed by reference. When a function returns more than one single result, then it returns a tuple. An atom is either a literal constant or a variable, and in several places of the IR atoms are mandatory. For instance, in actual arguments of an application. These restrictions make it easier the activities related to verification condition generation and proving. For a broader motivation of the decisions leading to the IR, see [7]. 1 Static Single Assignment. See for instance [1, Chap. 19]. 150 Processing an Intermediate Representation Written in Lisp quicksort (v :: array int; n :: int) :: (vres :: array int) = fQ : n = length(v)g fR : sorted(vres) ^ permut all(v;vres)g v o i d q s o r t ( i n t v [ ] , i n t i , i n t j )f letfun // Pre:0 <=i <=j < l e n g t h(v) qsort (v :: array int; i :: int; j :: int) :: (vres :: array int) = i n t p ; fQ1 : 0 ≤ i ≤ j < length(v)g fR1 : sorted sub(vres;i; j + 1) ^ permut sub(v;vres;i; j + 1)g i f ( i <j )f let (b :: bool) = <(i; j) in partition(v,i ,j,p); case b of (true bool) ! f (v;i; j) qsort(v,i ,p −1); 1 qsort(v,p+1,j ); (false :: bool) ! v g // Post: sorted s u b(vres,i,j+1) && f (v :: array int; i :: int; j :: int) :: (vres :: array int) = // permut sub(v,vres,i,j+1) 1 let (v :: array int; p :: int) = partition(v;i; j) in g 1 let (p1 :: int) = -(p;1) in v o i d q u i c k S o r t ( i n t v [ ] , i n t n )f let (v2 :: array int) = qsort(v1;i; p1) in // Pre:n= length(v) let (p2 :: int) = +(p;1) in qsort(v,0,n −1); qsort(v2; p2; j) // Post: sorted(vres) && in // permut a l l(v,vres) let (n1 :: int) = -(n;1) in g qsort(v;0;n1) (a) Original C++ version (b) Transformed to the CAVI-ART IR Figure 2: The quickSort algorithm we will use as running example We will use the algorithm quicksort as a running example in this paper. Fig. 2a shows a source ver- sion in C++. Once transformed into the IR, it has adopted the form shown in Fig. 2b. Preconditions and postconditions are logic formulas which may use atomic predicates introduced in externally axiomatized theories. In the example, we use the predicates sorted, permut for checking the corresponding proper- ties on the full length of vectors, and sorted sub, permut sub for reasoning about subvectors. These predicates are part of Why3 [3], which is the underlying platform that CAVI-ART uses for discharging formulas. The IR is strongly typed in a polymorphic Hindley–Milner-like type system. After the transformation from the source language (either by translating the user given types, or by inferring them), the IR is assumed to be annotated with types. In Fig. 1 it can be seen that types are included at all the defining occurrences of variables, i.e. in the formal arguments and results of function definitions, in the let-bound variables, and in the case-bound variables introduced by pattern-matching. The syntax of types is also shown in Fig.
Recommended publications
  • Rubicon: Bounded Verification of Web Applications
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by DSpace@MIT Rubicon: Bounded Verification of Web Applications Joseph P. Near, Daniel Jackson Computer Science and Artificial Intelligence Lab Massachusetts Institute of Technology Cambridge, MA, USA {jnear,dnj}@csail.mit.edu ABSTRACT ification language is an extension of the Ruby-based RSpec We present Rubicon, an application of lightweight formal domain-specific language for testing [7]; Rubicon adds the methods to web programming. Rubicon provides an embed- quantifiers of first-order logic, allowing programmers to re- ded domain-specific language for writing formal specifica- place RSpec tests over a set of mock objects with general tions of web applications and performs automatic bounded specifications over all objects. This compatibility with the checking that those specifications hold. Rubicon's language existing RSpec language makes it easy for programmers al- is based on the RSpec testing framework, and is designed to ready familiar with testing to write specifications, and to be both powerful and familiar to programmers experienced convert existing RSpec tests into specifications. in testing. Rubicon's analysis leverages the standard Ruby interpreter to perform symbolic execution, generating veri- Rubicon's automated analysis comprises two parts: first, fication conditions that Rubicon discharges using the Alloy Rubicon generates verification conditions based on specifica- Analyzer. We have tested Rubicon's scalability on five real- tions; second, Rubicon invokes a constraint solver to check world applications, and found a previously unknown secu- those conditions. The Rubicon library modifies the envi- rity bug in Fat Free CRM, a popular customer relationship ronment so that executing a specification performs symbolic management system.
    [Show full text]
  • An Optimized R5RS Macro Expander
    An Optimized R5RS Macro Expander Sean Reque A thesis submitted to the faculty of Brigham Young University in partial fulfillment of the requirements for the degree of Master of Science Jay McCarthy, Chair Eric Mercer Quinn Snell Department of Computer Science Brigham Young University February 2013 Copyright c 2013 Sean Reque All Rights Reserved ABSTRACT An Optimized R5RS Macro Expander Sean Reque Department of Computer Science, BYU Master of Science Macro systems allow programmers abstractions over the syntax of a programming language. This gives the programmer some of the same power posessed by a programming language designer, namely, the ability to extend the programming language to fit the needs of the programmer. The value of such systems has been demonstrated by their continued adoption in more languages and platforms. However, several barriers to widespread adoption of macro systems still exist. The language Racket [6] defines a small core of primitive language constructs, including a powerful macro system, upon which all other features are built. Because of this design, many features of other programming languages can be implemented through libraries, keeping the core language simple without sacrificing power or flexibility. However, slow macro expansion remains a lingering problem in the language's primary implementation, and in fact macro expansion currently dominates compile times for Racket modules and programs. Besides the typical problems associated with slow compile times, such as slower testing feedback, increased mental disruption during the programming process, and unscalable build times for large projects, slow macro expansion carries its own unique problems, such as poorer performance for IDEs and other software analysis tools.
    [Show full text]
  • The Evolution of Lisp
    1 The Evolution of Lisp Guy L. Steele Jr. Richard P. Gabriel Thinking Machines Corporation Lucid, Inc. 245 First Street 707 Laurel Street Cambridge, Massachusetts 02142 Menlo Park, California 94025 Phone: (617) 234-2860 Phone: (415) 329-8400 FAX: (617) 243-4444 FAX: (415) 329-8480 E-mail: [email protected] E-mail: [email protected] Abstract Lisp is the world’s greatest programming language—or so its proponents think. The structure of Lisp makes it easy to extend the language or even to implement entirely new dialects without starting from scratch. Overall, the evolution of Lisp has been guided more by institutional rivalry, one-upsmanship, and the glee born of technical cleverness that is characteristic of the “hacker culture” than by sober assessments of technical requirements. Nevertheless this process has eventually produced both an industrial- strength programming language, messy but powerful, and a technically pure dialect, small but powerful, that is suitable for use by programming-language theoreticians. We pick up where McCarthy’s paper in the first HOPL conference left off. We trace the development chronologically from the era of the PDP-6, through the heyday of Interlisp and MacLisp, past the ascension and decline of special purpose Lisp machines, to the present era of standardization activities. We then examine the technical evolution of a few representative language features, including both some notable successes and some notable failures, that illuminate design issues that distinguish Lisp from other programming languages. We also discuss the use of Lisp as a laboratory for designing other programming languages. We conclude with some reflections on the forces that have driven the evolution of Lisp.
    [Show full text]
  • HOL-Testgen Version 1.8 USER GUIDE
    HOL-TestGen Version 1.8 USER GUIDE Achim Brucker, Lukas Brügger, Abderrahmane Feliachi, Chantal Keller, Matthias Krieger, Delphine Longuet, Yakoub Nemouchi, Frédéric Tuong, Burkhart Wolff To cite this version: Achim Brucker, Lukas Brügger, Abderrahmane Feliachi, Chantal Keller, Matthias Krieger, et al.. HOL-TestGen Version 1.8 USER GUIDE. [Technical Report] Univeristé Paris-Saclay; LRI - CNRS, University Paris-Sud. 2016. hal-01765526 HAL Id: hal-01765526 https://hal.inria.fr/hal-01765526 Submitted on 12 Apr 2018 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. R L R I A P P O R HOL-TestGen Version 1.8 USER GUIDE T BRUCKER A D / BRUGGER L / FELIACHI A / KELLER C / KRIEGER M P / LONGUET D / NEMOUCHI Y / TUONG F / WOLFF B D Unité Mixte de Recherche 8623 E CNRS-Université Paris Sud-LRI 04/2016 R Rapport de Recherche N° 1586 E C H E R C CNRS – Université de Paris Sud H Centre d’Orsay LABORATOIRE DE RECHERCHE EN INFORMATIQUE E Bâtiment 650 91405 ORSAY Cedex (France) HOL-TestGen 1.8.0 User Guide http://www.brucker.ch/projects/hol-testgen/ Achim D. Brucker Lukas Brügger Abderrahmane Feliachi Chantal Keller Matthias P.
    [Show full text]
  • Exploring Languages with Interpreters and Functional Programming Chapter 11
    Exploring Languages with Interpreters and Functional Programming Chapter 11 H. Conrad Cunningham 24 January 2019 Contents 11 Software Testing Concepts 2 11.1 Chapter Introduction . .2 11.2 Software Requirements Specification . .2 11.3 What is Software Testing? . .3 11.4 Goals of Testing . .3 11.5 Dimensions of Testing . .3 11.5.1 Testing levels . .4 11.5.2 Testing methods . .6 11.5.2.1 Black-box testing . .6 11.5.2.2 White-box testing . .8 11.5.2.3 Gray-box testing . .9 11.5.2.4 Ad hoc testing . .9 11.5.3 Testing types . .9 11.5.4 Combining levels, methods, and types . 10 11.6 Aside: Test-Driven Development . 10 11.7 Principles for Test Automation . 12 11.8 What Next? . 15 11.9 Exercises . 15 11.10Acknowledgements . 15 11.11References . 16 11.12Terms and Concepts . 17 Copyright (C) 2018, H. Conrad Cunningham Professor of Computer and Information Science University of Mississippi 211 Weir Hall P.O. Box 1848 University, MS 38677 1 (662) 915-5358 Browser Advisory: The HTML version of this textbook requires a browser that supports the display of MathML. A good choice as of October 2018 is a recent version of Firefox from Mozilla. 2 11 Software Testing Concepts 11.1 Chapter Introduction The goal of this chapter is to survey the important concepts, terminology, and techniques of software testing in general. The next chapter illustrates these techniques by manually constructing test scripts for Haskell functions and modules. 11.2 Software Requirements Specification The purpose of a software development project is to meet particular needs and expectations of the project’s stakeholders.
    [Show full text]
  • The Semantics of Syntax Applying Denotational Semantics to Hygienic Macro Systems
    The Semantics of Syntax Applying Denotational Semantics to Hygienic Macro Systems Neelakantan R. Krishnaswami University of Birmingham <[email protected]> 1. Introduction body of a macro definition do not interfere with names oc- Typically, when semanticists hear the words “Scheme” or curring in the macro’s arguments. Consider this definition of and “Lisp”, what comes to mind is “untyped lambda calculus a short-circuiting operator: plus higher-order state and first-class control”. Given our (define-syntax and typical concerns, this seems to be the essence of Scheme: it (syntax-rules () is a dynamically typed applied lambda calculus that sup- ((and e1 e2) (let ((tmp e1)) ports mutable data and exposes first-class continuations to (if tmp the programmer. These features expose a complete com- e2 putational substrate to programmers so elegant that it can tmp))))) even be characterized mathematically; every monadically- representable effect can be implemented with state and first- In this definition, even if the variable tmp occurs freely class control [4]. in e2, it will not be in the scope of the variable definition However, these days even mundane languages like Java in the body of the and macro. As a result, it is important to support higher-order functions and state. So from the point interpret the body of the macro not merely as a piece of raw of view of a working programmer, the most distinctive fea- syntax, but as an alpha-equivalence class. ture of Scheme is something quite different – its support for 2.2 Open Recursion macros. The intuitive explanation is that a macro is a way of defining rewrites on abstract syntax trees.
    [Show full text]
  • Beginner's Luck
    Beginner’s Luck A Language for Property-Based Generators Leonidas Lampropoulos1 Diane Gallois-Wong2,3 Cat˘ alin˘ Hrit¸cu2 John Hughes4 Benjamin C. Pierce1 Li-yao Xia2,3 1University of Pennsylvania, USA 2INRIA Paris, France 3ENS Paris, France 4Chalmers University, Sweden Abstract An appealing feature of QuickCheck is that it offers a library Property-based random testing a` la QuickCheck requires build- of property combinators resembling standard logical operators. For ing efficient generators for well-distributed random data satisfying example, a property of the form p ==> q, built using the impli- complex logical predicates, but writing these generators can be dif- cation combinator ==>, will be tested automatically by generating ficult and error prone. We propose a domain-specific language in valuations (assignments of random values, of appropriate type, to which generators are conveniently expressed by decorating pred- the free variables of p and q), discarding those valuations that fail icates with lightweight annotations to control both the distribu- to satisfy p, and checking whether any of the ones that remain are tion of generated values and the amount of constraint solving that counterexamples to q. happens before each variable is instantiated. This language, called QuickCheck users soon learn that this default generate-and-test Luck, makes generators easier to write, read, and maintain. approach sometimes does not give satisfactory results. In particular, We give Luck a formal semantics and prove several fundamen- if the precondition p is satisfied by relatively few values of the ap- tal properties, including the soundness and completeness of random propriate type, then most of the random inputs that QuickCheck generation with respect to a standard predicate semantics.
    [Show full text]
  • Quickcheck Tutorial
    QuickCheck Tutorial Thomas Arts John Hughes Quviq AB Queues Erlang contains a queue data structure (see stdlib documentation) We want to test that these queues behave as expected What is “expected” behaviour? We have a mental model of queues that the software should conform to. Erlang Exchange 2008 © Quviq AB 2 Queue Mental model of a fifo queue …… first last …… Remove from head Insert at tail / rear Erlang Exchange 2008 © Quviq AB 3 Queue Traditional test cases could look like: We want to check for arbitrary elements that Q0 = queue:new(), if we add an element, Q1 = queue:cons(1,Q0), it's there. 1 = queue:last(Q1). Q0 = queue:new(), We want to check for Q1 = queue:cons(8,Q0), arbitrary queues that last added element is Q2 = queue:cons(0,Q1), "last" 0 = queue:last(Q2), Property is like an abstraction of a test case Erlang Exchange 2008 © Quviq AB 4 QuickCheck property We want to know that for any element, when we add it, it's there prop_itsthere() -> ?FORALL(I,int(), I == queue:last( queue:cons(I, queue:new()))). Erlang Exchange 2008 © Quviq AB 5 QuickCheck property We want to know that for any element, when we add it, it's there prop_itsthere() -> ?FORALL(I,int(), I == queue:last( queue:cons(I, queue:new()))). This is a property Test cases are generasted from such properties Erlang Exchange 2008 © Quviq AB 6 QuickCheck property We want to know that for any element, when we add it, it's there int() is a generator for random integers prop_itsthere() -> ?FORALL(I,int(), I == queue:last( queue:cons(I, I represents one queue:new()))).
    [Show full text]
  • What Macros Are and How to Write Correct Ones
    What Macros Are and How to Write Correct Ones Brian Goslinga University of Minnesota, Morris 600 E. 4th Street Morris, Minnesota 56267 [email protected] ABSTRACT such as Java and Scheme. There arise situations in most Macros are a powerful programming construct found in some high-level languages where code falls short of this ideal by programming languages. Macros can be thought of a way having redundancy and complexity not inherent to the prob- to define an abbreviation for some source code by providing lem being solved. Common abstraction mechanisms, such as a program that will take the abbreviated source code and functions and objects, are often unable to solve the problem return the unabbreviated version of it. In essence, macros [6]. Languages like Scheme provide macros to handle these enable the extension of the programming language by the situations. By using macros when appropriate, code can be programmer, thus allowing for compact programs. declarative, and thus easier to read, write, and understand. Although very powerful, macros can introduce subtle and Macros can be thought of as providing an abbreviation for hard to find errors in programs if the macro is not written common source code in a program. In English, an abbrevi- correctly. Some programming languages provide a hygienic ation (UMM) only makes sense if one knows how to expand macro system to ensure that macros written in that pro- it into what it stands for (University of Minnesota, Morris). gramming language do not result in these sort of errors. Likewise, macros undergo macro expansion to derive their In a programming language with a hygienic macro system, meaning.
    [Show full text]
  • Branching Processes for Quickcheck Generators (Extended Version)
    Branching Processes for QuickCheck Generators (extended version) Agustín Mista Alejandro Russo John Hughes Universidad Nacional de Rosario Chalmers University of Technology Chalmers University of Technology Rosario, Argentina Gothenburg, Sweden Gothenburg, Sweden [email protected] [email protected] [email protected] Abstract random test data generators or QuickCheck generators for short. In QuickCheck (or, more generally, random testing), it is chal- The generation of test cases is guided by the types involved in lenging to control random data generators’ distributions— the testing properties. It defines default generators for many specially when it comes to user-defined algebraic data types built-in types like booleans, integers, and lists. However, when (ADT). In this paper, we adapt results from an area of mathe- it comes to user-defined ADTs, developers are usually required matics known as branching processes, and show how they help to specify the generation process. The difficulty is, however, to analytically predict (at compile-time) the expected number that it might become intricate to define generators so that they of generated constructors, even in the presence of mutually result in a suitable distribution or enforce data invariants. recursive or composite ADTs. Using our probabilistic formu- The state-of-the-art tools to derive generators for user- las, we design heuristics capable of automatically adjusting defined ADTs can be classified based on the automation level probabilities in order to synthesize generators which distribu- as well as the sort of invariants enforced at the data genera- tions are aligned with users’ demands. We provide a Haskell tion phase. QuickCheck and SmallCheck [27] (a tool for writing implementation of our mechanism in a tool called generators that synthesize small test cases) use type-driven and perform case studies with real-world applications.
    [Show full text]
  • Foundational Property-Based Testing
    Foundational Property-Based Testing Zoe Paraskevopoulou1,2 Cat˘ alin˘ Hrit¸cu1 Maxime Den´ es` 1 Leonidas Lampropoulos3 Benjamin C. Pierce3 1Inria Paris-Rocquencourt 2ENS Cachan 3University of Pennsylvania Abstract Integrating property-based testing with a proof assistant creates an interesting opportunity: reusable or tricky testing code can be formally verified using the proof assistant itself. In this work we introduce a novel methodology for formally verified property-based testing and implement it as a foundational verification framework for QuickChick, a port of QuickCheck to Coq. Our frame- work enables one to verify that the executable testing code is testing the right Coq property. To make verification tractable, we provide a systematic way for reasoning about the set of outcomes a random data generator can produce with non-zero probability, while abstracting away from the actual probabilities. Our framework is firmly grounded in a fully verified implementation of QuickChick itself, using the same underlying verification methodology. We also apply this methodology to a complex case study on testing an information-flow control abstract machine, demonstrating that our verification methodology is modular and scalable and that it requires minimal changes to existing code. 1 Introduction Property-based testing (PBT) allows programmers to capture informal conjectures about their code as executable specifications and to thoroughly test these conjectures on a large number of inputs, usually randomly generated. When a counterexample is found it is shrunk to a minimal one, which is displayed to the programmer. The original Haskell QuickCheck [19], the first popular PBT tool, has inspired ports to every mainstream programming language and led to a growing body of continuing research [6,18,26,35,39] and industrial interest [2,33].
    [Show full text]
  • Specification Based Testing with Quickcheck
    Specification Based Testing with QuickCheck (Tutorial Talk) John Hughes Chalmers University of Technology and QuviQ AB ABSTRACT QuickCheck is a tool which tests software against a formal specification, reporting discrepancies as minimal failing examples. QuickCheck uses properties specified by the developer both to generate test cases, and to identify failing tests. A property such as 8xs : list(int()): reverse(reverse(xs)) = xs is tested by generating random lists of integers, binding them to the variable xs, then evaluating the boolean expression under the quantifier and reporting a failure if the value is false. If a failing test case is found, QuickCheck “shrinks” it by searching for smaller, but similar test cases that also fail, terminating with a minimal example that cannot be shrunk further. In this example, if the developer accidentally wrote 8xs : list(int()): reverse(xs) = xs instead, then QuickCheck would report the list [0; 1] as the minimal failing case, containing as few list elements as possible, with the smallest absolute values possible. The approach is very practical: QuickCheck is implemented just a library in a host programming language; it needs only to execute the code under test, so requires no tools other than a compiler (in particular, no static analysis); the shrinking process “extracts the signal from the noise” of random testing, and usually results in very easy-to-debug failures. First developed in Haskell by Koen Claessen and myself, QuickCheck has been emulated in many programming languages, and in 2006 I founded QuviQ to develop and market an Erlang version. Of course, customers’ code is much more complex than the simple reverse function above, and requires much more complex properties to test it.
    [Show full text]