Clean Property-based Contract Tests Dr. Frank Raiser Konzept Informationssysteme GmbH Overview nProperty-based Testing Predicate Logic, Contracts, Invariants How Property-based Testing works nProperty-based Contracts Definition and Application of contract tests nQuality Assurance Detect untested contracts 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 2 Properties – Predicate Logic nequals implementation should be Reflexive, Symmetric, Transitive, Consistent, Null-safe F.ex. Symmetry: ∀�∀�. (� ≠ ���� ∧ � ≠ ����) → ������ �, � ↔ ������ �, � Predicate logic Equals is a uses ∧ for logical predicate that and (&&) needs to be satisfied. 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 3 Properties – Contracts nInterfaces define contracts nMany prominent examples in Java Comparator#compare, Comparable#compareTo List#contains, List#remove nNo choice – you have to agree to that contract java.lang.IllegalArgumentException: Comparison method violates its general contract! How do you test these contracts? 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 4 Example Scenario: Premium Calculation nCalculation of the premium factor for a car insurance Adapted from “Developer Testing” by Alexander Tarlinder [ISBN-13: 978-0-13-429106-2] nPotential indicators: Younger persons are more likely to have accidents Female drivers are less likely to claim insurance 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 5 Example model Interface and different implementations for premium factors Represents someone who can request an insurance 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 6 Standard Unit Testing 1. Create a “test” person Factor == 1.75 2. Run the calculator Factor == 1.0 AgePremiumCalculator 3. Verify the correct factor Factor == 1.35 4. Repeat 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 7 Standard Unit Testing – Limited Sample Space What new Person(Gender.FEMALE, new Age(19)) about these? new Person(Gender.FEMALE, new Age(35)) new Person(Gender.FEMALE, new Age(73)) 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 8 Property-Based Testing with junit-quickcheck nElderly drivers should get a premium factor of 1.35 ∀�. � �� � ������ ∧ �. ��� ≥ 60 → ��������������� � = 1.35 Factor == 1.35 AgePremium 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH Calculator 9 Generators nHow to get a Person ? nGenerators do that Configurable selection of appropriate generator Configuration of generator Generators can be straightforward possible as well (f.ex. only female persons) 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 10 junit-quickcheck – Background Language Library License C Theft ISC License https://github.com/silentbicycle/theft Clojure test.check Eclipse Public License https://github.com/clojure/test.check Erlang Triq – Trifork QuickCheck for Erlang Apache 2.0 https://github.com/krestenkrab/triq Haskell QuickCheck BSD 3 https://hackage.haskell.org/package/QuickCheck Java junit-quickcheck MIT https://github.com/pholser/junit-quickcheck Javascript qc.js Revised BSD https://bitbucket.org/darrint/qc.js .NET FsCheck Revised BSD https://github.com/fscheck/FsCheck Python factcheck Apache 2.0 https://github.com/npryce/python-factcheck Scala ScalaCheck Revised BSD http://scalacheck.org/ 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 11 Possibilities and Limitations nMany tests with little code nRandom input data nBetter readability Can be reproduced (seed) no guarantee for edge-cases Property explicitly formulated Fewer tests needed nGenerators needed No object construction code can also get complex nReusable generators nRuntime penalty Falsifications tried 100 times 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 12 Example: combining calculators Product of base class factor and other calculators’ factors 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 13 Contracts Standard PremiumCalculator Factor == 1.925 Factor == 1.1 (male 1.1 * young 1.75) (male) nVarious factors involved, then a new business rule: All factors must be within the range of 0.5 and 2.0 ‒ Essentially a contract on the PremiumCalculator interface 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 14 Property-based Contracts Codifies contractual agreement for all interface implementations 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 15 Clean Property-based Contracts How much effort and code is it to Explicit verify the adherence contract for an to contract implementation class? Provides test subject Sufficient for 100 test cases against the contract! 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 16 Liskov Substitution Principle nBasePremiumCalculator should satisfy the contract nLSP requires derived classes to adhere to contract Can be easily tested now But: What if a developer forgot to add the contract test? 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 17 Validate Define contracts contract adherence jQAssistant is a QA tool which allows the definition and validation of project specific rules on a structural level. [Source: jqAssistant docs] Define LSP rules 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 18 Taking quality assurance even further jqassistant any Continuous Integration system Analyze source Detect contracts Execute tests to Detect verify all contracts classes for for all contracts implementations Ensure Test classes exist Ensure contract is 2017-06-22 implemented Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 19 Contract Definition Neo4j’s Cypher Interface query syntax must end in “Contract” and have some @Property Mark as a contract * The precise rules may need to be Create tweaked in project- relationship to specific ways (f.ex. If you want a base class the type the for contracts instead contract of interface w/ applies to default methods) 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 20 Contract Adherence Constraint Query types for which contract applies and their test classes Make sure test class adheres to the contract 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 21 Contract Testing Does not implement PremiumCalculatorConcept 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 22 Summary nExplicit contract definitionsnEffort to create generators nTiny amount of code nLarge amount of tests nReusable generators being executed nSafety-net against nQuality (dependent on accidentally ignoring generators and contracts properties) may be misleading 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 23 Clean Property-based Contract Tests Konzept Informationssysteme GmbH Pfarrer-Weiss-Weg 12 89073 Ulm Dr. Frank Raiser Software Entwicklungsingenieur Tel.: +49 731 1403434 – 51 Fax.: +49 731 1403434 – 34 [email protected] www.konzept-is.de Thank you Special thanks to pholser for extending junit-quickcheck to make this work Paul Holser [All images Public Domain.] 2017-06-22 Clean Property-based Contract Tests © 2017 Konzept Informationssysteme GmbH 24.