Make Your Testing Groovy

groovy -

tests -

/make asert/MakeTestingGroovy - paulk_asert Make your testingGroovy your Make paulk_asert http:/slideshare.net/ https://github.com/paulk Groovy Lead for Object Computing Inc. Computing Object Groovy for Lead @ Dr Paul King Dr Paul

• … but great second question? 

- 2016

• Consider first the task at hand and the organization and people fit! “People Fit” / “Organization Fit” • People – Developers (familiarity with languages) – Testers (tools language familiarity) – Responsibility to create/run/maintain

– BAs/SMEs (ability to read technical tests) © ASERT 2006 ASERT ©

– Consider both development and maintenance -

2016 – Expected feedback from tests (JUnit/Reports) • Organization – Maturity level – Degree of automation – Tools in use – Need for adaptability Why Test With Groovy? • Advantages – Easy to learn – Particularly good for Java shops – Easy to plug and play different testing tools

– Good community & tools for professional agile © ASERT 2006 ASERT ©

• Disadvantages - 2016 – Requires JVM – Less advantages if your developers using Python, .Net, PHP – Maybe your testers already know Ruby Scripting/Dynamic Languages • Advantages – Lend themselves to succinct code/DSLs – Powerful – Increased Refactoring

– Increased Reuse © ASERT 2006 ASERT ©

– Less prone to Brittleness -

2016 – Flexibility for tool integration – Open APIs provide extensibility • Disadvantages – Can be too low level – Sometimes less tool support Test Characteristics… • Coverage/Traceability – Requirement coverage/traceability – Code coverage: line, branch, path, state – Transactional Tracing

– Unit, Integration, System, Customer

- 2016 • Manageability – Removing duplication – Managing lifecycle: shared setUp/tearDown (@Before @After) …Test Characteristics • Handling test data – Data-driven, databases, Spreadsheets, tables, keywords, random, model-driven – Large Test Volumes

• Tool evolution, longevity, cost, - 2016 support, documentation – Open Source/Commercial, Critical mass/popularity • Combinability Key Testing Practices… • Use testing DSL’s • Look to move up the testing stack – It used to be all about the driver – Now the driver is hidden in the

• Apply good testing practices

- 2016 – Pareto analysis, bug clusters, mutation testing, test early, all pairs/equivalence partitions/orthogonal array testing, risk-based test selection, coding for testability, use CI, boundary value analysis, defensive programming …Key Testing Practices • Plug and play testing tools – Run different drivers with different runners and different tools • Complement automated tests with

• Expand testing scope

- 2016 – Test environment readiness, test deployments

Groovy and Testing Tool Spectrum*

- 2016

* Tools/libraries/frameworks don't always neatly fall into one category – still useful conceptually time - class class FP) st metaprogramming boiler plate code plate boiler + runtime & compile & + runtime (DSLs) grammar + flexible & GDK library + scripting – (1 + closures type system + extensible What is Groovy? is What

Groovy = Java Groovy = Java

extensibility.” provides great flexibility and flexibility great provides adds productivity features and features productivity adds It leverages Java features but features Java It leverages “Groovy is like a super version of Java. of version a super like is “Groovy

System.out.println("Hello, World!"); // supports Java syntax println 'Hello, World!' // but can remove some syntax

String name = 'Guillaume' // Explicit typing/awareness println "$name, I'll get the car." // Gstring (interpolation)

def longer = """${name}, the car

is in the next row.""" // multi-line, implicit type © ASERT 2006 ASERT © assert 0.5 == 1/2 // BigDecimal equals()

assert 0.1 + 0.2 == 0.3 // and arithmetic - 2016 def printSize(obj) { // implicit/duck typing print obj?.size() // safe dereferencing }

def pets = ['ant', 'bee', 'cat'] // literal list syntax pets.each { pet -> // closure support assert pet < 'dog' // overloading '<' on String } // or: for (pet in pets)... Target audience • Testing for developers – Organising tests – Running tests – Data generation – Mocking – What can be tested – Code coverage • Testing beyond developers – Structuring – Better reporting – Data-driven – Domain specific language (DSL) – Test coverage Testing Frameworks • None • JUnit 3 • JUnit 4 • JUnit 5 • TestNG • Spock Testing Frameworks • None ??? • JUnit 3 • JUnit 4 • JUnit 5 • TestNG • Spock Testing Frameworks • None ??? – Groovy deems testing so important that it comes with built-in testing: – Built-in asserts, mocks – Built-in JUnit 3 GroovyTestCase and utilities – Built-in runners for tests – Bundled JUnit 4 • JUnit 3 • JUnit 4 • JUnit 5 • TestNG • Spock Built-in assertions

class Converter { static celsius (fahrenheit) { (fahrenheit - 32) * 5 / 9 }

}

2016 import static Converter.celsius

assert 20 == celsius(68) assert 35 == celsius(95) assert -17 == celsius(0).toInteger() assert 0 == celsius(32) Built-in assertions • But what about errors?

a friendly description of what went wrong

- 2016 Built-in assertions • But what about errors?

a friendly description of what went wrong

- 2016 GroovyTestCase (extends JUnit 3)

import org.junit.Assert import static Converter.celsius import static org.junit.Assert.assertEquals

class SimpleGroovyTest extends GroovyTestCase {

assertEquals("Should convert freezing", 0.0, celsius(32.0)) - 2016 assertEquals("Should convert nearly freezing", 0.0, celsius(32.1), 0.1) } } JUnit4

import org.junit.Test import static org.junit.Assert.assertEquals import static Converter.celsius

class SimpleJUnit4Test { © ASERT 2006 ASERT © @Test void shouldConvert() {

assert celsius(68) == 20 - 2016 assert celsius(212) == 100, "Should convert boiling" assertEquals("Should convert freezing", 0.0, celsius(32.0)) assertEquals("Also for nearly freezing", 0.0, celsius(32.1), 0.1) } } JUnit5

@Grab('org.junit.platform:junit-platform-runner:1.0.0-M2') @Grab('org.junit.jupiter:junit-jupiter-engine:5.0.0-M2') import org.junit.jupiter.api.Test import org.junit.platform.runner.JUnitPlatform import org.junit.runner.RunWith

import static Converter.celsius © ASERT 2006 ASERT © @RunWith(JUnitPlatform) class ConverterJUnit5Tests {

- @Test 2016 void freezing() { assert celsius(32) == 0 } @Test void boiling() { assert celsius(212) == 100 } } JUnit5

import static Converter.celsius © ASERT 2006 ASERT © @RunWith(JUnitPlatform) class ConverterJUnit5Tests {

- @Test 2016 void freezing() { assert celsius(32) == 0 } @Test void boiling() { assert celsius(212) == 100 } } Spock - BDD style

@Grab('org.spockframework:spock-core:1.0-groovy-2.4') import spock.lang.Specification

class GivenWhenThenSpec extends Specification {

def "test adding a new item to a set"() {

def items = [4, 6, 3, 2] as Set - 2016 when: items << 1

then: items.size() == 5 } } Parameterized import org.junit.Test import org.junit.runner.RunWith import org.junit.runners.Parameterized import org.junit.runners.Parameterized.Parameters import static Converter.celsius

@RunWith(Parameterized) class DataDrivenJUnitTest { private c, f, scenario

@Parameters static scenarios() {[

[0, 32, 'Freezing'], © ASERT 2006 ASERT © [20, 68, 'Garden party conditions'], [35, 95, 'Beach conditions'], [100, 212, 'Boiling']

]*.toArray()}

- 2016 DataDrivenJUnitTest(c, f, scenario) this.c = c this.f = f this.scenario = scenario }

@Test void convert() { def actual = celsius(f) def msg = "$scenario: ${f}°F should convert into ${c}°C" assert c == actual, msg } } Spock @Grab('org.spockframework:spock-core:1.0-groovy-2.4') import spock.lang.* import static Converter.celsius

class SpockDataDriven extends Specification { def "test temperature scenarios"() { expect:

where: -

2016 scenario | tempF || tempC 'Freezing' | 32 || 0 'Garden party conditions' | 68 || 20 'Beach conditions' | 95 || 35 'Boiling' | 212 || 100 } } Spock - Celsius @Grab('org.spockframework:spock-core:1.0-groovy-2.4') import spock.lang.* import static Converter.celsius

class SpockDataDriven extends Specification { @Unroll def "Scenario #scenario: #tempFºF should convert to #tempCºC"() { expect:

celsius(tempF) == tempC © ASERT 2006 ASERT © where: scenario | tempF || tempC

'Freezing' | 32 || 0 - 2016 'Garden party conditions' | 68 || 20 'Beach conditions' | 95 || 34 'Boiling' | 212 || 100 } } Spock - Celsius @Grab('org.spockframework:spock-core:1.0-groovy-2.4') import spock.lang.* import static Converter.celsius

class SpockDataDriven extends Specification { @Unroll def "Scenario #scenario: #tempFºF should convert to #tempCºC"() { expect:

celsius(tempF) == tempC © ASERT 2006 ASERT © where: scenario | tempF || tempC

'Freezing' | 32 || 0 - 2016 'Garden party conditions' | 68 || 20 'Beach conditions' | 95 || 34 'Boiling' | 212 || 100 } } Spock - Mocks

@Grab('org.spockframework:spock-core:1.0-groovy-2.4') import spock.lang.Specification

class SpockMock extends Specification {

def "buy ticket for a movie theater"() { given:

theater.hasSeatsAvailable("Lord of the Rings", 2) >> true - 2016 when: purchase.fill(theater)

then: purchase.completed 1 * theater.purchaseTicket("Lord of the Rings", 2) }

} Spock - Mocks

@Grab('org.spockframework:spock-core:1.0-groovy-2.4') import spock.lang.Specification

class SpockMock extends Specification {

def "buy ticket for a movie theater"() { given:

theater.hasSeatsAvailable("Lord of the Rings", 2) >> true - 2016 when: purchase.fill(theater)

then: purchase.completed 1 * theater.purchaseTicket("Lord of the Rings", 2) }

} Spock - Mocks

@Grab('org.spockframework:spock-core:1.0-groovy-2.4') import spock.lang.Specification

class SpockMockWildcards extends Specification { def "cannot buy a ticket when the movie is sold out"() { given: def purchase = new Purchase("Lord of the rings", 2) © ASERT 2006 ASERT © MovieTheater theater = Mock()

when: - 2016 theater.hasSeatsAvailable(_, _) >> false purchase.fill(theater)

then: !purchase.completed 0 * theater.purchaseTicket(_, _) } } Spock - Mocks

@Grab('org.spockframework:spock-core:1.0-groovy-2.4') import spock.lang.Specification

class SpockMockClosureChecks extends Specification { def "on couples night tickets are sold in pairs"() { given:

theater.hasSeatsAvailable("Lord of the Rings", 2) >> true

- 2016 when: purchase.fill(theater)

then: 1 * theater.purchaseTicket(_, { it % 2 == 0 }) } } Assertion frameworks • None • JUnit 3 assertions • Hamcrest • FEST • Google truth

assertThat(googleColors).containsNoneOf(PINK, BLACK, WHITE, ORANGE)

assert ![PINK, BLACK, WHITE, ORANGE].any { color -> color in googleColors } Property-based testing • Agile testing game (TDD) – Minimum test code to steer design of minimal production code with desired business functionality but 100% code coverage – “Grey box” testing

- 2016 Property-based testing • Agile testing game (TDD) – Minimum test code to steer design of minimal production code with desired business functionality but 100% code coverage – “Grey box” testing

– Instead validate certain properties

- 2016 Property-based testing

@Grab('net.java.quickcheck:quickcheck:0.6') import static net.java.quickcheck.generator.PrimitiveGenerators.* import static java.lang.Math.round

def liquidF = 32..212 - 2016 100.times { int f = gen.next() int c = round(celsius(f)) assert c <= f assert c in liquidC == f in liquidF }

Property-based testing with Spock

- 2016

https://github.com/Bijnagte/spock-genesis Property-based testing: spock genesis

@Grab('com.nagternal:spock-genesis:0.6.0') @GrabExclude('org.codehaus.groovy:groovy-all') import spock.genesis.transform.Iterations import spock.lang.Specification import static Converter.celsius import static java.lang.Math.round import static spock.genesis.Gen.integer

class ConverterSpec extends Specification { def liquidC = 0..100

@Iterations(100)

def "test phase maintained"() { - 2016 given: int tempF = integer(-40..240).iterator().next()

when: int tempC = round(celsius(tempF))

then: tempC <= tempF tempC in liquidC == tempF in liquidF } … Property-based testing: spock genesis

class ConverterSpec extends Specification { def liquidC = 0..100 … © ASERT 2006 ASERT © def liquidF = 32..212 @Iterations(100) @Iterations(100) def "test order maintained"() {

def "test phase maintained"() { -

2016 given: given: int tempF1 = integer(-273..999).iterator().next() int tempF = integer(-40..240).iterator().next() int tempF2 = integer(-273..999).iterator().next() when: when: int tempC = round(celsius(tempF)) int tempC1 = round(celsius(tempF1)) int tempC2 = round(celsius(tempF2)) then: tempC <= tempF then: tempC in liquidC == tempF in liquidF (tempF1 <=> tempF2) == (tempC1 <=> tempC2) } } … } Case Study Case Study Case Study Case Study Case Study Case Study Case Study Case Study Case Study Case Study Case Study Case Study Case Study All Combinations All Combinations Case Study All Pairs All Pairs Case Study 58 GPars • Library classes and DSL allowing you to handle tasks concurrently: – Data Parallelism map, filter, reduce functionality in parallel with parallel array support – Asynchronous functions extend the Java executor services to enable multi-threaded closure processing – Dataflow Concurrency supports natural shared-memory concurrency model, using single-assignment variables – Actors provide Erlang/Scala-like actors including "remote" actors on other machines – Safe Agents provide a non-blocking mt-safe reference to mutable state; like "agents" in Clojure Case Study with GPars Constraint/Logic Programming • Description – Style of programming where relations between variables are stated in the form of constraints – First made popular by logic programming languages such as Prolog but the style is now also used outside logic programming specific languages – Constraints differ from the common primitives of other programming languages in that they do not specify one or more steps to execute but rather the properties of a solution to be found – Popular libraries used with Groovy supporting constraint programming include Gecode/J, Choco and tuProlog – We'll look at Choco as an example Case Study with Constraint Programming • You have been asked to set up some test cases representing the Simpsons’ weekly blogging habits • After some careful study you observe the following strange behavior – They never blog on the same day – Marge blogs only on a Saturday or Sunday – Maggie blogs only on a Tuesday or Thursday – Lisa blogs only on a Monday, Wednesday or Friday – Bart blogs only on the day after Lisa – Homer only blogs if noone else blogged the previous day and doesn't allow anyone to blog the next day Case Study with Constraint Programming Case Study with Constraint Programming Case Study with Constraint Programming ModelJUnit ModelJUnit ModelJUnit Case Study with ModelJUnit Case Study with ModelJUnit Case Study with ModelJUnit Case Study with ModelJUnit Case Study with ModelJUnit Testing DSLs • Low-level DSL

• Higher-level DSL post blog from Bart with title "Bart rulz!" and category School and content "Cowabunga Dude!" More Information: Groovy in Action