Continuous Testing & Integration

Philippe Collet (avec beaucoup de M1 PROJET - 2019-2020 (images: pixabay) 1 slides de Guilhem. Molines) Delivering Software

P. o l l e t 2 Delivering software…

• In 1995, MS released Windows 95 – Major breakthrough for consumers – Real multi-tasking with background apps

• How many service packs were released?

3 Delivering software…

• In 1995, MS released Windows 95 – Major breakthrough for consumers – Real multi-tasking with background apps

• How many service packs were released?

à 2 (yes, two) (6 months and 1 year after release)

4 Delivering software…

• In 2015, MS released Windows 10 – 4 editions – Continuous, forced updates

• How many service packs were released?

5 Delivering software…

• In 2015, MS released Windows 10 – 4 editions – Continuous, forced updates

• How many service packs were released?

à couldn’t count… (1 GB on first day)

6 What is the most important thing you need to deliver software continuously?

Trust

7 How do you get trust in your software delivery? • Good architecture • Development guidelines • Project management • Controlled processes • Traceability of requirements • Automation • Repeatable pipeline • Testing • Testing • And… Testing

8 Software Delivery Life Cycle (SDLC)

In a nutshell

P. C o l l e t 9 Single student, simple project

• Ant • Maven? • SCM? • Code and click

10 Group of students, simple project

• Ant / Maven • Github • Code and click • Build on local machine • Ticket management ?

11 Industrial complex project

• 20 – 200 contributors • One release / year • One patch / month • > 20M LOCs • Deployment complexity

• à What do you need?

12 Industrial complex project

• Componentization • Independence of builds • Each component tested • Clear quality indicators • Requirement traceability • Fast builds • Each contributor only builds what they code • Deployment automation

13 Software quality

• What is it? • No Defects? • Or….

• Known defects?

14 Known defects???

• Goals of quality are: – Know and document bugs – Verify them for regression – Find workarounds – Feed more requirements

15 How do you choose your defects?

• Dev methodology • Team culture • Good practices • Correct soft. Architecture • Early Integration • And…

• Testing

16 Types of tests

• Unit-tests • Integration tests • Acceptation tests

• …ok…. What else?

17 Types of tests

• Unit Tests • Usability Tests • Integration Tests • Security Tests • GUI Tests • Recovery Tests • Non-regression Tests • L10N/I18N Tests • Coverage Tests • Accessibility Tests • Load Tests • Installation/Configuration • Stress Tests Tests • Performance Tests • Documentation Tests • Scalability Tests • Platform testing • Reliability Tests • Samples/Tutorials Testing • Volume Tests • Code inspections

18 Seriously, you do all of this stuff???

How the hell do you get organized???

TEST ARCHITECT

19 Test architect role

• Choose test tools • Define practices • Build base frameworks à reusability • Find test data • Global test bucket consistency, strategy

20 Unit-, Functional- & Integration tests

21 Unit-Test

22 Unit-Test

Example:

CartModifier add(Customer, Item) KitchenBean process(Order)

23 Functional Test

24 Functional Test

Example:

CartWebService addItemToCustomerCart(String, Item)

25 Functional - specs

GIVEN The Cookie Factory server and its Webservices The customer Bob WHEN Bob calls the CartWebService addItemToCart WS with 3 chocolate cookies THEN - Bob’s cart is updated (incremented by 3 items) - TCF inventory is updated (3 cookies removed) - …

26 Unit vs Functional

Technology <-> Function <-> User Does it work? <-> Does it implement the function I need? Small <-> Bigger

27 Integration Test

28 Integration Test

Example:

CLI OrderCookie.execute

29 Integration Test - specs

GIVEN - TCF server, connected to “a” (*) bank system - The CLI client - The right sample data (Bob, some cookies…) WHEN - Bob builds a cookie order and calls OrderCookie.execute THEN - TCF inventory is updated (3 cookies removed) - Bob’s cart is updated (added 3 chocolate cookies) - Bob’s bank accounted is billed

(*: real or mocked)

30 Unit vs Integration

http://imgur.com/qSN5SFR 31 Scenario Testing

32 Scenario Testing - specs

33 (From: Nhan Ngo) 34 Building components

• What does B consume of A? A • Contract B C – API

– Teams • Enforced through tests E F

35 Where to run

• Unit – Dev command line / dev. Env – Build plan, right after compile – Prevents artifactory publication – Because • it’s fast • It’s the contract with dependents

36 Where to run

• Functional – Dev command line / dev. Env – Build plan, dedicated • Use profiles – Prevents artifactory publication – or not… – Not so fast anymore – May require to be run in-container • Arquilian, Cactus à server-side

37 Where to run

• Integration – Dedicated env • Use VM if possible – Separate build plan – Long running – Requires prod-like env.

38 Continuous Integration

P. C o l l e t 39 Continuous Integration

• Definition (May 2006) by Martin Fowler

• https://martinfowler.com/articles/continuousIntegration.html

40 Principles

• Integrate frequently – At least once a day (e.g. git)

• Integrated system is automatically built – Making a repeatable process (e.g. maven)

• Build triggers Tests – (e.g. Junit, cucumber…)

And that’s all…

41 DevOps

http://meshfields.de/continuous-integration-testing-delivery-ionic2-hybrid-mobile-apps-buddybuild/

P. C o l l e t 42 DevOps Pipeline

https://insights.sei.cmu.edu/devops/2015/04/continuous-integration-in-devops.html

P. C o l l e t 43 CI servers: some examples

P. Collet 44 travis-ci.org travis-ci.com

• Full Cloud • Integrated with , free for open-source repos • Free for github classroom organization (1 cpu, 1 job) – https://travis-ci.com/uca-m1informatique-softeng

• When you run a build, – Travis CI clones your GitHub repository into a brand-new virtual environment, – carries out a series of tasks to build and test your code. – If one or more of those tasks fail, the build is considered broken – If none of the tasks fail, the build is considered passed and Travis CI can deploy your code to a web server or application host.

45 Setup

• Signup with your github account – Authorize application within github • Add a repository • Configure the .travis.yml file – For Java: https://docs.travis-ci.com/user/languages/java/

– You can lint the file: https://config.travis-ci.com/explore • Tutorial: https://docs.travis-ci.com/user/tutorial/

46 A more complex .travis.yml file

P. C o l l e t 47 https://github.com/SonarSource/sq-com_example_java-maven-travis/blob/master/.travis.yml When the build passes

P. C o l l e t 48 When the build fails

P. C o l l e t 49 When the build fails

P. C o l l e t 50 An important settings option

P. C o l l e t 51 P. C o l l e t 52