Continuous Testing & Integration
Philippe Collet (avec beaucoup de M1 PROJET - 2019-2020 (images: pixabay) 1 slides de Guilhem. Molines) Delivering Software
P. C o l l e t 2 Delivering software…
• In 1995, MS released Windows 95 – Major breakthrough for consumers – Real multi-tasking with background apps
• How many service packs were released?
3 Delivering software…
• In 1995, MS released Windows 95 – Major breakthrough for consumers – Real multi-tasking with background apps
• How many service packs were released?
à 2 (yes, two) (6 months and 1 year after release)
4 Delivering software…
• In 2015, MS released Windows 10 – 4 editions – Continuous, forced updates
• How many service packs were released?
5 Delivering software…
• In 2015, MS released Windows 10 – 4 editions – Continuous, forced updates
• How many service packs were released?
à couldn’t count… (1 GB on first day)
6 What is the most important thing you need to deliver software continuously?
Trust
7 How do you get trust in your software delivery? • Good architecture • Development guidelines • Project management • Controlled processes • Traceability of requirements • Automation • Repeatable pipeline • Testing • Testing • And… Testing
8 Software Delivery Life Cycle (SDLC)
In a nutshell
P. C o l l e t 9 Single student, simple project
• Ant • Maven? • SCM? • Code and click
10 Group of students, simple project
• Ant / Maven • Github • Code and click • Build on local machine • Ticket management ?
11 Industrial complex project
• 20 – 200 contributors • One release / year • One patch / month • > 20M LOCs • Deployment complexity
• à What do you need?
12 Industrial complex project
• Componentization • Independence of builds • Each component tested • Clear quality indicators • Requirement traceability • Fast builds • Each contributor only builds what they code • Deployment automation
13 Software quality
• What is it? • No Defects? • Or….
• Known defects?
14 Known defects???
• Goals of quality are: – Know and document bugs – Verify them for regression – Find workarounds – Feed more requirements
15 How do you choose your defects?
• Dev methodology • Team culture • Good practices • Correct soft. Architecture • Early Integration • And…
• Testing
16 Types of tests
• Unit-tests • Integration tests • Acceptation tests
• …ok…. What else?
17 Types of tests
• Unit Tests • Usability Tests • Integration Tests • Security Tests • GUI Tests • Recovery Tests • Non-regression Tests • L10N/I18N Tests • Coverage Tests • Accessibility Tests • Load Tests • Installation/Configuration • Stress Tests Tests • Performance Tests • Documentation Tests • Scalability Tests • Platform testing • Reliability Tests • Samples/Tutorials Testing • Volume Tests • Code inspections
18 Seriously, you do all of this stuff???
How the hell do you get organized???
TEST ARCHITECT
19 Test architect role
• Choose test tools • Define practices • Build base frameworks à reusability • Find test data • Global test bucket consistency, strategy
20 Unit-, Functional- & Integration tests
21 Unit-Test
22 Unit-Test
Example:
CartModifier add(Customer, Item) KitchenBean process(Order)
23 Functional Test
24 Functional Test
Example:
CartWebService addItemToCustomerCart(String, Item)
25 Functional - specs
GIVEN The Cookie Factory server and its Webservices The customer Bob WHEN Bob calls the CartWebService addItemToCart WS with 3 chocolate cookies THEN - Bob’s cart is updated (incremented by 3 items) - TCF inventory is updated (3 cookies removed) - …
26 Unit vs Functional
Technology <-> Function Programmer <-> User Does it work? <-> Does it implement the function I need? Small <-> Bigger
27 Integration Test
28 Integration Test
Example:
CLI OrderCookie.execute
29 Integration Test - specs
GIVEN - TCF server, connected to “a” (*) bank system - The CLI client - The right sample data (Bob, some cookies…) WHEN - Bob builds a cookie order and calls OrderCookie.execute THEN - TCF inventory is updated (3 cookies removed) - Bob’s cart is updated (added 3 chocolate cookies) - Bob’s bank accounted is billed
(*: real or mocked)
30 Unit vs Integration
http://imgur.com/qSN5SFR 31 Scenario Testing
32 Scenario Testing - specs
33 (From: Nhan Ngo) 34 Building components
• What does B consume of A? A • Contract B C – API
– Teams D • Enforced through tests E F
35 Where to run
• Unit – Dev command line / dev. Env – Build plan, right after compile – Prevents artifactory publication – Because • it’s fast • It’s the contract with dependents
36 Where to run
• Functional – Dev command line / dev. Env – Build plan, dedicated • Use profiles – Prevents artifactory publication – or not… – Not so fast anymore – May require to be run in-container • Arquilian, Cactus à server-side
37 Where to run
• Integration – Dedicated env • Use VM if possible – Separate build plan – Long running – Requires prod-like env.
38 Continuous Integration
P. C o l l e t 39 Continuous Integration
• Definition (May 2006) by Martin Fowler
• https://martinfowler.com/articles/continuousIntegration.html
40 Principles
• Integrate frequently – At least once a day (e.g. git)
• Integrated system is automatically built – Making a repeatable process (e.g. maven)
• Build triggers Tests – (e.g. Junit, cucumber…)
And that’s all…
41 DevOps
http://meshfields.de/continuous-integration-testing-delivery-ionic2-hybrid-mobile-apps-buddybuild/
P. C o l l e t 42 DevOps Pipeline
https://insights.sei.cmu.edu/devops/2015/04/continuous-integration-in-devops.html
P. C o l l e t 43 CI servers: some examples
P. Collet 44 travis-ci.org travis-ci.com
• Full Cloud • Integrated with github, free for open-source repos • Free for github classroom organization (1 cpu, 1 job) – https://travis-ci.com/uca-m1informatique-softeng
• When you run a build, – Travis CI clones your GitHub repository into a brand-new virtual environment, – carries out a series of tasks to build and test your code. – If one or more of those tasks fail, the build is considered broken – If none of the tasks fail, the build is considered passed and Travis CI can deploy your code to a web server or application host.
45 Setup
• Signup with your github account – Authorize application within github • Add a repository • Configure the .travis.yml file – For Java: https://docs.travis-ci.com/user/languages/java/
– You can lint the file: https://config.travis-ci.com/explore • Tutorial: https://docs.travis-ci.com/user/tutorial/
46 A more complex .travis.yml file
P. C o l l e t 47 https://github.com/SonarSource/sq-com_example_java-maven-travis/blob/master/.travis.yml When the build passes
P. C o l l e t 48 When the build fails
P. C o l l e t 49 When the build fails
P. C o l l e t 50 An important settings option
P. C o l l e t 51 P. C o l l e t 52