Continuous Testing & Integration Philippe Collet (avec beaucoup de M1 PROJET - 2019-2020 (images: pixabay) 1 slides de Guilhem. Molines) Delivering Software P. C o l l e t 2 Delivering software… • In 1995, MS released Windows 95 – Major breakthrough for consumers – Real multi-tasking with background apps • How many service packs were released? 3 Delivering software… • In 1995, MS released Windows 95 – Major breakthrough for consumers – Real multi-tasking with background apps • How many service packs were released? à 2 (yes, two) (6 months and 1 year after release) 4 Delivering software… • In 2015, MS released Windows 10 – 4 editions – Continuous, forced updates • How many service packs were released? 5 Delivering software… • In 2015, MS released Windows 10 – 4 editions – Continuous, forced updates • How many service packs were released? à couldn’t count… (1 GB on first day) 6 What is the most important thing you need to deliver software continuously? Trust 7 How do you get trust in your software delivery? • Good architecture • Development guidelines • Project management • Controlled processes • Traceability of requirements • Automation • Repeatable pipeline • Testing • Testing • And… Testing 8 Software Delivery Life Cycle (SDLC) In a nutshell P. C o l l e t 9 Single student, simple project • Ant • Maven? • SCM? • Code and click 10 Group of students, simple project • Ant / Maven • Github • Code and click • Build on local machine • Ticket management ? 11 Industrial complex project • 20 – 200 contributors • One release / year • One patch / month • > 20M LOCs • Deployment complexity • à What do you need? 12 Industrial complex project • Componentization • Independence of builds • Each component tested • Clear quality indicators • Requirement traceability • Fast builds • Each contributor only builds what they code • Deployment automation 13 Software quality • What is it? • No Defects? • Or…. • Known defects? 14 Known defects??? • Goals of quality are: – Know and document bugs – Verify them for regression – Find workarounds – Feed more requirements 15 How do you choose your defects? • Dev methodology • Team culture • Good practices • Correct soft. Architecture • Early Integration • And… • Testing 16 Types of tests • Unit-tests • Integration tests • Acceptation tests • …ok…. What else? 17 Types of tests • Unit Tests • Usability Tests • Integration Tests • Security Tests • GUI Tests • Recovery Tests • Non-regression Tests • L10N/I18N Tests • Coverage Tests • Accessibility Tests • Load Tests • Installation/Configuration • Stress Tests Tests • Performance Tests • Documentation Tests • Scalability Tests • Platform testing • Reliability Tests • Samples/Tutorials Testing • Volume Tests • Code inspections 18 Seriously, you do all of this stuff??? How the hell do you get organized??? TEST ARCHITECT 19 Test architect role • Choose test tools • Define practices • Build base frameworks à reusability • Find test data • Global test bucket consistency, strategy 20 Unit-, Functional- & Integration tests 21 Unit-Test 22 Unit-Test Example: CartModifier add(Customer, Item) KitchenBean process(Order) 23 Functional Test 24 Functional Test Example: CartWebService addItemToCustomerCart(String, Item) 25 Functional - specs GIVEN The Cookie Factory server and its Webservices The customer Bob WHEN Bob calls the CartWebService addItemToCart WS with 3 chocolate cookies THEN - Bob’s cart is updated (incremented by 3 items) - TCF inventory is updated (3 cookies removed) - … 26 Unit vs Functional Technology <-> Function Programmer <-> User Does it work? <-> Does it implement the function I need? Small <-> Bigger 27 Integration Test 28 Integration Test Example: CLI OrderCookie.execute 29 Integration Test - specs GIVEN - TCF server, connected to “a” (*) bank system - The CLI client - The right sample data (Bob, some cookies…) WHEN - Bob builds a cookie order and calls OrderCookie.execute THEN - TCF inventory is updated (3 cookies removed) - Bob’s cart is updated (added 3 chocolate cookies) - Bob’s bank accounted is billed (*: real or mocked) 30 Unit vs Integration http://imgur.com/qSN5SFR 31 Scenario Testing 32 Scenario Testing - specs 33 (From: Nhan Ngo) 34 Building components • What does B consume of A? A • Contract B C – API – Teams D • Enforced through tests E F 35 Where to run • Unit – Dev command line / dev. Env – Build plan, right after compile – Prevents artifactory publication – Because • it’s fast • It’s the contract with dependents 36 Where to run • Functional – Dev command line / dev. Env – Build plan, dedicated • Use profiles – Prevents artifactory publication – or not… – Not so fast anymore – May require to be run in-container • Arquilian, Cactus à server-side 37 Where to run • Integration – Dedicated env • Use VM if possible – Separate build plan – Long running – Requires prod-like env. 38 Continuous Integration P. C o l l e t 39 Continuous Integration • Definition (May 2006) by Martin Fowler • https://martinfowler.com/articles/continuousIntegration.html 40 Principles • Integrate frequently – At least once a day (e.g. git) • Integrated system is automatically built – Making a repeatable process (e.g. maven) • Build triggers Tests – (e.g. Junit, cucumber…) And that’s all… 41 DevOps http://meshfields.de/continuous-integration-testing-delivery-ionic2-hybrid-mobile-apps-buddybuild/ P. C o l l e t 42 DevOps Pipeline https://insights.sei.cmu.edu/devops/2015/04/continuous-integration-in-devops.html P. C o l l e t 43 CI servers: some examples P. Collet 44 travis-ci.org travis-ci.com • Full Cloud • Integrated with github, free for open-source repos • Free for github classroom organization (1 cpu, 1 job) – https://travis-ci.com/uca-m1informatique-softeng • When you run a build, – Travis CI clones your GitHub repository into a brand-new virtual environment, – carries out a series of tasks to build and test your code. – If one or more of those tasks fail, the build is considered broken – If none of the tasks fail, the build is considered passed and Travis CI can deploy your code to a web server or application host. 45 Setup • Signup with your github account – Authorize application within github • Add a repository • Configure the .travis.yml file – For Java: https://docs.travis-ci.com/user/languages/java/ – You can lint the file: https://config.travis-ci.com/explore • Tutorial: https://docs.travis-ci.com/user/tutorial/ 46 A more complex .travis.yml file P. C o l l e t 47 https://github.com/SonarSource/sq-com_example_java-maven-travis/blob/master/.travis.yml When the build passes P. C o l l e t 48 When the build fails P. C o l l e t 49 When the build fails P. C o l l e t 50 An important settings option P. C o l l e t 51 P. C o l l e t 52.