Intelligent Tools for Policy Design Deliverable 3.3 Test Strategy and Test Cases Project Reference No. 287119 Deliverable No. D 3.3 Relevant workpackage: WP 3 Nature: Report Dissemination Level: Public Document version: FINAL Editor(s): Rumm Nikolaus, Kamenicky Alexander, Kagitcioglu Hakan, Hönigschnabl Ilja Contributors: Sonntagbauer Peter, Mondragon Arturo, Bouchard Guillaume, Nazemi Kawa, Burkhardt Dirk, Ginters Egils, Aizstrauts Artis, Boscolo Paolo, Palmisano Elena Document description: This is the test plan and the test strategy for WP3 - Open Core Platform and Web 2.0/3.0 Integration The objective of this document is to describe the quality management procedures that the WP3 team has decided upon in order to achieve a sufficiently high level of product quality for the FUPOL Core Platform. This includes an overview of the test items, the features to be tested and those that won't be tested, the test environment, the various test types (including logical test cases for some of them), the test approach taken, management issues like controlling and reporting, the test deliverables, risks and contingencies, the test tools and finally some notes on real-life test data from the pilot cities. Please note that D3.3 is based on the requirements defined in D3.1. and on the user stories that we used to describe the functionality. History Version Date Reason Prepared / Revised by 0.1 Initial release Rumm Nikolaus 0.2 2012-05-25 Intermediate release for internal review Rumm Nikolaus 0.3 2012-07-30 Intermediate release for internal review Rumm Nikolaus 0.4 2012-09-28 Intermediate release for internal review Rumm Nikolaus 1.0 2013-01-28 Final release Rumm Nikolaus All rights reserved. No parts of this document may be reproduced without written permission from the FUPOL programme steering committee and/or cellent AG. This includes copying, printing, processing, duplicating and all other forms of distributing this document on any media. Company names, product name, trademarks and brand names used in this document might be protected by law and belong to their respective owners. We acknowledge that this document uses material from ISTQB. Deliverable D3.3 2 Table of Contents 1 INTRODUCTION ...................................................................................... 9 1.1 The Purpose of the Project ..................................................................... 10 1.2 Scientific Approach and Overall Concept .................................................. 12 1.3 Goals of the Project ................................................................................ 14 2 TEST OBJECT ......................................................................................... 18 2.1.1 Business Context ............................................................................. 18 2.1.2 Logical Context ............................................................................... 21 2.1.3 Technical/Infrastructure Context ...................................................... 22 2.1.4 Implementation Context .................................................................. 23 3 FEATURES TO BE TESTED ..................................................................... 25 3.1 Quality Goals and Non-Functional Requirements ...................................... 26 3.2 Functional Requirements ........................................................................ 28 4 FEATURES NOT TO BE TESTED ............................................................. 31 5 TEST ENVIRONMENT ............................................................................ 34 5.1 Hardware .............................................................................................. 35 5.2 Software ............................................................................................... 37 5.3 Stages ................................................................................................... 38 5.4 Database ............................................................................................... 40 5.5 Project URLs .......................................................................................... 41 6 TEST TYPES ........................................................................................... 43 6.1 Test Layers ............................................................................................ 44 6.1.1 Unit/Unit Integration Test ................................................................ 44 6.1.2 System Test .................................................................................... 48 6.1.3 System Integration Test .................................................................. 50 6.1.4 Acceptance Test .............................................................................. 50 6.2 Functional and Non-Functional Tests ....................................................... 53 Deliverable D3.3 3 6.2.1 Functional Tests .............................................................................. 53 6.2.2 Non-Functional Tests ....................................................................... 54 6.3 Validating vs. Verifying Tests .................................................................. 56 6.4 Manual vs. Automated Tests ................................................................... 59 7 TEST APPROACH ................................................................................... 61 7.1 Agile Testing .......................................................................................... 62 7.2 Quality Gates ......................................................................................... 69 7.2.1 Quality Gate 1: Commitment to the Delivery ..................................... 70 7.2.2 Quality Gate 2: Definition of Done .................................................... 71 7.2.3 Quality Gate 3: Review Meeting ....................................................... 73 7.2.4 Pilot Phase ...................................................................................... 74 7.3 Test Case Design Methodology ............................................................... 75 7.3.1 Unit/Unit Integration Test ................................................................ 75 7.3.2 System Test .................................................................................... 76 7.3.3 System Integration Test .................................................................. 76 7.3.4 Acceptance Test .............................................................................. 77 7.4 Notification / Escalation Procedures ......................................................... 78 8 TEST CONTROLLING AND REPORTING ................................................ 79 9 TEST MEASUREMENT AND METRICS .................................................... 81 10 TESTING DELIVERABLES ................................................................... 92 11 RISK AND CONTINGENCIES .............................................................. 93 12 TOOLS ................................................................................................ 95 12.1 Test Management .................................................................................. 96 12.2 Defect Tracking ..................................................................................... 97 12.3 Test Automation .................................................................................... 98 12.4 Test Result Management ...................................................................... 100 12.5 Code Metrics ........................................................................................ 101 13 SYSTEM TEST CASE REPOSITORY ................................................... 104 Deliverable D3.3 4 13.1 System Tests ....................................................................................... 105 13.2 Baseline for System Tests - Preconditions .............................................. 106 13.3 System Test Case List .......................................................................... 109 13.4 Sprint Overview and Requirements Coverage ......................................... 117 13.4.1 Access Management ...................................................................... 128 13.4.2 Account Management .................................................................... 135 13.4.3 Campaign Management ................................................................. 174 13.4.4 Client Management........................................................................ 206 13.4.5 Social Media Management .............................................................. 216 14 GLOSSARY ....................................................................................... 270 15 APPENDIX I: REAL-LIFE TEST DATA ............................................... 286 15.1 Prato ................................................................................................... 287 15.1.1 Social Media Data .......................................................................... 287 15.1.2 Geographic Data ........................................................................... 288 15.1.3 Statistical Data .............................................................................. 288 15.2 Zagreb ................................................................................................ 289 15.2.1 Social Media Data .......................................................................... 289 15.2.2 Geographic Data ..........................................................................