Architecting TIBCO StreamBase Applications Unit 4 Functional Testing © Copyright 2000-2014 TIBCO Software Inc. StreamBase Functional Testing Overview • Test Methodology and Tooling • Test Development Tasks • Types of Tests • Tool Overview • Functional Test Areas © Copyright 2000-2014 TIBCO Software Inc. 2 StreamBase Test Methodology and Tooling 3 © Copyright 2000-2013 TIBCO Software Inc. Test Methodology and Tooling Partial list taken from Project Methodology Unit: We Can Help. • by validating/ creating recipes for the application of SB specifics to the general rules • by establishing delivery team discipline around particularly important good practices that often fall by the wayside • by promoting systematic use of lifecycle tool support and integrating into customer SDLC practices and infrastructure © Copyright 2000-2014 TIBCO Software Inc. 4 StreamBase Functional Testing: Tasks 5 © Copyright 2000-2013 TIBCO Software Inc. Functional Testing Tasks • Test Development Tasks – Test Plan – Test Case Specification – Test Development – Test Data Generation • Functional Testing Infrastructure and DevOps Setup and Integration • CI Server Integration and SLAs • Regression Suite Initiation and Frequency, Results Review and Notification © Copyright 2000-2014 TIBCO Software Inc. 6 StreamBase Types of Tests 7 © Copyright 2000-2013 TIBCO Software Inc. Types of Application Testing • Functional Requirements Testing • System Integration Testing • Performance Testing and Tuning • Throughput Metrics • Latency Metric • This is a science; see performance unit © Copyright 2000-2014 TIBCO Software Inc. 8 More Types of Testing – Failover/Failback Testing • Application Server Failure • Inbound Messaging Server(s) Failover/Failure • Outbound Messaging Server(s) Failover/Failure • Persistence (RDBMS) Server Failover/Failure • Network Failure • Storage/Cache Failure – Disaster Recovery Testing – Stress Testing/Burn-in © Copyright 2000-2014 TIBCO Software Inc. 9 Testing Types You Need Other People’s Help For • QA Environment Tasks – Production, Adapter and Application Deployment – Knowledge Transfer to QA Personnel – Issue Review and Resolution – Release packaging and re-deployments • Customer Acceptance Testing – Deployment – Issue Review and Resolution – Release packaging and re-deployments • User Acceptance Testing (if required) © Copyright 2000-2014 TIBCO Software Inc. 10 StreamBase Functional Testing: Tool Overview 11 © Copyright 2000-2013 TIBCO Software Inc. StreamBase Testing Tool Overview (1 of 2) • StreamBase Unit – JUnit-based; leverages existing tooling – Wizards for automated test generation • Acceptance testing framework example – FitNesse and SbFit (https://github.com/streambase/SbFit) – Why just example? Based on older Fit framework; should be migrated to newr SLiM – Fixtures for StreamBase Enqueue/Dequeue and JDBC tests using StreamBase Datasources • Sbfeedsim to drive simulations and performance tests © Copyright 2000-2014 TIBCO Software Inc. 12 StreamBase Testing Tool Overview (2 of 2) • StreamBase Unit Extensions – Encourages creation of schema-change resilient tests – Open source/GitHub (https://github.com/streambase/SBUnit-Extensions) - latest – SBX (Use “Unit Testing Extensions for StreamBase 7.6 v1.0” ) – a little easier to set up – Tutorial, Samples, Exercises – Good “docs” in the unit tests for the extensions themselves – Intention to productize and revise wizards; no action in a long time, though © Copyright 2000-2014 TIBCO Software Inc. 13 StreamBase JUnit Test Creation Wizard © Copyright 2000-2014 TIBCO Software Inc. 14 Using Ant with StreamBase JUnit • Example build.xml: https://answer.streambase.com/display/test/How+do+I+run+StreamBase+JUnit+tests+using+Ant <target name="sbunit" depends="build"> <junitreport todir="${sbunitlog}"> <mkdir dir="${sbunitlog}" /> <fileset dir="${sbunitlog}"> <junit printsummary="yes" haltonfailure="no" fork="yes"> <include name="TEST-*.xml" /> <classpath refid="MYPROJECT.classpath" /> </fileset> <sysproperty key="streambase.unit-test.server-conf" <report format="frames" todir="${sbunitlog}" /> value="sbd.sbconf"/> </junitreport> <jvmarg value="-Xmx2g"/> </target> <jvmarg value="-Dstreambase.log-level=2"/> <formatter type="xml" /> <batchtest fork="yes" todir="${sbunitlog}"> <fileset dir="java-src"> <include name="**/test/*.java" /> <include name="**/tests/*.java" /> <include name="**/Test*Test.java" /> </fileset> </batchtest> </junit> © Copyright 2000-2014 TIBCO Software Inc. 15 StreamBase Functional Testing: Test Areas 16 © Copyright 2000-2013 TIBCO Software Inc. Some Broad Functional Test Areas • EventFlow Application Logic • Messaging Connectivity (integration test?) • RDBMS/Active Spaces Access (integration test?) • Client/User Interface • System Integration © Copyright 2000-2014 TIBCO Software Inc. 17 StreamBase Application Development Test Tasks • Main Business Logic Flow • State CRUD Logic • Initialization/State Recovery • Shutdown • Failover, Failback, and Connection and Dependent State Recovery • Automated Functional Unit Tests © Copyright 2000-2014 TIBCO Software Inc. 18 Messaging Connectivity Development Test Tasks • Messaging Queue/Topic Configuration – Message/Field Definition – QoS Configuration • Authentication/Authorization • Connectivity Testing • Recovery Testing • Messaging Connections – Schema definition – Adapter Configuration – Connectivity Testing • Test Message Generator© Copyright 2000-2014Setup TIBCO and Software Inc. Configuration 19 RDBMS/Caching Integration Test Tasks • Schemas • Stored procedure interface, if required • Reference data population • Permissions/user accounts • Connectivity/JDBC/Cache Adapter Configuration • Distributed cache integration (ActiveSpaces, etc.) © Copyright 2000-2014 TIBCO Software Inc. 20 Client And UI Functional Testing • UI Test Automation Framework • Manual UI Testing Approach, Frequency, and Cost © Copyright 2000-2014 TIBCO Software Inc. 21 System Integration Testing • Automating end to end and integration tests © Copyright 2000-2014 TIBCO Software Inc. 22.