The Promise of Model-Based Testing
Total Page：16
File Type：pdf, Size：1020Kb
T17 Concurrent Session Thursday 05/08/2008 3:00 PM The Promise of Model-Based Testing Presented by: Mahesh Velliyur Maveric Testing Solutions Presented at: STAREAST Software Testing Analysis & Review May 5-9, 2008; Orlando, FL, USA 330 Corporate Way, Suite 300, Orange Park, FL 32043 888-268-8770 904-278-0524 [email protected] www.sqe.com Mahesh Velliyur Executive Director and Co-Founder, Maveric Systems Ltd. Mahesh Velliyur is an Executive Director and co-founder of Maveric Systems. He has over 20 years of experience in the IT industry in the area of IT Strategy, architecture, design and implementation of IT solutions to the private and government sector. After serving premier consulting and technology organizations, Mahesh started his first entrepreneurial venture in 1992. He has wide ranging experience in delivering complex development and testing solutions to clients in Europe and US through innovative models of delivery. At Maveric, apart from being the architect of Maveric's proprietary testing frameworks and methodologies, Mahesh heads delivery operations of Maveric across the various delivery centers. He takes accountability for all offshore delivery in Maveric. With Maveric rapidly becoming multi-locational in its delivery, Mahesh’s primary challenge is ensuring that clients perceive the same level of responsiveness and capability in Maveric teams across various delivery locations. Mahesh and his team also ensure that we are constantly innovating and improving our testing methods and processes to significantly enhance the quality of our client’s applications. Mahesh holds the belief that test automation will be the one area in testing where we will see the most rapid and radical changes in the coming years. He has therefore created a dedicated Centers of Excellence for Test Automation in Maveric Systems. The automation CoE invests in creating proprietary, re-usable and tool independent automation frameworks to support Maveric’s clients in the design, execution and management of tests. The CoE also focuses on evaluating best of breed automation tools across the product lifecycle to deliver cost effective and maintainable solutions to our clients. Mahesh holds an MBA from XLRI, India and a Bachelors in Engineering from the Indian Institute of Technology, Madras. Promise of Model Based Testing New Jersey Mumbai Chennai Bangalore Dubai London New Jersey Mumbai Chennai Bangalore Dubai London New Jersey Mumbai Chennai Bangalore Dubai London Maveric Systems confidential Coverage Background Automation Landscape Test design – A perspective Model based testing – A positive influence UAT Challenges – Promise of MBT Design process overview – Retail banking Background Software Testing - Relatively nascent as an independent industry Key drivers – over past 10 years Early years - need for objectivity / independence Mid Years – domain / vertical competencies, regression & repeatability What now ? What now? Productivity improvement across the testing lifecycle, achieved through Process improvement initiatives (Alignment to TMM, TPI etc) Automation initiatives Automation Landscape Test Defect Test Planning Test Design Test Automation Test Closure Execution Mgmt. Scoping & estimation tools Test automation tools Takes up to 50-65% of time in Environment preparation tools Testing Test Construction – simulation tools, integration stubs Most significantly impacts the quality Test management tools of testing Defect management tools Currently very limited automation Data generation tools SOA testing tools A Model based design Configuration & Version control tools Build & release management tools approach is the focus of this presentation Test Design – A Perspective Good design takes up to 50-65% of time in the testing life cycle Difficult to standardize a structured approach to design No scientific method to ensure comprehensiveness – no optimization on the number of test conditions Increased dependency on the Individual’s expertise and experience. Process is not documented; therefore not Auditable. Warrants a high understanding of the domain among all team members or high dependence on business users Model Based Testing – in the right direction Key Benefits Model • An independent basis for based verification testing Boundary Pre/Post • Objective coverage Value assessment Equivalence Transition Application partitioning Model • Scientific mechanism to of design definition achieve desired focus on Functional techniques Pair-wise specific areas Data-flow Rule of ‘n’ • Ability for the business user to relate to process Design pack and outputs generation Maveric has been leading extensive research over the last two years in adoption of Model based Testing for UAT UAT Design Automation – A Wish List A generic model pertaining to the domain; allowing user customisation DOMAIN Automated generation of test cases Principal component using a repertoire of testing principles Ancillary components applied on the above model resulting in Coverage predictability Duplication avoidance / reduction Greater Test Data accuracy Testing Test Better Test results (Oracles) Principles Management identification Boundary Value, Run-plan, Functional Better execution planning and Equivalence Partitioning Decomposition, prioritisation etc. Control Reports etc. Focus is on “what to test” and “how much to test” UAT Challenges Coverage Predictability Duplication avoidance / reduction Ensuring Greater Data Accuracy Identification of Expected results Execution Planning / Prioritisation Needs a solution that uses and extends Model Based Testing UAT Challenges - Coverage Predictability Through well defined business models encompassing a. Definition of Product Groups, Products and Lifecycle Events For example, the following might be relevant in the Retail Banking area: UAT Challenges - Coverage Predictability Through well defined business models encompassing b. Product / Transaction rules Definition For example, a loan cannot be sanctioned when the Nature of Borrower is ‘Joint – Greater than Maximum’ Attributes of Relevance Business Rules Type of Interest Loan Nature of Interest Curre Interest Statu Borrower Base Product Borrowers Rate ncy Base s s Definition Attribute Value Individual Home Single Rack rate Yen Actual Sanctioned Balance Amount Nature of Borrowers Joint – Greater than Fail Maximum Corporate Auto Joint – Less than Negotiate USD Expected Disbursed Maximum d Rate Balance Amount Personal Joint – Equal to Preferred GBP Outstandin Maximum Rate g Principal Joint – Greater AED than Maximum Euro Type of Loan Nature of Interest Base Product Def ID Interest Rate Currency Interest Base Status Borrowers Product Borrowers Definition LN01 Individual Home Joint – Greater Rack rate Yen Actual Sanctioned FAIL than Maximum Balance Amount UAT Challenges - Coverage Predictability Through well defined business models encompassing c. Definition of business flow related rules These would include negative rules related o sequencing of transactions as well as definition of commonly used business flows For e.g.: In Loans, an institution may permit assignment of loans only after the drawdown is complete . Assignment Timing : Value Status Before Setup FAIL After Setup, Pre-Drawal FAIL Post-Drawal, before repayment PASS Post Drawal and repayment FAIL Post closure FAIL UAT Challenges - Coverage Predictability Through well defined business models encompassing d. Capturing Intuitive intelligence - For example, you may wish to test each Loan Product for each type of borrower based on past experience even though the same is not technically necessary Attributes of Relevance Distribution Rules Type of Loan Product Nature of Borrowers Interest Rate Currency Borrowers Type of Borrowers Loan Product Individual Home Single Rack rate Yen 1 2 Corporate Auto Joint – Less than Maximum Negotiated Rate USD Personal Joint – Equal to Maximum Preferred Rate GBP Joint – Greater than Maximum AED Product Nature of Borrowers Currency Interest Rate Status Type of Borrowers Loan Product Definitions LN 01 Individual Auto Single Yen Rack rate LN 02 Individual Home Joint – Less than USD Negotiated Rate Maximum LN 03 Individual Personal Joint – Equal to GBP Preferred Rate Maximum CLEAR LN 04 Corporate Auto Single AED Rack rate LN 05 Corporate Home Joint – Less than Euro Negotiated Rate Maximum LN 06 Corporate Personal Joint – Equal to Yen Preferred Rate Maximum UAT Challenges - Coverage Predictability Through Consistent application of testing Principles a. Equivalence partitioning For example, the following set presents the Equivalence partitioning of the Tenor in Loan setup; optimised to cover partitioning across other attributes as well: b. Boundary value analysis For example, the Amount boundary for a cash deposit in a Savings Account is Rs. (10,000 - 15,000) UAT Challenges – Coverage Predictability By Enabling Tracking at various levels Functional Decomposition Reports enable the users in reviewing outputs generated at various levels - helps in tracking changes in the outputs at various stages Business Scenarios Control Reports UAT Challenges – Duplication Avoidance Application of Rule of N Duplication occurs when the product level attribute values are independently considered for testing without an attempt to combine them. The following example explains how this can be restricted at Product definition – Using Rule of N we can cover all 25 attribute values in 5 conditions Product setup definitions Below is the Product Matrix (Test Conditions) without the duplication of the definitions UAT Challenges – Duplication