MCGILL UNIVERSITY Montreal, Quebec September 20 – 21, 2016 A DMAIC Framework for Improving Software Quality in Organizations: Case Study at RK Company Team Composition • Racha Karout • Anjali Awasthi Outline 1. Introduction 1.1 Background 1.2 Problem Definition 1.3 Research Objectives 2. Literature Review 3. Solution Approach 4. Conclusions and Future Works 1.1 Background • To compete in today‘s world, every business needs to improve. • Software has increasingly become a critical component in many industries (telecoms, banking, insurance, … etc.). • Software quality is crucial and poor quality is not acceptable. • Software development has not been consistently successful. What is Software Quality? • Software quality is the degree to which a system, component or process meets specified requirements, in other words, the degree to which a system, component or process meets customer or user needs or expectation (IEEE, 1991). • The software should not have bugs that reduce the quality attributes (functionality, reliability, usability and maintainability) (Chang et al., 2006). • There should not be issues that affect its ability to maintain or re-establish its level of performance. • Easy to use and maintain. 1.2 Problem Definition • In today’s market competition and the need for rapid delivery, software quality is often sacrificed, leading to the failure of the software project • The use of traditional methodology (waterfall) with the current market pace, continuous change of customer requirements and rapid development of technology plays a major role in poor software quality. • People jump to solutions without fully understanding the problem or finding the root cause of poor quality. 1.3 Research Objectives 1. Improve software quality in organizations. 2. Identify the root cause(s) of the problems that result in poor quality. 3. Identify improvement opportunities based on the root cause(s) of poor quality. 4. Show how companies can control their processes to ensure the improvements applied are directing the project to the expected level of quality. 2. Literature Review • Six Sigma • Common Software Quality Problems • Software Development Processes • Kanban • Test Automation • Metrics and Reports • Research Gaps 2.1 Six Sigma • Is a business process improvement approach that seeks to find and eliminate causes of defects and errors, reduce cycle times and cost of operations (Evans & Lindsay, 2005) • It is based on a simple problem solving methodology – “DMAIC” (Hung & Sung, 2011) • Quality principles and six sigma: 1. Focus on customers 2. Participation and teamwork 3. A process focus supported by continuous improvement. 2.2 DMAIC Framework • Define: clearly define the problem • Measure: gather facts, collect data and observations and measure the internal process. • Analyze: identify the root cause(s) of the problems and focus on why defects occur. • Improve: generate and implement ideas for improving or resolving the problem. • Control: focus on maintaining the improvements. 2.3 Common Software Quality Problems • During project implementation both technology and business environment change. This leads to products that do not meet the needs of customers. (Williams & Cockburn, 2003) • Customers are only involved during requirement collecting in traditional software development (Hossain et al., 2013), while in today’s market demand, customer requirements continuously change. • In the traditional plan-driven software development process, work is coordinated by managers (Moe et al., 2010), thus giving software development team less control (Stankovic et al., 2013) • When developers and testers do not develop a shared mental model, they will not work together to address the project issues (Yu & Petter, 2014). • Late discovery of bugs in the traditional software model leads to an increased cost to fix the faults (Monassar & Govardhan, 2010) 2.4 Software Development Processes - Waterfall Requirement Analysis System Design Implementation Testing Deployment Maintenance 2.5 Software Development Processes – Agile • Agile promotes quick response to changing environments, change in user requirements and accelerated deadlines (Eriksson et al., 2005). • It focuses on four core values 1. Individuals and interactions over processes and tools. 2. Working software over comprehensive documentation. 3. Customer collaboration over contract negotiation. 4. Responding to change over following a plan (Dyba & Dingsoyr, 2008). How can agile techniques enhance software quality • System metaphor: It is helpful for communication between the team members and users. It enhances maintainability, efficiency, reliability and flexibility. • Architectural spike: It helps to determine and test potential architecture. It enhances correctness. • Onsite customer feedback: It helps developers refine and correct requirements. It enhances responsiveness to changing customer requirements. • Refactoring: It improves software quality factors such as efficiency, reliability and testability. • Pair programming: It improves design quality factors such as correctness, verifiability and testability and reduce defects. • Stand-up-meeting: It improves software reliability and flexibility. • Continuous integration: It improves integrity, usability and testability. • Acceptance testing: It improves testability. Agile Methods - Scrum 2.6 What is Kanban System? • Number of cards equivalent to the capacity of a system are placed in circulation. • It is a pull system as new work is pulled into the system when there is capacity to handle it, rather than being pushed. • The core properties that Kanban uses are: 1. Visualize workflow 2. Limit work in progress 3. Measure and manage flow 4. Make process policies explicit (Anderson, 2010) 2.7 Test Automation • It is the process of writing a computer program to do testing. Once test have been automated, they can be run quickly and repeatedly (Hooda, 2012) • Benefits of test automation: Defects can be found at an early stage. Can discover defects that manual testing can’t find. Helps increase test coverage. Helps increase management confidence in the software. It executes significantly faster than human users. Can be reused to test different versions with different configurations or platforms. With increased test coverage, more bugs are prevented from escaping to production which in turn reduces cost radically. Can be considered as a safety net for developers especially in case of necessary code refactoring. 2.8 Metrics and management Reporting • The purpose is to show that the system is predictable, the organization exhibits business agility, and there is focus on flow, and there is clear development of continuous improvement • Some of the metrics that can be used: • Tracking work-in-progress • Lead time • Throughput • Defect removal efficiency • DRE=E/E+D 2.9 Research Gaps • Subramanian et al. (2007) conducted a study that examines how CMM and the IS implementation strategies impact software quality and project performance. They mentioned a limitation in their study that is focused primarily on CMM and mentioned other methods such as Total Quality Management and Six Sigma could also be considered in empirical research. • Card (2004) argued the need for more academic research in software process improvement methods. • However, this thesis focuses on using six sigma DMAIC framework to improve software quality in organizations and as a consequence ensures the success of the software project. 3. Solution Approach • ‘RK’ Company Description • Define Phase • Measure Phase • Analysis Phase • Improvement Phase • Control Phase DMAIC - Tools Phase Technique Define CTQs SIPOC diagram Measure Pareto charts Analyze 5-Why technique Cause and effect Interrelationship diagram Improve QFD Control Measurement metrics 4.‘RK’ Company and DMAIC • One of the fast growing software companies in Canada (Medium size) • DMAIC is applied to one of the biggest and core projects • The project has been in development for the past few years. • Waterfall methodology. • Team consists of: • Six sub development teams • System test team (test the product manually) • Test automation team (Current focus is to create test tools for the project) • Data collection: • Interviews conducted with development manager and QA manager. • Production bugs were collected from the reporting system and classified by type, severity and seasonality. 4.1 DMAIC – Define Phase - CTQs • Critical to quality (CTQs): are what the customer expect of a product or service. • The most important quality attributes for the project under study are: • Security • Reliability • Assurance • Efficiency • Maintainability DMAIC – Define Phase – SIPOC Diagram Supplier Input Process Output Customer Organization Test Manager Tool Create manual test cases for new planned features New feature test suite Project manager QA team High level requirements Execute test cases for each fully implemented new feature New feature test results document (HLR) Development QA manager Report bugs for new features List of identified bugs for new feature manager System requirements document Development team Verify bug fixes for new features Modified build after bug fixes QA manager Project stable build Development manager Retest all new features when all new planned features are Regression test results Organization Testing Tools fully implemented. Project manager Updated test cases Testing environment Perform regression testing Updated regression test suite to include Test plan Report bugs found in regression priority one test cases