DOCSLIB.ORG

An Introduction to Continuous Integration Testing What It Is, Why Use It and How to Test

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
An Introduction to Continuous Integration Testing What It Is, Why Use It and How to Test An Introduction to Continuous Integration Testing What it is, why use it and how to test Continuous integration means building your software every time a developer pushes code. This affects your testing and requires careful planning and the use of test automation. Introduction Continuous integration (CI) is the process of building a new version of your software every time you push a change. Clearly, CI has a big impact on testing, since it means you are never testing stable software. On the other hand, the changes you are testing are much smaller. This means you have to adopt a different approach to testing than was traditionally the case. In this article, we will explore CI and look at the challenges it creates for testers. Continuous Integration The background and motivation for CI CI isn’t new, but it marked a change from the traditional way of building software. To do CI well requires careful coordination and buy-in from both thedevelopers and testers. Once upon a time, developers would work individually and in teams on developing their part of the software. Every few weeks they would upload all their new code to the build server, along with the code from all other teams. The next stage was to try and resolve all the merge conflicts. Then you could run your build, at which point large numbers of errors probably showed up. Finally, you would have a working build, ready to pass to the testers. It doesn’t take a genius to see the flaws in this approach. Various tricks were used to try and improve functionize.com © 2020 Functionize, Inc. All Rights Reserved. 1 this process. For instance, large pieces of software would be broken into small modules and treated as black boxes. As long as the interface worked, you removed the inter-dependencies. However, that can only go a short way to solving the issue. And so, we come to continuous integration. What exactly is CI? CI isn’t rocket science. The problem is that merging large amounts of new code caused things to break. So, the solution is to merge smaller amounts of code! Martin Fowler (one of the authors of the Agile manifesto) describes CI as follows: Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. CI is a pre-requisite for continuous deployment (which we discussed in a recent blog). In continuous deployment, you release the newly built code to production as soon as it is ready. However, CI and CD are distinct from each other, and many companies only do CI. Why is CI such a good approach? CI is a powerful approach that can be applied in most companies. CI will benefit any size of team, so let’s look at a few of the key benefits. 1 Frequent merges reduce merge conflicts. Anyone who has fought with git merge will know the pain of merge conflicts. Sometimes solving these is simple, but other times it needs major changes. At the least, it is a time sink. 2 Developers benefit from rapid feedback, especially when it’s related to an error they have to fix. So, if you build and test the code at least daily, the errors are likely to be in code that has just been committed. This means the developer won’t have to reload state about code she worked on weeks ago. 3 Everyone is always working on the newest codebase. This significantly reduces the risk of problems in complex pieces of software. It also means that developers can see what each other is doing much more easily. 4 Any changes to the backend environment are instantly available for frontend developers to work with. This removes a key source of problems for many developers. functionize.com © 2020 Functionize, Inc. All Rights Reserved. 2 5 It enables continuous deployment. Even if you don’t practice this as a company, being in a position where your code is always deployable is a distinct advantage. 6 Product managers can also benefit because they get to see the latest product changes instantly. There are other benefits too. For instance, developers are learning from each other’s mistakes and you know who to blame when things break. What are the requirements for CI? Enabling continuous integration requires a few things to be put in place. Let’s look at the most important of these in a bit of detail: 7 Code repo. You need all your code to be in a central repository, ideally something like GitHub, Bitbucket or the like. Where you rely on external libraries you need copies of these too. 8 Single-action builds. You must be able to launch your build process with a single action, such as a CLI command, mouse click or by automatically monitoring code commits. This is where CI tools like Jenkins come into their own 9 Automated tests. All your tests must be automated so that as soon as the build is complete it can be verified. More on this later. The CI process flow Continuous integration follows a simple flow which we show in the diagram below. When a developer pushes new code to the commit system, the CI system is notified. It then triggers a build. If the build fails, then the developer is told. If the build succeeds, the test system is updated. The automated test system now runs a suitable set of tests (see later). This may be triggered automatically, periodically or by the CI system. The results of the test are passed to the CI system. This either tells the developer the code is OK or indicates that it failed a test. functionize.com © 2020 Functionize, Inc. All Rights Reserved. 3 How to design your testing for CI True continuous integration would imply that every single build gets fully tested. However, even with the best test automation system in the world, this simply isn’t possible. Some tests take a long time to run, and not all tests are equally important. Let’s have a look at both these issues. Time to test A full set of tests covers everything from unit testing to full regression and UI testing. Unit tests are, by definition, small. So, unit tests can be completed rapidly. By contrast, a full set ofregression tests can easily take hours to complete. This is especially the case if you have to reset the system state between each test (as is often the case). Often your developers will push code several times a day. This means there simply isn’t time to run all the tests for every build. The relative importance of tests As you will know, not all tests are of equal value. Some may be testing an obscure corner case, while others test the central functionality of your entire system. This factor can be used to prioritize your tests. Test prioritization By splitting your tests into three categories you can streamline your CI testing. Firstly, every build should be subjected to a full set of (suitable) unit tests and a smoke test. A smoke test is designed to test the minimal necessary functionality of your system. It should show up the most fundamental issues but may miss some obscure ones. You should design your smoke test to run in a maximum of a couple of minutes. Every hour you can subject the latest build to a more thorough set of tests, so long as they will complete within the time you have. You should choose the tests that are of highest importance. Finally, you should run ALL your regression tests overnight or over the weekend if any really take a long time. (I once worked for a company designing storage solutions. Since our tests required reformatting entire RAID arrays, some of them took a day to complete). Potential issues with CI testing There are a couple of real gotchas that can cause problems for testing with continuous integration. The first is the test maintenance issue. By definition, your CI system is relying on your tests working. The only failures should be genuine code failures. However, many automated test systems suffer from spurious failures (just speak to anyone that has used Selenium). The second is that new code functionize.com © 2020 Functionize, Inc. All Rights Reserved. 4 requires you to create new tests. The combination of both these things can take a lot of time and resources to solve. And while they are being solved your testing may be blocked. How Functionize helps The Functionize system simplifies and streamlines automated testing of web applications. We make it easy to integrate with CI systems. Our system is powered by AI, which is why we talk about autonomous testing rather than automated testing. What we mean by that is that our system can recover from most test failures without any human intervention. We call this feature “self-healing”. We also make writing new tests as simple as writing plain English. Our Adaptive Language Processing (ALP™) engine then takes these and automatically converts them into tests for you. Intelligent Testing As the world becomes more agile traditional test automation creates bottlenecks that can slow your release cycles to a crawl. Functionize combines natural language processing, deep-learning ML models and other AI-based technologies to empower your team to build tests faster that don’t break and run at scale in the cloud.
Load more

Recommended publications
  • Core Elements of Continuous Testing
    WHITE PAPER CORE ELEMENTS OF CONTINUOUS TESTING Today’s modern development disciplines -- whether Agile, Continuous Integration (CI) or Continuous Delivery (CD) -- have completely transformed how teams develop and deliver applications. Companies that need to compete in today’s fast-paced digital economy must also transform how they test. Successful teams know the secret sauce to delivering high quality digital experiences fast is continuous testing. This paper will define continuous testing, explain what it is, why it’s important, and the core elements and tactical changes development and QA teams need to make in order to succeed at this emerging practice. TABLE OF CONTENTS 3 What is Continuous Testing? 6 Tactical Engineering Considerations 3 Why Continuous Testing? 7 Benefits of Continuous Testing 4 Core Elements of Continuous Testing WHAT IS CONTINUOUS TESTING? Continuous testing is the practice of executing automated tests throughout the software development cycle. It’s more than just automated testing; it’s applying the right level of automation at each stage in the development process. Unlike legacy testing methods that occur at the end of the development cycle, continuous testing occurs at multiple stages, including development, integration, pre-release, and in production. Continuous testing ensures that bugs are caught and fixed far earlier in the development process, improving overall quality while saving significant time and money. WHY CONTINUOUS TESTING? Continuous testing is a critical requirement for organizations that are shifting left towards CI or CD, both modern development practices that ensure faster time to market. When automated testing is coupled with a CI server, tests can instantly be kicked off with every build, and alerts with passing or failing test results can be delivered directly to the development team in real time.
    [Show full text]
  • Rugby - a Process Model for Continuous Software Engineering
    INSTITUT FUR¨ INFORMATIK DER TECHNISCHEN UNIVERSITAT¨ MUNCHEN¨ Forschungs- und Lehreinheit I Angewandte Softwaretechnik Rugby - A Process Model for Continuous Software Engineering Stephan Tobias Krusche Vollstandiger¨ Abdruck der von der Fakultat¨ fur¨ Informatik der Technischen Universitat¨ Munchen¨ zur Erlangung des akademischen Grades eines Doktors der Naturwissenschaften (Dr. rer. nat.) genehmigten Dissertation. Vorsitzender: Univ.-Prof. Dr. Helmut Seidl Prufer¨ der Dissertation: 1. Univ.-Prof. Bernd Brugge,¨ Ph.D. 2. Prof. Dr. Jurgen¨ Borstler,¨ Blekinge Institute of Technology, Karlskrona, Schweden Die Dissertation wurde am 28.01.2016 bei der Technischen Universitat¨ Munchen¨ eingereicht und durch die Fakultat¨ fur¨ Informatik am 29.02.2016 angenommen. Abstract Software is developed in increasingly dynamic environments. Organizations need the capability to deal with uncertainty and to react to unexpected changes in require- ments and technologies. Agile methods already improve the flexibility towards changes and with the emergence of continuous delivery, regular feedback loops have become possible. The abilities to maintain high code quality through reviews, to regularly re- lease software, and to collect and prioritize user feedback, are necessary for con- tinuous software engineering. However, there exists no uniform process model that handles the increasing number of reviews, releases and feedback reports. In this dissertation, we describe Rugby, a process model for continuous software en- gineering that is based on a meta model, which treats development activities as parallel workflows and which allows tailoring, customization and extension. Rugby includes a change model and treats changes as events that activate workflows. It integrates re- view management, release management, and feedback management as workflows. As a consequence, Rugby handles the increasing number of reviews, releases and feedback and at the same time decreases their size and effort.
    [Show full text]
  • Towards Embedded System Agile Development Challenging Verification, Validation and Accreditation
    Towards Embedded System Agile Development Challenging Verification, Validation and Accreditation : Application in a Healthcare Company Clément Duffau, Bartosz Grabiec, Mireille Blay-Fornarino To cite this version: Clément Duffau, Bartosz Grabiec, Mireille Blay-Fornarino. Towards Embedded System Agile Develop- ment Challenging Verification, Validation and Accreditation : Application in a Healthcare Company. ISSRE 2017- IEEE International Symposium on Software Reliability Engineering, Oct 2017, Toulouse, France. pp.1-4. hal-01678815 HAL Id: hal-01678815 https://hal.archives-ouvertes.fr/hal-01678815 Submitted on 9 Jan 2018 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Towards Embedded System Agile Development Challenging Verification, Validation and Accreditation Application in a Healthcare Company Clement´ Duffau Bartosz Grabiec Mireille Blay-Fornarino AXONIC and I3S AXONIC I3S Universite´ Coteˆ d’Azur, CNRS Sophia Antipolis, France Universite´ Coteˆ d’Azur, CNRS Sophia Antipolis, France [email protected] Sophia Antipolis, France [email protected] [email protected] Abstract—When Agile development meets critical embedded to survive. Moreover, with limited human resources, quality systems, verification, validation and accreditation activities are improvement with Agility usage remains to be demonstrated. impacted. Challenges such as tests increase or accreditation The remainder of this paper is organized as follows.
    [Show full text]
  • Leading Practice: Test Strategy and Approach in Agile Projects
    CA SERVICES | LEADING PRACTICE Leading Practice: Test Strategy and Approach in Agile Projects Abstract This document provides best practices on how to strategize testing CA Project and Portfolio Management (CA PPM) in an agile project. The document does not include specific test cases; the list of test cases and steps for each test case are provided in a separate document. This document should be used by the agile project team that is planning the testing activities, and by end users who perform user acceptance testing (UAT). Concepts Concept Description Test Approach Defines testing strategy, roles and responsibilities of various team members, and test types. Testing Environments Outlines which testing is carried out in which environment. Testing Automation and Tools Addresses test management and automation tools required for test execution. Risk Analysis Defines the approach for risk identification and plans to mitigate risks as well as a contingency plan. Test Planning and Execution Defines the approach to plan the test cases, test scripts, and execution. Review and Approval Lists individuals who should review, approve and sign off on test results. Test Approach The test approach defines testing strategy, roles and responsibilities of various team members, and the test types. The first step is to define the testing strategy. It should describe how and when the testing will be conducted, who will do the testing, the type of testing being conducted, features being tested, environment(s) where the testing takes place, what testing tools are used, and how are defects tracked and managed. The testing strategy should be prepared by the agile core team.
    [Show full text]
  • Chap 3. Test Models and Strategies 3.5 Test Integration and Automation 1
    Chap 3. Test Models and Strategies 3.5 Test Integration and Automation 1. Introduction 2. Integration Testing 3. System Testing 4. Test Automation Appendix: JUnit Overview 1 1. Introduction -A system is composed of components. System of software components can be defined at any physical scope. Component System Typical intercomponent interfaces (locus of (Focus of Integration) (Scope of integration faults) Integration) Method Class Instance variables Intraclass messages Class Cluster Intraclass messages Cluster Subsystem Interclass messages Interpackage messages Subsystem System Inteprocess communication Remote procedure call ORB services, OS services -Integration test design is concerned with several primary questions: 1. Which components and interfaces should be exercised? 2. In what sequence will component interfaces be exercised? 2 3. Which test design technique should be used to exercise each interface? -Integration testing is a search for component faults that cause intercomponent failures. -System scope testing is a search for faults that lead to a failure to meet a system scope responsibility. ÷System scope testing cannot be done unless components interoperate sufficiently well to exercise system scope responsibilities. -Effective testing at system scope requires a concrete and testable system-level specification. ÷System test cases must be derived from some kind of functional specification. Traditionally, user documentation, product literature, line-item narrative requirements, and system scope models have been used. 3 2. Integration Testing -Unit testing focuses on individual components. Once faults in each component have been removed and the test cases do not reveal any new fault, components are ready to be integrated into larger subsystems. -Integration testing detects faults that have not been detected during unit testing, by focusing on small groups of components.
    [Show full text]
  • Devops Point of View an Enterprise Architecture Perspective
    DevOps Point of View An Enterprise Architecture perspective Amsterdam, 2020 Management summary “It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change.”1 Setting the scene Goal of this Point of View In the current world of IT and the development of This point of view aims to create awareness around the IT-related products or services, companies from transformation towards the DevOps way of working, to enterprise level to smaller sizes are starting to help gain understanding what DevOps is, why you need it use the DevOps processes and methods as a part and what is needed to implement DevOps. of their day-to-day organization process. The goal is to reduce the time involved in all the An Enterprise Architecture perspective software development phases, to achieve greater Even though it is DevOps from an Enterprise Architecture application stability and faster development service line perspective, this material has been gathered cycles. from our experiences with customers, combined with However not only on the technical side of the knowledge from subject matter experts and theory from organization is DevOps changing the playing within and outside Deloitte. field, also an organizational change that involves merging development and operations teams is Targeted audience required with an hint of cultural changes. And last but not least the skillset of all people It is specifically for the people within Deloitte that want to involved is changing. use this as an accelerator for conversations and proposals & to get in contact with the people who have performed these type of projects.
    [Show full text]
  • White Paper Getting Started with Continuous Integration in Software
    WHITE PAPER GETTING STARTED WITH CONTINUOUS INTEGRATION IN SOFTWARE DEVELOPMENT Amruta Kumbhar, Madhavi Shailaja & Ravi Shankar Anupindi Introduction DevOps culture is gaining rapid momentum in the IT industry as it enables business to adopt agile software delivery methodologies like Continuous Integration (henceforth referred to as CI) & Continuous Delivery (henceforth referred to as CD) .These methodologies enable quicker issue resolution, instant feedback loops, improved software quality and cost saving to meet the ever- increasing demand to deliver better software faster. It would not be wrong to say that CI-CD practices will soon become the de- facto software delivery standards across the industry. Though both these methodologies (CI and CD) This article compiles insights gained from complement each other, CI is the pre-requisite our practical experiences across various CI phase for enabling CD as the latter is built on enablement programs which we have been Gartner says “By 2016, top of the former. The primary goal of CI is associated with. not only to enable build automation through DevOps will evolve Some of these experiences might vary continuous test and quality checks but also to from a niche to a depending on the choice of tools, provide project insights through reports and infrastructure setup, organization policies mainstream strategy dashboards. Since this phase is all about tools, and project requirements. In spite of the employed by 25 it imposes various integration challenges. differences, we hope, this article will act as Having a good knowledge of the tools Percent of global 2000 a helping guide to all those planning for CI involved, their integration aspects and the organizations” setup in their projects.
    [Show full text]
  • Integration Testing of Object-Oriented Software
    POLITECNICO DI MILANO DOTTORATO DI RICERCA IN INGEGNERIA INFORMATICA E AUTOMATICA Integration Testing of Object-Oriented Software Ph.D. Thesis of: Alessandro Orso Advisor: Prof. Mauro Pezze` Tutor: Prof. Carlo Ghezzi Supervisor of the Ph.D. Program: Prof. Carlo Ghezzi XI ciclo To my family Acknowledgments Finding the right words and the right way for expressing acknowledgments is a diffi- cult task. I hope the following will not sound as a set of ritual formulas, since I mean every single word. First of all I wish to thank professor Mauro Pezze` , for his guidance, his support, and his patience during my work. I know that “taking care” of me has been a hard work, but he only has himself to blame for my starting a Ph.D. program. A very special thank to Professor Carlo Ghezzi for his teachings, for his willingness to help me, and for allowing me to restlessly “steal” books and journals from his office. Now, I can bring them back (at least the one I remember...) Then, I wish to thank my family. I owe them a lot (and even if I don't show this very often; I know this very well). All my love goes to them. Special thanks are due to all my long time and not-so-long time friends. They are (stricty in alphabetical order): Alessandro “Pari” Parimbelli, Ambrogio “Bobo” Usuelli, Andrea “Maken” Machini, Antonio “the Awesome” Carzaniga, Dario “Pitone” Galbiati, Federico “Fede” Clonfero, Flavio “Spadone” Spada, Gianpaolo “the Red One” Cugola, Giovanni “Negroni” Denaro, Giovanni “Muscle Man” Vigna, Lorenzo “the Diver” Riva, Matteo “Prada” Pradella, Mattia “il Monga” Monga, Niels “l’e´ semper chi” Kierkegaard, Pierluigi “San Peter” Sanpietro, Sergio “Que viva Mex- ico” Silva.
    [Show full text]
  • A Confused Tester in Agile World … Qa a Liability Or an Asset
    A CONFUSED TESTER IN AGILE WORLD … QA A LIABILITY OR AN ASSET THIS IS A WORK OF FACTS & FINDINGS BASED ON TRUE STORIES OF ONE & MANY TESTERS !! J Presented By Ashish Kumar, WHAT’S AHEAD • A STORY OF TESTING. • FROM THE MIND OF A CONFUSED TESTER. • FEW CASE STUDIES. • CHALLENGES IDENTIFIED. • SURVEY STUDIES. • GLOBAL RESPONSES. • SOLUTION APPROACH. • PRINCIPLES AND PRACTICES. • CONCLUSION & RECAP. • Q & A. A STORY OF TESTING IN AGILE… HAVE YOU HEARD ANY OF THESE ?? • YOU DON’T NEED A DEDICATED SOFTWARE TESTING TEAM ON YOUR AGILE TEAMS • IF WE HAVE BDD,ATDD,TDD,UI AUTOMATION , UNIT TEST >> WHAT IS THE NEED OF MANUAL TESTING ?? • WE WANT 100% AUTOMATION IN THIS PROJECT • TESTING IS BECOMING BOTTLENECK AND REASON OF SPRINT FAILURE • REPEATING REGRESSION IS A BIG TASK AND AN OVERHEAD • MICROSOFT HAS NO TESTERS NOT EVEN GOOGLE, FACEBOOK AND CISCO • 15K+ DEVELOPERS /4K+ PROJECTS UNDER ACTIVE • IN A “MOBILE-FIRST AND CLOUD-FIRST WORLD.” DEVELOPMENT/50% CODE CHANGES PER MONTH. • THE EFFORT, KNOWN AS AGILE SOFTWARE DEVELOPMENT, • 5500+ SUBMISSION PER DAY ON AVERAGE IS DESIGNED TO LOWER COSTS AND HONE OPERATIONS AS THE COMPANY FOCUSES ON BUILDING CLOUD AND • 20+ SUSTAINED CODE CHANGES/MIN WITH 60+PEAKS MOBILE SOFTWARE, SAY ANALYSTS • 75+ MILLION TEST CASES RUN PER DAY. • MR. NADELLA TOLD BLOOMBERG THAT IT MAKES MORE • DEVELOPERS OWN TESTING AND DEVELOPERS OWN SENSE TO HAVE DEVELOPERS TEST & FIX BUGS INSTEAD OF QUALITY. SEPARATE TEAM OF TESTERS TO BUILD CLOUD SOFTWARE. • GOOGLE HAVE PEOPLE WHO COULD CODE AND WANTED • SUCH AN APPROACH, A DEPARTURE FROM THE TO APPLY THAT SKILL TO THE DEVELOPMENT OF TOOLS, COMPANY’S TRADITIONAL PRACTICE OF DIVIDING INFRASTRUCTURE, AND TEST AUTOMATION.
    [Show full text]
  • Softwareprozesse: Timeboxing & Continuous Delivery
    Softwareprozesse: Timeboxing & Continuous Delivery BENJAMIN PROJDAKOV The Timeboxing Process Model •Voller Titel: The Timeboxing Process Model for Iterative Software Development •Verfasser: • Professor Pankaj Jalote Department of Computer Science and Engineering Indian Institute of Technology Kanpur • Aveejeet Palit, Priya Kurien Infosys Technologies Limited Bangalore Einführung: Wasserfallmodell •Linear •Nicht iterativ •Lange Entwicklungsdauer typisch •Anforderungen: • Müssen vollständig bekannt sein • Änderungen schwierig •Entwicklung in Phasen •Ergebnis: Vollständige Software •Komplexität der Software eher gering Einführung: Iterative Modelle •Linear •Entwicklung: • In Zyklen • Zyklus unterteilt sich in Phasen •Kurze Entwicklungsdauer pro Zyklus •Anforderungen: • Müssen pro Iteration bekannt sein • Änderungen in nächster Iteration möglich •Ergebnis: Funktionierende Software nach jedem Zyklus Timeboxing Modell •Entwicklung: • In Zyklen • Zyklus unterteilt sich in Phasen • Ausführung mit Versatz parallel •Feste Entwicklungsdauer pro Zyklus • Wochen bis Monate •Anforderungen: • Müssen pro Iteration bekannt sein • Änderungen in nächster Iteration •Ergebnis: • Häufige Releases • Funktionierende Software nach jedem Zyklus Timeboxing Modell: Phasen & Teams •Phasen: •Teams: • Anzahl Variabel • Eine Aufgabe pro Team • Gleiche Dauer ideal • Größe abgestimmt auf Dauer Timeboxing Modell: Phasen & Teams •Ungleiche Boxen führen zu ineffizienter Teamauslastung •Lösungen: • “Right-scaling” von Teams • Ressourcenumverteilung •Kann auch durch Verzögerungen
    [Show full text]
  • ISO 26262 Software Compliance with Parasoft C++Test
    ISO 26262 Software Compliance with Parasoft: Achieving Functional Safety in the Automotive Industry Some modern automobiles have more lines of code than a jet fighter. Even moderately sophisticated cars ship with larger and more complex codebases than the same line from just a few years ago. The inclusion of multi-featured infotainment systems, driver-assist technologies, and electronically controlled safety features as standard components—even in economy models—have fueled the growth of software in the automotive industry. Additionally, the emergence of driverless technology and “connected” cars that function as IoT systems on wheels will mean even larger and more complex codebases. All of the innovation taking place in the automotive industry, though, raises concerns over the safety, security, and reliability of automotive electronic systems. The concerns are appropriate given that the automotive software supply chain is a long convoluted system of third-party providers spanning several tiers. Consider, for example, that software developed for a specific microcontroller unit (MCU) may be integrated by a third-tier provider into a component they’re shipping to a second-tier provider and so on—until a composite component is delivered for final integration by the automaker. While not all automotive software is critical to the safe operation of the vehicle, code that carries out functional safety operations must be safe, secure, and reliable. Organizations must implement strong software quality process controls around the development of safety-critical software in accordance with ISO 26262, which is a functional safety standard for automotive software. ISO 26262 provides guidance on processes associated with software development for electrical and/or electronic (E/E) systems in automobiles.
    [Show full text]
  • Testing and Debugging C Programming and Software Tools N.C
    Testing and Debugging C Programming and Software Tools N.C. State Department of Computer Science Introduction • Majority of software development is testing, debugging, and bug fixing • The best software developers are 10X (!) more productive than other developers; why??? CSC230: C and Software Tools © NC State Computer Science Faculty 2 Why Do Bugs Happen ? • OS problem? Compiler? Hardware? – not likely • Unclear requirements / specifications, constantly changing, unreasonable schedules, … • Lack of mastery of programming tools / language • Inadequate testing procedures • Faulty logic Addressed in this course CSC230: C and Software Tools © NC State Computer Science Faculty 3 Testing Procedures • Types of testing – Unit testing – test each function – Integration testing – test interaction between units and components – System testing – testing complete system – Regression testing – selective retesting – Acceptance Testing – testing of acceptance criteria – Beta Testing – 3rd party testing • Test logging • Bug fixing – test one change at a time – maintain old versions, and log the changes CSC230: C and Software Tools © NC State Computer Science Faculty 4 Test Case Information • Unique Identifier – Black box: name of test input/output files • Input into the program or program unit – Black box: how the user runs and interacts with the program • Could be redirection input and output • Expected output from the program or program unit – What you expect to get based on input and requirements • Stored in a file that can be compared with actual
    [Show full text]