DOCSLIB.ORG

Ubiquitous Automation Civilization Advances by Extending the Number of Important Operations We Can Perform Without Thinking

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Ubiquitous Automation Civilization Advances by Extending the Number of Important Operations We Can Perform Without Thinking software construction Editors: Andy Hunt and Dave Thomas I The Pragmatic Programmers [email protected] I [email protected] Ubiquitous Automation Civilization advances by extending the number of important operations we can perform without thinking. —Alfred North Whitehead Welcome to our new column on software Many authors and pundits have compelling argu- construction. We hope that you’ll enjoy follow- ments for each of these viewpoints.1–3 This is un- ing this series as we explore the practical nuts- derstandable because all the views have merit— and-bolts issues of building today’s software. software development is all these things, and this Before we start, though, we need to talk about plurality is what causes so much misunderstand- the column’s title. In some ways, “construction” is ing. When should we act like engineers, and an unfortunate term, in that the most immediate when shouldn’t we? (and often used) example involves building con- Many organizations still view coding as merely struction, with the attendant idea of an initial and a mechanical activity. They take the position that inviolate architecture from which springs forth de- design and architecture are of paramount impor- sign and code. tance and that coding should be a rigorous, re- peatable, mechanistic process that can be rele- gated to inexpensive, inexperienced novices. We wish these organizations the best of luck. We’d like to think we know a better way. The SWEBOK (Software Engineering Body of Knowl- edge—view an online draft at www.swebok.org), for instance, states unequivocally that coding is far from a mechanistic translation of good design into code, but instead is one of those messy, im- precise human activities that requires creativity, in- sight, and good communication skills. Good soft- ware is grown organically and evolves; it is not Of course, nothing could be further from the built slavishly and rigidly. Design and coding must truth. Software development, including its con- be flexible. We should not unduly constrain it in a struction, is utterly unlike any other human en- misguided attempt to turn coders into robots. deavor. (Software development is also exactly the However, the way we construct software same as all other human endeavors, but that’s a should not be arbitrary. It must be perfectly con- topic for another time.) Software is unique in both sistent, reliable, and repeatable, time after time. its malleability and its ephemerality; it is (to bor- And that’s the topic of this first column. row the title of a Thomas Disch book) the dreams —Andy Hunt and Dave Thomas our stuff is made of. Yet we cannot simply wish a software system into being. We must create it us- ing some semblance of engineering practice. This tension between the nonrepeatable, ill-defined, References 1. P. McBreen, Software Craftsmanship: The New Impera- chaotic creative process and the scientific, repeat- tive, Addison-Wesley, Reading, Mass., 2001. able, well-defined aspects of engineering is what 2. T. Bollinger, “The Interplay of Art and Science in Soft- causes so much heartburn in practitioners, au- ware,” Computer, vol. 30, no. 10, Oct. 1997, pp. 128, 125–126. thors, and scholars. 3. W. Humphrey, Managing the Software Process, Addi- Is software engineering? Is it art? Is it craft? son-Wesley, Reading, Mass., 1989. 0740-7459/02/$17.00 © 2002 IEEE January/February 2002 IEEE SOFTWARE 11 SOFTWAREDEPT CONSTRUCTION TITLE f software construction is to in- that every developer on a project com- tests to try to find holes in our soft- volve engineering, the process piles his or her software the same way, ware. (There are many other, equally must be consistent and repeat- using the same tools, against the same important, reasons for using unit able. Without consistency, know- set of dependencies. Make sure this tests, but that’s the subject of another ing how to build, test, or ship compilation process is automated. article.) However, most developers I someone else’s software (or even This advice might seem obvious, we’ve seen skip unit testing or at best your own software two years later) is but it’s surprising how often it’s ig- do it in an ad hoc way. The standard hard, if not impossible. Without re- nored. We know teams where some technique goes something like this: peatability, how can you guarantee developers compile using the auto- the results of a build or a test run— matic dependencies calculated by 1. Write a wad of code. how do you know the 100,000 CDs their integrated development environ- 2. Get scared enough about some as- you just burned contain the same ment, others use a second IDE, and pect of it to feel the need to try it. software you think you just tested? still others use command-line build 3. Write some kind of driver that in- The simple answer is discipline: tools. The result? Hours wasted every vokes the code just written. Add a software construction must be disci- week tracking down problems that few print statements to the code plined if it is to succeed. But people aren’t really problems, as each devel- under test to verify it’s doing what don’t seem to come that way; in gen- oper tests against subtly different ex- you thought it should. eral, folks find discipline hard to take ecutables generated by his or her in- 4. Run the test, eyeball the output, and even harder to maintain. Fortu- dividual compilation system. and then delete (or comment out) nately, there’s a pragmatic solution: Nowadays, automated compila- the prints. automate everything you can, so that tions are pretty straightforward. Most 5. Go back to Step 1. the development environment itself IDEs offer a single-key “compile the provides the disciplined consistency project” command. If you’re using Let us be clear. This is not unit test- and repeatability needed to make the Java from the command line, there’s ing. This is appeasing the gods. Why process run smoothly and reliably. the Ant tool (http://jakarta.apache.org/ invest in building tests only to throw This approach leaves developers free ant/index.html). In other environ- them away after you’ve run them to work on the more creative (and ments, the “make” system, or less once? And why rely on simply scan- fun) side of software construction, common variants such as Aegis (www. ning the results when you can have which makes the programmers hap- pcug.org.au/~millerp/aegis/aegis.html), the computer check them for you? pier. At the same time, it makes the do the same job. Whatever the tool, Fortunately, easy-to-use automated accountants happier by creating a de- the ground rules are the same: provide testing frameworks are available for velopment organization that is more a single command that works out most common programming lan- efficient and less prone to costly hu- what needs to be done, does it, and re- guages. A popular choice is the set of man-induced errors and omissions. ports any errors encountered. xUnit frameworks, based on the Gamma/Beck JUnit and SUnit systems Compilation automation Testing automation (www.xprogramming.com/software. If you do nothing else, make sure During construction, we use unit htm). We’ll look at these frameworks Career Opportunities PURDUE UNIVERSITY compilers, databases, distributed and research. Salary and benefits are highly Department of Computer Sciences parallel computing, geometric modeling competitive. Special departmental and and scientific visualization, graphics, in- university initiatives are available for ju- The Department of Computer Sciences formation security, networking and oper- nior faculty. Candidates should send a at Purdue University invites applications ating systems, programming languages, curriculum vitae, a statement of career for tenure-track positions beginning Au- scientific computing, and software engi- objectives, and names and contact infor- gust 2002. Positions are available at the neering. The department implements a mation of at least three references to: assistant professor level; senior positions strategic plan for future growth which is Chair, Faculty Search Committee will be considered for highly qualified ap- strongly supported by the higher admin- Department of Computer Sciences plicants. Applications from outstanding istration. This plan includes a new build- Purdue University candidates in all areas of computer sci- ing expected to be operational in 2004 to West Lafayette, IN 47907-1398 ence will be considered. Areas of particu- accommodate the significant growth in Applications are being accepted now lar interest include security, networking faculty size. Further information about and will be considered until the positions and distributed systems, scientific com- the department is available at are filled. Inquiries may be sent to puting, and software engineering. http://www.cs.purdue.edu. [email protected]. The Department of Computer Sciences Applicants should hold a Ph.D. in Purdue University is an Equal Opportu- offers a stimulating and nurturing acade- Computer Science, or a closely related nity/Affirmative Action mic environment. Thirty-five faculty discipline, and should be committed to employer. Women and minorities are es- members have research programs in excellence in teaching and have demon- pecially encouraged to apply. analysis of algorithms, bioinformatics, strated strong potential for excellence in 12 IEEE SOFTWARE January/February 2002 SOFTWAREDEPT CONSTRUCTION TITLE in later articles. For now, it’s enough it work when it gets there? Think have no shoes. Often, people who de- to point out that they are simple to about how much of this process you velop software use the poorest tools use, composable (so you can build can automate, allowing the quality to do the job. You can do better than suites of tests), and fully automated.
Load more

Recommended publications
  • Version Control Systems, Build Automation and Continuous Integration Integration and Verification Techniques
    Version Control Systems, Build Automation and Continuous Integration Integration and Verification Techniques Systematic methods and automation can highly enhance the teamwork on software projects. During this laboratory we will learn the basics of the following techniques: • Version control systems to support efficient teamwork. • Build automation to ensure stable and consistent builds. • Continuous Integration to provide automatic and systematic build, test execution and also deployments. 1 Version Control Systems Version control systems allow to keep all the historical versions of your software for easy tracking. Using version control also benefits team collaboration and improves the efficiency of the development team. In addition, it can be used as a central repository for the data, making build automation and continuous integration possible. There are historically two different approaches for managing the repositories: • Centralized model (CVS and Subversion/SVN): there is one server that contains “the repository” and everyone else checks code into and out of that repository. An important feature of these systems is that only the repository has the full history of all changes made. A checkout from this central repository will place a “working copy” on the user’s machine. This is a snapshot from a certain version of the project on his/her disk. • Distributed model (Git): In a distributed version control system, instead of a checkout, a user will clone a repository from a remote server. In return, he/she receives a full-fledged repository, not just a working copy. The user then has his/her own repository on the local machine – including all of the project’s history.
    [Show full text]
  • Devops Point of View an Enterprise Architecture Perspective
    DevOps Point of View An Enterprise Architecture perspective Amsterdam, 2020 Management summary “It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change.”1 Setting the scene Goal of this Point of View In the current world of IT and the development of This point of view aims to create awareness around the IT-related products or services, companies from transformation towards the DevOps way of working, to enterprise level to smaller sizes are starting to help gain understanding what DevOps is, why you need it use the DevOps processes and methods as a part and what is needed to implement DevOps. of their day-to-day organization process. The goal is to reduce the time involved in all the An Enterprise Architecture perspective software development phases, to achieve greater Even though it is DevOps from an Enterprise Architecture application stability and faster development service line perspective, this material has been gathered cycles. from our experiences with customers, combined with However not only on the technical side of the knowledge from subject matter experts and theory from organization is DevOps changing the playing within and outside Deloitte. field, also an organizational change that involves merging development and operations teams is Targeted audience required with an hint of cultural changes. And last but not least the skillset of all people It is specifically for the people within Deloitte that want to involved is changing. use this as an accelerator for conversations and proposals & to get in contact with the people who have performed these type of projects.
    [Show full text]
  • Coverity Static Analysis
    Coverity Static Analysis Quickly find and fix Overview critical security and Coverity® gives you the speed, ease of use, accuracy, industry standards compliance, and quality issues as you scalability that you need to develop high-quality, secure applications. Coverity identifies code critical software quality defects and security vulnerabilities in code as it’s written, early in the development process when it’s least costly and easiest to fix. Precise actionable remediation advice and context-specific eLearning help your developers understand how to fix their prioritized issues quickly, without having to become security experts. Coverity Benefits seamlessly integrates automated security testing into your CI/CD pipelines and supports your existing development tools and workflows. Choose where and how to do your • Get improved visibility into development: on-premises or in the cloud with the Polaris Software Integrity Platform™ security risk. Cross-product (SaaS), a highly scalable, cloud-based application security platform. Coverity supports 22 reporting provides a holistic, more languages and over 70 frameworks and templates. complete view of a project’s risk using best-in-class AppSec tools. Coverity includes Rapid Scan, a fast, lightweight static analysis engine optimized • Deployment flexibility. You for cloud-native applications and Infrastructure-as-Code (IaC). Rapid Scan runs decide which set of projects to do automatically, without additional configuration, with every Coverity scan and can also AppSec testing for: on-premises be run as part of full CI builds with conventional scan completion times. Rapid Scan can or in the cloud. also be deployed as a standalone scan engine in Code Sight™ or via the command line • Shift security testing left.
    [Show full text]
  • Security Automation Best Practices
    SECURITY AUTOMATION BEST PRACTICES A Guide to Making Your Security Team Successful with Automation TABLE OF CONTENTS Introduction 3 What Is Security Automation? 3 Security Automation: A Tough Nut to Crack 4 Prepare Your Security Organization for Success 6 Make a Choice: Build or buy? 8 Add Automation When the Time Is Right 10 Know Which Tasks Are Ideal for Automation 12 Testing Automation’s Capabilities 14 Implementing Security Automation 15 About Rapid7 16 Appendix 17 | Rapid7.com Security Automation Best Practices - 2 INTRODUCTION The best security postures are those that are built on efficiency and time-to-response. While processes make it possible to get a job done faster, creating ones that solve practical problems and result in measurable efficiency gains can be a time-consuming task, and without the expertise required to create and build them, they simply don’t get done. This is where security automation comes in. WHAT IS SECURITY AUTOMATION? Security automation streamlines a series of repetitive, manual tasks into cohesive and automated workflows. By plugging a set of tasks into an automated system (such as those involved in phishing investigations), security processes become: • More efficient • Less prone to human error With increased efficiency, better and faster decisions can be made, which in turn can improve your organization’s entire security posture. Even better, with repetitive and manual tasks taken care of by automation, security personnel can instead focus on more strategic work, which boosts their job satisfaction and ensures you’re retaining good talent. | Rapid7.com Security Automation Best Practices - 3 SECURITY AUTOMATION: A TOUGH NUT TO CRACK Historically, security automation has been difficult to implement, which is why many companies have yet to take advantage of it.
    [Show full text]
  • How to Get the Most out of Your Ci/Cd Workflow Using Automated Testing
    WHITE PAPER HOW TO GET THE MOST OUT OF YOUR CI/CD WORKFLOW USING AUTOMATED TESTING This paper is aimed at Test and QA Executives as well as Project Managers who are considering adopting automated testing, but are unsure of how to get started. It highlights the benefits of automated testing, the recommended technical approach to take, and suggests tools that enable teams to successfully adopt automated testing as part of a healthy continuous integration and delivery process. It also examines which tests to automate and which to continue to do manually. TABLE OF CONTENTS 3 Executive Summary 10 Mobile Testing 3 Automated Testing as Part of the Broader 11 Which Tests to Continue Manually CI/CD Pipeline 11 Usability Tests 4 The Kubernetes Effect 12 One-off Tests 5 GitOps = Fully-automated CI/CD 12 Selecting the Right Test Automation Solution 5 Reality Check - The Majority of Testing is 12 The Crucial Decision - In-house, Open Still Manual Source, or Commercial 6 What is Test Automation? 13 Selenium - The Leading Test Automation Tool 7 Manual Testing vs Automated Testing - for Web Apps Weighing the Benefits 13 Appium - The Leading Test Automation Tool 7 Obstacles to Adopting Automated Testing for Mobile Apps 8 The Right Approach to Test Automation 14 Open Source Tools Require Expertise to Run In-House 9 Unit and Component Testing 14 The Ideal Solution Should Combine the Best of 9 Headless Testing Both Worlds - Selenium & Appium 9 API or Web Services Testing 15 Conclusion 9 UI Testing 15 About Sauce Labs 10 Regression Testing 15 Appendix 10 Functional Testing EXECUTIVE SUMMARY In today’s hyper-competitive cloud economy, it’s important to be first to market to gain a competitive edge.
    [Show full text]
  • From Build Automation to Continuous Integration
    Beginners guide to continuous integration Gilles QUERRET Riverside Software About the speaker • Working with Progress and Java since 10 years • Started Riverside Software 7 years ago • Based in Lyon, France • Focused on technical expertise and continuous integration in those environments • Code analysis for OpenEdge http://geekandpoke.typepad.com define:continuous integration Continuous Integration is a software development practice of performing software integration frequently…several times a day, in fact. Ideally, your software application or system should be built automatically after each commit into a shared version control repository. Upon each successful build, the system integrity should be verified using automated tests that cover if not all, then at least most of the functionality. If some tests fail, the developer responsible is notified instantly and the problem can be identified and solved quickly. Using this approach, you can deliver working and reliable code to the customer much faster, while also mitigating the risk of releasing unstable, buggy software to your users. define:continuous integration 5 steps to continuous integration 1 •Source code repository 2 •Build automation 3 •CI server setup 4 •Automated deployment 5 •Automated tests Use the right tools Step 1 : Source code repository • Keeps track of every change in your codebase • Using homegrown solution can be challenging in a continuous integration environment • Lots of tools on the market : CVS, Subversion, Mercurial, Perforce, BitKeeper, Roundtable… • Side note
    [Show full text]
  • Static Code Analysis: a Buyer’S Guide
    WHITE PAPER STATIC CODE ANALYSIS: A BUYER’S GUIDE KEY DECISION CRITERIA FOR SELECTING TOOLS TO DEVELOP EMBEDDED SOFTWARE THAT IS MORE RELIABLE, SAFE, AND SECURE. www.programmingresearch.com TABLE OF CONTENTS OVERVIEW ..................................................................................................................... 3 WHAT IS STATIC CODE ANALYSIS? ............................................................................. 4 BENEFITS OF STATIC CODE ANALYSIS ...................................................................... 4 Reduces Risks Associated with Large, Complex Code Bases .............................. 4 Improves Security and Reduces Vulnerabilities..................................................... 4 Enhances Reliability and Safety ............................................................................ 5 Streamlines Processes .......................................................................................... 5 Saves Time and Reduce Costs through Early Detection and Remediation ........... 5 CHALLENGES OF SOURCE CODE ANALYSIS ............................................................ 6 Static Analysis is Not Dynamic .............................................................................. 6 Too Much Noise and Too Many False Results ...................................................... 6 Too Many Errors To Fix ......................................................................................... 7 Diffculties with Process Integration ......................................................................
    [Show full text]
  • Understanding and Improving Software Build Teams
    Understanding and Improving Software Build Teams Shaun Phillips Thomas Zimmermann Christian Bird University of Calgary Microsoft Research Microsoft Research Calgary, Alberta, Canada Redmond, WA, USA Redmond, WA, USA [email protected] [email protected] [email protected] ABSTRACT In real-world scenarios, a build process will likely consist of Build, creating software from source code, is a fundamental activity many more complex tasks. For instance, there may be scheduling in software development. Build teams manage this process and en- of build machines, build metric collection (such as time or power sure builds are produced reliably and efficiently. This paper presents consumption), security checks, so-called “smoke” tests to ensure an exploration into the nature of build teams—how they form, work, some baseline level of quality, language localization, reporting of and relate to other teams—through three multi-method studies con- build results, and triaging of build problems. All together, the ducted at Microsoft. We also consider build team effectiveness and collection of scripts, programs, and services that comprise and find that many challenges are social, not technical: role ambiguity, automate a build process is called a build system. knowledge sharing, communication, trust, and conflict. Our findings The build process is cyclical. Development teams write code validate theories from group dynamics and organization science, and which at some point is processed by the build system. The output, using a cross-discipline approach, we apply learnings from these the build, is deployed back to the development teams to verify the fields to inform the design of engineering tools and practices to changes [19] and the cycle begins again.
    [Show full text]
  • Mgr Inż. Marcin Kawalerowicz Classification Of
    MGR INŻ. MARCIN KAWALEROWICZ CLASSIFICATION OF AUTOMATIC SOFTWARE BUILD METHODS Summary: The process of creating working software from source code and other components (like libraries, database files, etc.) is called “software build”. Apart from linking and compiling, it can include other steps like automated testing, static code analysis, documentation generation, deployment and other. All that steps can be automated using a build description of some sort (e.g. script). This article classifies the automatic software build processes beginning at build script and reaching the various types of continuous integration. Keywords: software build automation, continuous integration, commanded integration, scheduled software build, triggered software build Chapter 1. Classification of automatic software build methods Software build process at a basic level consists of the translating the source code into working software. The translated code is often unit tested. More mature software build processes include more steps like complex testing, code metrics, code analysis, deployment [1] and other. So the build process contains a set of repetitive manual tasks that integrate the various components (source code, libraries, data files, etc.) into software. The tasks need to be executed in certain chronological order. Output of every task can vary from execution to execution, depending on the state the source code is in. So the next task to execute can vary depending on the output of the previous task. As a series of repetitive tasks the software build process done by humans is error prone [2]. To eliminate the uncertainty of the “human factor” the build process is often recommended to be automated [3]. Figure 1 shows the build automation diagram containing the classification of the automatic software build processes.
    [Show full text]
  • Plan, Code, Collaborate, and Ship Your Applications Faster with Azure Devops
    Guide Azure DevOps Services Plan smarter, collaborate better, and ship faster with a set of modern dev services. 0 Contents Azure DevOps user guide Overview What is Azure DevOps? Overview of services Where's VSTS? Quickstarts for users Code with Git Set up continuous integration & delivery Plan & track work Add & run manual tests View permissions Quickstarts for admins Sign up for Azure DevOps Create organizations Add users to a project or team Configure team resources Get started as an admin Install a Marketplace extension Tutorials for users Set favorites Follow work & pull requests Get started as a Stakeholder Tutorials for admins Change individual permissions Grant or restrict permissions to select tasks Concepts Key concepts Define organizations and projects Source control Backlogs, boards, & Agile tools Clients & tools Software development roles What's the difference? Azure DevOps Services vs. TFS Glossary How-to guides Sign-in to the web or a client Troubleshooting Troubleshoot connection TF31002: Unable to connect Troubleshoot network connections and whitelist addresses Get support, provide feedback Reference Permissions & access (Security) About access levels Keyboard shortcuts Navigate in Team Explorer FAQs Resources Web portal navigation Manage projects Security & identity Billing Visual Studio IDE Visual Studio Code Visual Studio for Mac What is Azure DevOps Services? 9/10/2018 • 2 minutes to read • Edit Online Azure DevOps Services Azure DevOps Services is a cloud service for collaborating on code development. It provides
    [Show full text]
  • Devsecops Fundamentals Guidebook
    Unclassified UNCLASSIFIED DevSecOps Fundamentals Guidebook: DevSecOps Tools & Activities March 2021 Version 2.0 DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited. 1 UNCLASSIFIEDUnclassified UNCLASSIFIED Document Set Reference 2 UNCLASSIFIED UNCLASSIFIED Trademark Information Names, products, and services referenced within this document may be the trade names, trademarks, or service marks of their respective owners. References to commercial vendors and their products or services are provided strictly as a convenience to our readers, and do not constitute or imply endorsement by the Department of any non-Federal entity, event, product, service, or enterprise. 3 UNCLASSIFIED UNCLASSIFIED Contents Document Set Reference .............................................................................................................. 2 Trademark Information .................................................................................................................. 3 Introduction ................................................................................................................................... 6 Audience and Scope ................................................................................................................. 6 DevSecOps Tools and Activities ................................................................................................... 7 Security Tools & Activities Cross Reference ............................................................................. 8 Plan Tools and
    [Show full text]
  • Accelerating Embedded Software Verification with Polyspace Static Code Analysis
    Accelerating embedded software verification with Polyspace static code analysis Stefan David © 2019 The MathWorks, Inc.1 Agenda 1. Making Software Safe and Secure 2. Polyspace Static Analysis 3. Team Collaboration with Polyspace 3 1. Making Software Safe and Secure 4 Security is on consumers’ minds …of customers would …of automakers admit their never buy from an OEM if organization had been they had been hacked hacked in the past 2 years according to 2016 KPMG Consumer Loss Barometer study 5 In the News.... Embedded Software Security - New Challenge Source: https://www.wired.com/2016/08/jeep-hackers-return-high-speed-steering-acceleration-hacks/ 6 In the News.... Embedded Software Security - New Challenge Source: https://www.wired.com/2016/08/jeep-hackers-return-high-speed-steering-acceleration-hacks/ 7 Safety & Security Goals Safety System issue Environment Security Environment attack System Note: Security issues may cause safety issues 9 source: https://www.cvedetails.com (CVE ... Common Vulnerabilities and Exposures) 10 When Software Safety and Security Matter ▪ Industries where safety and security matter – Automotive, Aerospace, Medical Device, Industrial Machinery ▪ Governed by functional safety and other standards – ISO 26262, DO-178, IEC 62304, IEC 61508 – ISO/SAE 21434, RTCA DO-326 – MISRA, CERT, AUTOSAR ▪ Static analysis provides certification credits – For standards such as ISO 26262 and DO-178 11 source: https://www.securecoding.cert.org Validate inputs Heed compiler warnings and use static and dynamic analysis tools Architect/Design Software for security policies 12 “Program testing can be used to show the presence of bugs, but neverProgram to show their Testing absence” Edsger Dijkstra, Computer Science Pioneer “Given that we cannot really show there are no more errors in the program, when do we stop testing?” Brent Hailpern, Head of Computer Science, IBM Dijstra, “Notes on Structured Programming” (1972) Hailern, Santhanam, “Software Debugging, Testing, and Verification”, IBM Systems Journal, (2002) 14 2.
    [Show full text]