DOCSLIB.ORG

Refactoring of Preprocessor Directives in the #Ifdef Hell

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Refactoring of Preprocessor Directives in the #Ifdef Hell 1 Discipline Matters: Refactoring of Preprocessor Directives in the #ifdef Hell Flávio Medeiros, Márcio Ribeiro, Rohit Gheyi, Sven Apel, Christian Kästner, Bruno Ferreira, Luiz Carvalho, and Baldoino Fonseca Abstract—The C preprocessor is used in many C projects to support variability and portability. However, researchers and practitioners criticize the C preprocessor because of its negative effect on code understanding and maintainability and its error proneness. More importantly, the use of the preprocessor hinders the development of tool support that is standard in other languages, such as automated refactoring. Developers aggravate these problems when using the preprocessor in undisciplined ways (e.g., conditional blocks that do not align with the syntactic structure of the code). In this article, we proposed a catalogue of refactorings and we evaluated the number of application possibilities of the refactorings in practice, the opinion of developers about the usefulness of the refactorings, and whether the refactorings preserve behavior. Overall, we found 5670 application possibilities for the refactorings in 63 real-world C projects. In addition, we performed an online survey among 246 developers, and we submitted 28 patches to convert undisciplined directives into disciplined ones. According to our results, 63% of developers prefer to use the refactored (i.e., disciplined) version of the code instead of the original code with undisciplined preprocessor usage. To verify that the refactorings are indeed behavior preserving, we applied them to more than 36 thousand programs generated automatically using a model of a subset of the C language, running the same test cases in the original and refactored programs. Furthermore, we applied the refactorings to three real-world projects: BusyBox, OpenSSL, and SQLite. This way, we detected and fixed a few behavioral changes, 62% caused by unspecified behavior in the C programming language. Index Terms—Configurable Systems, Preprocessors, and Refactoring F 1 INTRODUCTION [5], [14], [26]. We call preprocessor usage that does not respect the syntactic structure of the source code The C preprocessor is a language-independent tool for undisciplined (e.g., wrapping a single bracket without its lightweight meta-programming that provides no percep- corresponding closing one). tible form of modularity [44]. The preprocessor is used in To resolve undisciplined preprocessor usage, in a pre- many projects written in C. Developers use preprocessor vious work, we proposed a catalog of refactorings (and directives, such as #ifdef and #endif, to mark blocks we developed an Eclipse plug-in to automate the refactor- of source code as optional or conditional, with the pur- ings) [32]. The catalog contains 14 refactorings to resolve pose of tailoring software systems to different hardware undisciplined directives divided into 4 categories: single platforms, operating systems, and application scenarios. statements, conditions, wrappers, and comma-separated Researchers and practitioners have been criticizing the elements. In this article, we report on a mixed-method C preprocessor because of its negative effect on code evaluation [8], [9] of our refactorings to answer the understanding and maintainability and its error prone- following research questions: ness [10], [5], [26], [31], [30], [33]. More importantly, the preprocessor hinders the development of tool support • RQ1: What is the number of possibilities to apply that is standard in other languages, such as automated the refactorings in practice? refactoring [49], [23], [15], [29], [21], [48]. Developers • RQ2: What opinion do developers have on our aggravate these problems when using the preprocessor catalog of refactorings in practice? in undisciplined ways, for example, using preprocessor • RQ3: Do the refactorings of the catalog preserve directives that split up a statement or expression [10], program behavior? To answer RQ1 (number of possibilities), we ana- • F. Medeiros is affiliated with the Federal Institute of Alagoas, AL, lyzed 63 open source C projects searching for opportuni- 57020-600, and with the Department of Computing and Systems, Federal ties to apply the refactorings in practice. We considered University of Campina Grande, PB, 58429-900, Brazil. R. Gheyi is C projects of different sizes and from various domains, affiliated with the Department of Computing and Systems, Federal University of Campina Grande, PB, 58429-900, Brazil. M. Ribeiro, B. such as games, operating systems, Web servers, and Ferreira, Luiz Carvalho, and B. Fonseca are affiliated with the Computing database systems. Overall, we found 5670 application Institute, Federal University of Alagoas, Maceió, AL, 57072-900, Brazil. possibilities, in 59 out of the 63 open-source projects C. Kästner is affiliated with the Institute for Software Research, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 15213, (one refactoring possibility for every 3704 lines of code), USA. S. Apel is affiliated with the Department of Informatics and showing that our refactorings are indeed relevant to real- Mathematics, University of Passau, 94032, Germany. world C projects. 2 We answer RQ2 (opinion of developers) with two This work builds upon our prior work on studying different evaluations. First, we analyzed data from an #ifdef directives and possible refactorings. Specifically, online survey among 246 developers. We asked them we interviewed 40 developers and performed a survey about their preferences showing pairs of behaviorally with 202 participants in our previous study to learn equivalent code snippets: (1) the original code from a about the opinion of developers regarding undisciplined real-world project with undisciplined preprocessor us- directives and its common problems, such as code un- age, and (2) a disciplined version of the original code derstanding, maintainability, and error proneness [30]. snippet, created by applying one of our refactorings. By studying 12 C real-world projects, we identified Second, we selected a subset of application possibilities patterns of undisciplined preprocessor directives and in the studied subject systems and submitted patches to we proposed a preliminary version of our catalogue of the respective project converting undisciplined directives refactorings to remove these patterns of undisciplined into disciplined ones to get feedback from developers. directives [32]. In this preliminary version, we evaluated The results of our survey reveal that 63% of developers the refactorings showing that they were syntactically prefer to use the refactored code (i.e., the disciplined corrected, that is, we did not introduce syntax errors version of the code) instead of using the preprocessor with our refactorings. This article significantly expands in undisciplined ways. Likewise, we received positive on prior work by implementing and evaluating the feedback from developers when submitting patches to refactorings with multiple studies. We complement our resolve undisciplined directives. previous studies by submitting patches to remove undis- To answer RQ3 (behavior preservation), we created a ciplined preprocessor directives and by performing a model of a subset of the C language and a corresponding totally new survey with 246 developers. Furthermore, code generator, based on Alloy [19], to automatically we performed studies to verify behavior preservation in generate programs with application possibilities for our our refactorings using automatically generated programs refactorings, along with corresponding test cases. In ad- and three real-world systems. A replication package for dition, we applied our refactorings in BusyBox, OpenSSL, our current study is available at the project’s Web site.1 and SQLite, three real-world projects with test cases available. For all subjects (Alloy-based and real-world), 2 UNDISCIPLINED PREPROCESSOR USAGE we ran the test cases before and after applying our refac- The lexical operation mode, which allows introducing torings to check behavior preservation. We found and undisciplined directives at arbitrary tokens, is one of fixed some behavioral changes in one refactoring of our the most criticized aspects of the C preprocessor [10], catalog, and a few behavioral changes in the refactoring [5], [14], [26], [30], [31], [33]. Undisciplined preprocessor implementations. Most behavioral changes (62%) are re- directives do not respect the syntactic structure of the lated to unspecified behavior in the C language, though. source code (e.g., wrapping a single bracket without its We modified one refactoring that introduced behavioral correspondent closing one) [26]. For instance, Figure1 (a) changes and updated another refactoring with feedback presents part of the source code of Vim including pre- from developers. We found no behavioral changes when processor directives that we consider as undisciplined. running the test cases before and after applying the In this case, the #ifdefs wrap only part of expressions. refactorings to the three real-world projects. In Figure1 (b), we present an alternative, disciplined In summary, the main contributions of this article are: version of the code, in which the preprocessor directives • We present a mixed-method evaluation of our cata- surround entire statements only. log and implementation of refactorings to remove In previous work, we gathered evidence that devel- undisciplined preprocessor directives. The results opers do not recommend using undisciplined direc- provide
Load more

Recommended publications
  • Thriving in a Crowded and Changing World: C++ 2006–2020
    Thriving in a Crowded and Changing World: C++ 2006–2020 BJARNE STROUSTRUP, Morgan Stanley and Columbia University, USA Shepherd: Yannis Smaragdakis, University of Athens, Greece By 2006, C++ had been in widespread industrial use for 20 years. It contained parts that had survived unchanged since introduced into C in the early 1970s as well as features that were novel in the early 2000s. From 2006 to 2020, the C++ developer community grew from about 3 million to about 4.5 million. It was a period where new programming models emerged, hardware architectures evolved, new application domains gained massive importance, and quite a few well-financed and professionally marketed languages fought for dominance. How did C++ ś an older language without serious commercial backing ś manage to thrive in the face of all that? This paper focuses on the major changes to the ISO C++ standard for the 2011, 2014, 2017, and 2020 revisions. The standard library is about 3/4 of the C++20 standard, but this paper’s primary focus is on language features and the programming techniques they support. The paper contains long lists of features documenting the growth of C++. Significant technical points are discussed and illustrated with short code fragments. In addition, it presents some failed proposals and the discussions that led to their failure. It offers a perspective on the bewildering flow of facts and features across the years. The emphasis is on the ideas, people, and processes that shaped the language. Themes include efforts to preserve the essence of C++ through evolutionary changes, to simplify itsuse,to improve support for generic programming, to better support compile-time programming, to extend support for concurrency and parallel programming, and to maintain stable support for decades’ old code.
    [Show full text]
  • Red Hat Enterprise Linux 8 Security Hardening
    Red Hat Enterprise Linux 8 Security hardening Securing Red Hat Enterprise Linux 8 Last Updated: 2021-09-06 Red Hat Enterprise Linux 8 Security hardening Securing Red Hat Enterprise Linux 8 Legal Notice Copyright © 2021 Red Hat, Inc. The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/ . In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. Linux ® is the registered trademark of Linus Torvalds in the United States and other countries. Java ® is a registered trademark of Oracle and/or its affiliates. XFS ® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries. MySQL ® is a registered trademark of MySQL AB in the United States, the European Union and other countries. Node.js ® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
    [Show full text]
  • Defining the Undefinedness of C
    Technical Report: Defining the Undefinedness of C Chucky Ellison Grigore Ros, u University of Illinois {celliso2,grosu}@illinois.edu Abstract naturally capture undefined behavior simply by exclusion, be- This paper investigates undefined behavior in C and offers cause of the complexity of undefined behavior, it takes active a few simple techniques for operationally specifying such work to avoid giving many undefined programs semantics. behavior formally. A semantics-based undefinedness checker In addition, capturing the undefined behavior is at least as for C is developed using these techniques, as well as a test important as capturing the defined behavior, as it represents suite of undefined programs. The tool is evaluated against a source of many subtle program bugs. While a semantics other popular analysis tools, using the new test suite in of defined programs can be used to prove their behavioral addition to a third-party test suite. The semantics-based tool correctness, any results are contingent upon programs actu- performs at least as well or better than the other tools tested. ally being defined—it takes a semantics capturing undefined behavior to decide whether this is the case. 1. Introduction C, together with C++, is the king of undefined behavior—C has over 200 explicitly undefined categories of behavior, and A programming language specification or semantics has dual more that are left implicitly undefined [11]. Many of these duty: to describe the behavior of correct programs and to behaviors can not be detected statically, and as we show later identify incorrect programs. The process of identifying incor- (Section 2.6), detecting them is actually undecidable even rect programs can also be seen as describing which programs dynamically.
    [Show full text]
  • Web Browser Access to Cryptographic Hardware
    Universidade do Minho Escola de Engenharia Leonel João Fernandes Braga Web Browser Access to Cryptographic Hardware Outubro de 2012 Universidade do Minho Escola de Engenharia Leonel João Fernandes Braga Web Browser Access to Cryptographic Harware Tese de Mestrado Mestrado em Engenharia Informática Trabalho realizado sob orientação de Doutor Vítor Francisco Fonte Supervisão na empresa de Engenheiro Renato Portela Outubro de 2012 Acknowledgments I could not conclude this work without acknowledge all the support, time, and understanding of all the people who have been around me during this phase and during my journey of life. I am sure that without them everything would be much more difficult, and the success would be harder to achieve. First of all, I want to thank to my supervisor Professor Victor Fonte for being so helpful and supportive. His guidance certainly improved my work and my knowledge as well. I want also to thank to Engenheiro Renato Portela from MULTICERT for enlightening me when I was more doubtful. A special thanks to MULTICERT for letting me enrol in this project: it made me grow professionally and enhanced my knowledge. I want also to thank the Firebreath community for clarifying all the doubts I had. Congrat- ulations for your great work as well. In this context, there is one person to whom I could not be more grateful: Pedro, thank you for your help and patience, even when I had lots of questions. I am also grateful for the discussions I had with Pedro and Ulisses: they gave me lots of ideas of how I could improve my work.
    [Show full text]
  • Automatic Detection of Unspecified Expression Evaluation in Freertos Programs
    IT 14 022 Examensarbete 30 hp Juni 2014 Automatic Detection of Unspecified Expression Evaluation in FreeRTOS Programs Shahrzad Khodayari Institutionen för informationsteknologi Department of Information Technology . ... ... .... . . .. . . Acknowledgements This is a master thesis submitted in Embedded Systems to Department of Information Technology, Uppsala University, Uppsala, Sweden. I would like to express my deepest gratitude to my suppervisor Philipp Rümmer, Programme Director for Master’s programme in Embedded System at Uppsala University, for his patience in supporting continuously and generously guiding me with this project. I would like to appriciate professor Bengt Jonsson for reviewing my master thesis and offering valuable suggestions and comments. I would like to thank professor Daniel Kroening for helping me and providing updates of CBMC. Sincere thanks to my husband and my incredible parents who gave me courage and support throughout the project. Contents 1 Introduction..........................................................................................................1 Contributions.................................................................................................................3 Structure of the thesis report..........................................................................................3 2 Background...........................................................................................................5 2.1 Verification..............................................................................................................5
    [Show full text]
  • Lenient Execution of C on a JVM How I Learned to Stop Worrying and Execute the Code
    Lenient Execution of C on a JVM How I Learned to Stop Worrying and Execute the Code Manuel Rigger Roland Schatz Johannes Kepler University Linz, Austria Oracle Labs, Austria [email protected] [email protected] Matthias Grimmer Hanspeter M¨ossenb¨ock Oracle Labs, Austria Johannes Kepler University Linz, Austria [email protected] [email protected] ABSTRACT that rely on undefined behavior risk introducing bugs that Most C programs do not strictly conform to the C standard, are hard to find, can result in security vulnerabilities, or and often show undefined behavior, e.g., on signed integer remain as time bombs in the code that explode after compiler overflow. When compiled by non-optimizing compilers, such updates [31, 44, 45]. programs often behave as the programmer intended. However, While bug-finding tools help programmers to find and optimizing compilers may exploit undefined semantics for eliminate undefined behavior in C programs, the majority more aggressive optimizations, thus possibly breaking the of C code will still contain at least some occurrences of non- code. Analysis tools can help to find and fix such issues. Alter- portable code. This includes unspecified and implementation- natively, one could define a C dialect in which clear semantics defined patterns, which do not render the whole program are defined for frequent program patterns whose behavior invalid, but can still cause surprising results. To address would otherwise be undefined. In this paper, we present such this, it has been advocated to come up with a more lenient a dialect, called Lenient C, that specifies semantics for behav- C dialect, that better suits the programmers' needs and ior that the standard left open for interpretation.
    [Show full text]
  • Bring Your Own Token to Replace Traditional Smart Cards
    White paper Cisco Public Bring Your Own Token to Replace Traditional Smart Cards This case study discusses implementing a Bring Your Own Token (BYOT) solution with support for self-provisioning and management options to enable users to provision digital identities used for strong authentication and signing. Abstract Smart cards are a good way to enable strong authentication to enterprise network and applications because they provide identification, authentication, and the ability to store cryptographic key information on a card using the embedded microchip and memory. Enterprises can provision smart cards with a digital identity, in the form of a X509 certificate uniquely associated with a user, to enable smart card login to servers and mutual TLS authentication to services. Traditionally, hybrid cards that provide both the proximity card and smart card functionalities are used for this purpose so that users can have a single card for facility access and strong authentication to IT servers or applications. There are some limitations and challenges with using a single card as both proximity and smart card. Proximity cards can generally preprovision in bulk because the association of the user identity to the proximity ID can be done after the card is assigned to a user. But for the smart card, the X509 certificates provisioned to the smart cards contain the user information that must be known at provisioning time. This slows down the provisioning process. There are also other challenges related to issuing temporary replacement cards for lost cards. This white paper describes the Cisco solution to replace traditional hybrid smart cards with a Bring Your Own Token (BYOT) model, to overcome the limitations and challenges with traditional smart card solutions.
    [Show full text]
  • C/C++ Coding Standard Recommendations for IEC 61508
    C/C++ Coding Standard Recommendations for IEC 61508 Version V1, Revision R2 Feb 23, 2011 CONFIDENTIAL INFORMATION © exida Consulting LLC exida c c++ coding standard - iec61508.doc, Feb 23, 2011 Mike Medoff - John Grebe Page 1 of 18 CONFIDENTIAL INFORMATION Table of Contents 1. C/C++ Source Code Standard ................................................................................... 3 1.1 Introduction ..................................................................................................................... 3 1.1.1 Purpose and Scope ..............................................................................................3 1.1.2 Standards used ....................................................................................................4 2. Background ................................................................................................................ 4 2.1 IEC 61508 Requirements ................................................................................................4 2.2 C/C++ Programming languages compared to IEC 61508 language selection requirements ........................................................................................................................................ 5 2.2.1 IEC 61508 language selection requirements ....................................................... 5 2.2.2 Gap Analysis ........................................................................................................6 2.3 Knowledge of typical Coding Errors Made by C/C++ Programmers ..............................
    [Show full text]
  • High Integrity C++ Version
    Coding Standard Version 4.0 www.codingstandard.com 3 October 2013 Programming Research Ltd Mark House - 9/11 Queens Road - Hersham - Surrey KT12 5LU - United Kingdom Tel: +44 (0) 1932 888 080 - Fax: +44 (0) 1932 888 081 [email protected] www.programmingresearch.com Registered in England No. 2844401 © 2013 Programming Research Ltd Contents Contents 0 Introduction 7 0.1 Typographical Conventions.........................................7 0.2 Escalation policy...............................................7 0.3 Base Standard and Policy..........................................8 0.3.1 ISO Standard C++..........................................8 0.3.2 Statically detectable restrictions...................................8 0.3.3 Examples...............................................8 0.4 Basis of requirements............................................8 0.5 Rule Enforcement..............................................8 0.6 Deviations...................................................9 0.7 Glossary....................................................9 1 General 10 1.1 Implementation compliance......................................... 10 1.1.1 Ensure that code complies with the 2011 ISO C++ Language Standard............ 10 1.2 Redundancy................................................. 10 1.2.1 Ensure that all statements are reachable............................. 10 1.2.2 Ensure that no expression or sub-expression is redundant.................... 11 1.3 Deprecated features............................................. 12 1.3.1 Do not
    [Show full text]
  • Security in Programming Languages Security in Programming Languages
    Security in programming languages Security in programming languages • Languages have long been related to security. • Modern languages should contribute to security: – Constructs for protection (e.g., objects). – Techniques for static analysis, in particular for ensuring safety by typechecking. – A tractable theory, with sophisticated methods. • Several security techniques rely on language ideas, with static and dynamic checks. See Morris’s “Protection in Programming Languages”. A class with a secret field class C { • A possible conjecture: Any two instances of this // the field class are observationally private int x; equivalent (that is, they // a constructor cannot be distinguished public C(int v) { x = v; } within the language). } • More realistic examples use constructs similarly. • Objects are unforgeable. // two instances of C E.g., integers cannot be C c1 = new C(17); cast into objects. C c2 = new C(28); Mediated access [example from A. Kennedy] class Widget {// No checking of argument virtual void Operation(string s) {…}; } class SecureWidget : Widget { // Validate argument and pass on // Could also authenticate the caller override void Operation(string s) { Validate(s); base.Operation(s); } } … SecureWidget sw = new SecureWidget(); sw.Operation(“Nice string”); // Can’t avoid validation of argument Some application areas Avoiding or detecting Using language constructs to implementation flaws embody policies • Various static analyses (e.g., • Data abstraction for for C programs). protecting private state. • High-level languages with • Objects for capabilities. automatic memory • Method calls enriched with management (e.g., Java). security semantics (as in • Typed assembly and stack inspection). intermediate languages; • Information-flow control proof-carrying code. (e.g., in Jif). Other aspects of language-based security • Secure coding practices.
    [Show full text]
  • The Java™ Language Specification Java SE 7 Edition
    The Java™ Language Specification Java SE 7 Edition James Gosling Bill Joy Guy Steele Gilad Bracha Alex Buckley Copyright © 1997, 2011, Oracle and/or its affiliates. All rights reserved. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. Duke logo™ designed by Joe Palrang. LIMITED LICENSE GRANTS 1. License for Evaluation Purposes. Oracle hereby grants you a fully-paid, non-exclusive, non-transferable, worldwide, limited license (without the right to sublicense), under Oracle's applicable intellectual property rights to view, download, use and reproduce the Specification only for the purpose of internal evaluation. This includes (i) developing applications intended to run on an implementation of the Specification, provided that such applications do not themselves implement any portion(s) of the Specification, and (ii) discussing the Specification with any third party; and (iii) excerpting brief portions of the Specification in oral or written communications which discuss the Specification provided that such excerpts do not in the aggregate constitute a significant portion of the Specification. 2. License for the Distribution of Compliant Implementations. Oracle also grants you a perpetual, non-exclusive, non-transferable, worldwide, fully paid-up, royalty free, limited license (without the right to sublicense) under any applicable copyrights or, subject to the provisions of subsection 4 below, patent rights it may have covering the Specification to create and/or
    [Show full text]
  • Using Cryptographic Hardware to Secure Applications
    Using Cryptographic Hardware to Secure Applications BRUCE MOMJIAN This presentation explains how to use cryptographic hardware in client applications. https://momjian.us/presentations Creative Commons Attribution License Last updated: July, 2021 1/99 Outline 1. Openssh configuration 2. OpenPGP configuration 3. OpenPGP usage 4. PIV vs OpenPGP 5. Postgres usage 6. Database encryption scope 7. Private key storage options 8. Conclusion 2/99 1. Openssh Configuration # host does not allow password authentication $ ssh [email protected] Permission denied (publickey). # can also use ssh-keygen -D opensc-pkcs11.so -e # use the PIV AUTH key’’ (1) $ pkcs15-tool --read-ssh-key 1 --output ssh.pub Using reader with a card: Yubico Yubikey 4 OTP+U2F+CCID 00 00 Please enter PIN [PIV Card Holder pin]: $ cat ssh.pub ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBrGGJqMxb… $ sudo sh -c ’cat ssh.pub >> ˜postgres/.ssh/authorized_keys’ $ rm ssh.pub $ ssh -I ‘‘$OPENSC’’ [email protected] Enter PIN for ’PIV_II (PIV Card Holder pin)’: Last login: Wed Aug 16 22:52:21 2017 from momjian.us $ id uid=109(postgres) gid=117(postgres) groups=117(postgres),111(ssl-cert) 3/99 Add PKCS#11 Provider for a Host $ cp ˜/.ssh/config ˜/.ssh/config.orig # OPENSC set previously $ echo " > Host momjian.us > PKCS11Provider $OPENSC" >> ˜/.ssh/config # -I not needed $ ssh [email protected] Enter PIN for ’PIV_II (PIV Card Holder pin)’: Last login: Fri Aug 18 15:23:09 2017 from momjian.us $ https://ef.gy/hardening-ssh 4/99 Use ssh-agent To Avoid Repeated PIN Entry # restore config file since
    [Show full text]