Virtual Separation of Concerns: Toward Preprocessors

This article is protected by German copyright law. You may copy and distribute this article for your personal use only. Other use is only allowed with written permission by the copyright holder. , features software prod- 2]. For example, in the Bedingte Kompilierung ist ein explicitly support preproces- 1 BigLever ) in a single domain, generated from or variants Commercial product-line tools like those from Zusammenfassung , Philipps University Marburg 2 Oldenbourg Wissenschaftsverlag which are domain abstractionsalities and characterizing differences common- between variantstransactions, – in replication, our example, orA flash variant is support specified by are a feature features. selection, e. g., the data- uct lines. pure::systems sors. A software productsystems line ( is aa set common of managed related codedomain software base of [ embedded dataent management variants systems, differ- arescenario: needed with or depending withoutreplication, on with transactions, or the with without support application ordifferent for without flash power-saving drives, algorithms, with andof so forth. a Variants product line are distinguished in terms of evolved into a common way to implement  einfaches und häufigvon benutztes Variabilität Mittel ingrund Softwareproduktlinien, zur negativer welches Implementierung aber Auswirkungenbarkeit auf- auf stark Codequalität kritisiertstützung und wird. – Sichten, Wart- Wir Visualisierung, zeigenProduktlinien-Typsystem kontrollierte – Annotationen, wie die wesentlichen Werkzeugunter- Problemekann beheben und viele Vorteile einerWir modularen bieten Entwicklung emuliert. damitvon eine Belangen Alternative mittelsweise zur Modulen. in klassischen Statt Dateien Quelltext Trennung Trennung zu von notwendiger- Belangen“ separieren, durch entsprechende erzielenstüzung. Werkzeugunter- wir eine „virtuelle © Softwareproduktlinien, Variabilität, Präprozessoren, Bedingte #endif astner and ¨ and directives, n Schritt zu Präprozessoren 2.0 10.1524/itit.2012.0662 #ifdef #endif Schlagwörter DOI and , 8(6):59–78, 2009.  astner has been awarded by the ¨ #ifdef and similar lexical tools are and similar preprocessor direc- cpp #ifdef it 1/2012 Conditional compilation with preprocessors like D.2.9 [Software: Software Engineering: Management]; Software product lines, variability, preprocessors, conditional Journal of Object Technology (JOT) is a simple but effective means to implement variabil- Distinguished Dissertations The usage of – Information Technology 54 (2012) 1 / ity. By annotating code fragments with Summary cpp directives, different programfragments variants can with be created, orto which without can implement these be software usedsors (among are product others) frequently lines.for used Although, in their practice, preproces- negative theyity. effect We are show on often how code criticized disciplined simple tool annotations, quality support and and – variability-awarecan views, maintainabil- type visualizations, address systems these – modularized problems implementations. and Insteadcode emulate of into some files, separating benefits we pursue source of a “virtual separation of concerns”. The dissertation of Dr. Christian K This summary shares text with a previous overview: C. K it GI DisserationSaake, Award Otto-von-Guericke-University 2010.The The Magdeburg, University examiners of Prof. DonUniversity Texas were of at Batory, Waterloo. Austin, Prof. Dr. Gunter and Prof. Dr. Krzysztof Czarnecki, 2 these can later beferent excluded compiler from options, compilation. different Withor program dif- without variants these with fragments can be created. tives, as exemplified in1 the codeS. Apel. fragment Virtual below, separationcessors. of has concerns – A second chance for prepro- 1 Introduction The C preprocessor broadly used in practice tonotating code implement fragments variability. with By an- Keywords compilation, views, type systems Christian Kästner, Laureate of the GI Dissertation Award 2010 Kompilierung, Sichten, Typsysteme Virtuelle Trennung von Belangen: Ei Toward Preprocessors 2.0 Virtual Separation of Concerns: 42 This article is protected by German copyright law. You may copy and distribute this article for your personal use only. Other use is only allowed with written permission by the copyright holder. 43 n all in our case).  virtual separation NO_QUEUE Using preprocessors can eas- 6]. independent optional features), we might [ is not selected). We introduced this error n . Although we cannot claim to eliminate all views One of the keythat developers motivations can of find modularizingand all code features reason of is a about featureconcerns. it in Clearly, a one without scattered, spot being preprocessor-basedmentation imple- distracted does by not other reasoning, support but the this corethis kind question feature” “what can of still code lookupform be belongs answered of and to by tool support in the tation scatters featurewhere code it across is the entangledtures. entire closely Lack code with of the base separation codefor of of a concerns other is lot fea- a held of responsible feature problems: such Tofrom as the understand product transactions the line, or webase behavior instead need to of to of just search remove looking theno into a direct entire a traceability single code from feature module. a There featureits is as implementation. domain concept Tangled to codetracts the of programmer other in the features search.annotations Additionally, dis- can textual entirely obfuscateits the control source flow. codecontrary Scattering and to and decades tangling of software feature engineering code education. is Sensitivity to subtle errors. ily introduce errorsdifficult at to different detect. This levels alreadyand that begins with type can simple errors. syntax belike Developers very annotating are a proneone to closing as illustrated simple bracket in errors the butbracket code in not excerpt Line above the 5 (theNO_QUEUE is opening opening closed in Linedeliberately, 18 but only such when errorsand feature can are easily difficult to occur detect. inimplementations The practice scattered intensifies nature of this feature ers problem. cannot The detect compil- customer) such eventually errors, builds unlessatic the a feature developer combination variant (with (or withHowever, a since problem- there arevariants so for many potentialnot variants compile (2 variants withnation a during problematic initial feature development. combi- Simply compiling 3 VirtualInstead Separation of of suggesting Concerns abandoningof preprocessors in more favor ‘modular’investigate mechanisms, how as we most candeveloped improve critics tool preprocessors. do, support We we that have of achieves a concern disadvantages, we will point outand some unique new advantages opportunities that preprocessors offer. 3.1 Separation of Concerns variants is also not feasible, dueeven to simple their syntax huge number, and so, for type a errors long might time. goonly The undetected late bottom in line is theexpensive that development to errors cycle, fix. are when found they are more . 2], fea- and similar #ifdef rfp; { * and is prone to introduce 9], and others. Instead of separating all code virtual separation of concerns itivity to subtle errors. 11]. Numerous studies discuss the 11]. The use of 5; 4; _ _rep_queue_filedone(dbenv, dbenv; 10], aspects [ * (_ _db_no_queue_am(dbenv)); rep; DIAGNOSTIC NO_QUEUE * empty, ret, t_ret; Berkeley DB rep, rfp) separation of concerns int return db_pgno_t first,u_int32_t last; flags; DB_MSGBUF mb; // over 100 further lines of C code COMPQUIET(rep, NULL); COMPQUIET(rfp, NULL); DB_ENV REP _ _rep_fileinfo_args Virtual Separation of Concerns: Toward Preprocessors 2.0 static int #endif #endif #else #ifdef #ifdef } //Adapted code excerpt of Oracle’s We do not intend to give a definitive answer on how to Here, we take sides with preprocessors. We show how By this point, many readers may already object to pre- 6 7 8 9 2 3 4 5 1 19 15 16 17 18 10 11 12 13 14 implement a product line (actually,selves and we explore are different not paths in sureto parallel), our- but bring we preprocessors want backresearch into the toward race novel and preprocessor-based encourage approaches. Separation of concerns. that implements afile, feature class, package, into etc.), a a preprocessor-based implemen- separate module (or There are many argumentsbut against the lexical two preprocessors, most commonconcerns are and their their lack sens of separation of 2 Criticism directives breaks withcept the of fundamentally accepted con- subtle errors. Manyentirely academics abandoning recommend the use limiting ofimplement preprocessors or product and lines instead withtechniques ‘modern’ implementation that encapsulatemodules, features such as in componentsture some and modules frameworks [ form [ of simple extensions of conceptspitfalls and tools of can avoid preprocessorunique many usage advantages and overtechniques we in contemporary the highlight modularization contextSince some we of aim product-line for separation development. feature-related of concerns code without into dividing we physically name separated this approach modules, negative effect of preprocessor usagemaintainability on code [ quality and processor usage – andcriticized in in literature fact, as preprocessors summarizedConsidered are Harmful” in [ heavily the claim “#ifdef management system with transactionssupport. but without flash This article is protected by German copyright law. You may copy and distribute this article for your personal use only. Other use is only allowed with written permission by the copyright holder. 8]. can help ; return can check that all vari- nically, disciplined annota- e preprocessor’s applicabil- each variant in isolation [ e syntax errors in any vari- module can be checked for are an approach to limit the ex- disciplined annotations , Transaction txn ) { || txn==null ) annotation is disciplined as it aligns annotation is undisciplined. By enforcing 6] that manage annotations at tool level and push(Object o (o==null Stack { Lock l=txn.lock(o); l.unlock(); if annotations can be detected by adding tool sup- elementData[size++] = o; fireStackChanged(); void } DIAGNOSTIC Disciplined annotations In addition, disciplined annotations enable a clear Variability-aware type systems class } 1 2 3 4 5 6 7 8 9 distinguish a certainare number many of interesting colors), possibilities to but explore. still, there 3.2 Sensitivity to Subtle Errors Also various kinds#ifdef of errors thatport. can We easily illustrate occur how regarding with syntax errorstype and systems can how help new regardingfocus on type variability-aware semantic errors. errors, (We suchare did as not not deadlocks, a because specific they equally problem in of modular annotations implementations.) but can occur pressive power of annotations inerrors, order without to restricting prevent syntax th ity to practical problems.preprocessor Syntax errors usage arise that from considerstext, lexical a in source which file everyets) can as token be plain (including annotated. A individualconsider safer the brack- way underlying to structure annotateprogrammers code of to is the annotate to code (and andsyntactical thus allow program remove) elements, only such entire and as statements. classes, We functions, say analigns with annotation the is underlying disciplined, structure. In if ourthe it code example, with a local declaration,NO_QUEUE whereas the elsedisciplined branch annotations, of we the generation can will guarantee not introduc thatant. variant mapping from features to codeical structures tokens), (instead which of is lex- beneficialviews for and many tools, type including systems.tions Tech require moreunderstanding elaborate of tools, which thecheck whether have underlying annotations a with artifacts.sor a basic are traditional in Such a preproces- disciplined tools formapproaches, (this is in equivalent to modular whichsyntax each errors in isolation).like CIDE Alternatively, [ there areensure that tools only structural elementsthe can first be place. annotated in ants in thecompiled), without checking product lineThis detects are typical well-typed problems, suchthat (i. as e., are functions removed or can types in be some variants but still referenced, . 12]orthe 3]: Relevant NO_QUEUE ){ [] directives. Back- #ifdef directives, because we could end up 1]. [ % #ifdef (_ _db_no_queue_am(dbenv)); NO_QUEUE Distinguished Dissertations return [] COMPQUIET(rep, NULL); COMPQUIET(rfp, NULL); With simple tool support for providing views, we In addition to views on individual features, (editable) Beyond views, we also explored different visual repre- With relatively simple tool support, it is possible to #else #endif #ifdef } static int _ _rep_queue_filedone( 5 6 7 8 9 1 2 3 4 ground colors andhelpful similar for visual long and support nestederwise are annotations, be especially which hard may to oth- problems track. of We using are colors aware (e. g., of humans some are potential only able to can even emulate challengingproaches, problems such of as modularimplementation ap- the of expression problemsource feature code [ can interactions simply appear [ in multiple views. views on variants arethe possible. source code That that is,ture would a be selection tool generated and can for hidefeatures. a show all given remaining This fea- code goes ofproaches, beyond unselected with which the the powerthe developers behavior have of of to multiple modular reconstruct components/plug-ins/aspectstheir ap- in minds. Especially, when manyinteract, fine-grained from features ourdous experience, views help. can Althoughseparate be compilation some a or desirable tremen- be modular features achieved type with checking suchal. views, cannot as have in measured a anwith similar increase views context, by in 40 developer Atkins productivity et sentations of annotations thatless. obfuscate Among the others source we code exploredwith representing background annotations colors at tooltual level, directives instead at of source-code using level. tex- Inwe the annotate example a below, synchronizationa feature background in color. Java With code theseat with fine-grained substatement annotations level, thehighly source obfuscated with code traditional would have been create an (editable) viewall on irrelevant the code sourceirrelevant of files code from other by a file hiding features.code browser from That editor and windows we (technically, is, hide thismented irrelevant can we like be imple- filter code foldingexcerpt in below, modern we IDEs). show In a the view code on feature with completely unrelated statements.to Instead, provide we some context need function (italic is and this gray; statementcode e. g., located) (‘[]’). in and Interestingly, which indicate similaralso present hidden context in modularized information implementations is of in the extension form points and interfaces. Note that we cannot simply removeannotated everything by that is not 44 This article is protected by German copyright law. You may copy and distribute this article for your personal use only. Other use is only allowed with written permission by the copyright holder. 45  ) and to consider cpp , because, even though 7]. , available as open source directives and build system CIDE . We evaluated the concepts and #ifdef 12]. In exactly those cases, views on scattered virtual separation of concerns Finally, annotating code does not prohibit traditional We do not have a definitive answer whether physi- To evaluate visualizing annotations with background Currently, we explore scaling variability-aware type http://fosd.net/CIDE language (e. g., AspectJ forUML for Java, UML), AspectC preprocessors (and forcan most C, be improvements) Aspect- applied uniformly across languages. means of separationable of to concerns. still decompose Inpreprocessors the fact, system only into it where modules isadd necessary. and use Preprocessors reason- additional only expressiveness,ularization where techniques traditional comecrosscutting mod- concerns to or multi-dimensional separation their of concerns limits [ regarding code can be helpful forautomated understanding. refactorings We from even annotations explored to modularplementations im- (and vice versa) [ 4 Conclusion We have argued thatfor preprocessors are product-line not development. beyondwe hope With can little address tool manyare support, problems often for which criticized. preprocessors modularity Views and on separation the ofannotations source concerns, code and and emulate disciplined variability-awareimplementation type errors. systems Together,forts detect we namefeatures are not these physically separated into ef- ration modules, is a emulated sepa- by tools.problems While of we do preprocessors not (for eliminate example,lation all separate is compi- still notdistinct possible), advantages preprocessors like also ease havemity. of some use and language unifor- cal or virtual separationdepends of very concerns much isinvestigating better on different (and what approaches this a in one look at parallel measures). possible andwe We integration want have to scenarios. are encourage With researchers ourudices to (usually overcome work, from their experience prej- with annotation-based implementations. At the samewant time, to we encourage current practitioners thatusing are preprocessors currently to look forsupport improvements. Since is tool necessaryanyway, it for is well product-line worth implementation investing also into tool support 3.4 Tools and Evaluation We implemented all presented improvements intotype our product-line pro- tool at their combinations withcluding 13 non-trivial a case product-line studies,Berkeley version in- DB. of Weaware the proved type correctness database system of formally system for the a variability- subset ofcolors, Java. we conducted controlled experiments. checking to the Linuxall implemented kernel with with overvariability. 10 000 features, very when annotating language indepen- in all variants. If both, ref- : Developers annotate and op- uniform experience WRITE storeData(...) { ... } main() { [] readData() { ... } Virtual Separation of Concerns: Toward Preprocessors 2.0 and provide a readData(); ... storeData(...); First, and most important, preprocessors have a Second, preprocessors are usually Whereas a conventional type system checks whether it A variability-aware type system can efficiently check int int int #ifdef #endif } 6 7 8 9 1 2 3 4 5 different artifact types. WeJava applied code, our but toolscode, explored grammars, mostly also documentation to and C otherInstead code, artifact types. of C# providing code, Haskell a tool or language extension per 3.3 Unique Advantages of Preprocessors Despite all their problems, preprocessors also haveadvantages unique over modular approaches. simple programming model dent tionally remove code. Preprocessorsand are understand. very In easy contrast tonew to use languages, tools, modular or approaches, processesconjecture no have that to be this learned. simplicity We drivesuse practitioners preprocessors to despite still all disadvantages. can resolve all functionsystem calls, knows our about variability-awarefunction variability type calls and can be checks resolved whether all erence and target are annotated withreference the same can feature, be the resolvedhave in to every check variant; the relationship otherwise,(the between call’s we annotation both must annotations implya the check target’s that annotation, we cansolvers). encode If and there solve is efficiently any withthe variant SAT reference in which is the removed targettype (as but not system in our issues example anline above), is error. checked the That in a is,of single the all step by invocations entire comparing and product annotations checking their every respective variant in targets, isolation. instead of entire product lines, usuallyhead, with avoiding almost constant thevariants over- exponential in explosion isolation.lates of Type some form checking checking ofthem. annotations modules So emu- and instead dependencies ofports between specifying another, we that check onecode these component using dependencies im- in relationships scattered line. between With features disciplined in annotationstype a systems, and we product avoid variability-aware or detectmake the preprocessors typical so problems that error prone. like in the codeless fragment common below in modular (such approaches problems sinceinterfaces often that common and are separate compilation are used). This article is protected by German copyright law. You may copy and distribute this article for your personal use only. Other use is only allowed with written permission by the copyright holder. , Proc. of Proc. USENIX Conf. is a PostDoc at Prof. Klaus , pages 107–119, Los Alamitos, CA, Dr. Christian Kästner, Laureatesertation of Award 2010 the GI Dis- Ostermann’s Group for Programmingand Languages Software Engineering at thesity Philipps Univer- Marburg, Germany. He receivedComputer his Science Ph. for D. his in worktion on of virtual concerns in separa- May 2010of from the Magdeburg, University Germany, whereber he of was a Prof.Group mem- Gunter since Saake’s 2007. Databaseceived For Research the his GI dissertation,focuses Dissertation on Award. he correctness His re- and research tems understanding of with sys- variability,mentation including work mechanisms, on tools, imple-analysis, variability-aware type systems,refactoring. feature He interactions, is and thefifty author peer-reviewed or scientific coauthor publications. of over Address: PhilippsMeerwein-Straße, University D-35032 Marburg, Germany Marburg, Hans- Proc. of Europ. Conf. Object-Oriented Programming , LNCS 1241, pages 419–443, 1997. Springer-Verlag. objects. In: (ECOOP) bility experiencepages 185–198, with Berkeley, CA, 1992. C USENIX Association. news.separation: Multi-dimensional In: separation ofInt’l concerns. Conf. In: Softw. Eng.1999. (ICSE) IEEE Computer Society. [10] C. Prehofer. Feature-oriented programming: A fresh[11] H. look Spencer and at G. Collyer. #ifdef considered harmful or[12] porta- P. Tarr, H. Ossher, W. Harrison, and S. M. Sutton Jr. N degrees of Received: October 1, 2011, accepted: October 9, 2011 , Proc. Platform-independent Recognition of Procedures , LNCS , pages 311– , pages 157– , 28(12):1146– Proc. of Int’l Conf. Proc. of Int’l Work- Reverse Code Engineering – State of the Art and Experiments with P2P botnet detection Anomaly detection at supersonic speed Software Architecture in ACM Trans. Softw. Eng. IEEE Trans. Softw. Eng. (TSE) rhards-Padilla, E.: , page 29, Los Alamitos, CA, 1997. separated features. In: IEEE Trans. Softw. Eng. (TSE) , 2011. accepted for publication. , 41(1):115–141, 2003. Proc. of Int’l Conf. Softw. Eng. (ICSE) . Addison-Wesley, Boston, MA, 1998. Eschweiler, S. and Ge Laskov, P. and Grozea, C.: Willems,C.andFreiling,F.: Countermeasures Riviere, L. and Dietrich, S.: in Binaries based on simple Characteristics Distinguished Dissertations control data tostudy evaluate of the the impact Version of Editor. software In: tools: A case 1241, pages 220–242, 1997. Springer-Verlag. 28(7):625–637, 2002. Practice ture interaction: AComputer Networks critical review and considered forecast. In: 1170, 2002. shop on Program Comprehension of Europ. Conf. Object-Oriented Programming (ECOOP) preprocessor use. In: IEEE Computer Society. product lines. In: Methodol. (TOSEM) M. Loingtier, and J. Irwin. Aspect-oriented programming. In: 320, New York, 2008. ACM Press. physically and virtually Generative Programming and Component Eng. (GPCE) 166, New York, 2009. ACM Press. annotation-based product lines. In: • • • • Preview on issue 2/2012 The topic of our next issuefollowing will articles: be “Reactive Security” (Editor: U. Flegel) and it will contain the [1] D. L. Atkins, T. Ball, T. L. Graves, and A. Mockus. Using version [2] L. Bass, P. Clements, and[3] R. Kazman. M. Calder, M. Kolberg, E. H. Magill, and S. Reiff-Marganiec. Fea- [5] J.-M. Favre. Understanding-in-the-large. In: [4] M. Ernst, G. Badros, and D. Notkin. An empirical analysis of C [6] C. Kästner, S. Apel, and M. Kuhlemann. Granularity in software [9] G. Kiczales, J. Lamping, A. Menhdhekar, C. Maeda, C. Lopes, J.- [7] C. Kästner, S. Apel, and M. Kuhlemann. A model of refactoring [8] C. Kästner, S. Apel, T. Thüm, and G. Saake. Type checking for new preprocessors and virtualGive separation of preprocessors concerns. a second chance! References 46