UCAM-CL-TR-845 Technical Report ISSN 1476-2986 Number 845 Computer Laboratory Black-box composition of mismatched software components Stephen Kell December 2013 15 JJ Thomson Avenue Cambridge CB3 0FD United Kingdom phone +44 1223 763500 http://www.cl.cam.ac.uk/ c 2013 Stephen Kell This technical report is based on a dissertation submitted December 2010 by the author for the degree of Doctor of Philosophy to the University of Cambridge, Christ’s College. Technical reports published by the University of Cambridge Computer Laboratory are freely available via the Internet: http://www.cl.cam.ac.uk/techreports/ ISSN 1476-2986 Black-box composition of mismatched software components Stephen Kell Summary Software is expensive to develop. Much of that expense can be blamed on difficulties in combining, integrating or re-using separate pieces of software, and in maintaining such compositions. Conventional development tools approach composition in an inherently narrow way. Specifically, they insist on modules that are plug-compatible, meaning that they must fit together down to a very fine level of detail, and that are homogeneous, meaning that they must be written according to the same conventions and (usually) in the same programming language. In summary, modules must have matched interfaces to compose. These inflexibilities, in turn, motivate more software creation and concomitant expense: they make programming approaches based on integration and re-use unduly expensive. This means that reimplementation from scratch is often chosen in preference to adaptation of existing implementations. This dissertation presents several contributions towards lessening this problem. It centres on the design of a new special-purpose programming language, called Cake. This language is specialised to the task of describing how components having mismatched in- terfaces (i.e. not plug-compatible, and perhaps not homogeneous) may be adapted so that they compose as required. It is a language significantly more effective at captur- ing relationships between mismatched interfaces than general-purpose programming lan- guages. Firstly, we outline the language’s design, which centres on reconciling interface differences in the form high-level correspondence rules which relate different interfaces. Secondly, since Cake is designed to be a practical tool which can be a convenient and easily-integrated tool under existing development practices, we describe an implementa- tion of Cake in detail and explain how it achieves this integration. Thirdly, we evaluate Cake on real tasks: by applying it to integration tasks which have already been performed under conventional approaches, we draw meaningful comparisons demonstrating a smaller (quantitative) size of required code and lesser (qualitative) complexity of the code that is required. Finally, Cake applies to a wide range of input components; we sketch extensions to Cake which render it capable of composing components that are heterogeneous with respect to a carefully identified set of stylistic concerns which we describe in detail. Acknowledgments Research is never finished. Rather than thanking people for anything they helped me complete, I’d like to thank a lot of people without whom I either couldn’t have started or couldn’t have continued. Firstly, I thank my supervisor, David Greaves, for the keen interest he showed in my research goals, balanced with a quiet faith in letting me find out where they led. I am grateful for his patient feedback when it became time to put this dissertation together. I thank Steven Hand for letting me through the door, finding funding, and always being available to offer honest and insightful feedback. Jon Crowcroft deserves special thanks, for his boundless positivity and ability to give sound feedback at a moment’s notice. Alan Mycroft also provided generous feedback at several decisive points. I also thank Sandy Fraser for his generosity and encouragement during a valuable summer in Princeton. Certain people deserve special mention. Michael Dales, Alastair Reid and Simon Ford gave me early encouragement which, unbeknownst to them, was essential in convincing me that I could and should pursue my research ambitions. Michael Hicks appeared at a critical time; his insight and positivity were vital in keeping me going. Somewhat later, Yvonne Coady rescued me from a trough out of which I might otherwise not have emerged. My many hugely talented colleagues and friends in the Atlas Room have indulged my eccentricities on a daily basis over the years. In particular, I thank Chris Purcell, Henry Robinson, Steven Smith, Theodore Hong, Mark Williamson, Derek Murray, Nishanth Sastry, Periklis Akritidis, Chris Smowton, Malte Schwarzkopf and Malcolm Scott. I also thank all the other members of the Networks and Operating Systems group for their willingness to listen to my increasingly off-topic ideas, and the Cambridge Programming Research Group, especially Dominic Orchard, Robin Message, Max Bolingbroke and Boris Feigin, for letting me pretend to belong to their group when it suited me. Finally, a PhD student needs post-docs: Scott Owens, Kathy Gray, David Evans, David Eyers, Mike Dodds and Alan Lawrence all gave vital advice and encouragement. Vaughan Wittorff, while not a post-doc, belongs in this list too. I thank the Computer Laboratory support staff for the department’s efficiency and generous resourcing; EPSRC and Cambridge Philosophical Society for essential funding; Christ’s College, ACM SIGPLAN, ACM SIGSOFT, EuroSys and the Royal Academy of Engineering for equally essential travel funding. I also thank various constituencies of Christ’s College for enhancing my life in various ways, with special mentions to the Graduate Society, the porters and the catering staff. Finally, I thank my family and friends for their support, for providing distractions, and generally for being more important than the contents of this dissertation. ‘If we succeed in making an Intergalactic Network, then our main problem will be learning to communicate with Aliens.’ J.C.R. Licklider Contents 1 Introduction 23 1.1 Diversityandchange .............................. 23 1.2 Existingpractice ................................ 25 1.2.1 Library-baseddevelopment. 25 1.2.2 Abstractionlayers............................ 27 1.2.3 Reimplementation . 27 1.2.4 Non-implementation .......................... 28 1.2.5 Standardisation ............................. 29 1.2.6 Information hiding and modular programming . 29 1.2.7 Porting ................................. 30 1.2.8 Automatedsourcecodetransformation . 31 1.2.9 Gluecoding ............................... 31 1.3 Examining tool support for glue coding . 32 1.3.1 Aptitudes of conventional languages . 32 1.3.2 Plug-compatibility assumptions . 34 1.3.3 Homogeneityassumptions . 35 1.3.4 The separation of functionality from integration . 35 1.4 Researchapproaches .............................. 36 1.4.1 Clean-slate versus adoptable solutions . 36 1.4.2 White-box versus black-box approaches . 36 1.5 Goals....................................... 39 1.5.1 Listofgoals............................... 39 1.5.2 Non-goals ................................ 39 1.6 Thesisstatement ................................ 40 1.6.1 Stateoftheart ............................. 41 CONTENTS CONTENTS 1.7 Approach .................................... 42 1.8 Contributions .................................. 43 1.9 Technicalbackground.............................. 44 1.9.1 Challenges of C and C++ programs .................. 44 1.9.2 Tools manipulating binaries . 45 1.10 Outlineofsubsequentchapters. .. 47 2 The Cake language 49 2.1 Motivation.................................... 49 2.2 ThedesignofCake ............................... 51 2.2.1 Designapproach ............................ 52 2.2.2 High-level view . 52 2.2.3 Introducingtherunningexample . 53 2.2.4 Insights ................................. 54 2.2.5 Requirements .............................. 55 2.2.6 Characterising Cake-generated code as a transducer . .... 56 2.2.7 Toolchaincontext............................ 56 2.2.8 Syntacticconventions. 58 2.2.9 Simplecorrespondences. 58 2.2.10 RemarksonsimpleCakeusage . 59 2.2.11 Correspondences for free: name matching . .. 61 2.3 MorepowerfulfeaturesofCake . 62 2.3.1 Corresponding sequences of events: event context . .... 62 2.3.2 Generating data-dependent call sequences: stubs . .... 64 2.3.3 Practicalities . 65 2.3.4 Many-to-manyvaluecorrespondences . 68 2.3.5 Quantificationandguards . 71 2.3.6 Relating individual values . 71 2.3.7 Inputandoutputparameters . 72 2.3.8 Arraysandlists............................. 73 2.3.9 Functionpointers ............................ 74 2.3.10 Completingtheexample . 75 2.4 Semanticquestions ............................... 76 2.4.1 Component-managedstate. 76 CONTENTS CONTENTS 2.4.2 Run-timeerrors............................. 76 2.4.3 Dynamic matching and binding . 78 2.4.4 Ruleprecedence............................. 80 2.5 Summary .................................... 81 3 Limitations of the Cake language 83 3.1 Event context predicate connectives . ... 83 3.2 Cakeandclassesofformallanguage. 84 3.3 Cross-cuttinglogic ............................... 85 3.4 Callingconventions ............................... 85 3.5 Controlstructures................................ 86 3.5.1 Proceduralbias ............................. 86 3.5.2 Threadedcontrolmodels . 86 3.5.3 Blocking versus nonblocking calls . 86 3.5.4 Conversationaladaptations . 87 3.5.5 Callsequencingconstraints. 87 3.5.6 Generalisedsequencemappings