Application Binary Interface Compatability Through A

Total Page:16

File Type:pdf, Size:1020Kb

Application Binary Interface Compatability Through A View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by The University of Utah: J. Willard Marriott Digital Library APPLICATION BINARY INTERFACE COMPATIBILITY THROUGH A CUSTOMIZABLE LANGUAGE by Kevin Jay Atkinson A dissertation submitted to the faculty of The University of Utah in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science School of Computing The University of Utah December 2011 Copyright c Kevin Jay Atkinson 2011 All Rights Reserved The University of Utah Graduate School STATEMENT OF DISSERTATION APPROVAL The dissertation of Kevin Jay Atkinson has been approved by the following supervisory committee members: Matthew Flatt , Chair 11/3/2011 Date Approved Gary Lindstrom , Member 11/17/2011 Date Approved Eric Eide , Member 11/3/2011 Date Approved Robert Kessler , Member 11/3/2011 Date Approved Olin Shivers , Member 11/29/2011 Date Approved and by Al Davis , Chair of the Department of School of Computing and by Charles A. Wight, Dean of The Graduate School. ABSTRACT ZL is a C++-compatible language in which high-level constructs, such as classes, are defined using macros over a C-like core language. This approach is similar in spirit to Scheme and makes many parts of the language easily customizable. For example, since the class construct can be defined using macros, a programmer can have complete control over the memory layout of objects. Using this capability, a programmer can mitigate certain problems in software evolution such as fragile ABIs (Application Binary Interfaces) due to software changes and incompatible ABIs due to compiler changes. ZL’s parser and macro expander is similar to that of Scheme. Unlike Scheme, however, ZL must deal with C’s richer syntax. Specifically, support for context-sensitive parsing and multiple syntactic categories (expressions, statements, types, etc.) leads to novel strategies for parsing and macro expansion. In this dissertation we describe ZL’s approach to parsing and macros. We demonstrate how to use ZL to avoid problems with ABI instability through techniques such as fixing the size of class instances and controlling the layout of virtual method dispatch tables. We also demonstrate how to avoid problems with ABI incompatibility by implementing another compiler’s ABI. Future work includes a more complete implementation of C++ and elevating the ap- proach so that it is driven by a declarative ABI specification language. CONTENTS ABSTRACT :::::::::::::::::::::::::::::::::::::::::::::::::::: iii LIST OF FIGURES :::::::::::::::::::::::::::::::::::::::::::::: viii LIST OF TABLES ::::::::::::::::::::::::::::::::::::::::::::::: x ACKNOWLEDGEMENTS :::::::::::::::::::::::::::::::::::::::: xi CHAPTERS 1. INTRODUCTION :::::::::::::::::::::::::::::::::::::::::::: 1 1.1 Dissertation Statement . .3 1.2 Approach . .4 1.3 Contributions . .4 2. PROBLEMS WITH THE C++ ABI ::::::::::::::::::::::::::::::: 6 2.1 The C++ ABI . .6 2.2 The Problem of Fragile ABIs . .9 2.2.1 Solutions Within C++ . .9 2.2.2 Defining a Better ABI . 10 2.3 The Problem of Compiler Specific ABIs . 11 3. SOLVING ABI PROBLEMS :::::::::::::::::::::::::::::::::::: 12 3.1 Overview . 12 3.1.1 User Roles . 13 3.2 Adding Private Data Members . 14 3.2.1 Reserving Space Ahead of Time . 14 3.2.2 Storing the Private Data in a Separate Object . 16 3.2.3 Avoiding Direct Allocation . 17 3.2.4 Why Not a Fixed Set of Language Extensions? . 18 3.3 Adding New Virtual Methods . 19 3.4 Reordering . 20 3.5 Removing Members . 21 3.6 Migrating Method Upwards . 22 3.7 Adding Parameters . 22 3.8 Other Difficult Transformations . 22 3.9 A Better ABI . 23 3.10 Changing Compilers . 23 4. ZL OVERVIEW ::::::::::::::::::::::::::::::::::::::::::::: 25 4.1 ZL Primitives . 25 4.2 Macros . 26 4.3 Parsing and Expanding . 27 4.4 Procedural Macros . 28 4.5 The Class Macro . 32 5. USING ZL TO MITIGATE ABI PROBLEMS :::::::::::::::::::::: 33 5.1 Adding Data Members without Changing Class Size . 33 5.1.1 Fixing the Size of a Class . 34 5.1.2 Allowing Expansion . 36 5.1.3 Validation . 38 5.2 Fixing the Size of the Virtual Table . 38 5.3 A Better ABI . 38 5.4 Matching an Existing ABI . 40 5.5 Matching GCC’s ABI . 40 5.6 Matching Another ABI . 41 5.7 Other ABI Problems . 41 6. THE CASE OF A SIMPLE SPELL CHECKER ::::::::::::::::::::: 42 6.1 Simple Spell . 42 6.2 The Spell Checker API . 42 6.2.1 The Application API . 43 6.2.2 The Extension API . 45 6.3 A Simple Application and Binary Compatibility . 47 6.4 Adding a Filter, Compiled with GCC . 47 6.4.1 The Bridge Class . 47 6.4.2 Adding The Email Filter . 49 6.4.3 Automating the Creation of the Bridge Class . 50 6.5 Adding Support for a Personal Dictionary . 50 6.6 A Better ABI to Allow Future Enhancements . 54 6.7 A Simple Spell Checker, Version 2 . 59 6.8 An Opportunity for an Even Better ABI . 60 6.9 Comparison to a Real Spell Checker: Aspell . 60 7. USING ZL ::::::::::::::::::::::::::::::::::::::::::::::::: 62 7.1 Classes and User Types . 62 7.2 Pattern-Based Macros and Lexical Extensions . 64 7.2.1 Extending the Parser . 65 7.2.2 The Parser . 66 7.2.3 Built-in Macros . 66 7.3 Macro API . 68 7.3.1 The Syntax Object . 68 7.3.2 The Syntax List . 70 7.3.3 Matching and Replacing . 71 7.3.4 Match Patterns . 72 v 7.3.5 Creating Marks . 73 7.3.6 Controlling Visibility . 73 7.3.7 Fluid Binding . 74 7.3.8 Partly Expanding Syntax . 75 7.3.9 Compile-Time Reflection . 76 7.3.10 Misc API Functions . 77 7.4 Procedural Macro Implementation and State Management . 78 7.4.1 The Details . 78 7.4.2 Macro Libraries . 78 7.4.3 State Management . 80 7.4.4 Symbol Properties . 80 7.5 ABI Related APIs . 81 7.5.1 User Type and Module API . 81 7.5.2 User Type Builder . ..
Recommended publications
  • GNU MP the GNU Multiple Precision Arithmetic Library Edition 6.2.1 14 November 2020
    GNU MP The GNU Multiple Precision Arithmetic Library Edition 6.2.1 14 November 2020 by Torbj¨ornGranlund and the GMP development team This manual describes how to install and use the GNU multiple precision arithmetic library, version 6.2.1. Copyright 1991, 1993-2016, 2018-2020 Free Software Foundation, Inc. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with the Front-Cover Texts being \A GNU Manual", and with the Back-Cover Texts being \You have freedom to copy and modify this GNU Manual, like GNU software". A copy of the license is included in Appendix C [GNU Free Documentation License], page 132. i Table of Contents GNU MP Copying Conditions :::::::::::::::::::::::::::::::::::: 1 1 Introduction to GNU MP ::::::::::::::::::::::::::::::::::::: 2 1.1 How to use this Manual :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 2 2 Installing GMP ::::::::::::::::::::::::::::::::::::::::::::::::: 3 2.1 Build Options:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 3 2.2 ABI and ISA :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 8 2.3 Notes for Package Builds:::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 11 2.4 Notes for Particular Systems :::::::::::::::::::::::::::::::::::::::::::::::::::::: 12 2.5 Known Build Problems ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 14 2.6 Performance
    [Show full text]
  • Majnemer-Fuzzingclang.Pdf
    Fuzzing Clang to find ABI Bugs David Majnemer What’s in an ABI? • The size, alignment, etc. of types • Layout of records, RTTI, virtual tables, etc. • The decoration of types, functions, etc. • To generalize: anything that you need N > 1 compilers to agree upon C++: A complicated language union U { int a; int b; }; ! int U::*x = &U::a; int U::*y = &U::b; ! Does ‘x’ equal ‘y’ ? We’ve got a standard How hard could it be? “[T]wo pointers to members compare equal if they would refer to the same member of the same most derived object or the same subobject if indirection with a hypothetical object of the associated class type were performed, otherwise they compare unequal.” No ABI correctly implements this. Why does any of this matter? • Data passed across ABI boundaries may be interpreted by another compiler • Unpredictable things may happen if two compilers disagree about how to interpret this data • Subtle bugs can be some of the worst bugs Finding bugs isn’t easy • ABI implementation techniques may collide with each other in unpredictable ways • One compiler permutes field order in structs if the alignment is 16 AND it has an empty virtual base AND it has at least one bitfield member AND … • Some ABIs are not documented • Even if they are, you can’t always trust the documentation What happens if we aren’t proactive • Let users find our bugs for us • This can be demoralizing for users, eroding their trust • Altruistic; we must hope that the user will file the bug • At best, the user’s time has been spent on something they probably didn’t want to do Let computers find the bugs 1.
    [Show full text]
  • Comparative Studies of 10 Programming Languages Within 10 Diverse Criteria
    Department of Computer Science and Software Engineering Comparative Studies of 10 Programming Languages within 10 Diverse Criteria Jiang Li Sleiman Rabah Concordia University Concordia University Montreal, Quebec, Concordia Montreal, Quebec, Concordia [email protected] [email protected] Mingzhi Liu Yuanwei Lai Concordia University Concordia University Montreal, Quebec, Concordia Montreal, Quebec, Concordia [email protected] [email protected] COMP 6411 - A Comparative studies of programming languages 1/139 Sleiman Rabah, Jiang Li, Mingzhi Liu, Yuanwei Lai This page was intentionally left blank COMP 6411 - A Comparative studies of programming languages 2/139 Sleiman Rabah, Jiang Li, Mingzhi Liu, Yuanwei Lai Abstract There are many programming languages in the world today.Each language has their advantage and disavantage. In this paper, we will discuss ten programming languages: C++, C#, Java, Groovy, JavaScript, PHP, Schalar, Scheme, Haskell and AspectJ. We summarize and compare these ten languages on ten different criterion. For example, Default more secure programming practices, Web applications development, OO-based abstraction and etc. At the end, we will give our conclusion that which languages are suitable and which are not for using in some cases. We will also provide evidence and our analysis on why some language are better than other or have advantages over the other on some criterion. 1 Introduction Since there are hundreds of programming languages existing nowadays, it is impossible and inefficient
    [Show full text]
  • ILE C/C++ Language Reference, SC09-7852
    IBM IBM i Websphere Development Studio ILE C/C++ Language Reference 7.1 SC09-7852-02 IBM IBM i Websphere Development Studio ILE C/C++ Language Reference 7.1 SC09-7852-02 Note! Before using this information and the product it supports, be sure to read the general information under “Notices” on page 355. This edition applies to IBM i 7.1, (program 5770-WDS), ILE C/C++ compilers, and to all subsequent releases and modifications until otherwise indicated in new editions. This version does not run on all reduced instruction set computer (RISC) models nor does it run on CISC models. © Copyright IBM Corporation 1998, 2010. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents About ILE C/C++ Language Reference Digraph characters ........... 27 (SC09-7852-01) ........... ix Trigraph sequences ........... 28 Who should read this book ......... ix Comments............... 28 Highlighting Conventions .......... x How to Read the Syntax Diagrams ....... x Chapter 3. Data objects and Prerequisite and related information ...... xii declarations ............ 31 How to send your comments ........ xii Overview of data objects and declarations .... 31 Overview of data objects ......... 31 What's new for IBM i 7.1 ....... xv Incomplete types .......... 32 Compatible and composite types ..... 32 Chapter 1. Scope and linkage ..... 1 Overview of data declarations and definitions .. 33 Tentative definitions ......... 34 Scope ................. 1 Storage class specifiers........... 35 Block/local scope ............ 2 The auto storage class specifier ....... 35 Function scope ............ 2 Storage duration of automatic variables ... 35 Function prototype scope ......... 3 Linkage of automatic variables ...... 36 File/global scope ...........
    [Show full text]
  • Application Binary Interface for the ARM Architecture
    ABI for the ARM Architecture (Base Standard) Application Binary Interface for the ARM® Architecture The Base Standard Document number: ARM IHI 0036B, current through ABI release 2.10 Date of Issue: 10th October 2008, reissued 24th November 2015 Abstract This document describes the structure of the Application Binary Interface (ABI) for the ARM architecture, and links to the documents that define the base standard for the ABI for the ARM Architecture. The base standard governs inter-operation between independently generated binary files and sets standards common to ARM- based execution environments. Keywords ABI for the ARM architecture, ABI base standard, embedded ABI How to find the latest release of this specification or report a defect in it Please check the ARM Information Center (http://infocenter.arm.com/) for a later release if your copy is more than one year old (navigate to the ARM Software development tools section, ABI for the ARM Architecture subsection). Please report defects in this specification to arm dot eabi at arm dot com. Licence THE TERMS OF YOUR ROYALTY FREE LIMITED LICENCE TO USE THIS ABI SPECIFICATION ARE GIVEN IN SECTION 1.4, Your licence to use this specification (ARM contract reference LEC-ELA-00081 V2.0). PLEASE READ THEM CAREFULLY. BY DOWNLOADING OR OTHERWISE USING THIS SPECIFICATION, YOU AGREE TO BE BOUND BY ALL OF ITS TERMS. IF YOU DO NOT AGREE TO THIS, DO NOT DOWNLOAD OR USE THIS SPECIFICATION. THIS ABI SPECIFICATION IS PROVIDED “AS IS” WITH NO WARRANTIES (SEE SECTION 1.4 FOR DETAILS). Proprietary notice ARM, Thumb, RealView, ARM7TDMI and ARM9TDMI are registered trademarks of ARM Limited.
    [Show full text]
  • GNU/Linux AI & Alife HOWTO
    GNU/Linux AI & Alife HOWTO GNU/Linux AI & Alife HOWTO Table of Contents GNU/Linux AI & Alife HOWTO......................................................................................................................1 by John Eikenberry..................................................................................................................................1 1. Introduction..........................................................................................................................................1 2. Traditional Artificial Intelligence........................................................................................................1 3. Connectionism.....................................................................................................................................1 4. Evolutionary Computing......................................................................................................................1 5. Alife & Complex Systems...................................................................................................................1 6. Agents & Robotics...............................................................................................................................1 7. Programming languages.......................................................................................................................2 8. Missing & Dead...................................................................................................................................2 1. Introduction.........................................................................................................................................2
    [Show full text]
  • Linkers and Loaders Do?
    Linkers & Loaders by John R. Levine Table of Contents 1 Table of Contents Chapter 0: Front Matter ........................................................ 1 Dedication .............................................................................................. 1 Introduction ............................................................................................ 1 Who is this book for? ......................................................................... 2 Chapter summaries ............................................................................. 3 The project ......................................................................................... 4 Acknowledgements ............................................................................ 5 Contact us ........................................................................................... 6 Chapter 1: Linking and Loading ........................................... 7 What do linkers and loaders do? ............................................................ 7 Address binding: a historical perspective .............................................. 7 Linking vs. loading .............................................................................. 10 Tw o-pass linking .............................................................................. 12 Object code libraries ........................................................................ 15 Relocation and code modification .................................................... 17 Compiler Drivers .................................................................................
    [Show full text]
  • Kawa - Compiling Dynamic Languages to the Java VM
    Kawa - Compiling Dynamic Languages to the Java VM Per Bothner Cygnus Solutions 1325 Chesapeake Terrace Sunnyvale CA 94089, USA <[email protected]> Abstract: in a project in conjunction with Java. A language im- plemented on top of Java gives programmers many of Many are interested in Java for its portable bytecodes the extra-linguistic benefits of Java, including libraries, and extensive libraries, but prefer a different language, portable bytecodes, web applets, and the existing efforts especially for scripting. People have implemented other to improve Java implementations and tools. languages using an interpreter (which is slow), or by translating into Java source (with poor responsiveness The Kawa toolkit supports compiling and running vari- for eval). Kawa uses an interpreter only for “simple” ous languages on the Java Virtual Machine. Currently, expressions; all non-trivial expressions (such as function Scheme is fully supported (except for a few difficult fea- definitions) are compiled into Java bytecodes, which are tures discussed later). An implementation of ECMA- emitted into an in-memory byte array. This can be saved Script is coming along, but at the time of writing it is for later, or quickly loaded using the Java ClassLoader. not usable. Kawa is intended to be a framework that supports mul- Scheme [R RS] is a simple yet powerful language. It tiple source languages. Currently, it only supports is a non-pure functional language (i.e. it has first-class Scheme, which is a lexically-scoped language in the Lisp functions, lexical scoping, non-lazy evaluation, and side family. The Kawa dialect of Scheme implements almost effects).
    [Show full text]
  • Codewarrior Development Studio for Starcore 3900FP Dsps Application Binary Interface (ABI) Reference Manual
    CodeWarrior Development Studio for StarCore 3900FP DSPs Application Binary Interface (ABI) Reference Manual Document Number: CWSCABIREF Rev. 10.9.0, 06/2015 CodeWarrior Development Studio for StarCore 3900FP DSPs Application Binary Interface (ABI) Reference Manual, Rev. 10.9.0, 06/2015 2 Freescale Semiconductor, Inc. Contents Section number Title Page Chapter 1 Introduction 1.1 Standards Covered............................................................................................................................................................ 7 1.2 Accompanying Documentation........................................................................................................................................ 8 1.3 Conventions...................................................................................................................................................................... 8 1.3.1 Numbering Systems............................................................................................................................................. 8 1.3.2 Typographic Notation.......................................................................................................................................... 9 1.3.3 Special Terms.......................................................................................................................................................9 Chapter 2 Low-level Binary Interface 2.1 StarCore Architectures......................................................................................................................................................11
    [Show full text]
  • N2761=08-0271 Date: 2008-09-18 Project: Programming Language C++, Core Working Group Reply-To: Michael Wong ([email protected]) Revision: 6
    Towards support for attributes in C++ (Revision 6) Jens Maurer, Michael Wong [email protected] [email protected] Document number: N2761=08-0271 Date: 2008-09-18 Project: Programming Language C++, Core Working Group Reply-to: Michael Wong ([email protected]) Revision: 6 General Attributes for C++ 1 Overview The idea is to be able to annotate some entities in C++ with additional information. Currently, there is no means to do that short of inventing a new keyword and augmenting the grammar accordingly, thereby reserving yet another name of the user's namespace. This proposal will survey existing industry practice for extending the C++ syntax, and presents a general means for such annotations, including its integration into the C++ grammar. Specific attributes are not introduced in this proposal. It does not obviate the ability to add or overload keywords where appropriate, but it does reduce such need and add an ability to extend the language. This proposal will allow many C++0x proposals to move forward. A draft form of this proposal was presented in Oxford and received acceptance in EWG to proceed to wording stage. This proposal integrates suggestions and comments from the Oxford presentation, and email conversations post-Oxford. It addresses many of the controversial aspects from the Oxford presentation and includes comprehensive Standard wordings. Specifically, it adds: Sept 15, 2008, Revision 6 • Updated based on latest draft N2723 • Added support for late-specified return type attributes in 8.1p1 and 8.3.5p2 • Added support
    [Show full text]
  • C++ ABI: the Only Thing That Is More Important Than Performance Icsc 2020
    C++ ABI: the only thing that is more important than performance iCSC 2020 Nis Meinert German Aerospace Center Reading x86-64 Assembly …for fun and profit Function Prologue & Epilogue → Few lines of code at the beginning (prologue) and end (epilogue) of a function, which prepares (and eventually restores) → the stack and → registers → Not part of assembly: convention (defined & interpreted differently by different OS and compilers) Prologue Epilogue 1 push rbp ; rbp: frame pointer 1 mov rsp, rbp 2 mov rbp, rsp ; rsp: stack pointer 2 pop rbp 3 sub rsp, N 3 ret alternatively alternatively 1 enter N, 0 1 leave 2 ret (reserve N bytes on stack for local use) C++ ABI: the only thing that is more important than performance – Reading x86-64 Nis Meinert – German Aerospace Center Assembly 2 / 29 Stack frame for function call ┌──────────────┐ │ ... │ │ 8th Argument │ (rbp + 24) │ 7th Argument │ (rbp + 16) → CALL = PUSH address of next ├──────────────┤ instruction + JMP target │ rip │ (return address) → RET pops return address and │ rbp │ (rbp) transfers control there ├──────────────┤ → pass arguments 1 …6 in │ rbx │ registers (rsi, rdx, …) │ r12 │ │ r13 │ (rsp) └──────────────┘ (stack frame for function call with 8 arguments and local registers rbx, r12 and r13) C++ ABI: the only thing that is more important than performance – Reading x86-64 Nis Meinert – German Aerospace Center Assembly 3 / 29 Reading assembly for fun and profit 1 int f(int x, int y, int z) { # g92 -O0 2 int sum = x + y + z; | _Z1fiii: 3 return sum; 1| push rbp 4 } 1| mov rbp, rsp
    [Show full text]
  • Fivos Kefallonitis
    1 Name mangling demystified 29/10/07 Fivos Kefallonitis [email protected] http://www.int0x80.gr/papers/name_mangling.pdf Copyright © Fivos Kefallonitis 2007 2 Contents Introduction Other examples of name mangling Mixing C and C++ Mangling I. GCC II. MSVC Demangling I. GCC II. MSVC Taking advantage of name mangling Case I Case II Case III Source Code References & Further Reading Introduction This paper will discuss name mangling, also known as name decoration. Name mangling is the process of generating unique names. Name mangling is applied in several areas, although it's primarily used for generating unique (unambiguous) names for identifiers in a computer programming language. Programming languages that deal with function/method overloading have compilers that implement some name mangling scheme. Some examples are C++ (most people mean C++ when they address name mangling), C#, Java, Python, Objective C and Fortran. Name mangling is highly compiler-dependent and it may also vary from one version to another of the same compiler. The mangled name contains all the necessary information that the linker needs, such as linkage type, scope, calling convention, number of arguments, type of arguments etc. Name mangling applies primarily to overloaded functions overloaded operators and variables. Name mangling allows the linker to distinguish between the different overloaded functions that have the same identifier but different parameter lists. Name demangling in programming context is the exact opposite, the process of changing the internal representation of identifiers back to their original source names. Other examples of name mangling Name mangling can be used to achieve "security through obscurity". Name mangling is a very important aspect of code obfuscators.
    [Show full text]