Automated Fortran–C++ Bindings for Large-Scale Scientific Applications Seth R Johnson HPC Methods for Nuclear Applications Group Nuclear Energy and Fuel Cycle Division Oak Ridge National Laboratory ORNL is managed by UT–Battelle, LLC for the US Department of Energy github.com/swig-fortran Overview • Introduction • Tools • SWIG+Fortran • Strategies • Example libraries 2 Introduction 3 How did I get involved? • SCALE (1969–present): Fortran/C++ • VERA: multiphysics, C++/Fortran • MPACT: hand-wrapped calls to C++ Trilinos 4 Project background • Exascale Computing Project: at inception, many scientific app codes were primarily Fortran • Numerical/scientific libraries are primarily C/C++ • Expose Trilinos solver library to Fortran app developers: ForTrilinos product 5 ECP: more exascale, less Fortran Higher-level { }Fortran ECP application codes over time (credit: Tom Evans) 6 Motivation • C++ library developers: expand user base, more F opportunities for development and follow-on funding • Fortran scientific app developers: use newly exposed algorithms and tools for your code C • Multiphysics project integration: in-memory coupling of C++ physics code to Fortran physics code • Transitioning application teams: bite-size migration from Fortran to C++ C++ 7 Tools 8 Wrapper “report card” • Portability: Does it use standardized interoperability? • Reusability: How much manual duplication needed for new interfaces? • Capability: Does the Fortran interface have parity with the C++? • Maintainability: Do changes to the C++ code automatically update the Fortran interface? • Robustness: Is it possible for silent failures to creep in? • Integration: How much overhead is needed to get the glue code working in development/deployment? 9 Hand-rolled binding code Portability � • Often based on hand-rolled C interface layer Reusablity � • Often targets F77/90 using configure-time bindings Capability � Maintainability � • Examples: SuperLU, STRUMPACK, TASMANIAN Robustness � Integration �/� 10 Project-specific scripts • Text processing engines hardcoded to a Portability � particular project’s data types, header Reusablity � formatting Capability � • Simple translations of function signatures Maintainability and types to “glue code” � Robustness � • Examples: MPICH, HDF5, Silo, PETSc Integration �/� 11 Automated code generators (manual C++ declaration) Portability � • CIX: in-house ORNL tool Reusablity � (template substitution) Capability � • Shroud: recent LLNL development Maintainability � (custom YAML) Robustness � Integration �/� 12 Automated code generators (integrated C++ parsing) Portability � Reusablity � Capability � • SWIG+Fortran: fork of Simplified Wrapper Interface Generator Maintainability � Robustness � Integration � 13 SWIG+Fortran 14 Supported SWIG: Simplified Wrapper and Interface Generator Languages • Allegro CL • C# • CFFI • CLISP • • Chicken Generate interfaces to existing C and C++ code and data • D types so that a target language can invoke functions and use • Go • Guile the data • Java • Javascript • “Glue” code: flat C-linkage wrappers to C++ functions, • Lua • Modula-3 corresponding interfaces in target language • Mzscheme • OCAML • Does not couple target languages to other target languages • Octave • Perl • Does not parse target languages or create C++ proxy • PHP • Python wrappers • R • Ruby • Scilab • Tcl • UFFI 15 SWIG execution sequence • SWIG reads .i interface file which may %include other C/C++ headers to parse them • Interface file can be as little as a couple of lines for simple interfaces • More difficult C++/Fortran conversions require more interface code • SWIG generates source code files: .cxx and .f90 • This wrapper code can be distributed like regular source files • Library users don’t need to know SWIG User-written code SWIG-generated code Foo.cxx Foo.hh SWIG foo.f90 main.f90 Foo.i foo_wrap.cxx 16 Control flow and data conversion Call “native” procedure user.f90 Public module procedure Convert Fortran types to ISO_C_BINDING types module.f90 Call BIND(C) interface • Fortran 2003 standard defines C-compatible datatypes • Use only Fortran-compatible ISO C datatypes extern "C" wrapper function Convert C type to C++ type • Minimize data movement of numerical arrays Call C++ library function module_wrap.cxx C++ Library code library.cxx 17 Features • Primitive types • Function overloading • Enumerations • Template instantiation • Classes (with inheritance) • Compile-time constants • C strings and std::string • Exception handling • Function pointers • OpenACC and Thrust • Arrays and std::vector 18 Addition with “native” int: input and generated C code F/C interface #ifndef simplest_h SWIGEXPORT int _wrap_add( #define simplest_h int const *farg1, int add(int a, int b); #endif int const *farg2) { int fresult ; Input argument int arg1 ; conversion simplest.h int arg2 ; int result; arg1 = (int)(*farg1); arg2 = (int)(*farg2); %module simplest Wrapped result = (int)add(arg1,arg2); function call %typemap(ftype, in="integer, intent(in)") fresult = (int)(result); int "integer" return fresult; %typemap(fin) int "$1 = int($input, C_INT)" %typemap(fout) int "$result = int($1)" } %include "simplest.h" simplest.i Output simplest_wrap.c argument conversion 19 Addition with “native” int: generated Fortran Fortran proxy function module simplest .... use, intrinsic :: ISO_C_BINDING F/C implicit none interface function add(a, b) & private result(swig_result) public :: add integer :: swig_result interface integer, intent(in) :: a function swigc_add(farg1, farg2) & integer, intent(in) :: b bind(C, name="_wrap_add") & integer(C_INT) :: fresult Input result(fresult) integer(C_INT) :: farg1 argument integer(C_INT), intent(in) :: farg1 integer(C_INT) :: farg2 integer(C_INT), intent(in) :: farg2 conversion integer(C_INT) :: fresult farg1 = int(a, C_INT) Wrapper end function farg2 = int(b, C_INT) function call end interface fresult = swigc_add(farg1, farg2) contains Output swig_result = int(fresult) argument end function .... conversion end module simplest.f90 20 Simple addition function: the part app devs care about SWIGEXPORT int swigc_add(int const *farg1, program main int const *farg2); use simplest, only : add write (0,*) add(10, 20) end program simplest_wrap.c module simplest use, intrinsic :: ISO_C_BINDING main.f90 … contains function add(a, b) & result(swig_result) integer :: swig_result integer, intent(in) :: a integer, intent(in) :: b … end function $ ./main.exe end module simplest.f90 30 21 Automatic BIND(C) wrapping %module bindc module bindc Only generate … interface code type, bind(C), public :: Point %fortranbindc; real(C_FLOAT), public :: x %fortran_struct(Point) real(C_FLOAT), public :: y Don’t create real(C_FLOAT), public :: z proxy class end type Point %inline { … interface typedef struct { float x, y, z; } Point; subroutine print_point(p) & void print_point(const Point* p); bind(C, name="print_point") void make_point(Point* pt, use, intrinsic :: ISO_C_BINDING import :: point const float xyz[3]); type(Point), intent(in) :: p } end subroutine subroutine make_point(pt, xyz) & bind(C, name="make_point") use, intrinsic :: ISO_C_BINDING import :: point type(Point) :: pt real(C_FLOAT), dimension(3), intent(in) :: xyz end subroutine end interface bindc.i end module bindc.f90 22 Templated class template<typename T> Insert raw C++ %module "templated" Tell SWIG to class Thing { code into parse the T val_; generated %{ header file public: wrapper file #include "templated.hpp" Thing(T val); %} T get() const; %include "templated.hpp" }; // Instantiate templated classes template<typename T> %template(Thing_Int) Thing<int>; void print_thing(const Thing<T>& t); %template(Thing_Dbl) Thing<double>; // Instantiate and overload a function Instantiate %template(print_thing) print_thing<int>; templates %template(print_thing) print_thing<double>; at SWIG time templated.i 23 Templated class: generated Fortran wrapper code Memory module templated ownership interface print_thing … module procedure swigf_print_thing__SWIG_1, integer, parameter, public :: swig_cmem_own_bit = 0 swigf_print_thing__SWIG_2 integer, parameter, public :: swig_cmem_rvalue_bit = 1 end interface public :: print_thing type, bind(C) :: SwigClassWrapper interface Overloaded type(C_PTR), public :: cptr = C_NULL_PTR subroutine swigc_delete_Thing_Int(farg1) & integer(C_INT), public :: cmemflags = 0 bind(C, name="_wrap_delete_Thing_Int") function end type Opaque class … ! class Thing< int > contains type, public :: Thing_Int wrapper subroutine swigf_release_Thing_Int(self) type(SwigClassWrapper), public :: swigdata Call delete if use, intrinsic :: ISO_C_BINDING contains we “own” procedure :: get => swigf_Thing_Int_get class(Thing_Int), intent(inout) :: self procedure :: release => swigf_release_Thing_Int type(SwigClassWrapper) :: farg1 the memory procedure, private :: swigf_Thing_Int_op_assign__ farg1 = self%swigdata generic :: assignment(=) => swigf_Thing_Int_op_assign__ if (btest(farg1%cmemflags, swig_cmem_own_bit)) then end type Thing_Int call swigc_delete_Thing_Int(farg1) interface Thing_Int endif module procedure swigf_create_Thing_Int Fortran farg1%cptr = C_NULL_PTR end interface farg1%cmemflags = 0 ! class Thing< double > proxy class self%swigdata = farg1 type, public :: Thing_Dbl Second end subroutine … template … end type Thing_Dbl instantiation end module … templated.f90 (2/2) 24 Exception handling %module except program main %include <std_except.i> use except, only : do_it, do_it_again, ierr, get_serr %exception { call do_it(-3) SWIG_check_unhandled_exception();