Please do not remove this page Extensions to the Franz, Inc.'s Allegro Common Lisp foreign function interface Keane, John https://scholarship.libraries.rutgers.edu/discovery/delivery/01RUT_INST:ResearchRepository/12643408560004646?l#13643533500004646 Keane, J. (1996). Extensions to the Franz, Inc.’s Allegro Common Lisp foreign function interface. Rutgers University. https://doi.org/10.7282/t3-pqns-7h57 This work is protected by copyright. You are free to use this resource, with proper attribution, for research and educational purposes. Other uses, such as reproduction or publication, may require the permission of the copyright holder. Downloaded On 2021/09/29 22:58:32 -0400 Extensions to the Franz Incs Allegro Common Lisp Foreign Function Interface John Keane Department of Computer Science Rutgers University New Brunswick NJ keanecsrutgersedu January Abstract As provided by Franz Inc the foreign function interface of Allegro Com mon Lisp has a number of limitations This pap er describ es extensions to the interface that facilitate the inclusion of C and Fortran co de into Common Lisp systems In particular these extensions make it easy to utilize libraries of numerical subroutines such as those from Numerical Recip es in C from within ACL including those routines that take functions as arguments A mechanism for creating Lisplike dynamic runtime closures for C routines is also describ ed Acknowledgments The research presented in this do cument is supp orted in part by NASA grants NCC and NAG This research is also part of the Rutgers based HPCD Hyp ercomputing and Design pro ject supp orted by the Ad vanced Research Pro jects Agency of the Department of Defense through contract ARPADABT C Motivation Allegro Common Lisp from Franz Inc provides a means for Lisp routines to utilize subroutines written in C and Fortran called the foreign function interface Franz Though adequate for most purp oses the foreign function interface is not as easy to use as might b e desired and do es not supp ort certain kinds of interface with foreign co de as straightforwardly as we might wish In particular we found it dicult to dene and use foreign co de that takes functions as arguments such as a NewtonRaphson ro ot nder b ecause of the way in which Ccallable Lisp functions are dened Additionally ACL version and later utilizes two dierent mecha nisms for loading foreign co de into the Lisp image one for SVR and Posix op erating systems and one for BSD op erating systems Under SVR sys tems all foreign co de must b e rst linked into sharedob ject so libraries While this provides signicant improvement in the sp eed of loading foreign co de and reduces the size of the Lisp image it requires that the sharedob ject libraries b e dened and built ahead of time The two load mechanisms place an additional burden on Lisp user programs to b e aware of the system under which they are running and act accordingly if they are to b e p ortable In writing these extensions we wished to make using foreign co de as transparent to the user as p ossible We built on top of the solid foundation of the existing foreign function interface These extensions do not replace functionality in ACL they complement it The Foreign Function Extensions Calling Mo del Figure illustrates the mo del that is used for all foreign function inter actions dened through the extensions In the example the Lisp function foo wishes to call the C routine bar which wishes to call the Lisp func tion baz These calls are not made directly but are made through help er routines that are automatically generated when the functions are dened in Lisp These help er routines add a small amount of additional overhead to each foreign function call but provide imp ortant functionality in providing parameter management In Figure the Lisp function foo calls the Lisp standin function bar (foo ...) (bar (x y z) ...) bar_c_fe(P) bar(x',y',z') (baz-lisp-fe (P) ...) baz(a,b,c) (baz (a',b',c') ...) Lisp C Figure Foreign Function Extensions Calling Mo del This function takes the arguments of the C routine bar which it then re packages into a parameter control blo ck and calls the C frontend routine bar c fe This routine takes the parameter control blo ck and calls the target C routine bar after recovering the parameters The C routine bar calls the C standin routine baz This function takes the arguments of the Lisp function baz which it then repackages into a parameter control blo ck and calls the Lisp frontend routine bazlispfe This function takes the parameter control blo ck and calls the target Lisp function baz after recovering the parameters Dening Foreign Functions Two new functions have b een added to dene foreign functions These are ffxdefineCroutine header key typecheck nil writefiles t compile nil interfacelispdir interfacecdir ffxdefineFortranroutine header key typecheck nil writefiles t compile nil interfacelispdir interfacecdir Both functions have a required argument the header This sp ecies the calling parameters and return values of the function using C data types The header can b e automatically created by parsing the C header for the 1 function The typecheck argument sp ecies whether argument type checking co de is to b e included in the interface functions The writefiles argument sp ecies whether the generated interface co de is to b e written out The compile argument sp ecies whether the interface routines are to b e compiled The last two arguments give the target directories for the interface co de These routines dier from the ACL ffdefforeign function in that they do not need to b e invoked every time the foreign co de is included into a system If the interface routines are written out and compiled only the load function b elow need b e used Dening Ccallable Lisp Functions ffxdefineexternalcallable header key callback nil writefiles t compile nil interfacelispdir interfacecdir 1 C headers can b e generated for Fortran routines by using the widelyavailable fc Fortran to C compiler The arguments to this routine are much the same as for the previous two routines except that the function sp ecied in the header is a Lisp function If the callback argument is t the interface is generated as a callback which will allow C routines to call dynamic lambda expressions Loading Foreign Functions ffxloadexternalsystem key routines files libraries interfacelispdir interfacecdir solibname solibdir rebuildsolib t The routines argument is a list of strings containing the names of rou tines dened by ffxdefineCroutine ffxdefineFortranroutine or ffxdefineexternalcallable The files argument is a list of strings containing the names of ob ject les to load The libraries argument is a list of strings containing the names of system libraries or lo cal libraries to load with the foreign co de Under SVR systems the user must sp ecify solibname the name of the sharedob ject so lib If rebuildsolib is T the so lib will b e rebuilt b efore loading This function handles the entire foreign load pro cess under b oth SVR and BSD systems It issues the required ffdefforeign calls and invokes the foreign function loader It also takes care of registering the externally callable Lisp functions and passing the call p ointers to the asso ciated C interface functions If the interface routines have b een previously generated and compiled this function is the only one required to load external co de into the Lisp system Passing Lisp Functions to Foreign Func tions Many C routines take a C function p ointer as an argument If the user wishes to pass a named Lisp routine to a C function she may use the function ffxgetCfunctionpointer which takes the name of a Lisp function previously dened with ffxdefineexternalcallable and returns the p ointer to the C standin interface routine for that function Supp ose that the user wishes to pass a dynamically created lambda ex pression instead such as a function closure Because of their dynamic na ture it would b e highly inecient to have to generate a separate unique interface for each dynamic closure Instead the user may use the function ffxbindlispcallback This function takes as arguments a lambda ex pression and the name of a callback previously dened with ffxdefineexternalcallable taking the same formals as the lambda A C function p ointer is returned that when dereferenced by a C routine will cause the closure to b e called through the callback interface co de Any number of dynamically generated closures can b e b ound to a single callback and a unique C function p ointer will b e created for each The function ffxbindlispcallback works by creating a tiny machine language program called a function vector Each function vector is ab out bytes long and is malloced in Lisp foreign function space The function vec tor contains the address of the C standin interface routine for the callback the address of a designated storage lo cation in Lisp foreign function space and a unique Lisp index to the closure The address of the function vector is what is returned by ffxbindlispcallback When control is transfered to the function vector it stores the closure index into the designated storage lo cation and transfers control unconditionall y to the C standin interface routine Since the stack is not touched the C standin interface routine nds all the arguments it is exp ecting just as if it had b een called directly by the C routine It then builds the parameter blo ck and calls the Lisp front end for the callback The frontend function lo oks at the designated storage lo cation nds the index