S Allegro Common Lisp Foreign Function Interface

Home , Allegro Common Lisp, T (programming language)

Please do not remove this page

Extensions to the Franz, Inc.'s Allegro Common Lisp foreign function interface

Keane, John https://scholarship.libraries.rutgers.edu/discovery/delivery/01RUT_INST:ResearchRepository/12643408560004646?l#13643533500004646

Keane, J. (1996). Extensions to the Franz, Inc.’s Allegro Common Lisp foreign function interface. Rutgers University. https://doi.org/10.7282/t3-pqns-7h57

This work is protected by copyright. You are free to use this resource, with proper attribution, for research and educational purposes. Other uses, such as reproduction or publication, may require the permission of the copyright holder. Downloaded On 2021/09/29 22:58:32 -0400

Extensions to the Franz Incs

Allegro Common Lisp Foreign

Function Interface

John Keane

Department of Computer Science

Rutgers University

New Brunswick NJ

keanecsrutgersedu

January

Abstract

As provided by Franz Inc the foreign function interface of Allegro Com

mon Lisp has a number of limitations This pap er describ es extensions to

the interface that facilitate the inclusion of C and Fortran co de into Common

Lisp systems In particular these extensions make it easy to utilize libraries

of numerical subroutines such as those from Numerical Recip es in C from

within ACL including those routines that take functions as arguments A

mechanism for creating Lisplike dynamic runtime closures for C routines

is also describ ed

Acknowledgments

The research presented in this do cument is supp orted in part by NASA

grants NCC and NAG This research is also part of the Rutgers

based HPCD Hyp ercomputing and Design pro ject supp orted by the Ad

vanced Research Pro jects Agency of the Department of Defense through

contract ARPADABT C

Motivation

Allegro Common Lisp from Franz Inc provides a means for Lisp routines

to utilize subroutines written in C and Fortran called the foreign function

interface Franz Though adequate for most purp oses the foreign

function interface is not as easy to use as might b e desired and do es not

supp ort certain kinds of interface with foreign co de as straightforwardly as

we might wish In particular we found it dicult to dene and use foreign

co de that takes functions as arguments such as a NewtonRaphson ro ot

nder b ecause of the way in which Ccallable Lisp functions are dened

Additionally ACL version and later utilizes two dierent mecha

nisms for loading foreign co de into the Lisp image one for SVR and Posix

op erating systems and one for BSD op erating systems Under SVR sys

tems all foreign co de must b e rst linked into sharedob ject so libraries

While this provides signicant improvement in the sp eed of loading foreign

co de and reduces the size of the Lisp image it requires that the sharedob ject

libraries b e dened and built ahead of time The two load mechanisms place

an additional burden on Lisp user programs to b e aware of the system under

which they are running and act accordingly if they are to b e p ortable

In writing these extensions we wished to make using foreign co de as

transparent to the user as p ossible We built on top of the solid foundation

of the existing foreign function interface These extensions do not replace

functionality in ACL they complement it

The Foreign Function Extensions Calling

Mo del

Figure illustrates the mo del that is used for all foreign function inter

actions dened through the extensions In the example the Lisp function

foo wishes to call the C routine bar which wishes to call the Lisp func

tion baz These calls are not made directly but are made through help er

routines that are automatically generated when the functions are dened in

Lisp These help er routines add a small amount of additional overhead to

each foreign function call but provide imp ortant functionality in providing

parameter management

In Figure the Lisp function foo calls the Lisp standin function bar (foo ...)

(bar (x y z) ...) bar_c_fe(P)

bar(x',y',z')

(baz-lisp-fe (P) ...) baz(a,b,c)

(baz (a',b',c') ...)

Lisp C

Figure Foreign Function Extensions Calling Mo del

This function takes the arguments of the C routine bar which it then re

packages into a parameter control blo ck and calls the C frontend routine

bar c fe This routine takes the parameter control blo ck and calls the target

C routine bar after recovering the parameters

The C routine bar calls the C standin routine baz This function takes

the arguments of the Lisp function baz which it then repackages into a

parameter control blo ck and calls the Lisp frontend routine bazlispfe

This function takes the parameter control blo ck and calls the target Lisp

function baz after recovering the parameters

Dening Foreign Functions

Two new functions have b een added to dene foreign functions These are

ffxdefineCroutine header key typecheck nil

writefiles t

compile nil

interfacelispdir

interfacecdir

ffxdefineFortranroutine header key typecheck nil

writefiles t

compile nil

interfacelispdir

interfacecdir

Both functions have a required argument the header This sp ecies the

calling parameters and return values of the function using C data types

The header can b e automatically created by parsing the C header for the

function

The typecheck argument sp ecies whether argument type checking co de

is to b e included in the interface functions The writefiles argument

sp ecies whether the generated interface co de is to b e written out The

compile argument sp ecies whether the interface routines are to b e compiled

The last two arguments give the target directories for the interface co de

These routines dier from the ACL ffdefforeign function in that they

do not need to b e invoked every time the foreign co de is included into a

system If the interface routines are written out and compiled only the load

function b elow need b e used

Dening Ccallable Lisp Functions

ffxdefineexternalcallable header key callback nil

writefiles t

compile nil

interfacelispdir

interfacecdir

C headers can b e generated for Fortran routines by using the widelyavailable fc

Fortran to C compiler

The arguments to this routine are much the same as for the previous two

routines except that the function sp ecied in the header is a Lisp function

If the callback argument is t the interface is generated as a callback which

will allow C routines to call dynamic lambda expressions

Loading Foreign Functions

ffxloadexternalsystem key routines

files

libraries

interfacelispdir

interfacecdir

solibname

solibdir

rebuildsolib t

The routines argument is a list of strings containing the names of rou

tines dened by ffxdefineCroutine ffxdefineFortranroutine or

ffxdefineexternalcallable The files argument is a list of strings

containing the names of ob ject les to load The libraries argument is

a list of strings containing the names of system libraries or lo cal libraries

to load with the foreign co de Under SVR systems the user must sp ecify

solibname the name of the sharedob ject so lib If rebuildsolib is T

the so lib will b e rebuilt b efore loading

This function handles the entire foreign load pro cess under b oth SVR

and BSD systems It issues the required ffdefforeign calls and invokes

the foreign function loader It also takes care of registering the externally

callable Lisp functions and passing the call p ointers to the asso ciated C

interface functions If the interface routines have b een previously generated

and compiled this function is the only one required to load external co de

into the Lisp system

Passing Lisp Functions to Foreign Func

tions

Many C routines take a C function p ointer as an argument If the user wishes

to pass a named Lisp routine to a C function she may use the function

ffxgetCfunctionpointer which takes the name of a Lisp function

previously dened with ffxdefineexternalcallable and returns the

p ointer to the C standin interface routine for that function

Supp ose that the user wishes to pass a dynamically created lambda ex

pression instead such as a function closure Because of their dynamic na

ture it would b e highly inecient to have to generate a separate unique

interface for each dynamic closure Instead the user may use the function

ffxbindlispcallback This function takes as arguments a lambda ex

pression and the name of a callback previously dened with

ffxdefineexternalcallable taking the same formals as the lambda

A C function p ointer is returned that when dereferenced by a C routine

will cause the closure to b e called through the callback interface co de Any

number of dynamically generated closures can b e b ound to a single callback

and a unique C function p ointer will b e created for each

The function ffxbindlispcallback works by creating a tiny machine

language program called a function vector Each function vector is ab out

bytes long and is malloced in Lisp foreign function space The function vec

tor contains the address of the C standin interface routine for the callback

the address of a designated storage lo cation in Lisp foreign function space

and a unique Lisp index to the closure The address of the function vector is

what is returned by ffxbindlispcallback When control is transfered

to the function vector it stores the closure index into the designated storage

lo cation and transfers control unconditionall y to the C standin interface

routine Since the stack is not touched the C standin interface routine nds

all the arguments it is exp ecting just as if it had b een called directly by

the C routine It then builds the parameter blo ck and calls the Lisp front

end for the callback The frontend function lo oks at the designated storage

lo cation nds the index to the closure and calls it

We note in passing that this mechanism for function vectors can also b e

used within systems written entirely in C to give the eect of Lisp function

closures without the need to pass additional information through global

variables This allows many problems to b e avoided that o ccur when global

variables are overwritten during nested function calls

Example

This is an example of the foreign function extensions applied to the function

newt from Numerical Recipes in C Press et al This routine takes

two function p ointers as arguments the function to solve for the ro ots of

and a function to compute the Jacobian of the rst function Here is the

denition of newt and the callbacks for the Lisp routines

This routine is part of the Numerical Recipes in C library

setf newtheader makeinstance header

routinename newt

returntype void

carglist x double array

n int none

check int pointer

fixits int none

vecfunc void function

jacfunc void function

cheader

void newtdouble x int n

int check int fixits

void vecfunc

int double double

void jacfunc

int double double

double

void vecfunc

int double double

ffxdefinecroutine newtheader typecheck lispcinterfacetypecheck

interfacelispdir lispcinterfacelispdir

interfacecdir lispcinterfacecdir

writefiles t

compile t

setf newtvecfuncheader makeinstance header

routinename newtvecfunc

returntype void

carglist n int none

x double array

f double array

cheader

void newtvecfuncint n double x

double f

ffxdefineexternalcallable newtvecfuncheader callback t

interfacelispdir

lispcinterfacelispdir

interfacecdir

lispcinterfacecdir

writefiles t

compile t

setf newtjacfuncheader makeinstance header

routinename newtjacfunc

returntype void

carglist n int none

x double array

fvec double array

df double array

vecfunc void function

cheader

void newtjacfuncint n double x

double fvec

double df

void vecfunc

int double

double

ffxdefineexternalcallable newtjacfuncheader callback t

interfacelispdir

lispcinterfacelispdir

interfacecdir

lispcinterfacecdir

writefiles t

compile t

Heres how the routines are loaded into Lisp

ffxloadexternalsystem routines newt

newtvecfunc

newtjacfunc

files

list