Fast FloatingPoint Pro cessing in Common Lisp

RICHARD J FATEMAN

UniversityofCalifornia Berkeley

and

KEVIN A BROUGHAN and DIANE K WILLCOCK

UniversityofWaikato

and

DUANE RETTIG

Franz Inc

Lisp one of the oldest higherlevel programming languages has rarely b een used for

fast numerical oatingp oint computationWe explore the b enets of Common Lisp an

emerging new language standard with some excellent implementations for numerical computation

We compare it to Fortran in terms of the sp eed of eciency of generated co de as well as the

structure and convenience of the language There are a surprising number of advantages to Lisp

esp ecially in cases where a mixture of symbolic and numeric pro cessing is needed

Categories and Sub ject Descriptors G Mathematics of Computing Mathematical Software

eciency portability D Programming Languages Pro cessorscompilers interpreters

optimization

General terms Common Lisp Fortran C Floatingp oint arithmetic

Additional Key Words and Phrases Compiler optimization Symb olic computation Numerical

algorithms

INTRODUCTION

The combination of symbolic and numeric computation has b ecome p opular in sev

eral application areas These include for example motion planning for rob otics

signal pro cessing and image understanding One burgeoning area is the develop

mentofscientic computing environments including symb olic mathematics sys

tems numerical computing and graphics These tend to mix traditional strengths

of the Lisp programming language interactivity and symb ol manipulation with

the additional need for numerics

In general mathematical mo deling of complex physical phenomena includes a

symb olic manipulation phase of mo del formulation followed byanumerical solu

tion phase and output visualization Mo del formulation has traditionally b een a

Authors addresses Richard J Fateman Computer Science Division University of California

Berkeley CA USA Kevin A Broughan and Diane K Willco ck Dept of Mathematics and

Statistics UniversityofWaikato Private Bag Hamilton NZ

Duane Rettig Franz Inc UniversityAve Berkeley CA USA email fate

manp eoplesparcBerkeleyedu kabwaikatoacnz duanefranzcom

Permission to copy without fee all or part of this material is granted provided that the copies are

not made or distributed for direct commercial advantage the ACM copyright notice and the title

of the publication and its date app ear and notice is given that copying is by p ermission of the

Asso ciation for Computing MachineryTo copy otherwise or to republish requires a fee andor sp ecic p ermission

R J Fateman et al

p encilandpap er activity but in recentyears computer algebra systems haveas

sisted in analysis and subsequentFortran co de generation for numerical compu

tation Since dierent languages are b eing used for the analysis and nal execution

this approach precludes the development of algorithms which use mixed symb olic

numeric pro cessing at a deep er level A b etter arrangmentwould allow the analysis

and execution phases to exist simultaneouslyallowing eachtointeract with the

other Still even if the symb olic and numeric phases coexist as a practical mat

ter errors or exceptional conditions on one side of the software b oundary tend to

b e dealt with inconveniently on the other side suchevents include oatingp oint

exceptions keyb oard or mousegenerated interrupts op erating system actions and

storagereclamation pro cessing garbage collection in Lisp A b etter metho d of

joining the two kinds of pro cessing without descending to lowlevel implementation

co ding would b e useful

For some time now the need for a symb olicnumeric mixture has b een well

recognized by designers of computer algebra systems Sometimes this need is met

entirely in the Lisp system environmentby writing in Lisp Sometimes the numeric

or graphical routines are loaded from a standard library or written in advance using

CorFortran but integrated at compile or load time with the computer algebra

system Sometimes the need for numerical co de is only satised by pro ducing on

the y Fortran sourceco de compiling and loading it backinto Lisp This last

approach tends to b e delicate but several systems have b een built around the

use of symb olic systems to help co de Fortran subroutines which are then t into

templates of other Fortran co de to pro duce complete packages Some recen twork

includes work by Wirth Lanam Wang Broughan Co ok and

Ka jler

As a principal example of the approach of linking Lisp programs with numerics

consider the developmentofsymb olicnumeric algorithms for the Senac environ

ment under way at the UniversityofWaikato New Zealand Mathematical Soft

ware Pro ject The goal of this development has b een to combine computer algebra

languages with numeric and graphic subroutine libraries The combination has

until recently b een achieved using Lispbased algebra languages combined with

Fortranbased numeric and graphics co de The linking of these systems has of

ten employed Lisp foreign function interfaces to C and Fortranlanguage co de

Such links to foreign functions are included in a numb er of mo dern Lisp systems

although the linkage sp ecication syntax is not yet standardized

Another technique sometimes used is to set up interpro cess communication IPC

between dierent pro cesses The IPC may in fact link programs running on dierent

computers Either technique provides an alternative to the traditional concept of

the users explicit editing of symb olic mathematical text into a numeric source

program although the IPC technique is even less likely to b e robust in the face of

errors than the foreignfunction linkage

Using b oth of these metho ds the Waikato group has written a number of in

terfaces Naglink was written b etween Macsyma and the NAG

Library Numlink between Senac and Graink and the NAG

Fortran Library and Graphics Library

These references describ e howcomponents have b een integrated to provide a very

highly automated problemsolving environment combining symb olic numeric and

graphic to ols As exp ected the approaches retain the eciency and robustness of

Fast FloatingPoint Pro cessing in Common Lisp

the Fortran co de They automate using interface programs written in Lisp the

calling and return pro cesses and userwritten subroutines demanded bysome

Fortran packages eg numerical integration minimization can in fact b e written

by Senac rather than the user An additional b enet of the Senac approachisthat

the user need not attend to the tedious requirements to provide output formatting

In spite of success in researchenvironments there have b een obstacles to the

widespread propagation and generalization of these concepts They include

The lack of a language standard for foreign function interfaces in Common Lisp

Standardization on this topic is arguably to o dicult or controversial to hold

out much exp ectation for a standard so on

The low standard of maintenance of implementation dep endent foreign func

tion interfaces and linkers in general by language and op erating system or

hardware vendors The Waikato exp erience has b een that of grappling with a

succession of serious bugs

Lack of compatibilitybetween libraries for dierent language compilers and

between ob jectco de formats for the same compiler at dierent revision levels

This diversity has created a degree of insecurity for this and other comp osite

language environment develop ers

Op erating system defects Wecontinue to see ma jor computer corp orations

releasing versions of their op erating systems for highsp eed workstations with

no provision for dynamic linking

In this pap er we explore an approachwhich enables all of the problems listed

ab ovetobesolved at a single stroke use Lisp as the source language for the

numeric and graphical co de

This is not a new idea it was tried at MIT and UCB in the s While

these exp eriments were mo destly successful the particular systems are obsolete

Fortunately some of those ideas used in Maclisp NIL andFranz Lisp

were incorp orated in the subsequent standardization of Common Lisp CL

In this new setting it is appropriate to reexamine the theoretical and practical

implications of writing numeric co de in Lisp

The p opular conceptions of Lisps ineciency for numerics have b een based on

rumor supp osition and exp erience with early and in fact inecient implemen

tations It is certainly p ossible to continue to write inecient programs As one

example of the results of deemphasizing numerics in the design consider the situ

ation of the basic arithmetic op erators The denitions of these functions require

that they are generic eg must b e able to add anycombination of several pre

cisions of oats arbitraryprecision integers rational numb ers and complexes The

very simple way of implementing this arithmetic b y subroutine calls is also v ery

inecient Even with appropriate declarations to enable more sp ecic treatmentof

numeric typ es compilers are free to ignore declarations and such implementations

naturally do not accommo date the needs of intensivenumb ercrunching See the

app endix for further discussion of declarations

Be this as it may the situation with resp ect to Lisp has changed for the b etter

in recentyears With the advent of ANSI standard Common Lisp several active

vendors of implementations and one active university research group Carnegie Mel

lon there has b een a signicantimprovement in the quality and eciency of the

R J Fateman et al

numeric facilities available in Lisp systems Based on our exp erience rep orted in

part in this pap er we exp ect that co de from Common Lisp compilers can demon

strate comparable eciency to that of go o d Fortran and C compilers

Another recentdevelopmentmakes it imp ortant to address numerical Lisp pro

cessing It is the now nearlyuniversal adoption of the IEEE binary oating

p oint arithmetic standard by hardware manufacturers Standards for semantics

for Lisp arithmetic as well as the sp eed of compiled co de inuence each other

Although we will not discuss it in this pap er clearly a language design should not

enforce arithmetic semantics at o dds with the data typ es op erations and error

control of the underlying hardware

Given these developments we are therefore encouraged to explore numeric en

vironments in Lisp so as to combine soundlybased and easytouse programming

environments with new designs for go o d oatingp oint implementations of Com

mon Lisp Further encouragement comes from the observation that languages with

Lisps functional and ob jectoriented programming style and exible storage allo

cation are b eing prop osed for to days more challenging programming tasks even

those that have b een seen as primarily numerical Lisp may emerge as an esp ecially

strong contender in such a comp etition

The plan of this pap er is as follows in section we briey review those features

of Common Lisp as a language which might reasonably b e picked out as esp e

cially advantageous or disadvantageous for numeric programming Section then

elab orates on our general motivation for seriously considering Lisp for numerical

programming Section discusses some of the interaction b etween the language

its sp ecications including the use of declarations and machine architectures In

section we consider at some length the kinds of demands we mightmakeona

mo dern Lisp compiler in general and then more sp ecically using several graded

examples clearing an arraymultiplying matrices and nally singularvalue de

comp osition We examine the dierences b etween compiled Lisp and Fortran co de

as well as run times In section we rep ort on the design and implementation of an

automated FortrantoLisp translator together with some ideas for its extension

Finally section provides some conclusions

Toanticipate our conclusions weintend to show

a Many features of CL are useful in the context of numeric computing

b CL co de as pro duced by current compilers is really fairly go o d

c Even more improved basic numeric capabilities can b e incorp orated in CL

d Converting existing Fortranlikenumeric co de to Lisp can b e automated

e Numerical Recipes or other numerical co de libraries can b e written in Lisp

and may b e viable alternatives to versions in Fortran or C

LANGUAGE FEATURES AND NUMERICAL PROGRAMMING

Intro duction

In this section we deal primarily with the Common Lisp CL language as describ ed

informally by Steele and the sub ject of a forthcoming language standard the

ANSI XJ committee We o ccasionally deal with issues often misconceptions

ab out the earlier language design dating back to McCarthy Our ob jective

is to examine the ma jor reasons for and against numerical programming in Lisp

Fast FloatingPoint Pro cessing in Common Lisp

In comparing features of CL and Fortran it is imp ortant to realize that many

programmers use implementations of Fortran or even Fortran language stan

dards The presence of a particular feature in Fortran do es not mean that

Fortran programmers often committed to p ortability and backward compatibility

can use it

Indeed to remedy some failings of Fortran some programmers are moving not to

Fortran but to C or C Others are using interactive highlevel numerical

computation systems like Matlab MathCad Gauss etc as well as computer alge

bra systems such as Maple or Mathematica It app ears that C and C are not

entirely satisfactory substitutes for Fortran they app ear to b e displacing Fortran

in only sp ecialized contexts While we view the success of interactive systems tar

geted for scientic computing as a very p ositivedevelopment they generally fail to

supp ort the development and integration of new fast compiled co de in the same

interactive language

We emphasize this last p oint without ecient userpro duced compiled co de

it is dicult to extend the application of systems b eyond the b oundaries of their

predened capabilities Such a barrier limits the convenientgrowth of environments

for scientic computing to new applications where p erformance is an imp ortant

comp onent

With this in mind can Lisp accommo date the needs of numeric computing

Positive asp ects of CL

First of all we will list some of the more imp ortant features viewed from a numerical

computing p ersp ective In section wegive more details of our motivation and in

section additional examples

Lisp provides the user with

A richenvironmen t including access to interactive and compiled co de with

the same dynamic b ehavior interactive debugging in the same environmentand

instant turnaround for testing and selfeducation

Language features supp orting functional programming lexical scoping clo

sures and lo cal functions user or system dened generic functions uniform context

indep endent syntax exible functiondening mechanisms allowing keyword op

tional or an arbitrary numb er of arguments and p owerful macro denition capabil

ities

A data typ e system including supp ort for structures vectors bitvectors

strings arrays les hashtables readtables There is also an elab orate sys

tem for ob jectoriented programming generic functions with inheritance

A library of programs and predened data typ es covering a large number of

supp orted arithmetic comp onents These include exact integer rational complex

rational complex integer single oat double oat complex builtin elementary

trigonometric hyp erb olic logical and other functions

Declarations which are in fact hints for the compiler Although these are

optional they can in most implementations b e used to sp ecify the desirabilityof

various attributes of the compiled co de including sp eed size errorchecking See

app endix for further comments

R J Fateman et al

Negative asp ects of CL

Wenow consider some of the reasons usually given against numerical programming

in Lisp and a brief outline of our resp onses Fuller details follow in Sections and

b elow

Tradition Most engineers learn Fortran not Lisp Most programs that must

be maintained in scientic applications are written in Fortran often Fortran

Resp onse Times are changing

Sp eed Lisp is slowfornumerical computation Resp onse Lisp compiled co de

still seems somewhat inferior to that of the b est optimized Fortran Sp eed maybe

only one criterion however and we also exp ect that Lisp will get faster Weprovide

more detail on this in Section b elow

Size Lisp programs have a large runtime size compared to standalone C or

Fortran Resp onse Of course those other systems are getting bigger in some cases

memory prices are low compared to software and even small computers can b e

loaded with substantial RAM nally Lisp vendors are paying more attention to

size reduction esp ecially for runtime delivery systems

InputOutput Some typ es of raw input or output are not available in Lisp

Resp onse We dont intend to address this issue here If necessary sp ecial IO can

be achieved using foreign functions to move data into or out of memory

Garbage Collection Lisps storage reclamation constitutes an uninterruptable

and lengthy pause in real pro cessing Resp onse this has b een largely overcome

technically by socalled generationscavenging garbage collection Using Lisp for

realtime pro cessing is still not p opular but then most timesharing systems used

to supp ort Lisp are not truly realtime anywayMyfavorite text editor has a

slightly more noticeable memory allo cation b ehavior If a pause for a second every

few minutes is strongly ob jectionable you migh t indeed register an ob jection to

timesharing as well as Lisp

Syntax The language p ermits abstruse programs Resp onse Wehopetoshow

this is a matter of familiarity that is easily solved and that there are advantages

to Lisps syntax

Semantics There are indeed some language features in the CL design that

can lead to some inecient programs and program interfaces H Baker claims

The p olymorphic typ e complexity of the Common Lisp library functions is mostly

gratuitous and b oth the eciency of compiled co de and the eciency of the pro

grammer could b e increased by rationalizing this complexity For example the

typ e of the result of the some functions eg sqrt or log cannot b e computed

based solely on the type of argument One must know something ab out the value

a negative argumenttosqrt will result in a complex numb er Determining typ es

at compiletime is imp ortant for ecientcode One remedywould b e to use sp e

cialized versions of squarero ot for example realdoublesqrt which will either

return a doubleoat or signal an error for a negative argument Rob MacLachlan

p ointed out another p ossibility used in CMUCL use a rangelimiting declaration

Regarding the squarero ot If the argument is the square of an integer the result maybeap

parently at the option of the implementor either an integer or a oat Thus sqrt maybe

c or c

Fast FloatingPoint Pro cessing in Common Lisp

That is sqrt the doublefloat d x asserts to the compiler that the

argument is nonnegative and so it can compile it as a call to realdoublesqrt

Tosolve some of the problem Baker identies one can also use overloading as

distinct from p olymorphism and demand that the compiler b e suciently clever

As an example consider this example from Baker

defun asinhz

typecase z

singlefloat log z sqrt z z

doublefloat log z sqrt z z

complex singlefloat log z sqrt z z

etc

where each of the apparently identical arms of the typecase are compiled quite

dierently b ecause the typ e of z diers in each case In the case of inline expansion

the CMUCL compiler can given appropriate declarations removethetyp e

checkaswell as the dead co de from the unused arms of the typecaseSee the

app endix for further discussion

MOTIVATION FOR NUMERICAL LISP

We contend that Lisp features are useful for numerical programs sp ecicallyas

well as programming in general Some of these features although often in an

attenuated form may b e found in C or C and thus our arguments mayprovide

some ammunition as well for arguing in favor of C or other mo dern languages

versus Fortran Nevertheless these particular motivating examples in this section

help supp ort our contention

Scoping information hiding and sharing

Here is an example of a simple but neat use of lexical scoping Groups of Fortran

programs sometimes need private copies of constan ts or need access to shared but

not public workspace The traditional use of common andor data statements in

Fortran are for a constants to b e set at load time or set in one initialization

program and used by others or b shared memory for communication of values

amongst some subset of programs This leads to rather unwieldy and errorprone

programs Anychanges in common statements in subprograms must b e co ordinated

it can also b e insecure since unintentional changes of shared values may o ccur

through aliasing or errors

So far as we can tell the use of p ositional information in common and data

statements is just a regrettable design The most nearly equivalent option in Lisp

would b e to use the package system whichprovides separate name spaces but

allows access to symb ols in dierentpackages bythe packagesymbol syntax

But there is a much stronger informationhiding mechanism in Lisp It allows

denitions of variables in shared but restricted scop e by using lexical closures

let work makearray elementtype doublefloat

initialelement d

the array work is known only to f g and h defun f x use work

R J Fateman et al

defun g y z use work

defun h x use work

This technique can b e used to share data among a selected family of functions while

protecting it from any others It provides a level of information hiding that can b e

used also for applications such as seedsetting with random numb er generators

let seed

defun initializeseed r setf seed r

defun randomgenerator

compute a random number as a function of the seed

Note that any program can call randomgenerator but only it and initializeseed

can directly aect seed

System to ols

Most mo dern CL systems provide systembuilding to ols so that whole new packages

or even sublanguages might b e written in Lisp Examples of interactive application

languages built on Lisp dialects include Auto cad Derive Emacs Interleaf Jacal

Macsyma and variations of it Maxima P aramax Reduce Senac and Axiom

To ols for tokenscannning and language parsing are easily at hand temp orarily

mo difying the Lisp readtable so that individual characters or groups of them take

on sp ecial signicance assists intelligent parsing for new user languages For

example the concatenation of characters to form an identier token or the in

terpretation of digits commas or other characters within numb ers can b e easily

changed eg x could b e parsed as times x

The CL package mechanism assists in avoiding inadvertent nameclashes that

might develop from multiple indep endent programmers dierent system versions

co existing or imp orting from multiple libraries For example one could nd names

like solvebandedlinearequation in b oth NAG and LAPACK packages in the

same CL image They migh t refer to each other if necessary using prexes nag

and lapack The programmer do es not ordinarily need prexes within a single

package except when symb ols are imp orted or exp orted

Most implementations include other although not standardized to ols for prol

ing debugging and interfaces to functions written in other languages There are

numerous graphics packages written for access from Common Lisp several vendors

have standardized on CLIM Common Lisp Interface Manager for user interface

co de

Parameters optional and named

In a Fortran program wemightseea PARAMETER statement for setting sizes of arrays

and other variables

PARAMETERAd N M

Changes to the values of these variables requires editing source co de While Lisp

has a defparameter function for such uses it is our feeling that Fortrans parameter

statements are more often used for assigning values to extra subroutine param

eters In Lisp suchvalues can b e directly indicated passed to a function through

optional or keyword arguments whichmaybegiven defaults at the time a function

Fast FloatingPoint Pro cessing in Common Lisp

is dened Then if the default setting needs to b e overridden the parameter value

can b e sp ecied as an option available with each call to the function In the exam

ple b elow the function f uses the defaults on the rst call but changes the values

of a and m on the second

defun f x y key a d n m

f

f a d m

Functional programming supp ort

A collection of functions that op erate using functions as data provide an alterna

tive and often very simple view of complex op erations As a simple example if v is

avector of numb ers then to pro duce a new vector of the absolute values of those

numbers typ e map vector abs v



To pro duce a new vector of eachvalue v just a bit more programming is

i

needed We can dene a function CubePlus and then map that over v

defun CubePlus x x

map vector singlefloat CubePlus v

The next statement p erforms the op eration in one gulp using a lamb da expres

sion The advantage to the programmer one which only b ecomes apparentafter

some exp erience is that no new onetimeuse function name symb ol need b e in

tro duced These lamb daexpressions can b e written as any other kind of program

comp onent o ccasionally rarely it is useful though quite p ossible to build suc h

an anonymous lamb daexpression at run time and even compile it into ecient

co de Lisps inherentsymb olic manipulation facilities can treat data as programs

and vice versa

map lambdax x v

Syntax macroexpansion expressionalit y

Advo cates of Lisp often hear complaints by nonLisp programmers that the syntax

is poor Many of the complaints are probably based on a supp osition that human

Lisp programmers are forced to count parentheses to make them balance In fact

computers do an excellent job of this They also are excellent at indenting programs

to show structure whether the language is C Fortran or Lisp One rapidly b ecomes

accustomed to seeing parentheses rather than the b eginend or doo d pairs of

wordier languages

Heres an extract of Fortran from Numerical Recipes p computing an

approximation to a Bessel function

DATA QQQQQQQQQDD

DDDD

DDD

BESSIEXPAXSQRTAXQYQYQYQ

YQYQYQYQYQ

The assignment statement has pairs of parentheses and do es not makeeasy

reading If there were an error in it would you notice

R J Fateman et al

It fails to b e expressive b ecause although the values of Q are initialized in a

data statement it is not clear that they are constants They might b e initial

values of variables If the compiler knew they were never going to b e changed

it mightmake some simplications Perhaps if Q were zero it could even save

an addition Furthermore the Fortran fails to express the intent accurately the

constants app ear to b e double precision for example Q is d and yet

Q is presumably accurate to only decimal places We exp ect that Q is declared

to b e double precision solely to promote the use of doubleprecision arithmetic later

on

Here is the same assignment statement in Lisp It has pairs of parentheses

Which is easier to read Which has its structure displayed more clearly You are

entitled to your opinion but keep reading please

setf

bessi

exp ax sqrt ax

q

y

q

y

q

y

q

y

q

y

q

y

q

y

q y q

Such indentations and parenthesis counting is almost always done by edit pro

grams not by programmers But consider another version

setf

bessi

exp ax sqrt ax

polyeval y

d d d d

d d d d

d

This is m uch simpler to view and wehave not necessarily committed ourselves to

a particular wayofevaluating the given constantco ecient p olynomial We could

dene polyeval by macroexpansion to generate exactly the same form Horners

rule as ab ove

To illustrate the virtuosity of Lisp here is one such denition of polyeval

Fast FloatingPoint Pro cessing in Common Lisp

lines one of which is a comment

defmacro polyeval y l

Expand out Horners Rule for py a ay

where la a

if endp rest l first l

y polyeval y rest l first l

The numerical language programmer writing in Lisp need not b e concerned with

the details of such a macro expansion

In fact a professional version of this might b e more elab orate although just



as easy to use it would put typ e declarations in all appropriate places make

conversions at compiletime from rationals or integers to single or doubleoats as

appropriate checkforvarious erroneous uses and simplify away additions of It

also assures that y is evaluated only once It could even b e used as a way of generat

ing machine co de in some highly optimized fashion taking into accountoverlapping

of multiplication addition and loading of oating p oint registers or generating a

call to a p olynomialevaluation instruction available on some computers suchas

the DEC VAX

But p erhaps we could do something b etter In fact by not committing to a partic

ular implementation of polyeval for all machines wemightbeabletosave time

on some architectures For some newer architectures reducing the number of mul

tiplicationaddition op erations even by a factor of two may not b e as imp ortant

as computing the b est ordering of op erations for the overlap of memory accesses

with registertoregister instructions Therefore implementations of polyeval may



dier for Lisps in tended for sucharchitectures

On another tack we can take the same higherlevel syntax of polyeval and

consider one of a numb er of dierent techniques for p olynomial evaluation that are



theoretically and practically faster than Horners rule



Implementations of Common Lisp vary signicantly in their ability to infer typ es from declara

tions Some are fairly clever CMUCL and others Kyoto CL require declarations of virtually

every o ccurrence of a symb ol A useful feature in some compilers is an explanation facility where

the compilers inability to determine a sp ecic typ e results in a warning message suggesting that

more ecient co de could b e generated if the programmer were to know enough ab out the typ es

of arguments to insert a declaration At least one Lisp system CMUCL takes the view that

compilation with its static analysis of the program provides b etter debugging information than

their interpreter Other implementations place a heavier debugging burden on the interpreter

while assuming the compiler is used only on co de that is correct These approaches are not mu

tually exclusive several systems supp ort variants of compilers andor interpreters with dierent

ob jectives controlled by optimization directives



In one test on one compiler we found that writing out Horners rule as a lo op stepping through an

arraywas faster than the inline expansion for a p olynomial of degree This was counter

intuitive so we explored further By reversing the order of arguments to the in the macro

we reduced the time for the inline expansion by more than a factor of two The explanation is

actually simple the Lisp compiler was evaluating and saving the arguments to lefttoright

and then adding them Thus wehave a heuristic for complicated expressions in the case of

commutative arguments to and it is preferable to place the most elab orate argumentto

compile on the left



Although Horners rule is b oth admirably stable numerically and in one sense optimal in that

it uses the fewest numb er of op erations if q through q are not known until runtime it do es

NOT use the fewest numb er of op erations if a preconditioning computation with the co ecients

R J Fateman et al

The smallest case in which this matters is the general th degree p olynomial

which can b e evaluated with rather than multiplications

In fact we can expand the example p olynomial from bessi ab ove to the fol

lowing Lisp program which uses only multiplications and adds instead of the



exp ected each

polyeval x d d d d

d d d d

d

expand to

let z x d x d

w x d z d

x x w d w z

d

d

d

d

Handling p olynomialsofspeciclow orders as well as the general nth order

p olynomial can also b e programmed as part of an elab orate compiletime expansion

of polyeval Preconditioned forms can magnify oatingp oint roundo errors

by cancellations although even Horners rule can have bad evaluation p oints To

b e esp ecially cautious one could attempt a careful analysis of the range for the

indep endentvariable and the formulas numerical accuracy In this case knowing

that this p olynomial is used only for jxj allows us to b ound the maximum

dierence b etween the evaluations of the two expressions or the expression and

the exact inniteprecision answer If this particular preconditioned expression

is unstable there are other choices for preconditioning co ecients with dierent

prop erties There are alternativetechniques known to evaluate a p olynomial with

optimal accuracy eg Meszteny and Witsgall involving a kind of T aylor

series expansion of the p olynomial ab out a zero near the interval of interest

While the preconditioned formula app ears in fact to b e faster on our Sun Mi

crosystems Sparc computer as p ointed out previously there are considerations other

than the countofmultiplications that are relevant in minimizing the cost of evalu

ating common relatively low degree p olynomials As one example if you havetwo

 

indep endent units to evaluate p olynomials decomp osing pxinto p x x p x



maysave ab out half the computing time if you are just counting multipli

cations and additions

In an environment devoted to generating and making available topqualitysoft

ware implementation of alternative ideas like polyeval or stable evaluation

could reside in a symb olic computation optimization kernel indep endentofthe

is p ossible In general ab out n multiplications and ab out n additions suce see Knuth

section for a general discussion



Computing the preconditioned co ecients used in the macroexpansion can require substantial

arithmetic here weusedthenumerical solution of a cubic equation

Fast FloatingPoint Pro cessing in Common Lisp

language This would have a complementary role to play along with more conven

tional numerical libraries

Other Lisp features

Although we will not dwell on these features we should mention a numb er of other

asp ects of the language

There are substantial libraries of often free software for sophisticated manipu

lation of text data programs These include packages for graphics statistics im

age understanding symb olic mathematics combinatorics parsing of computer lan

guages natural language parsing knowledge representation exp ert system shells

etc

The CL Ob ject System CLOS is quite p owerful Adding to the core facili

ties of the language using this extension facility can b e relatively convenientand

straightforward

There are numerous builtin function capabilities dealing with numeric and non

numeric typ es p erhaps worthy of sp ecial note in the context of oating p oint

arithmetic is the provision for handling traps and other nonstandard owofcontrol

constructs

Strengthening the p osition of the language as a vehicle for p ortable programming

is the detailed denition and standardization that is well under way

SYSTEM ARCHITECTURE AND NUMBER REPRESENTATION IN LISP

In this section we consider some of the deep er issues concerning Lisp compilation

and how the sp eed of compiled co de for numeric problems might b e improved

Pointers and numb er values

In order to b e consistent with Lisps usual memory mo del on a sto ckhardware



computer it is often necessary to manipulate p ointers ie addresses of ob jects

rather than ob jects or values themselves That is setf x d stores in the

v alue cell of the symbol xapointer to a lo cation often referred to in Lisp

literature as a b ox in memory where bits of is stored as a oatingp oint

number Why the indirection The generality of the concept of value in Lisp

requires that xs value could b e an ob ject of any size a singleprecision oat an

array a structure etc This is usually handled by pointing to the real value from

the value cell of the symbol x As usually implemented the p ointer provides some

space in the high or low bits of the address to mark what kind of ob ject is b eing

addressed This typ e information is vital to maintain Lisps memory consistency

ho w else to trace through memory cells in use One must b e able to distinguish

between a p ointer whichmust b e followed during garbage collection in order to

mark other accessible data and integer or a oat in which case the buck stops

here It is also useful for error checking based on typ es

But what if we know that xs value is constrained to b e a doubleoat For

eciencys sake esp ecially if we are comp eting with Fortran which has no such

overhead wemust try to take advantage of this knowledge and eliminate the

indirectness of p ointers



As opp osed to sp ecial Lisp machines with tagged architectures

R J Fateman et al



Integers The rst way out of this indirectness is to establish a set of bit

values that could not p ossibly b e interpreted as addresses themselves These bit

patterns then can b e used to enco de small integers For example if p ointers must

always p oint to doubleword bit b oundaries we are using only the bit patterns

with loworder s Alternatively p erhaps addresses must b e nonnegative so we

have all negativeintegers to use Such an enco ding leaves us with a generally low

cost implementation of immediate numb erlike ob jects sometimes called INOBS

array indexes counters and runofthemill xedprecision small integers These

are of subtyp e fixnum in Common Lisp Typically these can accommo date values

n n

that are in the range to where n in a bit implementation is likely

to b e or The remaining few bits are generally used for some kind of xnum

tag For ob jects to b e fixnum typ e they must b e declared to the Lisp compiler

since an undeclared integer value could b e outside this range Note that the

Common Lisp standard requires an arbitraryprecision range for integers

The p oint here is that declaring integers as fixnums can cut down on the overhead

of p ointers and b oxes for storing numb ers given a suitable compiler to p erform that

elimination Some systems have full bit unb o xed integers available as well

the advantage here is unrestricted direct access to the integer arithmetic on the

host machine without ddling with high or low order bits

Arrays and oats unboxed The second imp ortanttyp e of declaration for

Fortranlike co de is that of oatingp ointarrays Common Lisp implementations

mayhave several oatingp ointtyp es but for concreteness lets use doublefloat

Most implementations treat suchanumb er as a bit quantity A element

singleindexed array of doubleoats D can b e allo cated by setf D makearray

elementtype doublefloat and programs using D but compiled out of

view of the allo cation can contain the declaration declare type simplearray

doublefloat D This annotation if prop erly treated by the compiler in

dicates that eachentry of D can b e treated as a doubleoat Not as a p ointer to

a doubleoat but the bit quantity itself Such an arrayissaidtocontain un

boxed oats A clever enough compiler can therefore generate co de to add two

elements in D and store in a third element with the same eciency as in Fortran

A ma jor gross transformation of anyFortranstyle oatingp oint arithmetic to

ecient Lisp representations is therefore p ossible if we are willing to forego the

usefulness of individual oating p oint names like x and treat all op erations as di

rect manipulations on elements in some array of oatingp ointnumb ers That is

instead of writing Lisps version of the Fortran X YZ as setf x y z we

must do this by array references In rawLispthiswould lo ok like setf aref

d i aref d jaref d k where the indexes fi j kg denote the lo ca

tions of fx y zg resp ectively in the array dwhich is itself a kind of alias for

all memory used to store doubleprecision numb er This array reference alias

business should certainly b e hidden from the ordinary programmer who should

b e able by appropriate mappings easily accomplished in Lisp to write setf x

y z or even X YZ for that matter It could b e incorp orated into one of

the optimization stages of fcl describ ed b elow and could b e carried through to



or whatever the basic address space dictates



One might more correctly view the situation as having the whole array protected from the

attentions of the garbage collector by b eing stored all in one b ox

Fast FloatingPoint Pro cessing in Common Lisp

include the transmission of oating p ointorinteger arguments to subroutines etc

As a target for the compiler we could map the users program into a severely

overdeclared program as illustrated by the fragments

declare optimize speed safety

type simplearray doublefloat d

type fixnum i j k

setf aref n i

the doublefloat

the doublefloat aref d j

the doublefloat aref d k

Because it would substantially alter the readabilityofournumerical programs

we have not used the technique outlinedabove in the rest of this paper It could

however b e used for situations in whicha human never lo oks at the co de again

A more subtle wayofachieving some of the same eect for individual variables

is for the compiler and runtime system to allow for some storage for unboxed lo cal

variables in a section of the stackframe for a function That is the stacklocal

variables area is divided for purp ose of the garbage collector into two sections

one which the gc lo oks at and one which it do esnt Unboxed oats are placed into

the latter Some compilers implement this rightnow See also the discussion in

section b elow ab out function calls

Lo ops

One compiler for our Sparc machine typically uses instructions to get aref d j

in a oatingp oint register the whole op erational part of the expression ab oveisof

these sequences a oatingp ointmultiplication and another instructions to store

in the right place A lo op to compute a vector crosspro duct adds instructions

a test an incrementaloopback A total of instructions By a minor hackin

index calculations it app ears to b e

This is neither the shortest nor the fastest sequence there are substan tial sub

tleties in ordering of op eration co des to allow for maximum overlap b est use of

caches etc The particular implementation of the SPARCarchitecture is also rel

evant However the co de pro duced by the Lisp compiler is not outrightstupid

it is shorter and probably faster than the op erations including one nop in the

main lo op pro duced by the standard Fortran compiler f from this co de

DIMENSION U V W

DO I

UIVIWI

END

But the Lisp co de at least from Allegro CL is slightly longer than the

op erations obtained from the optimized f fast option

By hand co ding we can remove one more instruction whether this runs faster or

not and whether it should b e unrolled for sp eed dep ends on the hardware in

ways we are not prepared to explain here

R J Fateman et al

The Lisp compiler generating the co de do es not use sophisticated techniques for

oatingp oint and in fact do es no common sub expressions elimination But it is not

a bad base to build up on and it is plausible that we can by agreement on particular

restricted typ es of sourceco de idioms get the compiler writers to pro duce excellent

co de for those sequences

Because lo ops including our example ab ove generally involvearray accessing

where the index dep ends on the lo op variable this discussion continues in section

on arrays indexing

Function Calls

When one Lisp function calls another they can agree in principle on the typ e of

the arguments and the return value They can then agree to leavethemunboxed

By passing arrays of known typ e in to receive results we are implementing this in a

manner that requires less elab oration on the part of the compiler writer And it is

agenerallysticky p oint to compile appropriate interfaces to functions in Lisp Lisp

is to o exible Given functions foo and bar where foo calls bar the user is free to

redene either foo or bar on the yThus it is hard to force the consistency b etween

formal and actual arguments or return values via compiler checks it is imp ossible

for the compiler to know without some explicit declaration how to compile the

arguments For example if one denes bar to accept two arguments an integer

and a doubleoat the compiler do esnt necessarily know whether to receivethe

oat argumentasunboxed or not In particular foo maynotyet hav e b een seen

by the compiler or for that matter foo may not yet have b een written

Such closely linked calls can b e handled indeed the Allegro compiler al

lows through suitable prior declaration passing unboxed values directly to foreign

functions The CMUCL compiler allows blo ck compilation and sp ecial fast calls

to internal programs dened in a labels list The diculty is cultural it is an

unusual burden to place on Lisp programmers that they must declare the external

interface to a program prior to the compilation of any program that uses it Of

course many other programming languages including Pascal Mo dula and Ada

require this

A simple mo del forarithmetic computation

What then should our idiomatic Lisp arithmetic source co de lo ok like Ideally it

would provide an intermediate form suitable for expressing all the mathematical

computation but might also include some level of hints as a concession to the

Lisp compiler writer who might not wish to sp end time analyzing oatingp oint

co de Suchhints have a mixed history early C implem entations used register

declarations mo dern implementations tend to ignore them on the grounds that the

compilers can generally make b etter choices than the programmer in such matters

Ideally we could design such a source co de level that is indep endent of the target

architecture and can run in a numb er of Lisp compilers Wemust then make

provision for the mapping from the users lispfortranish into the primitive forms

In Lisp this would ordinarily b e done by macroexpansion optionally followed by

compilation

For exp osition purp oses we will make some simplifying assumptions whichcan

via calldirect

Fast FloatingPoint Pro cessing in Common Lisp

however b e removed without conceptual dicultyWe assume only one oating

p ointnumeric format or more are p ossible in CL We will assume real as

opp osed to complex arithmetic is of primary interest We will assume that ex

act arbitraryprecision integer arithmetic and ratios exact fractions need not b e

handled esp ecially rapidlyWe will assume that subscript calculations lo op indices

and decisionmaking can b e done with the fixnum typ e

Two mo dels emerge from machine architecture considerations Machines with

oatingp oint stackmodelsliketheIntel and its successors provide one mo del

Machines with a register orientation like the SPARC have another These mo dels

are computationally equivalent but at the level of ecientmachine instruction

issue it mightpay to address each separately

Any mo del should ultimately pro duce optimal co de on a few oatingp ointcon

structions such as a simple assignment

setf y x

avector crosspro duct

dotimes i k setf aref u i aref v i aref w i

and a p olynomial evaluation

RPN arithmetic and a oating p oint stack

Generating Reverse Polish Notation RPN for computation with a stackiswell

known and may b e most appropriate if we consider that there is a copro cessor

which implements the following op erations lo ok at the Intel copro cessor

architecture for example

push n push arrayn on stack

pop n p op and store top in arrayn

jump fgtgeltleeqne notcomparableg

swap swap top two items on stack

plus changesign subtract reversesubtract times invert divide

reversedivide square squareroot sin cos exp log

In the absence of any other eorts toward oatingp oint eciencyeach of these

could b e implemented as a compilermacro in the LISP compiler in such a form as

to emit no more co de than necessary

A function which computes yx for oating p oint x could b e written in CL

where n n and n were appropriate indexes as push n push n push

n times plus Representations for x and for the lo cation of the result on

the top of the oatingp oint stack must b e taken into account at the transition

p oints b etween normal Lisp and numericallyoriented Lisp More sophisticated

translation can easily b e constructed using techniques in many texts eg on

programming language compilation

Register or other mo dels

In fact one could argue that the Lisp compiler writers could very well directly

compute a translation from delimited prex arithmetic notation that is normal

Lisp to a conventional nregister mo del and thus the intermediate description in

the previous section is not necessary or even desirable

R J Fateman et al

Some compilers emit Clanguage co de or some kind of byteco de which is halfway

compiled This somewhat b egs the question since the eciency of either technique

dep ends signicantly on other software

Note that compilers may already make clever use of registers the Allegro CL

compiler with appropriate requests for high optimization can already allo cate

a doubleoat solely in a register when it is a dolo op index as is acbelow

defun scalarprod x y k

doublefloat scalar product of the klength doublefloat

vectors x and y

declare optimize speed safety debug

type simplearray doublefloat x y

type fixnum k

do i i

ac d ac aref x iaref y i

i k ac

declare fixnum i doublefloat ac empty dobody

In other circumstances or for other compilers it may not b e obvious that a

variable is eligible to b e in eect a synonym for a register and so a declaration

declare doublefloatregister acc might b e helpful

An implementation technique for closed numeric subroutines suchascos is for the

Lisp system to provide two or usually more entry p oints A highsp eed routine is

central to the approach this entry p ointisforrealdoublecosunboxed to com

pute the unboxed real doubleoat result of the cosine of an unboxed doubleoat

The result might b e left in a known oatingp oint register The generic slow entry

p ointwould b e used by the interpreter or from compiled co de where the argument

typ e must b e determined by examining the datum itself and the result must b e

put in a b ox This routine could call among others realdoublecosunboxed

Other variations would include assuming a b oxed argumentbutleaving the result

in a register or taking the argument from a register but b oxing the result

Given the p ossibility of either the register or the RPN mo del w e exp ect that

compiler writers havetheirown prejudices based on the technology they are al

ready using and its ability to supp ort other optimizations such as array addressing

in the next section An imp ortant p ointwe wish to emphasize is the use of higher

level targets lik ethe polyeval macro illustrated earlier a scalarpro duct array

primitive see b elow or canned BLAS which form the core building blo cks for

p ortable highsp eed reliable linear algebra programs By appropriately redening

the BLAS in machinedep endent fashion a large p ercentage of the vector and par

allel sp eedups p ossible in parallel machines can b e exploited while still using the

same toplevel subprograms The machine indep endent BLAS are available in

Fortran and some vendors have reprogrammed them in assembly language but

this do es not mean that they must b e called from Fortran indeed they could b e

called from Lisp

The rstlevel BLAS dene vector op erations suchascomputingx y for scalar

but vectors x and y These do not provide high leverage but only convenience

they do only a linear in the vector size amountofwork

The secondlevel BLAS are used for matrixvector op erations suchasAx y

 

for a matrix AThislevel p erforms O n op erations on O n datasothaton

Fast FloatingPoint Pro cessing in Common Lisp

some pro cessors ma jor sp eedups are limited by the rate of data movement On

some other vector pro cessors nearp eak p erformance can b e obtained

The thirdlevel BLAS are used for matrixmatrix op erations This level provides



ma jor scop e for improvements in sp eed b ecause they p erform O n op erations on



O n data A typical computation would b e AB C

Co ding in terms of these routines is a ma jor step in eliminating dierences in

eciency amongst Fortran Lisp C or other systems

Note Since Fortran uses callbyreference it is p ossible to identify a location in

an arrayby Ak a BLAS routine can then copy the next n elements to another

array Using Lisps callbyvalue conventions only the value of an array elementis

conveyed by aref A k and therefore one of several lessdirect schemes must

b e used to notate sub comp onents of arrays

HOW GOOD ARE MODERN LISP COMPILERS

Manytalented p eople often academics have lab ored over the problem of compiling

Lisp into fastrunning co de Since traditional Lisp source co de tends to consist of

many relatively small indep endentlydened functions whose relationships to each

other and to global names can b e changed some optimizations are dicult or im

p ossible to p erform without violating some seman tic requirements Considerable

eort has b een exp ended in some compilers to nd appropriate optimizations and

compromises with semantics Typically you cant change the denition for func

tion f or the typ edeclaration for variable v without recompiling all optimized

functions referring to f and v in the same compile blo ck

Fortunately the situation is somewhat dierentifwe are not compiling tradi

tional Lisp source co de but Lisp programs written in a Fortran traditional style

to perform numerical computation Judging from existing Fortran programs we

susp ect that they will consist of longer stretches of co de dealing primarily with

primitive data typ es like oats integers and arrays of oats without intervening

functioncalls The typ es of all data ob jects are xed at compile time Control

structures are simple While historically it has not b een p ossible in most systems

to express such simple co de in its stark simplicity within the Lisp data represen

tations and op erations Common Lisp has changed this substantially byproviding

b etter expressibility through declarations simpler co de can b e generated

Can we build compilers that generate appropriate optimized co de for these sim

ple programs

We b elieve the answer is yes Many of the lo op optimizing and rewriting tech

niques develop ed for other languages can b e applied to Lisp Ironically Lisp is an

esp ecially wellsuited language for developing and sp ecifying such to ols an inter

mediate form for a Fortran program as it is compiled is likelytolookvery much like

Lisp Lo op optimizations unrolling and other transformations can often

b e expressed nicely in Lisp sub ject to a reasonable underlying mo del of the archi

tecture knowing how far to unroll a lo op or howmanyaccumulator registers are

available or howmuch instruction overlap is p ossible dep ends on the architecture

Then the question b ecomes one of economics Is it worth implementing the to ols

in Lisp to p erform the necessary optimizations

This is the big stickingp oint as we see it Programmers have in the past viewed

Lisp as a notforfastcomputation language And b ecause users view it as such Lisp

compilerwriters dont tend to see the p otential market in this area And b ecause

R J Fateman et al

they dont concentrate in this area b ecause there is no market the Lisp compilers

tend to stay b ehind the times with resp ect to other languages thus fullling the

prophecy

Even so experiments with some commercial compilers suggest that the existing

compilers arepretty good

Our goal in this section of this pap er is an existence pro of of sorts From that it

could b e argued that there may b e a p otential market for more highlyoptimizednu

meric computation expressedinLisp and that Lisp compilerwriters should satisfy

this market

To illustrate our p oints ab out expressibilityweprovide a set of three graduated

examples of Lisp co de that webelieve represent subsets of the kind of computation

typicalofmostlynumeric programs in Fortran

Clearing an array

Array accessing and element setting is a key ingredientinmostnumerical co de

An inconvenient dierence in convention is the numb ering of arrays and matrices

in dierent languages and asso ciated mathematical conventions Fortran bycon

vention numb ers arrays starting with CL byconvention b egins with Mapping

numerical algorithms phrased in terms of Fortran conventions into Lisp can either

b e done byhuman programmers in the transcription pro cess or by some automated

transformation from a FortraninLisp notation to a Normal Lisp notation

In a dimensional arrayofsizem by n using Lisps zerobased array the entries

are laid out in memory in rowma jor order

a a an

a a an

am amn

So if r is the address of a and the stride of the arrayiss for example bytes

if the array holds doubleoats then ai j is stored at lo cation r s i n j

Fortrans based array has the entries laid out this way for an m by n array

a a am

a a am

an amn

So if r is the address of a and the stride of the arrayiss then ai j is stored

at lo cation r s j m i which can b e rewritten as r s j m

i sm So an equivalent computation to zerobased addressing is to cho ose

r r s m and use based addressing

Actually the stride can b e separated out from the indexing Then wecansay

that the Fortran ai j in a Lisp array a is aref a j m i m

Let us start with a Fortranbased lo op that lo oks like

DO IM

DO JN

AI J D

We insist that there b e no branches to statement and all access is from coming

through the top the compiler terminology is that this is a basic block Pre

Fast FloatingPoint Pro cessing in Common Lisp

sumably the Fortran language standard allows jumps into this lo op p erhaps after

resetting i and j but it would certainly b e p o or programming practice to write a

program to do so There are in any case techniques for determining basic blo cks

and we could insist that prior to our optimization a determination b e made that

this is a basic blo ck

We are going to optimize this lo op entirely in p ortable Common Lisp

First note that it could b e translated into a Lisp lo op

dotimes i m dotimes j n setf aref a i j d

but the aij indices would b e o and p erhaps confusingly so In particular for

Lisp wehave zero ed out a through am nFor Fortran wehave set

A through Amn If the b o dy of the Fortran lo op were AIJIJ

the conversion to Lisp would have tolooklike setf aref i j i j

We could do this but if we are targeting the conversion of Fortran to Lisp

by automatic means we could alternatively adopt the Fortran convention at no

particular extra cost and we gain consistency with other uses of subscripts in

the Fortran co de So wehyp othesize some new constructions fordotimes and

foraref in Lisp that are consistent with Fortran in this way

fordotimes i m

fordotimes j n

setf foraref a i j n d

This inner lo op computation is clumsy b ecause inside the array reference we

will b e rep eatedly computing the index i n j n How can we

sp eed this up There are standard ideas from compiler optimization that can

b e applied For a starter lets precompute the invariantquantity mnp n

Then if we treat a as D vector rather than D array the index simplies to

i n j mnp and the lo op lo oks likethis

prog

mnp n

fordotimes i m

fordotimes j n

setf aref a i n j mnp d

It is advantageous to removeinteger multiplications from an inner lo op In fact

the SPARC has no integer multiplication instruction so we consider this

prog

k n

fordotimes i m

setf k k n add here instead of multiplying below

fordotimes j n

setf aref a k j d

Next we replace the fordotimes with its macroexpanded denition to get the

program segment

prog

k n do i i

R J Fateman et al

i m nil

setf k k n

do j j

j n nil

setf aref k j d

and nally we put the pieces together inserting declarations The function cleara

imp osing a D access mo del on a D array can b e compared with the b est direct

D arrayversion whichweshow as the function clearaa below

The times on a SPARC in Allegro to clear a by array are

ms by the D metho d in the normal D co de The time for the equivalent

unoptimized Fortran co de is ms but with f fast set it is ms slightly

b etter than Lisp

defun cleara a m n

simulated D array accessing using D

declare type simplearray doublefloat a

type fixnum m n

optimize speed safety

prog

k n

declare fixnum k

do i i i m nil declare fixnum i

setf k k n

do j j j n nil declare fixnum j

setf aref a k j d

defun clearaa a m n

normal D array accessing

declare type simplearray doublefloat a

type fixnum m n

optimize speed safety

dotimes i m declare fixnum i

dotimes j n declare type fixnum j

setf aref a i j d

The advantages of referencing a D array as though it were a singleindex vector

has o ccurred to the XJ Common Lisp standardization committee and in fact a



function rowmajoraref has b een prop osed for that purp ose As it happ ens

a D array and a D array are likely to have dierent header structures and an

attempt to lie to the compiler passing a D arra y in but claiming it is D may

lead to unpredictable results

Here is what it lo oks like in Allegro Common Lisp

exclarraybase Allegro Extended Common Lisp

given an array arraybase

returns values

A nonp ortableway to extract the underlying D vector for a D arrayislikely to already

exist in every CL implementation

Fast FloatingPoint Pro cessing in Common Lisp

the data vector holding the arrays elements

the index in this vector of the element representing

aref array

true iff the array is displaced

defun clearav a

use the underlying vector to clear an array a of any dimension

declare optimize speed safety

multiplevaluebind sv ix

exclarraybasea

declare type simplearray doublefloat sv

type fixnum ix

let tot arraytotalsize a

declare type fixnum tot

do k ix the fixnum k

k tot nil

declare type fixnum k

setf aref sv k d

This program is bytes long and compiles to the same inner lo op as cleara

Matrix multiplication

Heres a simple Fortran program to multiply two matrices to get a third

dimension uvwtarr

C INITIALIZE INPUT MATRICES

do i

do j

vij

uij

C START THE TIMER

startdtimetarr

call matmuluvw

C READ THE TIMER

elapseddtimetarr

C PRINT THE TIMER user seconds system seconds

print tarrtarr

formatf

end

subroutine matmulabcnmk

dimension anmbmkcnk

do in

do jk

sum

do lm

sumsumailblj

cijsum end

R J Fateman et al

Here is a Lisp program to p erform the same matrix multiplication op eration

defun matmul a b c n m k

CAB

aisnXmbismXkcisnXk

dotimes i n c

dotimes j k

setf aref c i j

let sum

dotimes l m sum

setf sum sum aref a i laref b l j

If wechange the ab ove program through sourcetosource transformations which

we fully intend to automate to replace D arrays with D vectors and add

declarations we get this

defun matmul a b c n m k

declare optimize speed safety

type simplearray singlefloat a b c

fixnum n m k

let sum

i m

k

declare singlefloat sum fixnum i k

dotimes i n c

declare fixnum i

setf i i m iim

dotimes j k

declare fixnum j

setf sum

setf k k

dotimes l m

declare fixnum l

setf k k k k lk

setf sum sum aref a i l

aref b k j

setf aref c i j sum

How do es this aect the timing and how do es this compare to Fortran Heres

a short table of timings computing the pro duct of two singleoat arrays

on a Sun SPARC We use dierentversions The rst two are run in Allegro

CL The next two columns are the Fortran times for the Sun f in default

mo de f and in the fastest mo de f fast O

If we really want this program to run fast wemust consider taking into account

the sp eed of access to memory in most machines We usually have a memory sp eed

hierarchyfavoring registers then memory lo cations near those that have b een

referenced recently and are therefore in the CPU cache then ordinary physical

memory lo cations then virtual memory store on secondary storage One way

of achieving a higher cache hit ratio is to copy columns from matrix B in to a simple

vector Doing this in Lisp sp eeds up our b enchmark by ab out p ercent

Fast FloatingPoint Pro cessing in Common Lisp

Taking full advantage of lo cality requires substantial eort to tune the program

to the memory characteristics of the architecture and its implementation The nal

column is the time estimated taken by a highly tuned program written byKen

Stanley using a combination of C and assembler This program is based on

partitioning the arrays into submatrices and multiplying these blo cks This

is rather elab orate but worth it if matrix multiplication is the critical op eration

matmul matmul f f fast Cassembler

s s s s s

Other Lisp compilers Lucid CMU CL f pro duce similar times for the

matmul version the newer Allegro CL compiler in b eta test pro duces co de

running at s With some additional suggestions to the compiler writers at

these organizations it is p ossible that they could further narrow the gap with the

best Fortran Note incidentallythatifyou were real ly concerned with sp eed of

multiplication of matrices as part of a larger calculation you might b e b etter o

lo oking at a stateofthe art implementation of the BLAS for your machine

This function is p erhaps atypically small and thus we consider a more compli

cated program the singular value decomp osition example b elow

Before going on to that example though it is a natural question to ask if this

slowdown factor of to using compiled Lisp rather than optimized Fortran is

tolerable There are several ways to answer this

If you generally use Fortran without the optimization you are slower by a factor

of on this problem Did that matter

If you need fast computation why are you using Fortran Stanleys program

takes advantage of the SPARCcache architecture and multiplies matrices at

more than twice the rate of the b est Fortran to date

Singular value decomp osition

The nal extended example of our exp eriments with Fortranlike co de is the singular

value decomp osition algorithm found as Fortran co de in Numerical Recipes

Numerical Recipes is the pro duct of a collab oration among four scientists in

academic research and industry It comes in the form of a text on the art of

scientic computing and numerical metho ds together with the source co de for the

programs describ ed in the text on diskette The text contains a fully annotated

form of each of the programs As well as Fortran there are versions in C and Pascal

Numerical Recipes has found widespread use in science and engineering in spite of

receiving a thoroughly negative reception bythenumerical analysis community

Nevertheless the coverage is useful for simple applications over a wide range of

problems areas The suite of subprograms consists of ab out functions or sub

routines covering the areas solution of linear equations interp olation and extrap o

lation integration of functions evaluation of functions sp ecial functions random

In most cases the programs are not the most sophisticated of their typ e they do not represent

the high level of bulletproong in some libraries nor do they use algorithms that takeadvantage

of subtle features and dierences among arithmetics of various computers NRs advantage of

easy to explain do es not hold muchweight with professional numerical analysts nor should it

You probably wouldnt want to b e a passenger in an airplane whose primary attribute was that

it was easy to explain

R J Fateman et al

numb ers sorting ro ot nding solution to nonlinear sets of equations minimization

and maximization of functions eigensystems Fourier transform sp ectral metho ds

statistics mo deling of data integration of ordinary dierential equations twopoint

b oundary value problems and partial dierential equations

Singular value decomp ositions provide p owerful metho ds for the solutions to sys

tems of equations whichmay b e singular or nearly singular Given a matrix A

whichis m by n representing a system of m equations in n unknowns with m n

the singular value decomp osition of A is the representation

T

A UWV

where U is m by n and column orthogonal W is n by n and diagonal and V is n by

n and orthogonal The columns of U corresp onding to nonzero elements w span

ii

the range of A and rows of V corresp onding to zero w span the null space of A

ii

If each w is b ounded away from zero then the inverse of A may b e represented by

ii

the expression

 T

A Vdiag w U

jj

If A is singular then the solution vector of the equation Ax b of smallest length

is given by

T

x Vdiag u U b

j

where u is w if w is nonzero and zero otherwise if such a solution vector

j jj jj

exists ie if b is in the range of AIfb is not in the range of A then the ab ove

expression minimizes the size of the residual Ax b

The svd function was chosen for this comparison b etween Fortran and Lisp b e

cause of its imp ortance and b ecause it exhibits many of the features of Fortran

programs taken together The Lisp co de contains nested do when and cond state

ments intermixed to depth It also contains go statements and uses nonlo cal do

variables The Fortran co de was lines and the Lisp lines of handtailored

co de Note that fcl describ ed in section pro duces somewhat fewer lines than

this principally b ecause the handco ded version used the Lisp do rather than the

more compact macro fdo

The same test matrix of size by with entries b eing random integers chosen

between and were presented to the Lisp and Fortran versions of the algorithm

The environment for the timings was a Sparc using Allegro CL and the system

provided Fortran compiler The time for the Lisp computation was obtained using

the time function and for Fortran the dtime routine The Allegro CL times

of seconds b eat the f time of seconds setting on optimization for f

brought its time down to seconds

Thus for this system the compiled co de can have quite comparable sp eed to

that of the corresp onding unoptimized Fortran in this case as well We exp ect that

substantial sourcetosource co de improvement is p ossible in the Lisp gures ab ove

simply byintelligently displacing the arrays in use rather than micro translating

each index op eration Further improvements on compiling Lisp can also b e exp ected

as the Common Lisp technology continues to mature and thus we are hesitantabout

running extensivebenchmarks Although improved Fortran compilers can also b e

exp ected they tend to b e at the moment more mature and p erhaps less likely to

b e improved so dramaticallyWepro vide some notes on this in the next section

Fast FloatingPoint Pro cessing in Common Lisp

Further notes on compiling

Some architectures have a storewithup date or loadwithup date instruction that

can assist further in stepping through an array although it seems likely to us

that the particular compiler would havetomake optimizations at that level The

RS C compiler among others do es b oth the lo op strengthreduction chang

ing multiplications to additions as indicated ab ove as well as using loadstore with

up date

Our conclusions on array accessing based on our measurements ab ove as well

as b olstered by times given for matrix multiplication are these

a Allegro CL do es inadequate optimization of D array indexing Probably

some other systems do worse some eg Lucid do b etter

b By an automatic transformation all D arrays of the Fortran variety can b e

addressed as D arrays of the Lisp varietyFurthermore Lisp programs can

b e written to make these transformations as sourcetosource changes This

improves Allegros p erformance but do es not help Lucids

c We probably have to demand that a Lisp compiler provide for ecientD

addressing at stride sizes suitable for oatingp oint single and doubleprecision

and p erhaps their complex counterparts

Although one might b e tempted to optimize by microadvising on stride in

dexing by explicitly adding byte counts to index variables one would b e delving

rather far b elow the level of p ortability This do es not lo ok advisable to us

A FORTRAN TO LISP TRANSLATION TOOL

Motivation

A rapid prototyping to ol fcl was written in Lisp to assist in the developmentof

aformofNumerical Re cipes in Lisp Lispack as a prelude to its inclusion in a

numerical library for Senac Senpack Fcl pro duces an automatic derivation

of an equivalent prototypeLispcodeforaFortran program We then handco ded

such enhancements as the separations of input and output parameters elimination

of unnecessary parameters such as array dimensions which are carried along with

Lisp arrays and therefore do not need to b e passed in as separate parameters and

other improvements We outline the design of this to ol in its currentprototyp e

state in this section Weexpecttocontinue to rene the to ol

The principle reason for writing fcl is to demonstrate that with reasonable

ease wecanmake eective use of algorithmic content of the large b o dy of existing

numeric co de mostly written in Fortran within Lispbased environments This

should weaken one argumentinfavor of continuing to use Fortran regardless of its

suitability for new co de that there is to o high a value in the large b o dy of existing

co de already written in Fortran to switch

Because the range of micro features of Lisp is generally a sup erset of those in

Fortran the mapping of Fortran to Lisp is in many resp ects straightforward We

intend to extend the fcl environment to include a rst stage based on a translation

of the PFORT verier to ensure that standard Fortran is b eing handed to

the translator a stage where intensive Lisp declarations are intro duced and an

optimization phase where the program is restructured to improve p erformance as

outlined in previous sections Thus weprovide from a single Fortran source program

R J Fateman et al

several Lisp versions ranging from the simple and obvious primary translation to

a highlymo died version for highest sp eed

The translator outlined

The following Fortran statementtyp es are translated declaration function

and subroutine sp ecications parameter common data implicit goto if if

then if then else do continue computed goto arithmetic assignments ar

ray references call of subroutines and functions and return statements Because

numerical library co des are of the highest imp ortance to us we leave for the o cca

sional handtranslation the few inputoutput control characterstring and format

statements that are needed In a welldesigned library these usually o ccur only

in the driver test and errorrep orting programs rather than the computational

subroutines

Integers and single and double precision numb ers are not altered The basic

program logic is not altered The translation takes place in the main Fortran

statementby statement

REALDOUBLE PRECISIONINTEGERCOMPLEXARRAY IDENTIFIER X

singlefloat doublefloat fixnum complex array symbol x

Assignments and algebraic expressions are translated in the obvious way making

inx to prex transformations If the typ es of the left and right hand sides of an

assignment dier a conversion is required but in the usual case one sees something

lik e this

A A BC

setf a a b c

Functions and subroutines in Fortran are translated in a programlogicpreserving

manner to functions dened using defun in Lisp

REALDOUBLE PRECISIONINTEGER FUNCTION FX defun f x

SUBROUTINE SA B C defun s a b c

All function arguments from the original co de are included even though some

of these are unnecessary For example the subroutine SUBX N where N is

the dimension of the array X could translate to defun sub x setq n

arraydimension x This simplication has generally b een done in

the case of Lispack However where families of Fortran subroutines call and are

called byeach other in complex ways we retained the arguments exactly as in

the Fortran co de to ensure compatibilitybetween the corresp onding Lisp programs

without requiring sp ecial user intervention or revision of existing do cumentation

Functions return an equivalentvalue but subroutines whichchange the values of

their input parameters must b e handled dierently from the usual Lisp conventions

Calls to subroutines are therefore translated into a form where all arguments are



returned using the Lisp multiplevalue return

FUNCTION FX return f

SUBROUTINE SA B C return values a b c



In Lispackwe mo died the co de to separate the conceptually distinct input and output argu

ments Only the input arguments o ccur in the Lisp function formal parameter list and only the

output values in the return list Inputoutput parameters o ccur in b oth places

Fast FloatingPoint Pro cessing in Common Lisp

The scop e of variables in the Lisp target co de is that of a normal Lisp function

ie variables are lexically scop ed The b o dy of the function is a Lisp prog or prog

We found parameter statements in Numerical Recipes used primarily to set de

faults We b elieve these are more appropriately dealt with by optional keyword

parameters in Lisp Note in the example b elow that the constant one is probably

dierent in concept from epsnevertheless in Lispack these would b e included as

lo cal variables with a default value

SUBROUTINE SUBX Y Z

PARAMETERA NMAX EPSD ONED

b ecomes

defun sub x y z

key a nmax eps d one d

Array references are based in Fortran and based in Lisp To preserve the rela

tionship b etween the original and translated co de an array referencing macro fref

is provided so that the Lisp arrays are referenced with a displacement Statements

in whicharrayvalues are assigned are translated to fset statements where fset is

a supplied macro which acts like setf in the presence of arrays but recognizes the

sp ecial nature of fref See however the comments on array referencing in Sections

and ab ove

XIJM fref x i j m

X D YIJ fset fref x d fref y i j

In a Lisp do the lo op variables are lo cal to the lo op under consideration In

Fortran they are not In particular the value of a lo op variable is left at its nal

value when the lo op exits For this reason lo ops are translated into a sp ecial macro

form fdo which has the simple syntax

fdo var init final increment body body

The fdo macro will translate into a Lisp tagbody with no lo cal variables The

variable b eing stepp ed must b e established in the surrounding tagbody in order to

allow it to p ersist when the fdo is completed Another asp ect of lo op translation

return statements frequently o ccur within a b o dy of a do blo ckinFortran co de

to signal exit from a subroutine not exit from a lo op Therefore the return is

translated to a returnfrom in Lisp

Unstructured uses of gotosuch as jumping from outside to inside a lo op are

not translated but tagged as errors

In the case of if then else statements the following translations are p erformed

ANEB not a b

AGEB a b etc

IF pred statement if pred statement

IF pred THEN ENDIF when pred

IF pred THEN

ELSE IF pred

ELSE

R J Fateman et al

ENDIF

ENDIF

cond pred

pred

t

Where a function o ccurs as an argument to a subroutine and is declared as external

in the Fortran co de then at least in simple cases the co de is translated to a funcall

in the Lisp co de

SUBROUTINE SAF

EXTERNAL F

UFX Y

setq u funcall f x y

Functions used in the slot for f need to pro cess the arguments whichwould b e

presented to them in translated co de

Very little additional errorchecking is p erformed in the translated co de Lisp

implementations generally makeavailable much more general error checking than

Fortran eg checking the numb ers of arguments and their typ es to builtin and

user functions as well as a more sophisticated tracing and debugging environment

In the case of SenpackLispack explicit user function argument datatyp e check

ing is p erformed by a Senpack shell function which calls the basic Lispack function

which then need not p erform checking In this way the Lispack function is able to

have all of its arguments fully declared and programs compiled for extra sp eed yet

never will b e presented with illformed data

Subroutine calls are translated to expressions returning multiple values allowing

the full callbycopyreturn of f

CALL SUBA B multiplevaluesetq a b sub a b

As mentioned previously handco ding w as used to reduce this burden in the case

of Lispack each function was considered separately and the nature of eachvariable

in the argument list whether it was input or output or b oth determined and the

Lisp function structured accordingly Since passed arrays are altered destructively

there is no need to have these returned and the multiple value returns can b e re

stricted to just the nonarraytrueoutputvariables Performing these optimization

transformations by hand was found to b e error prone esp ecially where manycom

plex subroutine calls are b eing made This is one of the asp ects of fcl whichhas

b een agged for further investigation

As many declarations as p ossible are included in the translated co de to improve

the eciency of the Lisp compiled version For example the Lispack singular value

decomp osition routine svdcmp of Numerical Recipes is dened as follows

defun svdmcp a declare type simplearray doublefloat a

Fast FloatingPoint Pro cessing in Common Lisp

prog

m arraydimension a

n arraydimension a

w makearray n elementtype doublefloat

initialelement d

v makearray list n n

elementtype doublefloat

initialelement d

rv makearray n elementtype doublefloat

initialelement d

g d scale d anorm d s d h d f d

c d x d y d z d l ll

declare type fixnum l ll m n

declare type doublefloat c anorm f g h s scale x y z

declare type simplearray doublefloat w

declare type simplearray doublefloat rv

declare type simplearray doublefloat v

end

return values a w v

Statements not currently translated

Variables which are identied with common storage are established as global with

a sp ecial pro clamation We do not p ermit equivalencing two dierently named

pieces of data by allo cating them to the same lo cation in a shared commonIn

the context of this restriction we sampled the usage of common blo cks the NAG

Library with ab out program les contains ab out named common blo cks

Numerical Recipes has ab out Wehave come across co de where common blo cks

are used uniformly to pass data b etween subroutines bypassing the normal safer

mechanism eg UNAFEM The cases where named common blo cks are used

with dierent substructure and names within dierent subprograms are in our

exp erience quite rare As indicated in section ab ove lexical closures or p erhaps

global data in packages are two appropriate mechanisms for functions to share data

in Lisp Wehave not automated this however

Some Fortran co de passes what amounts to a p ointer into an arraytoasub

routine sayby binding the array z to a thereby making references to z

corresp onds to a etc This can b e done in Lisp by declaring a displaced array

a corresp onding to a and passing into the subroutine a

Another rarely used Fortran structure is the equivalence statement This con

struction is used ab out times in the NAG Library and once in Numerical Recipes

Some programs use the statementinanessential way XMP for example The

Common Lisp displacedarray enables these to b e mo deled eectively although

wehave not chosen to automate this feature

Although Common Lisp has an elab orate set of formatting features the format

function more than equal to those of Fortran for formatting numerical input and

output at this stage in the developmentoffcl read and write statements are not

translated Wehave b een exp erimenting with formatted output

R J Fateman et al

No provision has b een made for Fortran statement functions ie function de

nitions o ccurring b efore the rst executable statement A numb er of other minor

features are also not covered We do not automatically handle the Fortran entry

statement used only once in Numerical Recipes and in the NAG Library only in

Chapter F devoted to NAGs implementation of the Basic Linear Algebra Rou

tines BLAS where the essentially trivial use of the statementistosetup

aliases for subroutines names FAAF and DROTG for example where the latter is

the original BLAS name

Although the translator do es not oer a complete range of facilities it has proved

to b e useful even in its presentform

CONCLUSION

In this article wehave asserted and shown through a numb er of examples that

numerical computing in Lisp need not b e slow and that many features of Common

Lisp are useful in the context of numerical computing Wehave demonstrated that

the sp eed of compiled Common Lisp co de though to day somewhat slower than

that of the b est compiled Fortran could probably b e as ecient and in some ways

sup erior Wehave suggested ways in which the sp eed of this co de might b e further

improved

Since sp eed is not the absolute criterion for most programmers or else wewould

use more assembler other factors must b e considered We b elieve that Lisp has

some interesting advantages in pro ductivity and as more am bitious programs are

attempted Lisp will b e more appreciated for its strengths in comparison to Fortran

or C

Wehave indicated how the algorithmic contentofexistingFortran co de might

be converted to Lisp to make use of the features of Lisp as well as other programs

written in or interfaced to Lisp

It is our b elief that the advantages of using Lisp will b ecome more apparentas

highquality standardsconforming implementations of Common Lisp b ecome more

widespread The ambitions of scientic computation systems constructors continue

to grow from individual programs to libraries to in tegrated seamless environ

ments Such systems are not easily constructed by writing more Fortran Even

if truly successful standardization of foreignfunction interfaces were implemented

ecientlyineachCommon Lisp system the demands of scientic computing en

vironments mayoverwhelm this b order Suchsystemsmay require a far tighter

integration in terms of symb olic and numeric data than can b e managed comfort

ably via callouts to numeric libraries or Fortran subroutines After all the natural

datatyp es of list tree etc used for constructing say a mathematical symb olic

formula in Lisp are dicult to manipulate in Fortran or C or C

Perhaps one key to writing advanced environments will b e the use of Lispbased

to ols Making Lisp credible even in numerical computing will encourage program

mers to takeadvantage of the additional features of what is simultaneously one of

the oldest and yet most mo dern of programming languages

ACKNOWLEDGMENTS

The assistance of the UniversityofWaikato Research Committee in supp orting

this research is gratefully acknowledged R Fateman was supp orted in part by

Fast FloatingPoint Pro cessing in Common Lisp

NSF Grantnumb er CCR and by NSF Infrastructure Grantnumber CDA

Comments from Rob MacLachlan of CMU have b een esp ecially helpful

concerning CMU Common Lisp

REFERENCES

A V Aho J Hop croft J D Ullman The Design and Analysis of Computer Algorithms

AddisonWesley Publ Co

A V Aho Ravi Sethi J D Ullman Compiler Principles Techniques and Tools Addison

Wesley Publ Co

Allegro CL Users Guide Release Franz Inc Berkeley CA

E Anderson Z Bai C Bischof J Demmel J Dongarra J Du Croz A GreenbaumS

Hammarling A McKenneySOstrouchov and D Sorensen LAPACK Users Guide Release

SIAM

Henry G Baker The Nimble Typ e Inferencer for Common Lisp Nimble Computer Corp

Encino CA submitted for publication

R F Boisvert S E Howe and D K Kahaner GAMS a framework for the management

of scientic softwareACM Trans Math Soft pp

K A Broughan The SENAC ManualVols and University of London London

K A Broughan Interactive access to subroutine libraries an interlink for SENACinPro

ceedings of the International Conference on Computation al Techniques and Applications

CTAC J Horvath ed Hemisphere Washington pp

K A Broughan G Keady T D Robb M G Richardson and M C Dewar Some Symbolic

Computing Links to the NAG Numeric Library SIGSAM Bulletin pp

K A Broughan SENAC a high level interface for the NAG LibraryACM Trans Math

Soft pp

K A Broughan SENAC Lisp as a platform for constructing a problem solving environment

in Programming Environments for Highlevel Scientic Problem solving P W Ganey and

E N Houstis Eds NorthHollandIFIP pp

K A Broughan and G Keady Numlink and Naglink links to the NAG library from SENAC

and Macsyma Pro ceedings on the Interface b etween Symb olic and Numeric Computing

Helsinki Helsinki UniversityTechnical Rep ort pp

K A Broughan Naglink a working symbolicnumeric interface in IFIP TCWG Work

ing Conference on Problem solving Environments for Scientic Computing B Ford and F

chatelin Eds Elsevier Amsterdam pp

K A Broughan The Naglink Manual An interfacebetween MACSYMA and NAGVersion

Development Finance Corp oration of New Zealand

K A Broughan Asymbolic numeric interface for the NAG Library Newsletter of the Nu

merical Algorithms Group August pp

hou and K A Broughan The Risch algorithms of Macsyma and SENAC SIGSAM WCSc

Bulletin pp

D S Burnett Finite Element Analysis AddisonWesley Reading MA

S I Feldman Personal Communication Marc h author of f compiler

Grant O Co ok Jr Code Generation in ALPAL using Symbolic Techniques in Pro ceedings

of the International Symp osium on Symb olic and Algebraic Computation PWang Ed

Berkeley CA ACM New York pp

J K Fo deraro K L Sklower and K Layer The Franz Lisp Manual University of California

Berkeley CA

JKFo deraro editor Sp ecial Section on Lisp Communications of the ACM volume

Sept pp

B Ford Transportable Numerical Software Lecture Notes in Computer Science vol

pp SpringerVerlag

N Ka jler A Portable and Extensible Interface for Computer Algebra Systems in Pro ceedings

of the International Symp osium on Symb olic and Algebraic Computation PWang Ed

Berkeley CA ACM New York pp

esley Publ Co D E Knuth The Art of Computer Programming Vol nd ed AddisonW

R J Fateman et al

DHLanamAnAlgebraic Frontend for the Production and Use of Numeric Programs in

Pro c of ACM Symp on Symb olic and Algebraic Computation PWang Ed Snowbird

Utah Aug pp

CLLawson R J Hanson D Kincaid and F T Krogh Basic linear algebra subprograms

for FORTRAN usageACM Trans Math Soft pp

Lucid Common LispSUN Advanced Users Guide Release Lucid Inc Menlo Park CA

The Macsyma Reference ManualVersion Mathlab Group Lab oratory for Computer Sci

ence MIT Cambridge MA

J McCarthyP W Abrahams D J Edwards PAFox T P Hart and M J Levin Lisp

Programmers Manual MIT Press

C Meszteny and C Witzgall Stable Evaluation of Polynomials J of Res of the Natl

Bur of StdsB B no Jan pp

R E Marsten The design of the XMP linear programming libraryACM Trans Math Soft

pp

Rob ert A MacLachlan The Python Compiler for CMU Common Lisp Pro c ACM Conf

on Lisp and Functional Programming Oct See also CMU Common Lisp Users Manual

CMUCS or up dates

W H Press B P FlannerySATeukolsky and W T Vetterling Numerical Recipes

FortranCambridge University Press Cambridge UK

K Stanley private communication

Guy L Steele Jr Common Lisp the Language nd ed Digital Press

PSWang FINGER A Symb olic System for Automatic Generation of Numerical Programs

in Finite Element Analysis J Symbolic Computation pp

Jon L White Lisp Program is Data A historical perspective on MACLISP Pro ceedings of

the MACSYMA Users Conference MIT Lab oratory for Computer Science Cambridge

Mass published as NASA CP pp

Jon L White NIL A Perspective Pro ceedings of the MACSYMA Users Conference

MIT Lab oratory for Computer Science Cambridge Mass

Michael C Wirth On the Automation of ComputationalPhysics PhD diss Univ Calif

Davis Scho ol of Applied Science Lawrence Livermore Lab Sept

App endix Declarations in Common Lisp

For an extensive discussion of declarations see Steele chapter

Without going into the details of their precise meaning and placement in programs

we can state simply that declarations are additional optional advisory information

provided by the programmer to the Lisp system

Steele explains that with the exception of special declarations whichwe will

not discuss declarations are completely optional and correct declarations do not

aect the meaning of a correct program Furthermore it is considered an error

for a program to violate a declaration suchasatype declaration but an im

plementation is not required to detect such errors though such detection where

feasible is to b e encouraged

In fact how should such errors b e detected and when should they b e rep orted

Information that x isavariable whose value is a oatingp ointdouble numb er

may b e used by the system to insert additional runtime errorchecking co de into

a compiled program or paradoxically to remove errorchecking co de On the one

hand one might pro duce a program that checks at all appropriate interfaces that

x has the appropriate typ e as declared On the other hand by assuming the typ e

declaration is correct the compiler might remove all runtime checks and generate

co de that consists solely of machine co de to execute oatingp oint op erations One

can sometimes combine these two ideas of checking and sp eed if one can identify

the appropriate interfaces for typ echecking in a low frequency part of the co de

Fast FloatingPoint Pro cessing in Common Lisp

one can then run fullsp eed in the innermost lo ops

The use of declarations is further complicated by the fact that manyCommon

Lisp implementations provide an interpreter as well as a compiler On correct

programs these must op erate identicallyHow they op erate on programs that are

correct except for declarations is not dened since it is an error to have incorrect

declarations

The common wisdom is to debug using an interpreter but then compile for

sp eed For many systems compiletime static checks are helpful in nding errors

and for at least one system CMUCommon Lisp the compiler literature strongly

emphasizes the advantage of using the compiler for debugging As p ointed out by

R MacLachlan private communication the aim in the Python CMUCL compiler

is to provide a high level of static checking of co de during compiling He observes

Debugging with a typical interpreter do esnt guarantee that the co de will work

compiled since the interpreter will ignore typ e declarations that the compiler

counts on b eing correct Thus the debug in the interpreter run with compiled

co de ignores real problems The compiler provides a stronger comp onentof

reality

In addition to the usual alternatives of ignoring declarations or trusting them

CMU CL oers the alternative of using them but checking them This oers

useful debuggable errors when a declaration is incorrect while providing run

times with nearp eak sp eed

With appropriate optimization settings even a compiled program can b e provided

with a high degree of sourcelevel debugging supp ort approaching the capabilities

provided by traditional Lisp interpreters

Certainly the Common Lisp sp ecication provides wide latitude for the study of

the interaction b etween compiletime and runtime errorchecking Naturally it is

quite valuable to uncover errors including errors in declarations whic h after all

serve not only as optimization hints but sp ecications and do cumentation

Received May revised January accepted January