Using Automatic Memoization as a Software Engineering To ol in RealWorld AI Systems James Mayeld Marty Hall Computer Science Department Science Applications International Corp University of Maryland Baltimore County Broken Land Parkway Suite Baltimore MD Columbia MD mayeldcsumb cedu hallcsumb cedu Tim Finin Computer Science Department University of Maryland Baltimore County Baltimore MD nincsumb cedu April Abstract Memo functions and memoization are well known concepts in AI programming They have b een dis cussed since the Sixties and are often used as examples in intro ductory programming texts However the automation of memoization as a practical software engineering to ol for AI systems has never received a detailed treatment This pap er describ es how automatic memoization can b e made viable on a large scale It p oints out advantages and uses of automatic memoization not previously describ ed describ es the com p onents of an automatic memoization facility enumerates p otential memoization failures and presents a publicly available memoization package CLAMP for the Lisp programming language Exp erience in applying these techniques in the development of a large planning system are briey discussed Track Enabling Technology Subtrack Software Engineering Words Corresp ondence Please address all corresp ondence to James Mayeld Computer Science Department University of Maryland Baltimore County Baltimore MD Voice Fax Email mayeldcsumb cedu Intro duction Memo functions and memoization are well known concepts in AI programming They have b een discussed since the Sixties and are often used as examples in intro ductory programming texts However the automation of memoization as a practical software engineering to ol for AI systems has never received a detailed treatment This pap er describ es how automatic memoization can b e made viable on a large scale It p oints out advantages and uses of automatic memoization not previously describ ed itemizes the comp onents of an automatic memoization facility enumerates p otential memoization failures and presents a publicly available memoization package CLAMPCommon Lisp Automatic Memoization Package for 1 the Lisp programming language We will also briey discuss our exp erience in applying these techniques in the development of the Signature Management System SMS a decision supp ort system that provides submarine crews with situational awareness and op erational advice in order to reduce detectability The term memoization is derived from the term memo function which was coined by David Michie It refers to the tabulation of the results of a set of calculations to avoid rep eating those calculations Auto matic memoization refers to a metho d by which an ordinary function can b e changed mechanically into one that memoizes or caches its results We place the decision ab out which functions to memoize in the hands of the user this contrasts with the approach of Mostow and Cohen which tries to automatically infer which functions should b e memoized These two approaches to automatic memoization are not incompatible although as Mostow and Cohen p oint out the latter approach is not a practical to ol Memoization is particularly apt for AI applications Rapid prototyping is a hallmark of AI programming Automatic memoization allows the programmer to write certain functions without regard to eciency while b eing assured of reasonable p erformance By allowing the programmer to avoide these eciency concerns at the outset automatic memoization facilitates the kind of exploratory programming used in the development of most AI systems The principle of memoization and examples of its use in areas as varied as logic programming functional programming and natural language parsing have b een describ ed in the literature In all of the cases that we have reviewed the use of memoization was either built into in a sp ecial purp ose 1 Throughout this pap er we use the name Lisp to refer to the language Common Lisp as describ ed in Steele computing engine eg for rulebased deduction or CFG parsing or treated in a cursory way as an example rather than taken seriously as a practical technique The automation of function memoization as a practical software engineering technique under human control has never received a detailed treatment In this pap er we rep ort on our exp erience in developing CLAMP a practical automatic memoization package for Common Lisp and its use in a largescale AI pro ject By means of illustration consider the divideddierence algorithm dd which is shown in pseudoco de in Figure This algorithm is used to determine co ecients of interp olated p olynomials The algorithm which is a standard one in numerical metho ds is taken directly from Numerical Mathematics and Computing The application itself is not particularly imp ortant it is the runtime b ehavior of the algorithm that is of interest to us here The call tree for the divided dierence algorithm dd forms a network rather than a tree thus a recursive call to dd with particular arguments may b e rep eated many times during a single computation For example a call to dd with low and high will generate one recursive call with low and high and another recursive call with low and high each of these will in turn make a recursive call with low and high Memoization of dd causes each calculation to b e p erformed only once and stored in a table thereafter each rep eated recursive call is replaced by a table lo okup of the appropriate value Figure compares the p erformance of memoized and unmemoized versions of a Lisp implementation of the divided dierence algorithm using f n cosn and the rst n natural numb ers as arguments Since a single function call on n p oints generates two recursive calls on n p oints the unmemoized version has n 2 time complexity After memoization the rst invo cation requires n time since no subsequence 2 of p oints is calculated more than once and there are n n subsequences Subsequent invo cations take nearconstant time This algorithm for divided dierence is typical of a large class of algorithms which have very elegant recursive denitions which are simply unusable for most real problems The conventional resp onse to this situation is to manually rewrite the algorithm in a dynamic programming style Of course such manual rewriting of the co de takes eort and involves the risk of intro ducing new bugs An attractive alternative is to use an automatic memoization package to convert the elegant but inecient recursive algorithm into a divideddifference algorithm ddpoints array low arrayindex high arrayindex fn function begin if low high then return fnpointslow else return ddpoints low high fn ddpoints low high fn pointshigh pointslow end Figure The divided dierence algorithm for determining co ecients of interp olated p olynomials can b e elegantly written using recursion and made ecient through the use of automatic memoization n Unmemoized Memoized Memoized rst run subsequent runs Centuries Figure This table shows the b enets of memoization on a Lisp implementation of dd The time complexity n 2 is reduced from to n for initial runs and to nearconstant time on subsequent ones useful one This is attractive of course only if such a package can address the practical problems faced in real application In the next section we describ e the some of the asp ects and uses of automatic memoization not previously describ ed in the literature Section three compares the use of automatic memoization to the alternative of rewriting co de by hand Section four describ es the general comp onents that should b e present in any automatic memoization facility Section ve presents problems inherent in the use of memoization and the various ways to address them Section six describ es our exp erience in using automatic memoization in in the development of SMSa large decision supp ort system Uses of Memoization There are four main uses of automatic memoization Two of these involve the avoidance of redundant calculation rst within a single function invo cation and second across invo cations The third use of automatic memoization is as a precalculation to ol while the fourth is as a timing and proling to ol These are not conjectured uses but ones which we found to b e eective in many situations in a large realworld AI application We discuss each of these uses in more detail in the following subsections Rep etition within a Function Call The most common use of memoization is to avoid the rep etition of subcalculations within a single function call In the divided dierence example presented ab ove there were many rep eated recursive calls within a single function invo cation This typ e of rep etition is common For example a simple recursive backtracking parser may parse the same constituent many times during a single parse thus its p erformance is p o or Norvig has shown that such an algorithm can obtain the p erformance of chart parsing or of Earleys algorithm through the application of memoization Thus we can view memoization as a technique to automatically convert a recursive algorithm into one that has the same b ehavior as a dynamic programming algorithm rather than determining the prop er order in which to construct subpieces memoization of a simple solution can typically achieve the same p erformance Again through automatic memoization one can in many cases achieve the b enets of a dynamic programming approach without rewriting the control structure or intro ducing new data structures