sign in sign up
<<
Home , Computer algebra, Numerical analysis

Can the Eureqa Symbolic Regression Program, , and Numerical Help Each Other?

David . Stoutemyer

ureqa is a symbolic regression program was curious to learn if the program could also described by Schmidt and Lipson [11], be used as a supplementary tool for experimental freely downloadable from [13], where , , and numerical there are press citations, a bibliography analysis. In the first few weeks of using this tool, I of its use in articles, a blog, and a have already found that: discussionE group. In contrast to typical regression , the user does not have to explicitly or 1) Eureqa can sometimes do a job of exact simpli- implicitly provide a specific containing fication better than existing computer algebra unknown constants for the software to determine. With little or no guidance, symbolic regression systems if there is a concise equivalent determines not only unknown coefficients but also expression. the class and form of the model expression. See [9] 2) Eureqa can often discover simple expres- for quick insight into the underlying “survival of sions that approximate more complicated the fittest” paradigm for symbolic regression. An expressions or fit a set of numerical results Internet search on “symbolic regression” reveals well. that there are other such programs. However, my 3) Even when the fit isn’t as accurate as de- skeptic’s curiosity was aroused by such extreme press praise as sired, the form of a returned expression Move over, Einstein: Machines will take it often suggests a class of forms to try for from here. [10] classic regressions, , or expansions giving higher accuracy. There are very clever “thinking machines” in existence today, such as Watson, the IBM 4) Eureqa could do an even better job if it computer that conquered Jeopardy! last supplemented its search with exploitation year. But next to Eureqa, Watson is merely of more computer algebra and numerical a glorified search engine. [5] methods, including classic regression. By one yardstick, Eureqa has even discov- 5) There are important caveats about how to ered the answer to the ultimate question of use Eureqa effectively. life, the universe and everything. [5] 6) The most extreme press quotes are exag- The program is designed to work with noisy gerations, but Eureqa really is quite impres- experimental data, searching for, then returning a sive. set of result expressions that attempt to optimally trade off conciseness with accuracy. However, I This article has examples that illustrate these findings, using Eureqa 0.93.1 beta. David Stoutemyer is a retired professor of information and at the University of Hawaii. His email Regarding times reported herein, the address is [email protected]. computer is a 1.60GHz Intel Core 2 Duo U9600 DOI: http://dx.doi.org/10.1090/noti1000 CPU with 3 gigabytes of RAM.

June/July 2013 Notices of the AMS 713 Exact Simplification and Transformation 1) First I plotted expression (1) in Mathematica, A Trigonometric Simplification Example revealing that it is antisymmetric and that its fundamental period appeared to be π, with Here is a 15 assignment of an input trigono- higher frequency components appearing to metric expression to a dependent y: have a minimum period of π/4. This was cos(x)3 sin(x) confirmed by TrigReduce[y], which returned y := cos(x)3 sin(x) + 2 the equivalent expression 3 1 + 2 cos(x) cos(2x) sin(x) (sin(2x)+6 sin(4x)+4 sin (6x)+sin(8x)) . 16 (1) 2) I decided to use evenly spaced values of x cos(x)3 cos(4x) sin(x) + because expression (1) is periodic, bounded, 2 and ∞ for all real x. I guessed that it 3 − cos(x) sin(x)3 − 2 cos(x) cos(2x) sin(x)3 might help Eureqa discover and exploit 2 the antisymmetry if I used sample points cos(x) cos(4x) sin(x)3 − . symmetric about x = 0. I also guessed that it 2 might help Eureqa discover the periodicities The default simplification merely combines the if I used exactly two fundamental periods. first two terms, which are similar. I guessed that using 16 samples within the However, the simplify(y) function1 required minimum period π/4 would be sufficient only 0.03 seconds to return the much simpler to resolve it quite well; then I doubled   that because Eureqa uses some points for (2) 4 sin(x) cos(x)5 2 cos(x)2 − 1 . fitting and others for error assessment. This implies 128 intervals of width π/64 from An equivalent expression produced by the Math- −π through π. I then created a table of ematica FullSimplify[. . .] in only 0.08 17-digit floating-point pairs of x and y, then seconds is also much simpler: exported it to a file by entering 2 (sin(3x) − sin(x)) cos5(x) .  Export “trigExample.csv”, Table But the equivalent even simpler form discovered  π  by Eureqa is N [{x, . . .}, 17], {x, −π, π, } 128 4 (3) y = cos(x) sin(4x) . where the ellipsis was expression (1).2 That even the simplified model of evolution in 3) I then launched Eureqa, opened its Eureqa can work so well might change some minds. -like Enter Data tab, then replaced the default data there with mine Caveat: The uses a random by importing file trigExample.csv. In the row generator with no current user control over its labeled var, I then entered x in column A seeding, which appears to be done by the clock. and y in column B. Therefore are not currently repeatable, 4) The Prepare Data tab then showed superim- and the computing times to obtain expression (3) posed plots of the x and y data values and varied dramatically, from 3 seconds to several offered preprocessing options that aren’t minutes. It conceivably could require more time relevant for this very accurate data. than you would ever be willing to invest. However, 5) Figure 1 shows the Set Target tab: although even 3 seconds is much longer than the (a) The pane labeled The Target Expression computer algebra times, it is certainly worth a suggests the fitting model y = f (x), which is few minutes wait to obtain such a nice result, and what I want, so I didn’t change it. such experiments can be done while away from (b) The pane labeled Primary Options has check the computer, such as while eating, sleeping, or boxes for the desired building- playing cell phone games. There is also an blocks used in composing candidate f (x) to use parallel cloud computing, which reduces the expressions. The default checked ones are mean and variance of the elapsed time necessary sufficient for the sort of concise equivalent to obtain a satisfactory result. Here is how I used Eureqa to obtain this 2The reason for requesting 17 significant digits was that I delightfully simple exact result (3): wanted Mathematica to use its adaptive significance arith- metic to give me results estimated to be accurate to 17 digits despite any catastrophic cancellations, and I wanted Eureqa 1simplify(. . .) has an optional second keyword argument to receive a 17th significant digit to help it round the se- “size” that is intended to minimize size, but for this example quences of input digit characters to the closest representable the result is less concise than (2). 16-digit IEEE double values, which are used by Eureqa.

714 Notices of the AMS Volume 60, Number 6 Figure 1. Eureqa Set Target tab for exact trigonometric simplification example.

to expression (1) that I am seeking. Therefore Maximize the R-squared goodness of fit or I didn’t check any of the other offered Maximize the correlation coefficient is more functions and operators, not all of which scale and mean invariant, which are desirable are shown on this screen shot.3 Formula properties too. However, the data is already building blocks also shows the corresponding well scaled with means of 0. Complexity measures, which can be altered () For the covered Data Splitting drop-down by the user. The complexity measure of an menu the default alternative is to designate expression is the sum of the complexities of a certain percentage of the data points its parts. for fitting the data (training) and a certain (c) The drop-down menu labeled Error metric percentage for assessing the error measure offers different built-in error measures for (validation). The individual assignments to Eureqa to try optimizing. A bound is usually these categories are done randomly, with more reassuring than the alternatives, so some overlap if there aren’t many data I chose Minimize the worst-case maximum points. For the almost exact data in this error. The documentation suggests that article, I would prefer that alternative points be assigned to alternative categories, with the 3 I could have saved search time by unchecking , be- end points being used for training. However, cause the floating-point coefficients make a denominator none of the alternatives offered this, so I unnecessary for this class of expressions. I could also have chose the default.4 saved search time and perhaps obtained a more accurate result by checking the Constant box, because integer 6) I then pressed the Run button on the coefficients are quite likely for exact equivalent expressions, Start Search tab and watched the tempo- making it worth having Eureqa try rounding to see if that ral progress of the search. The plot in improves the accuracy. However, I decided to accept the de- fault checked boxes to see if Eureqa could find a good exact 4I have since learned that, with such precise data, another equivalent without any more help from me. Other building promising alternative would be to use all of the points for blocks are described at [6]. both training and validation, which is a current option.

June/July 2013 Notices of the AMS 715 Figure 2. Eureqa Start Search tab for exact trigonometric simplification example.

Figure 2 dynamically zoomed out to show (b) Result (4) appears first with a reported Fit of a log-log plot of the best error measure 0.000111, followed by the same expression obtained so far as a function of computing with monotonically better fits up through time. The Project Log dynamically showed 3.81 × 10−10 at 39 seconds.6 I terminated the successive candidates that are better than any so far on the basis of either complexity or 6The same displayed formula was associated with such dra- accuracy. The entertainment of watching the matically different error measures because Eureqa always evolution of these panes is very appealing. It rounds its displayed coefficients to about four significant dig- its, and in this case it enabled display of an exact result to is similar to rooting for the home team at a which it was merely converging. If I had checked the Integer sporting event. Notice that: Constant building block, then Eureqa probably would have (a) Integer powers of subexpressions are rep- rounded the coefficients to . However, y(0.0) = 0.0 resented in the Project Log as repeated and the data varies between about ±3.8, so a maximum −10 multiplications, and their complexity is mea- residual of 3.81 × 10 is enough to convince most peo- ple that the displayed result is exact. Nonetheless, the result sured that way. For example, several results expression might not be equivalent to the sampled expres- are displayed as sion everywhere between samples even if all the residuals are 0.0. Also, one or more of those floating-point zeros might (4) y =cos(x)∗cos(x)∗cos(x)∗cos(x)∗sin(4∗x) be caused by underflow of whose exact values are with a complexity measure of 26.5 nonzero. For this example we can prove equivalence for all complex x because " ! 5 cos(x)3 sin(x) The complexity measure seems less than ideal: the ulti- TrigReduce cos(x)3 sin(x) + + · · · mately reported formula in the View Results tab of Figure 3 2 is cos(x)4 sin(4x). This is more concise and requires only # 4 one cosine and two multiplies to compute the cos(x) ∗ − Cos[x] Sin[4x] → 0 . cos(x) ∗ cos(x) ∗ cos(x) factor if compiled by any decent .

716 Notices of the AMS Volume 60, Number 6 Figure 3. Eureqa View Results tab for exact trigonometric simplification example.

search at 8 minutes, during which Eureqa etc. This minimal computer algebra is also tried different that didn’t fit as well, used to make the displayed results in the regardless of how complicated they were. Best solutions of different sizes pane more 7) Figure 3 shows the results on the View attractive. These transformations include: Results tab: • Integer powers and products of sums are (a) In the pane labeled Best Solutions of Different expanded to make them more nearly canonical.7 Sizes, the first column lists the complexity • Factors and terms are sorted; then similar measure, the second column lists an error factors and terms are collected. measure, and the third column lists the • Numeric subexpressions are reduced to a single corresponding candidate . If you number. click on a row, the pane in the lower left corner • A few rules such as abs(abs(u)) → abs(u) and gives more detail, with various rounded error 1 ∗ u → u are applied. measures, including the Primary Objective that I chose, which was maximum absolute However, you will often see a sub-expression error. such as 1 cos(1x). This is because the 1’s are (b) Eureqa does some automatic expression sim- a result of rounding noninteger coefficients plification: Whenever a mutation produces to the typically displayed 3 or 4 significant a new candidate expression or a crossover digits. Nonetheless, I noticed unexploited produces two new candidates containing opportunities, such as Eureqa unnecessarily mixtures of the two parents’ subexpressions, trying the sub-expression sin(x + 6.283). Ex- a few transformations are applied to make ploiting symmetry and angle reduction could the expression more nearly canonical and to avoid having a misleadingly high com- 7Unfortunately, expanded forms are often less concise plexity on account of uncollected similar than equivalent factored forms that would have smaller terms, uncombined numeric subexpressions, Complexity.

June/July 2013 Notices of the AMS 717 preclude the need to try candidates hav- • I tabulated only the real part for example (6), be- ing constant terms outside a much smaller cause rounding errors for the principal branch interval. of the fractional powers of the negative quantity p (c) The pane in the lower right corner plots an 81q2 − 12−9q generated some relatively very error measure as a function of complexity small magnitude imaginary parts. for the twelve reported optimal candidates, • To generate 172 → 289 rows of data for example with the currently highlighted candidate as (7), I used a larger red dot. 2 (d) The pane in the upper right corner shows a Apply[Join,Table[N[{x, y, Max[x− √ y, 0] plot of the highlighted expression superim- 3 3 1 1 posed on the validation points in dark blue −Max[y−x, 0]}, 17], {x, −1, 1, }, {y, −1, 1, }]]. and the training points in lighter blue. The 8 8 drop-down menu above it offers the option The examples so far illustrate that Eureqa can of instead plotting the residuals at all of the sometimes determine a simpler exact result than a data points, which gives a much better idea . However, I constructed of the fitting errors as a function of x. If these very simple results, then transformed subex- these residuals had revealed a nonnegligible pressions in ways that I suspected would be difficult recognizable pattern such as being approxi- for common transformations to reverse, particu- mately proportional to sin(16x), then I would larly if applied to the entire expression rather than have tried another search with the model well-chosen pieces. Despite this it was not easy to y = f (cos(x), sin(4x), sin(16x)). I would it- find examples for which neither FullSimplify[. . .] erate this process until there was no further nor simplify(. . .) could determine the very simple improvement or the residuals revealed no equivalent. Thus, Eureqa should be regarded as an nonnegligible recognizable pattern. occasionally beneficial supplement to these func- Instead of searching for the most concise tions rather than as a replacement. Also, effective equivalent, one can also search for equivalent use of Eureqa requires good judgment in: expressions of a particular form. For example, we • the interval spanned by the sample points, their could request that expression (1) be transformed number and spacing; into a function without cosines by unchecking • the form selected for the target expression; and that building block. As another example, the • the building blocks that are selected together request z = f1(x) ∗ f2(y) can be used to search with their complexities. for an equivalent form having separated variables. Effective use also probably depends on experience (Nonnegative integer suffixes on variable and with Eureqa. Therefore, the benefits that I have function names are pretty-printed as subscripts.) discovered should be regarded as a lower bound Some Other Exact Simplification Examples because of my novice status.

Using “Assuming[. . . , FullSimplify[. . .]]” in Mathe- How to Reduce the matica and “simplify(. . .), assuming ...” in Maple, As illustrated by example (7), data for representing I could not make them accomplish the follow- all combinations of n different values for each of m ing quickly successful Eureqa simplifications, for different independent variables requires nm rows, which I selected the default building blocks, Integer and we must have n ≥ 2 for Eureqa to discern Constants, and other functions or operators that any nonconstant dependence on each variable. occur in the specific given expressions: Thus, for a given n, this exponential growth q  √ √ (5) bxc +  x | x ≥ 0 → 2  x . in data rows with the number of independent variables can greatly increase the time required p 2/3 21/3 81q2 −12−9q +241/3 for Eureqa to find a good fit. Moreover, yielding to (6) q2 + the consequent temptation to reduce computation p 1/3 62/3 81q2 − 12 − 9q time by reducing n as m increases tends to reduce   √  the precision of the results. Fortunately, there are 2 cos 1 acos − 3 3q − 3 √ 2 several complementary techniques that sometimes 3 reduce the effective dimensionality: 2 | |q|< √ → q2. 1) Dimensional analysis can sometimes re- 3 3 duce the number of independent variables, (7) max(x−y, 0) − max(y −x, 0) → x − y. and [15] describes a program that • Example (5) comes from [7]. automates this.

718 Notices of the AMS Volume 60, Number 6 2) If you can partition an expanded or factored Eureqa can help by discovering form of your expression into two or more concise approximate expressions that meet the components having distinct variable sets, silver standard. then you can independently fit each of those components. For example, you can An Example separately fit max(x−y, 0) − max(y −x, 0) Many well-known and special-purpose functions and max(y−z, 0)−max(z−y, 0) in the sum of are defined by a definite containing a these two expressions, converting a trivariate variable that is not the integration variable, a example into two bivariate examples. definite integral for which there is no known 3) If in every term of a given sum the exponents closed-form expression in terms of simpler known of some subset of the variables sum to functions. For example, suppose that I want to the same homogeneity exponent k, then implement an IEEE double version of the Dogbert substitute 1 for one of those variables v, W function defined by fit this isomorphic problem, then multiply (8) every resulting term by the individual power x Z dt of v necessary to restore the homogeneity. W (x) := | − 1 ≤ x ≤ 1 . r   4) Unlike many experimental situations, for 2 2 πt 0 1 − t + cos we usually have π 2 complete freedom to choose the data values No computer algebra systems that I tried can of the independent variables. In both uni- determine a closed form for W (x). variate and multivariate problems there can Plan B: Do a gold standard numeric integration be sweet spots that deliver more accuracy for a of x values in the interval −1 ≤ x ≤ 1, for a given number of samples than does a then use Eureqa to discover a silver standard brute force Cartesian product of uniformly expression that fits those values well. Here is an spaced points. These special values are often abridged account of my attempt to do this. related to the zeros or extrema of orthogonal 1) I entered the Mathematica input . For example, see the multidi- mensional integral formulas in Chapter 25 xWPairs = Table[{N[x, 17], of [1]. 1 NIntegrate[ , r   (1−t)(1+t) + 2 cos πt Approximate Symbolic Results (9) π 2 Ideally a floating point result is either exact or {t, 0, x}, PrecisionGoal → 17, the closest representable number to the exact WorkingPrecision → 25], {x, −1, 1, 1/64}]. result, with ties broken in favor of having the last significant bit be 0. Well-designed floating-point Then I inspected the 129 number pairs to comes very close to this ideal: If the exact make sure there were no imaginary parts, 9 result is representable within the thresholds for infinities, or undefined values. overflow and for gradual underflow, then the result 9 is the exact result for inputs that differ from the • The data in the Enter Data tab must currently be in- tegers or finite real floats. A fraction, a floating point actual inputs by factors between 1 − εm and 1 + εm, where ε is about 1.11 × 10−16 infinity, a nan, or a nonreal number is not accepted. m If an imaginary part of your data is negligible noise, for IEEE double, which corresponds to about 16 then replace it with 0.0. Otherwise you must sepa- significant digits. A relative error bound of ±εm is rately fit the real and imaginary parts or the absolute the gold standard. There are similar expectations value and the arg, which is the Two-Argument Arc- for operations such as and for tangent in Eureqa. If your data has an undefined functions such as sinusoids or logarithms that value due to an indeterminate form, then replace it are commonly built into but allowing a with the value returned by your computer algebra limit (. . .) function. If your table has an infinity, then few times εm, which I call the silver standard. It is subtract out or divide out the singularity that is caus- a matter of professional pride among numerical ing it. Don’t expect a good fit if you simply omit a value analysts designing general purpose mathematical that has infinite magnitude or replace it with an ex- software to strive for the silver standard for all ceptionally large magnitude having the correct sign, functions provided with the software, and many either of which can dramatically mislead the search. users assume this.8 I was curious to know if • I entered 1 − t2 as (1 − t)(1 + t) in input (9) and else- where to reduce the magnification of rounding errors 8For example, early Microsoft Basic computed elementary by catastrophic cancellation near x = ±1. functions, fractional powers, and integer powers to half • The reason for WorkingPrecision → 25 is that I the precision of the other arithmetic operators, causing un- wanted to further reduce this catastrophic cancella- suspecting users to have results that were often far less tion. accurate than assumed.

June/July 2013 Notices of the AMS 719 2) All the values were real and finite, so I then to polish a result with classic regression entered software to obtain a silver standard result.10 (b) I edited the copied formula because it con- Export [“xWPairs.csv”, xyPairs] ; tained annoying superfluous parentheses and represented integer powers by repeated then I imported the resulting file into Eureqa multiplication. and tried to fit W = f (x) with the default (c) The even symmetry of the integrand and the formula building blocks. After three minutes centered integration from 0 imply that the of search the most accurate formula was antiderivative should have odd symmetry. Therefore the spurious constant term in 0.7773112536 x W = , expression (10) should be discarded. Terms cos(0.4696613973 x7) − 0.2294845216 x2 that are contrary to odd symmetry might arise from the random partitioning of the which is noninsightful and accurate to a data into training and validation sets. disappointing maximum absolute error of (d) Converting the result to recursive form by about 0.4%. collecting similar powers of x or asin(x) 3) A Mathematica plot of the integrand reveals can significantly reduce the Complexity. For a probable explanation: The integrand has example, deleting the spurious constant endpoint singularities, and although they term, rounding the coefficients in result (10) are integrable, the resulting integral has for brevity, then collecting with respect to infinite magnitude low-order at asin(x)2 give the endpoints, which probably makes the W = 0.78 x + 0.015 x3 − 0.03 x2asin(x) less accurate and the Eureqa + + 3 2 search slower than otherwise. (0.076 x 0.00091 x ) asin(x) 4) The plot of numeric antiderivative values on + 0.082 asin(x)3, the Prepare Data tab revealed that it looks which saves one instance of ∗asin(x)2. At approximately proportional to asin(x). the expense of comprehensibility, the Com- 5) So next I did another Eureqa search after plexity and the floating-point substitution disabling the sine and cosine building blocks time can be further reduced by Hornerizing and making the target expression be W = this recursive representation to f (x, asin(x)). I did not enable the arcsine W = x(0.78 + 0.015x2) + asin(x)(−0.03 x2 building block because I didn’t want Eureqa to waste time trying arguments that cause + (x (0.076 + 0.00091 x2) asin(. . .) to be nonreal. In 11 seconds Eureqa + 0.082 asin(x)) asin(x))). found the following seven-term expression This is another place where more computer having a maximum absolute error of only algebra could help Eureqa. about 3.1 × 10−9: (e) Notice that result (10) has no term that is sim- (10) W = −1.870027576 × 10−13 ply a numeric multiple of asin(x). Therefore it doesn’t model the infinite endpoint slopes + 0.7816747744x associated with the integrand singularities + 0.0147770774 x3 there. But that is my fault, because the target − 0.03033616234 x2 asin(x) expression contains no special encourage- ment to include a term of that form, and 2 + 0.07586494202 x asin(x) the plot from the Eureqa Prepare Data tab + 0.0818165982 asin(x)3 reveals that the sample points were not 3 2 closely enough spaced near the endpoints to + 0.0009144579166 x asin(x) . suggest the infinite slope magnitudes there. I aborted the search at 3 minutes with no 10The rounding of coefficients in the Project Log and Best further improvement. Regarding this result: solutions of different sizes panes does increase compre- (a) To view the coefficients rounded to 10 hensibility, particularly since those panes are regrettably significant digits rather than about 4, I unscrollable. Even more rounding could justifiably be done had to copy the expression from the Best for coefficients of terms whose relative maximum contribu- solutions of different sizes pane in the View tion is small. For example, if the maximum contribution of a term over all data points is 1%, then its coefficient justifiably Results tab into Mathematica or a text editor. could be rounded to two less significant digits than the coef- Unfortunately there is currently no way to ficient that contributes the most over all data points. See [3] view or access all 16 digits, so I will have for more about this idea.

720 Notices of the AMS Volume 60, Number 6 6) Result (10) suggests that a set of particularly Thus it is not promising to try larger total degrees good fits would be a truncated series of the with this basis in pursuit of a concise numerically form stable silver standard. Nonetheless, this has been a nice combined use of Eureqa and Mathematica: (c1 asin (x) + c2 x) After the decision to try a target expression of the  3 2 + c3 asin (x) + c4 x asin (x) form f (x, asin(x)), Eureqa discovered a concise  form having 9 significant digits and basis functions + c x2asin (x) + c x3 + · · · 5 6 of the form xj asin(x). I then used Mathematica with numeric coefficients ck. This can be to obtain a result having only one more term regarded as a truncated bivariate series in that is accurate to 11 significant digits, which is terms having a power of x times a power more than adequate for many purposes. For this of arcsin(x) that have nondecreasing odd example Eureqa has served as a muse rather than total degrees. I could constrain Eureqa to as an oracle. search only within this class of expressions However, I couldn’t help wondering: Is there a by entering a target expression having a relatively simple class of forms that Eureqa might particular total degree, such as have revealed for further polishing to a concise silver standard fit by Mathematica? W = f1() asin (x) + f2() x With the help of the Mathematica Series [. . .] 3 2 + f3() asin (x) + f4() x asin (x) function and an embarrassingly large amount of my 2 3 time adapting its results to my desires via a sixteen + f5() x asin (x) + f6() x . line Mathematica function downloadable from a However, standard regression software is website [16], I was able to determine an exactly much faster for merely optimizing some integrable truncated infinite series expansion of the parameters in a specific form/and more integrand that converged sufficiently fast without accurate if done with arbitrary-precision undue catastrophic cancellation over the entire arithmetic. The largest total degree in Eureqa interval −1 ≤ x ≤ 1. Using Assuming[−1 ≤ x ≤ result (10) is 5. Consequently, I next used the 1, Integrate[. . .]], the Hornerized representation Mathematica LinearModelFit [. . .] function of the corresponding antiderivative is with the 12 basis expressions of the form (12) xj asin (x)k having odd total degree 1 ≤ j + k ≤ 5. After 0.27 seconds I received the W˜ (x):=c0 asin(x) following eight-term result, whose significant q  2 2  digits I have truncated for brevity: +x (1−x)(1+x) c1 +x c2 +x (c3 +· · ·) . (11) 22.9 x + 0.012 x3 + 0.000097 x5 Being a single series bifocally expanded about x = ±1, it is C∞ over the entire interior inter- − 22.1 asin (x) − 0.068 x2asin (x) val −1 < x < 1. Notice that, contrary to the + 0.78 x asin (x)2 + 3.09 asin (x)3 previous efforts, asin(x) occurs only to the first 5 power and that all the other terms are multiplied − 0.078 asin (x) . p by (1 − x)(1 + x). A good set of basis expres- p The maximum absolute error at the tabulated sions is thus {asin(x), x1+2k (1 − x)(1 + x)} for − points was about 4 × 10 11, which is two k = 0, 1,... . The higher powers of asin(x) in the significant digits more than (10) for the same previous basis were being used as flawed surro- p total degree and only one more term. The 96 gates for x1+2k (1 − x)(1 + x): octagonal pegs in percent cancellation between the dominant circumscribed round holes—they sort of fit, but in terms 22.9 x and −22.1 asin (x) for x near 0 an impaired way. is less than ideal, but the small error measure The Mathematica function that I wrote computes√ is very gratifying. the exact coefficients ck, which involve 6 and Encouraged, I next tried LinearModelFit [. . .] powers of π. After writing that function I used with total degree 7, which entails twenty basis LinearModelFit [. . .] to effectively economize a terms. However, the most accurate returned result higher-degree approximation to a lower-degree also had eight terms and cancellation between the one having nearly the same precision. For the two dominant terms. Moreover, the error measure data I used high-precision was only slightly smaller, and the coefficients after using truncated series (12) to subtract out changed dramatically from those of correspond- and exactly integrate two terms of the endpoint ing terms in (11). Therefore it seems likely that singularities. This made equally spaced x values something important is missing from the basis, pre- quite acceptable. Experiments revealed that using venting results from corresponding to economized terms through coefficient c7 gave an approximation truncations of a convergent infinite series. to y(x) that had a worst absolute error of about

June/July 2013 Notices of the AMS 721 6×10−16 at the right endpoint, where the value was approximate implicit . (See [12] for about 1.255. A plot of the residuals revealed that tips on how to use Eureqa for this purpose.) this eight-coefficient approximation has a relative error of about 5 × 10−16, making it silver standard. Additional Tips and Tricks Moreover, this plot was essentially noise, revealing Here are a few additional tips and tricks for using no remaining exploitable residual for IEEE double. Eureqa effectively in conjunction with computer Eureqa did not by itself discover this better basis, algebra or numerical analysis: but that is my fault: I never allowed square roots as 1) To increase the chance of obtaining exact a building block. I should have done a little analysis rational coefficients, reduce your input ex- earlier to wisely suggest a target expression of the p pression over a common denominator, then form f (x, asin(x), (1 − x)(1 + x)). With a square separately tabulate and fit the numerator root in the integrand denominator, I should have and denominator with the Integer Constant anticipated a square root in a good approximate building block checked. antiderivative numerator. Better yet, I could have 2) To possibly include foreseen exact symbolic√ tried the square root that also contains the known irrational constants such as π and 2 cosine term. Using Eureqa effectively for computer in your result, for each such constant make algebra and numerical analysis should be viewed a labeled column of corresponding floating- as a collaboration rather than as a competition. And point values, preferably accurate to 16 or 17 Eureqa is not a black box oracle. You cannot leave digits. For example if a column labeled pi your brain behind. contains 3.1415926535897932 and a column labeled sqrt2 contains 1.414213562373095, Other Possible Approximate Symbolic Results then check the Integer Constant building There are many other possibilities for using Eureqa block and use a target expression of a to find a concise expression or to suggest a good form such as “... = f (pi, sqrt2, . . .)”. These class of expressions that closely approximates a symbolic constants don’t contribute much result that would otherwise have to be presented to the curse of dimensionality because they only graphically or as a table. For example, are invariant, so there is no need to include • Inverse functions and solving parametric alge- more rows on their account. braic equations: Suppose that for the example 3) To see if an unrecognizable floating-point (8) we also or instead wanted an approximate constant in a result is a close approximation symbolic expression for the inverse Dogbert to some unknown relatively simple√ rational or W function: x as a function of W . Eureqa can irrational constant such as 3/7 or 7π, enter search for such expressions if we simply enter all of the computed digits into the remarkable x = f (W ) or, more efficiently for this example, free online Inverse Symbolic [2] or use the Maple identify(. . .) function, which x = sin(f2() W ) + f3(W ). 11 • Solution of differential, integral, delay, and other is an earlier version thereof. kinds of functional equations: For example, sup- 4) As with most regression software, Eureqa pose that we have a system of first-order tends to express results in terms of the ordinary differential equations and a tabulated notoriously ill-conditioned monomial basis 2 3 numerical solution vector [y1(t), y2(t), . . .] for 1, x, x , x ,.... To express your result more ac- curately in terms of an orthogonal t = t0, t1, . . . , tn. Then we can use Eureqa to separately search for good expressions for basis, such as the Chebychev T polynomials, generate columns labeled T ,T ,...,T , the y1(t), y2(t), . . . . If we want tn = ∞, then we 0 1 n should first make a change of independent desired constant n at the tabulated values variable to map that endpoint to a finite value of the independent variables. After 1 in the in a way that doesn’t introduce a singularity column labeled T0 and the sample values elsewhere. of x in the column labeled T1, successive • Implicitization: A major application of com- columns can be computed from these by puter algebra is implicitization of paramet- entering = 2 ∗ x ∗ T1 − T0 in the column la- ric equations defining curves, surfaces, or beled T2, then = 2∗x∗T2 −T1 in the column higher-dimensional manifolds. Implicit rep- labeled T3, etc. You can then use a target resentations are better than parametric for 11 determining whether a point is inside, outside, Inverse Symbolic Calculator can be regarded as symbolic regression of a single floating-point number to an exact nu- or on a closed manifold. However, exact im- meric expression that it approximates well. It would be nice plicitization are known only for if Eureqa included building blocks for at least simple ratio- certain classes of parametric equations. For nal constants, quadratic numbers, and multiples of π or e, other classes we can try using Eureqa to find which is easy to implement.

722 Notices of the AMS Volume 60, Number 6 expression of the form “... = f (T0,T1, . . .)”. the denominator will contain a pole, but they The Tk variables don’t contribute much to still could contain a logarithmic singularity that the curse of dimensionality because they would have to be handled by subtracting or are totally correlated rather than truly inde- dividing it out. pendent. Moreover, these variables have the • It is probably best to choose endpoints that same Complexity as any other variable, thus extend modestly beyond all stationary points preventing them from being discriminated and the real parts of all zeros, including at least against as they would be if you otherwise one fundamental period, if any, with enough equivalently made the target expression be points to resolve the shortest period. “... = f (1, x, x2 − 1, x3 − 3x, . . .)”. • If there is a symmetry and the expression is C∞ at the symmetric point, then it is probably Seamless Integration of Software Packages best to center the data values on that point. It is usually somewhat of a nuisance to work Otherwise it might be more efficient to make between two or more separate software packages that symmetry point be one of the endpoints. compared to working entirely within one that has Even a purely numeric program that searches for all of the necessary features built in. The extra effort zeros, singularities, extrema, periodicities, and and the lesser chance of even knowing about two or symmetries could help in this regard. more complementary software packages is a great deterrent to such use. Thus it is good to know that Conclusions interfaces are under development for using Eureqa 1) With wise use, Eureqa can sometimes deter- from within Mathematica, MatLab, Python, .Net, mine a simpler exact equivalent expression and Knime, with current versions downloadable better than current computer algebra sys- from [13]. Such added interfaces are rarely as tems or sometimes transform an expression seamless as features that are built in, but they are into a desired form that isn’t provided by a often great improvements over communicating by computer algebra system. export-import of files or by copy and paste. Among 2) With wise use, Eureqa can sometimes suggest other things, the interfaces can accommodate promising forms of expressions that approx- differences in syntax for expressions. imate a result for which an exact closed form There is also a different symbolic regression is unobtainable or excessively complicated. add-on Gptips for MatLab [8] and one named However, often you will want to obtain a DataModeler for Mathematica [4]. more accurate result in a thus-revealed class by using a linear or pro- The Need for Automation gram built into a computer algebra system At any one time, most of us are amateurs with or a package. most of the software that we use. We need all of the 3) Some simple interface within Eu- help we can obtain. In response to that need, many reqa could increase its utility for the above of the best software packages automate certain two purposes. Interfaces to Eureqa within portions. However, the previous examples were more software packages would encourage not automatic. Training, experience, and judgment more use of Eureqa. Embedding Eureqa were involved in choosing the data points, target within those packages would probably be expression, etc. Some of this could and should be even more seamless. automated. A qualitative analysis program such as 4) Eureqa uses some custom computer alge- the Maxima program described in [17] could help bra and some standard numerical methods in this regard: Given an expression, this program internally. Eureqa would almost certainly attempts to return bounds and indications of benefit from embedding and exploiting a monotonicity, convexity, symmetries, periodicities, well-developed full-fledged computer alge- zeros, singularities, limits, and stationary points. To bra system together with classic regression the extent that this is successful, this information and powerful numerical methods. could be used to choose automatically the function 5) Eureqa’s most frequent and notable suc- blocks and for each independent variable the cesses will probably continue to be with endpoints, the number of samples, and perhaps noisy experimental data, but Eureqa shows even their distribution. For example: promise for purposes (1) and (2) above. • If there are poles, then it is best to automatically 6) Some of the most extreme press praise is convert any tangents to sines divided by cosines, poetic license sound bites, but symbolic then form a reduced common denominator, regression deserves a place in the tool kits then use Eureqa to fit separately the numerator of many , engineers, and and denominator. Neither the numerator nor scientists.

June/July 2013 Notices of the AMS 723 Acknowledgments I thank Michael Schmidt and some referees for their helpful suggestions.

References [1] M. Abramowitz and I. A. Segun, Handbook of Mathematical Functions, Dover Publications, 1965. Professor of Mathematics [2] D. Bailey and J. Borwein, Inverse symbolic calculator, http://isc.carma.newcastle.edu.au/. The Department of Mathematics at ETH Zurich [3] R. Corless, E. Postma and D. R. Stoutemyer, Round- ing coefficients and artificially underflowing terms (www.math.ethz.ch) invites applications for a in nonnumeric expressions, ACM Communications in position in an algorithmic area of mathematics Computer Algebra 45 (1/2), 2011, 17–48. related to optimization. The duties of the future [4] DataModeler, http://www.evolved-analytics.com/?q=node/77. professor include teaching in mathematics and [5] R. Ehrenberg, Science News 181(1), 2012, p. 20, related areas. http://www.sciencenews.org/view/feature/id/ 337207/title/Software_Scientist. We are seeking candidates with an internationally [6] Eureqa building blocks, http://creativemachines. cornell.edu/eureqa_ops. recognized research record and with proven [7] R. L. Graham, D. E. Knuth and O. Patashnik, Con- ability to direct research of highest quality. crete Mathematics, 2nd edition, Addison Wesley, 1994, Expertise and a strong background in optimization Section 3.2. [8] Gptips, and/or computation will be especially appreciated. http://sites.google.com/site/gptips4matlab/. Willingness to teach at all university levels and [9] J. R. Koza, Example of a run of genetic program- to collaborate with colleagues from departments ming, http://www.genetic-programming.com/ gpquadraticexample.html. outside mathematics is expected. [10] J. Mullins, Move over Einstein: Machines will take it from here, New Scientist, March 22, 2011. Together with the colleagues from the department, [11] M. Schmidt and H. Lipson, Distilling free-form natural the new professor will be responsible for under- laws from experimental data, Science 324 (5923) 2009, 81–85. graduate courses in mathematics at ETH Zurich for [12] M. Schmidt and H. Lipson, Symbolic regression of students of mathematics, engineering and natural implicit equations, Genetic Programming Theory sciences, and for graduate courses in the programs and Practice, Volume 7, Chapter 5, 2009, 73–85, http://creativemachines.cornell.edu/sites/ MSc in , MSc in Computational default/files/Schmidt-Final-2009-06-05.pdf. Science and Engineering, MSc in Statistics, and MSc [13] M. Schmidt, Eureqa webpage, in Quantitative Finance (joint degree with the http://creativemachines.cornell.edu/eureqa. [14] SourceForge, University of Zurich). The successful candidate will http://maxima.sourceforge.net/compalg.html. be expected to teach undergraduate level courses [15] D. R. Stoutemyer, Dimensional analysis using (German or English) and graduate level courses computer symbolic mathematics, Journal of Compu- tational Physics 24 (2), 1977, 141–149. (English). [16] , Can the Eureqa symbolic regression program, computer algebra and numerical analysis help each Please apply online at www.facultyaffairs.ethz.ch other? e-print, http://arxiv.org/abs/1203.1023. [17] , Qualitative analysis of mathematical expres- Applications should include a curriculum vitae and a sions using computer algebra, Proceedings of the 1976 ACM Symposium on Symbolic and Algebraic list of publications. The letter of application should Computation, 97–104. be addressed to the President of ETH Zurich, Prof. Dr. Ralph Eichler. The closing date for applications is 30 September 2013. ETH Zurich is an equal oppor- tunity and family friendly employer and is further responsive to the needs of dual career couples. In order to increase the number of women in leading academic positions, we specifically encourage women to apply.

724 Notices of the AMS Volume 60, Number 6