Invitation to embarrassingly parallel computing Barbara J. Breen and Christine E. Weidert Physics Department, The University of Portland, Portland, Oregon 97203 John F. Lindner, Lisa May Walker, Kasey Kelly, and Evan Heidtmann Physics Department, The College of Wooster, Wooster, Ohio 44691 ͑Received 2 September 2007; accepted 18 December 2007͒ A surprising number of physical systems problems are well suited to “embarrassingly parallelЉ computations which do not require complicated algorithms or specialized hardware. As faculty and students at small institutions, we are readily incorporating parallel computing in diverse levels of our curricula and are embracing the opportunity to utilize high performance computing to do cutting edge research. We describe three typical examples: spatiotemporal patterns of one-way coupled oscillators, ray-tracing in curved spacetime, and solar escape as a three-body problem. © 2008 American Association of Physics Teachers. ͓DOI: 10.1119/1.2834738͔

I. INTRODUCTION its descendants͒ that solves a piece of the problem for some parameters. To solve the entire problem, we run multiple jobs Until recently, the idea of parallel computing might have across a range of parameters on remote computers. We seemed intimidating to many physics departments with lim- automate4 the distribution of jobs using a few short scripts, ited computing resources. Specialized supercomputers were which are collections of Unix commands. To demonstrate very expensive and required their own support staff. Further- that distributing the jobs requires only a small knowledge of more, many interesting physical systems are difficult to par- Unix and shell scripting, the Appendix contains a transcript allelize, and the necessary algorithms are the subject of re- of a complete terminal session using Xgrid in a Bash shell to search in computer science. perform a parallel computation. This perception began to change about a decade ago when the first Beowulf clusters1 were developed. These were typi- cally inexpensive personal computers running a Unix oper- II. ONE-WAY COUPLING ating system and connected via Ethernet. What is perhaps still under-appreciated is the richness and diversity of inter- The dynamics of arrays of one-way coupled oscillators5,6 esting physical systems that are easily parallelizeable. There are of much interest. One-way coupling might seem impos- are many problems where inter-computer or inter-processor sible because it appears to violate Newton’s third law. How- communication is unnecessary, and the speed-up of the cor- ever, such systems can be built if the position of one oscil- responding computation can be proportional to the number lator modifies the external force acting on a neighboring of processors used. We refer to such computations as “em- oscillator.7 barrassingly parallel,” not because the technique is embar- We model the oscillators by a bistable potential energy rassing, but because it is embarrassingly simple to imple- function, ment. There is no special infrastructure, no sophisticated ͓ ͔ 1 2 1 4 ͑ ͒ parallel algorithms, and no need to use a message passing V x =− 2 ax + 4 bx , 1 interface. where a and b determine the height and width of the barrier In this article we describe how we are using embarrass- separating the two stable equilibria. Such potentials can ingly parallel computing to study a variety of interesting re- model a wide variety of systems, from a simple light switch search problems that are accessible to students and appropri- to a threshold-mitigated neuron.8 Arrays of such oscillators ate for independent study and the upper-level physics have been studied extensively because they are robust mod- ͑ ͒ curriculum. In one case Portland , we are using a dedicated els of a variety of physical systems, from mattress springs to ͑ ͒ rack-mounted computer cluster. In another case Wooster , neural networks. We add noise to make them more realistic we are using the spare cycles of otherwise idle computer and phenomenologically richer. classrooms. A well-studied equation of motion for the nth oscillator in We do have one advantage: our computers run not just such an array is Unix, but Mac OS X,2 which supports Xgrid,3 the first dis- ͓ ͔ ␥ ␬͑ ͒ ␬͑ ͒ ␰ tributed computing architecture built into a desktop operating mx¨n =−VЈ xn − x˙n + xn − xn+1 + xn − xn−1 + n, system. Xgrid enables the user to add a Mac OS X computer ͑2͒ to a cluster by clicking a checkbox in a system preferences panel. It facilitates the submission and retrieval of jobs and where an overdot denotes time differentiation and a prime can automatically lower its priority if someone else uses a denotes space differentiation, xn is the displacement of the computer in the cluster. Although Xgrid is an advantage, our nth oscillator, m is its mass, ␥ is the damping coefficient, ␬ is ␰ ͓ ͔ main message, including our distribution strategies and the coupling constant, and n t represents the noise. ␬͑ ͒ physical examples, applies to any kind of computer cluster. For one-way coupling, we eliminate the xn −xn+1 term To accomplish our parallel computation, we utilize the C in Eq. ͑2͒. For simplicity, we study the overdamped case and family of programming languages and a Unix operating sys- neglect the inertial term relative to the viscous term, so that tem. We first write a job, a computer program in C ͑or one of the equation of motion thus becomes

347 Am. J. Phys. 76 ͑4&5͒, April/May 2008 http://aapt.org/ajp © 2008 American Association of Physics Teachers 347 Fig. 2. The spacetime complexity C as a function of ␬ and D for an array of length N=161. The terrain colors represent complexity, with blue sea being Fig. 1. Matrix of spacetime plots depicting the oscillator speed S as a func- low and white mountain being high. This complexity metric reliably quan- tion of the coupling strength ␬ and noise D for an array of length N=161. In tifies the behavior in Fig. 1. each plot, time increases to the right and oscillator number increases up- ward. The spacetime behavior is complicated for small coupling and large noise, but is simple for large coupling and small noise. grate the set of stochastic differential equations at different points in their parameter space. Each calculation is indepen- dent of the other and does not require messages to be passed ␥ ͓ ͔ ␬͑ ͒ ␰ ͑ ͒ x˙n =−VЈ xn + xn − xn+1 + n. 3 between processors. We code a range of coupling values and distribute a small To implement periodic boundary conditions, we couple the range of noise values to each processor in our cluster. Each last oscillator to the first, creating a ring. processor returns a value of C for given values of ␬ and D, Ideal white noise has the same power at all frequencies. which we combine to visualize the complexity as a function However, because viscosity suppresses high frequency dy- of coupling and noise, as in Fig. 2. namics, our noise is colored. We assume it is Gaussian The complexity C quantifies our observations of the spa- shaped and exponentially correlated and characterized by tiotemporal behavior of the one-way coupled arrays. In gen- three interdependent parameters, eral, C is largest for small ␬ and large D and smallest for D = ␴2␶. ͑4͒ large ␬ and small D. Contours of constant complexity are parabolic, which is likely related to the creation and annihi- The noise intensity D is a measure of the noise power per lation of soliton pairs by intermediate values of the noise and 2 unit frequency. The variance, ␴ , is the square of the width of coupling and is the focus of ongoing research. the noise probability distribution. The correlation time, ␶,is The ability to sweep the parameter space of such a non- the time over which the noise values are closely related. linear system to assess its range of behavior is extremely For sufficiently small noise, the array of oscillators useful. Modeling this stochastic system provides an excellent evolves toward a quiescent steady state in which adjacent example for an upper-level course in nonlinear dynamics. oscillators are in opposite wells and the restoring and cou- pling forces are balanced. However, this spatially alternating III. SPINNING BLACK HOLES equilibrium is frustrated in an array with an odd number of elements, resulting in a solitary wave, or soliton, that propa- Black holes are typically formed by the collapse of mas- gates endlessly around the ring. sive stars. In general relativity, the consequent spacetime cur- We numerically integrate the stochastic equations of Eq. vature is so great that the interior of the black hole becomes ͑3͒ using the Euler–Maruyama algorithm, which is the Euler causally disconnected from the rest of the universe. Light algorithm applied to stochastic equations. For such equa- bent around but not into a black hole can cause lensing and tions, higher order algorithms must be modified.9 We gener- other visual distortions, which are a hindrance and an oppor- ate the colored noise using the algorithm of Fox et al.10 tunity for astronomers. For example, the interpretation of Figure 1 is a matrix of spacetime plots of the oscillator black hole accretion disk observations requires and reveals speed S at different coupling strengths and noise intensities, detailed knowledge of how black holes affect spacetime. with the initial transient dynamics omitted. For small D and Black holes are completely determined by their mass, large ␬, and an odd number N of elements, only a single spin, and charge. The Kerr–Newman solution12 to Einstein’s soliton propagates around the ring. The soliton speed in- equations completely describes the corresponding spacetime. creases with increasing coupling. For intermediate noise and The solution can be expressed as a generalization of the coupling, soliton pairs are created and annihilated. For large Pythagorean theorem. In flat Euclidean space, the distance dr D and small ␬, all the solitons disintegrate into random os- between two nearby points separated by coordinate differ- cillator motion. We quantify this activity by defining the ences dx, dy, and dz is given by complexity C=͗S͘, which is a kind of spatiotemporal aver- dr2 = dx2 + dy2 + dz2. ͑5͒ age of the oscillator speed.11 Determining the complexity C across a range of coupling Einstein and Minkowski generalized this metric to flat space- and noise is an embarrassingly parallel procedure. We inte- time, where the interval ds between two nearby events, sepa-

348 Am. J. Phys., Vol. 76, Nos.4&5,April/May 2008 Breen et al. 348 rated by coordinate differences dt, dx, dy, and dz, is given by ds2 =−͑cdt͒2 + dx2 + dy2 + dz2, ͑6͒ where c is the speed of light. In spherical coordinates and relativistic units ͑where Newton’s gravitational constant G and the speed of light are set to unity͒, Eq. ͑6͒ becomes ds2 =−dt2 + dr2 + ͑rd␪͒2 + ͑r sin ␪d␸͒2. ͑7͒ In curved spacetime, Eq. ͑7͒ generalizes to 2 ␣ ␤ ds = g␣␤dx dx , ͑8͒ ␣ where g␣␤ is the metric tensor, x represents the spacetime coordinates; a double sum over the repeated indices is im- Fig. 3. Schematic of the ray tracing algorithm applied to four adjacent plied. In Kerr–Newman spacetime, pixels. For efficiency, light rays terminating on all pixels are traced back- ward from the camera to the background. Q2 −2Mr ⌺ ds2 = ͩ−1− ͪdt2 + ͩ ͪdr2 − ⌺d␪2 ⌺ ⌬ As in any ray-tracing problem, this procedure is embar- ͑r2 + a2͒2 − ⌬2 sin2 ␪ rassingly parallel. Each processor in the cluster maps one + ͩ ͪsin2 ␪d␸2 ⌺ portion of the image. For example, a simple script can dis- tribute the task of mapping a 400ϫ400 pixel image over 40 Q2 −2Mr + ͩ ͪa sin2 ␪ dt d␸, ͑9͒ processors at ten lines per processor. Different lines of pix- ⌺ els, such as those containing center rays that interact most strongly with the black hole, may require more computation where time. In this case we manually assign more processors to ⌬ = r2 −2Mr + a2 + Q2, ͑10a͒ them. We employ abstract background images to elucidate the optical distortions when a black hole is interposed between ⌺ = r2 + a2 cos2 ␪, ͑10b͒ the camera and the background.15 The background image on / ͑ ͒ the left side of Fig. 4 is an array of squares whose hues vary a = J M. 10c monotonically with angle and whose brightnesses increase M is the mass of the black hole, J is its angular momentum, monotonically from inside out. This coloring enables the and Q is its charge.12 The off-diagonal dt d␸ term of Eq. ͑9͒ mapping of the radius, orientation, and horizontal and verti- models the swirling of spacetime in the vicinity of the black cal edges. We impose a regular blue and yellow checker- hole, the famous frame-dragging effect.13 board pattern on the immediate exterior of the black hole’s Einstein also generalized the law of inertia of flat space, event horizon to track the rays that pass through it. The resulting Escher-like16 optical effects are shown in the k d2x camera image on the right side of Fig. 4. Every line through =0, ͑11͒ dt2 the center of the background image is repeated twice in a line through the center of the camera image, once on either side to the geodesic equation of curved spacetime, of the black hole. The black center square of the background 2 ␣ ␮ ␯ image is mapped into a concentric annulus in the camera d x ␣ dx dx =−⌫ , ͑12͒ image. The four external corners of the background image ␭2 ␮␯ ␭ ␭ d d d appear twice in the camera image at the four interior and four where again there is an implied sum over the repeated indi- exterior cusps. The event horizon’s checkerboard pattern is ces. The connection coefficients, distorted so that the north and south poles of the black hole are both visible ͑multiple times͒ due to the bending of the gץ gץ gץ 1 ⌫␣ ␣␤ͩ ␤␮ ␯␤ ␮␯ ͪ ͑ ͒ light rays by the curved spacetime. The vertical spin axis ␮␯ = g + − , 13 x␤ induces a left-right asymmetry as the right side of the blackץ x␮ץ x␯ץ 2 hole moves away from us and the left side moves toward us. ␣␤ quantify the spacetime curvature, and g is the inverse of Such visualizations provide novel and valuable insight the metric g␣␤. ͑For a massive particle, the parameter ␭ of into spacetime curvature. They may be introduced as com- Eq. ͑12͒ would be the proper time ␶.͒ putational aids in relativity courses to elucidate the basic In principle, we would numerically integrate the four phenomena and to provide an exciting way to incorporate second-order equations of Eq. ͑12͒ to find the light ray paths contemporary research into the curriculum. or null geodesics. In practice, we exploit several constants of ␣ ␣ / ␭ the motion and introduce the four-velocity u =dx d to IV. SOLAR ESCAPE integrate five first-order differential equations. For efficiency, we trace the null geodesics in reverse to map the light rays We are studying a special case of the three-body problem from the camera to the background, both of which are far in which two primary massive bodies, which we call Sun and from the black hole, as shown schematically in Fig. 3. Be- Earth, orbit their common center of mass and a projectile of cause the null geodesics can be highly curved near the black infinitesimal mass is launched radially from one of the mas- hole, we employ an adaptive fourth-order Runge–Kutta algo- sive bodies.17 We construct a model using Newton’s laws and rithm which provides higher accuracy at larger curvature.14 then numerically integrate millions of trajectories. We plot

349 Am. J. Phys., Vol. 76, Nos.4&5,April/May 2008 Breen et al. 349 Fig. 4. When an uncharged, spinning black hole is placed in front of the background image on the left, the result is the Escher-like camera image on the right. In order to map the radius, orientation, and edges, the background image is an array of squares whose hues vary with angle and whose brightness increases from inside out. The blue and yellow checkerboard pattern represents the event horizon of the black hole. The vertical spin axis induces the left-right asymmetry. Every line through the center of the background image is repeated twice in a line through the center of the camera image, once on either side of the black hole. The parameters are M =1, a=J/M =1/ͱ2, and Q=0.

the distance of the projectile from the center of mass after a GM GM E ͑ ␻ ͒ S ͑ ␻ ͒ ͑ ͒ constant time versus the initial launch angle ␸ and launch y¨ =− y − LE sin t − y + LS sin t , 14b d3 d3 energy ⑀ ͑see Fig. 5͒. These plots underscore the chaotic E S nature of the three-body problem and help us uncover fami- where M is the mass of Earth; L is the radial distance of lies of special orbits. E E Earth from the center of mass; dE is the current distance from We assume the primaries orbit in concentric coplanar the projectile to Earth; and M , L , and d are the corre- ␻ S S S circles at constant angular speed . We launch the projectile sponding quantities for Sun. The angular frequency ␻ into the plane of the primaries. By Newton’s laws of motion =2␲/T , where T is one year. We freely vary the ratio of the ͕ ͖ 0 0 and gravity, the position x,y of the projectile satisfies masses of Sun and Earth. In the restricted three-body problem, energy is not strictly GM GM conserved because the two primary massive bodies are unaf- x¨ =− E ͑x − L cos ␻t͒ − S ͑x + L cos ␻t͒, ͑14a͒ d3 E d3 S fected by the gravitational attraction of the projectile’s infini- E S tesimal mass. Fortunately, each trajectory conserves the Ja- cobi constant,18 which is essentially the pseudoenergy per unit mass, 1 GM GM 1 3 ⑀ ͑ 2 2 ͒ ͩ E S ͪ ␻2͑ 2 2 ͒ = x˙R + y˙R − + − xR + yR + , 2 dE dS 2 2 ͑15͒ ͕ ͖ where xR ,yR is the projectile position in coordinates rotat- ing with the primaries.17 We use the pseudoenergy per unit mass as a parameter for specifying the initial conditions, and we use its constancy along each trajectory to check the va- lidity of the numerical integration. Given the initial launch angle ␸ and the pseudoenergy ⑀, we derive the launch speed v0. We numerically integrate Eq. ͑ ͒ 14 for a fixed time T and compute the displacement dC of the projectile from the center of mass of the primaries. After benchmarking fourth-order Runge–Kutta, NDSolve from Mathematica, and CVODE from Lawrence Livermore Na- tional Laboratory,19 we used the CVODE integrator because / / Fig. 5. Solar escape initial velocity space plot for mass ratio ME MS =3 7. ͑ ͒ ͕ ␸͖ it is the fastest for a given accuracy. Orbital geometry see text for symbols superimposed on a polar plot v0 , Computing the displacement dC as a function of the initial of the initial velocity. Yellow-hot colors represent displacements dC after ͕ ␸͖ time T=5T , with yellow being near and deep red being far. Green indicates velocity v0 , is an embarrassingly parallel procedure. 0 ␸ Earth collision and blue indicates Sun collision. Varying both v0 and , we assign each processor in our clus-

350 Am. J. Phys., Vol. 76, Nos.4&5,April/May 2008 Breen et al. 350 / / Fig. 6. A close-up of the initial velocity space plot dramatizes chaos in the three-body problem; ME MS =3 7 and T=50T0. Same color palette as in Fig. 5. Note that order and chaos intermix.

ter a subset of initial velocities. Each processor integrates V. CONCLUSION independently of all others. There is no need for interproces- sor communication. The resulting speed-up in generating the We have described several embarrassingly parallel com- puting projects within easy reach of even small physics de- displacement plots is proportional to the number of proces- partments. Many similar projects can be investigated across sors. broad parameter ranges on multiple computers concurrently In Fig. 5 we show a scan over a range of initial velocities with no message-passing. Creating a cluster to study such and color each one according to the corresponding final dis- problems can galvanize a department and the larger aca- placement, using a “yellow-hot” palette that maps small dis- demic community around the possibilities of computational placements to yellows and large displacements to reds. We research across the disciplines. terminate the simulation if the projectile comes too close to Our experience testifies to the excitement that this power- either Sun or Earth, assume a collision, and color the corre- ful tool can provide. When an otherwise idle room of com- sponding initial velocities blue or green. puters becomes a tool for teaching and learning, it is invigo- For small initial speeds the projectile falls back to Earth, rating for both faculty and students. When a new computer and for large initial speeds it escapes quickly. For intermedi- cluster enables teachers and students to collaborate on cut- ate speeds, there is a chaotic region of initial conditions, ting edge research, it is exciting for both them and their where small differences in the initial velocities lead to large department. Parallel computing opens new horizons for both differences in the final displacements. This threefold division education and research. is reminiscent of how the Mandelbrot set illustrates the intri- cacy implicit in a quadratic map.20 ACKNOWLEDGMENTS Figure 6 is a close-up of the chaotic region of the initial velocity space with mass ratio M /M =3/7 and time T This work was supported in part by NSF Grant No. DMR- E S 0649112 and the Clare Boothe Luce Program of the Henry =50T0. In regions where the colors vary rapidly, the system 21 Luce Foundation. We thank Information Technology at The exhibits a sensitive dependence on the initial conditions. College of Wooster for the use of one of its computer labs. Broad bands of similar colors indicate nonchaotic regions. Chaos and order dramatically intermix. The structure of the initial velocity space plot is dominated by hyperbolic, spiral, APPENDIX: SIMPLE EXAMPLE OF A PARALLEL and elliptical patterns, whose details are being studied. For PROCESSING JOB example, for a family of initial conditions that coil into a Here we provide an example of a simple parallel process- spiral, the corresponding spatial orbits wind more and more ing job using a C/Cϩϩ program, a Bash shell script, and about the center of mass of the system. Xgrid.22 The program takes an integer, doubles it, and writes Numerical exploration of this variation of the classic the result to an output file labeled with the input integer. The three-body problem underscores the inherent complexity of shell script distributes a designated number of jobs over mul- even simple nonlinear systems. Exploring this problem com- tiple processors, enabling us to double many integers in par- putationally provides an opportunity to develop significant allel. In the following, we run only two parallel calculations. new insights into the physics of the three-body problem in This example can be generalized to much larger and non- upper-level mechanics courses. trivial jobs.

351 Am. J. Phys., Vol. 76, Nos.4&5,April/May 2008 Breen et al. 351 The following C/Cϩϩ program, Doubler.cpp, can be A second simple shell script can automate the retrieval of the compiled to an executable Doubler. It scans in an integer, files, which is especially useful when there are many of doubles it, creates a file with a name based on the input them. integer, and then prints the output integer to that file: #include ͗stdio.h͘ 1 T. L. Sterling, J. Salmon, D. J. Becker, and D. F. Savarese, How to Build -const argv͓͔͒ a Beowulf: A Guide to the Implementation and Application of PC Clus ء int main ͑int argc, char ͕ ters ͑MIT Press, Cambridge, MA, 1999͒. 2 int input; Unix includes UNIX-like languages, such as Linux. Mac OS X 10.5 is ͑ ͓ ͔ ͒ UNIX 03 certified. sscanf argv 1 , “%i”, &input ; 3 Xgrid is freely available for Mac OS X, version 10.3 and higher. input; 4 Formerly, we employed “sneakernet” and manually started each job onءint output=2 ͓ ͔ each computer. char FileName 256 5 V. In, A. R. Bulsara, A. Palacios, P. Longhini, A. Kho, and J. D. Neff, sprintf͑FileName,“outputគ%i.txt”, input͒; “Coupling-induced oscillations in overdamped bistable systems,” Phys. .FileP=fopen ͑FileName,“w”͒; Rev. E 68, 045102-1–4 ͑2003͒ء FILE fprintf͑FileP,“%i\n”, output͒; 6 J. F. Lindner and A. R. Bulsara, “One-way coupling enables noise- fclose͑FileP͒; mediated spatiotemporal patterns in media of otherwise quiescent multi- stable elements,” Phys. Rev. E 74, 020105͑R͒-1–4 ͑2006͒. return 0; 7 J. Gallagher, “Designing and building arrays of one-way coupled bistable ͖ oscillators ͑The College of Wooster, Ohio, 2007͒, pp. 1–10. 8 A. Longtin, A. Bulsara, and F. Moss, “Sensory information processing by The following shell script can be compiled to an execut- noisy bistable neurons,” Mod. Phys. Lett. B 6͑21͒, 1299–1308 ͑1992͒. able named Doubler.sh. It can only be executed by a 9 P. E. Kloeden and E. Platen, Numerical Solutions of Stochastic Differen- command-line call from a Bash terminal. The shell script tial Equations ͑Springer, Berlin, 1992͒. instructs Xgrid to run Doubler nMax times on the next 10 R. F. Fox, I. R. Gatland, R. Roy, and G. Vemuri, “Fast, accurate algo- available computers in the cluster: rithm for numerical simulation of exponentially correlated colored noise,” Phys. Rev. A 38, 5938–5940 ͑1988͒. 11 #!/bin/bash The complexity is formally defined by Eqs. ͑4͒ and ͑5͒ of Ref. 9. Here, nMax=$1 we first binary filter the time series and then average over a fixed number of soliton periods. n=1 12 C. W. Misner, K. S. Thorne, and J. A. Wheeler, Gravitation ͑Freeman, while ͓$n -le $nMax͔ ;do New York, 1973͒, pp. 875–891. xgrid -job submit ./Doubler $n 13 J. Lense and H. Thirring, “Über den Einfluss der Eigenrotation der Zen- n=$ ͓$n+1͔ tralkörper auf die Bewegung der Planeten und Monde nach der Einstein- ͑ done schen Gravitationstheorie” On the influence of the proper rotation of central bodies on the motions of planets and moons according to Ein- The following Bash terminal session demonstrates the use stein’s theory of gravitation͒, Phys. Z. 19, 156–163 ͑1918͒. of Doubler and Doubler.sh. First, we confirm our Xgrid 14 W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Nu- connection with a test of “Hello, World.” We then call our merical Recipes in C: The Art of Scientific Computing ͑Cambridge U. P., Cambridge, 1992͒, 2nd ed., pp. 710–722. shell script, Doubler.sh, providing it with an nMax of 2. 15 Xgrid assigns job identifier numbers to each iteration of L. M. Walker, “Spacetime whirlpools,” Senior Thesis in Physics, The College of Wooster ͑2007͒. Doubler. We retrieve and display the resulting data files 16 See, in particular, M. C. Escher, “High and Low” ͑1947͒ or “Print Gal- from the command line of the terminal using the job identi- lery” ͑1956͒. fier numbers. 17 N. J. Harmon, C. Leidel, and J. F. Lindner, “Optimal exit: Solar escape as a restricted three-body problem” Am. J. Phys. 71͑9͒, 871–877 ͑2003͒. breen$ xgrid -job run /bin/echo “Hello, World” 18 J. M. A. Danby, Fundamentals of Celestial Mechanics ͑Willmann-Bell, Hello, World Richmond, VA, 1989͒, 2nd ed., pp. 253–254. breen$./Doubler.sh 2 19 S. D. Cohen and A. C. Hindmarsh, “CVODE, a stiff/nonstiff ODE solver ͕ ͖ in C,” Comput. Phys. 10,͑2͒, 138–143 ͑1996͒. jobIdentifier=101; 20 ͑ ͒ ͕ ͖ B. Mandelbrot, Fractal Geometry of Nature Freeman, New York, 1989 . jobIdentifier=102; 21 P. T. Boyd and S. L. W. McMillan, “Parameter space structure in irregular breen$ xgrid -job results -id 101 gravitational scattering,” Phys. Rev. A 46, 6277–6287 ͑1992͒; “Chaotic breen$ xgrid -job results -id 102 scattering in the gravitational three-body problem,” Chaos 3, 507–523 breen$ cat ouputគ1.txt outputគ2.txt ͑1993͒. 2 22 For complete instructions on installing Xgrid on Mac OS X, including 4 the use of batch files instead of shell scripts for job submission, see ͗www.wooster.edu/physics/cowcpp/Xgrid/͘.

352 Am. J. Phys., Vol. 76, Nos.4&5,April/May 2008 Breen et al. 352