General Index

Accelerated convergence of series 172ff. Annealing, method of simulated 398f., 448ff. Accuracy 31f. assessment 459 achievable in minimization 402, 408, 415 for continuous variables 448, 455f. achievable in root finding 357 schedule 449 contrasted with fidelity 843, 851 thermodynamic analogy 448f. CPU different from memory 192 traveling salesman problem 449ff. vs. stability 715, 740, 841, 855 ANSI/ISO C++ standard 3 Accuracy parameters 17, 956f. Antonov-Saleev variant of Sobol’ sequence Acknowledgments xiii 314ff. Adams-Bashford-Moulton method 752 Apple xx Adams’ stopping criterion 377 Macintosh 896 Adaptive integration 133, 146, 714, 719ff., Approximate inverse of matrix 60 729ff., 737f., 742, 748, 753f., 799 Approximation of functions 108f. Adaptive Monte Carlo integration 320ff., by Chebyshev polynomials 197f., 524 323ff. Pade« approximant 206ff. Addition, multiple precision 917 by rational functions 209ff. Addition theorem, elliptic integrals 266 by wavelets 605f., 793 ADI (alternating direction implicit) method see also Fitting 858, 872f., 917 Arithmetic Adjoint operator 878 arbitrary precision 891, 916ff. Adobe Illustrator xiv, xx floating point 891 Advective equation 837 IEEE standard 289, 893 AGM (arithmetic geometric mean) 917 rounding 892 Airy function 216, 245, 255 Arithmetic coding 891, 912ff. routine for 255f. Arithmetic-geometric mean (AGM) method Aitken’s delta squared process 172 917 Aitken’s interpolation algorithm 111 Array AIX (IBM) 4 C style 20, 941 Algorithms, non-numerical 891ff. centered subarray of 122 Aliasing 506, 581 classes 20ff., 25ff., 941, 950, 953 see also Fourier transform constructors 26, 29 All-poles model 578 index range xi, 21 see also Maximum entropy method (MEM) one-dimensional 20 All-zeros model 578 resize function not assumed 23 see also Periodogram size function 26, 30 Allocation of storage 941 three-dimensional 27 Alternating-direction implicit method (ADI) two-dimensional 20 858, 872f., 917 unit-offset xi, 21 Alternating series 172f. user-defined library 28ff., 949 Alternative extended Simpson’s rule 138 variable dimension 20 Amoeba 414 zero-offset xi, 21 see also Simplex, method of Nelder and Artificial viscosity 842, 848 Mead Ascending transformation, elliptic integrals Amplification factor 839, 841, 843, 851, 856f. 267 Amplitude error 842 ASCII character set 6, 898, 906, 912 Analog-to-digital converter 823, 896 Assembly language 282 Analyticity 207 Associated Legendre polynomials 257f., 775 Analyze/factorize/operate package 74f., 835 recurrence relation for 258 Anderson-Darling statistic 631f. relation to Legendre polynomials 257 Andrew’s sine 707 Association, measures of 615, 633ff.

972 Index 973

Asymptotic series 173 normalization formula 187 exponential integral 229 recurrence relation 184, 236, 243f., 246f. Attenuation factors 595 reflection formulas 247 Autocorrelation reflection formulas, modified functions in linear prediction 570 252 use of FFT 550 routines for 237ff., 248ff. Wiener-Khinchin theorem 503, 579 routines for modified functions 253f. AUTODIN-II polynomial 900 series for 172, 235 Autonomous differential equations 740f. series for Kν 252 Autoregressive model (AR) see Maximum en- series for Yν 247 tropy method (MEM) spherical 245, 256 Average deviation of distribution 616 turning point 246 Averaging kernel, in Backus-Gilbert method Wronskian 245, 251 818 Best-fit parameters 661, 667, 671, 708f. see also Fitting Beta function 218 acksubstitution 45, 50, 53, 102 B incomplete see Incomplete beta function in band diagonal matrix 57 BFGS algorithm see Broyden-Fletcher-Goldfarb- in Cholesky decomposition 100 Shanno algorithm complex equations 52 −1 Bias, of exponent 31 direct for computing A · B 51 Bias, removal in linear prediction 575 relaxation solution of boundary value prob- Biconjugacy 87 lems 768 Biconjugate gradient method in singular value decomposition 67 Backtracking 431 elliptic partial differential equations 835 in quasi-Newton methods 388 preconditioning 88f., 835 Backus-Gilbert method 818ff. for sparse system 87f., 610 Backward deflation 374 Bicubic interpolation 128f. Bader-Deuflhard method 741, 746f. Bicubic spline 130f. Bairstow’s method 375, 381 Big-endian 306 Balancing 488 Bilinear interpolation 126f. Band diagonal matrix 53, 55ff. Binomial coefficients 218 backsubstitution 57 recurrences for 220 LU decomposition 56f. Binomial probability function 220 multiply by vector 55f. cumulative 234 storage 55 deviates from 294, 299f. Band-pass filter 563, 567 Binormal distribution 642, 701 wavelets 597, 603f. Biorthogonality 87 Bandwidth limited function 506 Bisection 120, 370 Bank accounts, checksum for 904 compared to minimum bracketing 401ff. Bar codes, checksum for 904 minimum finding with derivatives 410 Bartlett window 559 root finding 354, 357f., 363ff., 401, 481 Base of representation 31, 892 BISYNCH 900 BASIC, Numerical Recipes in xvi, 1 Bit 31 Basis functions in general linear least squares reversal in fast Fourier transform (FFT) 676 510f., 536 Bayes’ Theorem 821 Bitwise logical functions 300ff., 900f. Bayesian Block-by-block method 799 approach to inverse problems 810, 822, Block of statements 7 828 Bode’s rule 136 contrasted with frequentist 821 Boltzmann probability distribution 449 vs. historic maximum entropy method 828 Boltzmann’s constant 449 views on straight line fitting 675 bool, specialization of vector interferes with Bays’ shuffle 284 MTL wrapper 953 Bernoulli number 143 Bootstrap method 696f. Bessel functions 235ff., 245ff. Bordering method for Toeplitz matrix 95f. asymptotic form 235, 241 Borland C++ 4 complex 216 Borwein and Borwein method for π 917 continued fraction 245f., 251f. Boundary 166f., 437, 756 double precision 235 Boundary conditions fractional order 235, 245ff. for differential equations 712f. Miller’s algorithm 187, 239 initial value problems 713 modified 241ff. in multigrid method 879f. modified, fractional order 251ff. partial differential equations 519, 830ff., modified, normalization formula 244, 251 859ff. modified, routines for 241ff. for spheroidal harmonics 776 974 Index

two-point boundary value problems 713, Carlson’s elliptic integrals 266f. 756ff. Cash-Karp parameters 721f. Boundary value problems see Differential Casting types, using C++ constructor 23 equations; Elliptic partial differential Cauchy probability distribution see Lorentzian equations; Two-point boundary value probability distribution problems Cauchy problem for partial differential equa- Box-Muller algorithm for normal deviate 293 tions 829f. Bracketing Cayley’s representation of exp(−iHt) 855 of function minimum 354, 401ff., 413 CCITT (Comite« Consultatif International Tel« e-« of roots 352, 354ff., 364, 373, 375, 380, graphique et Tel« ephonique)« 900, 911 401 CCITT polynomial 900 Branch cut, for hypergeometric function 215f. CDROMs, how to order xviii Branching 8 Center of mass 309ff. Break iteration 13 Central limit theorem 663f. Brenner, N.M. 511, 527 Central tendency, measures of 615ff. Brent’s method Change of variable minimization 399f., 406ff., 672 in integration 148ff., 799 minimization, using derivative 400, 410 in Monte Carlo integration 311f. root finding 352, 360, 672 in probability distribution 291ff. Broyden-Fletcher-Goldfarb-Shanno algorithm Characteristic polynomial 401, 430ff. digital filter 566 Broyden’s method 384, 393ff., 396 eigensystems 461, 481 singular Jacobian 396 linear prediction 572 Bubble sort 333 matrix with a specified 379 Bugs of recurrence relation 186 in compilers xiv Characteristics of partial differential equations how to report iv 829 Bulirsch-Stoer Chebyshev acceleration in successive over- algorithm for rational function interpolation relaxation (SOR) 870f. 114f. Chebyshev approximation 94, 134, 195, 196ff. method (differential equations) 214, 276, Clenshaw-Curtis quadrature 202 713f., 717, 727, 729ff., 737, 751 Clenshaw’s recurrence formula 199 method (differential equations), stepsize coefficients for 197 control 729f., 737f. contrasted with Pade« approximation 207 for second order equations 737 derivative of approximated function 195, Burg’s LP algorithm 573 201 Byte 31 economization of series 204ff., 207 for error function 225f. even function 200 C (programming language) and fast cosine transform 524 Numerical Recipes in xvi, 1 gamma functions 247 C++ 7 integral of approximated function 201 compilers 3 odd function 200 control structures 5 polynomial fits derived from 203 deficiencies 16, 23 rational function 209ff. explicit keyword 26 Remes exchange algorithm for filter 566 features 16f. Chebyshev polynomials 161, 196ff. function declaration 17, 931 continuous orthonormality 196f. function definition 17 discrete orthonormality 197 header (.h) file 18, 931 explicit formulas for 196 k namespace 18, 931 formula for x in terms of 205 nature of 16f. Check digit 904 Numerical Recipes in 1 Checksum 891, 898 operator associativity 23f. cyclic redundancy (CRC) 898ff. operator precedence 23f. Cherry, sundae without a 821 static object 19 Chi-by-eye 662 unnamed namespace 19 Chi-square fitting see Fitting; Least squares user-defined conversions 20 fitting using declaration 19 Chi-square probability function 221, 226, using definition 19 626, 665, 808 valarray 20, 27 as boundary of confidence region 699 vectors in xi, 3, 20, 25ff., 941, 950, 953 related to incomplete gamma function 226 Calendar algorithms 1f., 12ff. Chi-square test 625f. Calibration 664 for binned data 625f. Cards, sorting a hand of 333 chi-by-eye 662 Index 975

and confidence limit estimation 699 Communication theory, use in adaptive integra- for contingency table 634ff. tion 732 degrees of freedom 626f. Communications protocol 898 for inverse problems 808 Comparison function for rejection method least squares fitting 664ff. 294f. nonlinear models 686ff. Complementary error function rule of thumb 666 see Error function for straight line fitting 666ff. Complete elliptic integral see Elliptic integrals for straight line fitting, errors in both coor- Complex arithmetic 182ff. dinates 671 avoidance of in path integration 214 for two binned data sets 627 cubic equations 191 unequal size samples 628 linear equations 52f. Chip rate 304 quadratic equations 190 Chirp signal 568 Complex error function 264 Cholesky decomposition 99ff., 434, 467 Complex plane backsubstitution 100 fractal structure for Newton’s rule 371f. operation count 100 path integration for function evaluation pivoting 100 213ff., 275 solution of normal equations 679 poles in 114, 172, 214f., 218, 566, 578, Circulant 597 729 Class Complex systems of linear equations 52f. as data type 7 Compression of data 607, 891, 906ff., 912ff. derived 30, 953 Concordant pair for Kendall’s tau 647f. implementation of vector and matrix classes Condition number 64, 88 25ff., 941, 950, 953 Confidence level 697f., 701ff. matrix 20ff., 25ff., 941, 950, 953 Confidence limits NRMat3d 27 bootstrap method 697f. NRMat 20ff., 25ff., 941, 950, 953 and chi-square 699 NRVec 20ff., 25ff., 941, 950, 953 confidence region, confidence interval 697 NRVec as valarray 27 on estimated model parameters 694ff. vector 20ff., 25ff., 941, 950, 953 by Monte Carlo simulation 695ff. wrapper 29f., 950, 953 from singular value decomposition (SVD) Clenshaw-Curtis quadrature 134, 202, 522, 703 524 Confluent hypergeometric function 216, 251 Clenshaw’s recurrence formula 187ff., 202 Conjugate directions 418f., 425ff. for Chebyshev polynomials 199 Conjugate gradient method stability 187ff. biconjugate 87 Clocking errors 901 compared to variable metric method 430 cn function 273 elliptic partial differential equations 835 Coarse-to-fine operator 875 for minimization 400f., 424ff., 815, 826 Coarse-grid correction 875f. minimum residual method 88 Coding preconditioner 88f. arithmetic 912ff. for sparse system 87ff., 610 checksums 898 and wavelets 610 decoding a Huffman-encoded message Conservative differential equations 737 907 const Huffman 906ff. to protect data 28 run-length 911 protects container, not contents 27f., 30f., variable length code 906 949 Ziv-Lempel 906 const correctness 19f., 27ff., 949 see also Arithmetic coding; Huffman cod- Constrained linear inversion method 811ff. ing Constrained linear optimization see Linear pro- Coefficients gramming binomial 220 Constrained optimization 398 for Gaussian quadrature 152ff. Constraints, deterministic 815ff. for Gaussian quadrature, nonclassical weight Constraints, linear 435 function 163f., 799 Constructor for quadrature formulas 135ff., 799 conversion 29 Column degeneracy 35 for matrices 26f. Column operations on matrix 40, 43f. for vectors 26, 29 Column totals 635 Contingency coefficient C 635f. Combinatorial minimization see Annealing Contingency table 633ff., 649 Comite« Consultatif International Tel« egraphique« statistics based on chi-square 634ff. et Tel« ephonique« (CCITT) 900, 911 statistics based on entropy 637ff. 976 Index

Continued fraction 175ff. Cooley-Tukey FFT algorithm 514 Bessel functions 245f. Co-processor, floating point 896 convergence criterion 177 Copy, deep 30 equivalence transformation 178 Copyright rules xix evaluation 175ff. Cornwell-Evans algorithm 827 evaluation along with normalization condi- Corporate promotion ladder 340 tion 252 Corrected two-pass algorithm 618 even and odd parts 178, 222, 227 Correction, in multigrid method 874 even part 260, 262 Correlation coefficient (linear) 641ff. exponential integral 227 Correlation function 503 Fresnel integral 260 autocorrelation 503, 551f., 570 incomplete beta function 232 and Fourier transforms 503 incomplete gamma function 222 theorem 503, 550 Lentz’s method 177, 224 treatment of end effects 551 modified Lentz’s method 177 using FFT 550f. Pincherle’s theorem 187 Wiener-Khinchin theorem 503, 579 ratio of Bessel functions 251 Correlation, statistical 614f., 633 rational function approximation 176, 222, Kendall’s tau 645, 647ff. 232 linear correlation coefficient 641ff., 669 recurrence for evaluating 176f. linear related to least square fitting 641, and recurrence relation 187 669 sine and cosine integrals 262 nonparametric or rank statistical 644ff. Steed’s method 176f. among parameters in a fit 668, 678, 681 tangent function 175 in random number generators 281 typography for 175 Spearman rank-order coefficient 645f. Continuous variable (statistics) 633 sum squared difference of ranks 645 Control structures 7ff. Cosine function, recurrence 184 bad 14 Cosine integral 259, 261ff. Conventions in C++ programs 23ff. continued fraction 262 Convergence routine for 262f. accelerated, for series 172ff. series 262 of algorithm for π 916 Cosine transform see Fast Fourier transform criteria for 357, 402f., 415, 495, 500, (FFT); Fourier transform 689f., 770 Coulomb wave function 216, 245 eigenvalues accelerated by shifting 482f. Courant condition 840, 843, 845, 847 golden ratio 358, 411 multidimensional 857 of golden section search 402f. Courant-Friedrichs-Lewy stability criterion see of Levenberg-Marquardt method 689f. Courant condition linear 357, 404 Covariance of QL method 482f. a priori 711 quadratic 60, 362, 368f., 420f., 431, 916 in general linear least squares 678, 682 rate 357, 363f., 368f. matrix, by Cholesky decomposition 101, recurrence relation 187 678 of Ridders’ method 362 matrix, of errors 807, 819 series vs. continued fraction 176 matrix, is inverse of Hessian matrix 690 and spectral radius 867ff., 873 matrix, when it is meaningful 699ff. Conversions, user-defined 20 in nonlinear models 690, 692 Convex sets, use in inverse problems 815f. relation to chi-square 699ff. Convolution from singular value decomposition (SVD) denoted by asterisk 503 703 finite impulse response (FIR) 543 in straight line fitting 668 of functions 503, 514 CR method see Cyclic reduction (CR) of large data sets 548f. Cramer’s V 635f. for multiple precision arithmetic 920f. Crank-Nicolson method 850, 855, 857 multiplication as 920f. CRC (cyclic redundancy check) 898ff. necessity for optimal filtering 547 CRC-12 900 overlap-add method 549 CRC-16 polynomial 900 overlap-save method 549 CRC-CCITT 900 and polynomial interpolation 123 Creativity, essay on 8 relation to wavelet transform 597 Critical (Nyquist) sampling 505, 555 theorem 503, 543ff., 558 Cross (denotes matrix outer product) 76 theorem, discrete 543ff. Crosstabulation analysis 634 treatment of end effects 545 see also Contingency table use of FFT 534f., 543ff. Crout’s algorithm 47ff., 56 wraparound problem 545 Cubic equations 189ff., 371 Index 977

Cubic spline interpolation 116ff. Deflation see also Spline of matrix 483 Cumulative binomial distribution 231, 234 of polynomials 373ff., 381f. Cumulative Poisson function 226 Degeneracy of linear algebraic equations 35, related to incomplete gamma function 226 64, 68, 681 Curvature matrix see Hessian matrix Degenerate kernel 796 Cycle, in multigrid method 876 Degenerate minimization principle 806 Cyclic Jacobi method 471 Degrees of freedom 626f., 665, 701 Cyclic reduction (CR) 859f., 863f. Demonstration programs 3 Cyclic redundancy check (CRC) 898ff. Derivatives Cyclic tridiagonal systems 77f. computation via Chebyshev approximation 195, 201 computation via Savitzky-Golay filters 195, 656 Danielson-Lanczos lemma 509f., 537 matrix of first partial see Jacobian determi- Data nant assigning keys to 899 matrix of second partial see Hessian ma- continuous vs. binned 625 trix entropy 637ff., 906f. numerical computation 192ff., 390, 656, essay on 614 743, 761, 782 fitting 661ff. of polynomial 179f. fraudulent 666 use in optimization 399f., 410 glitches in 664 Derived class 30, 953 iid (independent and identically distributed) DES see Data Encryption Standard 697 Descending transformation, elliptic integrals modeling 661ff. 267 serial port 902 Descent direction 387, 394, 431 smoothing 615, 655ff. Descriptive statistics 614ff. statistical tests 614ff. see also Statistical tests unevenly or irregularly sampled 581, Design matrix 656, 676, 806, 811f. 586f., 659 Determinant 37, 52 use of CRCs in manipulating 899 Deviates, random see Random deviates windowing 558ff. DFP algorithm see Davidon-Fletcher-Powell see also Statistical tests algorithm Data compression 607, 891 Diagonal dominance 54, 689, 791, 867 arithmetic coding 912ff. Difference equations, finite see Finite differ- cosine transform 524 ence equations (FDEs) Huffman coding 906ff., 912 Difference operator 173 linear predictive coding (LPC) 576f. Differential equations 712ff. lossless 906 accuracy vs. stability 715, 740 Data Encryption Standard (DES) 304ff. Adams-Bashforth-Moulton schemes 752 Data type 31 adaptive stepsize control 714, 719ff., accuracy parameters for 17, 956f. 729f., 737f., 743, 748, 753f. DAUB4 597ff., 600, 602f., 605 algebraically difficult sets 774 DAUB6 598 backward Euler’s method 740 DAUB12 609 Bader-Deuflhard method for stiff 741, DAUB20 602f. 746f. Daubechies wavelet coefficients 597ff., 601, boundary conditions 712f., 756ff., 760, 602f., 605, 609 763, 782f. Davidon-Fletcher-Powell algorithm 401, 430f. Bulirsch-Stoer method 214, 276, 713f., Dawson’s integral 264f., 611 717, 727, 729ff., 751 approximation for 264 Bulirsch-Stoer method for conservative routine for 265 equations 737 D.C. (direct current) 503 comparison of methods 713f., 751, 755 Debugging 7 conservative 737 DEC (Digital Equipment Corp.) xx, 896 danger of too small stepsize 725 Decomposition see Cholesky decomposition; eigenvalue problem 759, 775ff., 782ff. LU decomposition; QR decomposition; embedded Runge-Kutta method 720f., 743 Singular value decomposition (SVD) equivalence of multistep and multivalue Deconvolution 547, 554 methods 754 see also Convolution; Fast Fourier trans- Euler’s method 713, 715, 740 form (FFT); Fourier transform forward Euler’s method 740 Defect, in multigrid method 874 free boundary problem 759, 788 Deferred approach to the limit see Richard- high-order implicit methods 741ff. son’s deferred approach to the limit implicit differencing 740f., 752 978 Index

initial value problems 713 Disclaimer of warranty xvii internal boundary conditions 787ff. Discordant pair for Kendall’s tau 648 internal singular points 787ff. Discrete convolution theorem 543ff. interpolation on right-hand sides 120 Discrete Fourier transform (DFT) 505ff. Kaps-Rentrop method for stiff 741 as approximate continuous transform 508 local extrapolation 720 see also Fast Fourier transform (FFT) modified midpoint method 727f., 730 Discrete optimization 448ff. multistep methods 751ff. Discriminant 190, 469 multivalue methods 751 Dispersion 842 order of method 715f., 729f. DISPO see Savitzky-Golay filters path integration for function evaluation Dissipation, numerical 841 213ff., 275 Divergent series 173 predictor-corrector methods 713, 742, Division 751ff. complex 183 reduction to first-order sets 712, 756 multiple precision 922 relaxation method 757f., 765ff. of polynomials 181, 373, 381 relaxation method, example of 775ff. dn function 273 r.h.s. independent of x 740 Do-while iteration 13 Rosenbrock methods for stiff 741 Dogleg step methods 396 Runge-Kutta method 713f., 715ff., 719ff., Domain of integration 166f. 742, 751 Dominant solution of recurrence relation 185 Runge-Kutta method, high-order 716 Dot (denotes matrix multiplication) 36 Runge-Kutta-Fehlberg method 720f. Double exponential error distribution 706 scaling stepsize to required accuracy 721f. Double precision second order 737 as default 17, 957 semi-implicit differencing 741 as refuge of scoundrels 892 semi-implicit Euler method 741, 746 DP type 3, 17, 22, 939, 957 semi-implicit extrapolation method 741, Double root 352 746 Downhill simplex method see Simplex, method semi-implicit midpoint rule 746 of Nelder and Mead shooting method 757, 760ff. DP type (double precision) 3, 17, 22, 939, shooting method, example 782ff. 957 similarity to Volterra integral equations Driver programs 3 796f. Dual viewpoint, in multigrid method 885 singular points 729f., 763, 787ff. Duplication theorem, elliptic integrals 267 step doubling 720 DWT (discrete wavelet transform) see Wavelet stepsize control 714, 719ff., 729, 737f., transform 743, 748, 753f. stiff 714, 739ff. E ardley, D.M. 349 stiff methods compared 750 EBCDIC 900 Stoermer’s rule 737 Economization of power series 204ff., 207 see also Partial differential equations; Two- Eigensystems 461ff. point boundary value problems balancing matrix 488 Diffusion equation 829, 849ff., 866 bounds on eigenvalues 61 Crank-Nicolson method 850, 855, 857 calculation of few eigenvectors or eigenval- Forward Time Centered Space (FTCS) ues 466, 499 849ff., 852f., 866 canned routines 466 implicit differencing 850 characteristic polynomial 461, 481 multidimensional 857f. completeness 462 Digamma function 227 defective 462, 488, 499 Digital filtering see Filter deflation 483 Dihedral group D5 904 degenerate eigenvalues 461, 463 Dimensions (units) 689 elimination method 465, 490 Diminishing increment sort 334 factorization method 465 Dirac delta function 297, 791 fast Givens reduction 475 Direct method see Periodogram generalized eigenproblem 467 Direct methods for linear algebraic equations Givens reduction 474f. 38 Hermitian matrix 486f. Direct product see Outer product of matrices Hessenberg matrix 465, 482, 487ff., 499 Direction of largest decrease 421f. Householder transformation 465, 474ff., Direction numbers, Sobol’s sequence 315 481, 485, 486, 490 Direction-set methods for minimization 400, ill-conditioned eigenvalues 488 417ff. implicit shifts 483ff. Dirichlet boundary conditions 831, 851, 861, and integral equations 790ff., 796 867, 869 invariance under similarity transform 464 Index 979

inverse iteration 467, 481, 488, 498ff. multigrid method 835, 873ff. Jacobi transformation 465, 468ff., 475, rapid (Fourier) method 835, 859ff. 486, 500 relaxation method 834, 865ff. left eigenvalues 463 strongly implicit procedure 835 list of tasks 466 successive over-relaxation (SOR) 868ff., multiple eigenvalues 500 873, 877 nonlinear 467 Emacs, GNU xiv nonsymmetric matrix 487ff. Embedded Runge-Kutta method 720f., 743 operation count of balancing 488 Encapsulation, in programs 7f. operation count of Givens reduction 475 Encryption 304 operation count of Householder reduction Entropy 906f. 479 of data 637ff., 822 operation count of inverse iteration 499 EOM (end of message) 912 operation count of Jacobi method 472 Equality constraints 435 operation count of QL method 482, 485 Equations operation count of QR method for Hessen- cubic 189ff., 371 berg matrices 495 normal (fitting) 656, 677ff., 812 operation count of reduction to Hessenberg quadratic 32, 189ff. form 490 see also Differential equations; Partial dif- orthogonality 462 ferential equations; Root finding polynomial roots and 379 Equivalence classes 348f. QL method 481ff., 486, 499f. Equivalence transformation 178 QL method with implicit shifts 483ff. Error QR method 63, 465, 468, 481ff. checksums for preventing 901 QR method for Hessenberg matrices 491ff. clocking 901 real, symmetric matrix 162, 479, 796 double exponential distribution 706 reduction to Hessenberg form 489f. local truncation 885 right eigenvalues 463 Lorentzian distribution 707 shifting eigenvalues 461, 482f., 492 in multigrid method 874 special matrices 466 nonnormal 664, 700, 704ff. termination criterion 495, 500 relative truncation 885 tridiagonal matrix 465, 481ff., 499 roundoff 191, 891f. Eigenvalue and eigenvector, defined 461 series, advantage of an even 143, 728 Eigenvalue problem for differential equations systematic vs. statistical 664 759, 775ff., 782, 784 truncation 33, 192, 411, 720, 891f. Eigenvalues and polynomial root finding 379 varieties found by check digits 904 EISPACK 466, 487 varieties of, in PDEs 842ff. Electromagnetic potential 530 see also Roundoff error Elimination see Gaussian elimination Error function 220f., 612 Ellipse in confidence limit estimation 699 approximation via sampling theorem 612 Elliptic integrals 265ff., 917 Chebyshev approximation 225f. addition theorem 267 complex 264 Carlson’s forms and algorithms 266ff. for Fisher’s z-transformation 643 Cauchy principal value 268 relation to Dawson’s integral 264 duplication theorem 267 relation to Fresnel integrals 260 Legendre 266ff., 271ff. relation to incomplete gamma function routines for 268ff. 225 symmetric form 266f. routine for 225f. Weierstrass 266 for significance of correlation 641 Elliptic partial differential equations 829 for sum squared difference of ranks 646 alternating-direction implicit method (ADI) Error handling in programs 2, 941 872f., 917 Estimation of parameters see Fitting; Maxi- analyze/factorize/operate package 835 mum likelihood estimate biconjugate gradient method 835 Estimation of power spectrum 555ff., 577ff. boundary conditions 830f. Euler equation (fluid flow) 842 comparison of rapid methods 865 Euler-Maclaurin summation formula 143, 146 conjugate gradient method 835 Euler’s constant 228f., 262 cyclic reduction 859f., 863f. Euler’s method for differential equations 713, Fourier analysis and cyclic reduction (FACR) 715, 740 859ff., 865 Euler’s transformation 172ff. Gauss-Seidel method 866, 875ff., 886 generalized form 174f. incomplete Cholesky conjugate gradient Evaluation of functions see Function method (ICCG) 835 Even and odd parts, of continued fraction Jacobi’s method 866f., 875 178, 222, 227 matrix methods 835 Even parity 898 980 Index

Exception handling in programs 2, 941 cosine transform, second form 524, 863 exit() function 2 Danielson-Lanczos lemma 509f., 537 Explicit differencing 838 data sets not a power of 2 514 explicit keyword 26 data smoothing 655 Exponent in floating point format 31, 892 data windowing 558ff. Exponential deviate 291f. decimation-in-frequency algorithm 514 Exponential integral 227ff. decimation-in-time algorithm 514 asymptotic expansion 229 discrete autocorrelation 551f. continued fraction 227 discrete convolution theorem 543ff. recurrence relation 184 discrete correlation theorem 550 related to incomplete gamma function 227 at double frequency 587 relation to cosine integral 262 endpoint corrections 590f. routine for Ei(x) 230 external storage 536f. routine for En(x) 228f. figures of merit for data windows 559 series 227 filtering 563ff. Exponential probability distribution 582 FIR filter 564f. Extended midpoint rule 135, 140, 146 Fourier integrals 589ff. Extended precision, use in iterative improve- Fourier integrals, infinite range 595 ment 58 Hamming window 559 Extended Simpson’s rule 138, 798, 801 Hann window 559 Extended Simpson’s three-eighths rule 799 history 509 Extended trapezoidal rule 135, 138, 141ff., IIR filter 565f. 146, 797 image processing 814, 816 roundoff error 143 integrals using 134 Extirpolation (so-called) 586f. inverse of cosine transform 518f. Extrapolation 108ff. inverse of sine transform 522 in Bulirsch-Stoer method 729ff., 736 large data sets 536f. differential equations 713 leakage 557 by linear prediction 569ff. memory-local algorithm 540 local 720 multidimensional 526ff. maximum entropy method as type of 579 for multiple precision arithmetic 916 polynomial 733, 735, 751 for multiple precision multiplication 921 rational function 729ff., 736 number-theoretic transforms 514 relation to interpolation 108 operation count 509 for Romberg integration 144 optimal (Wiener) filtering 552ff., 570f. see also Interpolation order of storage in 512 Extremization see Minimization partial differential equations 835, 859ff. Parzen window 559 F -distribution probability function 231, 234 periodicity of 508 F-test for differences of variances 622, 624 periodogram 556ff., 579 FACR see Fourier analysis and cyclic reduc- power spectrum estimation 555ff. tion (FACR) for quadrature 134 Facsimile standard 911 of real data in 2D and 3D 530ff. Factorial of real functions 515ff., 530ff. double (denoted “!!") 258 related algorithms 513f. evaluation of 171 Sande-Tukey algorithm 514 relation to gamma function 218 sine transform 519ff., 861 routine for 219f. Singleton’s algorithm 536f. False position 358ff. square window 558 Family tree 349 treatment of end effects in convolution FAS (full approximation storage algorithm) 545 884ff. treatment of end effects in correlation 551 Fast Fourier transform (FFT) 509ff., 891 Tukey’s trick for frequency doubling 587 alternative algorithms 513f. use in smoothing data 655 applications 542ff. used for Lomb periodogram 586f. as approximation to continuous transform variance of power spectrum estimate 557, 508 561 Bartlett window 559 virtual memory machine 540 bit reversal 510f., 536 Welch window 559 and Clenshaw-Curtis quadrature 202 Winograd algorithms 514 convolution 514, 534f., 543ff., 921 see also Discrete Fourier transform (DFT); convolution of large data sets 548f. Fourier transform; Spectral density Cooley-Tukey algorithm 514 Faure sequence 314 correlation 550f. Fax (facsimile) Group 3 standard 911 cosine transform 202, 522ff., 862f. Feasible vector 435 Index 981

FFT see Fast Fourier transform (FFT) confidence levels related to chi-square val- Field, in data record 341 ues 701ff. Figure-of-merit function 661 confidence levels from singular value de- Filon’s method 595 composition (SVD) 703 Filter 563ff. confidence limits on fitted parameters 694ff. acausal 564 covariance matrix not always meaningful bilinear transformation method 566 662, 700 causal 564, 655 degeneracy of parameters 685 characteristic polynomial 566 an exponential 684 data smoothing 655 freezing parameters in 679, 711 digital 563ff. Gaussians, a sum of 692f. DISPO 655 general linear least squares 676ff. by fast Fourier transform (FFT) 534f., Kalman filter 711 563ff. KÐS test, caution regarding 632 finite impulse response (FIR) 543, 564f. least squares 662ff. homogeneous modes of 566 Legendre polynomials 685 infinite impulse response (IIR) 565f., 578 Levenberg-Marquardt method 688ff., 827 Kalman 711 linear regression 666ff. linear 564ff. maximum likelihood estimation 663, low-pass for smoothing 655 704ff. nonrecursive 564f. Monte Carlo simulation 632, 665, 695ff. optimal (Wiener) 547, 552ff., 570f., 655 multidimensional 686 quadrature mirror 597, 604 nonlinear models 686ff. realizable 564, 567 nonlinear models, advanced methods 693 recursive 565f., 578 nonlinear problems that are linear 684 Remes exchange algorithm 566 nonnormal errors 667, 700, 704ff. Savitzky-Golay 195, 655ff. polynomial 93, 123, 203, 655f., 676, 685f. stability of 566 by rational Chebyshev approximation 209ff. in the time domain 563ff. robust methods 704ff. Fine-to-coarse operator 875 of sharp spectral features 578 Finite difference equations (FDEs) 765, 774, standard (probable) errors on fitted parame- 786 ters 668, 672f., 678, 682, 694ff. alternating-direction implicit method (ADI) straight line 666ff., 678f., 708f. 858, 872f. straight line, errors in both coordinates art, not science 840 671ff. Cayley’s form for unitary operator 855 see also Error; Least squares fitting; Max- Courant condition 840, 843, 847 imum likelihood estimate; Robust esti- Courant condition (multidimensional) 857 mation Crank-Nicolson method 850, 855, 857 Five-point difference star 878 eigenmodes of 838f. Fixed point format 31 explicit vs. implicit schemes 838 Fletcher-Powell algorithm see Davidon-Fletcher- forward Euler 837f. Powell algorithm Forward Time Centered Space (FTCS) Fletcher-Reeves algorithm 400f., 425ff. 838ff., 849ff., 854, 866 Floating point co-processor 896 implicit scheme 850 Floating point format 31, 892 Lax method 839ff., 847 care in numerical derivatives 192 Lax method (multidimensional) 856f. IEEE 289, 893 mesh drifting instability 845f. Flux-conservative initial value problems 836ff. numerical derivatives 192 FMG (full multigrid method) 874, 879f. partial differential equations 832ff. for iteration 9 in relaxation methods 765ff. Formats of numbers 31, 892 staggered leapfrog method 844f. Fortran 16, 21 two-step Lax-Wendroff method 846ff. Numerical Recipes in xvi, 1 upwind differencing 843f., 848 Forward deflation 374 see also Partial differential equations Forward difference operator 173 Finite element methods, partial differential Forward Euler differencing 837f. equations 835f. Forward Time Centered Space see FTCS Finite impulse response (FIR) 543 Fourier analysis and cyclic reduction (FACR) Finkelstein, S. xiii 860, 865 FIR (finite impulse response) filter 564f. Fourier integrals Fisher’s z-transformation 642f. attenuation factors 595 Fitting 661ff. endpoint corrections 590f. basis functions 676 tail integration by parts 595 by Chebyshev approximation 197f. use of fast Fourier transform (FFT) 589ff. chi-square 664ff. Fourier and spectral applications 542ff. 982 Index

Fourier transform 108, 501ff. see also Inverse problems aliasing 506, 581 Frequency domain 501 approximation of Dawson’s integral 264 Frequency spectrum see Fast Fourier transform autocorrelation 503 (FFT) basis functions compared 519f. Frequentist, contrasted with Bayesian 821 contrasted with wavelet transform 596, Fresnel integrals 259ff. 605 asymptotic form 260 convolution 503, 514, 543ff., 921 continued fraction 260 correlation 503, 550f. routine for 260f. cosine transform 202, 522ff., 862f. series 259 cosine transform, second form 524, 863 Friday the Thirteenth 13f. critical sampling 505, 555, 557 FTCS (forward time centered space) 838ff., definition 501 849ff., 854 discrete Fourier transform (DFT) 196, stability of 838ff., 849ff., 866 505ff. Full approximation storage (FAS) algorithm Gaussian function 611 884ff. image processing 814, 816 Full moon 13f. infinite range 595 Full multigrid method (FMG) 874, 879f. inverse of discrete Fourier transform 508 Full Newton methods, nonlinear least squares method for partial differential equations 693 859ff. Full pivoting 41 missing data 581 Full weighting 878 missing data, fast algorithm 586f. Function Nyquist frequency 505ff., 530, 555, 557, Airy 216, 245, 255 581, 584 approximation 108f., 196ff. optimal (Wiener) filtering 552ff., 570f. associated Legendre polynomial 257f., Parseval’s theorem 503, 509, 556 775 power spectral density (PSD) 503f. autocorrelation of 503 power spectrum estimation by FFT 555ff. bandwidth limited 506 power spectrum estimation by maximum Bessel 184, 216, 235ff., 245ff. entropy method 577ff. beta 220f. properties of 502f. branch cuts of 215f. sampling theorem 506, 555, 557, 611f. chi-square probability 226, 808 scalings of 502 complex 213 significance of a peak in 582f. confluent hypergeometric 216, 251 sine transform 519ff., 861 convolution of 503 symmetries of 502 correlation of 503 uneven sampling, fast algorithm 586f. Coulomb wave 216, 245 unevenly sampled data 580ff., 586f. cumulative binomial probability 231, 234 and wavelets 603f. cumulative Poisson 221 Wiener-Khinchin theorem 503, 571, 579 Dawson’s integral 264f., 611 see also Fast Fourier transform (FFT); declaration 17, 931 Spectral density definition 17 Fractal region 371f. digamma 227 Fractional step methods 858f. elliptic integrals 265ff., 917 Fredholm alternative 791 error 225f., 260, 264, 612, 641, 646 Fredholm equations 790f. evaluation 171ff. eigenvalue problems 791, 796 evaluation by path integration 213ff., 275 error estimate in solution 795 exponential integral 184, 227ff., 262 first kind 790 F-distribution probability 231, 234 Fredholm alternative 791 Fresnel integral 259ff. homogeneous vs. inhomogeneous 791 gamma 218f. homogeneous, second kind 795f. header 17, 931 ill-conditioned 791 hypergeometric 214ff., 275ff. infinite range 800 incomplete beta 231ff., 621 inverse problems 791, 806ff. incomplete gamma 221ff., 626, 665, 668f. kernel 790f. inverse hyperbolic 190, 267 nonlinear 792 inverse trigonometric 267 Nystrom method 793ff., 799f. Jacobian elliptic 265, 273f. product Nystrom method 799 Kolmogorov-Smirnov probability 629f., second kind 791, 793f. 651f. with singularities 799 Legendre polynomial 184, 257f., 685 with singularities, worked example 803 logarithm 267 subtraction of singularity 800 modified Bessel 241ff. symmetric kernel 796 modified Bessel, fractional order 251ff. Index 983

path integration to evaluate 213ff. Gaussian integration 137, 152ff., 800 pathological 109, 354f. calculation of abscissas and weights 155ff. pointer to, passed as argument 23 error estimate in solution 795 Poisson cumulant 226 extensions of 165 prototype 17, 931 Golub-Welsch algorithm for weights and representations of 501 abscissas 162 routine for plotting a 353f. for integral equations 792, 794 sine and cosine integrals 259, 261ff. from known recurrence relation 161f. sn, dn, cn 273 nonclassical weight function 163f., 799 spherical Bessel 245 and orthogonal polynomials 154 spherical harmonics 257f. preassigned nodes 165 spheroidal harmonic 775ff., 782ff. weight function log x 165 Student’s probability 231, 233 weight functions 152ff., 799 Weber 216 Gear’s method (stiff ODEs) 742 Functional iteration, for implicit equations Geiger counter 278 751 Generalized eigenvalue problems 467 FWHM (full width at half maximum) 560 Generalized minimum residual method (GMRES) 88 Geophysics, use of Backus-Gilbert method G++ 4 820 Gamma deviate 294ff. Gerchberg-Saxton algorithm 816f. Gamma function 218ff. Gilbert and Sullivan 725 incomplete see Incomplete gamma func- Givens reduction 474f., 485 tion fast 475 Gauss-Chebyshev integration 153, 156, 161, operation count 475 522, 524 Glassman, A.J. 191 Gauss-Hermite integration 156, 800 Global optimization 398f., 448ff., 661 abscissas and weights 159 continuous variables 455f. normalization 159 Global variables 23 Gauss-Jacobi integration 156 Globally convergent abscissas and weights 160 minimization 430ff. Gauss-Jordan elimination 39ff., 44, 74 root finding 384, 387ff., 394, 760f., 763 operation count 45, 51 GMRES (generalized minimum residual method) solution of normal equations 678 88 storage requirements 41f. GNU Gauss-Kronrod quadrature 165 C++ 4 Gauss-Laguerre integration 156, 800 Emacs xiv Gauss-Legendre integration 156 Godunov’s method 848 see also Gaussian integration Golden mean (golden ratio) 33, 358, 403, 411 Gauss-Lobatto quadrature 165, 202, 522 Golden section search 352, 400ff., 407 Gauss-Radau quadrature 165 Golub-Welsch algorithm, for Gaussian quadra- Gauss-Seidel method (relaxation) 866, 868, ture 162 875ff. Goodness-of-fit 661, 665, 668f., 673, 701 nonlinear 886 goto statements, danger of 9 Gauss transformation 267 Gram-Schmidt Gaussian (normal) distribution 279, 663, 809 biorthogonalization 425f. central limit theorem 663f. orthogonalization 103, 462f. deviates from 292f., 583 SVD as alternative to 69 kurtosis of 617 Graphics, function plotting 353f. multivariate 701 Gravitational potential 530 semi-invariants of 619 Gray code 315, 891, 896ff. tails compared to Poisson 664 Greenbaum, A. 89 two-dimensional (binormal) 642 Gregorian calendar 12, 15 variance of skewness of 617 Grid square 126f. Gaussian elimination 44f., 62, 66 Group, dihedral 904 fill-in 56, 74 Guard digits 892 integral equations 797 operation count 45 in reduction to Hessenberg form 490 Half weighting 878 relaxation solution of boundary value prob- Halton’s quasi-random sequence 313f. lems 765ff., 789 Hamming window 559 Gaussian function Hamming’s motto 352 Hardy’s theorem on Fourier transforms Hann window 559 611 Harmonic analysis see Fourier transform see also Gaussian (normal) distribution Hashing 307 984 Index

HDLC checksum 900 Implicit Header (.h) files 18, 931 function theorem 351 Headers in C++ 17, 931 pivoting 41 Heap (data structure) 339f., 348, 907 shifts in QL method 483ff. Heapsort 332, 339f., 348 Implicit differencing 838 Helmholtz equation 863 for diffusion equation 850 Hermite polynomials 156, 159 for stiff equations 740f., 752 Hermitian matrix 462ff., 486f. Importance sampling, in Monte Carlo 320f. Hertz (unit of frequency) 501 Improper integrals 146ff. Hessenberg matrix 104, 465, 482, 487ff., 499 Impulse response function 543, 554, 564 see also Matrix IMSL xx, 38, 75, 217, 375, 380, 466 Hessian matrix 393, 419, 426, 431, 687ff., In-place selection 346 815, 826 Include files 18, 931 is inverse of covariance matrix 678, 690 Incomplete beta function 231ff. second derivatives in 688 for F-test 624 Hexadecimal constants 289, 307 routine for 232f. Hierarchically band diagonal matrix 609f. for Student’s t 621, 623 Hierarchy of program structure 5ff. Incomplete Cholesky conjugate gradient method High-order not same as high-accuracy 109f., (ICCG) 835 134, 400, 410, 716, 720, 752 Incomplete gamma function 221 High-pass filter 563 for chi-square 626, 665, 668f. Hilbert matrix 93 Historic maximum entropy method 828 deviates from 294ff. Homogeneous linear equations 64 in mode estimation 621 Hook step methods 396 routine for 223f. Hotelling’s method for matrix inverse 60, 610 Increment of linear congruential generator Householder transformation 63, 465, 474ff., 280 481, 485, 486, 490, 493ff. Indentation of blocks 12 operation count 479 Index 972ff. in QR decomposition 102 this entry 984 Huffman coding 576, 891, 906ff., 912 Index table 332, 341 Hyperbolic functions, explicit formulas for Inequality constraints 435 inverse 190 Inheritance 8 Hyperbolic partial differential equations 829 Initial value problems 713, 829f. advective equation 837 see also Differential equations; flux-conservative initial value problems Partial differential equations 836ff. Injection operator 875 Hypergeometric function 214ff., 275ff. Instability see Stability routine for 276f. Integer programming 447 Hypothesis, null 614 Integral equations 790ff. adaptive stepsize control 799 block-by-block method 799 I BM xx correspondence with linear algebraic equa- bad random number generator 281 tions 790ff. radix base for floating point arithmetic degenerate kernel 796 488 eigenvalue problems 791, 796 RS/6000 4 error estimate in solution 795 IBM checksum 904 Fredholm 790f., 793f. ICCG (incomplete Cholesky conjugate gradient Fredholm alternative 791 method) 835 ICF (intrinsic correlation function) model 828 homogeneous, second kind 795f. Identity (unit) matrix 37 ill-conditioned 791 IEEE floating point format 289, 893 infinite range 800 if structure 9 inverse problems 791, 806ff. warning about nesting 9 kernel 790f. IIR (infinite impulse response) filter 565f., nonlinear 792, 798 578 Nystrom method 793f., 799 Ill-conditioned integral equations 791 product Nystrom method 799 Image processing 530, 814 with singularities 799ff. cosine transform 524 with singularities, worked example 803 fast Fourier transform (FFT) 530, 534f., subtraction of singularity 800 814 symmetric kernel 796 as an inverse problem 814 unstable quadrature 798 maximum entropy method (MEM) 821ff. Volterra 791f., 796f. from modulus of Fourier transform 816 wavelets 793 wavelet transform 607f. see also Inverse problems Index 985

Integral operator, wavelet approximation of Gerchberg-Saxton algorithm 816f. 609, 793 incomplete Fourier coefficients 824 Integration of functions 133ff. and integral equations 791 cosine integrals 261 linear regularization 811ff. Fourier integrals 589ff. maximum entropy method (MEM) 821ff., Fourier integrals, infinite range 595 826f. Fresnel integrals 259 MEM demystified 825 Gauss-Hermite 159 Phillips-Twomey method 811ff. Gauss-Jacobi 160 principal solution 808 Gauss-Laguerre 158 regularization 807ff. Gauss-Legendre 156 regularizing operator 809 integrals that are elliptic integrals 265f. stabilizing functional 809 path integration 213ff. Tikhonov-Miller regularization 811ff. sine integrals 261 trade-off curve 806 see also Quadrature trade-off curve, Backus-Gilbert method Integro-differential equations 793 820 Intel 4 two-dimensional regularization 814 Intent in/out/inout 21 use of conjugate gradient minimization Interface, in programs 7, 18, 931 815, 826 Intermediate value theorem 354 use of convex sets 815f. Interpolation 108ff. use of Fourier transform 814, 816 Aitken’s algorithm 111 Van Cittert’s method 815 avoid 2-stage method 109 Inverse quadratic interpolation 364, 406ff. avoid in Fourier analysis 581 Inverse response kernel, in Backus-Gilbert bicubic 128f. method 818f. bilinear 126f. Inverse trigonometric function 267 caution on high-order 109f. ISBN (International Standard Book Number) coefficients of polynomial 109, 123ff., checksum 904 203, 586f. Iterated integrals 167 for computing Fourier integrals 591 Iteration 9 error estimates for 109 functional 751 of functions with poles 114ff. to improve solution of linear algebraic inverse quadratic 364, 406ff. equations 58ff., 207 multidimensional 111, 126ff. for linear algebraic equations 38 in multigrid method 878 required for two-point boundary value Neville’s algorithm 111f., 194 problems 756 Nystrom 794 in root finding 351f. operation count for 109 Iteration matrix 867 operator 875 ITPACK 81 order of 109 and ordinary differential equations 110 oscillations of polynomial 109, 123, 400, Jacobi matrix, for Gaussian quadrature 162 410 Jacobi transformation (or rotation) 104, 465, parabolic, for minimum finding 406f. 468ff., 475, 486, 500 polynomial 108, 111ff., 194 Jacobian determinant 292f., 786 rational Chebyshev approximation 209ff. Jacobian elliptic functions 265, 273f. rational function 108, 114ff., 206ff., 236f., Jacobian matrix 385, 387, 390, 393, 743 729ff., 736 singular in Newton’s rule 396 reverse (extirpolation) 586f. Jacobi’s method (relaxation) 866ff., 875 spline 109, 116ff., 130f. Jenkins-Traub method 380 on sub-range of an array 113, 122 Julian Day 1f., 12, 14f. trigonometric 108 Jump transposition errors 904 see also Fitting Interval variable (statistics) 633 Intrinsic correlation function (ICF) model 828 K -S test see Kolmogorov-Smirnov test Inverse hyperbolic function 190, 267 KAI C++ 4 Inverse iteration see Eigensystems Kalman filter 711 Inverse problems 791, 806ff. Kaps-Rentrop method 741 Backus-Gilbert method 818ff. Kendall’s tau 645, 647ff. Bayesian approach 810, 822, 828 Kermit checksum 899 central idea 810 Kernel 790f. constrained linear inversion method 811ff. averaging, in Backus-Gilbert method 818f. data inversion 818 degenerate 796 deterministic constraints 815ff. finite rank 796 in geophysics 820 inverse response 818f. 986 Index

separable 796 L’Ecuyer’s long period random generator 284ff. singular 799f. Left eigenvalues or eigenvectors 463 symmetric 795f. Legal matters xvii Keys used in sorting 341, 899 Legendre elliptic integral see Elliptic integrals Kolmogorov-Smirnov test 625, 628ff., 705 Legendre polynomials 257f. two-dimensional 650ff. fitting data to 685 variants 631ff., 650ff. recurrence relation 184 Kuiper’s statistic 632 shifted monic 165 Kurtosis 617, 619 see also Associated Legendre polynomials; Spherical harmonics Lehmer-Schur algorithm 380 L-estimate 705 Lemarie’s wavelet 605 Labels, named statement 9 Lentz’s method for continued fraction 177, Lag 503, 550f., 566 224 Lagrange multiplier 806 Lepage, P. 323 Lagrange’s formula for polynomial interpola- Leptokurtic distribution 617 tion 94, 111, 586f., 590 Levenberg-Marquardt algorithm 396, 688ff., Laguerre’s method 352, 371ff. 827 Lanczos lemma 509f. advanced implementation 693 Lanczos method for gamma function 218 Levinson’s method 95f. Landen transformation 267 Lewis, H.W. 289 LAPACK 38 Library Laplace’s equation 257, 829 matrix/vector xi, 20ff., 25ff., 941, 950, see also Poisson equation 953 Las Vegas 636 Latin square or hypercube 319 user-defined for arrays 28ff., 949 Laurent series 578 License information xvii Lax method 839ff., 847, 856f. Limbo 368 multidimensional 856f. Limit cycle, in Laguerre’s method 376 Lax-Wendroff method 846ff. Line minimization see Minimization, along a Leakage in power spectrum estimation 557, ray 559f. Line search see Minimization, along a ray Leakage width 559f. Linear algebraic equations 35ff. Leapfrog method 844f. band diagonal 55ff. Least squares filters see Savitzky-Golay filters biconjugate gradient method 87f. Least squares fitting 655f., 662ff., 666ff., Cholesky decomposition 99ff., 434, 467, 671ff., 676ff. 679 contrasted to general minimization prob- complex 52f. −1 · lems 693 computing A B 51 degeneracies in 682, 685 conjugate gradient method 87ff., 610 Fourier components 582 cyclic tridiagonal 77f. freezing parameters in 679, 711 direct methods 38, 74 general linear case 676ff. Gauss-Jordan elimination 39ff. Levenberg-Marquardt method 688ff., 827 Gaussian elimination 44f. Lomb periodogram 582 Hilbert matrix 93 as maximum likelihood estimator 663 Hotelling’s method 60, 610 as M-estimate for normal errors 706 and integral equations 790ff., 794 as method for smoothing data 655f. iterative improvement 58ff., 207 multidimensional 686 iterative methods 38, 87ff. nonlinear 396, 686ff., 827 large sets of 36 nonlinear, advanced methods 693 least squares solution 65, 68f., 211, 681 normal equations 656, 677ff., 812 LU decomposition 46ff., 207, 396, 743, normal equations often singular 681, 685 794, 797, 812 optimal (Wiener) filtering 552 nonsingular 36 QR method in 103, 679 overdetermined 37f., 211, 681, 808 for rational Chebyshev approximation 211 partitioned 80f. relation to linear correlation 641, 669 QR decomposition 101f., 393, 396, 679 Savitzky-Golay filter as 655f. row vs. column elimination 43f. singular value decomposition (SVD) 37f., Schultz’s method 60, 610 62ff., 211, 681ff. Sherman-Morrison formula 76ff., 93 skewed by outliers 664 singular 35, 64, 69, 211, 681 for spectral analysis 582 singular value decomposition (SVD) 62ff., standard (probable) errors on fitted parame- 211, 681ff., 808 ters 678, 682 sparse 36, 55ff., 74ff., 743, 815 weighted 663 summary of tasks 37 see also Fitting Toeplitz 93, 95ff., 207 Index 987

Vandermonde 93ff., 123 Local extremum 398, 449 wavelet solution 609f., 793 Localization of roots see Bracketing Woodbury formula 78ff., 93 Logarithmic function 267 see also Eigensystems Lomb periodogram method of spectral analysis Linear congruential random number generator 581f. 280f. fast algorithm 586f. choice of constants for 288 Loops 9 Linear constraints 435 Lorentzian probability distribution 296, 707 Linear convergence 357, 404 Low-pass filter 563, 655 Linear correlation (statistics) 641ff. LP coefficients see Linear prediction Linear dependency LPC (linear predictive coding) 576f. constructing orthonormal basis 69, 103 LU decomposition 46ff., 58f., 62, 66, 74, of directions in N-dimensional space 420 107, 385, 678, 743 −1 in linear algebraic equations 35f. for A · B 51 Linear equations see Differential equations; In- band diagonal matrix 55ff. tegral equations; Linear algebraic equa- complex equations 52f. tions Crout’s algorithm 47ff., 56 Linear inversion method, constrained 811ff. for integral equations 794, 797 Linear prediction 569ff. for inverse iteration of eigenvectors 499 characteristic polynomial 572 for inverse problems 812 coefficients 569ff. for matrix determinant 52 compared with regularization 812 for matrix inverse 51 contrasted to polynomial extrapolation for nonlinear sets of equations 385, 396 572 operation count 47, 51 related to optimal filtering 570f. for Pade« approximant 207 removal of bias in 575 pivoting 48f. stability 572 repeated backsubstitution 51, 57 Linear predictive coding (LPC) 576f. solution of linear algebraic equations 51 Linear programming 398, 434ff. solution of normal equations 678 artificial variables 441 for Toeplitz matrix 97 auxiliary objective function 441 Lucifer (encryption algorithm) 304 basic variables 438 composite simplex algorithm 447 constraints 435 M -estimates 704ff. convergence criteria 443 how to compute 708 degenerate feasible vector 440 local 705ff. dual problem 446 see also Maximum likelihood estimate equality constraints 435 Machine accuracy 31f., 892 feasible basis vector 437f. Macintosh, see Apple Macintosh feasible vector 435 Maehly’s procedure 375, 382 fundamental theorem 437 Magic inequality constraints 435 in MEM image restoration 825 left-hand variables 438 in Pade« approximation 207 nonbasic variables 438 Mantissa in floating point format 31, 892, normal form 438 921 objective function 435 Marginals 635 optimal feasible vector 435 Marquardt method (least squares fitting) 688ff., pivot element 439f. 827 primal-dual algorithm 447 Mass, center of 309ff. primal problem 447 MasterCard checksum 904 reduction to normal form 440ff. Mathematical Center (Amsterdam) 364 restricted normal form 438ff. Matrix 36ff. revised simplex method 447 approximation of 69f., 609 right-hand variables 438 band diagonal 53, 55ff., 74 simplex method 413, 434, 438ff., 443ff. band triangular 74 slack variables 440 banded 38, 466 tableau 438 bidiagonal 63 vertex of simplex 437 block diagonal 74, 765 Linear regression 666ff., 671ff. block triangular 74 see also Fitting block tridiagonal 74 Linear regularization 811ff. bordered 74 LINPACK 38 characteristic polynomial 461, 481 Linux 4, 16 Cholesky decomposition 99ff., 434, 467, Little-endian 306 679 Local extrapolation 720 column augmented 40 988 Index

complex 52f. symmetric 38, 99, 462, 466, 474ff., 679, condition number 64, 88 795f. curvature 688 threshold multiply of sparse 84ff. cyclic banded 74 Toeplitz 93, 95ff., 207 cyclic tridiagonal 77f. transpose of sparse 83f. defective 462, 488, 499 triangular 465 of derivatives see Hessian matrix; Jacobian tridiagonal 38, 53f., 74, 118, 162, 465f., determinant 474ff., 481ff., 499, 850f., 864, 872f. design (fitting) 656, 676, 811f. tridiagonal with fringes 833 determinant of 37, 52 unitary 462 diagonalization 464ff. updating 103, 393f. elementary row and column operations 40 upper triangular 46f., 101 finite differencing of partial differential user-defined library 28ff., 949 equations 832ff. Vandermonde 93ff., 123 Hermitian 462, 466, 486f. see also Eigensystems Hermitian conjugate 462 Matrix equations see Linear algebraic equa- Hessenberg 104, 465, 482, 487ff., 499 tions Hessian see Hessian matrix Matrix Template Library (MTL) 30, 953 hierarchically band diagonal 610 Matterhorn 617 Hilbert 93 Maximization see Minimization identity 37 Maximum entropy method (MEM) 577ff. ill-conditioned 64, 66, 123 algorithms for image restoration 826f. indexed storage of 81f. Bayesian 828 and integral equations 790, 794 Cornwell-Evans algorithm 827 inverse 37, 39, 45, 51f., 76ff., 80f., 105ff. demystified 825 inverse, approximate 60 historic vs. Bayesian 828 inverse by Hotelling’s method 60, 610 image restoration 821ff. inverse by Schultz’s method 60, 610 intrinsic correlation function (ICF) model inverse multiplied by a matrix 52 828 iteration for inverse 60, 610 for inverse problems 821ff. Jacobi transformation 465, 468ff., 475 operation count 579 Jacobian 743 see also Linear prediction lower triangular 46f., 99, 792 Maximum likelihood estimate (M-estimates) multiplication denoted by dot 36 699, 704ff. norm 61 and Bayes’ Theorem 822 normal 462f. chi-square test 699 nullity 64 defined 663 nullspace 37, 64, 66, 461, 806 how to compute 708 orthogonal 101, 462, 475, 599 mean absolute deviation 707, 708f. orthogonal transformation 464, 475ff., 482 relation to least squares 663 orthonormal basis 69, 100 Maxwell’s equations 837 outer product denoted by ⊗ 76, 432 Mean(s) partitioning for determinant 81 of distribution 615f., 619 partitioning for inverse 80f. statistical differences between two 620ff. pattern multiply of sparse 84f. Mean absolute deviation of distribution 616, positive definite 38, 99, 679 707 QR decomposition 101f., 393, 396, 679 related to median 708f. range 64 Measurement errors 661 rank 64 Median 332 residual 60 calculating 345 resize function not assumed 23 of distribution 616, 619f. row and column indices 36 as L-estimate 705 row vs. column operations 43f. role in robust straight line fitting 708f. self-adjoint 462 by selection 709 similarity transform 464ff., 468, 488, 490, Median-of-three, in Quicksort 336 493 MEM see Maximum entropy method (MEM) singular 64, 66, 69, 461 Merit function 661 singular value decomposition 37f., 62ff., in general linear least squares 676 808 for inverse problems 808 sparse 36, 74ff., 81, 609f., 743, 765, 815 nonlinear models 686 special forms 38 for straight line fitting 667, 708f. splitting in relaxation method 867f. for straight line fitting, errors in both coor- spread 819 dinates 671 square root of 434, 467 Mesh-drift instability 845f. storage schemes in C++ 36f., 941 Mesokurtic distribution 617 Index 989

Method of regularization 811ff. Modified Bessel functions see Bessel func- Metropolis algorithm 449f. tions Microsoft xx Modified Lentz’s method, for continued frac- Visual C++ 4 tions 177 Windows 4 Modified midpoint method 727f., 730 Midpoint method see Modified midpoint method; Modified moments 163 Semi-implicit midpoint rule Modula-2 7 Mikado, or Town of Titipu 725 Modular arithmetic, without overflow 282, Miller’s algorithm 187, 239 284, 289 Minimal solution of recurrence relation 185 Modularization, in programs 7f. Minimax polynomial 198, 210 Modulus of linear congruential generator 280 Minimax rational function 210 Moments Minimization 398ff. of distribution 615ff. along a ray 87, 388f., 400, 417f., 423, filter that preserves 655 428, 430 modified problem of 163 annealing, method of simulated 398f., problem of 93f. 448ff. and quadrature formulas 801 bracketing of minimum 401ff., 413 semi-invariants 619 Brent’s method 400, 406ff., 411, 672 Monic polynomial 154 Broyden-Fletcher-Goldfarb-Shanno algo- Monotonicity constraint, in upwind differenc- rithm 401, 430ff. ing 848 chi-square 664ff., 686ff. Monte Carlo 167, 279 choice of methods 399ff. adaptive 320ff., 323ff. combinatorial 448 bootstrap method 696f. conjugate gradient method 400f., 424ff., comparison of sampling methods 322f. 815, 826 exploration of binary tree 304 convergence rate 404, 420f. importance sampling 320f. Davidon-Fletcher-Powell algorithm 401, integration 134, 167, 308ff., 320ff. 430f. integration, recursive 328ff. degenerate 806 integration, using Sobol’ sequence 317ff. direction-set methods 400, 417ff. integration, VEGAS algorithm 323ff. and Kolmogorov-Smirnov statistic 632, downhill simplex method 400, 413ff., 456, 708 651f. partial differential equations 835 finding best-fit parameters 661 quasi-random sequences in 313ff. Fletcher-Reeves algorithm 400f., 425ff. quick and dirty 696f. functional 806 recursive 320ff., 328ff. global 398f., 455f., 661 significance of Lomb periodogram 583 globally convergent multidimensional 430ff. simulation of data 665, 695ff., 700 golden section search 401ff., 407 stratified sampling 321f., 328 multidimensional 399f., 413ff. Moon, calculate phases of 1f., 13f. in nonlinear model fitting 686f. Mother functions 596 Polak-Ribiere algorithm 400f., 427 Mother Nature 694, 697 Powell’s method 400, 413, 417ff. Moving average (MA) model 578 quasi-Newton methods 387, 401, 430ff. Moving window averaging 655 and root finding 386 Mozart 8 scaling of variables 432 MS-DOS xiii by searching smaller subspaces 826 Muller’s method 375, 383 steepest descent method 425, 815 Multidimensional termination criterion 402f., 415 confidence levels of fitting 699 use in finding double roots 352 data, use of binning 633f. use for sparse linear systems 87ff. Fourier transform 526ff. using derivatives 400f., 410ff. Fourier transform, real data 530ff. variable metric methods 401, 430ff. initial value problems 855ff. see also Linear programming integrals 134, 166ff., 308ff., 320ff. Minimum residual method, for sparse system interpolation 126ff. 88 Kolmogorov-Smirnov test 650ff. MINPACK 693 least squares fitting 686 MIPS 896 minimization 413ff., 417ff., 424ff. Missing data problem 581 Monte Carlo integration 308ff., 320ff. Mississippi River 450, 459 normal (Gaussian) distribution 701 Mode of distribution 616, 620 optimization 399f. Modeling of data see Fitting partial differential equations 855ff. Model-trust region 396, 693 root finding 351ff., 369, 381, 383ff., 386, Modes, homogeneous, of recursive filters 566 757, 760f., 763, 765 990 Index

search using quasi-random sequence 313 Nested iteration 879 secant method 384, 393ff. Neumann boundary conditions 831, 851, 862, wavelet transform 606 869 Multigrid method 835, 873ff. Neutrino 650 avoid SOR 877 Neville’s algorithm 111f., 114, 145, 194 boundary conditions 879f. Newton-Cotes formulas 135ff., 152 choice of operators 879 open 137 coarse-to-fine operator 875 Newton-Raphson method see Newton’s rule coarse-grid correction 875f. Newton’s rule 155f., 191, 352, 366ff., 373, cycle 876 375, 481 dual viewpoint 885 with backtracking 388f. fine-to-coarse operator 875 caution on use of numerical derivatives full approximation storage (FAS) algorithm 369 884ff. fractal domain of convergence 371f. full multigrid method (FMG) 874, 879f. globally convergent multidimensional 384, full weighting 878 387ff., 393, 760f., 763 Gauss-Seidel relaxation 876f. for matrix inverse 60, 610 half weighting 878 in multidimensions 381, 383ff., 760f., importance of adjoint operator 878 763, 765 injection operator 875 in nonlinear multigrid 884, 886 interpolation operator 875 nonlinear Volterra equations 798 line relaxation 877 for reciprocal of number 922 local truncation error 885 safe 370 Newton’s rule 884, 886 scaling of variables 393 nonlinear equations 884ff. nonlinear Gauss-Seidel relaxation 886 singular Jacobian 396 odd-even ordering 877, 880 solving stiff ODEs 751 operation count 873 for square root of number 923 prolongation operator 875 Niederreiter sequence 314 recursive nature 876 NL2SOL 693 relative truncation error 885 Noise relaxation as smoothing operator 876 bursty 899 restriction operator 875 effect on maximum entropy method 579 speeding up FMG algorithm 884 equivalent bandwidth 559 stopping criterion 886 fitting data which contains 658, 661 straight injection 878 model, for optimal filtering 553 symbol of operator 877f. Nominal variable (statistics) 633 use of Richardson extrapolation 880 Nonexpansive projection operator 816 V-cycle 876 Non-interfering directions see Conjugate direc- W-cycle 876 tions zebra relaxation 877 Nonlinear eigenvalue problems 467 Multiple precision arithmetic 916ff. Nonlinear equations Multiple roots 352, 373 finding roots of 351ff. Multiplication, complex 183 integral equations 792, 798 Multiplication, multiple precision 917, 920 in MEM inverse problems 824f. Multiplier of linear congruential generator multigrid method for elliptic PDEs 884ff. 280 Nonlinear instability 842 Multistep and multivalue methods (ODEs) Nonlinear programming 447 751ff. Nonnegativity constraints 435 see also Differential Equations; Predictor- Nonparametric statistics 644ff. corrector methods Nonpolynomial complete (NP-complete) 449 Multivariate normal distribution 701 Norm, of matrix 61 Murphy’s Law 418 Normal (Gaussian) distribution 279, 663, Musical scores 5 692f., 809 central limit theorem 663f. N AG xx, 38, 75, 217, 466 deviates from 292f., 583 Namespace 18, 931 kurtosis of 617 unnamed 19 multivariate 701 National Science Foundation (U.S.) xiv, xvi semi-invariants of 619 Natural cubic spline 118 tails compared to Poisson 664 Navier-Stokes equation 841f. two-dimensional (binormal) 642 Needle, eye of (minimization) 414 variance of skewness of 617 Negation, multiple precision 917 Normal equations (fitting) 37f., 656, 677ff., Negentropy 822, 906 806, 812 Nelder-Mead minimization method 400, 413ff. often are singular 681 Index 991

Normalization OEM information xix of Bessel functions 187 One-sided power spectral density 503 of floating-point representation 31, 892 Operation count of functions 154, 777 balancing 488 of modified Bessel functions 244 Bessel function evaluation 239 Notch filter 563, 567f. bisection method 357 NP-complete problem 449 Cholesky decomposition 100 nr.h function declarations 2, 18, 21, 931 coefficients of interpolating polynomial nrerror() utility 2, 941 123f. NRMat3d matrix class 27 complex multiplication 107 NRMat matrix class 20ff., 25ff., 941, 950, 953 cubic spline interpolation 118 nrtosp.pl Perl script 957 evaluating polynomial 180f. nrtypes.h type declarations 2, 21, 931, 939, fast Fourier transform (FFT) 509 949 Gauss-Jordan elimination 45, 51 for user-defined matrix libraries 30 Gaussian elimination 45 nrutil.h utility functions and classes 2f., Givens reduction 475 20f., 25ff., 931, 941, 950, 953 Householder reduction 479 NRVec vector class 20ff., 25ff., 941, 950, 953 interpolation 109 as valarray 27 inverse iteration 499 Null hypothesis 614 iterative improvement 59 Nullity 64 Jacobi transformation 472 Nullspace 37, 64, 66, 461, 806 Kendall’s tau 648f. Number-theoretic transforms 514 linear congruential generator 281 Numerical derivatives 192ff., 656 LU decomposition 47, 51 Numerical integration see Quadrature matrix inversion 107 Numerical Recipes matrix multiplication 106 as trademark xx maximum entropy method 579 compatibility with First Edition 4 multidimensional minimization 425 compilers tested 4 multigrid method 873 Example Book 3 multiplication 920 function declarations (nr.h) 2, 18, 21, polynomial evaluation 107, 180f. 931 QL method 482, 485 how to get CDROMs xviii QR decomposition 102 how to report bugs iv QR method for Hessenberg matrices 495 implementation of vector and matrix classes reduction to Hessenberg form 490 25ff., 941, 950, 953 selection by partitioning 345 license information xvii sorting 332ff. list of all 959ff. Toeplitz matrix 93 machines tested 4 Vandermonde matrix 93 no warranty on xvii Operator OEM information xix associativity, in C++ 23f. programming conventions 23ff. conversion 29 programs by chapter and section xxi overloading 8, 20, 26, 30 quick start 2, 22 precedence, in C++ 23f. table of dependencies 959ff. splitting 834, 858f., 872 type declarations (nrtypes.h) 2, 21, 30, Optimal feasible vector 435 931, 939, 949 Optimal (Wiener) filtering 547, 552ff., 570f., utility functions and classes (nrutil.h) 655 2f., 20f., 25ff., 931, 941, 950, 953 compared with regularization 812 Numerical Recipes Software xii, xx Optimization see Minimization address and fax number xix Ordinal variable (statistics) 633 Nyquist frequency 506ff., 530, 555, 557, 581, Ordinary differential equations see Differential 583f. equations Nystrom method 793f., 799f. Orthogonal see Orthonormal functions; Or- product version 799 thonormal polynomials Orthogonal transformation 464, 475ff., 482, 596 Object extensibility 8 Orthonormal basis, constructing 69, 103 Object-oriented programming 7 Orthonormal functions 154, 257 Objective function 435 Orthonormal polynomials Oblateness parameter 775 Chebyshev 156, 161, 196ff. Odd-even ordering construct for arbitrary weight 163f. in Gauss-Seidel relaxation 877, 880 in Gauss-Hermite integration 159 in successive over-relaxation (SOR) 870 and Gaussian quadrature 154 Odd parity 898 Gaussian weights from recurrence 161f. 992 Index

Hermite 156 forward Euler differencing 837f. Jacobi 156 Forward Time Centered Space (FTCS) Laguerre 156 838ff., 849ff., 854, 866 Legendre 156 Fourier analysis and cyclic reduction (FACR) weight function log x 165 859ff., 865 Orthonormality 62f., 154, 475 Gauss-Seidel method (relaxation) 866, Outer product of matrices (denoted by ⊗) 76, 875ff., 886 432 Godunov’s method 848 Outgoing wave boundary conditions 831 Helmholtz equation 863 Outlier 616, 664, 667, 704, 707 hyperbolic 829, 836f. see also Robust estimation implicit differencing 850 Overcorrection 868 incomplete Cholesky conjugate gradient Overflow 892 method (ICCG) 835 in complex arithmetic 183 inhomogeneous boundary conditions 861f. how to avoid in modulo multiplication initial value problems 829f. 282 initial value problems, recommendations on Overlap-add and overlap-save methods 549 849ff. Overrelaxation parameter 868 Jacobi’s method (relaxation) 866f., 875 choice of 868f. Laplace’s equation 829 Lax method 839ff., 847, 856f. Lax method (multidimensional) 856f. ade« approximant 114, 206ff. P matrix methods 835 Parabolic interpolation 407 mesh-drift instability 845f. Parabolic partial differential equations 829, Monte Carlo methods 835 849ff. Parallel axis theorem 322 multidimensional initial value problems Parameters in fitting function 662f., 694ff. 855ff. Parity bit 898 multigrid method 835, 873ff. Park and Miller minimal standard random gen- Neumann boundary conditions 831, 851, erator 282f. 862, 869 Parseval’s Theorem 503, 556 nonlinear diffusion equation 853 discrete form 509 nonlinear instability 842 Partial differential equations 829ff. numerical dissipation or viscosity 841 advective equation 837 operator splitting 834, 858f., 872 alternating-direction implicit method (ADI) outgoing wave boundary conditions 831 858, 872f. parabolic 829, 849ff. amplification factor 839, 845 periodic boundary conditions 861, 869 analyze/factorize/operate package 835 piecewise parabolic method (PPM) 848 artificial viscosity 842, 848 Poisson equation 829, 863 biconjugate gradient method 835 rapid (Fourier) methods 519ff., 835, 859ff. boundary conditions 830ff. relaxation methods 834, 865ff. boundary value problems 830ff., 859f. Schrodinger¬ equation 853ff. Cauchy problem 829f. second-order accuracy 844ff., 850f. caution on high-order methods 855f. shock 842, 848 Cayley’s form 855 sparse matrices from 74 characteristics 829 spectral methods 835f. Chebyshev acceleration 870f. spectral radius 867ff., 873 classification of 829ff. stability vs. accuracy 841 comparison of rapid methods 865 stability vs. efficiency 832 conjugate gradient method 835 staggered grids 524, 863 Courant condition 840, 843, 845, 847 staggered leapfrog method 844f. Courant condition (multidimensional) 857 strongly implicit procedure 835 Crank-Nicolson method 850, 853, 855, successive over-relaxation (SOR) 868ff., 857 873, 877 cyclic reduction (CR) method 859f., 863f. time splitting 858f., 872 diffusion equation 829, 849ff., 857, 866 two-step Lax-Wendroff method 846ff. Dirichlet boundary conditions 831, 850, upwind differencing 843f., 848 861, 867, 869 variational methods 835 elliptic, defined 829 varieties of error 842ff. error, varieties of 842ff. von Neumann stability analysis 838f., explicit vs. implicit differencing 838 841, 844, 851 FACR method 865 wave equation 829, 836f. finite difference method 832ff. see also Elliptic partial differential equa- finite element methods 835f. tions; Finite difference equations (FDEs) flux-conservative initial value problems Partial pivoting 41 836ff. Partition-exchange 336, 345 Index 993

Partitioned matrix, inverse of 80f. Poisson process 291, 295, 297 Party tricks 105ff., 180f. Polak-Ribiere algorithm 400f., 427 Parzen window 559 Poles see Complex plane, poles in Pascal 16 Polishing of roots 369, 374f., 380f. Numerical Recipes in 1 Polymorphism 8 Pascal, Numerical Recipes in xvi Polynomial interpolation 108, 111ff. Path integration, for function evaluation 213ff., Aitken’s algorithm 111 275 in Bulirsch-Stoer method 733, 735 Pattern multiply of sparse matrices 84f. coefficients for 123ff. PBCG (preconditioned biconjugate gradient Lagrange’s formula 94, 111f. method) 89, 835 multidimensional 126ff. PC methods see Predictor-corrector methods Neville’s algorithm 111f., 114, 145, 194 PCGPACK 81 pathology in determining coefficients for PDEs see Partial differential equations 123 Pearson’s r 641ff. in predictor-corrector method 751 PECE method 752 smoothing filters 655f. Pentagon, symmetries of 904 see also Interpolation Pentium processor 289, 307, 896 Polynomials 179ff. Percentile 332 algebraic manipulations 181 Period of linear congruential generator 280 approximating modified Bessel functions Periodic boundary conditions 861, 869 241 Periodogram 556ff., 579 approximation from Chebyshev coefficients Lomb’s normalized 581f., 586f. 203 variance of 557 AUTODIN-II 900 Perl xiv CCITT 900 for nrtosp.pl 957 characteristic 379 Perron’s theorems, for convergence of recur- characteristic, for digital filters 566, 572 rence relations 186f. characteristic, for eigenvalues of matrix Perturbation methods for matrix inversion 461, 481 76ff. Chebyshev 161, 190ff. Peter Principle 340 CRC-16 900 Phase error 842 deflation 373ff., 381 Phase-locked loop 711 derivatives of 179f. Phi statistic 636 division 94, 181, 373, 381 Phillips-Twomey method 811ff. evaluation of 179 Pi, computation of 916ff. evaluation of derivatives 179f. Piecewise parabolic method (PPM) 848 extrapolation in Bulirsch-Stoer method Pincherle’s theorem 187 733, 735 Pivot element 41, 44, 768 extrapolation in Romberg integration 144 in linear programming 439f. fitting 93, 123, 203, 655f., 676, 685 Pivoting 39, 41ff., 57, 76, 100 generator for CRC 899f. full 41 ill-conditioned 373 implicit 41, 49 matrix method for roots 379 in LU decomposition 48f. minimax 198, 210 partial 41, 44, 49 monic 154 and QR decomposition 102 multiplication 181 in reduction to Hessenberg form 490 operation count for 180f. in relaxation method 768 orthonormal 154, 190f. for tridiagonal systems 54 primitive modulo 2 300ff., 315f., 899 Pixel 530, 608, 814, 822 roots of 189ff., 373ff., 379 Planck’s constant 853 shifting of 204f. Plane rotation see Givens reduction; Jacobi stopping criterion in root finding 377 transformation (or rotation) Port, serial data 902 Platykurtic distribution 617 Portability 3f., 16 Plotting of functions 353f. Portable random number generator see Ran- POCS (method of projection onto convex sets) dom number generator 816 Positive definite matrix, testing for 100 Poetry 5 Positivity constraints 435 Pointer, use for matrices 941 Postal Service (U.S.), barcode 904 Poisson equation 530, 829, 863 PostScript xiv, xx Poisson probability function Powell’s method 400, 413, 417ff. cumulative 226 Power (in a signal) 503f. deviates from 294, 297ff., 583 Power series 171ff., 179f., 207 semi-invariants of 619 economization of 204ff. tails compared to Gaussian 664 Pade« approximant of 206ff. 994 Index

Power spectral density see Fourier transform; pivoting 102 Spectral density updating 103, 393 Power spectrum estimation see Fourier trans- use for orthonormal basis 69, 103 form; Spectral density Quadratic PPM (piecewise parabolic method) 848 convergence 60, 267, 362, 368f., 420f., Precedence of operators, in C++ 23f. 431, 916 Precision, converting to single 17, 957 equations 32, 189ff., 402, 469 Precision, floating point 17, 892, 957 interpolation 364, 375 Precision, multiple 916ff. programming 447 Preconditioned biconjugate gradient method Quadrature 133ff. (PBCG) 89 adaptive 133, 202, 799 Preconditioning, in conjugate gradient methods alternative extended Simpson’s rule 138 835 arbitrary weight function 163f., 799 Predictor-corrector methods 713, 742, 751ff. automatic 166 Adams-Bashforth-Moulton schemes 752 Bode’s rule 136 adaptive order methods 754f. change of variable in 148ff., 799 compared to other methods 751f. by Chebyshev fitting 134, 201 fallacy of multiple correction 752 classical formulas for 134ff. with fixed number of iterations 752 Clenshaw-Curtis 134, 202, 522, 524 functional iteration vs. Newton’s rule 753 closed formulas 135, 137f. multivalue compared with multistep 753f. and computer science 891 starting and stopping 753f. by cubic splines 134 stepsize control 753f. error estimate in solution 795 Prime numbers 926 extended midpoint rule 140, 146 Primitive polynomials modulo 2 300ff., 315f., 899 extended rules 137ff., 144, 797, 799, 801 Principal directions 418f. extended Simpson’s rule 138 Principal solution, of inverse problem 808 Fourier integrals 589ff. Prize, $1000 offered 285 Fourier integrals, infinite range 595 Probability see Random number generator; Gauss-Chebyshev 156, 161, 522, 524 Statistical tests Gauss-Hermite 156, 800 Probability density, change of variables in Gauss-Jacobi 156 291ff. Gauss-Kronrod 165 Process loss 559 Gauss-Laguerre 156, 800 Product Nystrom method 799 Gauss-Legendre 156, 794, 799 Program(s) Gauss-Lobatto 165, 202, 522 as black boxes xv, 6, 38, 63, 217, 352, Gauss-Radau 165 417 Gaussian integration 137, 152ff., 792, dependencies 959ff. 794, 799 encapsulation 7f. Gaussian integration, nonclassical weight interfaces 8 function 163f., 799 modularization 7f. for improper integrals 146ff., 799f. organization 5ff. for integral equations 792f., 797 recipes by chapter and section xxi Monte Carlo 134, 167, 308ff., 320ff. typography of 12 multidimensional 134, 166ff. validation 3f. Newton-Cotes formulas 135ff., 152 Projection onto convex sets (POCS) 816 Newton-Cotes open formulas 137 Projection operator, nonexpansive 816 open formulas 135, 137, 139f., 146 Prolongation operator 875 related to differential equations 133 Protocol, for communications 898 related to predictor-corrector methods 751f. Prototypes in C++ 17, 931 Romberg integration 134, 144f., 147, 194, PSD (power spectral density) see Fourier 728, 799 transform; Spectral density semi-open formulas 139f. Pseudo-random numbers 278ff. Simpson’s rule 136, 143, 147, 595, 793f., Puns, particularly bad 179, 755, 758 798, 801 Pyramidal algorithm 599 Simpson’s three-eighths rule 136, 799, Pythagoreans 403 801 singularity removal 148ff., 799f. singularity removal, worked example 803 QL see Eigensystems trapezoidal rule 135, 138, 141ff., 144, QR see Eigensystems 591, 595, 793f., 797 QR decomposition 101f., 393, 396 using FFTs 134 backsubstitution 102 weight function log x 165 and least squares 679 see also Integration of functions operation count 102 Quadrature mirror filter 597, 604 Index 995

Quantum mechanics, Uncertainty Principle timings 289f. 611 transformation method 291ff. Quartile value 332 trick for trigonometric functions 293 Quasi-Newton methods for minimization 401, Random numbers see Monte Carlo; Random 430ff. deviates Quasi-random sequence 313ff., 331, 891, 898 Random walk 32 Halton’s 313f. RANDU, infamous routine 281 for Monte Carlo integration 317ff., 323, Range 64, 66 331 Rank (matrix) 64 Sobol’s 315 kernel of finite 796 see also Random number generator Rank (sorting) 332, 343f. Quicksort 332, 336ff., 342, 345 Rank (statistics) 644ff., 705 Quotient-difference algorithm 176 Kendall’s tau 647ff. Spearman correlation coefficient 645f. sum squared differences of 645 R-estimates 705 Ratio variable (statistics) 633 Radioactive decay 291 Rational Chebyshev approximation 209ff. Radix base for floating point arithmetic 488, Rational function 108, 179ff., 206ff., 209ff. 892, 917, 924 approximation for Bessel functions 236f. Radix conversion 912, 916, 924 approximation for continued fraction 176, Ramanujan’s identity for π 926 222, 232 RAND_MAX macro 279f., 281 Chebyshev approximation 209ff. Random bits, generation of 300ff. evaluation of 182 Random deviates 278ff. extrapolation in Bulirsch-Stoer method binomial 299f. 729ff., 736 exponential 291f. interpolation and extrapolation using 108, gamma distribution 294ff. 114ff., 206ff., 209ff., 729ff., 736 Gaussian 279, 292f., 583, 809 minimax 210 normal 279, 292f., 583 as power spectrum estimate 578 Poisson 297ff., 583 Realizable (causal) 564, 567 quasi-random sequences 313ff., 891, 898 Rearranging see Sorting uniform 279ff. Reciprocal, multiple precision 922 uniform integer 284, 287ff. Record, in data file 341 Random number generator 278ff. Recurrence relation 184ff. bitwise operations 300ff. associated Legendre polynomials 258 Box-Muller algorithm 293 Bessel function 184, 236, 246f. Data Encryption Standard 304ff. binomial coefficients 220 good choices for modulus, multiplier and Bulirsch-Stoer 114f. increment 288 characteristic polynomial of tridiagonal for integer-valued probability distribution matrix 481 297 Clenshaw’s recurrence formula 187ff. integer vs. real implementation 286 and continued fraction 187 L’Ecuyer’s long period 284f. continued fraction evaluation 176f. linear congruential generator 280f. convergence 187 machine language 282 Minimal Standard, Park and Miller’s 282f. cosine function 184, 511 nonrandomness of low-order bits 281 dominant solution 185 perfect 285 exponential integrals 184 planes, numbers lie on 281 gamma function 218 portable 282ff. generation of random bits 301f. primitive polynomials modulo 2 300ff. Golden Mean 33 pseudo-DES 304 Legendre polynomials 184 quasi-random sequences 313ff., 891, 898 minimal vs. dominant solution 185 quick and dirty 287ff. modified Bessel function 243f. quicker and dirtier 289 Neville’s 112, 194 in Quicksort 336 orthonormal polynomials 154 random access to nth number 307 Perron’s theorems 186f. random bits 300ff. Pincherle’s theorem 187 recommendations 289f. polynomial interpolation 112, 195 rejection method 294ff. primitive polynomials modulo 2 301f. shuffling procedure 284, 285 random number generator 280 in simulated annealing method 449 rational function interpolation 114f. spectral test 289 sequence of trig functions 184f. subtractive method 286 sine function 184, 511 system-supplied 279ff. spherical harmonics 258 996 Index

stability of 33f., 185ff., 188f., 236, 244, Ridders’ method, for numerical derivatives 258 194 trig functions 584 Ridders’ method, root finding 352, 360, 362 weight of Gaussian quadrature 156 Riemann shock problem 848 Recursion, in multigrid method 876 Right eigenvalues and eigenvectors 463 Recursive Monte Carlo integration 320ff. Rise/fall time 559f. Recursive stratified sampling 328ff. Robust estimation 664, 704ff., 711 Red-black see Odd-even ordering Andrew’s sine 707 Reduction of variance in Monte Carlo integra- average deviation 616 tion 312, 320ff. double exponential errors 706 References (explanation) 5 Kalman filtering 711 References (general bibliography) 927ff. Lorentzian errors 707 Reflection formula for gamma function 218 mean absolute deviation 616 register storage class 23 nonparametric correlation 644ff. Regula falsi (false position) 358ff. Tukey’s biweight 707 Regularity condition 787 use of a priori covariances 711 Regularization see also Statistical tests compared with optimal filtering 812 Romberg integration 134, 144f., 147, 194, constrained linear inversion method 811ff. 728, 799 of inverse problems 807ff. Root finding 155f., 351ff. linear 811ff. advanced implementations of Newton’s rule nonlinear 824f. 396 objective criterion 814 Bairstow’s method 375, 381 Phillips-Twomey method 811ff. bisection 354, 357, 363ff., 370, 401, 481, Tikhonov-Miller 811ff. 709 trade-off curve 810 bracketing of roots 352, 354ff., 364, 373, two-dimensional 814 375, 380 zeroth order 807ff. Brent’s method 352, 360, 672 see also Inverse problems Broyden’s method 384, 393ff., 396 Regularizing operator 809 compared with multidimensional minimiza- Rejection method for random number genera- tion 386 tor 294ff. complex analytic functions 375 Relaxation method in complex plane 216 for algebraically difficult sets 774 convergence criteria 357, 385 automated allocation of mesh points 785ff., deflation of polynomials 373ff., 381 788 without derivatives 365 computation of spheroidal harmonics 775ff. double root 352 for differential equations 757f., 765ff. eigenvalue methods 379 elliptic partial differential equations 834f., false position 358ff. 865ff. Jenkins-Traub method 380 example 775ff. Laguerre’s method 352, 376ff. Gauss-Seidel method 866, 875ff., 886 Lehmer-Schur algorithm 380 internal boundary conditions 787ff. Maehly’s procedure 375, 382 internal singular points 787ff. matrix method 379 Jacobi’s method 866f., 875 Muller’s method 375, 383 successive over-relaxation (SOR) 868ff., multiple roots 352 873, 877 Newton’s rule 155f., 191, 352, 366ff., see also Multigrid method 373, 375, 381, 383ff., 387f., 481, 753, Remes algorithms 760f., 765, 798, 884, 886, 922f. exchange algorithm 566 pathological cases 354f., 366ff., 373, 384 for minimax rational function 210 polynomials 352, 373ff., 461 Residual 60, 65, 88 in relaxation method 765 in multigrid method 874 Ridders’ method 352, 360, 362 Resize function, existence not assumed 23 root-polishing 369, 374f., 380ff. Resolution function, in Backus-Gilbert method safe Newton’s rule 370 818 secant method 358ff., 369, 375, 411 Response function 543 in shooting method 757, 760f. Restriction operator 875 singular Jacobian in Newton’s rule 396 Reward, $1000 offered 285 stopping criterion for polynomials 377 Richardson’s deferred approach to the limit use of minimum finding 352 144, 147, 194, 713, 729ff., 737f., 798, using derivatives 366ff. 880 zero suppression 383 see also Bulirsch-Stoer method see also Roots Richtmyer artificial viscosity 848 Root polishing 369, 374, 380ff. Index 997

Roots uneven or irregular 581, 659 Chebyshev polynomials 196 Sampling theorem 506, 555 cubic equations 190f. for numerical approximation 611ff. multiple 352, 376ff. Sande-Tukey FFT algorithm 514 nonlinear equations 351ff. Savitzky-Golay filters polynomials 352, 373ff., 461 for data smoothing 655ff. quadratic equations 189f. for numerical derivatives 195, 656 reflection in unit circle 572 ScaLAPACK 38 square, multiple precision 923 Scallop loss 559 see also Root finding Schrage’s algorithm 282 Rosenbrock method 741ff. Schrodinger¬ equation 853ff. compared with semi-implicit extrapolation Schultz’s method for matrix inverse 60, 610 750 SDLC checksum 900 stepsize control 743 Searching Roundoff error 32, 891f., 957 with correlated values 120f. bracketing a minimum 411 an ordered table 120f. conjugate gradient method 835 selection 344ff. eigensystems 470, 479, 482, 483, 488, Secant method 352, 358ff., 369, 375, 411 490, 494 Broyden’s method 393ff. extended trapezoidal rule 143 multidimensional (Broyden’s) 384, 393ff. general linear least squares 679, 682 Second Euler-Maclaurin summation formula graceful 893 146 hardware aspects 892 Second order differential equations 737 Householder reduction 478 Seed of random number generator 279 IEEE standard 893 Selection 332, 344ff. interpolation 110 find m largest elements 348 least squares fitting 669, 679 heap algorithm 348 Levenberg-Marquardt method 690 for median 709 linear algebraic equations 35f., 39, 41, 58, operation count 344f. 67, 94 by partition-exchange 345 linear predictive coding (LPC) 576 without rearrangement 346 magnification of 32, 58 timings 347f. maximum entropy method (MEM) 579 use to find median 620 measuring 892 Semi-implicit Euler method 741, 746 multidimensional minimization 430, 434 Semi-implicit extrapolation method 741, 746 multiple roots 373f. compared with Rosenbrock method 750 numerical derivatives 192 stepsize control 748 recurrence relations 185 Semi-implicit midpoint rule 746 reduction to Hessenberg form 490 Semi-invariants of a distribution 619 series 177 Sentinel, in Quicksort 336, 345 straight line fitting 669 Separable kernel 796 variance 618 Separation of variables 257 Row degeneracy 35 Serial data port 902 Row-indexed sparse storage 81f. Series 171ff. transpose 83f. accelerating convergence of 172ff. Row operations on matrix 40, 43 alternating 172f. Row totals 635 asymptotic 173 RSS algorithm 328ff. Bessel function Kν 252 RST properties (reflexive, symmetric, transi- Bessel function Yν 247 tive) 349 Bessel functions 172, 235 Runge-Kutta method 713f., 715ff., 743, 751 cosine integral 262 Cash-Karp parameters 721f. divergent 173 embedded 720f., 743 economization 204ff., 207 high-order 716 quality control 733 Euler’s transformation 172ff. stepsize control 719ff. exponential integral 227, 229 Run-length encoding 911 Fresnel integral 259 Rybicki, G.B. 95, 123, 157, 264, 533, 586, hypergeometric 214, 275 611 incomplete beta function 232 incomplete gamma function 222 Laurent 578 Sampling relation to continued fractions 176 importance 320f. roundoff error in 177 Latin square or hypercube 319 sine and cosine integrals 262 recursive stratified 328ff. sine function 172 stratified 321f. Taylor 366, 418, 713, 720, 766, 770 998 Index

transformation of 172ff. in minimization 420 van Wijngaarden’s algorithm 173 more equations than unknowns 68f. Shaft encoder 896f. and rational Chebyshev approximation Shakespeare 8 211 Shampine’s Rosenbrock parameters 743 of square matrix 64ff. Shell algorithm (Shell’s sort) 333ff. use for ill-conditioned matrices 66f., 69, Sherman-Morrison formula 76ff., 93, 393 461 Shifting of eigenvalues 461, 482f., 492 use for orthonormal basis 69, 103 Shock wave 842, 848 Singularities Shooting method of hypergeometric function 214, 275 computation of spheroidal harmonics 784 in integral equations 799ff. for differential equations 757, 760ff., in integral equations, worked example 803 782ff. in integrands 146ff., 799f. for difficult cases 762f. removal in numerical integration 148ff., example 782ff. 799f. interior fitting point 762f. Singularity, subtraction of the 800 Shuffling to improve random number generator SIPSOL 835 284f. size function 26, 30 Sidelobe fall-off 559 Skewness of distribution 617, 619 Sidelobe level 559 Smoothing, importance in multigrid method Signal, bandwidth limited 506 876 Significance (numerical) 31 Smoothing of data 123, 655ff. Significance (statistical) 620f. in integral equations 792 one- vs. two-sided 643 sn function 273 peak in Lomb periodogram 582 Snyder, N.L. xiii of 2-d K-S test 651f. Sobol’s quasi-random sequence 315 two-tailed 624 Sonata 8 Similarity transform 464ff., 468, 488, 490, Sonnet 8 493 Sorting 332ff. Simplex bubble sort cautioned against 333 defined 413 compared to selection 344f. method in linear programming 400, 413f., covariance matrix 680, 692 433ff., 443ff. eigenvectors 473f. method of Nelder and Mead 400, 413ff., Heapsort 332, 339f., 348 456, 708 index table 332, 341 use in simulated annealing 456 operation count 332ff. Simpson’s rule 134ff., 138, 143, 147, 595, Quicksort 332, 336ff., 342, 345 793f., 798f. rank table 332, 343f. Simpson’s three-eighths rule 136, 799, 801 ranking 341 Simulated annealing see Annealing, method of Shell’s method 333ff. simulated straight insertion 333f., 473 Simulation see Monte Carlo Sparse linear equations 36, 74ff., 743 Sine function band diagonal 55ff. evaluated from tan(θ/2) 185 biconjugate gradient method 87f., 610 recurrence 184 indexed storage 81f. series 172 in inverse problems 815 Sine integral 259, 261ff. minimum residual method 88 continued fraction 262 named patterns 74, 833 series 262 partial differential equations 833ff. see also Cosine integral relaxation method for boundary value prob- Sine transform see Fast Fourier transform lems 765 (FFT); Fourier transform row-indexed storage 81f. Single precision wavelet transform 596, 610 converting to 17, 957 see also Matrix Singleton’s algorithm for FFT 536f. Spearman rank-order coefficient 645f., 705 Singular value decomposition (SVD) 37f., Special functions see Function 62ff. Spectral analysis see Fourier transform; Peri- approximation of matrices 69f. odogram backsubstitution 67 Spectral density 553 and bases for nullspace and range 64 and data windowing 558ff. confidence levels from 703 figures of merit for data windows 559f. covariance matrix 703 normalization conventions 555 fewer equations than unknowns 68 one-sided PSD 503 for inverse problems 808 periodogram 556ff., 579 and least squares 65, 68f., 211, 679, 681ff. power spectral density (PSD) 503f. Index 999

power spectral density per unit time 504 Statement labels, named 9 power spectrum estimation by FFT 555ff. static object 19 power spectrum estimation by MEM 577ff. Statistical error 664 two-sided PSD 504 Statistical tests 614ff. variance reduction in spectral estimation Anderson-Darling 631f. 557 average deviation 616 Spectral lines, how to smooth 655 bootstrap method 696f. Spectral methods for partial differential equa- chi-square 625f., 634ff. tions 835f. contingency coefficient C 635f. Spectral radius 865ff., 873 contingency tables 633ff., 649 Spectral test for random number generator correlation 614f. 289 Cramer’s V 635f. Spectrum see Fourier transform difference of distributions 625ff. Spherical Bessel functions 245 difference of means 620ff. routine for 256 difference of variances 622, 624 Spherical harmonics 257f. entropy measures of association 637ff. orthogonality 257 F-test 622, 624 routine for 258f. Fisher’s z-transformation 642f. stable recurrence for 258 general paradigm 614 table of 257 Kendall’s tau 645, 647ff. see also Associated Legendre polynomials Kolmogorov-Smirnov 625, 628ff., 650ff., Spheroidal harmonics 775ff., 782ff. 705 boundary conditions 776 Kuiper’s statistic 632 normalization 777 kurtosis 617, 619 routines for 780ff. L-estimates 705 Spline 109 linear correlation coefficient 641ff. cubic 116ff. M-estimates 704ff. gives tridiagonal system 118 mean 614ff., 619ff. natural 118 measures of association 615, 633ff. operation count 118 measures of central tendency 615ff. two-dimensional (bicubic) 130f. median 616, 705 Spread matrix 819 mode 616 Spread spectrum 304 moments 615ff., 619 Square root, complex 183f. nonparametric correlation 644ff. Square root, multiple precision 923 Pearson’s r 641ff. Square window 558 for periodic signal 582f. Squaring, templated function in C++ 25, 941 phi statistic 636 Stability 33f. R-estimates 705 of Clenshaw’s recurrence 188f. rank correlation 644ff. Courant condition 840, 843ff., 847, 857 robust 616, 645, 704ff. diffusion equation 851 semi-invariants 619 of Gauss-Jordan elimination 39, 41 for shift vs. for spread 631f. of implicit differencing 740f., 851 significance 620f. mesh-drift in PDEs 845f. significance, one- vs. two-sided 624, 643 nonlinear 842, 848 skewness 617, 619 partial differential equations 831, 838f. Spearman rank-order coefficient 645f., of polynomial deflation 374 705 in quadrature solution of Volterra equation standard deviation 616 798 strength vs. significance 620, 633 of recurrence relations 185ff., 188f., 236, Student’s t 621, 642 244, 258 Student’s t, for correlation 642 and stiff differential equations 740f. Student’s t, paired samples 623 von Neumann analysis for PDEs 838f., Student’s t, Spearman rank-order coefficient 841, 844, 851 645 see also Accuracy Student’s t, unequal variances 622 Stabilized Kolmogorov-Smirnov test 631f. sum squared difference of ranks 645f. Stabilizing functional 809 Tukey’s trimean 705 Staggered leapfrog method 844f. two-dimensional 650ff. Standard deviation variance 614ff., 618, 623 of a distribution 616 Wilcoxon 705 of Fisher’s z 642 see also Error; Robust estimation of linear correlation coefficient 641 Steak, without sizzle 821 of sum squared difference of ranks 646 Steed’s method Standard (probable) errors 621, 667, 672, Bessel functions 245ff., 251 678, 682, 694 continued fractions 176f. 1000 Index

Steepest descent method 425 Systematic errors 664 in inverse problems 815 Step doubling 141, 720 T ableau (interpolation) 112, 195 tripling 147 Tangent function, continued fraction 175 Stieltjes, procedure of 163 Taylor series 192, 366, 418, 713, 720, 753, Stiff equations 714, 739ff. 766, 770 Kaps-Rentrop method 741 Template Numerical Toolkit (TNT) 30, 950 methods compared 750 Test programs 4 predictor-corrector method 742 TEXxiv r.h.s. independent of x 741 Thermodynamics, analogy for simulated an- Rosenbrock method 741ff. nealing 448f. Threshold multiply of sparse matrices 84ff. scaling of variables 742 Tides 573 semi-implicit extrapolation method 741 Tikhonov-Miller regularization 811ff. semi-implicit midpoint rule 746 Time domain 501 Stiff functions 109, 410 Time splitting 858f., 872 Stirling’s approximation 218, 823 Toeplitz matrix 93, 95ff., 207 Stoermer’s rule 737 LU decomposition 97 Stopping criterion, in multigrid method 886 new, fast algorithms 98f. Stopping criterion, in polynomial root finding nonsymmetric 96ff. 377 Tongue twisters 344 Storage Torus 309ff., 317ff. band diagonal matrix 55 Trademarks xx scheme for matrix in C++ 941 Trade-off curve 806, 820 sparse matrices 81f. Transformation Straight injection 878 Gauss 267 Straight insertion 333f., 473 Landen 267 Straight line fitting 666ff., 678f. method for random number generator 291ff. errors in both coordinates 671ff. Transforms, number theoretic 514 robust estimation 708f. Transport error 842 Strassen’s fast matrix algorithms 106f. Transpose of sparse matrix 83f. Stratified sampling, Monte Carlo 321f., 328 Trapezoidal rule 135, 138, 141ff., 144, 591, Strongly implicit procedure (SIPSOL) 835 595, 793f., 797 Stroustrup, B. 3, 17 Traveling salesman problem 449ff. Structured programming 5ff. Tridiagonal matrix 53f., 69, 162, 465f., 499 Student’s probability distribution 231, 233 in alternating-direction implicit method Student’s t-test (ADI) 872f. for correlation 642 from cubic spline 118 for difference of means 621 cyclic 77f. for difference of means (paired samples) in cyclic reduction 864 623 eigenvalues 481ff. for difference of means (unequal variances) with fringes 833 622 from operator splitting 872f. for difference of ranks 646 reduction of symmetric matrix to 474ff., Spearman rank-order coefficient 645 481 Sturmian sequence 481 see also Matrix Sub-random sequences see Quasi-random se- Trigonometric quence functions, linear sequences 184f. Subtraction, multiple precision 917 functions, recurrence relation 184, 584 Subtractive method for random number genera- functions, tan(θ/2) as minimal 185 tor 286 interpolation 108 Successive over-relaxation (SOR) 868ff., 873 solution of cubic equation 190f. bad in multigrid method 877 Truncation error 33, 411, 720, 891f. Chebyshev acceleration 870f. in multigrid method 885 choice of overrelaxation parameter 868f. in numerical derivatives 192 Sum squared difference of ranks 645 Tukey’s biweight 707 Sums see Series Tukey’s trimean 705 Sun xx, 896 Twin errors 904 Solaris 4 Two-dimensional see Multidimensional Ultra workstation 4 Two-dimensional KÐS test 650ff. Workshop Compiler 4 Two-pass algorithm for variance 618 Supernova 1987A 650 Two-point boundary value problems 713, SVD see Singular value decomposition (SVD) 756ff. Symbol, of operator 877f. automated allocation of mesh points 785ff., Synthetic division 94, 180, 373, 381 788 Index 1001

boundary conditions 756ff., 760, 763, Variance(s) 782f. of distribution 614ff., 619, 622, 624 difficult cases 762f. pooled 621 eigenvalue problem for differential equa- reduction of (in Monte Carlo) 312, 320ff. tions 759, 775ff., 782ff. statistical differences between two 620 free boundary problem 759, 788 two-pass algorithm for computing 618 grid (mesh) points 757f., 765, 785ff., 788 see also Covariance internal boundary conditions 787ff. Variational methods, partial differential equa- internal singular points 787ff. tions 835 linear requires no iteration 762 VAX 289, 307 multiple shooting 764 Vector problems reducible to standard form 759 representation in C++ xi, 3, 20, 25ff., 941, regularity condition 787 950, 953 relaxation method 757f., 765ff. resize function not assumed 23 relaxation method, example of 775ff. user-defined library 28ff., 949 shooting to a fitting point 762ff. see also Array shooting method 757, 760ff., 782ff. VEGAS algorithm for Monte Carlo 323ff. shooting method, example of 782ff. Verhoeff’s algorithm for checksums 904 singular endpoints 763, 776, 782 Viete’s` formulas for cubic roots 190f. see also Elliptic partial differential equa- Virus, computer 899 tions Viscosity Two-sided exponential error distribution 706 artificial 842, 848 Two-sided power spectral density 504 numerical 841f., 848 Two-step Lax-Wendroff method 846ff. VMS xx Type conversion Volterra equations 791f. automatic 28 adaptive stepsize control 799 Type declarations analogy with ODEs 796f. nrtypes.h 2, 21, 30, 931, 939, 949 block-by-block method 799 first kind 792, 797 nonlinear 792, 798 Uncertainty coefficient 639 second kind 792, 796f. Uncertainty principle 611 unstable quadrature 798 Underflow, in IEEE arithmetic 893 von Neumann-Richtmyer artificial viscosity Underrelaxation 868 848 Uniform deviates see Random deviates, uni- von Neumann stability analysis for PDEs 838f., form 841, 844, 851 Unit-offset array xi, 21 Vowellish (coding example) 906f., 912 Unitary (function) 854f. Unitary (matrix) see Matrix UNIX xiii, xx, 16, 289, 307, 896 Unnamed namespace 19 W -cycle 876 Upper Hessenberg matrix see Hessenberg ma- Warranty, disclaimer of xvii trix Wave equation 257, 829, 836f. Upwind differencing 843f., 848 Wavelet transform 596ff. Using declaration 19 appearance of wavelets 602f. p Using directive 19 approximation condition of order 597 U.S. Postal Service barcode 904 coefficient values 599, 601 Utility functions and classes (nrutil.h)2, contrasted with Fourier transform 596f., 20, 25ff., 931, 941, 950, 953 605 Daubechies wavelet filter coefficients 597ff., 601, 602, 605, 609 V -cycle 876 detail information 598 valarray 20, 27 discrete wavelet transform (DWT) 599f. Validation of Numerical Recipes procedures DWT (discrete wavelet transform) 599f. 3f. eliminating wrap-around 599f. Valley, long or narrow 414, 418, 421 fast solution of linear equations 609f. Van Cittert’s method 815 filters 603f. Van Wijngaarden-Dekker-Brent method see and Fourier domain 603f. Brent’s method image processing 607f. Vandermonde matrix 93ff., 123 for integral equations 793 Variable length code 906 inverse 599 Variable metric method 401, 430ff. Lemarie’s wavelet 605 compared to conjugate gradient method of linear operator 609f. 430 mother-function coefficient 599 Variable step-size integration 133, 146, 714, mother functions 596 718, 729ff., 737f., 743, 748, 753f. multidimensional 606 1002 Index

nonsmoothness of wavelets 602f. pyramidal algorithm 599 quadrature mirror filter 597 smooth information 598 truncation 605f. wavelet filter coefficient 597, 599 wavelets 596, 602ff. Wavelets see Wavelet transform Weber function 216 Weighted Kolmogorov-Smirnov test 631f. Weighted least-squares fitting see Least squares fitting Weighting, full vs. half in multigrid 878 Weights for Gaussian quadrature 152ff., 799 nonclassical weight function 163f., 799 Welch window 559 while iteration 13 Wiener filtering 547, 552ff., 570f., 655 compared to regularization 812 Wiener-Khinchin theorem 503, 571, 579 Wilcoxon test 705 Window function Bartlett 559 flat-topped 560 Hamming 559 Hann 559 Parzen 559 square 558 Welch 559 Winograd Fourier transform algorithms 514 Woodbury formula 78ff., 93 Wordlength 31 Wraparound order for storing spectrum 512 problem in convolution 545 Wrapper class 29f., 950, 953 Wronskian, of Bessel functions 245, 251

X .25 protocol 900 XMODEM checksum 899 X-ray diffraction pattern, processing of 816

Y ale Sparse Matrix Package 75, 81

Z -transform 566, 572, 577 Z-transformation, Fisher’s 642f. Zealots 825 Zebra relaxation 877 Zero contours 383f. Zero-offset array xi, 21 Zeroth-order regularization 807ff. Zip code, barcode for 904 Ziv-Lempel compression 906