International Journal of Mo dern Physics C, Vol. 0, No. 0 (1992) 000{000
f
cWorld Scienti c Publishing Company
ANALYSIS OF RANDOM NUMBER GENERATORS
USING MONTE CARLO SIMULATION
P. D. CODDINGTON
Northeast Paral lel Architectures Center, Syracuse University
111 Col lege Place, Syracuse, NY 13244, U.S.A.
Received (received date)
Revised (revised date)
ABSTRACT
Monte Carlo simulation is one of the main applications involving the use of random
numb er generators. It is also one of the b est metho ds of testing the randomness prop-
erties of such generators, by comparing results of simulations using di erent generators
with each other, or with analytic results. Here we compare the p erformanceofsome
p opular random numb er generators by high precision Monte Carlo simulation of the 2-d
Ising mo del, for which exact results are known, using the Metrop olis, Swendsen-Wang,
and Wol Monte Carlo algorithms. Many widely used generators that p erform well in
standard statistical tests are shown to fail these Monte Carlo tests.
Keywords : Random Numb er Generators, Ising Mo del, Monte Carlo Simulation
1. Intro duction
Monte Carlo simulation is an imp ortantnumerical technique for studying a wide
1
range of problems in the physical sciences. Being a probabilistic technique, it re-
2;3;4
lies heavily on the use of pseudo-random numb er generators. The generation of
random numb ers on a computer is a notoriously dicult problem. An ideal random
numb er generator would provide numb ers that are uniformly distributed, uncorre-
lated, satisfy any statistical test of randomness, have a large p erio d of rep etition,
can b e changed by adjusting an initial \seed" value, are rep eatable, p ortable, and
can b e generated rapidly using minimal computer memory.
2;5
Many statistical tests havebeendevelop ed to check for randomness, and in
most cases the p erio d of the generator can b e calculated (at least approximately).
3;4;6
As noted in a number of recent review articles, random numb er generators
provided by computer vendors or recommended in computer science texts often
have b een (and unfortunately continue to b e) of p o or quality.Even generators that
p erform well in standard statistical tests for randomness may b e unreliable in certain
7;8;9;10;11;12;13
applications, as has b een found in some Monte Carlo simulations.
There are two main typ es of random numb er generators for pro ducing sequences
of pseudo-random integers X :
i
2;6
1. Linear congruential generators (LCGs) 1
2 P.D. Coddington
X = A X + B mo d M
i i 1
whichwe will denote byL(A; B ; M ). The p erio d is M for suitably chosen A
and B (M 1ifB = 0).
2;5
2. Lagged Fib onacci generators (LFGs)
X = X X
i i P i Q
whichwe will denote byF(P; Q; ), P>Q, where is any binary arithmetic
op eration, suchas+; ; or (the bitwise exclusive OR function XOR). The
arithmetic op erations are done mo dulo any large integer value, or mo dulo 1 if
the X 's are represented as oating p ointnumb ers in the interval [0,1), as can
b e done if the op eration is + or . Multiplication is done on the set of o dd
P b 1 P b 3
integers. For b-bit precision X 's, the p erio d is (2 1)2 ,or(2 1)2
5
for multiplication, for suitably chosen lags.
It is p ossible to nd sets of parameters (A; B ; M )or(P; Q; ) for which these
twotyp es of generators work well for most practical purp oses, and it is p ossible
5
to improve the p erformance of these generators by increasing M or P . There are
practical limits on these two parameters: M should not b e very much greater than
machine precision to avoid using slowmulti-precision arithmetic, and a large lag P
means storing a large array of previous numb ers in the sequence (the \lag table")
whichmay b e sub ject to memory constraints. However on most mo dern computers
adequate values of M and P can b e found which are well within these limits.
Linear congruential generators havetwo ma jor defects. The rst is that the
least signi cant bits of the numb ers pro duced are highly correlated, and a resultant
\scatter-plot" of ordered pairs of random oating p ointnumb ers in the interval
5;14;15;16
[0,1) shows regular lattice structure. They are also known to have long-
7;9;17;18
range correlations, esp ecially for intervals whichareapower of 2. Another
32
problem is that for 32-bit integers the p erio d of these generators is at most 2 ,or
9 8
of order 10 . On a mo dern RISC workstation capable of around 10 oating p oint
op erations p er second, this p erio d can b e exhausted in a matter of minutes. This
can b e alleviated by the use of 64-bit precision, however the correlation problems
still remain (although to a lesser degree). In spite of these problems, LCGs with
well-chosen parameters p erform well in most standard statistical tests, and an LCG
(unfortunately not always with well-chosen parameters!) is provided as the default
generator on many computer systems.
Lagged Fib onacci generators using arithmetic op erations (+; ; )give good re-
5
sults in standard statistical tests with very mo dest lags on the order of tens. When
the binary op eration used is XOR, these generators are referred to as generalized
19;20
feedback shift register generators. Marsaglia has shown that XOR is one of
the worst op erations one can use in a generator of this typ e, and strongly recom-
mends the use of standard arithmetic op erations that havemuch longer p erio ds
5
and p erform much b etter in statistical tests. Although shift register generators
Analysis of Random Number Generators Using Monte Carlo Simulation 3
5;21;22
pass statistical tests when the lag is large enough (of order hundreds), very
little (apart from the p erio d) is known theoretically ab out these generators, and
they have pro duced biased results in Monte Carlo studies of the Ising mo del in
11 8 12;13
two and three dimensions, and of self-avoiding random walks.
Mixing two di erent generators is b elieved to improve p erformance in some
5;15
cases, and many generators that p erform well in statistical tests are of this kind.
27;28
Marsaglia has suggested a fast, simple Weyl (or arithmetic sequence) generator
X = X K mo d M ,
i i 1
with K a constant relatively prime to M , that can b e e ectively combined with
a lagged Fib onacci generator. Adding a Weyl generator also increases the p erio d
of the combined generator by a factor of M (the p erio d of the Weyl generator).
15
L'Ecuyer has shown howtocombine two di erent 32-bit LCGs to pro duce a mixed
18
generator that passes the scatter-plot test and has a long p erio d of around 10 ,
thus overcoming some of the drawbacks of standard LCGs. Although these mixed
generators p erform well in empirical tests, there is little theoretical understanding
of their b ehavior, and it is quite p ossible that mixing two generators mayintro duce
new defects of whichwe are unaware. A go o d single generator may therefore b e
preferable to a mixed generator.
LCGs havetheadvantage that wehave a relatively go o d (although still limited)
theoretically understanding of their randomness prop erties. They are known to
b e defective, but their defects are fairly well understo o d (for example, the lattice
2
structure of an LCG can b e determined analytically using the sp ectral test ), and
in practice they work quite well. There is clearly a need for b etter random num-
b er generators, and LFGs and mixed generators are prime candidates. However
currently there is little or no theoretical understanding of these and other genera-
tors, and they are used mainly on the basis of their p erformance in statistical tests.
They are b elieved to overcome some of the aws of LCGs, although this has not
b een proven and they may p ossess other aws of whichwe are unaware. It is there-
fore extremely imp ortant to sub ject random numb er generators to a wide variety
of precise statistical tests.
2. Monte Carlo Tests
One practical way to test a random numb er generator is to use it for Monte Carlo
1
simulation of the two dimensional Ising mo del. This simple mo del has b een solved
23
exactly for a nite lattice, so that values of the energy and the sp eci c heat (the
variance of the energy) of the system calculated from the Monte Carlo simulation
can b e compared with the known exact values.
Anumb er of di erent Monte Carlo algorithms can b e used to simulate the Ising
mo del. Here we will concentrate on the three most widely used metho ds: the
1;24
Metrop olis algorithm, which up dates a single site of the lattice at a time; the
25
Swendsen-Wang algorithm, which forms clusters of sites to b e up dated collec-
26
tively; and the Wol algorithm, which up dates a single cluster of sites. Eachof
4 P.D. Coddington
these algorithms uses random numb ers in a very di erentway. The Swendsen-Wang
and Wol cluster up date algorithms are extremely ecient and allowvery precise
Monte Carlo simulations of the Ising mo del, easily reducing statistical errors in the
5
energy to b etter than one part in 10 . This precision provides us with a very e ec-
tive practical test of the randomness of a pseudo-random numb er generator, and in
particular its suitability for Monte Carlo simulation.
11
Ferrenberg et al. recently showed that some \go o d" random numb er genera-
tors, which p erform well in standard statistical tests, fail the \Monte Carlo test";
that is, they pro duce incorrect results when used in Monte Carlo simulations of
the Ising mo del, esp ecially using the Wol algorithm. The generators studied by
Ferrenb erg et al. were:
31 2;6
i. CONG, the linear congruential generator L(16807; 0; 2 1).
20
ii. Two shift register generators, F(250; 103; ) and F(1279; 1063; ).
27
iii. SWC, a subtract-with-carry generator based on F(43; 22; ).
27
iv. SWCW, a combined subtract-with-carry and Weyl generator.
In spite of the premise of that pap er, CONG and the shift register generators are
in fact known to b e not go o d random numb er generators. CONG has b een rec-
2;6
ommended bya numb er of authors as one of the b est 32-bit linear congruential
generators, however it still su ers the small p erio d and correlated low order bits
of these generators. Shift register generators have b een criticized by Marsaglia,
who showed that those with small lags (less than 100) p erformed p o orly in sta-
5 21;22
tistical tests. However similar tests of F(250; 103; )gave go o d results, and
20
Kirkpatrick and Stoll also obtained reasonable results with Monte Carlo tests.
Subtract-with-carry generators are another variation of LFGs, where the stan-
dard op eration of subtraction is replaced by subtraction with a carry bit C ,as
follows:
X = X X C;
i i P i Q
if X 0; C =0;
i
if X < 0; X = X + M; C =1:
i i i
P Q
This greatly increases the p erio d of the LFG, to M M for suitably chosen P; Q
27 P
and M , compared to approximately M 2 for a comparable LFG using subtraction.
32
Wehaveused M =2 5, which gives very long p erio ds for mo dest lags. Although
27
advo cated by Marsaglia, there were no known published results on statistical tests
of the SWCorSWCW generators prior to the results of Ferrenb erg et al.,soagain
there was little supp ort for their claim that these are \go o d" generators. Recently
the shift-with-carry generators were in fact shown to p erform p o orly in standard
22 statistical tests.
Analysis of Random Number Generators Using Monte Carlo Simulation 5
In this pap er the work of Ferrenberg et al. has b een extended by studies of b oth
the \go o d" generators of that pap er, and some \b etter" generators, which are listed
below. In this work there are also more, and in some cases longer, indep endent runs
for each generator, to obtain b etter error estimates and to b etter explore the e ect
of di erent initial seeds.
3
In a recent review of random numb er generators, James recommends 3 mixed
generators:
1. RANECU, L'Ecuyer's mixed LCG combining L(40014,0,2147483563) and
15
L(40692,0,2147483399).
28
2. RANMAR, Marsaglia's combined LFG F(97; 33; ) and Weyl generator.
27
3. RCARRY, a subtract-with-carry generator based on F(24; 10; ) (this is the
same as SWC but with a smaller lag).
We also tested the ab ove generators, plus the following:
31
4. RAND, the default 32-bit C and Unix generator L(1103515245; 12345; 2 1).
5. DRAND48, another standard C and Unix generator with larger mo dulus and
48
p erio d, based on L(5DEECE66D ;B ; 2 ).
16 16
48
6. RANF, another 48-bit LCG, L(2875A2E7B175 ; 0; 2 ), which is the stan-
16
35
dard generator used on CRAY and CDC CYBER machines.
7. RAN2, which is RANECU augmented byshuing the order of the output
29
values.
8. LFGs of di erent lags, using +; ; and .
30;31;32
9. LFGs using + and with 4 \taps", i.e.
X = X X X X ,
i i P i Q i R i S
whichwe will denote byF(P ; Q; R; S; ).
We followed Marsaglia and James by initializing eachbitoftheseedtablesinthe
LFGs by using a combination LFG and LCG (see the routines RSTARTinRef.28
and RMARIN in Ref. 3). We also tried using RAND to initialize every elementof
the seed tables, or everybitofevery element in the seed tables, which had little or
no e ect on the qualityoftheLFGs.
For each random numb er generator, 25 indep endentsimulation runs with di er-
ent initial seeds were p erformed, on a network of IBM RS/6000, HP Ap ollo 9000,
6 7
and DEC 5000 workstations. Eachsimulation was b etween 10 and 5 10 sweeps
1;23
of a 16 16 lattice at the critical p ointofthe2-d Ising mo del. The number of
random numb ers generated p er sweep p er site varies with the Monte Carlo algo-
rithm used, with an average of 0.87 for Metrop olis, 0.93 for Wol , and 1.85 for
Swendsen-Wang. For the Metrop olis algorithm wechose to visit the sites to b e
6 P.D. Coddington
up dated in order, rather than randomly,toprovide a more e ectiveway of probing
any regularity or lattice structure in the sequence of random numb ers, esp ecially
for the linear congruential style generators which are known to su er from this
5;7;14;15;16;35
problem.
Error estimates for eachsimulation were obtained by standard metho ds of bin-
1
ning the data, with a bin size much greater than the auto correlation time. The
error in the mean of the 25 combined results was also calculated, treating them as
indep endent data sets. Two measures were used to compare the Monte Carlo results
with the exact results: the deviation b etween the mean of the combined results
and the exact value as a multiple of the error in the mean , and the chi squared
2 36
p er degree of freedom for the 25 data sets. The rst test checks for any bias
in the average over all runs, the second checks for discrepancies in the statistical
uctuations exp ected b etween the individual runs. A generator is judged to have
2 2
failed the Monte Carlo test if > 3:3 , > 2:0, or < 0:34, all of which should
36
o ccur with probability less than 0.001 for a truly random generator.
3. Results
The results for = , the di erence b etween the exact and simulated values of the
energy and sp eci c heat given as a multiple of the errors in the mean, are presented
2
in Tables 1 and 2, along with the values of .Failure of a test is indicated in b old
typ e. The generators are group ed into 4 categories, determined by a di erent level
of precision of the simulations. Table 1 shows generators whichwewould classify
as bad or very bad (at least for this typ e of Monte Carlo application). The very
6
bad generators failed at least one of the tests with 10 Monte Carlo sweeps p er run,
7
with the bad generators failing after 10 sweeps p er run.
Table 2 shows generators whichwewould classify as go o d or very go o d. The
7
go o d generators failed one of the tests at a level of 5 10 sweeps for the Wol and
7
Metrop olis algorithms, and 10 sweeps for the SW algorithm (whichusesabouttwice
as many random numbers per sweep). The very go o d generators passed all the tests
10
at this level, whichinvolves generating on the order of 10 random numb ers for
11
each of the 25 indep endent simulations, or approximately 3 10 random numbers
in total. In contrast, the errors caused by using very bad generators were generally
9
apparent after using less than 10 random numb ers, in simulations whichtookonly
aboutanhouronaworkstation.
Fig. 1 shows the relative error in the sp eci c heat for the Wol algorithm versus
the lag of the Fib onacci generator, for the binary op erations addition, subtraction,
and XOR. In all cases the XOR op eration was ab out an order of magnitude worse
than addition and subtraction. Since in Monte Carlo simulation an order of mag-
nitude decrease in the error requires 100 times as many iterations, the di erence
between the qualityoftheLFG with di erent op erations is substantial. Quite large
lags of at least 1000 are required to reduce the error to less than 0:1%, however