DISCRETE AND CONTINUOUS Website: http://math.smsu.edu/journal DYNAMICAL SYSTEMS–SERIES B Volume 2, Number 2, May 2002 pp. 153–167

DISCRETE AND CONTINUOUS RATCHETS: FROM COIN TOSS TO

David Heath Department of Mathematical Sciences Carnegie Mellon University Pittsburgh, PA 15213 David Kinderlehrer Department of Mathematical Sciences Carnegie Mellon University Pittsburgh, PA 15213 Micha lKowalczyk Department of Mathematical Sciences Kent State University Kent, OH 44242, U.S.A.

Abstract. Directed or ratchet-like behavior in many molecular scale systems is a consequence of diffusion mediated transport. The Brownian motor serves as a paradigm. The Parrondo Paradox is a pair of coin toss games, each of which is fair, or even losing, but become winning with a schedule of playing them in alternation. It has been proposed as a discrete analog of the Brownian motor. We examine the relationship between these two systems. We discover a class of Parrondo games with unusual ratchet-like behavior and for which diffusion plays a fundamentally different role than it does in the Brownian motor. Detailed balance is an important feature in these considerations. The Brownian motor depends on details of the potential landscape in the system but the Parrondo game is decided on the potential difference alone. There are winning Parrondo games whose Brownian motor analogs move in the opposite direction. A general framework is discussed in section 7. The orig- inal Parrondo game, here in section 7.2, is completely determined by detailed balance.

1. Introduction. Directed motion, or ratchet-like behavior, in many molecular scale systems is a consequence of diffusion mediated transport. This is especially believed to be the situation for motor responsible for intracellular traffic in eukarya, where the Brownian motor serves as a paradigm, [1, 7, 8, 10, 14]. Other work in the is done by similar motor proteins, [12, 13]. It also has a role in the explanation of the Janossy effect, a actuated dye/nematic crystal interaction, [9]. Additional examples are found in the study of lipid bilayers, [15] and dielectrophoresis, [2]. All of these systems, extremely complex, appear to share this feature: there is a collaboration between a diffusive process, which tends to spread density uniformly through the medium, and a transport process, which concentrates density at specific sites, that results in movement through the

1991 Mathematics Subject Classification. 35K15, 35K20, 35B40, 60J10, 60J20, 60J60, 91A60, 92C37. Key words and phrases. Diffusion mediated transport, Brownian Motor, Parrondo Paradox.

153 154 D. HEATH, D. KINDERLEHRER AND M. KOWALCZYK system. Each process taken separately rapidly approaches equilibrium without net movement of density. The Parrondo Paradox is a pair of coin toss games, each of which is naively fair, or even losing, but a schedule of playing them in alternation can become winning. The result is directed motion of capital, although this outcome is favored by neither constituent game. It has been proposed as a discrete analog to the flashing ratchet example of a Brownian motor. Here we explore this game and explain that, although it functions quite differently from a Brownian motor, it has a novel ratcheting mechanism of its own. Our task, in summary, is to compare two dynamical systems, both of which are distant from equilibrium, one discrete and one continuous. Here we shall be concerned primarily with a Parrondo game played modulo 4, see below. In the course of our discussion, the notion of what fair means will evolve. In some sense, this game is not ergodic. We also address a game played modulo 3, see section 7.2, which is ergodic. Although many other games may be constructed, we believe that we have uncovered the important mechanisms that explain how these games function.

2. The Parrondo paradox and the flashing ratchet. We begin with an ex- ample of the Parrondo paradox which consists of a pair of coin-toss games, an A game and a B game [3, 4, 11]. The A game is single toss of a fair coin, to win or to lose a dollar. The B game has two parts. If the present capital is divisible by 4, then a coin with very unfair probability p is played. Otherwise a coin with quite favorable probability p0 is played. The numbers p and p0 are chosen so that the na¨ıve expectation of the B game is zero, or perhaps even less than zero. Let us say 1 3 EB = (2p − 1) + (2p0 − 1) = 0. (2.1) 4 4 Although both the A game and the B game are fair in the sense of (2.1), playing in alternation, eg, ABAB . . . or AABBAABB . . . is a winning strategy. We can seek to understand this by constructing an analogy with the flashing ratchet, the simplest example of diffusion mediated transport. The prototype and most accessible formulation of a Brownian motor is the flash- ing ratchet. It consists in solving the diffusion equation and the Fokker-Planck Equation for a periodic potential in alternation. Let ψ be a smooth potential func- tion in Ω = (0, 1) such that 1 ψ ≥ 0 and ψ periodic of period X = , b = ψ0. N Given the diffusion equation ∂ρ ∂2ρ = σ in Ω, t > 0 ∂t ∂x2 ∂ρ σ = 0 on ∂Ω, t > 0 (2.2) ∂x and the Fokker-Planck Equation ∂ρ ∂2ρ ∂ = σ + (bρ) in Ω, t > 0, ∂t ∂x2 ∂x ∂ρ σ + bρ = 0 on ∂Ω, t > 0, (2.3) ∂x DISCRETE AND CONTINUOUS RATCHETS 155

Figure 1. Dirac masses at 1/8 and 5/8 evolved according to a diffusion equation and the potential of a two toothed ratchet. Much more density has diffused into the left well from the right than vice-versa choose two time intervals Tdiff , diffusion time, and Ttr, transport time, and solve (2.2) for a time Tdiff and (2.3) for a time Ttr, with initial datum the preceding solution at time Tdiff , and iterate this process. We are flashing between the two equations. There has been extensive discussion concerning whether or not it is possible to have some net motion from this type of system in so far as this would appear to extract work from fluctuations. In fact, the Gibbs distribution of (2.2) is

ρdiff (x) ≡ 1 in Ω (2.4) and of (2.3) Z 1 −ψ(x)/σ −ψ(x)/σ ρFP (x) = e ,Z = e dx in Ω (2.5) Z Ω Naively, we expect the solution of our problem to oscillate between the two Gibbs distributions, resulting in no net movement of density. The intuitive surmise is that this solution will oscillate, for example, as a convex combination of ρFP and ρdiff . This does not represent transport since the mass of ρ will tend to be equally distributed in all the period intervals of ψ. Nor need it happen. The essential requirement for transport is that the minimum of ψ be located asymmetrically with respect to the local potential well. To see this, consider Figure 1 where two equal Dirac masses have diffused to a pair of gaussians. Because of the asymmetry, a greater proportion of the mass from the right has moved to the left well than vice versa. Transport with respect to the potential acts, essentially, to collect density in a period basin into a Dirac mass at the potential minimum. As a result of the 156 D. HEATH, D. KINDERLEHRER AND M. KOWALCZYK diffusion, there is now more mass in the left well than the right well, so after the transport step the new Dirac mass on the left will be larger than the one on the right. This is not the whole story, for other requirements see [5, 6], but it explains the importance of asymmetry in the potential.

3. The flashing ratchet analogy for the Parrondo game. First recall that an approximation to (2.3) with mesh length h and time step τ is given by στ τ ρ(x, t + τ) − ρ(x, t) = δ2ρ(x, t) + [(br) − (br) ], (3.1) h2 2h x+h x−h δ2ρ(x, t) = ρ(x + h, t) − 2ρ(x, t) + ρ(x − h, t). i Now let ρk denote the probability distribution of capital k at play i. Then i+1 i i ρk = pk−1ρk + (1 − pk+1)ρk+1, where (3.2) pk = probability of success from position k. This may be rewritten as 1 1 ρi+1 − ρi = (ρi − 2ρi + ρi ) + [−(2p − 1)ρi + (2p − 1)ρi ]. k k 2 k−1 k k+1 2 k+1 k+1 k−1 k−1 Thus 1 1 ρi+1 − ρi = (ρi − 2ρi + ρi ) + (b ρi − b ρi ), (3.3) k k 2 k−1 k k+1 2 k+1 k+1 k−1 k−1 with  −(2p − 1), k = 0 mod 4, b = −(2p − 1) = (3.4) k k−1 −(2p0 − 1), k = 1, 2, 3 mod 4. and so we arrive at a well known finite difference approximation to (3.2) where the drift term b assumes exactly two values. This is precisely the situation for the equation (2.3) of the flashing ratchet when the potential is a piecewise linear ’tent’ function. (2.1) ensures its periodicity. The fact that the minimum of the potential is asymmetrically located with respect to its basin of attraction is precisely the statement that one decision is made for c = 0 mod 4 and a different one for c = 1, 2, 3 mod 4. A Parrondo game may be realized by the pair of difference equations i+1 i 1 i i i A game ρk − ρk = 2 (ρk−1 − 2ρk + ρk+1), i+1 i 1 i i i 1 i i B game ρk − ρk = 2 (ρk−1 − 2ρk + ρk+1) + 2 (bk+1ρk+1 − bk−1ρk−1) (3.5) run according to a schedule, like ABAB . . . or AABBAABB . . . By analogy with the flashing ratchet, we can expect positive accumulation of capital, transport to the right, when p is small so that the minimum of the potential determined by {bk} is closer to its maximum to the right. We simulated the game directly but found no movement of capital. Playing the B game alone was found to be losing and not fair. What went wrong?

4. The Parrondo game redux. Playing the B game is not fair because not all integers mod 4 occur with the same frequency. In fact, the A game and the B game are Markov chains. To generalize, only slightly, our previous description of the B game, let pk, k = 1,..., 4, denote the success probabilities of 4 coins and suppose that we play coin k if the present capital c = (k − 1) mod 4. With

PBij = Pr(c mod 4 = j − 1 | c mod 4 = i − 1), 1 ≤ i, j ≤ 4, (4.1) DISCRETE AND CONTINUOUS RATCHETS 157 and   0 p1 0 1 − p1  1 − p2 0 p2 0  PB =   (4.2)  0 1 − p3 0 p3  p4 0 1 − p4 0 we obtain a transition matrix for the integers mod 4 for the B game. We refer to PB as a parity matrix. Likewise, the A game has the parity matrix  1 1  0 2 0 2 1 1  2 0 2 0  PA =  1 1  (4.3)  0 2 0 2  1 1 2 0 2 0 In this context we should refer to the B game as (asymptotically) fair provided that its expectation with respect to the stationary distributionρ ˆ of PB vanishes, namely, provided that X EasympB = (2pk − 1)ˆρk = 0. (4.4) This is a constraint on the B game. We can clarify our thoughts by briefly studying this condition and the separate condition of detailed balance. Detailed balance, or reversibility, (for a Markov chain) with outcomes a random variable X is the condition that in equilibrium the joint probability of (Xi,Xj) is the same as the joint probability of (Xj,Xi), or, in our case,

ρˆiPBij =ρ ˆjPBji. (4.5)

Explicitly,ρ ˆiPBii+1 =ρ ˆi+1PBi+1i, etc., or

PB12 p1 ρˆ2 = ρˆ1 = ρˆ1, PB21 1−p2

PB23 p2 ρˆ3 = P ρˆ2 = 1−p ρˆ2, B32 3 (4.6) PB34 p3 ρˆ4 = ρˆ3 = ρˆ3, PB43 1−p4

PB41 p4 ρˆ1 = ρˆ4 = ρˆ4. PB14 1−p1

This provides two expressions forρ ˆ4, which when combined yield that detailed balance is equivalent to Y pk (DB) = 1. (4.7) 1 − pk The condition is identical for detailed balance of an M-coin game. Suppose now (4.6), or equivalently, (4.7) holds. Then X X EasympB = (2pk − 1)ˆρk = [pkρˆk − (1 − pk)ˆρk]

=ρ ˆ1PB12 − ρˆ1PB14 +ρ ˆ2PB23 − ρˆ2PB21 +ρ ˆ3PB34 − ρˆ3PB32

+ˆρ4PB41 − ρˆ4PB43

= (ˆρ1PB12 − ρˆ2PB21) + (ˆρ2PB23 − ρˆ3PB32) + (ˆρ3PB34 − ρˆ4PB43)

+(ˆρ4PB41 − ρˆ1PB14). (4.8)

So EasympB = 0 if detailed balance holds, and the B game is asymptotically fair, as expected. Now observe the equilibrium equations

ρˆ =ρP ˆ B 158 D. HEATH, D. KINDERLEHRER AND M. KOWALCZYK are the equations ρˆ1 = (1 − p2)ˆρ2 + p4ρˆ4

ρˆ2 = p1ρˆ1 + (1 − p3)ˆρ3 (4.9) ρˆ3 = p2ρˆ2 + (1 − p4)ˆρ4

ρˆ4 = p3ρˆ3 + (1 − p1)ˆρ1 Thus ρˆ1PB12 − ρˆ2PB21 = p1ρˆ1 − (1 − p2)ˆρ2

=ρ ˆ2 − (1 − p3)ˆρ3 − (1 − p2)ˆρ2 (4.10) = p2ρˆ2 − (1 − p3)ˆρ3

=ρ ˆ2PB23 − ρˆ3PB32, and so forth, so the terms making up the sum in (4.8) are all equal. Thus

EasympB = 0 implies all the terms must vanish, so (DB) holds. To summarize, the B game is asymptotically fair if and only if it satisfies detailed balance. On the other hand, the na¨ıve version of ’fair game’, we shall refer to as ’fair for the uniform distribution’ and it takes the obvious form 1 X E B = (2p − 1) = 0. (4.11) unif 4 k It is unlikely that a given set of coins satisfies both conditions. One way of viewing the situation, indeed, is to say that playing the B game is playing a Markov chain with transition matrix PB and playing the A game and the B game in succession is playing another Markov chain, with a periodic transition matrix (k) P = PA for k odd and PB for k even, rather than a constant one. At any rate there is no reason to expect the two chains to have any special relationship. But they do.

5. Unraveling the Parrondo game. Summarizing our situation, we have two coin games, an A game and a B game with parity matrices PA and PB. These are transition matrices of Markov chains that determine the relative frequency mod 4 of capital. For an arbitrary Markov chain with transition matrices P (k) played with coins with probabilities of success p1, . . . , p4, the distribution after k trials is ρ(k) = ρ(k−1)P (k) = ρ(0)P (1) ··· P (k) (5.1) and the expectation at the kth trial is (k) (k−1) X (k−1) E = Eρ = (2pi − 1)ρi . (5.2) The remarkable feature of the Parrondo game is that  1 1  2 0 2 0 1 1 2  0 2 0 2  PBPA = PA =  1 1  . (5.3)  2 0 2 0  1 1 0 2 0 2 This means that the sequence of distributions {ρ(k)} is periodic. More exactly, let 1 1 1 1 ρ(0) = ρ = ( , , , ) (5.4) unif 4 4 4 4 DISCRETE AND CONTINUOUS RATCHETS 159 denote the uniform distribution-other situations may be treated identically-and suppose, for illustration, we play BABA . . . . Introduce the notations X X  1  E ρ = (2p − 1)ρ and E ρ = 2 − 1 ρ = 0. (5.5) B i i A 2 i Then 1 X ρ(1) = ρ P and E(1) = E ρ = (2p − 1), unif B B unif 4 i (2) 2 (2) ρ = ρunif PBPA = PAρunif = ρunif and E = EAρunif PB = 0, 1 X ρ(3) = ρ P and E(3) = E ρ = (2p − 1), unif B B unif 4 i (4) (4) ρ = ρunif PBPA = ρunif and E = EAρunif PB = 0, etc. The sequence of distributions is

{ρunif , ρunif PB, ρunif , ρunif PB, ρunif , ρunif PB,... } (5.6) So after a single play of the A game we are returned to the initial distribution, independent of the choice of PB. The expectation of a period of the game is the expectation of the uniform distribution ρunif with respect to the B game, 1 E ρ = (2p − 1). (5.7) B unif 4 i We may interpret this as a ratchet mechanism where the diffusion matrix resets the system to the initial distribution after each cycle, regardless of the probabilities of the b game. This is fundamentally different from the diffusion mechanism in the flashing ratchet, where mass is isotropically spread by diffusion. We shall have more to write about this later on. The Parrondo game resets the distribution in the manner of a screw in soft wood whose threads are stripped. When extracting the screw, after a turn or two it will emerge slightly but “ratchet” back into the hole. The algebraic interpretation of this situation is quite obvious. Given an ergodic (∞) k Markov chain with transition matrix P , P = limk→∞ P is the matrix all of whose rows are equal to the stationary vector for P . Our A game matrix PA is just a concatenation of two such 2×2 P (∞) matrices and thus maps any probability vector composed of just odd or even components to the odd or even uniform distribution. The special feature here is that in the 4 × 4 case, PA corresponds to a coin toss game. There are also special considerations here because of the constant odd-even switching, but this is elementary. Returning to the original description of the Parrondo game, one strategy is to choose coins with probabilities p and p0 satisfying the detailed balance criterion (4.7) p  p0 3 = 1. (5.8) 1 − p 1 − p0 (2.1) will no longer be valid. Indeed, we may choose p, p0 so that 1 3 E ρ = (2p − 1) + (2p0 − 1) > 0, (5.9) B unif 4 4 for example. The B game will be fair but playing in alternation, according to (5.6) and (5.7), will be winning. Figure 2 depicts the uniformly fair and detailed balance curves in this situation. Likewise, we could choose p0 small and p large; many options are now available and their outcome easily predicted. Obviously for a game 160 D. HEATH, D. KINDERLEHRER AND M. KOWALCZYK

Figure 2. Detailed balance and uniform distribution fair game lines for the game B. Above the curved line (p0, p) choice is win- ning for the detailed balance distribution and above the straight segment (p0, p) is winning for the uniform distribution. Choosing (p0, p) = (0.65, 0.2) results in a fair B game which is a winning Parrondo game. of period 2r, where the B game is played r times followed by r plays of the A game, the same analysis applies.

6. The flashing ratchet. To better compare the coin game mechanism with the 1 flashing ratchet, we review the latter, cf. [1, 6]. Let us set X = N , choose a, 0 < a < X, and i i − 1 x = , i = 0,...,N, and a = a + , i = 1,...,N, i N i N so that 0 = x0 < a1 < x1 < a2 < ··· < xN = 1. Suppose that ψ is a smooth function with

ψ(xi) = relative maximum and ψ(ai) = relative minimum, and ψ is decreasing on the intervals (xi−1, ai) and increasing on the intervals (ai, xi). Choose parameters Ttr > 0, Tdiff > 0, and let f(t) = 1 for 0 ≤ t < Ttr and f(t) = 0 for Ttr ≤ t < T = Ttr + Tdiff . Extend f periodically of period T . Consider the system, with b = ψ0 as usual, ∂ρ ∂2ρ = σ + f(t)(bρ) in Ω, t > 0, ∂t ∂x2 ∂ρ σ + f(t)bρ = 0 on ∂Ω, t > 0, (6.1) ∂x This system, as we have noticed, does not have a stationary state, but under some mild hypotheses does have a time periodic solutionρ ˆ(x, t). DISCRETE AND CONTINUOUS RATCHETS 161

During the “transport” phase, when kT ≤ t ≤ kT + Ttr, k = 0, 1, 2,... , (6.1) behaves approximately like the transport equation ∂ρ ∂ = (bρ) in Ω, t > 0, (6.2) ∂t ∂x and for finite time periods, namely, a time duration of Ttr , is close to an appropriate solution ρ# of this equation. To understand the behavior of ρ#, we solve (6.2) by characteristics. Inspection of the signs of b(x) in the intervals (xi−1, ai) and (ai, xi) reveals immediately that Z # ∗ # r → Cδai as t → ∞ where C = ρ (x, 0) dx, (6.3) E

E = (xi−1, xi), or the transport concentrates all the mass at the well minimum. That estimates of closeness between and ρ# only hold for finite times is evidence that the system is operating out of equilibrium in a metastable environment. As we have already mentioned, if Ttr → ∞ or Tdiff → ∞, the solution approximates (2.4) or (2.5) and does not exhibit transport, cf. [6] for additional details. The next ingredient is that the asymmetry of ai within the well now permits diffusion to reorganize the mass among the wells (xi−1, xi), as described in Figure 1. These features allow us to construct a Markov chain that does approximate the flashing ratchet. Suppose that ∗ X ∗ ∗ X ∗ µ = µi δai , 0 ≤ µi ≤ 1, µi = 1, and let w denote the solution of ∂w ∂2w = σ in Ω, t > 0, ∂t ∂x2 ∂w = 0 on ∂Ω, t > 0, (6.4) ∂x w = µ∗ at t = 0.

At time t = Tdiff , the mass has been reorganized and we have new partial densities given by Z µj = w(x, Tdiff ) dx, Ej = (xj−1, xj). Ej It is easy to check that we may write a new measure X ∗ µ = µjδai with µ = µ P, (6.5) where P is a matrix given explicitly in terms of Green’s functions of (6.4) with singularities at the ai. Letµ ˆ denote the stationary vector of this (ergodic) Markov chain. It is possible to prove thatµ ˆ andρ ˆ are close in some weak topology sense. Now if the points ai lie 1 to the left of the midpoints 2 (xi−1 + xi) of the intervals Ei, then the massesµ ˆi will be ordered so that there is more total mass in the left half of Ω than on the right and vice versa. The same is true forρ ˆ. The determining feature for transport is the asymmetric location of the minima of the potential, not the potential difference over the period intervals. Indeed, it should be possible to exhibit parameters for which a flashing ratchet moves density to the left while the discrete Parrondo game is winning, that is, moves capital to the right. We are able to do this. 162 D. HEATH, D. KINDERLEHRER AND M. KOWALCZYK

Figure 3. The dark triangle is the region where the Parrondo game is winning and the ratchet moves to the left. The shaded squares define monotone potentials and are excluded from ratchet- like behavior.

In our original Parrondo game played with two coins of success probabilities p, p0, the probabilities lie in the square 0 ≤ p0, p ≤ 1, cf. Figure 3. The condition for a winning Parrondo game is EBρunif > 0. (6.6) The condition for an oscillatory potential in a flashing ratchet is that the slope of b changes sign, or (1 − 2p)(1 − 2p0) < 0 (6.7) which excludes the shaded squares in Figure 3. Condition (6.6) is that 1 3 (2p − 1) + (2p0 − 1) > 0 or p + 3p0 > 2. (6.8) 4 4 If 1 − 2p < 0 and 1 − 2p0 > 0, density in the flashing ratchet will be transported to the left. Combining this with (6.8), we see that choosing (p0, p) in the triangle (1/2, 1/2), (1/2, 1), (1/3, 1) leads to a winning Parrondo game but a Brownian motor with transport to the left. Figure 3 depicts the potential for this motor and Figure 4 illustrates the transport when p0 = 0.44 and p = 0.8, approximately the centroid of the triangle.

7. General framework and variations. Suppose that we have in hand M × M probability matrices P1,...,Pn, set Q = P1 ...Pn and define a Markov chain by the schedule (k) P = Pi, if k = i mod n. (7.1) Then the subsequence of distributions ρ(jn) given by ρ(jn) = ρ((j−1)n)Q (7.2) is a particular Markov chain and tends to a stationary stateρ ˆ of Q. The interme- diate states are close to the periodic sequence of distributions

{ρ,ˆ ρPˆ 1,..., ρPˆ 1 ...Pn−1}, DISCRETE AND CONTINUOUS RATCHETS 163

Figure 4. Potential for winning Parrondo game where Brownian motor moves density to the left. Four wells of this potential are depicted on the unit interval with minimum at points k/4 + 1/16, corresponding to bets in a Parrondo game of 1/16.

Figure 5. Snapshots of the periodic solution (corresponding to ψ in Figure 3) at the end of diffusion phase (wavy line) and the transport phase (peaked curve). namely (jn+i) ρ ≈ ρPˆ 1 ...Pi, i = 1, . . . , n − 1. (7.3)

When Q is ergodic,ρ ˆ is unique and depends on all the Pi. The underlying feature of the Parrondo game is that special choices of the Pi can produce surprising behavior. We have seen that in the 4 × 4 case with a sequence defined by two matrices PA 2 and PB, the associated Q = PA is not ergodic and as long as the initial distribution 164 D. HEATH, D. KINDERLEHRER AND M. KOWALCZYK

Figure 6. Mesh dependence diagram for B, A game sequence.

ρ(0) satisfies 1 ρ(0) + ρ(0) = ρ(0) + ρ(0) = , (7.4) 1 3 2 4 2 we are immediately “ratcheted back” to the uniform distribution ρunif . When the Pi have odd dimension and non-zero rows, they are ergodic andρ ˆ depends generally on all the matrices. With this in mind, there are two variations of the Parrondo game we shall explore. First we consider a game played with 2N coins, where we shall devise a strategy to return us to the uniform distribution. Then we discuss the game with mod 3, which is the game originally introduced by Parrondo and Abbott and Harmer.

N 7.1. Consider a B game played with 2 coins with success probabilities p1, . . . , pn, n = 2N , where coin k is played if present capital c = (k − 1) mod n. This gives rise to a parity matrix PB = (PBij), where

PBij = Pr(c mod n = j − 1 | c mod n = i − 1), 1 ≤ i, j ≤ n, (7.5) and a corresponding parity matrix PA. It is easier to visualize this if we simply i return to the notations of section 3 and let ρk denote the probability distribution of capital k at play i. Then i+1 i i ρk = pk−1ρk−1 + (1 − pk+1)ρk+1, (7.6) where pk is the probability of success from position k, and, obviously, the subscripts are all taken mod n. After playing the sequence B, A the distribution, cf. the mesh dependence sketch Figure 6, is 1 1 1 ρi+2 = [p + (1 − p )]ρi + (1 − p )ρi + p ρi k 2 k k k 2 k+2 k+2 2 k−2 k−2 1 1 1 = p ρi + ρi + (1 − p )ρ . (7.7) 2 k−2 k−2 2 k 2 k+2 k+2 So odd and even values of capital have been separated in the obvious sense that DISCRETE AND CONTINUOUS RATCHETS 165

Figure 7. Mesh dependence diagram for B game, the A game, and doubling the bet.

i+2 ρk depends only on its nearest neighbors of the same parity. To take advantage of this situation, we shall invoke a strategy playing a fair game where we double i+3 our bet. So let us determine ρk by, cf Figure 7, 1 1 ρi+3 = ρi+2 + ρi+2 k 2 k−2 2 k+2 1 1 1 1 = [ ρi + (1 − p )ρi + p ρi ] 2 2 k−2 2 k k 2 k−4 k−4 1 1 1 1 + [ ρi + (1 − p )ρi + p ρi ] 2 2 k+2 2 k+4 k+4 2 k k 1 1 1 1 1 = p ρi + ρi + ρi + ρi + (1 − p )ρi (7.8) 4 k−4 k−4 4 k−2 4 k 4 k+2 4 k+4 k+4 N For example, if n = 2 = 8, then k − 4 = k + 4 mod 8, so that pk−4 = pk+4 and we have that 1 1 1 1 ρi+3 = ρi + ρi + ρi + ρi . (7.9) k 4 k−2 4 k 4 k+2 4 k+4 i i In particular, if ρi = (ρ1, . . . , ρn) is the uniform distribution, then after plays with two fair games, the first with one dollar bets and the second with two dollar bets, we are returned to the uniform distribution. In general, we continue with this procedure of doubling the bet until we have completed N −1 such fair trials. We shall then be returned to the uniform distribu- tion. Each of these trials corresponds to a Markov chain. Analogous to the 4-coin case, the product of the transition matrices is just a P (∞) matrix for the uniform distribution. Again, the interesting feature is that P (∞) is a finite product of fair coin toss games. 166 D. HEATH, D. KINDERLEHRER AND M. KOWALCZYK

7.2. The Parrondo game first discussed in [3] was based on A and B games played mod 3, namely, if present capital c = (i − 1) mod 3 then coin i with success prob- ability pi is played in the the next toss. This gives rise to parity matrices    1 1  0 p1 1 − p1 0 2 2 P =  1 − p 0 p  and P =  1 0 1  . (7.10) B  2 2  A  2 2  1 1 p3 1 − p3 0 2 2 0 Playing the A game and the B game in alternation then gives rise to a Markov chain with transition matrices ( PB k odd P (k) = (7.11) PA k even exactly as in section 5. Unlike the case of mod 4, our new PA and PB are er- godic and there does not seem to be as simple way to ratchet back to a specific distribution, like the uniform distribution. It does have special properties, namely 1 P P = (1 + P ∗ ), (7.12) B A 2 B ∗ where PB stands for the B game matrix with success probabilities 1 − p1, 1 − p2, 1 − p3, that is, reversing the roles of the sides of the coins. Let us refer to this game as the B∗ game. Beginning with an initial distribution ρ(0), we have (1) (0) ρ = ρ PB (2) (0) ρ = ρ PBPA (2k) (0) k (2k+1) (2k) ρ = ρ (PBPA) with E = EBρ , (2k) (2k−1) (2k) while E = EAρ = 0. After many plays, ρ ≈ ρˆ, whereρ ˆ is the stationary vector for the Markov chain with transition matrix PBPA. The equation forρ ˆ is 1 1 ρˆ =ρP ˆ P = ρˆ + ρPˆ ∗ . B A 2 2 B Hence ∗ ρˆ =ρP ˆ B, (7.13) ∗ orρ ˆ is the stationary vector for the Markov chain with transition matrix PB, the B∗ game. ∗ Now if PB satisfies detailed balance then so does PB, and both games are fair, cf. the (DB) condition (4.7). Indeed, the B game is losing or winning according to p1p2p3 f(p1, p2, p3) = < 1 or (1 − p1)(1 − p2)(1 − p3)

f(p1, p2, p3) > 1. (7.14)

This is easy to compute since the stationary vector ρ of PB satisfies

ρ ∝ (1 − p2 + p2p3, 1 − p3 + p1p3, 1 − p1 + p2p1) (7.15) so that EBρ ∝ 3[p1p2p3 − (1 − p1)(1 − p2)(1 − p3)], (7.16) where ∝ stands for proportional with a positive constant of proportionality. ∗ The strategy then is to choose p1, p2, p3 so the B game is losing but the B game is winning. On the other hand,

f(p1, p2, p3)f(1 − p1, 1 − p2, 1 − p3) = 1, (7.17) DISCRETE AND CONTINUOUS RATCHETS 167 so it is always the case that if the B game is losing the B∗ game is winning. The effect of the diffusion-like matrix is to reverse the long time behavior of the game, which can be interpreted as ratchet like, but which we consider more as a single twist of a wrench. This explains the original Parrando Paradox. It is completely determined by detailed balance.

Acknowledgments. David Kinderlehrer was partially supported by NSF Grant DMS 0072194. Micha lKowalczyk was partially supported by the Center of Nonlin- ear Analysis at Carnegie Mellon University, the NSF Mathematical Sciences Post- doctoral Fellowship DMS-9705972.

REFERENCES . [1] Astumian, R. D., Thermodynamics and kinetics of a Brownian motor, Science 276, (1998),917–222. [2] Gorre-Talini, L., Spatz, J. P., and Silberzan, P., Dielectrophoretic ratchets, Chaos, 8.3, (1998), 650–656. [3] Harmer, G. and Abbott, D., Losing strategies can win by Parrondo’s paradox, Nature, 402, (1999), 864. [4] Harmer, G., Abbott, D., Taylor, P., and Parrondo, J., Brownian ratchets and Parrondo’s games, CHAOS, 11,(2001), 705–714. [5] Kinderlehrer, D. and Kowalczyk, M., Remarks about diffusion mediated transport, Ric. Mat. XLIX suppl., (2000), 305–318. [6] Kinderlehrer, D. and Kowalczyk, M., Diffusion mediated transport and the flashing ratchet, Arch. Rat. Mech. Anal., to appear. [7] Okada, Y. and Hirokawa, N., A processive single-headed motor: super-family KIF 1A, Science, 283, (1997),1152–1157. [8] Okada, Y. and Hirokawa, N., Mechanism of the single headed processivity: diffu- sional anchoring between the K-loop of kinesin and the C terminus of tubulin, Proc. N.A.S., 7.2, (2000), 640–645. [9] Palffy-Muhoray, P. and E., W., Orientational ratchets and angular momentum bal- ance in the Janossy effect, Mol. Cryst. Liq. Cryst., 320, (1998),193–206. [10] Parmeggiani, A., J¨ulicher, F., Adjari, A., and Prost, J., transduction of isother- mal ratchets: Generic aspects and specific examples close to and far from equi- librim, Phys. Rev. E., 60.2, (1999), 2127–2140. [11] Parrondo, J., Harmer, G., and Abbott, D., New paradoxical games based on Brownian motors, Phys. Rev. Lett., 85.24, (2000), 5226–5229. [12] Peskin, C. S., Ermentrout, G. B., and Oster, G. F., Rotary DNA motors, Biophys. J., 69, (1995), 2256–2267. [13] Peskin, C. S., Ermentrout, G. B., and Oster, G. F., The correlation ratchet: a novel mechanism for generating directed motion by ATP hydrolysis, in Cell Mechanics and Cellular Engineering (Mow, V. C. et al. eds), Springer, New York 1995. [14] Vale, R. D. and Milligan, R. A., The way things move: looking under the hood of molecular motor proteins, Science, 288, (2000), 88–95. [15] van Oudenaarden, A. and Boxer, S., Brownian ratchets: molecular separations in lipid bilayers supported on patterned arrays, Science, 285, (1999), 1046–1048. Revised version received November 2001. E-mail address: [email protected] E-mail address: [email protected] E-mail address: [email protected]