Supplementary Information for TRAM: Estimating equilibrium expectations from time-correlated simulation data at multiple thermodynamic states Antonia S. J. S. Mey,1, ∗ Hao Wu,1, y and Frank Noe´1, z 1Freie Universit¨atBerlin, Arnimallee 6, 14195 Berlin (Dated: October 1, 2014) ∗Electronic Address: [email protected] yElectronic Address: [email protected] zElectronic Address: [email protected]; to whom correspondence should be addressed 1 I. PROOFS Here we proof the asymptotic convergence of xTRAM, and that the TRAM estimation equations (25,26,27) of the main manuscript become identical to the MBAR equations for the special case that each simulation samples from the global equilibrium at its respective thermodynamic state. A. Asymptotic convergence of xTRAM In the statistical limit N ! 1, we can use that the transition counts converge to their conditional expectation values: I I I cij Ni pij = (1) N N I I IJ bij N p = i i : (2) N N Inserting these into the xTRAM estimator for equilibrium probabilities results in the up- date: n I I I I m I IJ J JI 1 X Ni pij + Nj pji 1 X N p + N p xI = wI + i i i i ; i I N I N I N J (3) 2 Ni j 2 i i j=1 I + I J=1 wI πI + wJ πJ πi πj i i I I I I I I IJ J J JI using reversibility (πi pij = πj pji and w πi pi = w πi pi ), we get: πI I I n I I I I i m I IJ J IJ w πi Ni pij + Nj pij I N p + N p J J 1 X πj 1 X i i i i w π xI = wI + i i I N I N I N J (4) 2 Ni j 2 i i j=1 I + I J=1 wI πI + wJ πJ πi πj i i n m 1 X 1 X = wI πI pI + wI πI pIJ (5) 2 i ij 2 i i j=1 J=1 I I = w πi : (6) B. Free energies In order to show the relationship between TRAM and MBAR, we use the TRAM equa- tions and specialize them using the MBAR assumption that each thermodynamic state is sampled from global equilibrium. In order to relate the TRAM and MBAR free energy 2 estimates, f I , we merge all configuration states to one state. When converged, the TRAM equations (25,26,27) of the main manuscrip then become: bIJ + bJI πI pIJ = N I N J (7) πI + πJ N 0 = − ln πI : (8) N I From the second equation, we obtain πI = N I =N. Inserting into the first equation yields: 2N I pIJ = bIJ + bJI : (9) summing over J on both sides: X 2N I = (bIJ + bJI ) (10) J X = N I + bJI (11) J X bJI 1 = : (12) N I J Inserting the TRAM definition for temperature transition counts I I X N I ef −u (x) bJI = (13) P N K ef K −uK (x) x2SJ K results in: I I X X ef −u (x) 1 = ; (14) P N K ef K −uK (x) J x2SJ K and thus: −uI (x) I X e e−f = (15) P N K ef K −uK (x) all x K I X e−u (x) f I = − ln (16) P N K ef K −uK (x) all x K which is exactly the MBAR estimator for the reduced free energy of thermodynamic state I (See Eq. (11) in [14]). 3 C. Temperature-state transitions and resulting state probability expectations I We want to derive an expression for the normalized stationary probabilities πi that results from xTRAM under the assumption of sampling from global equilibrium within each of the thermodynamic states I and compare this to the corresponding xTRAM ex- pectation. I In order to find the xTRAM estimations of πi , we write down the reversible transition matrix optimality conditions. For transitions between thermodynamic states we have, I I using that the absolute stationary probability vector is given by elements N πi =N: IJ JI I I IJ bi + bi N πi pi = I J (17) Ni Ni I I + J J N πi N πi bIJ + bJI N I πI pIJ = N I πI N J πJ i i i i i i J J I I I J (18) N πi Ni + N πi Ni bIJ + bJI pIJ = N J πJ i i : i i J J I I I J (19) N πi Ni + N πi Ni I I I Using global equilibrium and the statistical limit N ! 1 allows us to write: Ni = πi N , J J J IJ I IJ Ni = πi N , and bi = Ni pi . Inserting these equations yields: bIJ bIJ + bJI i = N J πJ i i I i J J I I I I J J (20) Ni N πi πi N + N πi πi N bIJ + bJI = i i I I (21) 2πi N bIJ + bJI bIJ = i i (22) i 2 JI IJ bi = bi : (23) Using the TRAM estimators for thermodynamic-state transition counts: J J X N J ef −u (x) ^bIJ = (24) i P N K ef K −uK (x) I K x2Si I I X N I ef −u (x) ^bJI = ; (25) i P N K ef K −uK (x) J K x2Si we have the equality: J J I I X N J ef −u (x) X N I ef −u (x) = : (26) P N K ef K −uK (x) P N K ef K −uK (x) I K J K x2Si x2Si 4 Summing over J, it follows that J J I I X P N J ef −u (x) X X N I ef −u (x) J = (27) P N K ef K −uK (x) P N K ef K −uK (x) I K J J K x2Si x2Si I I X X N I ef −u (x) N I = (28) i P N K ef K −uK (x) J J K x2Si I −uI (x) N I X X e i e−f = : P K f K −uK (x) (29) NI N e J J K x2Si I I I Using again Ni = πi N we rewrite this equation into: I P P e−u (x) J K K J x2Si P N K ef −u (x) πI = K ; (30) i e−f I using Eq. (15) this results in I P e−u (x) x2Si P N K efK −uK (x) πI = K (31) i P e−uI (x) x2Ω P K fK −uK (x) K N e which is exactly the MBAR expectation value (using Eqs. (14-15) in [14] with the indicator function of set Si as function A(x)). D. MBAR expectation values from TRAM I I I Given the local free energies fi = f −ln πi computed from xTRAM, we consider each combination of configuration state and thermodynamic state as a thermodynamic state for MBAR and use the MBAR equations to compute expectation values [14]. We define the weights: e−u(x) g(x) = K K ; (32) P P K fi −u (x) K i Ni e and then obtain expectation values of the function A(x) as: P x g(x) A(x) E[A] = P : (33) x g(x) K K K This choice can be motivated as follows: Using fi = f − ln πi we obtain e−u(x) g(x) = K K K (34) P P K f −ln πi −u (x) K i Ni e e−u(x) = K : (35) P P Ni f K −uK (x) K i K e πi 5 K K K We now choose Ni = πi N (statistical limit and global equilibrium) and obtain: e−u(x) g(x) = ; P K f K −uK (x) (36) n K N e where the factor n−1 cancels in Eq. (33). We thus get exactly the MBAR equation for an expectation value (compare to Eqs. (14-15) in [14]). II. ESTIMATION OF FREE ENERGIES A. Initial choice for the free energies f I We first seek a way to come up with an initial guess of the free energies f I for all thermodynamic states I. Here we follow the approach of Bennett [1]. We first order the different thermodynamic states simulated at in a sequence (1; :::; I; I + 1; :::; m), e.g. as- cending temperatures, and then construct a reversible Metropolis-Hastings Monte Carlo process between neighboring thermodynamic states. We define the Metropolis function M(x) = minf1; exp(−x)g and request the detailed balance equation to be fulfilled: I I+1 M uI+1(x) − uI (x) e−u (x) = M uI (x) − uI+1(x) e−u (x): (37) Integrating over the configuration space Ω and multiplying the left-hand side by ZI =ZI and the right-hand side by ZI+1=ZI+1 yields dx M uI+1(x) − uI (x) e−uI (x) dx M uI (x) − uI+1(x) e−uI+1(x) ZI = ZI+1 ; (38) ´ ZI ´ ZI+1 and thus we can derive an estimator for the ratio of neighboring partition functions I+1 I Z^I+1 M u (x) − u (x) = I ; (39) I I I+1 Z^ hM (u (x) − u (x))iI+1 I I where h·iI denotes the expectation value at thermodynamic state I. Using f = − ln Z , we get: I+1 I I+1 I M u (x) − u (x) I f = f − ln I I+1 : (40) hM (u (x) − u (x))iI+1 All f I can be shifted by an arbitrary additive constant.