Virtual Tutorial Starts at 15:00 GMT
Total Page:16
File Type:pdf, Size:1020Kb
Welcome! Virtual tutorial starts at 15:00 GMT Please leave feedback afterwards at: www.archer.ac.uk/training/feedback/online-course-feedback.php Multi-resolution modelling of biological systems in LAMMPS ARCHER Virtual Tutorial, 19th Oct 2016 Iain Bethune [email protected] Oliver Henrich [email protected] Reusing this material This work is licensed under a Creative Commons Attribution- NonCommercial-ShareAlike 4.0 International License. http://creativecommons.org/licenses/by-nc-sa/4.0/deed.en_US This means you are free to copy and redistribute the material and adapt and build on the material under the following terms: You must give appropriate credit, provide a link to the license and indicate if changes were made. If you adapt or build on the material you must distribute your work under the same license as the original. Note that this presentation contains images owned by others. Please seek their permission before reusing these images. Outline • ARCHER eCSE programme • Implementation of Dual Resolution Simulation Methodology in LAMMPS • ELBA force-field • Implementation in LAMMPS • Performance testing • Summary ARCHER eCSE programme • Funding for the ARCHER user community to develop software • Implementation of algorithmic improvements within an existing code • Improving the scalability of software on higher core counts • Improvements to code which allows new science to be carried out • Porting and optimising a code to run efficiently on ARCHER • Adding new functionalities to existing codes • Code development to take a code from a Tier-2 (Regional) or local university cluster to Tier-1 (National) level bringing New Communities onto ARCHER • Projects typically 3 months – 1 year • Next call closes 31st Jan 2017 ARCHER eCSE programme • More information on the ARCHER website: • https://www.archer.ac.uk/community/eCSE/ • Project Reports • How To Apply • List of funded projects • Webinar from last month: • https://youtu.be/WRGsNKWrNIc Implementation of Dual Resolution Simulation Methodology in LAMMPS • eCSE04-7 (January 2015) • PI: Prof. Jonathan Essex, Southampton • 6 person-months funded: August 2015 – August 2016 • Objective: enable fast and reliable calculations with the ELBA force-field in LAMMPS • New integrators • Parallel load balancing ELBA Force-field • ELBA = ELectrostatics-BAsed coarse grained forcefield • Orsi & Essex, PLoS ONE 6(12) 2011 • Originally for studying lipids • Also applied to other biomolecules • Explicit solvent • One dipolar bead per water molecule • Allows for atomistic detail e.g. • Using CHARMM parameters 138 atoms -> 15 CG beads ELBA Force-field • Implemented in BRAHMS-MD (Biomembrane Reduced- ApproacH Multiresolution Simulator for Molecular Dynamics): • https://code.google.com/archive/p/brahms-md/ • Limited user base, single developer -> not sustainable • No parallelisation -> small systems • Why LAMMPS? • Main interaction types already implemented http://lammps.sandia.gov • Support for spherical particles • r-RESPA multiple timestepping • Flexible, scalable, large user base Implementation of Dual Resolution Simulation Methodology in LAMMPS 5 Then the rotation matrix Q is given by: 1 c Q = I + [v] + [v]2 − ⇥ ⇥ s2 Care has to be taken for the case were the dipole is parallel (or anti-parallel) to the space-frame z-axis, in which case Q = I (or I, respectively). In the DLM algorithm − the orientation is integrated by a sequence of rotations around each axis in the body- frame, while LAMMPS stores the angular velocity in the space-frame. Thus compared to the steps given in the DLM paper, our implementation has an additional transformation step to compute the angular velocity in the body-frame, and due to the definition of the orientation matrix, the orientation updates take the form of a right-multiplication by the transpose of the rotation matrix. The rotation matrices for a rotation around each axis are defined as (x rotation shown, the other two axes similarly): 10 0 2 3 6 7 ImplementationRx(φ) = 60 cos φ sin φ7 in LAMMPS 6 − 7 6 7 6 7 60 sin φ cos φ 7 6 7 • LAMMPS fix 6nve/sphere7 integrator does not conserve energy well and implemented (in the MathExtra46 namespace), using57 the more efficient small angle approximations:• Better scheme to integrate rotational d.o.f. - DLM Dullweber, Leimkuhler and McLachlan, JCP 107(15) 1997 φ 1 φ2/4 sin φ = , cos φ = − 1. Construct 1rotation+ φ2/4 matrix1 + φ2/ 4Q from dipole (taken as the body-fixed z-axis) The full rotational update is implemented as follows: 2. In body-space, apply rotations6 Implementation around of D ualeachResolution localSimulation axis:Methodology in LAMMPS 6 Implementation of Dual Resolution Simulation Methodology in LAMMPS !b = Q!s δt T R4 = Ry( !2), ! = R4!, Q = R4 Q δt T 2 R1 = Rx( !1), ! = R1!, Q = R1 Q δt T 2 R4 = Ry( !2), δ ! = R4!, Q = R4 Q 2 t T R5 = Rx( !1), ! = R5!, Q = R5 Q δt T 2 R2 = Ry( !2), ! = R2!, Q = R2 Q δt T 2 R5 = Rx( !1), ! = R5!, Q = R Q Finally, the space-frame2 dipole is computed as: 5 = (δt! ), ! = !, = T R3 Rz 3 R3 Finally,Q R the3 Q space-frame dipole is computed as: µ = QT [001] µ s · || || µ = QT [001] µ 3. Finally, compute the newTo dipole: correctness of the implementations was· || || tested on two systems. Firstly, a 49.34 Å3 3 Tocubic correctness box containing of the implementation 4000 ELBA water was testedbeads was on two equilibriated systems. Firstly,at 300K a using 49.34 a Å Langevin cubic boxthermostat containing for 104000 ps, ELBA then the water thermostat beads was was equilibriated removed and at 10ps 300K of using MD in a Langevinthe microcanon- thermostatical (NVE) for 10 ps, ensemble then the was thermostat performed. was removedThe timestep and 10ps for both of MD the in entire the microcanon- run was 10fs using ical (NVE)fix ensemble nve/sphere was performed. update dipole The timestep. Two runs, for both one with the entire the native run was ‘LAMMPS’ 10fs using integra- fix nve/spheretor and one with update our new dipole ‘DLM’. Two integrator runs, one are with shown the native in Figure ‘LAMMPS’2. While integra- the thermostat tor andis one enabled with our equipartition new ‘DLM’ of theintegrator kinetic are energy shown is maintained in Figure 2. in While both cases.the thermostat Once the ther- is enabledmostat equipartition is removed, of the the ‘leaking’ kinetic energy of energy is maintained from the inrotational both cases. modes Once is clear, the ther- and good mostatenergy is removed, conservation the ‘leaking’ is only of obtained energy from by using the rotationalthe DLM integrator. modes is clear, and good energy conservation is only obtained by using the DLM integrator. Figure 2: Rotational and translational components of kinetic energy for a box of 4000 FigureELBA 2: Rotational water beads and translationalduring 2000 steps components of NVT of and kinetic 2000 energy steps of for NVE a box molecular of 4000 dynam- ELBAics. water beads during 2000 steps of NVT and 2000 steps of NVE molecular dynam- ics. A similar experiment was carried out with a simulation of a lipid membrane consisting A similar experiment was carried out with a simulation of a lipid membrane consisting Implementation in LAMMPS • 4000 ELBA water beads, 10fs timestep, 20ps NVT, 20ps NVE fix thermostat all langevin 303 303 200 48279 omega yes Implementation in LAMMPS • 128 DPMC molecules in water, 75ps NVT, 100ps NVE Image from Sam Genheden Implementation in LAMMPS • DLM integrator enabled by an optional argument: • fix nve/sphere … update dipole/dlm • Also for other ensembles: • Constant temperature / NVT (Nosé-Hoover) • fix nvt/sphere … update dipole/dlm • Isothermal-isobaric (Nosé-Hoover / Parrinello-Rahman) • fix npt/sphere … update dipole/dlm • Isenthalpic (Parrinello-Rahman) • fix nph/sphere … update dipole/dlm Implementation in LAMMPS http://lammps.sandia.gov/doc/ • Load balancing schemes: balance.html None Shift RCB • Problem for dual-resolution simulations! • 90% of computational cost is force evaluation • Not all particles are the same • r-RESPA – some forces are computed more frequently than others Implementation in LAMMPS • New load balancing metrics: • Weighting by particle groups • Uses LAMMPS existing group command e.g. group solute type > 1 group water type 1 Iteration Threshold to limit stop balancing balance 1.1 shift xyz 50 1.1 weight group 1 solute 2.5 Threshold to start Balance all 3 Solute group has load balancing dimensions weight factor 2.5 Load balancing Weight particles algorithm by group (1 group) Implementation in LAMMPS • New load balancing metrics (subsequently added by LAMMPS developers): • Weighting by number of neighbors – weight neigh • Weighting by compute time – weight time • Doesn’t account for the different particles types contributing to different parts of the computation (pair, bond, kspace, neigh) • Weighting by arbitrary user-defined variables – weight var Performance testing • Bovine Pancreatic Trypsin Inhibitor (BPTI) dual-resolution model: • 882 atoms, CHARMM force-field • 6136 water molecules, ELBA beads • No r-RESPA • 1fs timestep • Up to 10% speedup over non-weighted balance Performance testing • 1:4 r-RESPA ratio • Water pair forces + dihedral forces computed every fourth step • Larger weightings better (2.5-3.0) • Up to 36% speedup over non- weighted balance Performance testing • 1:8 r-RESPA ratio • Larger weightings better (3.0-4.0) • Up to 65% speedup over non- weighted balance Summary • DLM integrator for NVE/NVT/NPT/NPH dynamics • Stable for water up to 16fs timestep Installed on ARCHER • Included in LAMMPS stable release 30 Jul 2016