Upload
buidieu
View
267
Download
2
Embed Size (px)
Citation preview
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation Summary
A λ-Dynamics Module for GROMACS
R. Thomas UllmannTheoretical and Computational BiophysicsMax Planck Institute for Biophysical ChemistryGöttingen, Germany
Gromacs WorkshopGöttingen May 19 2016
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation SummaryIntroduction to the GromEx projectExample: Protonation Triggers Opening of the Bacterial
Pentameric Ligand Gated Ion Channel (pLGIC) GLIC I
withCarstenKutzner,RudolfSchemm &Julian T.Brennecke
pLGICs mediate fast synaptic transmission in brain and muscle
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation SummaryIntroduction to the GromEx projectExample: Protonation Triggers Opening of the Bacterial
Pentameric Ligand Gated Ion Channel (pLGIC) GLIC II
110 Å
simulation system:
300,000 atoms
420 protonatable sites
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation SummaryIntroduction to the GromEx projectThe Two Major Challenges in the GromEx Project
Increase model realism by adding variableprotonationScaling to 10,000s – 1,000,000 coreså Ongoing work to develop a combinedλ-dynamics/fast multipole method forGROMACS
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation Summaryλ-Dynamics as Way to Model Chemically Variable SitesDynamical Protonation through λ-dynamics
protonatedform
deprotonated form
Smooth Interconversion Between Site Forms with λ-Dynamics• B. Tidor, 1993, J. Phys. Chem., 97, 1069–1073 • X. Kong & C. L. Brooks, 1996, J. Phys. Chem., 105, 2414–2423• S. Donnini, F. Tegeler, G. Groenhof & H. Grubmüller, 2011, J. Chem. Theory Comput., 7, 1962–1978• S. Donnini, R. T. Ullmann, G. Groenhof & H. Grubmüller, 2016, J. Chem. Theory Comput., 12, 1040–1051
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation Summaryλ-Dynamics as Way to Model Chemically Variable Sitesλ-dynamics on unit spheres, circles & hyper-spheres
φ1
φ2
λ2λ1
λ2
λ3
λ1
λ3
π+
π-
π2λ2
λ1λ2
λ1
φ1π
π23
0π
x2
x1
x3
λ2
λ1
λ3
2 forms on a circle 3 forms on a sphere• n forms on an n -dimensional hypersphere• n λ variables and n Cartesian coordinates xin∑ix2i =
n∑iλi = R 2 def
= L with λi def= x2i
• actual dynamics simulated in space ofm − 1independent, angular coordinates (φ1, ..., φn−1)
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation Summaryλ-Dynamics as Way to Model Chemically Variable SitesComputing Electrostatic Interactions With the
Fast Multipole Method (FMM) Enables Efficient Simulations on
Future Supercomputers with 10,000s to 1,000,000 CPU Cores
scalingPME FMM
atoms (N) O(n log n) O(N)
sites (s) O(s) O(1)
communicationnodes (p) O(p2) O(p log p)~~
~~
scaling bottlenecks
• C. Kutzner, R. Apostolov, B. Hess & H. Grubmüller, in Parallel Computing: Accelerating Computational Scienceand Engineering (CSE) , 722–730, IOS Press • C. Kutzner, S. Páll, M. Fechner, A. Esztermann, B. de Groot & H.Grubmüller, 2015, J. Comput. Chem., 36, 1990–2008 • I. Kabadshow & H. Dachsel 2012, IAS Series, 11, FZ Jülich
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation SummaryOverviewThe GromEx Project: Current State & Near Future
λ-dynamics
GROMACS
FMMenergies + forces
for λ
GPU-FMM
input format beyond
A/B topologies for λ sites
energies + forces
for λ
() CPU
reference implementation
to be discussed
enhance kernels for LJ + bonded
interactions for λ sites
Currently assembling the first workingversion:protonatable sites with two formsper site based on the current A & Btopologies, input files and freeenergy kernelscharge neutralization using aparsimonious proton bufferFMM with λ-support withnode-level parallelization
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation SummaryOverviewThe GromEx Project: Next Step
λ-dynamics
GROMACS
FMMenergies + forces
for λ
GPU-FMM
input format beyond
A/B topologies for λ sites
energies + forces
for λ
enhance kernels for LJ + bonded
interactions for λ sites
to be done for full gain from the newframework (constant-µ):add support for multiple localtopologies to the topology datastructures, .rtp, .top and .tng files(to be discussed)adapt bonded & Lennard-Jonesfree energy kernels to multiplelocal topologiesfull parallelization of the FMMopen questions to be discussed:
How to tell the FMM whichresources it is allowed to use?How to best utilize the currentdomain decomposition
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation SummaryOverviewThe GromEx Project: Next Step
GROMACS
FMMenergies + forces
for λ
GPU-FMM
input format beyond
A/B topologies for λ sites
energies + forces
for λ
enhance kernels for LJ + bonded
interactions for λ sites
λ-dynamics
to be done for full gain from the newframework (constant-µ):add support for multiple localtopologies to the topology datastructures, .rtp, .top and .tng files(to be discussed)adapt bonded & Lennard-Jonesfree energy kernels to multiplelocal topologiesfull parallelization of the FMMopen questions to be discussed:
How to tell the FMM whichresources it is allowed to use?How to best utilize the currentdomain decomposition
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation SummaryCommon PrinciplesInvariants for all λ-Dynamics variants
A site a can have n site forms with their respectivedimensionless weight factors λ = λ1, . . . , λnEach site obeys the constraint∑ni λi = L , where,canonically, L = 1.The actual dynamics takes place in a derived coordinatespace formed by the so-called simulation coordinates
s = s1, . . . , sm and the maximum λ value L .Coordinate transformations map between the coordinatespacesφ1
φ2
λ2λ1
λ2
λ3
λ1
λ3
π+
π-
π2λ2
λ1λ2
λ1
φ1π
π23
0π
x2
x1
x3
λ2
λ1
λ3
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation SummaryCommon PrinciplesThe Algorithm for a λ-Dynamics Time Step
Compute the potential energy E pot as function of thephysical coordinates and λ coordinates obtained in theprevious step.→ Fast Multipole Method, Electrostatics sessionFor each site a , Compute the vector of forces F i acting onthe λ particle along the simulation coordinates
F a =
[∂E pot
∂λa
]T [∂λa∂s a
]+ β−1
∂ ln√∣∣λgij ∣∣∂s a
Propagate the positions and velocities in simulationcoordinate spaceTransform the updated simulation coordinates s back tothe λ coordinate space
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation SummaryCommon PrinciplesThe Choice of Simulation Coordinates Distinguishes λ-dynamics variants
Simulation coordinates can be chosen with the following aimsimplicitly satisfy the constraint∑n
i λi = Lenhance sampling efficiency by facilitatinginterconversion between site forms
Fully implemented variants:λ-dynamics on hyperspheres
circles (2-sphere) for two forms, spheres (3-sphere) forthree forms, n -sphere for n formsm = n − 1 simulation coordinates
Nexp λ-dynamics (J. Knight & C. Brooks, 2011,J. Comput. Chem., 32, 3423–3432)no simple geometric interpretationm = n simulation coordinates
It’s easy to implement more variants.
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation SummaryThe Bias PotentialA Bias Potential Designed to Ensure Sampling of Physical States
and to Optimize Sampling Efficiency
Ωi =
( ni∑k
(λi ,kL
)2)−1
E bias = E bias, global +Nsites∑iE bias, sitei
E bias, sitei = K site
(Ωi − 1)2s
E bias, global = K global
Nsites∏i
Ωi − 12t
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation SummaryThe Bias PotentialThe Effect of Barrier Height And Shape on the Transition Rate as
Predicted by Eyring Theory for a Site with Two Forms
0
1
0.0 0.5 1.0
λ
S = 0.05
0
1
barr
ier
pote
ntial/barr
ier
heig
ht
S = 0.35harmonic
0
1 S = 4.0
harm
onic
S = 0.00
S = 0.01S = 0.02
S = 0.05
S =
0.1S
= 0
.35
S =
4.0
0
20
40
60
80
100
120
140tr
ansitio
n r
ate
[ns
−1]
0 1 2 3 4 5 6
barrier height [kcal/mol]
0102030405060708090
100110120130140
Eyring tra
nsitio
n r
ate
[ns
−1]
140140140 100100100
100
10
01
00
10
0
808080
80
80
80
80
606060
60
60
60
60
4040
40
40
40
40
2020
20
20
20
20
1010
10
10
10
10
55
55
55
2.52.5
2.5
2.5
2.5
2.5
1
1
11
11
0.5
0.5
0.5
0.5
0.5
0.25
0.25
0.2
50.2
50.2
5
0.1
0.1
0.1
0.1
0.1
0.025
0.0
25
0.0
25
0.0
25
0.0
25
0.01
0.0
1
0.0
10.0
10
.01
0.0025
0.0
025
0.0
025
0.0
025
0.001
0.0
01
0.0
01
0.0
01
0.0
0025
0.0
0025
0.0
0025
0.0
00
25
0.0
001
0.0
001
0.0
001
0.0
00
1
0.0
00025
0.0
00025
0.0
00025
0.0
00025
1e−
05
1e−
05
1e−
05
0.0
000025
0.0
000025
0.0
000025
1e−
06
0.0
0000025
1e−
08
0.0
00000025
1e−
09
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
shape p
ara
mete
r S
0 2 4 6 8 10 12
barrier height [kcal/mol]
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation SummaryThe Bias PotentialThe Effect of Barrier Height And Shape on Physical Purity and
Simulation Stability for a Site with Two Forms
1.0
1.1
1.2
1.3
1.4
1.5
effectively
popula
ted s
tate
s
1.551.551.55
1.5
5
1.5
51.5
51.5
5
1.51.51.5
1.5
1.5
1.5
1.5
1.451.451.45
1.4
51.4
51.4
51.4
5
1.41.41.4
1.4
1.4
1.4
1.4
1.351.35
1.35
1.3
5
1.3
5
1.3
5
1.35
1.31.3
1.3
1.3
1.3
1.3
1.3
1.251.25
1.2
5
1.25
1.25
1.25
1.21.2
1.2
1.2
1.2
1.2
1.151.15
1.15
1.15
1.15
1.11.1
1.1
1.1
1.081.08
1.081.08
1.06
1.06
1.06
1.04
1.041.04
1.02
1.02
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
shape p
ara
mete
r S
0 2 4 6 8 10 12
barrier height [kcal/mol]
10−2
10−1
100
101
102
103
104
105
106
107
maxim
um
forc
e [kJ
−1]
111
11
1
101010
10
10
10
100100100
100
100
100
100100100
100
100010001000
100010001000
100001000010000
100001000010000
100000100000100000
100000 100000 100000 100000
1000000100000010000001000000 1000000 1000000
10000000202020
20
20
20
20
303030
30
30
30
30
404040
40
40
40
40
505050
50
50
50
5050
606060
60
60
60
6060
707070
70
70
70
7070
808080
80
80
80
808080
909090
90
90
90
909090
110110110
110
110
110
110110110
110
120120120
120
120
120
120
120120120
130130130
130130130
130
130
130
130
140140140
140140140
140
140
140
140
150150150
150150150
150
150
150
150
160160160
160160160
160
160
160
170170170
170170170
170
170
170
180180180
180180180
180
180
180
190190190
190190190
190
190
190
200200200
200200200
200
200
200
210210210
210210210
210
210
220220220
220220220
220
220
230230230
230230230
230
230
240240240
240240240
240
240
250250250
250250250
250
250
260260260
260260260
260
270270270
270270270
270
280280280
280280280
280
290290290
290290290
290
300300300
300300300
300
310310310
310310310
320320320
320320320
330330330
330330330
340340340
340340340
350350350
350350350
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
shape p
ara
mete
r S
0 2 4 6 8 10 12
barrier height [kcal/mol]
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation SummaryImplementationThe Most Important Objects of the λ-Site Module
LambdaSiteModuleLambdaSiteData
global arraysLambdaSite
LambdaSiteInternalsImplHypersphereImplNexpLambdaSiteSimFunc
LambdaSiteBiasPotentialImplHarmonicImplOmega
outside code: FMM, ...coordinate transformations etc.
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation SummaryImplementationImplementation Details
class templates for exchangeability of data types (e.g.,float or double) and allocatorspolymorphic objectsabstract parent classes define pure virtual functions toensure that all daughter objects provide the samefunctionality and function signaturesanalytical functions for all quantities (potentials,coordinates, metric tensor determinants and their firstand second partial derivatives)all math functions are tested against numerical referencevaluesunit testing is automated for all implementations and allfloating point data types – adding a new implementationdoes not require writing new testsgeneral, allocator-aware array classes for data exchange,e.g., with the FMM and for computationSIMD support started, complete: vector-matrix products
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation Summary
Outlook: Full Chemical Variability in Molecular Simulations
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation Summary
Summary – Constant-pH & Beyond
First working version close to release (constant-pH).
The developed general λ-dynamics/FMM method will allowfor:modeling (bio)molecules in greater physical detail byenabling chemical variabilityimplementing physically correct electrostatic long-rangeinteractions also in presence of chemically variable sitesλ-dynamics simulations at small, constant overheadrelative to standard MDO (#sites) → ≈ O (1) (< 20% overhead)
Introduction The GromEx Project The λ-Dynamics Module: Physics & Implementation Summary
Acknowledgements
Bartosz Kohnke, Plamen Dobrev,Carsten Kutzner & Helmut GrubmüllerMPI for Biophysical Chemistry, Göttingen, GermanyIvo Kabadshow, Andreas Beckmann,David Haensel & Holger DachselJülich Supercomputing Centre, GermanySerena Donnini & Gerrit GroenhofUniversity of Jyväskylä, FinlandBerk Hess and other GROMACS developersKTH Royal Institute of Techology, Stockholm, Sweden
Discussion
Discussion
Discussion
DiscussionFMMA Fast Multipole Method for the Exascale
(Current Implementation, ongoing progress)
Scalability to largecore-counts – avoidcommunication
Optimal utilization ofHeterogeneous Hardware(CPUs + GPUs)
para
llel e
ffici
ency
21 24 27 210 213 2160%
50%
100%
number of MPI ranks
replication factor
1 2 4 816
64
128
32256
0.11
10100
1000
0 5 10 15 20 25
seco
nds
multipole order
CPU baseline
GPU
Discussionλ-Dynamics on Hyperspheresλ-dynamics on unit spheres, circles & hyper-spheres
φ1
φ2
λ2λ1
λ2
λ3
λ1
λ3
π+
π-
π2λ2
λ1λ2
λ1
φ1π
π23
0π
x2
x1
x3
λ2
λ1
λ3
2 forms on a circle 3 forms on a sphere• n forms on an n -dimensional hypersphere• n λ variables and n Cartesian coordinates xin∑ix2i =
n∑iλi = R 2 def
= L with λi def= x2i
• actual dynamics simulated in space ofm − 1independent, angular coordinates (φ1, ..., φn−1)
Discussionλ-Dynamics on HyperspheresThe Potential Energy Landscape on a Unit Sphere
E pot(λ) = E site + E bias
• E bias = K 22s (Ω− 1)2 s withbarrier steepness s and barrier height between two forms K• Ω = 1/∑n
i λ2i is the effective number of forms contributing to thecurrent state→ discourage population of unphysical states andprefer interconversion between pairs of forms
DiscussionNexp λ-DynamicsNexp λ-Dynamics
λi = L exp [C sin θi ]∑j exp [C sin θj ]
m = n angular simulation coordinates θ are mapped to n λcoordinates, where C is the steepness constant.The opposite mapping is, as in the hypersphere variant, not unique:reset all λ values to the realizable range of valuesλmin ≤ λi ≤ λmax, record the index j of the maximum λ value inthe course.
λmin =exp [−C ]
(N − 1) exp [+C ] + exp [−C ]
λmax =exp [+C ]
(N − 1) exp [−C ] + exp [+C ]
assign all λi according toθi = asin
[C−1 ln
[λiλj
]+ sin θj
]
DiscussionThe Extended Phase SpaceThe Extended Hamiltonian
two global topologies– conceivable as asingle site with twoforms
Nsites sites withmultiple localtopologies
H = λ1E pot1 + λ2E pot2 + E bias
+
realparticles∑r
[ p2r2mr]
+pλ22mλ
H =
Nsites∑a
na∑i
[λa ,i E site
a ,i
]
+
Nsites∑b
nb∑j
Nsites∑c 6=a
nc∑k
[λb ,jλc ,kWb ,j ,c ,k
]+ E bias
+
realparticles∑r
[ p2r2mr]
+
Nsites∑d
md∑l
[p sd ,l2
2mλd ,l
]
DiscussionThe Extended Phase SpaceThe Extended Phase Space: Momentum Integrals Cancel from
Free Energy Differences in Classical Statistical Thermodynamics
z = q, q, λ, s z′ = q, q, λ, λ
Zi =
∫Γi
exp [−βH (z)]dz
Zi =
∫Γi
(E pot (q, λ)dqdλ
) +∞∫−∞
(E kin,q (q)dq
) +∞∫−∞
(E kin,s (s)ds
)
∆FA→B = −β−1 ln[ZBZA
]