Request for Proposals for Biomolecular Simulation Time on Anton

Introduction

The Pittsburgh Supercomputing Center (PSC, www.psc.edu) is soliciting proposals for computer time on Anton 2, a special-purpose supercomputer for molecular dynamics (MD) simulation designed by D. E. Shaw Research (DESRES). Since late 2016, a 128-node Anton 2 machine has been made available without cost by DESRES for non-commercial research use by universities and other not-for-profit institutions. The machine is hosted by PSC. In the remainder of this RFP, Anton refers specifically to this Anton 2 machine.

To qualify for an allocation on Anton, the principal investigator (PI) must be a faculty or staff member at a U.S. academic or non-profit research institution and must have the appropriate level of authority and responsibility to direct the project supported by the allocation. International collaborations are allowed, and co-PIs do not need to be associated with U.S. academic or non-profit research institutions. Graduate students and postdoctoral researchers may not be PIs, but qualified advisors may apply for allocations on their behalf. Researchers with no previous experience with Anton are encouraged to apply.

Each investigator can serve as a PI for only a single application for computer time on Anton in a given allocation round. If there is a co-PI on the proposal, the PI must clearly demonstrate that the work described in the proposal will be conducted primarily in the PI’s lab and will be directly supervised by the PI. In addition, if a PI or co-PI is on another Anton proposal (as co-PI or PI, respectively), investigators involved in both proposals must also clearly demonstrate that the research efforts described in the two proposals are distinct and independent of each other, and thus do not constitute an effective doubling of the allowed allocation for a given research effort.

PIs who previously had an allocation on Anton 1 or the current Anton system at PSC are required to provide a progress report of their past work on the Anton machine at PSC. Repeat applications by a given PI that are unaccompanied by a progress report will not be considered. Further, applications from a PI with a previous allocation should not be identical in text to the previous application.

Trajectories generated on Anton and all their associated data will be kept private until publication, or up to one year after the end of an Anton allocation, whichever comes first. After this embargo period, the trajectories and data will become part of a unique collection of hundreds of long-timescale trajectories that will be made widely available to the research and education communities. To this end, investigators must agree to share their trajectories and data no later than one year after the end of the allocation period in which the data were generated. Investigators are responsible for ensuring that they comply with the data sharing policies of the agencies funding their research.

The remainder of this Request for Proposals (RFP) describes the intended focus of solicited projects, outlines simulation requirements, offers proposal preparation instructions, and explains the proposal review process.

Intended Focus of Proposed Projects

Anton enables scientists to perform MD simulations of biomolecular systems nearly two orders of magnitude faster than the previous state of the art (see the Estimating Requested Simulation Resources section). To maximize the benefit of Anton to the scientific community, proposed projects should focus exclusively on questions that will be greatly advanced by the unique capabilities of Anton. In their description of the scientific objectives, investigators must clearly explain why achieving their objectives requires access to Anton, and could not be efficiently achieved on other high performance computing systems available at their institutions or through national supercomputing resources made available by the NSF, DOE, or other programs. Generally, this justification will include the need for longer MD trajectories than are feasible on conventional systems. Proposals that do not clearly justify the need for Anton over other available computing resources will receive a lower ranking in the review process.

Simulation Requirements

Anton uses specialized hardware to perform molecular dynamics simulations orders of magnitude faster than general-purpose hardware running traditional MD software (see Reference [1] below). Importantly, Anton does not run Desmond, AMBER, NAMD, GROMACS, or any other MD simulation software package, although it uses a Desmond structure (DMS) file as an initial input and its trajectory output is compatible with Desmond’s (DTR files). Each simulation must be built, using tools available at PSC, specifically for Anton.

Anton is designed primarily to accelerate classical MD simulations of biomolecular systems with periodic boundary conditions and explicit solvent. To make best use of the available computational resources, consideration will be given only to applicants whose projects satisfy the criteria outlined below. Investigators who have questions regarding the suitability of their proposed simulations are encouraged to contact grants@psc.edu to discuss their planned project before submitting their proposal.

1. Simulations must be standard MD runs in the constant NVE, constant NVT (isothermal), or constant NPT (isothermal, isobaric) ensembles.  Constant NVT and NPT simulations must use the Multigrator framework described in Reference [2] with either Nose-Hoover or Langevin thermostats and isotropic or semi-isotropic MTK barostats. Simulation conditions may include the specification of a uniform constant applied electric field. Position restraints, on a per atom basis, are allowed. Enhanced sampling is also available in four forms as follows: (i) simulated tempering, including adaptive weighting, (ii) application of restraints between the centers of mass of groups of atoms, (iii) application of conformational restraints, each based on the calculation of RMSD (root mean squared deviation) with respect to atomic positions of a given reference structure, and (iv) tempered binding, as used in References [3] and [4], although note that it is incompatible with the NBFIX terms used in CHARMM.  For restraints in both (ii) and (iii), equilibria and spring constants can be varied during a simulation according to a schedule or adaptively to implement a form of umbrella sampling. Applicants with systems that have dozens of restraints and/or restraints involving thousands of atoms should contact grants@psc.edu before submitting a proposal.
2. The simulation cell must have only right angles (i.e., it must be a cubic or orthorhombic box), and must have a minimum of 45 Angstroms on each side. Applicants with systems shaped such that one dimension of the simulation cell is much larger than the others should contact grants@psc.edu before submitting a proposal.
3. Proposed simulations must use recent variants of the following standard, nonpolarizable biomolecular force fields: CHARMM (e.g., CHARMM22, CHARMM27 – including CMAP corrections, and CHARMM36), AMBER (e.g., AMBER99, AMBER99SB, AMBER03, and AMBER19SB). Modified versions of the CHARMM and AMBER force fields, based on published research by DESRES, are also acceptable (and available through the simulation setup tools). Water should be modeled with the SPC, TIP3P, or TIP4P models, or their variants. For more information on force force fields available on Anton, see here.
4. Chemical systems proposed for simulation must contain between 25,000 and 700,000 atoms (including solvent atoms), though systems between 50,000 and 600,000 atoms are recommended for maximum efficiency. Chemical systems proposed for simulation must consist of some combination of protein, DNA, RNA, lipids, water, and standard ions. Investigators who wish to use custom parameters or molecules that are not included in the standard distribution of the supported force fields (see 3 above) should contact grants@psc.edu to discuss the suitability of their simulations before submitting their proposal.
5. Anton is capable of producing long, continuous MD trajectories. To maximize the benefit of this resource to the research community, researchers should not propose to run simulation trajectories that would finish in less than one hour on Anton (e.g., less than ~2,500 ns for a 25,000 atom system, and less than ~150 ns for a 700,000 atom system; see Estimating Requested Simulation Resources).

Biomolecular systems should be well equilibrated prior to running on Anton. PSC will provide instructions to convert files to Anton format for equilibrated systems generated with the following programs: Desmond, Amber, CHARMM, Gromacs, or NAMD. However, for the easiest conversion, we strongly suggest using Desmond to equilibrate, especially if your system has custom molecules. Researchers can request allocations of compute time on the Bridges-2 system at PSC if they would like to use this resource for preparation and equilibration of systems they propose to run on Anton (https://www.psc.edu/resources/bridges-2/).

Estimating Requested Simulation Resources

Applicants may refer to the table below, with benchmarks for a number of systems of various sizes, to estimate the amount of machine time required for their project. The actual achievable simulation times, however, may vary even for different molecular systems of similar size. No more than a total of 460,000 MD simulation units (9.1 machine-days) will be allocated to any one principal investigator.  (An “MD simulation unit” is defined as the amount of machine-time required to simulate 1 nanosecond of a 50,000 atom system using Anton production parameters).  It is anticipated that 10 to 20 allocations will be made at or near 460,000 MD simulation units, and 30 to 40 allocations will be made at or near 230,000 MD simulation units.  Applicants are encouraged, but not required, to target their requested resources at one of these levels, and they must provide justification for the number of requested MD simulation units (see #6 in Structure of Proposal). Note that simulations on Anton 2 will run on all 128 nodes. In calculating their requests from the benchmark data below, it may help investigators to note that there are 50,550 MD simulation units per machine-day (460,000 MD simulation units per 9.1 machine-days). So, for example, simulating the 116,650 atom T7Lig system for 100 microseconds would require (100 microseconds / 21.9 microseconds per machine-day) * 50,550 MD simulation units per machine-day = 230,822 MD simulation units.

For more details regarding Anton 2, please see the References.

Structure of Proposal

Proposals should be two to six pages in length, not including references. PIs who previously received an allocation on Anton at PSC must also provide a separate progress report (2 pages maximum) demonstrating their successful use of Anton to produce high-impact scientific results. This report will be an important factor in assessing the project feasibility for proposals from returning investigators (see Project Feasibility and Team Qualifications and Progress Report sections below). In addition, applicants may submit up to two additional supporting documents (e.g., published papers) in PDF format on the submission page; however, all information required to assess the merits of the proposal must be contained within the text of the proposal and (for returning investigators) progress report. Reviewers will not be required to refer to supporting documents or any other external documents when reviewing proposals.

The main proposal document must have all the sections listed below. Proposals that do not clearly address all the points listed in each section will receive a lower ranking:

1. Summary of the project, including descriptive title of proposed research (400 words maximum for summary & title).
2. Name, address, email, and telephone number of the Principal Investigator and co-PIs. If a PI or co-PI is on another Anton proposal (as co-PI or PI, respectively), describe here how the two proposals are distinct and independent of each other, and thus do not constitute an effective doubling of the allowed allocation for a given research effort. Failure to provide such an explanation may result in a lower ranking.
3. Background information (1 page maximum): Investigators should include sufficient background information on the research field to allow reviewers to judge the scientific merit of the proposed research.
4. Scientific Objectives to be accomplished on Anton (2 pages maximum): Investigators should clearly explain, in detail, why Anton is necessary for the planned project and why the project could not be efficiently completed on conventional systems (see the Intended Focus of Proposed Projects section for more details). Applicants should also clearly explain the scientific impact of their proposed project.
5. Project Feasibility and Team Qualifications (2 pages maximum): Applicants must clearly and explicitly address all five points outlined in the Simulation Requirements section of this RFP, providing all necessary details regarding their proposed simulations to ensure technical feasibility. Please state clearly whether the proposed system has already been built and equilibrated. If the system has not yet been run in production, please provide evidence that the proposed system can be successfully built and simulated. In addition, applicants must describe the expertise and experience of their research team members (including prior experience running MD simulations) to ensure that the proposed simulations can be successfully completed on Anton. If there is a co-PI on a proposal, provide information here demonstrating that the work described in the proposal will be conducted primarily in the PI’s lab and will be directly supervised by the PI. It is particularly important for applicants who had previous Anton allocations to report successful outcomes from those awards, but these outcomes should be described in a separate progress report (see Progress Report below). 
6. Requested Resources (1 page maximum): Investigators must clearly state and provide justification for the number of Anton MD simulation units requested for their project subject to the limits given in Estimating Requested Simulation Resources. The justification should provide strong scientific arguments as to why the length and number of proposed simulation runs will be both sufficient and necessary to achieve the stated scientific objectives. Please refer to the Estimating Requested Simulation Resources section for benchmarks to facilitate estimation of requested resources. No more than a total of 460,000 MD simulation units (9.1 machine-days) can be requested per PI.

Progress Report (separate document required for groups with previous allocations only – 2 pages maximum): The Progress Report should describe how the PI’s current or prior allocation was used and summarize the findings or results. These results can be discussed in the context of publications, presentations, and other communications resulting from this work, but all information required to assess progress should be clearly communicated in the text of the progress report itself and should not depend on reviewers reading supporting documents or any other external documents. The Progress Report should also explain any significant digressions from the originally proposed resource usage plan for the prior period. Repeat applications by a given PI that are unaccompanied by a progress report will not be considered.

If a PI’s allocation was significantly underutilized, the Progress Report should briefly describe the reasons for the underutilization and any mitigation by the PI to avoid the situation with the current allocation. PIs that demonstrate a pattern of low utilization across multiple allocation periods, without strong justification for this underutilization, will receive a lower ranking.

Important Dates and Proposal Review Process

The electronic version of this RFP can be found at https://www.psc.edu/anton-rfp. Proposals should be submitted electronically in PDF format at https://allocations.psc.edu. You will need to have a PSC account to access the submission portal. Go to https://allocations.psc.edu and create a PSC account if you don’t already have one. Please do this at least 2 days prior to the submission deadline so you can get access to the submission system in time to submit your proposal.

The deadline for applications is 11:59 pm, EDT, Thursday, June 29, 2023.

Proposals will initially be assessed for technical feasibility by PSC staff (see the Simulation Requirements section) and will then be reviewed by a peer committee to be convened by the National Academies of Sciences, Engineering, and Medicine. Resource allocations by PSC will follow the recommendations of the National Academies of Sciences, Engineering, and Medicine. Resources are expected to be made available starting in December of 2023.

Proposals will be ranked based on the scientific merit of the proposed research, the strength of the justification for the requested resources, and evidence of the ability of the research team to successfully execute the proposed studies. In addition, proposals by PIs that previously received an allocation on Anton 1 or the current Anton at PSC will need to demonstrate that the scientific outcomes from the previous award(s) justify another award. At least 25% of the total allocated time will be reserved for PIs that have not previously had an allocation on an Anton system at PSC. Since only a limited number of allocations will be available on Anton, a successful proposal will clearly state how access to Anton will facilitate breakthrough science.

Anton 2 Proposal Webinar: To help with the proposal submission, we will offer two webinars on May 25, 2023:

• 2:30-3:30 PM (EDT) How to Write a Successful Anton 2 Proposal (Register here)
• 3:30-5:00 PM (EDT) Anton 2 Enhanced Sampling Techniques (Register here)

For general questions regarding this RFP, or to discuss feasibility and technical aspects of projects, please contact grants@psc.edu.

References

[1] Anton 2: Raising the Bar for Performance and Programmability in a Special-Purpose Molecular Dynamics Supercomputer, D. E. Shaw et al., Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC14), (2014).

[2] Accurate and Efficient Integration for Molecular Dynamics Simulations at Constant Temperature and Pressure, R. A. Lippert et al., Journal of Chemical Physics, vol. 139, no. 16, 2013, pp. 164106:1–11.

[3] Pan, Albert C., Daniel Jacobson, Konstantin Yatsenko, Duluxan Sritharan, Thomas M. Weinreich, and David E. Shaw. 2019. “Atomic-Level Characterization of Protein–Protein Association.” Proceedings of the National Academy of Sciences 116 (10): 4244–49. https://doi.org/10.1073/pnas.1815431116.

[4] Wang, Qi, Yakov Pechersky, Shiori Sagawa, Albert C. Pan, and David E. Shaw. 2019. “Structural Mechanism for Bruton’s Tyrosine Kinase Activation at the Cell Membrane.” Proceedings of the National Academy of Sciences 116 (19): 9390–99. https://doi.org/10.1073/pnas.1819301116.

[5] Desmond/GPU Performance as of April 2021, M. Bergdorf et al., D. E. Shaw Research Technical Report DESRES/TR—2021-01, 2021 (available at http://www.deshawresearch.com/publications.html)

[6] PSC Anton 2 benchmark systems used in this RFP can be downloaded at http://www.deshawresearch.com/downloads/download_trajectory_pscbenchmark.cgi

Anton RFP 2023 (released May 1, 2023)