A System for Load Balancing of Local Minimization and Energy Calculations in Crystal Structure Prediction Simulations

A. M. LUND; G. I. PAGOLA; A. M. ORENDT; M. B. FERRARO; J. C. FACELLI

St Louis

Conferencia; Extreme Science and Engineering Discovery Environment 2015 (XSEDE15); 2015

Crystal structure prediction (CSP) is an area of increasing scientific importance [1,2]. The development of effective computational methods for CSP would potentially lead to an increase in the understanding of crystal growth and structural analyses. In a more applied sense, CSP has importance in the pharmaceutical industry for detecting crystal polymorphs and formulating co-crystals to improve bioavailability of drugs. Over the past ten years great strides have been made showing that crystal structure prediction is viable and practical. Especially promising are recent advances using dispersion corrected density functional theory (DFT-D) in obtaining accurate predictions [3]. At the heart of CSP methods there are two fundamental tasks. The first one is the energetic ranking of structures In most cases a molecule will crystallize in an energetically stable form, consequently, the accurate assessment of lattice energies is essential in crystal structure prediction. Of particular importance is the balance between intra- and inter-molecular forces; correctly representing this interplay is critical. The second task is efficiently exploring the global energy hypersurface, as the energy landscape of an arbitrary molecular crystal can be quite complex, containing many local energy potential wells. Properly searching this surface requires a thorough but efficient search algorithm to find the global minimum as well as other local minima in close energetic proximity, which may exist in the case of polymorphism. The Modified Genetic Algorithm for Clusters and Crystals (MGAC) [4?6] is a parallel distributed, multithreaded application for crystal structure prediction of small organic molecules. The algorithm uses population based natural selection to perform the global search of the energy hypersurface. Most recently we have shown that DFT-D is effective when applied as the sole energy ranking method for MGAC [7]. By integrating the Quantum Espresso software suite [8] with MGAC we have been able to correctly predict the three atmospheric pressure polymorphs of glycine [3] and the single crystal form of histamine (unpublished results), when performing searches in the native space groups. The primary drawback of using DFT-D for CSP is the substantial computational time required. When compared with other commonly used methods for energetic ranking in CSP (such as molecular mechanics), DFT-D is computationally more expensive by a factor of at least 1000 when performing local optimization. In light of this fact, to perform blind predictions of crystal structures of molecules of pharmaceutical interest a significant scale up of the resources that MGAC can efficiently utilize will be required. The current version of MGAC relies on a server/client method which does not scale well for large node counts (greater than 100), nor does it allow for on the fly workload redistribution. Relevant to this is the inefficiency of the scheduling: when scaling to large numbers of nodes MGAC will often leave nodes idle for extended periods of time, leading to efficiency as low as 50% in some cases. MGAC is also highly susceptible to file system instabilities, and does not have a robust mechanism for dealing with node failures or sudden job termination. To mitigate these factors we have begun the development of MGAC2, which is projected to be functional and in production by mid-April, 2015. By mid-May we expect to have validated MGAC2 against glycine and histamine, results that we expect will be presented at this conference. As part of the development of MGAC2 we will be relying heavily on XSEDE to provide the resources required to perform blind CSP searches. We intend to show that blind CSP searches using DFT-D are both tenable and effective in finding crystal structures of pharmaceutical interest. We also intend to show that large scale simulations of population based systems can be scaled effectively to large numbers of cores, in a robust way that maximizes resources. Finally, we will present benchmarks for various molecules of pharmaceutical interest, as well preliminary results for the upcoming sixth blind test hosted by the Cambridge Crystallographic Data Centre. [1] G.M. Day, T.G. Cooper, A.J. Cruz-Cabeza, K.E. Hejczyk, H.L. Ammon, S.X.M. Boerrigter, et al., Significant progress in predicting the crystal structures of small organic molecules - a report on the fourth blind test, Acta Crystallogr. Sect. B-Structural Sci. 65 (2009) 107?125. [2] Bardwell DA, Adjiman CS, Ammon HL, Arnautova YA, Bartashevich E, Boerrigter SXM, et al., Towards crystal structure prediction of complex organic molecules - a report on the fifth blind test, Acta Cryst. B67 (2011). [3] A.M. Lund, G.I. Pagola, A.M. Orendt, M.B. Ferraro, J.C. Facelli, Crystal structure prediction from first principles: The crystal structures of glycine, Chem. Phys. Lett. 626 (2015) 20?24. [4] S. Kim, A.M. Orendt, M.B. Ferraro, J.C. Facelli, Crystal Structure Prediction of Flexible Molecules Using Parallel Geneic Algorithms with Standard Force Field, J. Comp. Chem. 30 (2009) 1973?1985. [5] V.E. Bazterra, M.B. Ferraro, J.C. Facelli, Modified genetic algorithm to model crystal structures. I. Benzene, naphthalene and anthracene, J. Chem. Phys. 116 (2002) 5984?5991. [6] V.E. Bazterra, M.B. Ferraro, J.C. Facelli, Modified genetic algorithm to model crystal structures. II. Determination of a polymorphic structure of benzene using enthalpy minimization, J. Chem. Phys. 116 (2002) 5992?5995. [7] A.M. Lund, A.M. Orendt, G.I. Pagola, M.B. Ferraro, J.C. Facelli, Optimization of Crystal Structures of Archetypical Pharmaceutical Compounds: A Plane-Wave DFT-D Study Using Quantum Espresso, Cryst. Growth Des. 13 (2013) 2181?2189. [8] P. Giannozzi, S. Baroni, N. Bonini, M. Calandra, R. Car, C. Cavazzoni, et al., QUANTUM ESPRESSO: a modular and open-source software project for quantum simulations of materials., J. Phys. Condens. Matter. 21 (2009) 395502.