Multiscale Materials Modelling on High Performance Computer Architectures

UNICORE-based Workflows for the Simulation of Organic Light-Emitting Diodes Stefan Bozic Ivan Kondov, Velimir Meded and Wolfgang Wenzel Karlsruhe Institute of Technology, Germany

The project MMM@HPC is funded by the 7th Framework Programme of the European Commission within the Research Infrastructures with grant agreement number RI-261594. Outline

. Project MMM@HPC overview . The Challenges . Integration concept: UNICORE . GridBean and UNICORE Workflow . Simulation of Organic Light Emitting Diodes (OLEDs) . Data Exchange . OpenMolGRID . OLED Workflow . Sharing the Workflow . Conclusions and outlook

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 2 MMM@HPC project overview

www.multiscale-modelling.eu

. HPC centres: CINECA, CSC, KIT and KIST (Korea) . Modelling and code developing groups: University Mons, CEA, CSC, STFC, University Patras, KIT . Industrial partners and users: CEA, SONY, KIT, project MINOTOR . Cooperating projects: PRACE, MINOTOR, D-Grid and NGI-DE

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 3 The challenges

Reusability

Data complexity

Solution for licensing issues

. Integration on different size and time scales Security & Reliability to address real-life problems in nano- materials science . Develop an easy to use solution for non- Capacity & Capability experts: industrial and experimental groups

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 4 Integration Concept: UNICORE

. Provision of simulation tools and Application Protocols services that can be combined in → Workflows many different application Application interfaces workflows → GridBeans . Adaptable, reusable and extendable interfaces & workflows UNICORE Client based on UNICORE UNICORE Services . Access to distributed HPC/HTC resources via UNICORE services UNICORE System

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 5 MOPAC GridBean

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 6 Application flow: Example

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 7 Data flow: Example

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 8 OLED: architecture and modelling tools

macroscopic scale molecular scale electronic scale ~ 10-6 m ~ 10-8 m ~ 10-10 m

continuum model coarse-grained model Atomistic model QM model (FEA) (CG) (MM) (QM) Elmer ToFeT (KMC) DEPOSIT MOPAC FEAP End-bridging MC LAMMPS TURBOMOLE Transporter DL_POLY BigDFT

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 9

10

hole) mobility hole) Workflow - 30 May 2012 30 May

- Calculate charge charge Calculate (electron density current Calculate Generate disordered film film disordered Generate morphologies LUMO, HOMO, Calculate energies. etc LUMO+1 couplings Electronic energies reorganization rates hopping charge Calculate ......

Film deposition Film (or MD) of hopping sites calculations QM (KMC) Carlo Monte Kinetic . . .

University of Technology, Dresden ofTechnology, University

-

1858. – , W. Wenzel, . and Wenzel, , W. UNICORE SummitUNICORE , Phys. Chem. Chem. Phys., Phys., Chem. Chem. , Phys.

–

Bredas

2008, 10, 1852 10, 2008, J. J. Kwiatkowski, J. Nelson, H. Li, H. Nelson, J. Kwiatkowski, J. J. L. J. Lennartz Stefan Bozic Bozic Stefan

25/05/2012

Alq3 disordered films disordered Alq3 OLED: simulation protocol for charge transport in in transport charge for protocol simulation OLED: Application Input/Output files

Structure

Program Parameters

Output

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 11 Application Input/Output files

Structure

Program Parameters

How to concatenate these applications? Output

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 12 Data exchange between applications

. Structure Data exchange format: (CML) . is the de facto XML for chemistry . Extendable with additional domain specific attributes (charges, temperature, density) in CML dictionaries . Data flow management with the OpenMolGRID library S. Sild et al., LNCS 3470, 464, Springer (2005); S. Sild et al., J. Chem. Inf. Model., 46, 953 (2006). . Read, write and convert . Provides a data model for molecular information . Application Wrapper API . Open Babel . Read, write and convert chemical file format

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 13 OpenMolGRID

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 14 Application Wrapper: OpenMolGRID

Application Wrapper Lifecycle 1. Preprocessing: Validation of App. Parameter, Generation of App. specific Input Files 2. Execution: Run App(s) in seperate Process(es), Monitoring of stdout/stderr (allows interaction with the application) 3. Postprocessing: Error Handling, Parsing App Output, Creation of Workflow Data

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 15 Application Wrapper Input/Output files

Structure

Program Parameters

Output With CML and Application Wrapper! Environment Variable

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 16 OLED Workflow

1 Molecule CML QM: Geometry

Optimization

MOPAC GridBean

MM: Film depostion CML DEPOSIT GridBean

MM: Determination of n Molecules PDB

protocol Site pairs BABEL GridBean

CML QM: Calculation of Hopping Sites Pairfinder GridBean CML PairPair Pair

CG: Compute Charge QM Batch GridBean mobility Edge Simulation Simulation CML Edge

Edge UNICORE Workflow UNICORE FEA: Calculate Current Reducer GridBean

Density CML Edge

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 17 UNICORE Workflow

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 18 Calculation of Hopping Sites

A deposition film has 1.000 – 1.000.000 Hopping Site candidates (Molecule Pairs) Each Hopping Site needs 3 QM calculations (2 Monomers and 1 Dimer)

QM-Batch Jobs • Short QM calculations (~ 0.5 - 5 min) • Each Pair file: includes n Molecule Pairs • Each Job: n * 3 QM calculations • Less Jobs: Minimize service overhead • Each QM Code needs additional GridBeans & Wrapper

QM-Single Jobs • Long QM calculation (30 min +) • Each Job: one QM calculation • Many Jobs & Files • Reuse of QM GridBeans • Each QM Code needs a special Reducer

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 19 Sharing the Workflow

1) Workflow Project

2. GridBeans in MMM@HPC Update Site

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 20 Conclusions and Outlook

. With UNICORE we provide an optimal low-effort/low-cost solution for multiscale modelling . GridBeans → App Interfaces . Workflows → Simulation protocol . Data Exchange in WFs between applications handled with CML, Open Babel and OpenMolGRID . Different approaches on the calculation of Hopping Sites

Current work . Integration of the CG and FEA steps into the OLED Workflow . Elmer, DL_POLY and BigDFT GridBeans . Simulation of whole OLED devices . Workflows for Molecular Electronics, Carbon Molecular Devices and Li- Ion Batteries

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 21 Acknowledgments

. All consortium partners in MMM@HPC . Funding from the EC

. Partner projects, supporting infrastructures and

25/05/2012 Stefan Bozic – UNICORE Summit - University of Technology, Dresden - 30 May 2012 22