CMTS Cyberinfrastructure
Total Page:16
File Type:pdf, Size:1020Kb
CMTS Cyberinfrastructure Michael Wilde - [email protected] hp://swi-lang.org Cyberinfrastructure Challenges • Expressing mul>-scale CMTS Cyberinfrastructure workflow protocols • Leveraging and expressing parallelism Campus-wide clusters and • Moving, processing and managing archival storage data • Traversing networks and security • Diverse environments: schedulers, storage… • Handling system failures • Tracking what was done: Departmental clusters, storage, and where, when, how, who? personal workstaons Cyberinfrastructure Tasks § Enable easy of use of mul>ple distributed large-scale systems § Reduce effort of wri>ng parallel simulaon and analysis scripts § Record and organize provenance of computaons § Annotate datasets to facilitate sharing, collaboraon, and validaon § Reduce the cost of modeling, both in terms of wall clock >me and computer >me. Swift Parallel Scripting Language: A Core Component of CMTS Cyberinfrastructure • Swi is a parallel scrip>ng language • Composes applica,ons linked by files • Captures protocols and processes as a CMTS library • Easy to write: a simple, high-level language • Small Swi: scripts can do large-scale work • Easy to run: on clusters, clouds, supercomputers and grids • Sends work to XSEDE, campus, and Amazon resources • Automates solu>ons to four hard problems • Implicit parallelism • Transparent execu,on loca,on • Automated failure recovery • Provenance tracking SwiSwi is a parallel scrip>ng language is a parallel scrip>ng language SwiSwiSwi is a parallel scrip>ng language is a parallel scrip>ng language is a parallel scrip>ng language Composes applica,ons linked by files Composes applica,ons linked by files Composes applica,ons linked by files Composes applica,ons linked by files Composes applica,ons linked by files Easy to write: a simple, high-level language Easy to write: a simple, high-level language Easy to write: a simple, high-level language Easy to write: a simple, high-level language Easy to write: a simple, high-level language Small Swi: scripts can do large-scale work Small Swi: scripts can do large-scale work Small Swi: scripts can do large-scale work Small Swi: scripts can do large-scale work Small Swi: scripts can do large-scale work Easy to run: on clusters, clouds and grids Easy to run: on clusters, clouds and grids Easy to run: on clusters, clouds and grids Easy to run: on clusters, clouds and grids Easy to run: on clusters, clouds and grids Sends work to XSEDE, Amazon, OSG, Cray Sends work to XSEDE, Amazon, OSG, Cray Sends work to XSEDE, Amazon, OSG, Cray Sends work to XSEDE, Amazon, OSG, Cray Sends work to XSEDE, Amazon, OSG, Cray Fast and highly parallel Fast and highly parallel Fast and highly parallel Fast and highly parallel Fast and highly parallel Runs a million tasks on thousands of cores Runs a million tasks on thousands of cores Runs a million tasks on thousands of cores Runs a million tasks on thousands of cores Runs a million tasks on thousands of cores hundreds of tasks per second hundreds of tasks per second hundreds of tasks per second hundreds of tasks per second hundreds of tasks per second Using Swift in the Larger Cyberinfrastructure Landscape Data servers Swift script Application Campus Local data systems programs Submit host (login node, laptop, Linux server) Swi9’s run>me system supports and aggregates diverse, distributed execu>on environments Expressing CMTS Algorithms in Swift Translang from a shell script of 1300 lines into code like this: 1. (…)EnergyLandscape(…) 2. { 3. loadPDB (pdbcode="3lzt"); 4. createSystem (LayerOfWater=7.5, 5. IonicStrength=0.15); 6. preequilibrate (waterEquilibration_ps=30, 7. totalEquilibration_ps=1000); 8. runMD (config="EL1",tinitial=0, tfinal=10, 9. frame_write_interval=1000); 10. postprocess(…); 11. forcematching(…); 12.} …to create a well-structured library of scripts that automate workflow-level parallelism CMTS Cyberinfrastructure Architecture Researchers • A challenging “breakthrough” project" • Integration & evaluation underway" CMTS Collaboraon Catalogs • Full implementation by the end of# Phase II" Files & Metadata External collaborators Provenance 0: Develop script Script libraries 1: Run script(EL1.trj) 2. Lookup file name=EL1.trj user=Anton 6: Update catalogs type=trajectory Storage 4: Run app 5: Transfer results locaons Compute Facili>es 3: Transfer inputs When do you need Swift? Typical application: protein-ligand docking for drug screening O(10) O(100K) … proteins X drug implicated candidates in a disease 1M compute jobs Tens of fruiul (B) candidates for wetlab & APS Work of M. Kubal, T.A.Binkowski, 8 www.ci.uchicago.edu/swi9 www.mcs.anl.gov/exm And B. Roux Numerous many-task applications T0623, 25 res., 8.2Å to 6.3Å (excluding tail) § A Simulaon of super- cooled glass materials § B Protein folding using homology-free approaches § C Climate model analysis and decision making in energy A B Inial policy Predicted Nave § D Simulaon of RNA-protein interac>on § E Mul>scale subsurface flow modeling F E § F Modeling of power grid for OE applicaons § > A-E have published science results obtained using Swi9 Protein loop modeling. Courtesy A. Adhikari D 9 C www.ci.uchicago.edu/swi9 www.mcs.anl.gov/exm Nested parallel prediction loops in Swift 1. Sweep( ) 2. { 3. int nSim = 1000; 4. int maxRounds = 3; 5. Protein pSet[ ] <ext; exec="Protein.map">; 6. float startTemp[ ] = [ 100.0, 200.0 ]; 7. float delT[ ] = [ 1.0, 1.5, 2.0, 5.0, 10.0 ]; 8. foreach p, pn in pSet { 9. foreach t in startTemp { 10. foreach d in delT { 11. ItFix(p, nSim, maxRounds, t, d); 12. } 13. } 14. } 10 proteins x 1000 simulaons x 15. } 3 rounds x 2 temps x 5 deltas 16. Sweep(); = 300K tasks 10 www.ci.uchicago.edu/swi9 www.mcs.anl.gov/exm Programming model: all execution driven by parallel data flow (int r) myproc (int i)! {! j = f(i); ! k = g(i);! r = j + k;! }! ! § f() and g() are computed in parallel § myproc() returns r when they are done § This parallelism is automa,c § Works recursively throughout the program’s call graph 11 www.ci.uchicago.edu/swi9 www.mcs.anl.gov/exm Programming model: Swift in a nutshell § All expressions are computed in parallel, thro5led by various levels of the run>me system § Simple data type model: scalars (boolean, int, float, string, file, external) and collec>ons (arrays and structs) § ALL data atoms and collec>ons have “future” behavior § A few simple control constructs – if, switch, foreach, iterate § A growing library – tracef(), strcat(), regexp(), readData(), writeData() 12 www.ci.uchicago.edu/swi9 www.mcs.anl.gov/exm Swift 2.0 § Mo>vaon for 2.0 – Scalability: 1M tasks/sec vs 500 – goal has been reached for basic tests – Richer programming model, broader spectrum of applicaon – Extensibility and Maintainability § Convergence issues – Some array closing; for loop extensions; library cleanup; mapper models – Data marshalling/passing for in-memory leaf func>ons § Informaon and downloads – hp://exm.xstack.org 13 www.ci.uchicago.edu/swi9 www.mcs.anl.gov/exm Encapsulation enables distributed parallelism Typed Swi data object Applicaon program Files expected Swi9 app( ) func>on or produced Interface defini>on by applicaon program Encapsula?on is the key to transparent distribu?on, paralleliza?on, and automa?c provenance capture 14 www.ci.uchicago.edu/swi9 www.mcs.anl.gov/exm app( ) func?ons specify cmd line argument passing To run: Fasta psim -s 1ubq.fas -pdb p -t 100.0 -d 25.0 >log file 100.0 25.0 In Swi code: seq dt t app (PDB pg, File log) predict (Protein seq, Float t, Float dt) { psim "-c" "-s" @pseq.fasta "-pdb" @pg -t -c -s -d "–t" temp "-d" dt; PSim applicaon } pdb > Protein p <ext; exec="Pmap", id="1ubq">; PDB structure; Swi9 app func>on File log; pg “predict()” log (structure, log) = predict(p, 100., 25.); 15 www.ci.uchicago.edu/swi9 www.mcs.anl.gov/exm Large scale parallelization with simple loops 1000 Runs of the “predict” applica,on … T1af7 T1b72 T1r6 9 Analyze() foreach sim in [1:1000] { (structure[sim], log[sim]) = predict(p, 100., 25.); } result = analyze(structure) 16 www.ci.uchicago.edu/swi9 www.mcs.anl.gov/exm Dataset mapping example: fMRI datasets type Study { Group g[ ]; } type Group { Subject s[ ]; On-Disk } Data Layout type Subject { Volume anat; Swi’s Run run[ ]; in-memory } data model type Run { Volume v[ ]; } type Volume { Mapping func>on Image img; or script Header hdr; } 17 www.ci.uchicago.edu/swi9 www.mcs.anl.gov/exm Metadata tag /d1/d2/f3 owner=asinitskiy group=cmts-chem \ create-date=2013.0415 \ type=trajectory-namd \ state=published \ note="trajectory for UCG case 1" \ molecule=1ubq domain=loop7, bead=u2 tag /d1/d2/f0 owner=jdama group=cmts-chem \ create-date=2013.0412 \ type=trajectory-namd \ state=validated \ note="trajectory for UCG case 2" molecule=clathrin, domain=loop1, bead=px 18 www.ci.uchicago.edu/swi9 www.mcs.anl.gov/exm Metadata Query ==> (owner=asinitskiy) and (molecule=1ubq)! ! ! dataset_id | name | value! ------------+-------------+------------------------! ! /d1/d2/f4 | owner | asinitskiy! | group | cmts-chem! | create-date | 2013.0416.12:20:29.456! | type | trajectory-namd! | state | published! | note | trajectory for case 1! | molecule | 2ubq! | domain | loop7! | bead | dt! ! 19 www.ci.uchicago.edu/swi9 www.mcs.anl.gov/exm Swift information § Swi Web: – hp://www.ci.uchicago.edu/swi or newer: hp://swi-lang.org § Swi User Guide: – hp://swi-lang.org/guides/trunk/userguide/userguide.html 20 www.ci.uchicago.edu/swi9 www.mcs.anl.gov/exm Running the CMTS “cyber” tutorials # pick up latest changes if instructed: cd /project/cmtsworkshop cp –rp tutorial/cyber $USER/cyber cd $USER/cyber source ./setup.sh cd basic_swi9 cat README cd part01 cat README swi9 p1.swi9 # etc etc etc # start with NAMD cd cyber/namd_sweep cat README source setup.sh # <== Don’t forget this step!!! swi rmsd.swi9 –etc etc etc Summary of the CMTS Cyberinfrastructure Approach § Streamlining scientist / computer interaction:" –! Large datasets (e.g.