sign in sign up
<<
Home , Cray XT3, Jaguar (supercomputer), Titan (supercomputer)

Data$intensive*science*executed* within*leadership$scale*computing* facilities.

Jack%.%Wells Director%of%Science Oak%Ridge%Leadership%Computing%Facility Oak%Ridge%National%Laboratory

ATLAS%Software%&%Computing%Week

This%research%used%resources%of%the%Oak%Ridge%Leadership%Computing%Facility%at%the%Oak% Ridge%National%Laboratory,%which%is%supported%by%the%Office%of%Science%of%the%U.S.% Department%of%Energy%under%Contract%No.%DE2AC05200OR22725.%Some%of%the%work% presented%here%is%from%the%TOTAL%and%Oak%Ridge%National%Laboratory%collaboration%which% is%done%under%the%CRADA%agreement%NFE214205227.%Some%of%the%experiments%were% ORNL%is%managed%by%UT2Battelle% supported%by%an%allocation%of%advanced%computing%resources%provided%by%the%National% for%the%US%Department%of%Energy Science%Foundation.%The%computations%were%performed%on%Nautilus%at%the%National% Institute%for%Computational%Sciences. What%is%a%Leadership%Computing%Facility%(LCF)?

• Collaborative,DOE,Office,of,Science,user6 • Highly,competitive,user,allocation,programs, facility,program,at,ORNL,and,ANL (INCITE,,ALCC). • Mission:,Provide,the,computational,and,data, • Projects,receive,10x,to,100x,more,resource, resources,required,to,solve,the,most, than,at,other,generally,available,centers. challenging,problems. • LCF,centers,partner,with,users,to,enable, • 26centers/26architectures,to,address,diverse, science,&,engineering,breakthroughs, and,growing,computational,needs,of,the, (Liaisons,,Catalysts). scientific,community ORNL%has%systematically%delivered%a%series% of%leadership6class%systems On(scope(•(On(budget(•(Within(schedule

OLCF%3

OLCF%2 27 PF OLCF%1 2012 2.5 1000%fold (XK7( 1( PF improvement PF 263( 2009 in(8(years TF 2008 Cray(XT5( 62( Cray(XT5( 54( 2008 25( TF TF Jaguar 18.5 TF Cray(XT4( TF Titan,(five(years(old(in(October(2017,(continues( 2006 2007 Jaguar Cray(XT3( Cray(XT4( to(deliver(world%class(science(research(in(support( 2004 2005 Jaguar of(our(user(community.(We(will(operate(Titan( Cray(X1E( Cray(XT3( Jaguar Phoenix( Jaguar through(2019(when(it(will(be(decommissioned. We#are#building#on#this#record#of#success# to#enable#exascale in#2021

OLCF#5 ~1 OLCF#4 EF 200 2021 PF 500#fold improvement Frontier 27 2018 in090years PF IBM0 2012 Cray0XK70 Titan Outline • Science'requirements'EOD'Science:'highlight'DOE/SC/ASCR' workshop'reports. • Case'Studies'(from'Titan'and'Summitdev): – Big'health'data'analytics:'massiveHscaleHanalysis'of'pathology'reports – Vertex'reconstruction'in''physics'experiments – Predict'plasma'disruptions'in'tokamak'fusion'reactor – Multimodal'characterization'of'materials – pbdR:'Scalable'R'Development'Platform'for'Big'Data'Analytics • Discussion'of'Summit’s'architecture'from'machine'learning'perspective • Preliminary'Summit'performance'on'CORAL'2'Deep'Learning'Suite' (DLS)'relative'to'performance'on'Titan • Roadmap'to'Frontier,'ORNL’s'first'exascale'system'&'CORAL'2' Procurement Experimental,and,Observational,Science,Data,is,Exploding

DOE$Workshop:$Data$Management,$Analysis$and$ $for$Experimental$and$Observational$ Data$(EOD)$Workshop,$(2015) Program$Leader:$Lucy$Nowell,$DOE/ASCR

https://extremescaleresearch.labworks.org/events/ dataQmanagementQvisualizationQandQanalysisQ experimentalQandQobservationalQdataQeodQ workshop Experimental,and,Observational,Science,Data,is,Exploding

• Growing(data(sizes(&(complexity Kalinin(et(al.,( ACS$Nano,(906819086,(2015 – Cannot&use&desktop&computers&for&analysis ! Need(HPC! • Multiple(file(formats – Multiple&data&structures – Incompatible&for&correlation ! Need(universal,(scalable,(format

• Disjoint(and(unorganized(communities – Similar&analysis&but&reinventing&the&wheel – Norm:&emailing&each&other&code,&data ! Need(centralized(repositories • Proprietary,(expensive(software ! Need(robust,(open,(free(software DOE$ASCR)Exascale)Requirements)Reviews

ASCR)facilities)conducted six)exascale) Schedule requirements)reviews)in)partnership)with)DOE) Science Programs June%10–12,%2015 HEP • Goals)included: November%3–5,%2015 BES – Identify mission)science)objectives)that)require) January%%27–29,%2016 FES advanced)scientific)computing,)storage)and) March%29–31,%2016 BER networking)in)exascale)timeframe June%15–17,%2016 NP Sept 27–29, 2016 ASCR – Determine future)requirements)for)a)computing) ecosystem)including)data,)software,) March%9–10,%2017% XCut libraries/tools,)etc. Common%Themes%Across%DOE%Science%Offices Data:%Large*scale%data%storage%and%analysis

Experimental,and,simulated,data,set, Methods,and,workflows,of,data,analytics, volumes,are,growing,exponentially., are,different,than,those,in,traditional,HPC., Examples:,High,luminosity,LHC, light, Machine,learning,is,revolutionizing,field., sources, climate,,cosmology,data, Established,analysis,programs,must,be, sets,~,100s,of,PBs., accommodated. Current,capability,is,lacking. DOE/SC'Requirements'Crosscut'Report: Executive)summary)findings)support)machine6learning)needs Data: • “[…]&performing&analyses&of&big&datasets&and& drawing&inferences&based&on&these&data&are& CROSSCUT revolutionizing&many&fields.&New'approaches'are' REPORT needed'for'analyzing'large'datasets'including' EXASCALE advanced'statistics'and'machine'learning.”& REQUIREMENTS REVIEWS Software'and'Application'Development: March 9–10, 2017 – Tysons Corner, Virginia • Scalable'data'processing,'data'analysis,'

machine'learning,'discrete'algorithms,&and& An Offce of Science review sponsored by: Advanced Scientifc Computing Research Basic Energy Sciences multiscale/&multiphysics simulations&are&crucial&for& Biological and Environmental Research Fusion Energy Sciences High Energy Physics reducing&and&understanding'the'largeEscale' Nuclear Physics data'that'will'be'produced'by'exascale' systems.'

All&6&workshop&reports&are&available&online,&XECut&online&soon: http://exascaleage.org/ LHC/ATLAS(OLCF+Integration Workflow(Management(on(Titan:( Next(Generation(Workflow(Management((for(High(Energy(Physics(

“BigPanDA*Workflow*Management*on*Titan*for*High*Energy*and*Nuclear*Physics* and*for*Future*Extreme*Scale*Scientific*Applications,”*DOE/SC/ASCR*NextJ Generation*Networking*for*Science,*Rich*Carlson.*PI:*Alexei*Klimentov (BNL)T* CoJPisT*K.*De*(U.*TexasJArlington),*S.*Jha (Rutgers*U)*J.C.*Wells*(ORNL) LHC$Upgrade$Timeline

In#10#years,#increase#by#factor#10#the#LHC#luminosity ➔ More#complex#events ➔ More#Computing#Capacity LHC$Upgrade$Timeline

Run4 Run3 Run2 2020)2022 2015%) 2018 Run1% 2009%) 2013

In#10#years,#increase#by#factor#10#the#LHC#luminosity ➔ More#complex#events ➔ More#Computing#Capacity Production*and*Distributed*Analysis* (PanDA) • 6#main#subsystems: – PanDA Server#(Run#1) – PanDA Pilot#(Run#1) – JEDI#(Run#2) – AutoPyFactory (Run#1) – PanDA Monitoring#(Run#1) – Event#Service#(Run#2) • Developed#for# – HEP#workloads#and#workflows – HTC#on#Grid#infrastructures • TaskMbased#and#jobMbased# execution#mode • Support#single#researchers,# research#groups,#ATLAS# production#workflows.# PanDA on'Titan

• Cannot'deploy'PanDA Pilot' on'Titan’s'worker'nodes' (WN) • PanDA Brokers'on'Titan’s' DTN • MPI'scripts'wrap'jobs’' payload • AthenaMP on'readBonly' NFS'shared'between'DTN' and'WN • Events'on'OLCF' FS Understanding+Backfill+Slot+Availability

• Efficiency:*fraction*of*core/hours*utilized*by*the*PanDA Brokers*to*Titan’s* total*backfill*availability. • Mean*BF*availability:*691*worker*nodes*for*126*minutes. • Up*to*15K*nodes*for*30/100*minutes NEEDS*UPDATING • large*margin*of*optimization Making'Use'of'“Unusable'Backfill”

NEEDS%UPDATING ATLAS%simulation%time%worldwide:% February%2017 NEEDS&UPDATING

• ATLAS&Detector&Simulation& integrated&with&Titan&(OLCF) • Titan&has&already&&contributed& a&large&fraction&of&computing& resources&for&MC&simulations – Titan&contributed&4.4%&of&total& simulation&time&in&February&2017. Emerging(Science(Activities: NEEDS%UPDATING Selected Machine,Learning,Projects,on,Titan:,2016:2017

Program PI( PI(Employer( Project(Name( Allocation((Titan(core7hrs) Discovering*Optimal*Deep*Learning*and*Neuromorphic*Network*Structures*using*Evolutionary* ALCC Robert*Patton* ORNL* 75,000,000 Approaches*on*High*Performance*Computers* ALCC Gabriel*Perdue* FNAL* Large*scale*deep*neural*network*optimization*for*neutrino*physics* 58,000,000 ALCC Gregory*Laskowski* GE HighKFidelity*Simulations*of*Gas*Turbine*Stages*for*Model*Development*using*Machine*Learning* 30,000,000 HighKThroughput*Screening*and*Machine*Learning*for*Predicting*Catalyst*Structure*and*Designing* ALCC Efthimions*Kaxiras* Harvard*U.* 17,500,000 Effective*Catalysts*

ALCC Georgia*Tourassi* ORNL* CANDLE*Treatment*Strategy*Challenge*for*Deep*Learning*Enabled*Cancer*Surveillance* 10,000,000

DD Abhinav*Vishnu* PNNL* Machine*Learning*on*Extreme*Scale*GPU*systems* 3,500,000 DD J.*Travis*Johnston* ORNL* Surrogate*Based*Modeling*for*Deep*Learning*HyperKparameter*Optimization* 3,500,000 DD Robert*Patton* ORNL* Scalable*Deep*Learning*Systems*for*Exascale*Data*Analysis* 6,500,000 DD William*M.*Tang* PPPL* Big*Data*Machine*Learning*for*Fusion*Energy*Applications* 3,000,000 DD Catherine*Schuman* ORNL* Scalable*Neuromorphic*Simulators:*High*and*Low*Level* 5,000,000 DD Boram*Yoon* LANL Artificial*Intelligence*for*Collider*Physics* 2,000,000 DD JeanKRoch*Vlimant* Caltech* HEP*DeepLearning* 2,000,000 DD Arvind*Ramanathan* ORNL* ECP*Cancer*Distributed*Learning*Environment* 1,500,000 DD John*Cavazos* U.*Delaware* LargeKScale*Distributed*and*Deep*Learning*of*Structured*Graph*Data*for*RealKTime*Program*Analysis 1,000,000 DD Abhinav*Vishnu* PNNL* Machine*Learning*on*Extreme*Scale*GPU*systems* 1,000,000 DD Gabriel*Perdue* FNAL* MACHINE*Learning*for*MINERvA* 1,000,000 TOTAL 220,500,000 Scaling(Deep(Learning(for(Science ORNL%designed,algorithm,leverages,Titan,to,create,high%performing,neural,networks

• Using'the'Titan'supercomputer,'a'research'team'led'by'Robert' Patton'of'ORNL'has'developed'an'evolutionary'algorithm' capable'of'generating'custom'neural'networks'that'match'or' exceed'the'performance'of'handcrafted'artificial'intelligence' systems.' • By'leveraging'the'GPU'computing'power'of'the'Cray'XK7'Titan' supercomputer,'these'autoIgenerated'networks'can'be' produced'quickly,'in'a'matter'of'hours'as'opposed'to'the'months' An'image'generated'from'neutrino'scattering'data' captured'by'the'MINERvA detector'at'Fermi'National' needed'using'conventional'methods. Accelerator'Laboratory.'Researchers'are'using' MENNDL'and'the'Titan'supercomputer'to'generate' • Scaled'across'Titan’s'18,688'GPUs,'the'team’s'algorithm'called' deep'neural'networks'that'can'classify'highIenergy' physics'data'and'improve'the'efficiencies'of' MENNDL'can'test'and'train'thousands'of'potential'networks'for' measurements.'' a'science'problem'simultaneously,'eliminating'poor'performers' and'averaging'high'performers'until'an'optimal'network' Related(Publication:(Young,'Steven'R.,'et'al.'“Evolving' Deep'Networks'Using'HPC.”'In'Proceedings,of,the, emerges.' Machine,Learning,on,HPC,Environments.'Paper' presented'at'The,International,Conference,for,High, • The'process'eliminates'much'of'the'timeIintensive,'trialIandI Performance,Computing,,Networking,,Storage,and, Analysis,'Denver,'CO,'November'2017.' error'tuning'traditionally'required'of'machine'learning'experts.' DOI:10.1145/3146347.3146355 Multi&node+Evolutionary+Neural+Networks+for+Deep+Learning+ (MENNDL)

Premise:((For+every+data+set,+there+exists+a+ corresponding+neural+network+that+performs+optimally+ with+that+data • ORNL+Data+Analytics+Group+used+Titan+to+develop+an+ evolutionary+algorithm+to+search+for+optimal+hyper¶meters+ and+topologies+for+ML+networks.+ • Demonstrated+on+18,000+nodes+of+Titan+using+high+energy+ physics+data+through+collaboration+with+Fermilab • Evaluated+against+multiple+datasets

• Standard+computer+vision+datasets Evolve • Neutrino+detector+vertex+finding+dataset d • SNS+Small+angle+scattering+model+fitting+dataset • Currently+exploring+additional+datasets+and+evaluating+performance+ on+Summit&Dev Coming'in'2018:'Summit'will'replace'Titan' as'the'OLCF’s'leadership'supercomputer'

Summit,'slated'to'be'more'powerful'than'any'other'existing' supercomputer,'is'the'Department'of'Energy’s'Oak'Ridge'National' Laboratory’s'newest'supercomputer'for'open'science. Compute&System Summit&Overview 10.2&PB&Total&Memory 256#compute#racks 4,608#compute#nodes Compute&Rack Mellanox EDR#IB#fabric 18#Compute#Servers 200#PFLOPS Warm#water#(70°F#directJcooled# ~13#MW# components) Compute&Node RDHX#for#airJcooled#components 2#x#POWER9 6#x##GV100 Components NVMeJcompatible#PCIe 1600#GB#SSD# IBM&POWER9 • 22#Cores • 4#Threads/core • NVLink

! 39.7#TB#Memory/rack GPFS&File&System ! 25#GB/s#EDR#IBJ (2#ports) 55#KW#max#power/rack 512#GB#DRAMJ (DDR4) 250&PB&storage 96#GB#HBMJ (3D#Stacked) 2.5#TB/s#read,#2.5#TB/s#write Coherent#Shared#Memory

NVIDIA&GV100 • 7#TF • 16#GB#@#0.9#TB/s • NVLink Summit&Node&Overview DRAM DRAM

7 TF 256 GB 256 GB 7 TF GPU GPU HBM HBM 16 GB 16 GB 900 900 GB/s 900 GB/s

50 GB/s 50 GB/s 50 GB/s50 GB/s50

135 GB/s 135 64 GB/s 135 GB/s P9 P9 7 TF 7 TF GPU GPU HBM HBM 16 GB 16 GB 50 GB/s50 GB/s50 900 900 GB/s 900 GB/s

16 GB/s

50 GB/s 50 GB/s 16 GB/s 50 GB/s50 GB/s50 NIC 7 TF 7 TF GPU GPU HBM HBM 16 GB 16 GB 900 900 GB/s 900 GB/s

6.0 GB/s Read NVM 2.2 GB/s Write 12.5 GB/s12.5 GB/s12.5

TF 42 TF (6x7 TF) HBM/DRAM Bus (aggregate B/W) HBM 96 GB (6x16 GB) NVLINK DRAM 512 GB (2x16x16 GB) X-Bus (SMP) NET 25 GB/s (2x12.5 GB/s) PCIe Gen4 MMsg/s 83 EDR IB HBM & DRAM speeds are aggregate (Read+Write). All other speeds (X-Bus, NVLink, PCIe, IB) are bi-directional. Coming'in'2018:'Summit'will'replace'Titan' as'the'OLCF’s'leadership'supercomputer'

Feature Titan Summit Application Performance Baseline 5@10x&Titan Number&of&Nodes 18,688 4,608 Node&performance 1.4&TF 42&TF • Many&fewer&nodes Memory per&Node 32 GB DDR3&+&6&GB&GDDR5 512&GB&DDR4&+&96&GB&HBM2 • Much&more&powerful&nodes NV&memory per&Node 0 1600&GB • Much&more&memory&per&node& Total&System&Memory 710&TB >10&PB&DDR4&+&HBM2&+ Non@volatile and&total&system&memory System&Interconnect Gemini&(6.4&GB/s) Dual&Rail&EDR@IB (25&GB/s) • Faster&interconnect Interconnect&Topology 3D Torus Non@blocking&Fat&Tree • Much&higher&bandwidth& Bi@Section&Bandwidth 15.6&TB/s 115.2 TB/s between&CPUs&and&GPUs 1&AMD&™ 2&IBM&POWER9™ Processors 1&NVIDIA&Kepler™ 6&NVIDIA Volta™

• Much&larger&and&faster&file& ® system File&System 32&PB,&1&TB/s, Lustre 250 PB,&2.5&TB/s,&GPFS™ Power&Consumption 9&MW 13&MW Summit&will&be&the&world’s&smartest& supercomputer&for&open&science But$what$makes$a$supercomputer$smart? Summit'provides'unprecedented'opportunities'for'the'integration' of'artificial'intelligence'(AI)'and'scientific'discovery.'Here’s'why:

• GPU&Brawn:&Summit'links'more'than'27,000'deep7learning' optimized'NVIDIA'GPUs'with'the'potential'to'deliver' exascale7level'performance'(a'billion7billion'calculations'per' second)'for'AI'applications. • High=speed&Data&Movement:&NVLink high7bandwidth' technology'built'into'all'of'Summit’s'processors'supplies'the' next7generation'“information'superhighways”'needed'to'train' deep'learning'algorithms'for'challenging'science'problems' quickly. • Memory&Where&it&Matters:'Summit’s'sizable'local'memory' One$of$Summit’s$4,600$IBM$AC922$nodes.$Each$node$ contains$six$NVIDIA$Volta$GPUs$and$two$IBM$Power9$ gives'AI'researchers'a'convenient'launching'point'for'data7 CPUs,$giving$scientists$new$opportunities$to$automate,$ intensive'tasks,'an'asset'that'allows'for'faster'AI'training'and' accelerate$and$drive$understanding$using$artificial$ greater'algorithmic'accuracy. intelligence$techniques. Summit&will&be&the&world’s&smartest& supercomputer&for&open&science But$what$can$a$smart$supercomputer$do?

Science&challenges&for&a&smart&supercomputer:&

Identifying&Next;generation&Materials Predicting&Fusion&Energy By&training&AI&algorithms&to&predict&material& Predictive&AI&software&is&already&helping& properties&from&experimental&data,& scientists&anticipate&disruptions&to&the&volatile& longstanding&questions&about&material& plasmas&inside&experimental&reactors.& behavior&at&atomic&scales&could&be&answered& Summit’s&arrival&allows&researchers&to&take& for&better&batteries,&more&resilient&building& this&work&to&the&next&level&and&further& materials,&and&more&efficient&semiconductors.& integrate&AI&with&fusion&technology.&

Deciphering&High;energy&Physics&Data Combating&Cancer With&AI&supercomputing,&physicists&can&lean&on& Through&the&development&of&scalable&deep& machines&to&identify&important&pieces&of& neural&networks,&scientists&at&the&US& information—data&that’s&too&massive&for&any& Department&of&Energy&and&the&National& single&human&to&handle&and&that&could&change& Cancer&Institute&are&making&strides&in& our&understanding&of&the&universe. improving&cancer&diagnosis&and&treatment.& We#have#a#lot#of#interest#in#AI1related#projects#on#Summit Summit)provides)unprecedented)opportunities)for)the)integration)of)artificial)intelligence)(AI))and) scientific)discovery.)Here)are)a)few)outcomes)that)teams)are)hoping)to)achieve)on)Summit:

• Determine)the)3D)atomic)structure)of)complex)materials)by)inverting)electron)scattering)data. • Virtually)dock)28)million)small?molecule)compounds)to)protein)structures)of)interest)by)combining) extensive)data)on)the)human)genome)and)protein)structures)with)molecular)dynamics) simulations. • Accelerate)understanding)of)human)health)and)disease)outcomes)through)analysis)of)large,) diverse)biomedical)datasets. • Advance)the)search)for)inhibitory)drugs)that)will)target)intrinsically)disordered)proteins,)linked)to)a) number)of)disease)states. • Enable)breakthrough)advances)in)global?scale)mapping,)with)applications)in)urban)development,) emergency)response,)and)national)security. • Transform)science?data)analytics,)specifically)in)cosmology)and)climate)science. • Develop)machine)learning)methods)to)advance)physics?based)models)for)additive)manufacturing) and)turbomachinery. 29 Installation)Nearing)Completion

• Hardware(installation(completed(in(March • OLCF(is(working(with(IBM,(NVIDIA,( Red(Hat,(and(Mellanox to(stabilize(and( • Continuing(to(stabilize(nodes,(disks,(and( debug(system(software network • In(December,(accepted(1,080(of(4,608( nodes(to(port(codes

ASCAC(4/17,(2018 Summit&is&still&under&construction

• We$expect$to$accept$the$machine$in$Summer$of$2018,$allow$early$users$on$this$ year,$and$allocate$our$first$users$through$the$INCITE$program$in$January$2019.$ Innovative)and)Novel)Computational)Impact)on)Theory) and)Experiment)(INCITE))Program for)2019

• Access%to%the%most%capable,%most%productive,% fastest%open%science%%in%the%nation • Call%for%proposals%submission%window: − Apr%16%– Jun%22,%2018 • Applicable%to%a%broad%array%of%science,%engineering,% and%computer%science%domains • Proposals%must%be: ⁻ HighFimpact,%computationally%and/or%data%intensive% campaigns ⁻ Must%take%advantage%of%unique%HPC%architectures% ⁻ Research%that%cannot%be%performed%anywhere%else. • For%more%information%visit% http://www.doeleadershipcomputing.org/%%% From%Summit%to%Frontier:% The%OLCF%2021%Exascale%computer

HighM Advanced$ performance$ Artificial simulations data$ intelligence analytics

2021$delivery$planned$with$>1000$PF$ Design$review Award$contract performance$(50–100! the$scientific$ Solicit$ productivity$of$current$Titan$system) bids$for$ computer$ Frontier$NRE:$hardware$and$$software$engineering$ build Prepare$power,$cooling,$and$space Installation Prepare$scientific$software$applications Frontier$available Summit$available:$200$PF$production$system Titan$available:$27$PF$ CY%2017 CY%2018 CY%2019 CY%2020 CY%2021 CY%2022 CY%2023

Frontier$is$scheduled$to$be$available$for$users$in$2022. What is CORAL?

• CORAL is a Collaboration of Oak Ridge, Argonne, and Lawrence Livermore Labs • Began in 2012 to acquire three systems for delivery in 2017 with a single RFP • This CORAL partnership and process was considered a big success in the first acquisition • So the partnership is being continued for the second round of system acquisitions (called CORAL-2) with a single RFP for delivery of a system in late 2021 and one or two systems in late 2022.

http://2procurement.ornl.gov/rfp/CORAL2/ DOE HPC Facilities Systems Pre-Exascale Systems Exascale Systems 2013 2016 2018 2020 2021-2022

Mira A21 Theta Argonne Argonne Argonne IBM BG/Q /Cray KNL Intel/Cray TBD Unclassified Unclassified Unclassified NERSC-9 CORI LBNL LBNL Cray/Intel Xeon/KNL TBD Unclassified Unclassified Frontier Titan Summit ORNL ORNL ORNL IBM/NVidia Cray/NVidia K20 TBD P9/Volta Unclassified Unclassified Unclassified El Capitan Sierra LLNL LLNL Trinity LLNL IBM/NVidia Crossroads TBD IBM BG/Q Classified Classified P9/Volta LANL/SNL LANL/SNL Classified TBD Cray/Intel Xeon/KNL Classified Classified 3 RFP Will Request Two Proposals

• ANL is getting a 2021 Intel system (Aurora) outside this RFP • The RFP requests proposals for an ORNL system delivered in late 2021 (accepted in 2022) with the requirement that the system must be diverse from the ALCF 2021 system. • The RFP requests proposals for a LLNL system delivered in late 2022 (accepted in 2023). There is no diversity requirement. • ANL, as CORAL partner, has the potential to choose a system from the same pool as LLNL . • The reasons for just one RFP: – Same RFP release time – Same system requirements (based on ECP exascale definition) – Only differences are facility descriptions and I/O subsystems – Potential opportunity to share NRE costs

4 Questions? Jack.Wells [email protected]