Report to AAAC on Euclid/LSST/WFIRST Coordinaon

Jason Rhodes (JPL) January 26, 2017 Starng point: Summary Slide from Steve Kahn Presentaon to AAAC 1 year ago Euclid: ESA-led with NASA and Canadian contribuon What is the TAG? LSST: DOE and NSF funded, plus internaonal partners WFIRST: NASA with potenal internaonal partners • Tri-Agency: DOE/NASA/NSF • Tri-Project: Euclid/LSST/WFIRST • Meet to discuss commonalies, coordinaon, opmizaon of data, simulaons, soware • Three aspects to TAG: 1. the agency program managers talking to each other, oen as part of wider program communicaon 2. the agencies plus project leads (what we usually mean by TAG) 3. the project leads gaining input from their collaboraons, e.g. informal “task forces” • Informal group started in ~2012 • Discuss coordinaon on these three projects, primarily for dark energy • Other science areas are considered, but haven’t driven discussions • Telecons every ~2 months, in person meengs as needed (~yearly) • Facilitates open communicaon about issues that affect all the projects and require joint or coordinated efforts of the agencies Who is the TAG

• HQ Reps • Eric Linder and Kathy Turner (DOE) • Dominic Benford and Linda Sparke(NASA) • Nigel Sharp (NSF) • Project Reps • Jason Rhodes (Euclid, also a WFIRST Deputy Project Scienst and on LSST) • Steve Kahn (LSST Project, also on Euclid) • Rachel Bean (LSST Dark Energy Science Collaboraon/DESC Spokesperson, also on Euclid and WFIRST) • Neil Gehrels (WFIRST Project Scienst) • David Spergel (WFIRST Adjutant Scienst, also on Euclid and LSST); added in 2016 Philosophies

• The best dark energy constraints in ~2030 will come from a combinaon of Euclid/LSST/WFIRST data • The teams that are processing those data best understand the data and should be involved in that combinaon • We should maintain US leadership or co-leadership in dark energy science using these combined data sets • Coordinang the cadences and survey footprints can increase science output • The opmal combinaon of the data sets may need to be done at the ‘pixel’ rather than ‘catalog’ level • This requires coordinaon and goes beyond the scope of the projects • This needs to be done with forethought in order to be done correctly • Simulaons of the universe are computaonally intensive and should be used by mulple groups where possible TAG-informed acvity, but done at LSST/Euclid coordinaon project level

• Requires internaonal agreements and careful examinaon of data rights • Requires agreement of Euclid Consorum, LSST Project and LSST DESC • Previous aempts at coordinaon have not yet reached an agreement on what would be shared, how it would be shared, and when it would be shared • Both projects have data they do not want to share immediately without safeguarding their core science • In 2015, the TAG endorsed an effort at defining the scienfic and technical benefits of coordinaon (see next slide) • Euclid Consorum, LSST DESC, LSST Project produced a joint charge that led to an in-person meeng of key sciensts in July 2015 TAG-informed acvity, but done at LSST/Euclid White paper project level Aendees: • Mario Juric One day meeng organized by Rhodes in July 2016 in Oxford, UK Andy Connolly • ~20 parcipants from Euclid and LSST (many in both) Steve Kahn Ian Shipsey • Agreed to write a white paper that did not concern itself with the Robert Lupton polics of data sharing or cadence coordinaon Jean Charles Cuillandre Marc Sauvage • White paper has progressed in fits and starts since July, but expect a Bob Nichol compete dra by end of February Tom Kitching Rachel Bean • Plan to publish white paper in a refereed journal for use as a guide Anja von der Linden for future MOU discussions. Outline: Peter Capak 1. Conceptual Raonale Richard Massey 2. Benefits for Cosmology Jason Rhodes 3. Benefits for Other Science (solar system, galaxy clusters, etc) Andy Taylor 4. Technical aspects of Coordinaon (shear, photometric redshis, etc) Will Percival 5. Implementaon Plan Dominique Bougny 6. Gap Analysis/Resources Needed Eric Aubourg 7. Summary Hendrik Hildebrandt Rachel Mandelbaum Jeff Newman TAG-informed acvity, but done at LSST/WFIRST Coordinaon project level

• Primarily requires agency and inter-team discussions • WFIRST data has no proprietary period; simplifies data rights issues • First steps of coordinaon are cosmology-based but other science areas considered • WFIRST (coordinated by Rhodes) contributed 3 secons (wide survey, supernova, microlensing) to LSST observing strategy white paper* • 3 day workshop organized by Doré (relevant WFIRST Science Invesgaon Team lead), Bean, Kahn, Rhodes in Pasadena in September

*hps://github.com/LSSTScienceCollaboraons/ObservingStrategy TAG-informed acvity, but done at WFIRST/LSST Workshop project level

• Workshop in September 2016; Science and data processing synergies

discussed Registrants: Alina Kiessling (JPL) Lee Armus (IPAC) Anton Koekemoer (STScI) Rachel Bean (Cornell) Elisabeth Krause (SLAC) • Cosmology focus, but future meengs will Andrew Benson (Carnegie) Rachel Mandelbaum (CMU) Phil Bull (Caltech) Dan Masters (IPAC) branch out further Peter Capak (IPAC) Peter Melchior (Princeton) Tzu-Ching Chang (ASIAA/JPL) Alex Merson (JPL) • Can be a guide for future collaborave Andy Connolly (Washington) Hironao Myatake (JPL) Roc Cutri (IPAC) Jeff Newman (Pisburgh) meengs Olivier Doré (JPL/Caltech) Peter Nugent (LBNL) Richard Dubois (Stanford) Saul Perlmuer (UC Berkeley) Tim Eifler (JPL) Andres Plazas-Malagon (JPL) • Doré plans to start a white paper in March Harry Ferguson (STScI) Graca Rocha (JPL) Ryan Foley (UC Santa Cruz) Jason Rhodes (JPL) based on the ideas developed at the Neil Gehrels (GSFC) Michael Schneider (LLNL) Daniel Gruen (Stanford) Sam Schmidt (Davis) workshop Salman Habib (Argonne) Melanie Simet (JPL) Katrin Heitmann (Argonne) Masahiro Takada (IPMU) George Helou (IPAC) Michael Troxel (OSU) Chris Hirata (Ohio State) Anja Von den Linden (Stony Brook) Shirley Ho (LBL) Yun Wang (IPAC) • First in a series? Eric Huff (JPL) Risa Wechsler (Stanford) Bhuvnesh Jain (Penn) Hao-Yi Wu (Caltech) Steve Kahn (SLAC) Joe Zuntz (Manchester) Joint Processing

• TAG commissioned a report on how joint pixel-level data processing could be done in the US and what work is needed to make that happen • George Helou (IPAC) coordinated the effort and produced a PPT report in February 2016 • Defined a 4-stage approach to joint processing to maximize science return • Report recommended the next step be conducng a scoping study of work to be done to fulfill joint processing Joint Processing Summary Joint Processing: 4 Stages

Phase 1 Complete. Projects ready to start Phase 2 in 2017 Joint Simulaon Efforts

• TAG commissioned a report on cosmological numerical simulaons and the required compung resources • Alina Kiessling (JPL) coordinated the group that wrote the report • Report to TAG in March, 2016 Simulaon Report Key Findings/Recommendaons

• Sharing simulaons makes sense (it’s the same Universe) • Sharing compung resources makes sense- High Performance Compung (HPC) infrastructure is expensive • Open access to simulaons makes scienfic sense • Data sharing infrastructure needs work (moving large amounts of data is challenging) • Common data formats help • Covariance matrix calculaon may be computaonally challenging • Modeling of baryonic physics needs work • Recommend seng up Tri Agency Cosmological Simulaons (TACS) Task Force to explore opons for joint simulaons, HPC resource coordinaon, data sharing Simulaon Data Sharing Opportunity (1)

• Recently, a data sharing experiment was carried out and showcased at the Internaonal Conference for High Performance Compung, Networking, Storage and Analysis. Data from a large simulaon run at one supercompung facility was moved to a different facility for longer term hosng and storage. • They obtained transfer speeds of ~0.5PB+ per day between the Argonne Leadership Compute Facility (ALCF) and the Naonal Center for Supercompung Applicaons (NCSA, in Illinois) without doing anything special other than opmizing file sizes with the ~100GB transfer link. Note that both facilies already have very good hardware for data transfer purposes. • Large simulaons are being produced by a number of groups involved in LSST, WFIRST, and Euclid. The facilies that run the simulaons are not necessarily the best facilies to host and serve the Level 1 (parcle data snapshots) and Level 2 (e.g., lightcone and mock catalog) data. Simulaon Data Sharing Opportunity (2)

• Currently, there are also difficules with long-term storage of very large datasets because appropriate policies with supercompung facilies are not in place. • In order to ensure that valuable data are not being deleted, it would be beneficial to have a facility that can both store and host these Level 1 and 2 cosmological simulaon data, including staff who could set up and maintain an efficient database for serving the data. • Kiessling and Heitmann have played a key role in iniang discussions of a possible arrangement where simulaons are run using predominantly DOE supercompung facilies and are then stored and hosted at a NASA facility. VERY PRELIMINARY. • The goal would be for all simulaons hosted by this facility to be fully public. Tri Agency Cosmological Simulaons (TACS) Task Force – Starng Soon • Projects (Rhodes, Kahn, Gehrels) have asked Alina Kiessling (JPL) and Katrin Heitmann (Argonne) to dra a charge for the TACS and recommend a TACS advisory board and task force members • Dra charge produced, will be sent to the TAG next week • Focus on: • Common infrastructure to share simulaon products • Base cosmological simulaons (info from projects) • Invesgaon of systemac effects • Large simulaon campaigns Conclusions

• TAG is providing excellent inter-agency and inter-project communicaons • Dark energy cosmology is driving current acvies but other science areas may benefit and drive future acvies • LSST/Euclid and LSST/WFIRST coordinaon meengs have engaged the community • Long term joint processing plans have been laid out • Sharing of simulaons and high performance compung resources will be explored in the near future Addional slides