Data Analysis Tools for the HST Community

EXPANDING THE FRONTIERS OF SPACE ASTRONOMY

Data Analysis Tools for the HST Community STUC 2019 Erik Tollerud Data Analysis Tools Branch Project Scientist What Are Data Analysis Tools? The Things After the Pipeline

• DATs: Post-Pipeline

Tools Spectral visualization tools Specutils Image visualization tools io models training & documentation modeling fitting - Analysis Tools astropy-helpers gwcs: generalized wcs wcs asdf: advanced data format JWST Tools tutorials ‣ Astropy software distribution astroquery MAST JWST data structures nddata ‣ Photutils models & fitting Astronomy Python Tool coordination photutils + others Development at STScI ‣ Specutils

STAK: Jupyter notebooks - Visualization Tools IRAF Add to existing python lib cosmology External replacement Build replacement pkg ‣ Python Imexam (+ds9) constants astropy dev IRAF switchers guide affiliated packages no replacement ‣ Ginga ‣ SpecViz ‣ (MOSViz) ‣ (CubeViz) Data Analysis Software at STScI is for the Community Software that is meant to be used by the astronomy community to do their science. This talk is about some newer developments of interest to the HST user community. 2. 3.

1. IRAF

• IRAF was amazing for its time, and the DAT for generations of astronomers

What is the long-term replacement strategy? Python’s Scientific Ecosystem Has what Astro Needs

Python is now the single most- used programming language in astronomy.

Momcheva & Tollerud 2015

And 3rd-most in industry… Largely because of the vibrant scientific and numerical ecosystem (=Data Science): (Which HST’s pipelines helped start!) IRAF<->Python is not 1-to-1. Hence STAK

Lead: Sara Ogaz Supporting both STScI’s internal scientists and the astronomy community at large (viewers like you) requires that the community be able to transition from IRAF to the Python-world. This is the purpose of the STAK notebooks

https://stak-notebooks.readthedocs.io/en/latest/

Fully supported via the STScI Help Desk

(And it is all open for the user community to contribute to and improve:) https://github.com/spacetelescope/stak-notebooks STAK Notebooks

Example: IRAF’s images.imfilter tasks map to several different packages and functions or objects in Python.

(And all the code is runnable if you download the notebook) Drizzlepac Notebooks https://spacetelescope.github.io/notebooks/

Inspired by this, a larger effort continues to write and serve Jupyter notebooks that show specific use cases to show the community how to use the Python-based DATs on HST data. New Development: Deep Synergy w/ JWST DATs w/Harry Ferguson, Susan Kassin JWST DATs are almost entirely in Python, and meet many of HST’s needs with only minor additions. • Data Analysis Tools - Astropy - Photutils (w/Astropy) - Specutils (w/Astropy) - WebbPSF • Visualization Tools - Python Imexam (+ds9) - Ginga - SpecViz - (MOSViz) - (CubeViz) STScI Community Software is Built on a Shared Foundation

Openness leads to sharing the load

Core Shared Philosophy:

Open Development (≠ Open Source)

Diagram credit: Jake vanderPlas What is Open Development?

• A way to build software that emphasizes processes where every step is done publicly. • Anyone, internal or external, can participate as a 1st- class citizen. • This makes the software also by the astronomy community. (Like you!) What is Open Development?

• Starting from a Pull Request… Anyone can review • Commenters may be scientists • Or engineers • The original author then chooses how to update it • The maintainers just hit “merge”. The user has become a contributor. Open Development is also Open Planning

• Design and planning of these tools also occurs in the open, in the same place the code lives. • Contributions/ You can do any of this right now! opinions are accepted from anyone. Astropy

http://docs.astropy.org • Data Analysis Tools • nddata: Image analysis and • Units and “Quantities” (arrays interoperability data - Astropy with units that act the way structures you’d expect). Integrated with - Photutils (w/Astropy) • Astro-appropriate convolution comprehensive astro- - Specutils (w/Astropy) appropriate physical constants • WCS (pixel ↔ sky mapping) - WebbPSF • Date/time good to • Extensible I/O: FITS, • Visualization Tools nanoseconds over a Hubble VOTable, hdf5, custom - Python Imexam time • Astro-relevant data models (+ds9) • Celestial coordinates and their and compatible ﬁtting/ transformations - Ginga optimization - SpecViz • Table manipulation, including • Common Astrostatistics tools - (MOSViz) many arcane astro formats • Cosmology tools - (CubeViz) Photutils https://photutils.readthedocs.io

• Data Analysis Tools - Astropy - Photutils (w/Astropy) - Specutils (w/Astropy) - WebbPSF • Visualization Tools - Python Imexam (+ds9) - Ginga - SpecViz - (MOSViz) - (CubeViz) A Demo of photutils on ACS data

Not shown here: Crowded-Field PSF Photometry (a la DAOPHOT/DOLPHOT) in the same framework. Specutils https://specutils.readthedocs.io

An Astropy-coordinated package with data structures and standard analysis functions for spectroscopy.

• Data Analysis Tools •Analysis functions -> - Astropy •Flux, Centroids, - Photutils (w/Astropy) FWHM - Specutils (w/Astropy) •Continuum ﬁtting/ - WebbPSF subtraction •Spectral arithmetic, • Visualization Tools respecting units - Python Imexam (+ds9) •Line modeling - Ginga - SpecViz - (MOSViz) •Pythonic data structures of spectra^ - (CubeViz) A Demo of specutils on STIS data Imexam (in Python) Lead: Megan Sosey

• Data Analysis Tools - Astropy - Photutils (w/Astropy) - Specutils (w/Astropy) - WebbPSF • Visualization Tools - Python Imexam (+ds9) - Ginga - SpecViz - (MOSViz) - (CubeViz) Specviz

• Data Analysis Tools - Astropy - Photutils (w/Astropy) - Specutils (w/Astropy) - WebbPSF • Visualization Tools - Python Imexam (+ds9) - Ginga - SpecViz - (MOSViz) - (CubeViz) But not everything comes for free

• Some of these tools require more directed effort to apply newly-developed JWST tools to HST. • Grism tools have been developed but tend to be problem-specific. JWST is more general but a lot of details are missing for HST • Sophisticated PSF modeling tools exist for for JWST (and WFIRST), and could replace tinytim, but need work to be backported and tested against HST’s cameras. • Tools for empirical PSFs for astrometry and photometry are being developed for JWST from knowledge of HST, but need customization for HST. • (Tour favorite problem could go here!) • We may be able to do this if you tell us it’s a priority, but we want to know what you want. This is how STScI is helping build a new generation of data analysis tools that are not just for science with HST, but also by and for, its community.

But we need your input! Comments/questions? Possibilities include: Should we focus on grism tools/aXe replacement? Or better PSFs/tinytim replacement?