Data Analysis Guide Version 11.1, Document Number: HERSCHEL-HSC-DOC-1199 07 December 2016 Build 15.0.3244 Data Analysis Guide Build 15.0.3244 Table of Contents Preface .................................................................................................................... xxvi 1. Conventions used in this manual ...................................................................... xxvi 1. Data input/output ....................................................................................................... 1 1.1. Components of an observation ........................................................................... 1 1.2. Typical workflow ............................................................................................ 3 1.3. How data are stored on your disk ...................................................................... 4 1.3.1. Managing storages and pools .................................................................. 5 1.4. Getting observations from the Herschel Science Archive ........................................ 7 1.4.1. Logging into the HSA ........................................................................... 8 1.4.2. Finding observations in the HSA ........................................................... 10 1.4.3. Inspecting the query results of an observation .......................................... 11 1.4.4. Finding observation IDs outside the HUI ................................................. 12 1.4.5. Downloading one entire observation ....................................................... 13 1.4.6. Browsing an observation in the HSA with known OBSID ........................... 14 1.4.7. Downloading multiple observations ........................................................ 17 1.5. Loading observations downloaded from the HSA into HIPE .................................. 19 1.6. Managing your HSA downloads ....................................................................... 21 1.6.1. Advanced configuration ....................................................................... 23 1.7. Retrieving an observation from disk ................................................................. 24 1.8. Customising the Product Browser results ........................................................... 27 1.9. How to use the Quick Analysis perspective ........................................................ 28 1.10. Saving data (products and observations) to disk ................................................. 30 1.11. Migrating pools across incompatible versions of HIPE ........................................ 32 1.12. Exporting an observation to a colleague ........................................................... 34 1.13. Retrieving products from disk ........................................................................ 35 1.14. Removing data from disk .............................................................................. 35 1.15. On-demand reprocessing of observations .......................................................... 35 1.16. Exchanging data with FITS files ..................................................................... 36 1.16.1. Saving a product to a FITS file ............................................................ 36 1.16.2. Retrieving a Herschel product from a FITS file ....................................... 38 1.16.3. Translation of Herschel metadata to FITS keywords ................................ 39 1.16.4. Structure of Herschel products when saved as FITS ................................. 41 1.16.5. Troubleshooting FITS import/export ..................................................... 46 1.16.6. Importing a non-Herschel FITS file into HIPE ........................................ 46 1.16.7. Importing a Herschel FITS file into external applications .......................... 48 1.17. Working with the VO (External Tools) ............................................................ 52 1.17.1. Sending products from HIPE to external tools ........................................ 52 1.17.2. Sending products from external tools to HIPE ........................................ 54 1.17.3. Opening VO Tables from HIPE ........................................................... 54 1.17.4. Writing tables to files in VO-table XML format ...................................... 55 2. Saving data as text files ............................................................................................ 56 2.1. Considerations and concepts for working with text files ........................................ 56 2.2. Worked example: Saving a Spectrum product as a text file .................................... 58 2.3. Worked example: Saving a SourceListProduct as a text file ................................... 60 2.4. Worked example: Reading a Spitzer spectrum into a table dataset ........................... 62 2.5. Worked example: Reading a VizieR catalogue into a table dataset .......................... 64 2.6. Reading a comma-separated-value (CSV) file into a table dataset ........................... 67 2.7. Reading a space-separated file into a table dataset ............................................... 68 2.8. Reading an IPAC, SExtractor or Topcat file into a table dataset ............................. 70 2.9. Reading a generic ASCII table file into a table dataset ......................................... 71 2.10. Writing a table dataset to a comma-separated-values (CSV) file ............................ 71 2.11. Writing a table dataset into a space-separated-value file ...................................... 73 2.12. Writing a spectrum to an ASCII table file ........................................................ 74 2.13. Writing a table dataset to a generic ASCII table file ........................................... 79 2.14. Reading column names from a file ................................................................. 80 iii Data Analysis Guide Build 15.0.3244 2.15. Defining which lines to ignore when reading a file ............................................ 80 2.16. Specifying the data types when reading a file .................................................... 82 2.17. Specifying how data values are separated when reading a file .............................. 83 2.18. Saving and loading a configuration for reading from file ..................................... 84 2.19. Adding a header to an ASCII table file ............................................................ 85 2.20. Adding table dataset metadata to an ASCII table file .......................................... 85 2.21. Defining a custom prefix for commented lines .................................................. 86 2.22. Choosing how to separate data values .............................................................. 86 2.23. Saving and loading options for writing to file ................................................... 86 2.24. Parsers, formatters and templates .................................................................... 87 2.25. Creating and configuring table templates .......................................................... 88 2.26. Creating and configuring parsers for reading in data ........................................... 89 2.27. Creating and configuring formatters for writing data .......................................... 90 2.28. Regular expressions ...................................................................................... 92 3. Plotting .................................................................................................................. 94 3.1. Getting started .............................................................................................. 94 3.2. Creating a plot .............................................................................................. 94 3.3. Customising title and subtitle ........................................................................... 95 3.4. Managing layers ............................................................................................ 97 3.5. Showing and customising a legend ................................................................. 100 3.6. Customising plot properties ........................................................................... 100 3.6.1. Command line equivalents .................................................................. 101 3.7. Setting margins ........................................................................................... 102 3.8. Saving and printing ...................................................................................... 102 3.9. Setting line and symbol styles ........................................................................ 102 3.10. Customising axes ....................................................................................... 104 3.11. Drawing grid lines ...................................................................................... 110 3.12. Managing annotations ................................................................................. 110 3.13. Drawing filled areas ................................................................................... 113 3.13.1. Drawing filled areas between curves ................................................... 114 3.14. Drawing a straight line ................................................................................ 115 3.14.1. Drawing an arbitrarily-positioned straight line ....................................... 116 3.15. Customising auxiliary axes .........................................................................