ESO Messenger N
Total Page:16
File Type:pdf, Size:1020Kb
VLVLTT SSCIENCECIENCE PPRODUCTSRODUCTS PPRODUCEDRODUCED BYBY PPIPELINESIPELINES:: AA SSTTAATUSTUS RREPOREPORTT USING THE DATA FLOW SYSTEM, ESO IS PRODUCING CALIBRATED SCIENCE DATA PRODUCTS FOR ALL VLT AND VLTI INSTRUMENTS. FOR MANY USERS, THESE PRODUCTS CAN BE USED DIRECTLY FOR SCIENTIFIC ANALYSIS WITHOUT FURTHER PROCESSING. IN THE END, THE SCIENTIFIC USEFULNESS OF THESE PRODUCTS DEPENDS STRONGLY ON THE NEEDS OF THE INDIVIDUAL INVESTIGATORS. DAVID R. SILILVA AND MICHÈLEICHÈLE PERON (ESO) NE OF THE GOALS for the VLT Data Flow System (Quinn 1996) is to provide calibrated data products ready for science analysis OforO the instrument modes used most heavily during service observing. Achieving this goal depends on a complex interaction elescopes and Instrumentation between the science goals of the astronomer, T the operational model of the observatory, and the implementation of instrument cali- bration pipelines. The VLT Science Policy document establishes the following guide- lines for instrument calibration and data products: ESO will execute and maintain a cali- bration plan for all VLT/VLTI instru- ments. The calibration data resulting from this plan shall be made available to the ESO community. ESO will also use these data to monitor the long-term evo- lution of instruments and to produce Figure 1: Science integration hours per VLT instrument, Period 70 – 73. The FORS1, FORS2, data products. These data products will ISAAC, and UVES instruments were operational during this entire two-year period. VIMOS and result from pipeline processing using the GIRAFFE operations did not start until February 2003. Exact dates are 2002 October 1 to 2004 September 30. Guaranteed Time Observations (GTO) data have been excluded. VLT/VLTI Data Flow System. Data products will consist of data with the instrumental signature removed as well as data calibrated into physical units. In this article, the current status of metaphor for all science instruments operat- The accuracy of the instrumentation and VLT/VLTI science products delivered by ed on Cerro Paranal. Messenger articles by physical unit calibration will be moni- instrument calibration pipelines is presented Comerón et al. (2003) and Mathys (2004) tored and maintained by ESO. with an emphasis towards science readiness, provide recent overviews of the VLT service In order to allow the ESO community to i.e. whether the currently delivered science observing operational model. Discussion of reproduce and modify the output of products can be used for science analysis instrument calibration pipelines and science instrument calibration pipelines, ESO without further processing. For the most products for La Silla instruments are outside will make the pipeline reduction recipes heavily used instrument modes over the last the scope of this article. and code available to the ESO commu- two years, we believe the answer is “yes”, nity for all supported instrument modes. depending on the science objective and the INSTRUMENT CALIBRATION PIPELINES Over time, the number of supported required calibration precision. In the end, Operational fundamentals modes for each instrument will be however, the usefulness and quality of any All VLT science observations are executed increased to ensure that the most active- given science data product is in the eye of using Observation Blocks (OBs) that are in ly used modes have calibrated data the beholder, i.e. it strongly depends on the turn composed of observation templates. products available. technical needs and science objectives of the Automatic instrument calibration pipelines The accuracy of ESO pipelines should end-user astronomer or data analysis team. have been implemented for the most heavily be such to satisfy a major fraction of the We hope this article will not only provide used instrument modes. These pipelines pro- scientific needs of the users and ESO information to the ESO user community but duce master calibration products used to will attempt to increase their accuracy also elicit feedback to help us improve our monitor instantaneous and long-term instru- over time following the guidance of its service. ment performance. Using the master calibra- community and in-house scientists. In the following article, VLT is used as a tion products, the pipelines also process raw 2 The Messenger 118 science data into science products. Data are tions during the last two years is summa- All released pipeline recipes come with processed on a template-by-template basis. rized in Figure 1. During this interval, more technical documentation – some good and Data from different templates (or different science data was produced in Service Mode some not so good. More useful for OBs) are not processed together. In general, than in Visitor Mode for all instruments. astronomers are the science reduction dis- only so-called standard instrument configu- This information is broken down by instru- cussions maintained for each of the VLT rations are supported. ment mode in Figure 2. Modes without instruments on the ESO QC Web pages The automatic production of science pipeline support are indicated by vertical (http://www.eso.org/qc). But astronomer-ori- products using instrument calibration stripes. The vast majority of science data ented cookbooks are really needed, prefer- pipelines requires an electro-mechanically obtained during this period came from ably written by experienced astronomers stable instrument (including detector), the instrument modes with pipeline support. The familiar with the pipeline recipes. Such a acquisition of sufficient calibration data, and most glaring exception is FORS2-MXU. guidebook exists for ISAAC. ESO is trying the implementation of proper software algo- Unfortunately, it was necessary in the last to find ways to facilitate the creation of such rithms, where proper is defined as an algo- two years to shift pipeline development cookbooks for other instruments, but unfor- rithm that does not degrade scientific useful- resources originally scheduled for FORS2- tunately no direct resources for this activity ness. How is ESO doing in these areas? MXU to VIMOS. exist right now. • Stability: at present, most of our instru- Recent pipeline recipes (VIMOS, ments are stable (FORS1, FORS2, UVES, Recipe development and availability GIRAFFE) were built partially or in whole GIRAFFE), some are semi-stable (ISAAC, At the core of each instrument calibration using the Common Pipeline Library (CPL) NACO) and one is (mechanically) unstable pipeline are a set of instrument-specific soft- developed by ESO. All future recipes will be (VIMOS). Despite various mechanical prob- ware data processing engines known as based on CPL (http://www.eso.org/cpl). This lems, high quality VIMOS science products pipeline recipes. For FORS1, FORS2, has two benefits for end-user astronomers. are often produced in automatic mode if suf- VIMOS, GIRAFFE, and MIDI, the instru- First, all CPL-based recipes will have com- ficient night-time calibrations are available. ment consortia that built and commissioned mon Unix shell behavior, creating a more • Calibration data: calibration plans those instruments developed the original unified and stable user interface. Such exist for all instruments and calibration data pipeline recipes. Since their delivery, these behavior allows rapid development of user- are obtained on a regular basis. These data recipes have been maintained by ESO and specific processing scripts using the Unix are used to monitor instrument health and modified to meet ESO operational goals. shell or such common scripting languages as process science data. The target photometric Recipes for the other currently operational Perl or Python. The tools dfits and fitsort calibration accuracy is 5–10% in magnitude instruments (VIMOS-IFU, ISAAC, UVES, (see http://archive.eso.org/saft) can be used to for imaging and in the relative flux calibra- and NACO) were developed within ESO extract and organize FITS header informa- tion for spectroscopy. The achieved photo- where they are still actively maintained and tion within such scripts. Second, CPL-based metric accuracy is often much better in both extended as needed. recipes can be executed within the Gasgano cases. Nevertheless, the achieved accuracy Recipes are released to the community data organization and management tool may not be sufficient for all science pro- as soon as possible after instrument opera- developed by ESO (available from grammes. tions begin, but not before their performance http://www.eso.org/gasgano/). Gasgano pro- • Proper algorithms: for imaging, has been verified using data from the as- vides a user-friendly Java-based environ- essentially no problems exist today. For built, as-operated instrument in all standard ment for data organization and processing spectroscopy, the situation is more complex. configurations. In the past, this verification within a single graphical user interface. While UVES pipeline results are universally process has taken as long as a year but has At ESO, the pipeline recipes are embed- proclaimed as science-ready for most users, recently accelerated. Recipe upgrades are ded in an infrastructure that automatically this is not necessarily true for other instru- released as needed. For information about classifies raw data frames, associates them ments. how to download available recipes, see with appropriate master calibration prod- Instrument usage for science observa- http://www.eso.org/pipelines/. ucts, and then executes the appropriate Figure 2: Science integration time per