Transcript A COURSE ON ASTRONOMY AND ASTROPHYSICS, IUCAA MODULE 9 ASTRONOMY FROM ARCHIVAL DATA Chapter 9.2 Astronomy from archival data - II Yogesh Wadadekar, NCRA-TIFR [00:00:10] The tools that we have looked at so far: TopCat, Aladin etc, were developed as part of a world-wide effort known as the Virtual Observatory. Let's spend a few minutes looking at what the Virtual Observatory is all about. Section 9.2.1 The Virtual Observatory [00:00:32] The Virtual Observatory had a vision that astronomical data sets and other resources related to astronomy should work as a seamless whole. Many different projects and many different organisations and data centres across the world are working towards this goal. They have come together and formed the International Virtual Observatory Alliance, which debates and agrees on the technical standards that are needed to make a working Virtual Observatory. [00:01:11] So the main contribution of the International Virtual Observatory Alliance is in the development of the standards and protocols that will ensure that astronomical data as taken with different telescopes on space and on the ground are completely interoperable. That means you can share data from one telescope annal.. with analyse it with any software and then share it with some other data obtained at some other telescope. [00:01:48] The organisation also acts as a focus group for VO aspirations and provides a framework for discussing and sharing ideas and technology. And is a body for promoting and publicising the Virtual Observatory. It's done a lot of good work over the last decade or decade and a half. And the tools that we have looked at: TOPCAT, Aladin etc. were all developed by the VO community. [00:02:23] India is one of the pioneers in this effort. And has developed some other tools like VO plot, which is a VO compatible plotting tool which are used by many applications Transcript within the VO community. In fact tools like Aladin and TOPCAT use the VO plot tool as a plotting tool for their application. There are a number of VO tools that have been developed for different applications. There is a full list available at the URL that is listed at the bottom of this page. There are some important concepts that are related to the VO which anyone who wants to use archival data for research should be aware of. So, I'm just going to ta lk about them very briefly. [00:03:24] The first is the FITS data format, which we've already encountered before. It stands for the flexible image transport system. It is the standard format for astronomical images and tables. It provides a standard way of expressing metadata, which means descriptions of the data contained in the image or table, as a pair of key word and its value. And specifies a small number of mandatory keywords along with a somewhat larger number of reserved keywords, which can't be used anywhere else, which have to have a particular meaning if a file uses them. [00:04:10] The second major innovation that was brought in by the VO was the concept of the Astronomy Data Query Language or ADQL for short. What is this? It's a standard language for querying any astronomical database. And it is very closely aligned to SQL or the Standard Query Language. Standardization of this kind was necessary because the popular commercial and open source variants of of SQL all differ slightly. And the Virt ual Observatory wanted a standard way of specifying a region which SQL does not provide. [00:04:59] So the idea of a region is basically a small part of the sky, which is a very it's it's concept which comes entirely from astronomy. And not unsurprisingly SQL, which is a general-purpose database processing language does not is not able to handle this concept. So what was done was that standard SQL was taken and it was enhanced with some astronomy specific capabilities. And when that was done that what emerged from that process was the ADQL language, which is used for querying any kind of astronomical database. [00:05:43] The most basic thing that one can do with a search is what is called as a Cone Search. It's basically you want to know: in your survey or in your catalog or in the image that you're providing, what are the sources that are located within so much distance so much angular distance of a certain position. So I might possess specify a position on the sky with its right ascension and declination and ask the Virtual Observatory service: give me a list of all sources that you have in your database that are within one degree of the position that I have specified. [00:06:29] Viewed in 3D, of course this radius defines a cone of space. And you get in a Cone Search a fixed set of columns back which are set by the data service in question. So if I query a data service, which is a radio source catalogue, it might return to me the positions Transcript of all the radio sources in the part of the sky that I am interested in, along with some information about those radio sources like: how bright they are? How large they are? And are they seen at other frequencies? What is their red shift and so on? [00:07:10] A related concept after a Cone Search is the concept of a Simple Image Access. Okay? This is specified in a protocol known as SIAP the Simple Image Access Protocol. A SIAP service is an archive that returns astronomical images within a specified position and radius. So it goes beyond what a Cone Search does. Cone Search will only give you a list, a table of astronomical sources within that part of the sky. A SIAP service will return to you an entire image in that part of the sky. [00:07:48] Similar to SIAP service is the SSAP service Simple Spectrum Access Protocol which instead of returning an image will return spectra for all objects that are located in that part of the sky. [00:08:07] In addition to understanding FITS for as a format for tables, the Virtual Observatory developed its own format for storing tabular data. They call it VOTable. It's the standard format for storing and exchanging tabular data within the Virtual Observatory. Most of the data archives now offer export of data in VOTable format. And a large number of tools read VOTables. When we looked at TOPCAT, we saw that it could read FITS tables, but it can just as happily read VOTables. Any FITS table can be expressed as a VOTable. So inter-conversion is possible. For example using TOPCAT, you could load a FITS file in TOPCAT and then write it out as a VOTable. [00:09:06] In order to improve access to astronomical catalogue data or tables, the VO developed a new protocol which they called TAP: Table Access Protocol. And what TAP does: it provides you query driven access to astronomical tables and databases. For example, when you do a simple Cone Search, you can only search by sky position. And it returns a fixed set of columns, which is specified by the data service that you are accessing. [00:09:38] On the other hand TAP service allows you to make searches along the lines of: give me all the records with B-V greater than 2.0. This refers to the colours of stars or galaxies. And give me just columns B, D, F and G. So you can choose what columns you want. And you can put conditions on the kind of sources that you are looking for. This kind of search can become arbitrarily complex. You can say that okay, I want to you to show me only quasars between redshifts 5 and 6 in this part of the sky. Or you can say I want you to show me a list of white dwarfs that are brighter than a certain magnitude in this part of the sky and so on. Transcript [00:10:30] Queries that use the TAP service will need to be formulated in the standard data query, astronomy data query language ADQL. But often the tool that you are using will automatically construct this for you. So there are now some advanced tools that allow you to do TAP queries without actually knowing the details of the ADQL. Section 9.2.2 Major astronomical archives [00:11:04] Let us now move on and look at some of the major astronomical archives that exist. Large archives are found at all wavelengths and there are a very large number of them. I found it really difficult to short list a few to show you because if I had simply listed all of them, that would not have been very useful. So I have chosen a few archives at optical, radio and x-ray wavelengths, which I have listed here. I feel that these are archives that are useful to a large community. Of course, this will be my biased view and I apologise to people if I have left your favourite archive out of this list. So let's look at some of the major archives in optical astronomy. [00:12:03] So as you know, the Hubble Space Telescope has been one of the major astronomy facilities over the last two and a half decades or so.