Satellite and Radiosonde Measurements of Atmospheric Humidity of Measurements Radiosonde Atmospheric and Satellite Kottayil Ajil

ISSN: 1402-1757 ISBN 978-91-7439-XXX-X Se i listan och fyll i siffror där kryssen är

LICENTIATE T H E SI S

Department of Computer Science, Electrical and Space Engineering Division of Space Technology Ajil Kottayil Satellite and Radiosonde Measurements of Atmospheric Humidity of Measurements Radiosonde Atmospheric and Satellite Kottayil Ajil

ISSN: 1402-1757 ISBN 978-91-7439-380-4 Satellite and Radiosonde Measurements

Luleå University of Technology 2011 of Atmospheric Humidity

Ajil Kottayil

Licentiate thesis

Satellite and Radiosonde Measurements of Atmospheric Humidity

Ajil Kottayil

13 December 2011

Dept. of Computer Science, Electrical and Space Engineering Division of Space Technology Graduate School of Space Technology Lule˚aUniversity of Technology Rymdcampus 1 98128, Kiruna, Sweden Printed by Universitetstryckeriet, Luleå 2011

ISSN: 1402-1757 ISBN 978-91-7439-380-4 Luleå 2011 www.ltu.se Abstract

This licentiate thesis is based on two papers which are related to the study of atmospheric humidity. The first paper mainly focuses on a non linear method for retrieving atmospheric humidity from infrared sounder satellite measurements based on data clustering and fuzzy logic which could potentially improve the retrieval accuracy. The main aim of this study was to provide a better first guess humidity profile for physical retrieval algorithms which can further improve retrieval accuracy. This method has been compared against linear and non linear regression retrievals which are the generally used methods to get the first guess profile. The results reveal that the retrieval accuracy is better for the new method as compared to the conventional methods. Generally, the accuracy of the humidity measurements of radiosonde is poor in the upper troposphere (UT) and is worse for daytime measurements due to solar heating of the humidity sensor. Several methods have been developed to correct the humidity measurements of radiosondes in the UT. The second paper presents a detailed analysis of the implications of these corrections and depicts how important they are for satellite validation. The corrections have been applied separately for daytime and nighttime radiosonde measurements and their effects have been quantified by comparing against the coinciding satellite measurements in the infrared and microwave spectral range used for humidity measurements.

iii iv Appended papers

Paper I A. Kottayil, P. K. Thapliyal, M. V. Shukla, P. K. Pal, P. C. Joshi, and R. R. Navalgund. A new technique for temperature and humidity proﬁle retrieval from infrared sounder observations using adaptive neuro-fuzzy inference system. IEEE Geosci. R. S., 48:1650 –1659, 2010. doi: 10.1109/TGRS.2009.2037314

Paper II A. Kottayil, S. A. Buehler, V. O. John, L. M. Miloshevich, M. Milz, and G. Holl. On the importance of Vaisala RS92 radiosonde humidity corrections for a better agreement between measured and modeled satellite radiances. J. Atmos. Oceanic Technol., 2011. doi: 10.1175/JTECH-D-11-00080.1. in press

v vi Related papers

• S. A. Buehler, V. O. John, A. Kottayil, M. Milz, and P. Eriksson. Eﬃcient radiative transfer simulations for a broadband infrared radiometer — combining a weighted mean of representative frequencies approach with frequency selection by simulated annealing. J. Quant. Spectrosc. Radiat. Transfer, 111(4):602–615, Mar. 2010. doi: 10.1016/j.jqsrt.2009.10.018

• P. K. Thapliyal, M. V. Shukla, S. Shah, P. K. Pal, P. C. Joshi, and A. Kottayil. An algorithm for the estimation of upper tropospheric humidity from kalpana observations: Methodology and validation. J. Geophys. Res., 116:1–16, 2011. doi: 10.1029/2010JD014291

• G. Holl, S. A. Buehler, J. Mendrok, and A. Kottayil. Simulating cloudy thermal infrared radiances with an optimised frequency grid in the radiative transfer model ARTS. J. Quant. Spectrosc. Radiat. Transfer, to be submitted

vii viii Contents

Abstract iii Appended papers v Related papers vii Table of contents ix Acknowledgements xi Chapter 1–Introduction 1 1.1Motivation...... 1 1.2 Observations of Humidity ...... 2 1.3LayoutoftheThesis...... 2 Chapter 2–Radiative Transfer 5 2.1 Clear Sky Radiative Transfer Equation ...... 5 2.2ARTS...... 7 Chapter 3–Humidity Retrieval Method 9 3.1FuzzyLogic...... 9 3.2DataClusteringMethods...... 12 3.3 ANFIS ...... 13 Chapter 4–Radiosonde Corrections 17 4.1RadiosondeCorrectionMethods...... 17 4.1.1 Mean Calibration Bias Correction ...... 18 4.1.2SolarRadiationErrorCorrection...... 18 4.1.3TimeLagCorrection...... 19 Chapter 5–Diurnal cycle of humidity 23 5.1Data...... 23 5.2ConstructionofDiurnalcycle...... 24 5.3Results...... 25 5.4FutureWork...... 26

ix Chapter 6–Summary of Papers 31 6.1PaperI...... 31 6.2PaperII...... 32 References 33 Paper I 39 Paper II 51

x Acknowledgements

I would like to extend my gratitude to my supervisor Stefan Buehler for giving me an opportunity to work in his group. Your valuable scientiﬁc inputs and suggestions have helped me in making better interpretations and careful introspection of my results. The scientiﬁc help and discussions that I have had with assistant supervisor Viju John were fruitful in carrying forward my work with more ease, thanks to you, Viju. Working with Larry Miloshevich was a good experience and I am grateful for your openness and productive discussions. I am grateful to all my present and former fellow members of the SAT group for their help. Thank you, Oliver Lemke, Gerrit Holl, Salomon Eliasson, Jana Mendrok, Thomas Kuhn, Mathias Milz and Isaac Moradi. I would also like to thank the Graduate School of Space Technology for their academic support.

xi xii Chapter 1 Introduction

1.1 Motivation

The increasing awareness of a changing climate and the role played in it by water vapour has elevated the interest of water vapour related studies in atmospheric science. Water vapour is the most important contributor to the natural greenhouse effect. The saturation vapour pressure is a strong function of temperature being governed by the well known Clausius-Claypeyron relationship. As the Earth’s atmosphere gets warmer due to enhanced emission of CO2, its water vapour concentration increases exponentially with temperature. Since water vapour is a strong infrared absorber, an increased amount of water vapour therefore absorbs more radiation re- sulting in an even warmer atmosphere. This positive feedback of water vapour is estimated to amplify the initial radiative forcing by about a factor of two as revealed from model simulations [Manabe and Wetherald, 1967, Held and Soden, 2000]. The global warming is now a phenomenon existing beyond any doubt and the increasing observed temperatures are an added proof. Parallel to this observation is the knowledge of an increasing trend in the tropospheric water vapour over the years as established by many studies [Trenberth et al., 2005, Wentz et al., 2007, Durre et al., 2009, Soden et al., 2005]. The monitoring of upper tropospheric water vapour in a scenario of changing climate plays a central role in the prediction of future climate. This is largely because of its sensitivity to outgoing long wave radiation which has a nearly negative logarithmic relation to specific humidity [Pierrehumbert et al., 2006]. Therefore, relatively small fluctuations in amount water vapour in the free troposphere will have a great influence on the radiation budget [Kiehl and Briegleb, 1992, Held and Soden, 2000]. However, a better picture of water vapour distribution, its trends over time and how various atmospheric processes are affected by it, are yet to be gained from scientific studies [Sherwood et al., 2010]. A direct step towards this would be to make available the water vapour concentration from all parts of the atmosphere and from all regions.

1 2 Introduction

1.2 Observations of Humidity

Although observations of humidity can be obtained from radiosondes, their data quality in upper troposphere is known to be poor [Elliott and Gaffen, 1991, Miloshevich et al., 2001, 2009]. In addition, their lack of global coverage, sparse data and utility confined to over land areas alone have posed a serious limitation in obtaining a wholesome picture on water vapour coverage. The limitations of the radiosondes are being overcome to a large extent with the use of satellite data which have a global coverage and a better quality. Satellite measurements of humidity trace their history to the very first meteorological satellites, the Television Infrared Observation Satellite (TIROS) series launched in the early 1960s. The wide use of remotely sensed water vapour measurements however was from High Resolution Infrared Sounder (HIRS) sensor onboard National Oceanic and Atmospheric Administration (NOAA) polar orbiting satellites launched in 1979. Data measuring capability has greatly been improved upon since then with the very recent version of HIRS/4 deployed in NOAA-19. The original HIRS had been ideal for tracking global changes in upper tropospheric humidity (UTH) though cloud cover tended to introduce a dry bias in observation [Soden and Bretherton, 1993, Lanzante and Gahrs, 2000, John et al., 2011a]. Cloud cover continued to limit the performance of Atmospheric Infrared Sounder (AIRS) and Infrared Atmospheric Sounding Interferometer (IASI), though these instruments had hundreds of channels with water vapour absorption bands and vertical resolving power of a few kilometers [Fetzer et al., 2008]. Geostationary satellites like European Space Agency’s (ESA) METEOSAT and India’s Kalpana Very High Resolution Radiometer (VHRR) also provides water vapour as one of their observations [Schmetz et al., 2002, Thapliyal et al., 2011]. Microwave based imagers are another preferred alternative on account of their lower sensitivity to clouds and notable among these are SSMI (Special Sensor Mi- crowave Imager) operating since 1988 and AMSU-B (Advanced Microwave Sounding Unit) sensor on NOAA satellites and these like HIRS have proven to be useful in providing water vapour climatology [Buehler et al., 2008]. The UTH retrieval meth- dology from microwave measurements and the advantage of microwave measurements over infared measurements have been demonstrated in various studies [Buehler and John, 2005, Buehler et al., 2008, John et al., 2011a]. The satellite data used in the present research belongs to microwave measurements from AMSU-B and Microwave Humidity Sounder (MHS) sensor onboard NOAA satellites. They are both five channel radiometers (Channels 16-20) which senses radiation from different levels of atmosphere to aid in getting humidity profiles on global scale. The channels 18-20 are placed proximal to strong water vapour absorption line at 183 GHz. Channels 16 and 17, at 89 GHz and 150 GHz, respectively sees the surface. More details on instrument scanning can be found at http: //www.ncdc.noaa.gov/oa/pod-guide/ncdc/docs/klm/index.htm. 1.3. Layout of the Thesis 3

1.3 Layout of the Thesis

The present thesis is a compilation of water vapour based studies which employ methods capable of improving the humidity measurements from radiosondes especially in the upper troposphere. It also includes a non linear method which can improve the accuracy of humidity retrieval from satellite infrared sounder measurments. An outline of the thesis is as follows. Chapter 2 brieﬂy discusses the clear sky radiative transfer equation and the radiative transfer model used in the study. One of the two papers (Paper I) presented in this thesis deals with the introduction of a retrieval method which is based on clustering and fuzzy logic for the retrieval of humidity from satellite observations. The background knowledge for a better understanding of the retrieval method is presented in Chapter 3. The second paper introduces various correction methods which can improve the quality of humidity measurements from radiosondes the details of which are presented in Chapter 4. A glimpse of an ongoing work which focuses on the diurnal cycle of humidity and cloud can be obtained in Chapter 5 along with some initial results. 4 Introduction Chapter 2

Radiative Transfer

Radiative transfer modeling has become an indispensable tool in various scientiﬁc domains as it can simulate the interaction of electromagnetic radiation with objects in a medium. Radiative transfer models use the radiative transfer equations to simulate the radiative processes of medium for a given wavelength and for a known set of atmospheric and surface parameters. The present chapter is motivated by the fact that the radiative transfer simulations in the thesis have been performed for clear sky conditions and the most used radiative transfer model for such studies is ARTS. In this chapter, the physical approximations of the clear sky radiative transfer equation are laid out and a brief description of the radiative transfer model ARTS is presented.

2.1 Clear Sky Radiative Transfer Equation

For a clear sky radiative transfer calculation, only absorption and emission of the medium are considered, avoiding the scattering term which usually occurs in the presence of clouds for infrared and microwave wavelengths. Consider radiation of intensity Iλ at wavelength λ travelling through an absorbing and emitting atmosphere. The change in its intensity due to absorption after travelling through a small layer dz is −kλIλρdz,whereρ is the density (mass per unit volume) of the medium, kλ is the mass absorption coeﬃcient (area per unit mass). According to Kirchoﬀ’s law, a selective absorber at any wavelength λ is also a selective emitter of radiation at the same wavelength, therefore the intensity emitted in the direction of propagation is kλBλ(T)ρdz. The net change in the intensity of radiation after travelling through a layer dz is given [Peixoto and Oort, 1992, Rees, 2001],

dIλ = −kλIλρdz + kλBλ(T)ρdz (2.1)

5 6 Radiative Transfer

−2 −1 −1 where Bλ(T) is the black body monochromatic radiance (Wm m sr ) speciﬁed by Planck’s law which is given by

2 2hc Bλ(T) = hc (2.2) λ5 e λkT − 1 where h =6.63 × 10−34 Js, the Planck constant, c =2.99 × 108ms−1, the speed of light, and k =1.38 × 1023 J K−1, the Boltzmann’s constant. Equation 2.1 is called Schwarzchild’s equation and is the basic equation for the clear sky radiative transfer. It shows that intensity of the radiation at any point in the atmosphere can be determined provided the distribution of absorbing mass and absorption coeﬃcients are known. Suppose the radiation emitted from the layer dz has to reach another level z1, but the radiation emitted from this layer will be partially absorbed before it reaches the level z1, so that the transmitted radiation is Tλ(z)Bλ(T)kλρdz (Figure 2.1). The quantity Tλ(z) is the transmittivity between the levels z and z1 deﬁned as [Peixoto and Oort, 1992], z1 Tλ(z)=exp(− kλρdz ) (2.3) z z1 The term z kλρdz is called optical depth or optical thickness. Since,

dTλ(z) = Tλ(z)kλρ (2.4) dz Therefore Equation 2.1 can be written as,

Tλ(z)dIλ = −(Iλ − Bλ(T))dTλ(z) (2.5)

d(IλTλ(z)) = Bλ(T)dTλ (2.6) Integrating Equation 2.6 from 0 to ∞

∞ dTλ (z) Iλ(∞)=Iλ(0)Tλ(0) + Bλ(T) dz (2.7) 0 dz Zero can be considered as the position of the Earth’s surface from where the radiation originates and inﬁnity (∞) indicates the atmospheric level where the intensity of the radiation is to be determined. Equation 2.7 is the expression for clear sky radiative transfer in the atmosphere. The ﬁrst term in Equation 2.7 is the spectral radiance emitted by the surface and attenuated by the atmosphere and the second term is the dTλ(z) spectral radiance emitted by the atmosphere. The term dz is called the weighting function and it gives an indication of where in the atmosphere the majority of the radiation for a given spectral band comes from. The atmospheric contribution is the weighted sum of Planck radiance from each layer in the atmosphere where the weights are provided by the weighting function. 2.2. ARTS 7

The first step towards solving the radiative transfer equation is the calculation of the mass absorption coefficient kλ. If there are n absorbing molecules in the medium then the transmittance can be written as [Rodgers, 2000], z T (λ, z )=exp − ki (λ, z ) ρi (λ, z ) dz (2.8) z i where i refers to the ith absorber and ρi is its density. In the case of modelling radiances in the infrared spectrum, where the molecular vibrational rotational bands are important, the absorption coefficients should be summed over large number of spectral lines. ki (λ, z )= kij [T (z )] fij [λ, p(z ),T(z )] (2.9) j where kij is the strength of jth line of the ith absorber and fij is its normalised shape. Any radiative transfer model which models radiances using the above procedure is called line by line radiative transfer model.

2.2 ARTS

Atmospheric Radiative Transfer Simulator (ARTS) is a line by line radiative transfer model which can simulate radiances for the spectral region spanning from infrared to microwave [Buehler et al., 2005a, Eriksson et al., 2011]. ARTS solves the radiative transfer equation for ID, 2D or 3D atmospheric geometries. The current simulations have been performed with ARTS for one-dimensional geometry where the atmospheric variables (temperature and gaseous concentrations) are allowed to vary only in the vertical direction. ARTS contains an inbuilt set up to simulate radiances as detected by radiometers such as HIRS and AMSU onboard NOAA satellites for different viewing angles of the instrument. For modelling the radiances in the infrared, ARTS employs a line by line calculation for computing the absorption coefficient. However, modelling the radiances using this method is computationally expensive; therefore, an absorption look-up-table is provided within ARTS to expedite the calculation [Buehler et al., 2011]. Further, for microwave simulations, PWR98 [Rosenkranz, 1998]acom- plete water vapour absorption model is provided within the ARTS. ARTS can also simulate the Jacobians for temperature and trace gas concentrations, which is generally defined as the partial derivative of radiance with respect to atmospheric parameters influencing it. The straight forward way to evaluate this quantity is the use of perturbation method where radiances are simulated for a reference state vector and re-determined successively through small perturbations in each element of the state vector. This procedure is time consuming due to which ARTS calculates this analytically [Buehler et al., 2005a]. For example, the humidity Jaco- bians simulated by ARTS for three different channels of AMSU-B is shown in Figure 2.2 8 Radiative Transfer

Figure 2.1: Transmission of radiation through atmosphere.

14 14 14 18 19 20 12 12 12

10 10 10

8 8 8

6 6 6 Altitude [ km ] 4 4 4

2 2 2 0 0 0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 -0.8 -0.6 -0.4 -0.2 0.0 0.2 -0.8 -0.6 -0.4 -0.2 0.0 0.2 Jacobian [ K / 1 ] Jacobian [ K / 1 ] Jacobian [ K / 1 ]

14 14 14 18 19 20 12 12 12

10 10 10

8 8 8

6 6 6 Altitude [ km ] 4 4 4

2 2 2 0 0 0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 -0.8 -0.6 -0.4 -0.2 0.0 0.2 -0.8 -0.6 -0.4 -0.2 0.0 0.2 Jacobian [ K / 1 ] Jacobian [ K / 1 ] Jacobian [ K / 1 ]

Figure 2.2: Water vapour Jacobians for AMSU-B channels 18,19 and 20. The upper and lower panels represent moist tropical atmosphere and dry mid latitude winter atmosphere, respectively. Simulations for nadir and oﬀ nadir views of the instrument are indicated as solids and dotted lines, respectively. Source: John [2005] Chapter 3 Humidity Retrieval Method

The measurements from space platforms are ideal in gaining a better understanding of global distribution of atmospheric humidity. The meteorological satellites capable of measuring atmospheric humidity do so by sensing the radiation emitted from diﬀerent parts of the Earth’s atmosphere. Satellite measurements, therefore do not provide direct measurements of humidity. Rather, they provide a radiance intensity which is a function of atmospheric constituents. Retrieval methods aid in getting the required information from the radiance measured by satellites. This chapter discusses a nonlinear method for retrieval of humidity which forms the basis of the paper I enclosed in the thesis. This method is based on data clustering and fuzzy logic. Although the paper demonstrates in detail the merits of the retrieval method over conventional methods in terms of accuracy, the very basic fundamental information have not been covered. The present chapter aims to furnish certain background information on retrieval methods which could help in better comprehension of the perspective of the paper. The detailed mathematical formulations however have been exempted.

3.1 Fuzzy Logic

Fuzzy logic is a problem-solving enabled mathematical concept, inspired from real life situations [Zadeh, 1965]. The real situations in human interactions are not centred wholly on an all or nothing alternative but somewhere between the both. While a computer is programmed to solve a problem through a binary approach of zeroes and ones where it has to take a yes or no decision, the fuzzy logic technique employs fuzzy logic variables that can be assigned a value between 0 and 1. The added advantage is that it can deal with non mathematical, linguistic variables such as which dominate human thought processes and communications. These linguistic variables are defined through IF-THEN statements and a host of other operators. A fuzzy set is defined as a set without a crisp, clearly defined boundary and is an extension of classical sets. If X is a universal set and its elements are denoted by x,

9 10 Humidity Retrieval Method then fuzzy set A in X is deﬁned as the set of ordered pair

A = {x, μA (x) |x ⊂ X} (3.1) where μA (x) is called membership function which maps universal set X to the real interval [0 1]. The closer μA (x) is to 1 the more is the chance of x belonging to A. μA (x) gives the degree of membership to x in A. The set-theoretic operations of union, intersection and complement for fuzzy sets are deﬁned through membership functions. Let A and B denote the pair of fuzzy set in X with membership functions μA (x)and μB (x) respectively. The membership function of union and the membership function of intersection are deﬁned as [Kaufmann, 1975]:

μA∪B = max (μA (x) ,μB (x)) (3.2)

μA∩B = min (μA (x) ,μB (x)) (3.3)

Complement of fuzzy set A is defined as μA¯ (x)=1− μA (x) The retrieval using fuzzy logic is enabled by formulating conditional statements using IF and THEN statements. Linguistic rules that describe a system consists of two parts; a premise part (between the IF and THEN) and a consequent block (following THEN). The approach to a problem using fuzzy logic is facilitated through fuzzy inference systems (FIS). The fuzzy reasoning mechanisms adopted in fuzzy inference systems are as follows: The first step is the fuzzification of the input which is a process of assigning membership value to each input using an appropriate membership function. In the present context input refers to the satellite sounder measurements. Fuzzification is the process of classifying the input into different value ranges, or more precisely, it is a process of assigning a linguistic label (small, medium and large) to the input as shown in Figure 3.1. Linguist variables, as the name implies are non- numerical variables that are used to define the rules in fuzzy logic. For example, the height of a person is a linguistic variable which can assume the values such as ’tall’ or ’short’. The ’tall’ or ’short’ attribute is referred to as linguistic label. The second step is to combine different rules generated by fuzzy rule base to get the firing strength of each rule. The concept of firing strength is explained here with thehelpofFigure3.2.TheFigure3.2 shows A1, A2, B1andB2 as linguistic labels assigned to the inputs which when combined using appropriate fuzzy operators (AND or OR) produces weights (w1 or w2) or firing strength. The third step is to make the consequent of each rule depending on the firing strength. The different steps of fuzzy reasoning are shown in Figure 3.2. The FIS uses different types of fuzzy reasonings and the one used in the paper has been proposed by Takagi and Sugeno [1985] where the output of each rule is a linear combination of input variables plus a constant term, and the final output is the weighted average of each rule’s output. This is depicted in the type 3 of the consequent part of the Figure 3.2. Hereafter this FIS architecture will be referred to as TS model. For example, a fuzzy rule using TS model can be written as, IF x is A1 and y is B1 THEN f1 = p1x1 + q1x1 + r1,whereA1 and B1 are linguistic labels. 3.1. Fuzzy Logic 11

Figure 3.1: Classiﬁcation of input into diﬀerent value ranges and assigning of membership values.

premise part consequent part

type 1 type 2 type 3

A1 B1 C1 C1

w1 w1 z1=ax+by+c

X Y z1 Z Z

A2 B2 C2 C2

w2 w2 z2=px+qy+r

X Y z2 Z Z xy weighted weighted average max average multiplication (or min)

w1*z1+w2*z2 w1*z1+w2*z2 z = z = w1+w2 w1+w2

z (centroid of area)

Figure 3.2: Various steps of Fuzzy reasoning mechanism adopted in FIS. Source: Shing and Jang [1993] 12 Humidity Retrieval Method

3.2 Data Clustering Methods

Data clustering technique is essentially a numerical method of classification, and as its name indicates, seeks application where a large group of data needs to be partitioned into certain nearly homogeneous classes. It is a common statistical tool finding use in almost all branches of science. Therefore, though the ultimate task is to have a classified data, the methods employed to attain it could be different depending on the nature of the dataset. There are different types of clusterings and depending on the application, clusters can vary. Cluster analysis includes a wide range of statistical techniques with the most common approaches to finding clusters being distance based, centroid based,or density based [S.Aldenderfer and Blashfield]. The main idea behind clustering methods is to find out a cluster center, which is an indicator of where the heart of each cluster is located. Most popular algorithms in use determine clusters in the dataset by minimizing a cost function of Euclidean distance. Among these, one of the widely used algorithms is hard-c means algorithm (HCM) [Mcqueen, 1967, Hartigan and Wong, 1979]. The HCM algorithm tries to minimize the cost function of the form:

c n 2 J = ||Xk − ci|| (3.4) i=1 k=1 where c is the total number of clusters and the cost function is based on Euclidean distance between a vector Xk and the corresponding cluster centre ci. The clusters formed as a result of classification are defined by a binary characteristic matrix, M also called the membership matrix Mik where Mik is 1 if kth data point Xk belongs to cluster i and 0 otherwise. The fuzzified c-means (FCM) is another popular clustering algorithm which clas- sifies data into clusters in such a way that each data point is not bound exclusively to any particular cluster but can associate to all the clusters specified by a membership grade [Jang et al., 1997]. This characteristic of assigning membership values distinguishes FCM from HCM. Fuzzy partitioning is achieved in FCM by allowing the membership to take all values between 0 and 1. The objective function which is to be minimized is the generalization of Equation 3.4.

c n 2 J = Mik||Xk − ci|| (3.5) i=1 k=1 where Mik is the membership matrix which can take the values between 0 and 1. The FCM and HCM algorithms start by randomly assigning a guess for the initial cluster centre. The cluster centres and membership grades are updated in successive iterations. Since the number of clusters is speciﬁed initially for both FCM and HCM, they come under the category of supervised algorithms. Conversely, if the number of clusters are not known, then unsupervised algorithms are employed. Subtractive 3.3. ANFIS 13 clustering belongs to the category of unsupervised algorithm and is based on the density of data points in the feature space [Chiu, 1994]. In subtractive clustering each data point is the possible candidate for the cluster centre. A density measure at each data point is deﬁned as

n 2 ||xi − xj|| Di = exp (3.6) ra 2 j=1 2 where ra is a positive constant representing neighbourhood radius. A data point will have a high density value if it has many neighboring data points. The ﬁrst cluster centre Xc1 is chosen as the point having maximum density value Dc1 . In the next step, the density value of each data point will be revised as follows:

2 ||xi − xc1 || Di = Di − Dc1 exp (3.7) rb 2 2 This process will exclude the selected cluster centre as the candidate for the next cluster, as the first cluster centre, Xc1 will have significantly reduced density measure. After revising the density function, the next cluster centre is selected as the point having the greatest density. Sufficient number of clusters can be obtained by contin- uing this process. The clustering method which is used in the paper is subtractive clustering and using this method the dataset is clustered prior to application of fuzzy rules as explained in the previous section.

3.3 ANFIS

ANFIS stands for adaptive neuron-fuzzy inference systems. In ANFIS an adaptive network has been included to fine tune the fuzzy rule base generated by TS model Shing and Jang [1993]. An adaptive network, like a neural network has nodes and connecting links through which nodes are connected. The nodes are called adaptive, which means that the output of these nodes depend on the parameter pertaining to this node, and the rules are learned by changing these parameters. ANFIS has five layers and each has its own dedicated function. In ANFIS the parameters associated with first and last node have been changed during its training phase. The first layer has a node function which is either bell shaped,

1 μA (x)= (3.8) i bi 1+ x−ci ai or Gaussian shaped − 2 − x ci μAi (x)=exp (3.9) ai 14 Humidity Retrieval Method

where x is the input to the node i, and Ai is the linguistic label (small, large, medium etc) associated with this node function. The parameter set {ai,bi,ci} ({ci,ai})contains the premise parameters and as the values of these parameters change, it changes the shape of the membership function. The output node has the parameter set {pi,qi,ri} and the output can be written as the linear combination of these parameters. These parameters are called consequent parameters. In ANFIS a hybrid algorithm is used to change the consequent and premise parameters using a set of training dataset. The premise parameters are changed using gradient descent method which tries to minimize an error function of the form:

E = Ep (3.10) p where Ep is the error in the pth training datset which is the diﬀerence in original and ANFIS output and is a function of premise and consequent parameters. For example if α is the parameter of the input node (premise parameter) then the update formula for α is ∂E Δα = −η (3.11) ∂α where η is the learning rate. This method can also be applied to update the consequent parameters but the method is generally slow and likely to be trapped in local minima. Therefore in ANFIS a least square method is applied to change the consequent parameters. As mentioned earlier the output of the ANFIS is a linear combination of consequent parameters which makes it possible to make an equation of the form AX=B using N training dataset for given values of premise parameters. X is the unknown vector whose elements are consequent parameters. A least square method can be used to ﬁnd X by minimizing the squared error ||AX − B||2. 3.3. ANFIS 15

AB1 1 w1 f 1 = p1 x +q1 y +r1 w f + w f X Y f= 1 1 2 2 2 2 AB w1 + w2 = w2 f 2 = p2 x +q2 y +r2 w1 f 1 + w2 f 2 X Y x y (a)

layer 1 layer 4 layer 2 layer 3

A1 x y layer 5

w1 w1 x w f 1 A2 1 f

y B1 w2 f 2 w2 w2

B2 x y (b)

Figure 3.3: TS model fuzzy reasoning having two fuzzy rules (a) and the equivalent ANFIS architecture (b). Source: Shing and Jang [1993] 16 Humidity Retrieval Method Chapter 4 Radiosonde Corrections

This chapter presents an overview of various types of correction applied to the Vaisala RS92 radiosonde humidity measurements. Vaisala radiosondes are the most widely used radiosondes in the world. Though radiosondes are the vital sources of information on atmospheric humidity distribution, its data quality in the upper troposphere is known to be poor [Elliott and Gaﬀen, 1991, Moradi et al., 2010, Miloshevich et al., 2009, Soden and Lanzante, 1996]. Their data quality issues may arise from several factors. Incorrect calibration of the humidity sensors, error caused due to solar heating of the humidity sensors and a slow sensor response at low temperatures are some of the most plausible reasons. Several studies have highlighted these errors and have introduced various correction methods [V¨omel et al., 2007, Turner et al., 2003, Miloshevich et al., 2001], but none of the studies so far have actually depicted the importance of correction through comparison with satellite measurements which is the main topic of the second paper. The corrections applied to the radiosondes are based on the work by Miloshevich et al. [2009]. This chapter has been written with an aim of furnishing the reader with a general idea of the radiosonde correction methods which could not be covered in detail in the article.

4.1 Radiosonde Correction Methods

The basis of the correction is the removal of bias in the Vaisala RS92 radiosonde humidity measurements through a comparison with three reference instruments of known accuracy. These instruments are cryogenic frost-point hygrometer which is intended for correction above 700 hPa, a microwave radiometer for lower troposphere, and a system of 6 calibrated RH probes for the surface. Mean calibration biases in RS92 humidity measurements were determined by comparing the RS92 measurements with simultaneous measurements from these instruments and then these biases were removed. The correction is a function of pressure (P) and relative humidity (RH) and is given by

17 18 Radiosonde Corrections

Figure 4.1: The Vaisala RS92 mean bias curve. Source: Miloshevich et al. [2009]

RHcorr =G(P, RH) × RH (4.1) where G(P,RH) is the correction factor.

4.1.1 Mean Calibration Bias Correction

The mean calibration correction removes the bias error in the RS92 humidity measurements by comparing these measurements against simultaneous measurements from three known reference instruments of known accuracy. The correction factor in Equation 4.1 is inferred from pressure dependent curves determined for several RH intervals as shown in Figure 4.1. The dashed lines in the Figure represents the polynomial approximation for each curve and are the best approximations of RS92 mean bias relative to the reference instruments. The polynomial coefficients (F(P, RH)) are given for the mid-value of each RH interval. The correction factor in G(P, RH) is determined from pressure dependent curve using the polynomial coefficients as given in Figure 4.2. The coefficients are different for day and nighttime measurements. The mean calibration error can be determined exclusively from nighttime measurements (Figure 4.1(a)) where as Figure 4.1(b) for daytime measurements represents a combination of mean calibration error and the error due to solar radiation which would be quantified separately. The details as to how they are quantified are explained in the next section. A more accurate result for the mean calibration correction can be obtained if one uses temperature instead of pressure since the temperature dependence is more consistent with the actual sensor calibration than pressure. 4.1. Radiosonde Correction Methods 19

Coefficients RH or Fit a0 a1 a2 a3 a4 a5 a6 Nightb 1.5 5.1993e+1 À7.9576eÀ1 3.9051eÀ3 À8.9666eÀ6 1.1825eÀ8 À8.4134eÀ12 2.4210eÀ15 2.5 4.3729e+1 À7.8757eÀ1 3.8100eÀ3 À8.4919eÀ6 1.0830eÀ8 À7.5247eÀ12 2.1433eÀ15 3 1.0102e+1 À3.5020eÀ1 1.3771eÀ3 À1.8918eÀ6 1.5448eÀ9 À1.0460eÀ12 3.7543eÀ16 4 À1.2053e+1 À1.3963eÀ1 5.0608eÀ4 8.7142eÀ8 À1.1580eÀ9 9.6029eÀ13 À2.2738eÀ16 6 À1.9292e+1 À5.3081eÀ2 1.1776eÀ5 1.5888eÀ6 À3.7721eÀ9 3.2351eÀ12 À9.7876eÀ16 8.5 À1.4220e+1 À1.5629eÀ1 7.3102eÀ4 À5.7830eÀ7 À6.9512eÀ10 1.1583eÀ12 À4.3573eÀ16 12 À8.6609e+0 À2.3153eÀ1 1.1601eÀ3 À1.6559eÀ6 4.7114eÀ10 6.4842eÀ13 À3.7600eÀ16 20 À1.2075e+1 À9.0493eÀ2 4.5730eÀ4 À4.4334eÀ7 À2.5251eÀ10 5.6512eÀ13 À2.1830eÀ16 30 À8.4463e+0 À6.7739eÀ2 2.1850eÀ4 2.4128eÀ7 À1.1680eÀ9 1.1593eÀ12 À3.6948eÀ16 42 À7.5226e+0 À9.4287eÀ2 5.6012eÀ4 À1.0285eÀ6 8.1621eÀ10 À2.4513eÀ13 3.3189eÀ18 50 3.7854e+1 À4.9026eÀ1 2.0313eÀ3 À3.9299eÀ6 3.9439eÀ9 À1.9776eÀ12 3.8808eÀ16 for P < P2 4.3867e+3 À3.7335e+2 1.2676e+1 À1.9717eÀ1 1.1628eÀ3

Dayc 0 6.8793e+0 1.6275eÀ1 À3.2097eÀ5 À4.1883eÀ7 5.0829eÀ10 À1.9028eÀ13 1.9 À1.3058e+1 1.5405eÀ1 3.0599eÀ5 À4.9033eÀ7 5.4030eÀ10 À1.9315eÀ13 2.4 À4.7161e+1 1.3916eÀ1 1.3784eÀ4 À6.1264eÀ7 5.9504eÀ10 À1.9805eÀ13 3.5 À6.0069e+1 1.3320eÀ1 1.8078eÀ4 À6.6256eÀ7 6.1467eÀ10 À1.9661eÀ13 5 À6.6681e+1 1.4741eÀ1 1.6426eÀ5 À1.4146eÀ7 8.9222eÀ12 4.0390eÀ14 11 À6.7112e+1 1.1009eÀ1 3.7366eÀ4 À1.2284eÀ6 1.2520eÀ9 À4.3857eÀ13 22 À6.6938e+1 1.1812eÀ1 2.8349eÀ4 À1.0166eÀ6 1.0377eÀ9 À3.5797eÀ13 34 À6.0024e+1 1.4726eÀ1 À6.9462eÀ5 À2.0216eÀ7 3.1579eÀ10 À1.3450eÀ13 for P < P2 5.4021e+3 À3.5312e+2 8.1766e+0 À6.4838eÀ2 SF(a) 9.6886eÀ1 3.3717eÀ3 À4.2343eÀ5 1.7882eÀ7 frac SRE(a) À1.6061eÀ3 3.7746eÀ2 À4.7402eÀ4 2.0018eÀ6

Figure 4.2: Polynomial coeﬃcients provided for mean bias correction for day and nighttime RS92 radiosonde proﬁles. Source: Miloshevich et al. [2009]

4.1.2 Solar Radiation Error Correction In addition to the correction for mean calibration error, the daytime radiosonde measurements need to be corrected for solar radiation error which is caused by solar heating of RH sensors [Vömel et al., 2007]. The solar radiation error (SRE) is primarily a function of incident solar flux which in turn is a function of solar altitude angle. If the solar altitude angle is high then SRE in the radiosonde will be higher due to enhanced heating of the RH sensor. Figure 4.1(b) shows the mean calibration bias curve of RS92 sensor determined during daytime sounding. The polynomial coefficients (F(P, RH, 66 ◦)) for daytime soundings are also given in Figure 4.2 and are provided for a higher solar altitude angle (66◦). These coefficients represent a combination of mean calibration error and the SRE. The SRE component of the daytime mean bias is the difference between daytime and nighttime mean biases (SRE(66◦) =F(P, RH, 66◦)-F(P, RH, night)). The SRE component in daytime dataset for other solar altitude angles can be obtained by multiplying SRE(66◦) with the necessary SRE fraction which can be determined from Figure 4.3 by using the polynomial fit given in Figure 4.2 (frac SRE(α)). Thus, SRE for any solar altitude angle α is given by

SRE(α)=SRE(66◦) × fraction(α) (4.2) Once the SRE for a particular solar angle is obtained, the coeﬃcients for daytime measurements can be obtained by adding the SRE(α) to F (P, RH, night).

F (P, RH, day)=SRE(α)+F (P, RH, night) (4.3) In brief, the daytime correction for Vaisala RS92 radiosonde consists of correction for mean calibration error (SRE=0) and the correction for SRE. 20 Radiosonde Corrections

Figure 4.3: Solar fraction error as the function of solar altitude angle. Source: Milo- shevich et al. [2009]

4.1.3 Time Lag Correction

The time lag error is the result of a slow sensor response to changes in the ambient humidity at low temperatures (<-45 °C) prevalent in the upper troposphere and the lower stratosphere. The time lag error affects the sensor’s ability to discern the detailed vertical structure of humidity profile, thus yielding a smooth and incorrect RH profile. The time lag error can be corrected, provided that the sensor time constant and the vertical humidity gradient information are known [Miloshevich et al., 2004]. The time constant (τ) refers to the time required for the sensor to respond to 63% of an instantaneous change in the ambient humidity and is a function of temperature. This value is usually determined in the laboratory. The time constant for RS92 sensor as a function of temperature is shown in Figure 4.4. The ambient humidity can be obtained from the measured humidity using the formula

dU Ua =U+τ(T ) × (4.4) dt 4.1. Radiosonde Correction Methods 21

Figure 4.4: RS92 time constant as a function of temperature. Source: http:// milo-scientific.com/prof/radiosonde.php.

dU where dt is the local humidity gradient, Ua is the ambient humidity and U is the measured humidity. The time constant τ is given by

τ(T )=0.8 × exp(−0.7399 + −0.07718 × T ) (4.5) The time lag correction is based on the assumption that the ambient humidity remains almost constant for a short interval of the measurement time. This necessitates a very good temporal resolution for the profiles. It is also necessary that the relative humidity values in the profiles have precision to certain decimal values in order that the correction may be effective. 22 Radiosonde Corrections Chapter 5 Diurnal cycle of humidity

Microwave observations of humidity from NOAA satellites were made possible with the deployment of Advanced Microwave Sounding Unit (AMSU-B) sensor in the year 1998. Long term homogenous data is essential in the present context of climate studies. The AMSU data obtained from the diﬀerent NOAA satellites are usually composited to get a long term dataset and these data are not homogeneous due to factors like changes in spectral characteristics, sensor degradation and drift in the satellite orbit. Orbital parameters of polar orbiting, sun-synchronous satellites are designed so that they measure the radiation emitted from any point on the Earth and its atmosphere at similar local times, thus sampling the same part of the diurnal cycle of the geophysical parameters measured. However, in reality, factors like atmospheric drag, the Earth’s non-spherical shape, the gravitational pull from celestial bodies and solar activity causes the orbit to drift towards the centre of the Earth [John et al., 2011b]. Such a drift in orbital height leads to changes in the orbiting period and hence a change in the local sampling time of the satellites leading to undesirable aliasing of diurnal cycle into the time series of data from the polar orbiters. The drift in the satellite dataset can be corrected provided we have knowledge of the diurnal behaviour of the measurements. In this study, our aim is to construct a diurnal behaviour of brightness temperatures obtained from microwave humidity sounder measurements by combining data from diﬀerent NOAA satellite platforms. Microwave data have the advantage of having an all-sky sampling as compared to only clear-sky sampling for infrared measurements [John et al., 2011a]. The diurnal cycle has been constructed following the approach of Anders et al. [2011] for HIRS measurements which is described in the following sections.

5.1 Data

The data used in the study are from AMSU-B and microwave humidity sounder (MHS) sensors on-board NOAA satellites. AMSU-B is ﬂown on NOAA satellite-15,

23 24 Diurnal cycle of humidity

Equator Crossing Time for NOAA/MetOp POES

22 NOAA-17 MetOpA 20 NOAA-15

16 Local Time [ hour ] NOAA-16 NOAA-19 14 NOAA-18

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Time [ years ]

Figure 5.1: Temporal span and equator crossing time of NOAA satellites for ascending orbit. Source: John et al. [2011b]

16 and 17 and MHS is on NOAA-18. They are both 5 channel microwave radiometers dedicated to measuring atmospheric humidity, an important climate variable [Held and Soden, 2000], but not very well simulated by current climate models [John and Soden, 2007]. Out of the five channels, three (18, 19, 20) are placed near a strong water absorption line at 183 GHz. The channels 16 and 17, at 89 GHZ and 150 GHz respectively look deeper through the atmosphere on to the Earth’s surface. Some of the five channels differ between MHS and AMSU-B; and the differences are in channels 17 and 20 [John et al., 2011b]. Both AMSU-B and MHS sensors are cross track scanning instruments having 90 Earth field-of-views per scan line. We have used the data spanning the years 2000-2010 taken from 4 different satellites, NOAA-15, 16, 17 and 18. With the exception of NOAA 15, data from the other satellites do not have the full temporal coverage from 2000-2010 as shown in Table 5.1. The measurements from each satellite for ascending and descending orbits for each day were gridded as 2.5◦ X2.5◦ latitude-longitude grid points. The satellites have different local observing time (Figure 5.1) so a single grid point can have multiple data from different satellites for different local times. Each grid point was then composited on a monthly basis over the period from 2000–2010. The near nadir observations alone have been used to avoid any uncertainty due to limb correction and scan asymmetry [Buehler et al., 2005b].

5.2 Construction of Diurnal cycle

As can be seen from the Figure 5.1, NOAA 16 has drifted by approximately around 5 hours over a period of 10 years. Drifts of varying magnitudes are observed in other satellites too. This drift can cause false trends in the time series [Jacobowitz et al., 2003]. On the other hand, this drift has an obvious advantage in that it facilitates a better means of extracting the diurnal behaviour of brightness temperature. A diurnal cycle has been ﬁtted for each grid point separately for each month using a second order Fourier series. 5.2. Construction of Diurnal cycle 25

Table 5.1: Data time period of diﬀerent NOAA satellites.

Satellite Time period NOAA-15 2000-2010 NOAA-16 2001-2010 NOAA-17 2003-2010 NOAA-18 2006-2010

π(t − t1) 2π(t − t2) Tb = a + a cos + a cos (5.1) 0 1 12 2 12 where Tb is the brightness temperature, t is the observational local time in hours, a0 is the mean value of Tb, a1 is the half peak to peak amplitude of the 24 hourly oscillation, a2 is the half peak to peak amplitude of the 12 hourly oscillation and t1 and t2 are phase. After expanding Equation 5.1 can be written as

πt πt 2πt 2πt Tb = b + b cos + b sin + b cos + b sin (5.2) 0 1 12 2 12 3 12 4 12

The coeffcients b0, b1, b2 b3 and b4 are determined using linear regression. Using these coefficients amplitudes a0 to a2 and the phase t1 and t2 are determined. For examples of fitted curve see Figure 5.2. A Monte carlo error analysis has been performed to determine the uncertainty of the derived fit parameters (amplitudes a1 and a2 ) from the Fourier series. Mea- surements in each grid point would comprise of data from the different satellites with each satellite having a different number of observations for their ascending and descending passes. Within a grid point, the data from each satellite for a particular pass containing say, ’n’ observations would then be replaced by a randomly generated dataset having a similar mean and standard deviation as that of the ’n’ observations. The random numbers are generated using a normal distribution. Each grid point would thus have randomly generated data for each satellite for their ascending and descending passes respectively. The fit parameters a1 and a2 were determined using Equation 5.1. The process of randomizing the grid point data is repeated 300 times, thus generating 300 fit parameters (a1 and a2). The standard deviation of a1 and a2 thus generated is the uncertainty of the amplitudes. If this uncertainty exceeds the original amplitude then the fit can not be accepted as real. In order that the fit be counted valid, we stipulate that the ratio of amplitude to standard deviation (signal to noise ratio) be greater than 1. 26 Diurnal cycle of humidity

5.3 Results

The results presented in this section mainly focus on the analysis of brightness temperature diurnal variations of channel 16. The channel 16 is a window channel located in the microwave region and it senses mostly the Earth’s surface, so under clear sky conditions, the variations in this channel are expected to reﬂect the variations in the surface temperature.

The amplitudes a1 and a2 of the diurnal variation of Channel 16 brightness temperature determined using the method described in section 5.2 are represented on a map for the months January and July (Figure 5.3). The amplitude a2 is significantly lower than that of a1 over most regions. The amplitude of diurnal variation is largest over land and lower over oceanic regions which is consistent with Anders et al. [2011]. Dry land regions exhibit maximum diurnal variation in amplitude a1 which reaches over 10 K. Seasonal amplitude of diurnal variations in amplitude a1 is clearly visible; Western north America, Sahara desert, the Middle East and northern parts of Indian subcontinent exhibit a greater diurnal cycle since summer. Over much of Africa, a1 often exeeds 6 K. A strong diurnal variation is observed over coastal Antarctic in Jan- uary and over the Arctic in July. This may be due to the large emissivity variation occurring due to the solar heating of the surface which causes larger diurnal variation in channel 16 brightness temperature. As explained in section 5.2, a Monte carlo error analysis has been performed to find the uncertainties in amplitude a1 and a2. The result is shown in Figure 5.4 which maps the signal to noise ratio, and a larger magnitude of the ratio implies a robust fit for the diurnal cycle. The map shows that the diurnal variations are significant over most of the land regions and insignificant over most of the oceanic regions. The diurnal peak time of channel 16 brightness temperature for the months January and July is shown in Figure 5.5. It is shown over those regions where the diurnal fit has been found to be significant from the Monte carlo error analysis (signal to noise ratio > 1). Since the diurnal amplitude of these regions is dominated by that of the 24 hour oscillation (a1), the peak time shown here represents the peak time of the 24 hour oscillation. Over most of the land regions, the diurnal peak occurs after noon (12 pm) local time. This result is consistent considering that the maximum land temperature due to solar heating occurs in the early afternoon. Exceptions however are seen over some of the oceanic regions and some regions of Arctic where the peak time is in the early morning.

5.4 Future Work

The main aim of constructing a diurnal cycle is to correct the drift in satellite measurements so as to generate a homogeneous long term climate dataset. The results shown in the previous section are preliminary. Prior to compositing the satellite measurements for creating the diurnal cycle, it is necessary to correct for inter-satellite biases. Inter-satellite bias calculation has not been implemented in this study. There- 5.4. Future Work 27

280 290 270

280 270 260 270

Tb 16 [K] 260 Tb 17 [K] Tb 18 [K] 250 260

250 250 240 0 6 12 18 24 0 6 12 18 24 0 6 12 18 24 Local Time (hr) Local Time (hr) Local Time (hr)

280 290 NOAA15 NOAA16 280 NOAA17 270 NOAA18 270

Tb 19 [K] 260 Tb 20 [K] 260

250 250 0 6 12 18 24 0 6 12 18 24 Local Time (hr) Local Time (hr)

Figure 5.2: The diurnal variation in brightness temperature of channels 16–20 of a particular grid point (31.25◦N, 1.25◦E) over Sahara desert. Black curve represents the diurnal ﬁt. fore, the immediate step would be to correct the inter-satellite biases based on the method developed by John et al. [2011b]. Subsequent to construction of diurnal cycle from inter-satellite bias corrected dataset, the measurements will be corrected for the drift and its impact would be studied by analysing the time-series against the uncorrected dataset. 28 Diurnal cycle of humidity

Figure 5.3: The diurnal amplitudes a1 (left panel) and a2 (right panel) for the months January (upper panels) and July (lower panels) for AMSU-B channel 16 . Unit of amplitude is in Kelvin.

Figure 5.4: The uncertainty of the amplitudes a1 (left panel) and a2 (right panel) for the months January (upper panels) and July (lower panels). The uncertainty is deﬁned as the ratio of amplitude to its standard deviation. Unit is dimensionless. The white regions in the map indicate areas with misssing data. 5.4. Future Work 29

Figure 5.5: The diurnal local peak times in hours for the months January (left) and July (right). The white regions in the map indicate areas with misssing areas. 30 Diurnal cycle of humidity Chapter 6 Summary of Papers

This thesis encloses two papers related to study of atmospheric humidity. The measurements of atmospheric humidity are made available from two main sources, the radiosondes and the satellites. Though an accurate representation of humidity can be obtained from radiosondes, its upper tropospheric humidity measurements are unreliable. On the other hand, measurements from satellites aid in getting global information on atmospheric humidity. However, the accuracy of humidity retrieval from satellite measurements largely depends on how well an algorithm takes into account the nonlinearity between satellite measurements and the atmospheric state. The paper I enclosed in the thesis deals with a retrieval method which could improve the humidity retrieval accuracy from satellite measurements and paper II investigates the impact of various radiosonde humidity corrections in the upper troposphere by comparing them with satellite measurements.

6.1 Paper I

This paper introduces a non linear retrieval method based on data clustering and fuzzy logic for the retrieval of humidity from satellite infrared sounder measurements. Simulated Geostationary Operational Environmental (GOES) infrared sounder measurements have been used in this study. The advantage of the present method over the conventional retrieval methods such as linear or nonlinear regression retrieval methods has been demonstrated. The data clustering prior to the application of fuzzy rule for the retrieval is proved to be very effective in improving the retrieval accuracy. It has been shown that the present method reduces the root mean square by 20% in humidity retrieval as compared to non linear regression method. For improving the retrieval accuracy using linear (or non linear) regression methods, the regression coefficient has to be determined separately for each latitude zone. Since, in the present study, the data clustering has already been applied prior to application of fuzzy rules, the regional classification is already taken into account in the retrieval method. This provides an advantage for this method to yield a better retrieval accuracy even if the

31 32 Summary of Papers algorithm is trained with a global training dataset.

6.2 Paper II

This paper mainly demonstrates the importance of radiosonde humidity corrections for satellite validation. In this paper various correction methods have been applied to the radiosonde humidity measurements which were then compared with satellite measurements both in infrared and microwave spectral range. This correction has been applied to daytime and nighttime datasets separately. The major correction which is required for the nighttime dataset is the bias removal associated with calibration error in the measurement where as for daytime soundings, major source of correction is due to solar radiation error. The application of this correction is shown to be very important for improving the accuracy of satellite estimate of upper troposheric humidity. After application of this correction, the agreement between the radiosonde and the satellite measurements for daytime and nighttime could be brought down to a comparable level. This study also cautions against the use of uncorrected radiosonde measurements for satellite validation. References

V. L. Anders, I. A. Mackenzie, S. F. B. Tett, and L. Shi. Climatological diurnal cycles in clear-sky brightness temperatures from the high-resolution infrared radiation sounder (hirs). J. Atmos. Oceanic Technol., 28:1199–1205, 2011. doi: 10.1175/ JTECH-D-11-00093.1.

S. A. Buehler and V. O. John. A simple method to relate microwave radiances to upper tropospheric humidity. J. Geophys. Res., 110:D02110, 2005. doi: 10.1029/ 2004JD005111.

S. A. Buehler, P. Eriksson, T. Kuhn, A. von Engeln, and C. Verdes. ARTS, the atmospheric radiative transfer simulator. J. Quant. Spectrosc. Radiat. Transfer,91 (1):65–93, Feb. 2005a. doi: 10.1016/j.jqsrt.2004.05.051.

S. A. Buehler, M. Kuvatov, and V. O. John. Scan asymmetries in AMSU-B data. Geophys. Res. Lett., 32:L24810, 2005b. doi: 10.1029/2005GL024747.

S. A. Buehler, M. Kuvatov, V. O. John, M. Milz, B. J. Soden, D. L. Jackson, and J. Notholt. An upper tropospheric humidity data set from operational satellite microwave data. J. Geophys. Res., 113:D14110, 2008. doi: 10.1029/2007JD009314.

S. A. Buehler, V. O. John, A. Kottayil, M. Milz, and P. Eriksson. Eﬃcient radiative transfer simulations for a broadband infrared radiometer — combining a weighted mean of representative frequencies approach with frequency selection by simulated annealing. J. Quant. Spectrosc. Radiat. Transfer, 111(4):602–615, Mar. 2010. doi: 10.1016/j.jqsrt.2009.10.018.

S. A. Buehler, P. Eriksson, and O. Lemke. Absorption lookup tables in the radiative transfer model arts. J. Quant. Spectrosc. Radiat. Transfer, 112(10):1559–1567, 2011. doi: 10.1016/j.jqsrt.2011.03.008.

S. Chiu. Fuzzy model identiﬁcation based on cluster estimation. J. of Intelligent and Fuzzy Systems, 2:267–278, 1994.

33 34 References

I. Durre, C. N. Williams, X. Yin, and R. S. Vose. Radiosonde-based trends in precipitable water over the Northern Hemisphere: An update. Journal of Geophysical Research (Atmospheres), 114:D05112, Mar. 2009. doi: 10.1029/2008JD010989.

W. P. Elliott and D. J. Gaﬀen. On the utility of radiosonde humidity archives for climate studies. Bull. Amer. Met. Soc., 72(10):1507–1520, 1991.

P. Eriksson, S. A. Buehler, C. P. Davis, C. Emde, and O. Lemke. ARTS, the atmospheric radiative transfer simulator, version 2. J. Quant. Spectrosc. Radiat. Transfer, 112(10):1551–1558, 2011. doi: 10.1016/j.jqsrt.2011.03.001.

E. J. Fetzer, W. G. Read, D. Waliser, B. H. Kahn, B. Tian, H. V¨omel, F. W. Irion, H. Su, A. Eldering, M. T. Juarez, J. Jiang, and V. Dang. Comparison of upper tropospheric water vapor observations from the microwave limb sounder and atmospheric infrared sounder. J. Geophys. Res., 113, 2008. doi: 10.1029/2008JD010000.

J. A. Hartigan and M. A. Wong. A k-means clustering algorithm. Applied Statistics, 28:100–108, 1979.

I. M. Held and B. J. Soden. Water vapor feedback and global warming. Annu. Rev. Energy Environ., 25:441–475, 2000.

G. Holl, S. A. Buehler, J. Mendrok, and A. Kottayil. Simulating cloudy thermal infrared radiances with an optimised frequency grid in the radiative transfer model ARTS. J. Quant. Spectrosc. Radiat. Transfer, to be submitted.

H. Jacobowitz, L. L. Stowe, G. Ohring, A. Heidinger, K. Knapp, and N. R. Nalli. The Advanced Very High Resolution Radiometer Pathﬁnder Atmosphere (PATMOS) Climate Dataset: A Resource for Climate Research. Bulletin of the American Meteorological Society, 84:785–793, June 2003. doi: 10.1175/BAMS-84-6-785.

J. S. R. Jang, C. T. Sun, and E. Mizutani, editors. Neuro-fuzzy and soft computing: a computational approach to learning and machine intelligence. Prentice Hall, 1997.

V. O. John. Analysis of upper tropospheric humidity measurements by microwave sounders and radiosondes. Berichte aus dem Institut fuer Umweltphysik. Logos Verlag Berlin, Apr. 2005. Ph. D. Thesis, University of Bremen, ISBN 3-8325-1010- 9.

V. O. John and B. J. Soden. Temperature and humidity biases in global climate models and their impact on climate feedbacks. Geophys. Res. Lett., 34:L18704, 2007. doi: 10.1029/2007GL030429.

V. O. John, G. Holl, R. P. Allan, S. A. Buehler, D. E. Parker, and B. J. Soden. Clear-sky biases in satellite infra-red estimates of upper tropospheric humidity and its trends. J. Geophys. Res., 116:D14108, 2011a. doi: 10.1029/2010JD015355. References 35

V. O. John, G. Holl, S. A. Buehler, B. Candy, R. W. Saunders, and D. E. Parker. Understanding inter-satellite biases of microwave humidity sounders using global SNOs. J. Geophys. Res., 2011b. in press.

A. Kaufmann, editor. Introduction to the theory of fuzzy sets. Academic Press, Newyork, 1975.

J. T. Kiehl and B. P. Briegleb. Comparison of the observed and calculated clear sky greenhouse eﬀect: Implications for climate studies. J. Geophys. Res., 97(D9), 1992. doi: 10.1029/92JD00729.

A. Kottayil, P. K. Thapliyal, M. V. Shukla, P. K. Pal, P. C. Joshi, and R. R. Naval- gund. A new technique for temperature and humidity proﬁle retrieval from infrared sounder observations using adaptive neuro-fuzzy inference system. IEEE Geosci. R. S., 48:1650 –1659, 2010. doi: 10.1109/TGRS.2009.2037314.

A. Kottayil, S. A. Buehler, V. O. John, L. M. Miloshevich, M. Milz, and G. Holl. On the importance of Vaisala RS92 radiosonde humidity corrections for a better agreement between measured and modeled satellite radiances. J. Atmos. Oceanic Technol., 2011. doi: 10.1175/JTECH-D-11-00080.1. in press.

J. R. Lanzante and G. E. Gahrs. The ”clear-sky bias” of TOVS upper-tropospheric humidity. J. Climate, 13:4034–4041, 2000.

S. Manabe and R. T. Wetherald. Thermal Equilibrium of the Atmosphere with a Given Distribution of Relative Humidity. Journal of Atmospheric Sciences, 24: 241–259, May 1967. doi: 10.1175/1520-0469(1967)0240241:TEOTAW2.0.CO;2.

J. Mcqueen. Some methods for classiﬁcation and analysis of multivariate observations. In Fifth Berkeley Symposium on Mathematical Statistics and Probability, pages 281– 297, 1967.

L. M. Miloshevich, H. V¨omel, A. Paukkunen, A. J. Heymsﬁeld, and S. J. Oltmans. Characterization and correction of relative humidity measurements from vaisala RS80-A radiosondes at cold temperatures. J. Atmos. Oceanic Technol., 18(2):135– 156, 2001. doi: 10.1175/1520-0426(2001)0180135:CACORH2.0.CO;2.

L. M. Miloshevich, A. Paukkunen, H. V¨omel, and S. J. Oltmans. Development and validation of a time-lag correction for Vaisala radiosonde humidity measurement. J. Atmos. Oceanic Technol., 21:1305–1327, 2004. doi: 10.1175/1520-0426(2004) 0211305:DAVOAT2.0.CO;2.

L. M. Miloshevich, H. V¨omel, D. N. Whiteman, and T. Leblanc. Accuracy assessment and correction of vaisala RS92 radiosonde water vapor measurements. J. Geophys. Res., 114:D11305, 2009. ISSN 0148-0227. doi: 10.1029/2008JD011565. 36 References

I. Moradi, S. A. Buehler, V. O. John, and S. Eliasson. Comparing upper tropospheric humidity data from microwave satellite instruments and tropical radiosondes. J. Geophys. Res., 115:D24310, 2010. doi: 10.1029/2010JD013962.

P. P. Peixoto and A. H. Oort, editors. PHYSICS OF CLIMATE. Springer-Verlag Newyork, Inc., 1992.

R. T. Pierrehumbert, H. Brogniez, and R. Roca, editors. The Global Circu lation of the Atmosphere, chap. On the Relative Humidity of the Atmosphere. Princeton, 2006.

W. G. Rees. Physical Principles of Remote Sensing. Cambridge university press, 2 edition, 2001. ISBN 978-0-521-66948-1.

C. D. Rodgers, editor. INVERSE METHODS FOR ATMOSPHERIC SOUNDING. World Scientiﬁc Publishing Co.Pte.Ltd, 2000.

P. W. Rosenkranz. Water vapor microwave continuum absorption: A comparison of measurements and models. Radio Sci., 33(4):919–928, 1998. (correction in 34, 1025, 1999), ftp://mesa.mit.edu/phil/lbl_rt. M. S.Aldenderfer and R. K. Blashﬁeld, editors. Cluster Analysis, volume 44. SAGA Publication, Inc.

J. Schmetz, P. Pili, S. Tjemkes, D. Just, J. Kerkmann, S. Rota, and A. Ratier. An introduction to meteosat second generation (MSG). Bull. Amer. Met. Soc., 83(7): 977–992, 2002.

S. C. Sherwood, R. Roca, T. M. Weckwerth, and N. G. Andronova. Tropospheric water vapor, convection, and climate. Rev. Geophys., 48:RG2001, 2010. doi: 10. 1029/2009RG000301.

J. Shing and R. Jang. Adaptive-network-based fuzzy inference system. IEEE Trans. Syst., Man, Cybern., 23:665–685, 1993.

B. J. Soden and F. P. Bretherton. Upper tropospheric relative humidity from the GOES 6.7 μm channel: Method and climatology for july 1987. J. Geophys. Res., 98(D9):16,669–16,688, 1993.

B. J. Soden and J. R. Lanzante. An assessment of satellite and radiosonde climatologies of upper-tropospheric water vapor. J. Climate, 9:1235–1250, 1996.

B. J. Soden, D. J. Jackson, V. Ramaswamy, M. D. Schwarzkopf, and X. Huang. The radiative signature of upper tropospheric moistening. Science, 310(5749):841–844, Nov. 2005. doi: 10.1126/science.1115602.

T. Takagi and M. Sugeno. Fuzzy identiﬁcation of systems and its applications to modeling and control. IEEE Trans. Syst., Man, Cybern., 15:116–132, 1985. References 37

P. K. Thapliyal, M. V. Shukla, S. Shah, P. K. Pal, P. C. Joshi, and A. Kottayil. An algorithm for the estimation of upper tropospheric humidity from kalpana observations: Methodology and validation. J. Geophys. Res., 116:1–16, 2011. doi: 10.1029/2010JD014291.

K. E. Trenberth, J. Fasullo, and L. Smith. Trends and variability in column-integrated atmospheric water vapor. Climate Dynamics, 24:741–758, 2005.

D. D. Turner, B. M. Lesht, S. A. Clough, J. C. Liljegren, H. E. Revercomb, and D. C. Tobin. Dry bias and variability in vaisala RS80-H radiosondes: The ARM experience. J. Atmos. Oceanic Technol., 20:117–132, 2003.

H. V¨omel, H. Selkirk, L. Miloshevich, J. Valverde-Canossa, J. Valdes, E. Kyroe, R. Kivi, W. Stolz, G. Peng, and J. A. Diaz. Radiation dry bias of the Vaisala RS92 humidity sensor. J. Atmos. Oceanic Technol., 24(6):953–963, 2007. doi: 10.1175/JTECH2019.1.

F. J. Wentz, L. Ricciardulli, K. Hilburn, and C. Mears. How much more rain will global warming bring? Science, 317:233–235, 2007. doi: 10.1126/science.1140746.

L. A. Zadeh. Fuzzy sets. Inf. Control, 8:338–353, 1965. 38 References Paper I A new technique for temperature and humidity proﬁle retrieval from infrared sounder observations using Adaptive Neuro-Fuzzy Inference System

Authors: A. Kottayil, P. K. Thapliyal, M. V. Shukla, P. K. Pal, P. C. Joshi and R. R. Naval- gund

Bibliography: A. Kottayil, P. K. Thapliyal, M. V. Shukla, P. K. Pal, P. C. Joshi, and R. R. Naval- gund. A new technique for temperature and humidity proﬁle retrieval from infrared sounder observations using adaptive neuro-fuzzy inference system. IEEE Geosci. R. S., 48:1650 –1659, 2010. doi: 10.1109/TGRS.2009.2037314

39 40 1650 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 48, NO. 4, APRIL 2010 A New Technique for Temperature and Humidity Proﬁle Retrieval From Infrared-Sounder Observations Using the Adaptive Neuro-Fuzzy Inference System Kottayil S. Ajil, Pradeep Kumar Thapliyal, Munn V. Shukla, Pradip K. Pal, Prakash C. Joshi, and Ranganath R. Navalgund

Abstract—The accuracy of the atmospheric profiles of tempera- polar-orbiting satellite, the Atmospheric InfraRed Sounder ture and humidity, retrieved from infrared-sounder observations (AIRS) onboard the Earth Observing System Aqua satellite, the using physical retrieval algorithms, depends directly on the quality Infrared Atmospheric Sounding Interferometer (IASI) onboard of the first-guess profiles. In the past, forecasts from the numerical- the METOP satellite, the Sounder onboard the Geostationary weather-prediction models were extensively used as the first guess. During the past few years, the first guess for physical retrieval is Operational Environmental Satellite (GOES), and the Moderate being estimated using regression techniques from sounder observ- Resolution Imaging Spectroradiometer (MODIS) onboard the ations. In this study, a new nonlinear technique has been described Terra and Aqua satellites. India is planning to launch INSAT- to improve the first guess using simulated infrared brightness tem- 3D in a geostationary orbit with an 18-channel infrared sounder peratures for GOES-12 Sounder channels. The present technique (plus a visible channel for the daytime cloud detection) on- uses fuzzy logic and data clustering to establish a relationship board, for continuous coverage of the Indian ocean region, between simulated sounder observations and atmospheric pro- which is, so far, available only from GOES over the U.S. region. files. This relationship is further strengthened using the Adap- tive Neuro-Fuzzy Inference System (ANFIS) by fine-tuning the The retrieval of atmospheric temperature and humidity pro- existing fuzzy-rule base. The results of ANFIS retrieval have been files from a finite number of sounder observations is an ill- compared with those from the nonlinear (polynomial) regression posed as well as an ill-conditioned problem. Various algorithms retrieval. It has been found that ANFIS is more robust and reduces have been developed to invert the sounding observations to ob- root mean squared error by 20% in humidity profile retrieval tain atmospheric profiles. Linear-regression-based algorithms compared with the nonlinear regression technique. In addition, it have been developed by Smith and Woolf [1] for Nimbus-6 has been shown that the ANFIS technique has an added advantage of its global application without any need for classification of the HIRS, Goldberg et al. [2] and Weisz et al. [3] for AIRS, training data that is required in the regression techniques. and Seemann et al. [4] for MODIS. Physical retrieval algorithms have been developed by Smith and Woolf [5], Smith Index Terms—Atmospheric profiles, fuzzy inference system [6], Hayden [7], Ma et al. [8], Li et al. [9], Susskind et al. (FIS), fuzzy neural networks, infrared sounder. [10], etc., for various satellite sounders. Physical retrieval uses I. INTRODUCTION forecast atmospheric profiles from NWP models as the first guess. However, the state-of-the-art physical retrieval algo- ATELLITE-DERIVED atmospheric parameters play a rithms use atmospheric profiles from regression retrieval as S major role in the improvement of numerical weather pre- the first guess [4], [9], [11], [12]. An advantage of using first diction (NWP) at various timescales. Infrared-sounder obser- guess from regression retrieval is that these atmospheric profiles vations from polar and geostationary platforms have provided are physically consistent with the sounder observations. Many a wealth of 3-D atmospheric data of temperature and hu- nonlinear statistical algorithms have also been developed for midity, particularly over oceanic regions, which is otherwise profile retrieval using artificial neural networks (ANNs) for very difficult to obtain using limited radiosonde observations. different types of infrared and microwave sounding instruments Various sounding instruments have been developed so far, [13]–[16]. An ANN is a derivative-based approach, which tries e.g., the High-resolution InfraRed Sounder (HIRS) onboard the to minimize a cost function by adjusting the weights inter- National Oceanic and Atmospheric Administration (NOAA) connecting the neurons. The problem with the ANN approach is that it can be very easily trapped in local minima, giving Manuscript received December 2, 2008; revised April 27, 2009 and the false impression of network training. Fuzzy logic, used in August 24, 2009. First published January 22, 2010; current version published this paper, on the other hand, uses basic human perception for March 24, 2010. solving the problem by formulating linguistic rules which can K. S. Ajil is with the Department of Space Science, Lulea Technical University, 971 87 Lulea, Sweden (e-mail: [email protected]). be very easily interpreted. P. K. Thapliyal, M. V. Shukla, P. K. Pal, P. C. Joshi, and R. R. Navalgund Linearization of sounder observations with respect to the are with the Space Applications Center, Indian Space Research Organisation, atmospheric states is very important in regression retrieval, par- Ahmedabad 380 015, India (e-mail: [email protected]; pradip_ ticularly for humidity retrieval, because satellite brightness tem- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]). perature observations have highly nonlinear dependence on the Digital Object Identifier 10.1109/TGRS.2009.2037314 atmospheric water vapor content. Contrary to this, atmospheric

Authorized licensed use limited to: LULEA TEKNISKA UNIVERSITET. Downloaded on March 29,2010 at 05:01:36 EDT from IEEE Xplore. Restrictions apply. AJIL et al.:NEWTECHNIQUEFORTEMPERATUREANDHUMIDITYPROFILERETRIEVAL 1651 temperature profiles have a linear relationship with the sounder- TABLE I observed brightness temperatures. Due to this reason, although GOES-12 SOUNDER CHANNEL SPECIFICATIONS the retrieval of atmospheric profiles from infrared-sounder measurements can give better temperature retrieval accuracy, humidity retrieval accuracy is not up to the appreciable level. In this paper, an attempt has been made to develop a new nonlinear technique to retrieve temperature and humidity profiles that will best serve the purpose of providing an improved first guess to the physical retrieval algorithms. The nonlinear algorithm developed in this paper is primarily based on the integration of fuzzy logic and the data clustering techniques. The retrieval using the present method consists of two parts. In the first part, the technique of data clustering (CLUST) has been applied to classify the output (temperature and humidity profiles at different atmospheric levels) depending on its homo- geneity. Following data classification, fuzzy rules are applied to establish a relationship between inputs (sounder measurements) and the clustered data set. Details are included in Section III-A. In the second part, the input (membership functions) and the output parameters are fine-tuned using an adaptive network, and this is realized through the Adaptive Neuro-Fuzzy Inference System (ANFIS). In other words, the network fine-tunes the existing fuzzy-rule base determined in the first part. The ANFIS training is equivalent to that of an ordinary neural network, but morphologically, it is different. A hybrid algorithm is employed to fine-tune the fuzzy-rule base through linear least square and gradient descent method and is described in Section III-A2. Algorithms have been developed separately for both parts to enable the profile retrieval, and the details of the same are described in Section IV. σ e2 f 2 e = j + j , where j is the instrument noise (Table I) and II. DATA SET AND RADIATIVE TRANSFER MODEL fj is the forward model error of channel “j.” The constant value To demonstrate the performance of the nonlinear technique of fj =0.2 K has been used in this paper. developed in this paper, we have used the simulated data set of Sounder channel brightness temperatures are simulated for the GOES-12 Sounder channels using the fast radiative transfer the SeeBor data set (version 4, University of Wisconsin [21]) model RTTOV-7 [17]–[19] developed at the European Center comprising of ∼15 000 diverse global profiles of temperature, for Medium Range Weather Forecasting (ECMWF). RTTOV- humidity, and ozone from the Thermodynamic Initial Guess 7 uses the Pressure Layer Optical Depth algorithm [20] for Retrieval data set (TIGR3), NOAA data set (NOAA88), and radiative transfer calculation. Radiance calculations are done ECMWF analysis data sets. TIGR3 and NOAA88 data sets at 43 pressure levels ranging from 1013 to 0.1 hPa, taking are diverse profiles of atmosphere collected from various ra- into account the observational zenith angle and the absorption diosonde observations. The retrieval algorithms are trained by well-mixed gases, water vapor, and ozone. The GOES-12 with two different data sets: regional data set covering the Sounder consists of 18 infrared spectral bands ranging from 3.7 area 30S–30N, 0–180E and global data set covering the area to 14.7 μm. These channels are sensitive to atmospheric gases, 80S–80N, 180W–180E. The numbers of training profiles used primarily CO2,O3, and H2O (Table I), that detect radiation for the regional and global retrievals are 2026 and 9167, respec- emanating from different levels of atmosphere. This enables tively, which comprise profiles from the SeeBor data set ex- the measurement of the vertical distribution of temperature, cluding NOAA88 profiles. Simulated brightness temperatures moisture, and ozone. Out of the 18 infrared channels, only for the NOAA88 data set (∼1200 profiles) covering the region 15 were used in this paper, excluding the ozone absorption (30S–30N, 0–180E) have been used for testing the algorithm. channel (Channel#9) and the shortwave infrared channels, as The effect of the regional and global training data sets in they are contaminated by reflected solar radiation (Channel#17 retrieving the profiles using nonlinear regression and ANFIS and 18). The radiative transfer calculations for the GOES-12 is discussed in Section IV. sounder have been performed using RTTOV-7, generating an ensemble of simulated brightness temperatures along with the III. RETRIEVAL METHODOLOGY actual atmospheric states. Clear-sky radiances, simulated for nadir looking observations (zenith angle =0◦), have been used This section describes the methodology of data clustering in this paper. In order to account for the measurement noise, (CLUST) and ANFIS for the retrieval of temperature and hu- a Gaussian-distributed random number with zero mean and midity profiles using simulated infrared-sounder observations. standard deviation σ has been added to the simulated radi- The results of retrieval using CLUST and ANFIS have been ances. σ includes instrument noise and forward model error: compared with the nonlinear regression technique (REGRE)

Authorized licensed use limited to: LULEA TEKNISKA UNIVERSITET. Downloaded on March 29,2010 at 05:01:36 EDT from IEEE Xplore. Restrictions apply. 1652 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 48, NO. 4, APRIL 2010

that is widely used as the first guess to the physical retrieval. where rb is a positive constant. Therefore, the data points These techniques are described in the following sections. near the first cluster center ucl will have significantly reduced density measures, making the points unlikely to be selected as A. Fuzzy Logic the next cluster center. The constant rb defines a neighborhood r Fuzzy-set theory [22] was born out of the realization that the to be reduced in density measure. It is normally larger than a r =1.5r world that surrounds us is defined by a nondistinct boundary. to prevent closely spaced cluster centers; typically, b a. It is a mathematical tool to deal with the linguistic variables, The data points within the selected cluster are removed i.e., the concept described in natural language. Fuzzy sets and (subtracted) to ensure their absence in the next cluster. The fuzzy operators are the basic building blocks of fuzzy logic. algorithm looks for a new point with the highest number Linguistic rules that describe a system consist of two blocks: of neighbors. This is continued until all the data points are an antecedent (premise) block and a consequent block. The ap- evaluated. proach to a problem using fuzzy logic is facilitated through FIS. The first part of the retrieval algorithm is realized by formu- Fuzzy inference is a process, which helps in the formulation of lating a FIS through data clustering. In the first step, subtractive a mapping from a given input to an output using fuzzy logic. clustering algorithm is applied to the brightness temperatures The process of fuzzy inference involves membership functions, as input and the humidity and temperature at each atmospheric a fuzzy logic operator, and IF–THEN rules. The fuzzification level as output. The subtractive clustering algorithm divides layer in FIS generates membership values for all the inputs the data points at each atmospheric level into a number of through membership functions which lie in the premise part. clusters, which, in turn, gives the same number of rules for This process is often known as fuzzification. A fuzzy logic desired parameters (temperature and humidity) at that particular controller in FIS combines all the membership values on the atmospheric level. More precisely, it is categorizing the entire premise part to get the firing strength (weight). The next step is data set into different clusters depending on the characteristic the generation of qualified consequents for each rule depending variability of the atmospheric states, which are observed in on the firing strength that is aggregated to produce a crisp different regions of the globe. For example, there is a wide output. This step is called defuzzification. difference in the range of humidity values found in the tropics 1) Data Clustering (CLUST): Data clustering, by definition, and in the polar region. The clustering algorithm takes into is the grouping of the data into similar categories and account this variability while doing the clustering. The number identifying natural groupings of data from a large data set. Data of clusters generated using subtractive clustering depends on clustering can be performed using supervised or unsupervised the search radius, which is the Euclidean distance between the learning. The fuzzified c-means algorithm [23] is a supervised cluster center and the data point. A small search radius will algorithm because aprioriknowledge of a number of clusters is have more number of clusters and vice versa. In the present required. If the number of clusters is not known beforehand, it algorithm, the search radius of 0.4 is found to be sufficient for is necessary to apply unsupervised algorithms. In unsupervised all the levels. It should be noted that there must be a tradeoff learning, algorithms are provided with just data points and no between the search radius and the desired retrieval accuracy. labels; the task is to find out a suitable representation of the One can choose this depending on how the variability of the underlying distribution of data. In this paper, we have used sub- data set under consideration fluctuates. The premise part of the tractive clustering that belongs to the category of unsupervised fuzzy rule has brightness temperatures whereas the consequent algorithms and is based on the density of data points in the part has a clustered data set. The advantage of data clustering feature space [23]. Here, brightness temperatures from sounder prior to the application of fuzzy rules is that rules can be devised channels have been used as data points for clustering. The aim separately for different regions. The input values are graded is to find a region in the feature space with a high density of between 0 and 1 and are coded in the form of fuzzy membership data points. Consider a collection of K data points (size of functions. In the present algorithm, we have used a Gaussian- training data set) specified by n-dimensional vectors (number shaped membership function to represent the input. Takagi and of sounder channels) uk, k =1, 2,...,n, which are assumed to Sugeno’s fuzzy rules are used [28], in which the output of each be normalized. Since each data point is a candidate for a cluster rule is a linear combination of the input variables in addition center, a density measure at the data point uk is defined as to a constant term. The final output is the weighted average of each rule’s output. The weight is the firing strength which is K u − u D = exp − k j (1) obtained by combining the membership values on the premise k (r /2)2 j=1 a part of each rule through a specific T-norm operator, usually multiplication or minimum. r where a is a positive constant. A data point will have a high Fuzzy rules were generated using the training data set. Using density value if it has many neighboring data points. Only the the regional training data set with a search radius of 0.4, we r fuzzy neighborhood within the radius a contributes to the have four data clusters for humidity in the lower levels and density measure. five in a few of the upper levels. The same numbers of clusters After calculating the density measure for each data point, the have been generated using the global training data set with a point with the highest density is selected as the first cluster cen- search radius of 0.4. Similarly, for temperature, we have four u D ter. Let cl be the point selected and cl be its density measure. clusters for all the retrieved levels. The numbers of fuzzy rules u Next, the density measure for each data point cl is revised by vary depending on the total number of clusters. Separate FISs K were devised for each atmospheric level. This part of the FIS uk − uj D = D − Dcl exp − (2) has been generated using the standard MATLAB routine genfis2 k k (r /2)2 j=1 b available in the fuzzy logic toolbox.

Layer 4 is an adaptive node in which the network calculates the node output as a linear combination of inputs

αi · fi = αi · (pi · x1 + qi · x2 + ···+ ri) (5) i i i where fi is the output of the ith rule and {p ,q ,...,r } is the parameter set of this node. The parameters in this layer are called consequent parameters. A hybrid algorithm in ANFIS updates the consequent and premise parameters through a learning procedure having two parts. In the first part, the entire training patterns are propagated, and the optimal consequent parameters are estimated by an iterative least mean square procedure, while the premise parameters are assumed to be fixed. In the second part, patterns are propagated again, and Fig. 1. Schematic diagram of ANFIS for two inputs (x1 and x2) and one output with the two fuzzy rules (A1 and A2 for input x1,andB1 and B2 for in this epoch, backpropagation gradient descent method [26] input x2) of Takagi and Sugeno type (α1 and α2 are firing strengths, and f1 is used to modify the premise parameters, while the consequent and f2 are outputs from different rules). parameters remain fixed. These techniques provide a method to learn information about a data set, by computing the mem- 2) ANFIS: System identification is the process of determin- bership function parameters that best allow the associated FIS ing a mathematical model of an unknown system by observing to track the given input/output data. The parameters associ- its input–output data. Some of the commonly used techniques ated with the membership functions will change through the for nonlinear system identification include fuzzy logic [28], learning process. The computation of these parameters (or their [29], neural network [30], genetic algorithm, etc. A neuro-fuzzy adjustment) is facilitated by a gradient vector, which provides system [23], [24] combines the fuzzy logic and an adaptive a measure of how well the FIS is modeling the input/output network so as to integrate human expert knowledge and the data for a given set of parameters. This learning method works learning capability. The generated FIS through the subtractive similarly to that of neural networks [31]. The readers may refer clustering algorithm has been trained using ANFIS [32]. AN- to [32] for a comprehensive understanding of the mathematical FIS is a method of tuning the existing fuzzy-rule base with a details of ANFIS. learning algorithm using a collection of training data sets. This In this paper, the utility “anfis” available in the MATLAB allows the rule base to adapt. The ANFIS architecture is func- fuzzy logic toolbox has been used for the fine-tuning of the tionally equivalent to a fuzzy-rule base similar to Takagi and fuzzy-rule base generated by subtractive clustering. Sugeno’s [25] which has five layers. Fig. 1 shows a schematic diagram of ANFIS for a simple case having two inputs, two B. Nonlinear Regression (REGRE) fuzzy IF–THEN rules, and one output. Here, A1, A2, B1, and B2 are the linguistic labels, e.g., small, medium, large, etc., as- Ill-posed and ill-conditioned problems, arising in the retrieval sociated with the inputs coded in the form of fuzzy membership of the atmospheric profiles of temperature and humidity from functions. This procedure facilitates the classification of the sounder observations, are usually solved by the least square input values. linear regression method [1], [4], [27]. If the atmospheric state The output membership function of the node i of layer 1, vector is represented by X and measurements by Y , then which is generally a Gaussian function, is given by the linear relation between these may be expressed by the x − c 2 following: μ x − i i Ai ( i)=exp (3) X CY T ai = (6) where C is the matrix of regression coefficients. If the di- where xi is the input (sounder channels’ Tb) to node i and A mension of X is “m” (number of outputs) and that of Y is i is the linguistic label associated with this node function. n C m × n The values of the parameter set {a ,c } determine the shape of “ ” (number of inputs), then the dimension of is “ .” i i T the Gaussian function. The parameters in this layer are called Superscript “ ” denotes the transpose of the matrix. In the premise parameters. Generally, continuous and piecewise dif- present regression procedure, the primary inputs are the bright- ferentiable functions (trapezoidal/triangular membership func- ness temperatures of 15 GOES Sounder channels and the tions) are the possible candidates for the node function in this surface pressure. The quadratic terms of the sounder brightness layer. The inputs are mapped into the membership functions temperatures are also used as additional inputs to make it through (3). The task of the second layer in the ANFIS is to nonlinear (or polynomial) regression to account for the nonlinearity of atmospheric humidity in relation to the Sounder combine the output values from node 1 so as to get the firing C strength for each rule. Each node output represents the firing observations. The least square solution of (6) to compute is given by the following: strength αi of the ith rule. The function that can perform a generalized AND operation can be used as the node function C = XY (Y TY )−1. (7) in this layer. Every node in layer 3 is a fixed node which is the normalized firing strength (αi), given by Regression methods use a statistical relationship between the α collocated data set of sounder-measured radiances and simul- i αi = . (4) taneously observed atmospheric profiles of temperature and αi i humidity from radiosonde. Perfectly collocated matchup data

Authorized licensed use limited to: LULEA TEKNISKA UNIVERSITET. Downloaded on March 29,2010 at 05:01:36 EDT from IEEE Xplore. Restrictions apply. 1654 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 48, NO. 4, APRIL 2010

Fig. 2. Block diagrams summarize the retrieval procedure. The letter T stands for temperature, and q is for humidity. sets are very difficult to obtain due to various constraints, such These coefficients have been applied on the brightness tem- as the spatial resolution difference, observation time difference, peratures in the independent regional testing data set to retrieve presence of clouds, errors in radiosonde measurements, etc. In the atmospheric profiles of temperature and humidity. Retrieval this paper, the matchup data set to compute regression coef- statistics has been computed in terms of the root mean squared ficients is generated from the simulated GOES-12 brightness error (rmse), the mean bias (BIAS), and the coefficient of temperatures and the corresponding atmospheric profiles. correlation (R). The detailed methodology of retrieval using the aforemen- tioned three methods: CLUST, ANFIS, and REGRE have been IV. RESULTS AND DISCUSSIONS shown schematically in Fig. 2 for better understanding. Two sets of coefficients have been generated for each of these meth- Fig. 3 shows the rmse of the retrieved humidity profiles using ods: 1) CLUST_GLO, ANFIS_GLO, and REGRE_GLO using regional and global coefficients for all the three retrieval tech- the global training data set and 2) CLUST_REG, ANFIS_REG, niques, namely, REGRE, CLUST, and ANFIS. In general, rmse and REGRE_REG using the regional training data set. decreases from the surface level to upper atmospheric levels.

Fig. 3. RMSEs of the humidity retrieval using (solid line) regional and Fig. 5. Coefficient of correlation of the humidity retrieval using (solid line) (dashed line) global coefficients for three different retrieval techniques: regional and (dashed line) global coefficients for three different retrieval REGRE, CLUST, and ANFIS. techniques: REGRE, CLUST, and ANFIS.

Fig. 4. Mean bias of the humidity retrieval using (solid line) regional Fig. 6. RMSE in the temperature profile retrieval with REGRE, CLUST, and and (dashed line) global coefficients for three different retrieval techniques: ANFIS retrieval techniques using regional coefficients. REGRE, CLUST, and ANFIS. the retrieval procedure based on fuzzy rules. Fig. 4 shows the This is due to the fact that the absolute humidity values are large mean bias in humidity retrieval. It is clear from this figure at surface levels that decrease with the height. It is observed that the bias observed from CLUST and ANFIS are very small that fuzzy-based methods (CLUST and ANFIS) provide much and ranges between −0.3 and +0.2 g/kg, whereas it is more better retrieval accuracy compared with the nonlinear regres- than 1.0 g/kg at the lower atmospheric levels for REGRE that sion (REGRE), using both global and regional coefficients. For decreases in the upper atmospheric levels. In REGRE, the bias regression retrieval, there is a large improvement in humidity using global coefficients is higher (∼1.5 g/kg) compared with retrieval when regional coefficients (REGRE_REG) are used that using regional coefficients (∼1.0 g/kg), whereas above the instead of global coefficients (REGRE_GLO). RMSE in near- 800-hPa level, REGRE with regional coefficients has higher surface humidity retrieval in REGRE_REG is nearly 15% bias compared with the global coefficients. This large bias smaller compared with REGRE_GLO. Due to this reason, sepa- in REGRE may be attributed to the limited ability of the rate sets of coefficients are generated in regression retrieval for nonlinear regression method to pick up the nonlinearity in the different latitude zones. However, using CLUST and ANFIS, “humidity–brightness-temperature relationship” as compared the results remain almost the same while using regional or with the ANFIS. In nonlinear regression, we have included T 2 global coefficients. There is more than 25% reduction in the only a quadratic term ( b ) to represent the nonlinearity in rmse of the humidity profile using the fuzzy-logic-based re- the relationship, whereas in ANFIS, the nonlinearity is very trieval technique compared with those from REGRE. It is in- well represented by fuzzy rules. Fig. 5 shows the coefficient teresting to note that, contrary to REGRE retrievals, the rmse in of correlation (R) between the actual and the retrieved humidi- CLUST and ANFIS is slightly lower with global coefficients ties at different atmospheric levels. It may be seen that R in compared with that with regional coefficients. This shows REGRE_GLO is lowest at all levels ranging from 0.7 at the the advantage of using fuzzy-logic- and data-clustering-based surface to 0.85 at 400 hPa. Improvement in terms of R is largest algorithms, as the data set is automatically categorized within (∼10%) at lower levels and smaller (∼2%) at higher levels. The

Authorized licensed use limited to: LULEA TEKNISKA UNIVERSITET. Downloaded on March 29,2010 at 05:01:36 EDT from IEEE Xplore. Restrictions apply. 1656 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 48, NO. 4, APRIL 2010

Fig. 7. Histogram of absolute error in humidity retrieval (in grams per kilogram) in REGRE and ANFIS for global and regional coefficients at different atmospheric levels: (a) 1000, (b) 850, (c) 750, and (d) 600 hPa. ANFIS_GLO-retrieved humidity has the highest R (> 0.80) quency become smaller. This shows that the improvement in followed closely by CLUST_GLO with the actual humidity at the humidity retrieval using ANFIS is very high in the lower all atmospheric levels. This high degree of correlation of re- atmosphere, whereas there is small improvement in the humid- trieved and actual humidities using fuzzy techniques with ity profiles above 600 hPa as compared with REGRE. Fig. 7(a) global coefficients may be due to the fact that the global coeffi- shows that, for REGRE retrieval, there are only 20%–25% data cients use a larger data set than the regional coefficients, thus points showing an absolute humidity retrieval error of less than having better resources for the data clustering. However, these 1.0 g/kg, which is considerably high (33%) in the ANFIS differences in the statistics using global and regional coeffi- humidity retrieval. Similarly, for absolute humidity errors cients are very small compared with those in the regression (> 4 g/kg), there are 10% points in each category for REGRE, technique. which is less than 5% for ANFIS retrieval. Similar features are It may be noted from Fig. 6 that there is a very small observed at other levels also, confirming the importance of the improvement in the temperature profile retrieval using ANFIS ANFIS technique in reducing the magnitude of retrieval errors. and CLUST compared with REGRE. The highest improvement Figs. 8 and 9 show scatter plots of actual humidity with in temperature profile using ANFIS in the range of 0.3– retrieved humidity using REGRE_REG and REGRE_GLO, 0.5 K is noted around the tropopause region (100–200 hPa) respectively, at 1000-, 850-, 750-, and 600-hPa atmospheric which is less than 0.2 K at other levels. This may be due to levels. It is observed that the scatter is small at lower humidity the linear relationship between the brightness temperature and values and large toward higher humidity values. Scatter is the temperature profile, thereby leaving very little scope of smaller in the REGRE_REG compared with REGRE_GLO, improvement in temperature profile retrieval using a nonlinear particularly at higher humidity values, thus indicating the im- technique. Contrary to this, large nonlinearity exists between portance of regional classification in the nonlinear regression atmospheric humidity and brightness temperature observations, technique. RMSE values for REGRE_REG are 2.97, 2.26, hence providing enough scope of improvement in the humidity 1.84, and 0.98 g/kg for 1000-, 850-, 750-, and 600-hPa lev- profile retrieval using a suitable nonlinear technique. This els, respectively, which are significantly on the higher side nonlinearity is taken care of in the ANFIS retrieval for humidity in the case of global regression REGRE_GLO, i.e., 3.64, in a better way compared with REGRE. It may also be noted 2.57, 2.04, and 1.21 g/kg, respectively. Similarly, Figs. 10 from Figs. 3–5 that, out of the two fuzzy-based techniques, and 11 show scatter plots of actual with retrieved humidity ANFIS shows improvement over CLUST. Therefore, ANFIS is using ANFIS_REG and ANFIS_GLO, respectively, at 1000-, chosen for further analysis and comparison with REGRE. 850-, 750-, and 600-hPa atmospheric levels. It is encour- Fig. 7 shows the histogram of the absolute humidity error aging to see that the scatter between the actual and the in REGRE and ANFIS for different atmospheric levels 1000, retrieved humidities at all levels has reduced significantly, 850, 750, and 600 hPa. The total numbers of profiles for these particularly for the higher values of humidity. RMSE values pressure levels are 712, 1206, 1227, and 1227, respectively. for ANFIS_REG are 2.67, 1.94, 1.46, and 0.80 g/kg at 1000-, It is evident that the frequency of smaller absolute humidity 850-, 750-, and 600-hPa levels, respectively, which are sig- errors in ANFIS is considerably higher compared with REGRE, nificantly smaller than those from the regression retrieval. It whereas the frequency of higher absolute humidity errors in has been observed that the rmse in ANFIS_GLO is slightly ANFIS is very small compared with the REGRE retrieval. lower (2.59, 1.87, 1.40, and 0.76 g/kg, respectively) than that For higher atmospheric levels, these differences in the fre- from ANFIS_REG. This indicates that the use of more data in

Fig. 8. Scatter diagram of retrieved humidity (in grams per kilogram) versus Fig. 10. Scatter diagram of retrieved humidity (in grams per kilogram) actual humidity (in grams per kilogram) using REGRE_REG at different versus actual humidity (in grams per kilogram) using ANFIS_REG at different atmospheric levels: (a) 1000, (b) 850, (c) 750, and (d) 600 hPa. atmospheric levels: (a) 1000, (b) 850, (c) 750, and (d) 600 hPa.

Fig. 9. Scatter diagram of retrieved humidity (in grams per kilogram) versus Fig. 11. Scatter diagram of retrieved humidity (in grams per kilogram) actual humidity (in grams per kilogram) using REGRE_GLO at different versus actual humidity (in grams per kilogram) using ANFIS_GLO at different atmospheric levels: (a) 1000, (b) 850, (c) 750, and (d) 600 hPa. atmospheric levels: (a) 1000, (b) 850, (c) 750, and (d) 600 hPa. training using fuzzy method will aid to the data clustering and The training time of ANFIS depends on the complexity of the hence improve the fuzzy retrieval technique using global train- network, the total number of epochs, and the search radius. ing data. In this paper, the training time for 100 epochs with a search Results from these analyses indicate the possibility of pro- radius of 0.4 is approximately 3 s for each parameter at each viding better estimates of humidity profiles using the ANFIS atmospheric level. However, this being a one-time exercise, the retrieval technique that can be used as the improved first guess time taken in the training is not a constraint. For the retrieval, for nonlinear physical retrieval as compared with the first guess ANFIS works very fast and takes only 1.2 s for retrieving from the conventional regression (linear or nonlinear) retrieval. the temperature and the humidity profile at 40 atmospheric Apart from the accuracy, the time taken in generating the first levels for ∼1200 testing data points on Intel computer having a guess is very crucial. We found ANFIS to be fast enough in Pentium 4 processor (2.66 MHz) compared with ∼50 ms using providing the first guess to the physical retrieval procedure. the nonlinear regression technique.

Authorized licensed use limited to: LULEA TEKNISKA UNIVERSITET. Downloaded on March 29,2010 at 05:01:36 EDT from IEEE Xplore. Restrictions apply. 1658 IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 48, NO. 4, APRIL 2010

V. C ONCLUSION in improving the contents and the quality of this paper and Dr. N. K. Vyas of SAC, ISRO, for critically going through the In this paper, a new technique based on fuzzy logic has manuscript. been developed for the retrieval of atmospheric temperature and humidity profiles using simulated infrared-sounder brightness temperatures, and its advantages have been demonstrated REFERENCES by comparison with a nonlinear regression technique. Two techniques based on fuzzy logic, i.e., data clustering and [1] W. L. Smith and H. M. Woolf, “The use of eigenvectors of statistical covariance matrices for interpreting satellite sounding radiometer obser- ANFIS, have been used to train the simulated training data for vations,” J. Atmos. Sci., vol. 33, no. 7, pp. 1127–1140, Jul. 1976. temperature and humidity retrieval from the sounder-observed [2] M. D. Goldberg, Y. Qu, L. M. McMillin, W. Wolf, L. Zhou, and brightness temperatures. Separate coefficients have been gen- M. Divakarla, “AIRS near-real-time products and algorithms in support of erated for the global and regional training data sets. Statistics operational numerical weather prediction,” IEEE Trans. Geosci. Remote Sens., vol. 41, no. 2, pp. 379–389, Feb. 2003. has been generated by applying these coefficients on the in- [3] E. Weisz, H.-L. Huang, J. Li, E. Borbas, K. Baggett, P. K. Thapliyal, dependent regional simulated data set. It has been observed and L. Guan, “International MODIS and AIRS processing package: AIRS that the fuzzy-logic-based techniques significantly improved products and applications,” J. Appl. Remote Sens., vol. 1, no. 1, p. 013519, Jul. 2008, DOI: 10.1117/1.2766867. the humidity retrieval compared with the nonlinear regression. [4] S. W. Seeman, J. Li, W. P. Menzel, and L. E. Gumley, “Operational The improvement in humidity profile retrieval using a fuzzy- retrieval of atmospheric temperature, moisture, and ozone from MODIS logic-based technique is more than 25% reduction in the rmse infrared radiances,” J. Appl. Meteorol., vol. 42, no. 8, pp. 1072–1091, compared with that with the nonlinear regression retrieval. The Aug. 2003. [5] W. L. Smith and H. M. Woolf, “A linear simultaneous solution for temper- present study also demonstrates that, while there is a need to ature and absorbing constituent profiles from radiance spectra,” in Proc. classify the training data in different latitude zones to improve 4th Int. TOVS Study Conf. Tech., Igls, Austria, 1988, pp. 330–344. the retrieval accuracy in nonlinear regression retrieval, there is [6] W. L. Smith, “The retrieval of atmospheric profiles from VAS geostation- no such need in the case of fuzzy-logic-based techniques. It is ary radiance observations,” J. Atmos. Sci., vol. 40, no. 8, pp. 2025–2035, Aug. 1983. also observed that, out of the two fuzzy-logic-based techniques, [7] C. M. Hayden, “GOES-VAS simultaneous temperature–moisture retrieval ANFIS provides slightly better accuracy compared to that with algorithm,” J. Appl. Meteorol., vol. 27, no. 6, pp. 705–733, Jun. 1988. CLUST. ANFIS retrieval using the global training data set [8] X. L. Ma, T. Schmit, and W. Smith, “A non-linear physical retrieval algorithm—Its application to the GOES-8/9 sounder,” J. Appl. Meteorol., provides better humidity retrieval accuracies compared with vol. 38, no. 5, pp. 501–503, May 1999. those obtained from the regional training data set. [9] J. Li, W. W. Wolf, W. P. Menzel, W. Zhang, H. L. Huang, and T. H. Achtor, Improvement in the humidity retrieval is required to make “Global sounding of the atmosphere from ATOVS measurements: The this information suitable for assimilation in NWP and climate algorithm and validation,” J. Appl. Meteorol., vol. 39, no. 8, pp. 1248– 1268, Aug. 2000. models. World Meteorological Organization requirements of [10] J. Susskind, C. D. Barnet, and J. M. Blaisdell, “Retrieval of atmospheric temperature and humidity accuracies for NWP are 1 K in the and surface parameters from AIRS/AMSU/HSB data in the presence of troposphere for temperature and 15%–25% in the humidity clouds,” IEEE Trans. Geosci. Remote Sens., vol. 41, no. 2, pp. 390–409, Feb. 2003. mixing ratio. The present ANFIS retrieval algorithm is an [11] X. L. Ma, Z. Wan, C. C. Moeller, W. P. Menzel, L. E. Gumely, and attempt to provide improved estimates of humidity as an input Y.Zhang, “Retrieval of geophysical parameters from Moderate Resolution first guess to the nonlinear physical retrieval compared with Imaging Spectroradiometer thermal infrared data: Evaluation of a two- that provided by conventional linear or nonlinear regression step physical algorithm,” Appl. Opt., vol. 39, no. 20, pp. 3537–3550, Jul. 2000. techniques. [12] X. Wu, J. Li, W. Zhang, and F. Wang, “Atmospheric profile retrieval with The application of the present technique to real satellite AIRS data and validation at the ARM CART site,” Adv. Atmos. Sci., observations needs to be investigated in detail. This includes vol. 22, no. 5, pp. 647–654, Sep. 2005. the investigation of the sensitivity of different sensor noise [13] R. J. Kuligowski and A. P. Barros, “Combined IR–microwave satellite retrieval of temperature and dewpoint profiles using artificial specifications on retrieval using satellite observations. The ex- neural networks,” J. Appl. Meteorol., vol. 40, no. 11, pp. 2051–2067, tension of the present ANFIS technique to the hyperspectral Nov. 2001. sounder measurements, such as IASI and AIRS, needs spe- [14] F. Aires, A. Chedin, N. A. Scott, and W. B. Rossow, “A regularized neural net approach for retrieval of atmospheric and surface temperature with cial investigation. This may require the application of data- the IASI instrument,” J. Appl. Meteorol., vol. 41, no. 2, pp. 144–159, compression techniques, like the principal component analysis, Feb. 2002. for reducing the dimensionality of the input measurements. The [15] Y. Zhigang, C. Hongbin, and L. Longfu, “Retrieving atmospheric tem- large number of inputs (2378 channels in AIRS and ∼8500 perature profiles from AMSU-A data with neural networks,” Adv. Atmos. Sci., vol. 22, no. 4, pp. 606–616, Jul. 2005. channels in IASI) will require more fuzzy rules and will make [16] C. T. Butler, R. V.Z. Meredith, and A. P. Stogryn, “Retrieving atmospheric the ANFIS architecture more complicated, leading to increased temperature parameters from DMSP SSM/T-1 data with a neural net- memory space and retrieval time. work,” J. Geophys. Res., vol. 101, no. D3, pp. 7075–7083, 1996. [17] J. R. Eyre, “A fast radiative transfer model for satellite sounding system,” Eur. Centre Medium-Range Weather Forecast., Reading, U.K., Tech. ACKNOWLEDGMENT Memo. 176, 1991. [18] R. W. Saunders, M. Matricardi, and P. Brunel, “An improved fast radiative The authors would like to thank Dr. J. S. Parihar, the Deputy transfer model for assimilation of satellite radiance observations,” Q. J. R. Meteorol. Soc., vol. 125, no. 556, pp. 1407–1425, Apr. 1999. Director of the Remote Sensing Applications Area, for the [19] S. Hannon, L. L. Strow, and W. W. McMillan, “Atmospheric infrared constant encouragement and Dr. A. Huang and Dr. J. Li of fast transmittance models: A comparison of two approaches,” in Proc. the University of Wisconsin, for their useful discussions and SPIE Conf. Opt. Spectroscopic Tech. Instrum. Atmos. Space Res. II, 1996, guidance in the sounding retrieval. The authors would like vol. 2830, pp. 94–105. [20] M. Matricardi, F. Chevallier, and S. Tjemkes, “An improved general fast to thank three anonymous reviewers and the Associate Editor radiative transfer model for the assimilation of radiance observations,” for the valuable comments and suggestions that greatly helped ECMWF, ECMWF Res. Dept., Tech. Memo. 345, 2001.

[21] E. Borbas, S. Seemann, H.-L. Huang, J. Li, and W. P. Menzel, “Global Munn V. Shukla was born in Uttar Pradesh, India, profile training database for satellite regression retrievals with estimates in 1981. He received the B.Sc. (Honors) degree in of skin temperature and emissivity,” in Proc. 14th Int. ATOVS Study Conf., physics from the University of Delhi, Delhi, India, in Beijing, China, 2005. 2000 and the M.Sc. degree in physics from the Indian [22] L. A. Zadeh, “Fuzzy sets,” Inf. Control, vol. 8, pp. 338–353, 1965. Institute of Technology, Delhi, in 2003. [23] J.-S. R. Jang, C. T. Sun, and E. Mizutani, Neuro–Fuzzy and He taught in the Hans Raj College, University Soft Computing. Upper Saddle River, NJ: Prentice-Hall, 1997, of Delhi, for a brief period. He joined the Space ser. MATLAB Curriculum Series. Applications Center, Indian Space Research Organi- [24] J.-S. R. Jang and C.-T. Sun, “Neuro–fuzzy modeling and control,” Proc. sation, Ahmedabad, India, in 2006. He is working on IEEE, vol. 83, no. 3, pp. 378–406, Mar. 1995. geophysical parameter retrieval, cloud detection, and [25] T. Takagi and M. Sugeno, “Fuzzy identification of systems and its ap- cloud clearing from infrared sounders, mainly for the plications to modeling and control,” IEEE Trans. Syst., Man, Cybern., future Indian geostationary satellite INSAT-3-D. vol. SMC-15, no. 1, pp. 116–132, Feb. 1985. [26] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Learning Internal Representations by Error Propagation, Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations,vol.I, Pradip K. Pal received the M.Sc. and Ph.D. degrees D. E. Rummelhart and J. L. McClelland, Eds. Cambridge, MA: MIT in applied mathematics from the People’s Friend- Press, 1986, pp. 318–362. ship University, Moscow, Russia, in 1981 and 1984, [27] W. L. Smith, H. M. Woolf, and W. J. Jacob, “A regression method for respectively. obtaining real time temperature and geopotential height profiles from He is currently the Group Director of the Mete- satellite spectrometer measurements and its application to NIMBUS3 orology and Oceanography Group, Space Applica- ‘SIRS’ observations,” Mon. Weather Rev., vol. 98, no. 8, pp. 582–603, tions Center, Indian Space Research Organisation, Aug. 1970. Ahmedabad, India. He is involved in the retrieval [28] J. Chanussot, G. Mauris, and P. Lambert, “Fuzzy fusion techniques for of geophysical parameters from satellite data and linear features detection in multitemporal SAR images,” IEEE Trans. their assimilation in numerical weather and climate Geosci. Remote Sens., vol. 37, no. 3, pp. 1292–1305, May 1999. prediction models. He has developed mathematical [29] J. Chanussot, J. A. Benediktsson, and M. Fauvel, “Classification of remote methods for retrieval and assimilation problems. sensing images from urban areas using a fuzzy possibilistic model,” IEEE Trans. Geosci. Remote Sens., vol. 3, no. 1, pp. 40–44, Jan. 2006. [30] C. Surussavadee and D. H. Staelin, “Global millimeter-wave precipitation Prakash C. Joshi received the M.Sc. degree in retrievals trained with a cloud-resolving numerical weather prediction physics and the Ph.D. degree in theoretical physics model: Part I: Retrieval design,” IEEE Trans. Geosci. Remote Sens., from Banaras Hindu University, Varanasi, India, in vol. 46, no. 1, pp. 99–108, Jan. 2008. 1970 and 1974, respectively. [31] M. Bishop, Neural Networks for Pattern Recognition. New York: Oxford He had a stint of four years in postgraduate teach- Univ. Press, 1995. ing in the Banaras Hindu University after acquiring [32] J.-S. R. Jang, “ANFIS: Adaptive-network-based fuzzy inference sys- the Ph.D. degree. In 1979, he joined the Indian Insti- tem,” IEEE Trans. Syst., Man, Cybern., vol. 23, no. 3, pp. 665–685, tute of Tropical Meteorology, Pune, India, as a Senior May/Jun. 1993. Scientific Officer and worked on stochastic dynamic prediction and tropical cyclone studies. Since 1983, he has been with the Space Applications Center, Kottayil S. Ajil was born in Kerala, India, in Indian Space Research Organisation, Ahmedabad, India, where he is heading 1981. He received the M.Sc. degree in physics from the Satellite Meteorology Group. His interests include the retrieval of geophys- Mahatma Gandhi University, Kottayam, Kerala, ical parameters from satellite data and assimilation in numerical models. His India, in 2004. He is currently working toward the main area of research is the application of satellite data in tropical weather Ph.D. degree in the Department of Space Science, studies. He has also taken a leading role in capacity building in satellite Lulea Technical University, Kiruna, Sweden. meteorology in the Asia Pacific region and popularizing the science of weather He was a Research Fellow in the Space Applica- through active participation in the Indian Meteorological Society. tions Center, Indian Space Research Organisation, in 2006–2008. He is currently involved in the development of a retrieval algorithm for ice cloud micro- Ranganath R. Navalgund received the M.Sc. de- physical parameters from submillimeter-wavelength gree in physics from the Indian Institute of Technol- observations. His research interests include the development of radiative trans- ogy, Mumbai, India, in 1970 and the Ph.D. degree fer models and retrieval of geophysical parameters from satellite observations. in physics from the Tata Institute of Fundamental Research, Mumbai, in 1977. He joined the Space Applications Center (SAC), Pradeep Kumar Thapliyal was born in Indian Space Research Organisation, Ahmedabad, Rudraprayag, India, in 1970. He received the India, in 1977. From 2001 to 2005, he was the M.Sc. degree in physics from H.N.B. Garhwal Director of the National Remote Sensing Agency, University, Srinagar, India, in 1990, the P.G. Hyderabad, India. He is currently a Distinguished Diploma in space science from Gujarat University, Scientist and the Director of SAC, where he oversees Ahmedabad, India, in 1992, and the Ph.D. degree the design and development of state-of-the-art communication, navigation, in mechanical sciences from the Indian Institute of meteorological, and remote sensing payloads flown on the INSAT and IRS Science, Bangalore, India, in 2004. series of satellites and the development of space application programs in the He has been a Scientist with the Meteorology and field of telemedicine, tele-education, survey of natural resources, etc. Oceanography Group, Space Applications Center, Dr. Navalgund is the recipient of many awards: 1) Maharana Udaisingh Indian Space Research Organisation, Ahmedabad, Award for Environment in 2008–2009 instituted by the Maharana of Mewar since 1995. Between 1995 and 2004, he worked on climate modeling, monsoon Foundation, Udaipur; 2) Bhaskara Award in 2006 from the Indian Society of prediction, and microwave remote sensing of soil moisture. Currently, he is Remote Sensing (ISRS) for his lifetime contributions in the area of remote the Principal Investigator for the development of an algorithm for the retrieval sensing; 3) Vasvik Award in 2003; 4) ASI Award in 1996 by the Astronautical of atmospheric profiles from the infrared sounder onboard the future Indian Society of India; 5) Indian National Remote Sensing Award in 1992 instituted geostationary satellite INSAT-3-D. He has developed algorithms for the esti- by ISRS; 6) Prof. K. R. Ramanathan Memorial Lecture Gold Medal in 2002 mation of outgoing longwave radiation and upper tropospheric humidity from instituted by the Indian Geophysical Union; and 7) Outstanding Leadership Indian geostationary satellites. From March 2005 to March 2006, he was a Award from the International Society for Photogrammetry and Remote Sensing. Visiting Scientist with the Space Science and Engineering Center, University He was the President of the Technical Commission II of the International of Wisconsin, where he worked on hyperspectral infrared sounding of the Society for Photogrammetry and Remote Sensing from 2000 to 2004. He atmosphere. represents India on the Group on Earth Observations, an international initiative.

Authorized licensed use limited to: LULEA TEKNISKA UNIVERSITET. Downloaded on March 29,2010 at 05:01:36 EDT from IEEE Xplore. Restrictions apply. Paper II On the importance of Vaisala RS92 radiosonde humidity corrections for a better agreement between measured and modeled satellite radiances

Authors: A. Kottayil, S. A. Buehler, V. O. John, L. M. Miloshevich, M. Milz and G. Holl

Bibliography: A. Kottayil, S. A. Buehler, V. O. John, L. M. Miloshevich, M. Milz, and G. Holl. On the importance of Vaisala RS92 radiosonde humidity corrections for a better agreement between measured and modeled satellite radiances. J. Atmos. Oceanic Technol., 2011. doi: 10.1175/JTECH-D-11-00080.1. in press

51 52 Generated using V3.1.2 of the oﬃcial AMS LATEX template–journal page layout FOR AUTHOR USE ONLY, NOT FOR SUBMISSION!

On the importance of Vaisala RS92 radiosonde humidity corrections for a better agreement between measured and modeled satellite radiances

Ajil Kottayil ∗ SRT–RY, Lule˚a University of Technology, Space Campus 1, SE-98128 Kiruna, Sweden Stefan. A. Buehler SRT–RY, Lule˚a University of Technology, Space Campus 1, SE-98128 Kiruna, Sweden Viju. O. John Met Oﬃce Hadley Centre, Exeter, UK Larry M. Miloshevich Milo Scientiﬁc LLC, Lafayette, Colorado, USA M. Milz SRT–RY, Lule˚a University of Technology, Space Campus 1, SE-98128 Kiruna, Sweden G. Holl SRT–RY, Lule˚a University of Technology, Space Campus 1, SE-98128 Kiruna, Sweden

ABSTRACT A study has been carried out to assess the importance of radiosonde corrections in improving the agreement between satellite and radiosonde measurements of upper tropospheric humidity. Infrared (HIRS 12) and microwave (AMSU 18) measurements from the NOAA-17 satellite were used for this purpose. The agreement was assessed by comparing the satellite measurements against simulated measurements using collocated radiosonde proﬁles of the Atmospheric Radiation Measurement program (ARM) undertaken at tropical and mid-latitude sites. The radiative transfer model ARTS was used to simulate the satellite radiances. The comparisons have been done under clear sky conditions, separately for daytime and nighttime soundings. Only Vaisala RS92 radiosonde sensors were used and an empirical correction (EC) was applied to the radiosonde measurements. The EC includes correction for mean calibration bias and for solar radiation error, and it removes radiosonde bias relative to three instruments of known accuracy. For the night time dataset, the EC signiﬁcantly reduces the bias from 0.63 K to -0.10 K in AMSU 18 and from 1.26 K to 0.35 K in HIRS 12. The EC has an even greater impact on the daytime dataset with a bias reduction from 2.38 K to 0.28 K in AMSU 18 and from 2.51 K to 0.59 K in HIRS 12. The present study promises a more accurate approach in future radiosonde based studies in the upper troposphere.

1. Introduction the need for an accurate monitoring of the upper tropospheric humidity (UTH) which is not very well simulated The vertical distribution of water vapour in the tropo- in current climate models (John and Soden 2007). One sphere is quite inhomogeneous ranging from concentrations of the sources of UTH measurements is radiosonde data of few percent near the surface to only a few parts per mil- since the mid 20th century. These observations however, lion near the tropopause. Despite the fact that the upper are limited mainly to land areas and are known to have tropospheric water vapour constitutes only a small frac- data quality issues (Elliott and Gaffen 1991; Miloshevich tion of the total water vapour, it still has a large effect on et al. 2009; Soden and Lanzante 1996). the outgoing longwave radiation (Kiehl and Briegleb 1992). The satellite era has offered us the advantage of mea- Model studies suggest that nearly two thirds of the total suring UTH globally along with a wealth of other informa- radiative feedback from water vapour occurs in the upper tion to understand the earth’s atmospheric system better. troposphere (Held and Soden 2000). These studies confirm In the microwave and infrared spectral range, certain fre-

1 qency ranges are dominated by water vapour emission from 2. Measurements, Models and Methods Used a broad range of pressure levels approximately between 200 a. Satellite Data and 500 hPa. These frequency ranges are suitable for deriving UTH. The typical frequencies used for this application The NOAA-17 (National Oceanic and Atmospheric Ad- are 6.7 μm for infrared and 183 ±1 GHz for microwave. The ministration) satellite carries the Advanced Microwave Sound- term UTH was first used by Soden and Bretherton in early ing Unit (AMSU) and the High Resolution Infrared Sounder 1990s, to describe relative humidity regressed from satellite (HIRS)/3. AMSU consists of AMSU-A and AMSU-B sen- observed brightness temperatures obtained by broadband sors with a total of 20 channels. The primary function of infrared instruments. A similar approach was followed by the 15-channel AMSU-A (channels 1–15) is to provide tem- Buehler and John (2005) to derive UTH from microwave perature sounding of the atmosphere. The five channels of measurements near the 183 GHz water vapour emission AMSU-B (channels 16–20) mainly measure water vapour line. and liquid precipitation over land and sea. Three channels Though there have been studies on both infrared and (18–20) are situated around the strong water vapour spec- microwave measurements (Buehler et al. 2004; Soden and tral line at 183.31 GHz with different offsets from the line Lanzante 1996), none of the studies so far have looked center. AMSU-B is a cross track, continuous line scanning, into both instrument types simultaneously to judge their total power radiometer with an instantaneous field of view consistency. For this purpose, the satellite infrared and of 1.18o (at the half-power points), equivalent to a nominal microwave measurements have been collocated with ra- spatial resolution of 16 km at nadir. The antenna provides diosonde profiles of the Atmospheric Radiation Measure- a cross-track scan, scanning ±49.5o from nadir with a total ments (ARM) program undertaken at mid-latitude and of 90 earth fields of view per scan line. tropical locations. The comparisons were performed in HIRS/3 on-board NOAA-17 is a 20 channel instrument radiance space by simulating the infrared and microwave with an instantaneous field of view of 1.38o providing a channel radiance from the radiosonde data using the radia- nominal spatial resolution of 18.9 km at nadir. The an- tive transfer model, ARTS (Atmospheric Radiative Trans- tenna provides a cross-track stepped scan covering ± 49.58o fer Simulator) (Eriksson et al. 2011). from nadir with a total of 56 fields of view per scan. The The relative humidity (RH) measurements from Vaisala spectral characteristics of the HIRS/3 channels are avail- radiosondes are known to have a dry bias in the upper tro- able at http://www.ncdc.noaa.gov/oa/pod-guide/ncdc/ posphere (Vömel et al. 2007b; Moradi et al. 2010). Re- docs/klm/index.htm. The spectral characteristics differ cently, Miloshevich et al. (2009) have implemented an em- between different satellites. Hereafter, the two channels pirical correction procedure to remove the mean bias error used in inferring UTH measurements viz., HIRS Channel in RS92 radiosonde sensors by characterizing their accu- 12 and AMSU Channel 18 will be referred to as HIRS 12 racy relative to three reference instruments of known ac- (6.7 μm) and AMSU 18 (183 ±1 GHz) (Soden and Brether- curacy. We have applied this correction to the RS92 ra- ton 1993; Buehler and John 2005; Buehler et al. 2008). We diosonde profiles. The correction is applied for mean cali- have used measurements of HIRS 12 and AMSU 18 for the bration bias as a function of RH and temperature. A cor- period 2005–2008 on NOAA-17. rection for solar radiation error caused due to solar heating of the RH sensor is also applied for daytime soundings. In b. Radiosonde Data and the Correction methods addition, a non-bias correction for sensor time lag error The ARM programme operated by the United States caused by slow response at low temperature is also exam- Department of Energy launches radiosondes from several ined. The time lag correction recovers vertical structure dedicated sites. We have used radiosonde data from 2005– in the RH profile that was ”smoothed” by slow sensor re- 2008 of two stations: one in the Southern Great Plains sponse (Miloshevich et al. 2004). Although the time lag (SGP) at 36.61◦N, 97.49◦W in the USA, and one in the correction is not directly a bias correction, it may remove tropical western Pacific, located at 12.42◦S, 130.88◦Enear some parts of a bias, in particular in regions with strong Darwin, on the north-western coast of Australia. We have gradients. An important objective of this study is to assess taken day and nighttime radiosonde data measured using whether the various corrections improves the agreement Vaisala RS92 sensors. The number of profiles used for between radiosonde and satellite data in both infrared and nighttime analysis is 624 out of which 457 belongs to SGP microwave band. and 167 belongs to Darwin. All the 419 profiles used for The paper is structured as follows.: Section 2 focuses daytime analysis belong to SGP. The profiles used to study on the details of measurements and the models and meth- the impact of the time lag correction also belong to SGP. ods used. Section 3 presents results and discussions on The accuracy of radiosonde RH measurements in the the impact of radiosonde correction procedure in satellite– upper troposphere is known to be poor. A number of stud- radiosonde comparison. Finally, Section 4 presents the ies have shown that RS92 radiosonde sensors exhibit a dry summary and conclusions. bias in the upper troposphere (Vömel et al. 2007b; Moradi

2 et al. 2010). Miloshevich et al. (2009) have characterized RS92 profiles were also corrected for time-lag error as de- the accuracy of RS92 RH sensors relative to three water scribed by Miloshevich et al. (2004). A subset of soundings vapor reference instruments of known accuracy. The ref- is used because older ARM data (and most operational erence instruments are a cryogenic frostpoint hygrometer RS92 data) have integer RH values, which results in a less (CFH; Vömel et al. (2007a)), which quantifies the accu- accurate time-lag correction because much of the detailed racy of RS92 above 700 mb, a microwave radiometer that information important for a time-lag correction is lost due mainly senses water vapor in the lower troposphere, and to ”rounding off” of the data. For this and other reasons, it a system of 6 calibrated RH probes at the surface. The is recommended that users of the Vaisala Digicora III data RS92 mean calibration bias was determined by compar- system output the higher-resolution RH data in so-called ing RS92 measurements with simultaneous measurements FLEDT files rather than the integer RH values in standard from these instruments, and an empirical correction (EC) EDT data files. procedure was described that removes the RS92 mean bias It should be noted here that in December 2010, Vaisala as a function of RH and pressure: updated their data processing algorithm to include time- lag and solar radiation corrections that differ from those of , × RHcorr =G(P RH) RH (1) Miloshevich et al. (2009), but this does not include the correction for mean calibration bias used in this study. This where G(P,RH) is the correction factor. The correc- change affects only new Digicora III data systems or older tion factor is determined from pressure-dependent curve systems updated with software version 3.64, and the effect fits for several RH intervals (Figure 9 in Miloshevich et al. on soundings is described in Vaisala’s ”RS92 Data Conti- (2009)), with different curve fits for nighttime and daytime nuity” webpage. soundings. The bias error that is corrected for nighttime soundings is the mean sensor calibration bias. The daytime c. Radiative Transfer Model soundings, are additionally affected by an error caused by solar heating of the RH sensor. Therefore, in addition to The Atmospheric Radiative Transfer Simulator (ARTS) the mean sensor calibration bias correction, the daytime is a line by line radiative transfer model which simulates soundings also require a correction to remove the bias due radiances from the infrared to the microwave spectral range to the so called solar radiation error (SRE). Hence the bias (Buehler et al. 2005; Eriksson et al. 2011). The model uses and the correction for daytime soundings would be higher pre-calculated absorption data to speed up the calculation, than the nighttime soundings. In presence of clouds the as described in Buehler et al. (2011). daytime corrections could in fact overcorrect the radiation The code is flexible from the user point of view and is bias. However this is not a factor in this study because only freely available along with comprehensive documentation clear conditions were used. We have applied these correc- (http://www.sat.ltu.se/arts/). The ARTS model has tions on RS92 radiosonde data and quantified their effect been validated against other models and satellite measure- by comparing the corrected data to satellite measurements. ments (John and Buehler 2005; Melsheimer et al. 2005; This study uses a modified version of the published cor- Saunders et al. 2007). The model has been used for a wide rection for mean calibration bias that improves the accu- range of atmospheric studies including applications from racy for very dry conditions, and extends the upper limit of the infrared to the microwave wavelength range (Deuber validity of the correction to encompass the tropical UT/LS et al. 2005; Fiorucci et al. 2008; Eriksson et al. 2003) . (Upper troposphere/Lower stratosphere). The primary dif- Buehler et al. (2010) recently implemented a fast setup ference is that the correction was converted from a func- to reduce the computational cost of the broadband radi- tion of pressure to a function of temperature based on the ance simulations of HIRS channels through representative mean T(P) relationship for the soundings used to derive the frequency selection by simulated annealing. The compu- correction. A temperature dependence is more consistent tational cost has been reduced significantly while keeping with the actual sensor calibration, and generalizes the cor- the accuracy of the radiance calculations the same as con- rection to atmospheric profiles substantially different from ventional line by line calculations. They have shown that the mid-latitude soundings used to derive the correction. the HIRS channel radiances on NOAA-14 can be simulated Further details of the modified correction for mean cali- with only 133 frequencies. For comparison, a full line-by- bration bias, and associated IDL code, can be found at line calculation would require 72717 frequencies. The accu- http://milo-scientific.com/prof/radiosonde.php. racy of this method is of the order of 0.03 K. This modified The correction procedure described by Miloshevich et setup has been used for the present study. al. (2009) also includes a correction for sensor time-lag Different continuum models have to be used for the in- error that causes ”smoothing” of the measured RH pro- frared and the microwave spectral range. The absorption file due to slow sensor response at low temperatures. To model setups for the two different spectral ranges are as fol- see whether the time-lag correction has a significant role lows. In the microwave, we have used PWR98 (Rosenkranz in reducing the brightness temperature bias, a subset of 1998) for H2O lines and continuum and PWR93 (Rosenkranz

3 1993) for N2 and O2 lines and their respective continuum. −4.0 K ≤ TB(18)-TB(8) ≤ 2.0 K Furthermore, line data of O3 was taken from HITRAN 2004 −4.0 K ≤ TB(19)-TB(18) ≤ 2.0 K (Rothman et al. 2003). In the infrared, line data were taken from HITRAN 2004 while the continua of H2O, CH4,N2, iv. During the day a pixel is clear if O2 and CO2 were taken from MT CKD 1.0 (Mlawer et al. |TB(18) − TB(8)| < 10 K 2003). A model inter-comparison study between ARTS and the Line-Line Radiative Transfer model (LBLRTM), an es- where TB(i) is the brightness temperature for a given tablished reference model for the infrared spectral range HIRS channel i. If there is more than one clear pixel in the was carried out for HIRS 12 on NOAA satellites 14, 15 16 target area, the average over all the clear pixels are used and 17. The model comparison approach is described in otherwise it is classified as cloudy. This very stringent clear Appendix A. The two radiative transfer (RT) models were scene requirement condition satisfies 25 to 100 % of the found to agree within a brightness temperature difference target area for HIRS depending on the cloud cover. We of 0.22 K. ARTS includes various algorithms to simulate assume that the AMSU 18 pixels coincident to the clear the influence of clouds (Davis et al. 2005; Emde et al. 2004). HIRS 12 pixels are not cloud contaminated. This could However, for the present study only clear sky simulations even result in the removal of the AMSU 18 pixel which are were performed. unaffected by clouds due to the much higher sensitivity of HIRS measurements to thin clouds. d. Collocations and Cloud Detection Approach e. Relation between UTH and Brightness temperature The collocations between NOAA-17 measurements and ARM radiosondes are based on spatial and temporal con- The relationship between UTH and brightness temper- straints. The spatial constraint is implemented by a target ature Tb is given by, area of radius 50 km around the radiosonde location as in Buehler et al. (2004). This approximately takes into ln(UTH) = a + b × Tb (2) account the drift of the radiosonde as it ascends through where a and b can be determined by linear regression. The the atmosphere. The number of collocated pixels in a tar- derivative yields get area varies from 2–8 for HIRS 12 and from 10–30 for AMSU 18 respectively. The time constraint imposed on ΔUTH =b× ΔT (3) the collocated dataset is ±2 hours. This should accommo- UTH date the time taken for the radiosonde to reach the satellite measuring altitude peak (300 hPa) which is approximately In other words, absolute changes in brightness temper- 25 minutes. Hence, the radiosonde time in the collocated ature map to relative changes in UTH. To ensure complete dataset is the sum of the radiosonde launch time and 25 consistency here, we have re-determined the parameters minutes. Radiances are simulated from radiosonde profiles bHIRS 12 and bAMSU 18 using linear regression on the simu- using ARTS to match with the satellite viewing geometry. lated measurements for the Chevallier et al. (2006) data set. A meaningful comparison between satellite measure- The values obtained for bHIRS 12 and bAMSU 18 are 0.110 ments and clear sky RT model simulations is possible only and 0.070 for the nadir viewing direction. For the extreme if the measurements are free of cloud contamination. There- off-nadir viewing direction the bHIRS 12 and bAMSU 18 val- fore we use the cloud detection tests developed by McMillin ues are 0.118 and 0.076. These values are in good agree- and Dean (1982) to filter out contaminated HIRS pixels ment with Soden and Bretherton (1993) and Buehler and from our analysis. Details of this procedure are as follows. John (2005). The regression coefficients a and b for dif- Each pixel from satellite measurements is subjected to ferent viewing angles of HIRS 12 are summarised in Table the following cloud detection tests. The first two tests are 1. based on the long-wave window channel (11 μm)bright- f. Ozone Impact on AMSU 18 Radiances ness temperature and is applicable to all the measurements. The impact of ozone lines on AMSU-B radiances was i. A pixel is classified as cloudy if its window channel investigated by John and Buehler (2004). They found that < brightness temperature is too cold ( 210 K). the ozone impact is largest for AMSU 18. This can re- ii. A pixel is classified as cloudy, if the long-wave window duce its brightness temperature, with a reduction that can channel brightness temperature is 4 K cooler than reach a maximum of 0.5 K. It was also reported that zonal, that of the warmest pixel from the target area. monthly mean climatological values of ozone profiles are sufficient to account for the ozone impact on AMSU 18 ra- iii. If any one of the followings conditions fail for night diances. With the above work as a base, temperature and time measurements, then the pixel is classified as humidity profiles from ARM radiosondes were taken up to cloudy. 100 hPa and the rest of the pressure levels up to 0.2 hPa

4 were filled in with climatological ozone and temperature The nighttime comparison results for AMSU 18 and profiles, for AMSU 18 simulations. A water vapor concen- HIRS 12 are summarised in Table 2. Also Figure 2 shows tration of 5 ppmv was assumed above 100 hPa. It is to be a scatter plot of simulated versus observed brightness tem- noted that the temperature and humidity profiles above peratures for AMSU 18 and HIRS 12 before and after cor- 100 hPa have negligible impact on AMSU 18 radiance. So, rection. The empirical correction reduces the bias from it is the presence of ozone that contributes to the impact. 0.63 to -0.1 K for AMSU 18 and from 1.26 K to 0.35 K for Climatological ozone and temperature profiles from Total HIRS 12. The reduction in the bias is higher for HIRS 12 Ozone Mapping Spectrometer (TOMS v8) which are given (0.91K)incomparisontoAMSU18 (0.73 K). The results for each 10 degree latitude bin and month were used to of daytime analysis are presented in Table 3. In contrast fill in the radiosondes above 100 hPa. In the case of the to the night time biases, the daytime biases in the uncor- HIRS 12 simulations, we have restricted the temperature rected datasets are much higher (2.38 K in AMSU 18 and and humidity profiles from ARM radiosondes to 90 hPa 2.51 K in HIRS 12). As a result, the empirical correction as the profiles above this level have negligible impact on on daytime datasets has a profound influence in reducing HIRS 12 radiances. Ozone has no influence on HIRS 12 the bias in AMSU 18 and HIRS 12. Accordingly, the bias simulations. reduction is of the order of 2.1 K for AMSU 18 and 1.92 K for HIRS 12. This clearly is an indication that the cor- 3. Results and Discussion rections are especially required for all daytime radiosonde observations which are subject to solar radiation error. Re- The radiosonde-satellite comparison results in terms of call that in addition to the mean bias correction as applied bias for AMSU 18 and HIRS 12 are discussed in this sec- in nighttime dataset, the daytime correction also includes a tion. The results of daytime and nighttime comparisons are correction for solar radiation error which is most significant presented separately. The bias is calculated as the mean of here. the difference between the simulated and measured bright- The bias as a function of brightness temperature binned ness temperature, so a positive bias means that the ra- at 5 K intervals for day and nighttime dataset is shown in diosonde measurements are dry and vice versa. The same Figure 3 . The reduction in bias due to correction is evi- convention is followed throughout the discussion that fol- dent in all brightness temperature ranges which represent lows. varying humidity conditions in the upper troposphere from The bias value can be used to determine the relative dif- wet (low brightness temperature) to dry (high brightness ference in the UTH measurements between satellite and ra- temperature). The impact of empirical correction on the diosonde. Applying the transformation coefficients (bHIRS 12 radiosonde profiles themselves can be inferred from Figure (0.114) and bAMSU 18 (0.073) ) in equation 3, it can be in- 4, showing the mean RH of all the profiles before and after ferred that a 1 K bias in AMSU 18 corresponds to a rel- correction. The magnitude of correction is much higher for ative error of 7% in UTH where as for HIRS 12, it corre- the daytime data as compared to nighttime. It is the solar sponds to a relative difference of 11%. Even though both radiation correction rather than the mean calibration bias AMSU 18 and HIRS 12 measure UTH, their sensitivity lies correction which accounts for this large difference in the at slightly different altitudes. As shown in Figure 1, the daytime dataset. The bias correction for SRE accounts peak of HIRS 12 sensitivity lies approximately 2 km higher for more than 50% of the bias reduction in the daytime in altitude than for AMSU 18. These differences in sens- dataset. It is also worthwhile to mention that the magni- ing height will lead to a bias rather than a random error. tude of the correction is dependent on different humidity Knowing the fact that radiosonde bias is altitude depen- conditions in the upper troposphere and is higher under dent (Miloshevich et al. 2009), the bias for HIRS 12 and drier conditions for the nighttime dataset while in the day- AMSU 18 could be different as the sounding altitudes of time dataset it is higher under moist conditions with the the two instruments are not the same. exception of some parts of the upper troposphere (Figure As a separate study to show the impact of ozone in 5). These results highlight the importance of corrected ra- AMSU 18 simulations, we have calculated the bias with diosonde profiles for satellite validation. In addition, the and without the inclusion of ozone. The nighttime un- results also reveal that the empirical correction is quite corrected radiosonde datasets were alone chosen for this effective in reducing the dry bias of radiosondes in the up- study. The results reveal that the inclusion of ozone im- per troposphere and leads to a better agreement between proved the bias to 0.63 K from a bias of 0.99 K. Obviously, HIRS 12 and AMSU 18. here the inclusion of ozone reduces the AMSU 18 bright- As mentioned earlier, we have used a different algo- ness temperature by 0.35 K. This is a clear confirmation rithm than the published algorithm for mean calibration that inclusion of ozone leads to a better agreement between bias correction. In the night time dataset, the correc- satellite and radiosonde measured UTH as argued by John tion with the new algorithm results in a bias of 0.35 K and Buehler (2004). in HIRS 12 whereas with the published algorithm it is 0.38

5 K, but for AMSU 18 these values are -0.07 K and -0.10 before correction in nighttime dataset was around 10% and K respectively. For the daytime dataset, correction with it was lowered to approximately 5% after correction. Sim- the new algorithm yields a bias of 1.71 K in HIRS 12 and ilarly, the discrepancy in the uncorrected daytime dataset a bias of 1.75 K with the published algorithm, while in was 12% which reduces to 4.7% after correction. These re- AMSU 18 these values are 1.76 K and 1.78 K respectively. sults show that the radiosonde correction is very important Though these differences are small the use of the new al- in bringing about a better agreement between satellite and gorithm is recommended for mean calibration correction radiosonde measurements of UTH. since the temperature dependence is more consistent with There have been a number of earlier correction meth- actual sensor calibration than the pressure dependence as ods for improving the accuracy of RS80 radiosonde sensors used in the published algorithm. (Turner et al. 2003; Miloshevich et al. 2001). However, So- The impact of the time lag (TL) correction is shown den et al. (2004) showed that these correction procedures in Table 4. This result reveals that the TL correction has offer only a small improvement in the upper troposphere, a negligible impact in reducing the bias in AMSU 18 and though they are effective in the lower troposphere. In con- HIRS 12 because the bias reduction is of the order of only trast to this, our work for the RS92 sensors and Miloshe- 0.03 K. Since TL correction needs profiles of higher res- vich et al. (2009) correction method, shows a significant olution, it is applied here only to a subset of RS92 pro- improvement, particularly, in the upper troposphere. The files. A noteworthy observation is that the bias reduction present study therefore may open up the possibility of ap- in AMSU 18 and HIRS 12 due to empirical correction on plying a similar correction approach to other radiosonde this subset is the same as that for all the RS92 profiles (not sensors. shown). Though the agreement between HIRS 12 and AMSU 18 4. Summary and Conclusions has been improved due to correction, a small discrepancy The present study evaluates the importance of radiosonde of the order of 0.45 K for nighttime and 0.31 K for day- correction in improving the agreement between satellite time data still remains. One of the major reasons for the and radiosonde measured UTH. The satellite measurements discrepancy can be attributed to the uncertainty in the from infrared (HIRS 12) and microwave (AMSU 18) in- corrected RS92 measurements. Miloshevich et al. (2009) struments were used for this purpose. The satellite mea- have shown that the bias uncertainty is ±(4% + 0.5 %RH), surements were collocated with ARM radiosonde profiles. which implies 4% of the measured value plus a 0.5 %RH The satellite-radiosonde comparison was performed by sim- offset component that is increasingly important for drier ulating the radiances from collocated radiosonde profiles conditions. For example, the relative uncertainty in the using the radiative transfer model ARTS. The compar- corrected RS92 data is ±5% for conditions of 50 %RH, isons were performed only for clear sky conditions. The ±6.5% at 20 %RH, ±9% at 10 %RH, and ±20% at 3 %RH. empirical correction procedure applied to RS92 radiosonde Therefore, there could be a larger uncertainty in the cor- sensors was found to have a significant impact in reducing rection occurring at the HIRS 12 sensing altitude than at the bias in AMSU 18 and HIRS 12. The nighttime bias in the AMSU 18 sensing altitude as it senses a drier atmo- AMSU 18 (from 0.63 K to -0.10 K) was reduced by 0.73 K sphere. A part of the difference could also be attributed to and by 0.91 K in HIRS 12 (from 1.26 K to 0.35 K). During error in the radiative transfer modeling. However, based daytime, the decrease in bias is significantly higher with a on the comparison to LBLRTM described in Appendix A, reduction of 2.1 K (from 2.38 K to 0.28 K) in AMSU 18 we expect this error to be smaller than 0.3 K. Systematic and 1.92 K (from 2.51 K to 0.59 K) in HIRS 12. This is on error in the satellite measurements which is of the order account of the bias removal associated with the solar radi- of 0.5 K in AMSU 18 and HIRS 12 on NOAA-17 could ation error. The correction also improved the consistency also play a role (Shi and Bates 2011; John et al. submitted between AMSU 18 and HIRS 12 and reduced the difference 2011). It is difficult to attribute the remaining discrep- between them by 5% in nighttime and by 7% in daytime ancy between AMSU 18 and HIRS 12 to any single cause datasets. as it a very small bias. Therefore, it is only fair to say Besides showing the importance of radiosonde correc- that the radiosonde, radiative transfer modelling and sys- tion for satellite validation, this study also cautions against tematic error in AMSU/HIRS measurements are equally the use of uncorrected radiosonde profiles for various water plausible contributors to the bias. vapour related studies in the upper troposphere. The need Coming back to the subject of radiosonde correction, for such corrections is especially due to the imperfect char- one can state that in addition to significantly reducing the acterization and calibration of the sensor output by the bias between satellite and radiosonde measurements, em- manufacturer. The occurrence of a similar or worse type pirical correction also brings forth a better agreement be- of imperfection as addressed here for Vaisala radiosondes tween AMSU 18 and HIRS 12. The discrepancy in the cannot be ruled out for many other operational radioson- apparent radiosonde bias between HIRS 12 and AMSU 18

6 des. In any case, accuracy is one aspect that cannot be zenith opacity and the SRF for HIRS 12 is shown in Figure ignored or compromised in climate related studies. The 6. In the normal line by line calculation mode, ARTS simu- correction method developed by Miloshevich et al. (2009) lates the radiances within the pass band of the sensor with and implemented in this study is an example as to how a high spectral resolution. These radiances are convolved these errors can be reduced to a significant extent. with the SRF to obtain the radiance of the HIRS channel. For n frequency grid points within the passband of the jth Acknowledgments. HIRS channel, the convolved radiance R(j)isgivenby The authors are thankful to Lisa Neclos at NOAA for n w r R(j)= i=1 i i (A1) providing access to NOAA-17 level 1B data. Thanks to n w the U.S. Department of Energy, Office of Science, Office i=1 i of Biological and Environmental Research, Climate and where ri is the radiance for the ith frequency grid point and Environmental Sciences Division for facilitating the use wi is the associated weight. The weights for the high res- of Radiosonde data obtained from their dedicated Atmo- olution frequency grids are obtained through interpolation spheric Radiation Measurement (ARM) Program. Thanks from the sensor specified weights defined for the frequen- are also due to the ARTS radiative transfer community. cies within the SRF. However, the fast set up for HIRS in We acknowledge the LBLRTM community for providing ARTS uses only a few frequencies to simulate the entire the code. Special thanks to Eli Mlawer, Oliver Lemke and HIRS channel radiance. These representative frequencies Isaac Moradi. Viju John has been supported by the U.K. are selected using a simulated annealing method (Buehler Joint DECC and DEFRA Met Office Hadley centre Cli- et al. 2010). In the fast setup for HIRS simulations, the mate Programme-GA01101. This work also contributes to weights for R(j ) are derived using multiple linear regres- COST Action ES604Water Vapour in the Climate System sion over a set of line by line calculations using an ensemble (WaVaCS). The work of the Kiruna group was funded by of atmospheric conditions. Swedish Space Board, grant, ”Satellite Atmospheric Sci- LBLRTM generates monochromatic infrared radiances ence”. We thank our three anonymous reviewers for their with high spectral resolution for a user specified frequency useful comments which have resulted in improving the con- range. It has no built-in option to consider the SRF or tent and quality of the paper. other instrumental properties. Thus to allow comparison with ARTS, the HIRS channel radiances have been gen- APPENDIX A erated by convolving the high resolution spectra with the respective SRF. The Planck function is used for the con- Model-Inter Comparison version of convolved radiance to brightness temperature. During the conversions, care was taken to ensure that the transformations were applied to both the models in the This is a comparison of results from the ARTS and same manner. Simulations were performed assuming clear LBLRTM forward radiance models. LBLRTM is a line sky conditions and nadir viewing geometry with a surface by line radiative transfer model (Clough et al. 2005). It emissivity of 1. The statistics of comparison for different has been validated against atmospheric radiance spectra NOAA satellites sensors are summarised in Table 5. from the ultraviolet to the sub-millimeter wavelength range (Turner et al. 2004). In this study infrared radiances were simulated using LBLRTM version 11.6 which uses the modified water vapour continuum model MT CKD 2.4. We verified that continuum changes are negligible in the frequency range where HIRS 12 is located, since ARTS uses MT CKD 1.0 as the continuum model. The line parameters are the same as for ARTS (HITRAN 2004). Model inter-comparisons were performed for the HIRS water vapour channel (HIRS 12) of NOAA satellites 14, 15, 16 and 17. The simulations were based on a subset of the diverse set of atmospheric profiles from European Centre for Medium-Range Weather Forecasts (ECMWF) re-analysis data (Chevallier et al. 2006). The number of used profiles is 4700. Each channel in HIRS has an individual sensor response function (SRF), specifying the weight as a function of frequency within the spectral range of the channel. The H2O

7 1 10 AMSU_18 HIRS_12

2 10 Pressure [hPa]

3 10 −0.05 0 0.05 0.1 0.15 0.2 Jacobian [K/1]

Fig. 1. Jacobian of AMSU 18 and HIRS 12 UTH channels for the standard tropical atmosphere (Garand et al. 2001) with a nadir viewing geometry. Jacobian values are divided by their sum.

8 Table 1. The regression coefficients a and b used to derive UTH from HIRS 12 for its different viewing angles. The UTH is defined with respect to liquid water. The angle θ is in degrees, a is dimensionless, and b is in K−1.

θ ab 0.900 29.554 -0.110 2.700 29.556 -0.110 4.500 29.560 -0.110 6.300 29.566 -0.110 8.100 29.574 -0.110 9.900 29.584 -0.110 11.700 29.596 -0.110 13.500 29.610 -0.110 15.300 29.626 -0.111 17.100 29.645 -0.111 18.900 29.667 -0.111 20.700 29.691 -0.111 22.500 29.718 -0.111 24.300 29.748 -0.111 26.100 29.781 -0.112 27.900 29.818 -0.112 29.700 29.859 -0.112 31.500 29.904 -0.112 33.300 29.955 -0.113 35.100 30.011 -0.113 36.900 30.073 -0.113 38.700 30.143 -0.114 40.500 30.222 -0.114 42.300 30.311 -0.115 44.100 30.412 -0.116 45.900 30.529 -0.116 47.700 30.666 -0.117 49.500 30.828 -0.118

9 280 280 Mean Bias = 0.63 K Mean Bias = 1.26 K 270 Slope = 1.029 270 Slope = 1.054 cc= 0.97 cc= 0.97 260 260

250 250

240 240

230 Simulated HIRS_12 [K] 230 Simulated AMSU_18 [K]

220 220 220 240 260 280 220 240 260 280 Satellite AMSU_18 [K] Satellite HIRS_12 [K]

280 280 Mean Bias = −0.10 K Mean Bias = 0.35 K 270 Slope = 1.045 270 Slope = 1.052 cc= 0.97 cc= 0.97 260 260

250 250

240 240

230 Simulated HIRS_12 [K] 230 Simulated AMSU_18 [K]

220 220 220 240 260 280 220 240 260 280 Satellite AMSU_18 [K] Satellite HIRS_12 [K]

Fig. 2. Scatter plots of simulated versus observed brightness temperature for AMSU 18 and HIRS 12 before (upper panels) and after empirical correction (lower panels, corrected RS92) for the nighttime data. The black and red lines are diagonal and regression lines respectively. The number of datapoints is 624 for each plot.

1.5 2.5 Corrected 1 Original 2 1.5 0.5 1

Bias [K] 0 Bias [K] 0.5

−0.5 0

−1 −0.5 230 240 250 260 270 220 230 240 250 260 AMSU_18 [K] HIRS_12 [K]

3 3

2.5 2 2

1 1.5 Bias [K] Bias [K] 1 0 0.5

−1 0 230 240 250 260 270 220 230 240 250 260 AMSU_18 [K] HIRS_12 [K]

Fig. 3. Bias as a function of brightness temperature binned at 5 K intervals before (red) and after (blue) Empirical correction for AMSU 18 (left) and HIRS 12 (right). The vertical bars represent the standard error. Upper panels are for nighttime data, lower panels are for daytime data.

10 2 2 10 10 Original Corrected Pressure [hPa] Pressure [hPa]

3 3 10 10 0 20 40 60 −20 0 20 40 Relative Humidity (%RH) Difference (%)

2 2 10 10 Pressure [hPa] Pressure [hPa]

3 3 10 10 0 20 40 60 −50 0 50 100 Relative Humidity (%RH) Difference (%)

Fig. 4. Left: The mean RH of RS92 radiosonde sensors before (black) and after Empirical correction. Right: The thick line represents mean of the percentage difference between the corrected and the original profiles. The horizontal bars represent the standard deviation. For the figure RH profiles from ARM radiosondes before and after the corrections were interpolated to specific pressure levels (53) ranging from 980 to 100 hPa. The Percentage difference is defined as corrected−original × original 100%. Upper panels are for nighttime data, lower panels are for daytime data.

2 2 10 10 Dry Moist Moderate Pressure [hPa] Pressure [hPa]

3 3 10 10 −10 0 10 20 30 0 20 40 60 Difference (%) Difference (%)

Fig. 5. The percentage difference between the corrected and original profiles classified in terms of different humidity conditions in the upper troposphere (dry, moist and moderate). Classification is based on the percentiles of the observed brightness temperature in HIRS 12 such that values ≥ 75 percentile are considerd as dry, ≤ 25 percentile moist and values ranging between these as moderate. Left plot is for nighttime data and the right plot is for daytime data.

11 5 10 0.2

4 10 0.16

3 10 0.12 Opacity

2 10 0.08 Channel Response

1 10 0.04

0 1480 1490 1500 1510 1520 1530 1540 1550 1560 1570 1580 Wavenumber [cm−1]

Fig. 6. The H2O zenith opacity spectrum (blue) and the SRF for HIRS 12 (black) on NOAA-17.

12 Table 2. Statistics comparing satellite radiance and simulated radiance in brightness temperature units for the nighttime datset. The bias is defined as the mean of simulated minus measured data. SD refers to the standard deviation. The number of profiles used is 624. Suffix bc and ac refers to before and after empirical correction.

Channel Bias (K) Bias in UTH (%) SD (K) SD UTH (%) AMSU 18bc 0.63 -4.5 1.66 11.62 AMSU 18ac -0.10 0.7 1.68 11.76 HIRS 12bc 1.26 -14.3 1.40 16 HIRS 12ac 0.35 -3.9 1.42 16.18

Table 3. Statistics comparing satellite radiance and simulated radiance in brightness temperature units for the daytime dataset. The bias is defined as the mean of simulated minus measured data. SD refers to the standard deviation. The number of profiles used is 419. Suffix bc and ac refers to before and after empirical correction.

Channel Bias (K) Bias in UTH (%) SD (K) SD UTH (%) AMSU 18bc 2.38 -16.6 1.31 9.17 AMSU 18ac 0.28 -1.96 1.39 9.73 HIRS 12bc 2.51 -28.61 1.19 13.57 HIRS 12ac 0.59 - 6.73 1.27 14.48

Table 4. Statistics comparing satellite radiance and simulated radiance in brightness temperature units for RS92 sensors. The bias is defined as the mean of simulated minus measured data. SD refers to the standard deviation. The number of profiles used is 85. Suffix btl and atl refers to before and after time lag correction.

Channel Bias (K) Bias in UTH (%) SD (K) SD UTH (%) AMSU 18btl 0.14 -0.9 1.52 10.64 AMSU 18atl 0.17 -1.2 1.51 10.57 HIRS 12btl 0.94 -10.7 1.21 13.80 HIRS 12atl 0.91 -10.4 1.19 13.56

Table 5. Statistics comparing ARTS and LBLRTM simulations. The bias is deﬁned as the mean of the diﬀerence ARTS minus LBLRTM. SD refers to the standard deviation. Satellite Channel Bias (K) SD (K) NOAA-14 HIRS 12 -0.07 0.15 NOAA-15 HIRS 12 -0.21 0.23 NOAA-16 HIRS 12 -0.17 0.22 NOAA-17 HIRS 12 -0.21 0.22

13 Deuber, B., J. Morland, L. Martin, and N. Kämpfer, 2005: Deriving the tropospheric integrated water vapor from REFERENCES tipping curve-derived opacity near 22 GHz. Radio Sci., 40, RS5011, doi:10.1029/2007RG000233. Buehler, S. A., P. Eriksson, T. Kuhn, A. von Engeln, and C. Verdes, 2005: ARTS, the atmospheric radiative Elliott, W. P. and D. J. Gaffen, 1991: On the Utility of transfer simulator. J. Quant. Spectrosc. Radiat. Trans- Radiosonde Humidity Archives for Climate Studies. Bul- fer, 91 (1), 65–93, doi:10.1016/j.jqsrt.2004.05.051. letin of the American Meteorological Society, 72, 1507– 1520, doi:10.1175/1520-0477(1991)0721507:OTUORH Buehler, S. A., P. Eriksson, and O. Lemke, 2011: Absorp- 2.0.CO;2. tion lookup tables in the radiative transfer model arts. J. Quant. Spectrosc. Radiat. Transfer, 112 (10), 1559– Emde, C., S. Buehler, C. Davis, P. Eriksson, T. Sreerekha, 1567, doi:10.1016/j.jqsrt.2011.03.008. and C. Teichmann, 2004: A polarized discrete or- dinate scattering model for simulations of limb and Buehler, S. A. and V. O. John, 2005: A simple method nadir long-wave measurements in 1-D/3-D spher- to relate microwave radiances to upper tropospheric hu- ical atmospheres. JOURNAL OF GEOPHYSICAL midity. J. Geophys. Res., 110, D02110, doi:10.1029/ RESEARCH-ATMOSPHERES, 109 (D24), doi:{10. 2004JD005111. 1029/2004JD005140}. Buehler, S. A., V. O. John, A. Kottayil, M. Milz, and Eriksson, P., S. A. Buehler, C. P. Davis, C. Emde, and P. Eriksson, 2010: Efficient radiative transfer simula- O. Lemke, 2011: ARTS, the atmospheric radiative trans- tions for a broadband infrared radiometer — combin- fer simulator, version 2. J. Quant. Spectrosc. Radiat. ing a weighted mean of representative frequencies ap- Transfer, 112 (10), 1551–1558, doi:10.1016/j.jqsrt.2011. proach with frequency selection by simulated annealing. 03.001. J. Quant. Spectrosc. Radiat. Transfer, 111 (4), 602–615, doi:10.1016/j.jqsrt.2009.10.018. Eriksson, P., C. Jiménez, D. Murtagh, G. Elgered, T. Kuhn, and S. Buehler, 2003: Measurement of tro- Buehler, S. A., M. Kuvatov, V. O. John, U. Leiterer, and pospheric/stratospheric transmission at 10–35 GHz for H. Dier, 2004: Comparison of microwave satellite humid- H2O retrieval in low earth orbiting satellite links. Radio ity data and radiosonde profiles: A case study. J. Geo- Sci., 38 (4), 8069, doi:10.1029/2002RS002638. phys. Res., 109, D13103, doi:10.1029/2004JD004605. Fiorucci, I., et al., 2008: Measurements of low amounts Buehler,S.A.,M.Kuvatov,V.O.John,M.Milz,B.J. of precipitable water vapor by millimeter wave spec- Soden, D. L. Jackson, and J. Notholt, 2008: An upper troscopy: An intercomparison with radiosonde, raman tropospheric humidity data set from operational satellite lidar, and fourier transform infrared data. J. Geophys. microwave data. J. Geophys. Res., 113, D14110, doi: Res., 113, D14314, doi:10.1029/2008JD009831. 10.1029/2007JD009314. Garand, L. et al., 2001: Radiance and jacobian intercom- Chevallier, F., S. Di Michele, and A. P. McNally, 2006: Di- parison of radiative transfer models applied to HIRS and verse profile datasets from the ECMWF 91-level short- AMSU channels. J. Geophys. Res., 106 (D20), 24,017– range forecasts. Tech. rep., NWP SAF Satellite Appli- 24,031. cation Facility for Numerical Weather Prediction. Doc- ument No. NWPSAF-EC-TR-010, Version 1.0. Held, I. M. and B. J. Soden, 2000: Water vapor feedback and global warming. Annu. Rev. Energy Environ., 25, Clough, S. A., M. W. Shephard, E. J. Mlawer, J. S. De- 441–475. lamere, M. J. Iacono, K. Cady-Pereira, S. Boukabara, and P. D. Brown, 2005: Atmospheric radiative trans- John, V. O. and S. A. Buehler, 2004: The impact of ozone 31 fer modeling: a summary of the AER codes. Journal of lines on AMSU-B radiances. Geophys. Res. Lett., , Quantitative Spectroscopy and Radiative Transfer, 91, L21108, doi:10.1029/2004GL021214. 233–244, doi:10.1016/j.jqsrt.2004.05.058. John, V. O. and S. A. Buehler, 2005: Comparison of mi- Davis, C., C. Emde, and R. Harwood, 2005: A 3D po- crowave satellite humidity data and radiosonde profiles: 5 larized reversed monte carlo radiative transfer model for A survey of European stations. Atmos. Chem. Phys., , mm and sub-mm passive remote sensing in cloudy atmo- 1843–1853, doi:10.5194/acp-5-1843-2005, sRef-ID:1680- spheres. IEEE T. Geosci. Remote, 43 (5), 1096–1101, 7324/acp/2005-5-1843. doi:10.1109/TGRS.2004.837505. John, V. O., G. Holl, S. A. Buehler, B. Candy, R. W. Saun- ders, and D. E. Parker, submitted 2011: Understanding

14 inter-satellite biases of microwave humidity sounders us- Rosenkranz, P. W., 1998: Water vapor microwave con- ing global SNOs. J. Geophys. Res. tinuum absorption: A comparison of measurements and models. Radio Sci., 33 (4), 919–928, (correction in 34, John, V. O. and B. J. Soden, 2007: Temperature and hu- 1025, 1999), ftp://mesa.mit.edu/phil/lbl_rt. midity biases in global climate models and their impact on climate feedbacks. Geophys. Res. Lett., 34, L18704, Rothman, L. S. et al., 2003: The HITRAN molecular spec- doi:10.1029/2007GL030429. troscopic database: edition of 2000 including updates through 2001. J. Quant. Spectrosc. Radiat. Transfer, 82, Kiehl, J. T. and B. P. Briegleb, 1992: Comparison of the 5–44. observed and calculated clear sky greenhouse implications for climate studies. J. Geophys. Res., 97, 10 037– Saunders, R., et al., 2007: A comparison of radiative trans- 10 049. fer models for simulating atmospheric infrared sounder (airs) radiances. J. Geophys. Res., 112, D01S90, doi: McMillin, L. M. and C. Dean, 1982: Evaluation of a new 10.1029/2006JD007088. operational technique for producing clear radiances. J. Appl. Meteorol., 21, 1005–1014. Shi, L. and J. J. Bates, 2011: Three decades of intersatellite-calibrated High-Resolution Infrared Radi- Melsheimer, C., et al., 2005: Intercomparison of general ation Sounder upper tropospheric water vapor. J. Geo- purpose clear sky atmospheric radiative transfer models phys. Res., 116, D04108, doi:10.1029/2010JD014847. for the millimeter/submillimeter spectral range. Radio Sci., RS1007, doi:10.1029/2004RS003110. Soden, B. J. and F. P. Bretherton, 1993: Upper tropospheric relative humidity from the GOES 6.7 μm chan- Miloshevich, L. M., A. Paukkunen, H. Vömel, and S. J. nel: Method and climatology for july 1987. J. Geophys. Oltmans, 2004: Development and validation of a time- Res., 98 (D9), 16,669–16,688. lag correction for Vaisala radiosonde humidity measurement. J. Atmos. Oceanic Technol., 21, 1305–1327, doi: Soden, B. J. and J. R. Lanzante, 1996: An assess- 10.1175/1520-0426(2004)0211305:DAVOAT2.0.CO;2. ment of satellite and radiosonde climatologies of upper- tropospheric water vapor. J. Climate, 9, 1235–1250. Miloshevich, L. M., H. Vömel, A. Paukkunen, A. J. Heymsfield, and S. J. Oltmans, 2001: Characterization Soden, B. J., D. D. Turner, B. M. Lesht, and L. M. Miloshe- and correction of relative humidity measurements from vich, 2004: An analysis of satellite, radiosonde, and lidar vaisala RS80-A radiosondes at cold temperatures. J. At- observations of upper tropospheric water vapor from the mos. Oceanic Technol., 18 (2), 135–156, doi:10.1175/ atmospheric radiation measurement program. J. Geo- 1520-0426(2001)0180135:CACORH2.0.CO;2. phys. Res., 109, D04105, doi:10.1029/2003JD003828. Miloshevich, L. M., H. Vömel, D. N. Whiteman, and Turner, D. D., B. M. Lesht, S. A. Clough, J. C. Liljegren, T. Leblanc, 2009: Accuracy assessment and correc- H. E. Revercomb, and D. C. Tobin, 2003: Dry bias and tion of vaisala RS92 radiosonde water vapor measure- variability in vaisala RS80-H radiosondes: The ARM ex- ments. J. Geophys. Res., 114, D11 305, doi:10.1029/ perience. J. Atmos. Oceanic Technol., 20, 117–132. 2008JD011565. Turner, D. D., et al., 2004: The QME AERI LBLRTM: Mlawer, E. J., S. A. Clough, and D. C. Tobin, 2003: A new A Closure Experiment for Downwelling High Spectral water vapor continuum model: MT CKD 1.0. Tech. rep., Resolution Infrared Radiance. Journal of Atmospheric University of Winsconsin. Sciences, 61, 2657–2675, doi:10.1175/JAS3300.1. Moradi, I., S. A. Buehler, V. O. John, and S. Elias- Vömel, H., D. E. David, and K. Smith, 2007a: Accuracy son, 2010: Comparing upper tropospheric humidity of tropospheric and stratospheric water vapor measure- data from microwave satellite instruments and tropi- ments by the cryogenic frost point hygrometer: Instru- cal radiosondes. J. Geophys. Res., 115, D24310, doi: mental details and observations. J. Geophys. Res., 112, 10.1029/2010JD013962. 1–14. Rosenkranz, P. W., 1993: Absorption of microwaves by Vömel, H., et al., 2007b: Radiation dry bias of the atmospheric gases. Atmospheric remote sensing by mi- Vaisala RS92 humidity sensor. J. Atmos. Oceanic Tech- crowave radiometry, M. A. Janssen, Ed., John Wi- nol., 24 (6), 953–963, doi:10.1175/JTECH2019.1. ley and Sons, Inc., 37–90, ISBN: 0-471-62891-3, ftp: //mesa.mit.edu/phil/lbl_rt.

The end.