SCIAMACHY Offline Processor Level1b-2 ATBD

SCIAMACHY Ofﬂine Processor Level1b-2 ATBD Algorithm Theoretical Baseline Document (SGP OL Version 7)

Issue 3

prepared institute date signature G. Lichtenberg, S. DLR-IMF 16.04.18 Hrechanyy checked SQWG copies to ESA Contents

I. Introduction9

1. About this document 10 1.1. Purpose and scope...... 10 1.2. Historical Background...... 10 1.3. Acronyms and Abbreviations...... 10 1.4. Document Overview...... 11

2. The Data Processor 12 2.1. Processing ﬂow overview...... 12 2.2. Components of the Data Processor...... 12 2.2.1. Database Server...... 13 2.2.2. Nadir Retrieval Server...... 13 2.2.3. Limb Retrieval Server...... 13 2.3. Overview of MDS...... 13

II. Nadir Retrieval Algorithms 14

3. Cloud Parameters 15 3.1. Cloud Top Height and Cloud Optical Thickness...... 15 3.1.1. Overview...... 15 3.1.2. Detailed Description...... 15 3.1.2.1. Cloud Top Height...... 15 3.1.2.2. Cloud Optical Thickness...... 16 3.2. Cloud Fraction...... 17 3.2.1. Introduction...... 17 3.2.2. Generation of the cloud-free composite...... 18 3.2.3. Determination of the cloud fraction...... 19 3.2.4. Future improvements...... 19

4. Absorbing Aerosol Index 20 4.1. Overview...... 20 4.1.1. Retrieval Settings...... 20 4.2. Detailed Description...... 20 4.2.1. Definition of used geometry...... 20 4.2.2. Definition of the Residue...... 21 4.2.3. Definition of the AAI...... 21 4.2.4. Calculation of the Residue...... 22 4.2.5. Look-up Tables...... 22 4.2.6. Degradation Correction...... 22

5. SDOAS Algorithm General Description 24 5.1. Slant column retrieval...... 24 5.2. Vertical column retrieval...... 25 5.3. Nadir Ozone Total Column Retrieval...... 26 5.3.1. Motivation...... 26 5.3.2. Retrieval Settings...... 27

1 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

5.3.3. Additional Details...... 27 5.3.3.1. Molecular Ring correction...... 27 5.3.3.2. Vertical column with iterative approach...... 28 5.4. Nadir NO2 Total Column Retrieval...... 29 5.4.1. Motivation...... 29 5.4.2. Retrieval Settings...... 29 5.5. Nadir SO2 Total Column Retrieval...... 30 5.5.1. Retrieval Settings...... 30 5.5.2. Additional Details...... 30 5.5.2.1. Slant Column...... 30 5.5.2.2. Vertical Column...... 31 5.6. Nadir BrO Total Column Retrieval...... 32 5.6.1. Motivation...... 32 5.6.2. Retrieval Settings...... 32 5.7. Nadir OClO Slant Column Retrieval...... 33 5.7.1. Retrieval Settings...... 33 5.7.2. Additional Details...... 33 5.8. Nadir HCHO Total Column Retrieval...... 33 5.8.1. Motivation...... 33 5.8.2. Retrieval Settings...... 34 5.8.3. Additional Details...... 34 5.8.4. User warning...... 35 5.9. Nadir CHOCHO Total Column Retrieval...... 35 5.9.1. Retrieval Settings...... 35 5.9.2. Additional Details...... 36 5.9.3. User warning...... 36

6. Water Vapour Retrieval 37 6.1. Inversion Algorithm...... 37 6.2. Forward Model and Databases...... 39 6.3. Auxiliary Data...... 39

7. Infra-red Retrieval using BIRRA 40 7.1. Introduction...... 40 7.1.1. A ﬁrst glimpse at near IR nadir observations...... 40 7.2. Forward Modelling...... 43 7.2.1. Radiative Transfer...... 43 7.2.1.1. General Considerations: Schwarzschild and Beer...... 43 7.2.1.2. Radiative transfer modelling of nadir near infra-red observations...... 43 7.2.2. Molecular Absorption...... 44 7.2.2.1. Line strength and partition functions...... 44 7.2.2.2. Line broadening...... 45 7.2.3. Geometry...... 48 7.2.3.1. Uplooking...... 48 7.2.3.2. Downlooking...... 49 7.2.3.3. Refraction...... 50 7.2.4. Instrument - Spectral Response...... 51 7.2.5. Input data for forward model...... 52 7.3. Inversion...... 55 7.3.1. Non-linear least squares...... 55 7.3.2. Separable non-linear least squares...... 55 7.4. CO Retrieval...... 56 7.4.1. Detailed Description...... 56 7.4.2. Retrival Settings...... 57 7.5. CH4 Retrieval...... 57 7.5.1. Detailed Description...... 57

SGP OL1b-2 ATBD Version 7 Page 2 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

7.5.2. Retrieval Settings...... 57

III. Limb Retrieval Algorithms 58

8. Limb Algorithm General Description 59 8.1. Summary of Retrieval Steps...... 59 8.2. Forward model radiative transfer...... 59 8.2.1. General considerations...... 59 8.2.2. Geometry of the scattering problem...... 61 8.2.3. Optical properties...... 67 8.2.3.1. Partial columns...... 67 8.2.3.2. Number density and VMR...... 68 8.2.3.3. Scattering optical depth of air molecules...... 68 8.2.3.4. Absorption optical depth of gas molecules...... 69 8.2.3.5. Scattering optical depth of aerosols...... 69 8.2.3.6. Total extinction optical depth...... 69 8.2.3.7. Phase functions...... 70 8.2.4. Recurrence relation for limb radiance and Jacobian...... 71 8.2.4.1. Recurrence relation for limb radiance...... 72 8.2.4.2. Recurrence relation for the Jacobian matrix...... 74 8.2.4.3. Input parameters for limb radiance and Jacobian calculation...... 77 8.2.5. Pseudo-spherical discrete ordinate radiative transfer equation...... 78 8.2.5.1. Requirement for a pseudo-spherical model...... 78 8.2.5.2. General considerations...... 79 8.2.5.3. Integral form of the layer equation...... 80 8.2.5.4. Layer equation...... 81 8.2.5.5. Matrix eigenvalue method...... 81 8.2.5.6. Padé approximation...... 84 8.2.5.7. System matrix of the entire atmosphere...... 85 8.2.5.8. Weighting functions calculation...... 86 8.2.6. Picard iteration...... 89 8.3. Inversion methods...... 92 8.3.1. Tikhonov regularization...... 93 8.3.1.1. Inversion models...... 93 8.3.1.2. Optimization methods for the Tikhonov function...... 94 8.3.1.3. Practical methods for computing the new iterate...... 100 8.3.1.4. Error characterisation...... 103 8.3.1.5. Parameter choice methods...... 108 8.4. Relation Number Density and VMR...... 112 8.5. Common Characteristics of the Profile retrieval...... 114 8.6. Limb Ozone Profile Retrieval Settings...... 115 8.7. Limb NO2 Profile Retrieval Settings...... 115 8.8. Limb BrO Profile Retrieval Settings...... 116

9. Limb Cloud Retrieval 117 9.1. Background...... 117 9.1.1. Motivation...... 117 9.1.2. Theory...... 117 9.1.3. Product description...... 118 9.2. Algorithm Description...... 118 9.2.1. Physical Considerations...... 118 9.2.2. Mathematical Description of the Algorithm...... 119 9.3. Application and Determination of Cloud Types...... 120 9.3.1. Tropospheric Clouds...... 120 9.3.2. Polar Stratospheric Clouds (PSCs)...... 121

SGP OL1b-2 ATBD Version 7 Page 3 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

9.3.3. Noctilucent Clouds (NLCs)...... 122 9.4. Summary and Implementation...... 123 9.4.1. Exception Rules...... 123

IV. Tropospheric Products Algorithms 125

10.Tropospheric NO2 126 10.1.Motivation...... 126 10.2.Retrieval Settings...... 126

11.Tropospheric BrO 129 11.1.Motivation...... 129 11.2.Retrieval Settings...... 129 11.2.1. Splitting of the tropospheric and tropospheric parts...... 129

Bibliography 133

SGP OL1b-2 ATBD Version 7 Page 4 of 137 List of Figures

4.1. Deﬁnition of viewing and solar angles...... 20

6.1. Example for AMC-DOAS fit results. Top: Spectral fit. Bottom: Residual ...... 38 6.2. Typical slant optical depths of water vapour and O2 in the spectral region around the AMC-DOAS fitting window...... 38

6.3. Example for RTM parameters τO2 , b and c...... 39

7.1. Atmospheric transmission in channel 8. MODTRAN band model, US standard atmosphere, vertical path 0 – 100km, no aerosols. The yellow bars indicate the micro-window 4282.686 – 4302.131 cm−1 as defined by the WFM–DOAS bad/dead pixel mask...... 41 7.2. Comparison of vertical transmission (0 – 100km) assuming various atmospheric models. .... 42 7.3. Comparison of vertical transmission assuming various aerosol models...... 42 7.4. Half widths (HWHM) for Lorentz-, Doppler- and Voigt-Profile as a function of altitude for a variety of line positions ν. The Lorentz width is essentially proportional to pressure and hence decays approximately exponentially with altitude. In contrast the Doppler width is only weakly altitude dependent. In the troposphere lines are generally pressure broadened, the transition to the Doppler regime depends on the spectral region. The dotted line indicated atmospheric temperature. (Pressure and temperature: US Standard atmosphere, molecular mass 36 amu). . 46 7.5. Observation geometry for SCIAMACHY nadir...... 48 7.6. Geometry of uplooking path...... 49 7.7. Geometry of downlooking path...... 50 7.8. Refraction for an uplooking path ...... 51 7.9. Water lines in SCIAMACHY channel 8 — Inter-comparison of recent Hitran 2004 and Geisa 2003 line data and corresponding cross sections. (H2O data in Hitran 2000 are approximately identical to Geisa)...... 52 7.10.Temperature dependence of molecular spectroscopic lines in SCIAMACHY channel 8 (Hitran 2004 database) indicated by the length of the vertical “error” bars...... 53 7.11.Molecular spectroscopic lines in SCIAMACHY channel 8 according to the Geisa database. ... 53 7.12.Temperature profiles and volume mixing ratios listed in the AFGL data set...... 54 7.13.Solar irradiance ...... 54

8.1. Integration along the line of sight ...... 60 8.2. For a sequence of limb scans characterized by different tangent levels, the solar zenith angle and the zenith angle of the line of sight are scan dependent...... 61 8.3. The Earth coordinate system is such that the X-axis is parallel to the line of sight ΩLOS. The solar coordinate system is such that the Zsun-axis is parallel to the sun direction esun = −Ωsun and the Ysun-axis is parallel to the azimuthal unit vector eϕ1. The spherical coordinate systems (er, eθ, eϕ) and (er, eθ1, eϕ1) are rotated by ϕ0...... 62 8.4. Limb viewing geometry. Top: The point Mp is situated at a distance sp from the point A. The depicted situation corresponds to the case p < p¯, where p¯ is the tangent level index. Bottom: The points on the line of sight are 1, ..., 2¯p − 1, the stages on the line of sight are 1, ..., 2¯p − 2 and the traversed limb layers are 1, ..., p¯ − 1...... 63 8.5. Top: The spherical coordinates of the generic point Mp in the solar coordinate system rlev(p), Φsun,p, Ψsun,p . Note that the near point A is situated in the XsunZsun-plane. Bottom: Spherical angles θLOS,p and ϕLOS,p of ΩLOS in the spherical coordinate system attached to Mp...... 65 8.6. Solar traversed layers in the cases Φsun,p < π/2 (top) and Φsun,p > π/2 (bottom)...... 66 8.7. Stage index p, point indices p and p + 1, layer index lay (p) and the level indices lev (p) and lev (p + 1)...... 71 8.8. Domain of analysis and boundary conditions...... 90

5 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

8.9. Long and short characteristics...... 91

9.1. Illustration of the principle of the cloud top height detection using scattered solar radiation at clouds for two wavelengths in the near infra-red (red and brown darts). SCIAMACHY scans the limb in tangent height steps with a difference ∆TH=3.3 km. The cloud top height CTH is calculated for the tangent height (the vector being perpendicular to the ground and the line-of- sight)...... 117 9.2. Calibrated Limb spectra for the SCIAMACHY channel 4 (top) and channel 6 (bottom) for an example measurement (von Savigny et al.(2005))...... 119 9.3. Taken from (Acarreta et al.(2004))and updated. The blue lines give wavelengths of SCIAMACHY channel 6+ suitable for ice cloud detection. The red dots indicate a spectral behaviour of a pure Rayleigh scattering atmosphere, showing the strong decrease of radiance...... 119 9.4. Colour index profiles of SCIAMACHY Limb radiances (left) and the corresponding colour index ratio profiles (right)...... 120 9.5. Colour index ratios for a SCIAMACHY orbit as a function of latitude and tangent height. The circles depict the cloud top heights. Full cloud coverage is given by solid circles above the ground, while partially cloudy scenes are illustrated by the black rings in the upper part of the plot. In the lower part solid circles depict ice phase clouds...... 121 9.6. Colour index profile (a) and colour index ratios (b) in case of a PSC contaminated measurement (blue) and a background profile (black)...... 122 9.7. Detection of NLC due to the increase in backscattered limb radiance above 80 km (purple) and a background measurement (red)...... 123

10.1.Flow chart of the tropospheric NO2 retrieval algorithm...... 128

11.1.Flow chart of the tropospheric BrO retrieval algorithm...... 131

SGP OL1b-2 ATBD Version 7 Page 6 of 137 List of Tables

8.1. Organization of the system matrix and of the source vector for the entire atmosphere...... 86

7 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18 Changelog

Change Page Section Issue 1 (10.05.2010) all new all new Issue 1A (26.01.2011) Changed: SACURA 3rd bit default value from ’full’ to ’no convergence’ - - Issue 2 (2.09.2014) Spelling errors corrected all all Updated AAIA algorithm description 22 4.2.5 Added SPICI to cloud fraction algorithm description 19 3.2.3 Clariﬁed background database explanation 30 5.5.2 Added section on HCHO 33 5.8 Added section on CHOCHO 35 5.9 Added retrieval settings table to CO section 57 7.4.2

Added section on CH4 57 7.5 Updated Limb Ozone Proﬁle retrieval settings 115 8.6 Added section on NLC detection 122 9.3.3 Corrected the bit notation to big endian - -

Added CH4 to format description - - Issue 2 A (27.03.2015)

Added Section on the Retrieval of tropospheric NO2 126 10 Issue 2 B (13.05.2015)

Added remark about trop. NO2to MDS description 13 2.3 Added SCODA setting table 123 9.4 Issue 3 (16.04.18) Limb Cloud settings updated 117 9 Section about tropospheric BrO added 129 11 Corrected averaging kernel deﬁnition and description for calculation of VMR for Limb 112 8.4 Updates to SWIR chapter 40 7 Removed Appendix with L2 structure and referrences thereto, no longer needed

SGP OL1b-2 ATBD Version 7 Page 8 of 137 Part I.

Introduction

9 1. About this document

1.1. Purpose and scope

SCIAMACHY is a joint project of Germany, the Netherlands and Belgium for atmospheric measurements. SCIAMACHY has been selected by the European Space Agency (ESA) for inclusion in the list of instruments for Earth observation research for the ENVISAT polar platform, which has been launched in 2002. After 10 years in space, the contact to the ENVISAT was lost on the 8th April 2012. On the 9th May 2012, ESA declared the ofﬁcial end of the ENVISAT Mission. The SCIAMACHY programme was under the supervision of the SCIAMACHY science team (SSAG), headed by the Principal Investigators Professor J. P. Burrows (University of Bremen, Germany), Professor I.A.A. Aben (SRON, The Netherlands) and Dr. C. Muller (BIRA, Belgium). The Quality Working Group has been installed in 2007 to intensify the development and implementation of the Algorithm Baseline for the operational data processing system of SCIAMACHY. Current members of the QWG are the University of Bremen (IUP-B) (Lead), BIRA, DLR, and SRON. The expertise of KNMI is brought in via an association with SRON. This document describes the algorithms used in the Level 1b-2 processing of SCIAMACHY data. Focus is the mathematical description and not the technical implementation.

1.2. Historical Background

The SCIAMACHY instrument was conceived in the mid-1980’s in recognition of the need to monitor chemically important trace species on a global basis using passive remote sensing devices. The importance of monitoring global ozone distributions was demonstrated by the TOMS (Total Ozone Monitoring Spectrometer) instrument on board Nimbus 7, especially after the discovery of the ozone hole in 1985. The SCIAMACHY measurement strategy and mission objectives were based in part on the successful extension in the early 1980s of differential spectroscopic methods to retrieve atmospheric trace gas amounts from ground-based measurements using moderate-resolution spectrometers. Following an approach to ESA (European Space Agency) an initial study was commissioned and the results published in 1988. The scientific consortium was then invited to carry out a detailed Phase A feasibility study, which was finished in 1991. During this time, a scaled-down version of SCIAMACHY was commissioned and accepted for inclusion on the ERS-2 satellite. This “mini-SCIAMACHY” became the GOME (Global Ozone Monitoring Experiment) instrument, which has been functioning successfully since the 1995 launch of ERS-2. SCIAMACHY was accepted as an AO (Announcement of Opportunity) instrument to be included on the ENVISAT satellite. For the Phase C/D completion, the flight model was built in 1998, and calibrated in early 1999. ENVISAT was launched in March 2002 and after the commissioning phase, SCIAMACHY was in operational mode between July 2002 and April 2012, when the contact to the ENVISAT was lost.

1.3. Acronyms and Abbreviations

AMC-DOAS Air Mass Corrected Differential Optical Absorption Spectroscopy AMF Air Mass Factor AO Announcement of Opportunity ATBD Algorithm Theoretical Baseline Document

10 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

BIRA Belgisch Instituut voor Ruimte-Aëronmie BIRRA Beer InfraRed Retrieval Algorithm DLR Deutsches Zentrum für Luft- und Raumfahrt (German Aerospace Centre) DOAS Differential Optical Absorption Spectroscopy DRACULA aDvanced Retrieval of the Atmosphere with Constrained and Unconstrained Least squares Al- gorithms ECMWF European Centre for Medium-Range Weather Forecasts ENVISAT ENVIronmental SATellite ESA European Space Agency GOME Global Ozone Monitoring Experiment IUP-B Institut für Umweltphysik Bremen (Institute for Environmental Physics) MDS Measurement Data Set PMD Polarisation Measurement Device RTM Radiative Transfer Model SGP SCIAMACHY Ground Processor SCIAMACHY Scanning Imaging Absorption spectroMeter for Atmospheric CHartographY SRON Space Research Organisation Netherlands TOMS Total Ozone Monitoring Spectrometer

1.4. Document Overview

The document is divided into three parts plus appendices: 1. Introduction (this part) 2. Nadir retrieval algorithms: Cloud parameters, AAI and total columns of trace gases 3. Limb retrieval algorithms: Cloud parameters and proﬁles of trace gases 4. Appendices describing data structure etc. Part two and three start with a general description of the retrieval method. After that the settings for the individual retrievals are described as well as additional or deviating methods used in the retrieval.

SGP OL1b-2 ATBD Version 7 Page 11 of 137 2. The Data Processor

2.1. Processing ﬂow overview

The processing from Level 1b to Level 2 products is separated into the following steps (generally performed on individual fitting windows): 1. Calibration of data, i.e. generation of internal Level 1c data, using the same algorithms as the scial1c applicator according to the calibration settings in the configuration file 2. Calculation of the ratio Sun/Earth (note that this is not necessarily the reflectance, since not radiometric calibrated data can be used) 3. Climatological pre-processing: a) Calculation of Cloud Parameters (fraction, optical depth, top height) b) Initialisation of data bases 4. Nadir- and limb retrievals 5. Writing of Level 2 MDS and offline product

2.2. Components of the Data Processor

The SGP consists of three main components 1. The database server 2. The nadir retrieval server 3. The limb retrieval server Other components taking care of communication between the different running processes, data delivery to and from the SGP_12OL (also referred to simply as SGP or offline processor in this document) and interfaces to the main host running the ENVISAT processors are left out here. A more technical view of the SGP_12OL, including the outward interfaces is given in the Architectural Design Document (Kretschel, 2006). The primary input to the SGP_12OL is a SCIAMACHY Level 1b file, processed by the Level 0-1b processor (Slijkhuis and Lichtenberg, 2014). Within the ground segment the SGP only has one Auxiliary file as input, the initialisation file (Lichtenberg and Kretschel, 2009). All other auxiliary inputs like topographic databases or pressure profiles needed for the trace gas retrieval are handled within the SGP itself, i.e. those are integral part of the offline processor itself. Generally, the processing is done state wise: Each state from an Level 1b input file (usually encompassing a complete orbit) is relegated to one instance of the Nadir or Limb retrieval server. The first step in the processing is the calibration of the Earthshine spectra to radiances. Note, that no Level 1c product is written in the Level 1b to 2 processing, the calibrated spectra solely exist as internal data within the processor. However, the scial1c tool (Scherbakov, 2008) is based on the same algorithms as the calibration in the SGP and can be used to generate Level 1c products. Both deliver the same results which makes it possible to easily verify the SGP w.r.t. scientific reference algorithms. More than one retrieval server can run at a given time. When the last state of the input product is processed, the results are collected and used to generate the Level 2 product.

12 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

2.2.1. Database Server

The database server is a separate process, which handles the auxiliary input for the retrievals. It is split up into the GeoDataServer which provides topographical data, the RefSpecDataServer which provides the reference spectra for the ﬁts of the measured spectrum and CommonDataServer for the remaining data like pressure or temperature proﬁles. Upon triggering by the retrieval server, the database server not only reads in the requested data, but also does simple tasks like interpolating the correct wavelength grid. It also caches data that are needed by more than one retrieval to accelerate the processing. The server is set up in a generic way making it easy to add or exchange data needed for a given retrieval.

2.2.2. Nadir Retrieval Server

Before the nadir measurements are processed, the cloud parameters and AAI are retrieved in the climatological pre-processing. The cloud parameters are used in the subsequent retrieval of the trace gases. For the Nadir DOAS retrievals the kernel of the GDP was modiﬁed to accept SCIAMACHY input. The retrieval algorithms themselves were not modiﬁed meaning that the SGP products have the same quality as the GOME products albeit on a higher spatial resolution. The basis for the SGP DOAS implementation is the SDOAS scheme developed by BIRA, which is closely based on the GDOAS developed for GOME.

2.2.3. Limb Retrieval Server

The Limb retrieval is done by the DLR developed package DRACULA. While various models are implemented in DRACULA, the SGP uses the Iterative Regularized Gauss-Newton Method. It includes sophisticated techniques to select the iteration and regularisation parameters. A polynomial is ﬁtted to radiances to remove broad spectral features and ratioed radiances are constructed for each tangent height. Finally the logarithm of the ratios determines the basis for the retrieval.

2.3. Overview of MDS

The so called measurement data sets (MDSs) contain the measurements of the atmosphere. Four MDS’s exist: Cloud & Aerosol MDS Contains the cloud parameters measured in nadir geometry: cloud fraction, cloud top height and cloud optical thickness. The algorithms are described in chapter 3.

Nadir MDS Contains the trace gas columns: O3, NO2, HCHO, SO2, BrO, H2O, CHOCHO, OClO, CO and CH4. The algorithms are explained in chapters5,6 and7

Limb MDS Contains the trace gas proﬁles of O3, NO2 and BrO. The algorithms are explained in chapter8. Limb Cloud MDS Contains height resolved indicators for cloud presence and type (water clouds, PSCs and NLCs). The algorithm is explained in chapter9.

SGP OL1b-2 ATBD Version 7 Page 13 of 137 Part II.

Nadir Retrieval Algorithms

14 3. Cloud Parameters

3.1. Cloud Top Height and Cloud Optical Thickness

3.1.1. Overview

Clouds play an important role in the Earth climate system (Kondratyev and Binenko, 1984) . The amount of radiation reflected by the Earth-atmosphere system into outer space depends not only on the cloud cover and the total amount of condensed water in the Earth atmosphere but also the size of droplets aef and their thermodynamic state is also of importance. The information about microphysical properties, cloud top height and spatial distributions of terrestrial clouds on a global scale can be obtained only with satellite remote sensing systems. Different spectrometers and radiometers , deployed on space-based platforms, measure the angular and spectral distribution of intensity and polarization of reflected solar light (see e.g. Deschamps et al., 1994). Generally, the measured values depend both on geometrical and microphysical characteristics of clouds. Thus, the inherent properties of clouds can be retrieved (at least in principle) by the solution of the inverse problem. The accuracy of the retrieved values depends on the accuracy of measurements and the accuracy of the forward radiative transfer model. In particular, it is often assumed that clouds can be represented by homogeneous and infinitely extended in the horizontal direction plane-parallel slabs (Kokhanovsky et al., 2003; Rossow, 1989). The range of applicability of such an assumption for real clouds is very limited as is shown by observations of light from the sky on a cloudy day. For example, the retrieved cloud optical thickness τ is apparently dependent on the viewing geometry (Loeb and Davies, 1996). This, of course, would not be the case for an idealized plane-parallel cloud layer. However, both the state-of-art radiative transfer theory and computer technology are not capable to incorporate 3-D effects into operational satellite retrieval schemes. As a result, cloud parameters retrieved should be considered as a rather coarse approximation to reality. However, even such limited tools produce valuable information on terrestrial clouds properties. For example, it was confirmed by satellite measurements that droplets in clouds over oceans are usually larger than those over land . This feature, for instance, is of importance for the simulation of the Earth’s climate . Concerning trace gas retrievals in the UV/VIS, clouds are considered as “contamination”. The part of the column, which is below the top of the clouds, cannot be detected by the satellite. This ghost vertical column has to be estimated from climatological vertical profiles and is added to the vertical column retrieved. It is determined by integrating the profile from surface up to the cloud top height. Partial cloudiness within the field of view can be taken into account using fractional cloud cover. The Semi-Analytical CloUd Retrieval Algorithm (SACURA) is used in the SGP OL1b-2 to retrieve the cloud top height and the cloud optical thickness. Cloud top height is derived from measurements in the oxygen A absorption band as recommended by (Yamomoto and Wark, 1961). In addition, we retrieve the cloud geometrical thickness from the fit of the measured spectrum in the oxygen A-band and optical thickness using the wavelength 755 nm outside of the band.

3.1.2. Detailed Description

3.1.2.1. Cloud Top Height

The determination of the cloud top height h using SACURA is based on the measurements of the top-of- atmosphere (TOA) reﬂection function R in the oxygen A-band (Rozanov and Kokhanovsky, 2004). The cloud

15 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

reflection function is extremely sensitive to the cloud top height in the centre of the oxygen absorption band. To find the value of h, we first assume that the TOA reflectance R can be presented in the form of a Taylor expansion around the assumed value of cloud top height equal to h0 :

∞ X i R(h) = R(h0) + ai(h − h0) (3.1) i=1

(i) (i) th where ai = R (ho)/I. Here R (h0) is the i derivative of R at the point h0. The next step is the linearisation, which is a standard technique in the inversion procedures (Rodgers, 2000; Rozanov et al., 1998). We found that the function R(h) is close to a linear one in a broad interval of the argument change (Kokhanovsky and Rozanov, 2004). Therefore, we neglect non-linear terms in equation (3.1). Then it follows:

0 R = R(h0) + R (ho) · (h − h0) (3.2)

0 dR where R = dh . We assume that R is measured at several wavelengths in the oxygen A-band. Then instead of the scalar quantity R we can introduce the vector Rmes with components R(λ1),R(λ2), ··· R(λn). The same applies to other scalars in equation (3.1). Therefore, it can be written in the following vector form:

y = ax (3.3)

0 where y = Rmes − R(h0), a = R (h0) , and x = h − h0. Note that both measurement and model errors are contained in equation (3.3). The solution xˆ of the inverse problem is obtained by the minimizing the following cost function: Φ = ky − axk2 (3.4)

where k· · · k means the norm in the Euclid space of the correspondent dimension. The value of xˆ, where the function Φ has a minimum can be represented as the scalar product in the Euclid space Pn i=1 aiyi xˆ = Pn 2 (3.5) i=1 ai with n being the number of spectral channels where the reﬂection function is measured.

Therefore, from known values of the measured spectral reﬂection function Rmes (and also values of the calcu- 0 lated reﬂection function R and its derivative R at h = h0) at several at wavelengths, the value of the cloud top height can be found from equation (3.5) and equality: h =x ˆ + h0. The value of h0 can be taken equal to 1.0 km, which is a typical value for low level clouds. The main assumption in our derivation is that the dependence of R on h can be presented by a linear function on the interval x (Kokhanovsky and Rozanov, 2004). An error of 0.25 km can be expected for full convergence; otherwise 0.5 km.

3.1.2.2. Cloud Optical Thickness

Cloud optical thickness τ, together with the cloud fraction c, has a significant impact on the transfer of solar and infra-red radiation through a cloudy atmosphere. τ is defined as the cloud extinction coefficient kext integrated across the cloud vertical extent. For a vertically homogeneous cloud with geometrical thickness L, τ = kextL. For optically thick clouds, the optical thickness in the visible is estimated from the following equation (Kokhanovsky et al., 2003): R(µ, µ0, φ) = R0∞(µ, µ0, φ) − tcloudK0(µ)K0(µ0), (3.6) ↑ where R(µ, µ0, φ) = πI (µ, µ0, φ)/µ0F0 is the cloud reflection function as measured from the satellite, µ0 and µ are the cosines of the solar and observation zenith angles, φ is the relative azimuth, F0 is the top-of-atmosphere irradiance on the plane perpendicular to the incident light beam, and I↑ is the intensity of the reflected light, tcloud = 1/(a + bτ). Here tcloud is the cloud spherical transmittance, a = 1.07 and b = 0.75(1 − g). Assuming a given cloud model, e.g., spherical particles and a polydisperse distribution with a given effective radius of droplets, aef , or predefined ice crystals shapes and size distributions, the escape function K0(µ),

SGP OL1b-2 ATBD Version 7 Page 16 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

and the reﬂection function for a semi-inﬁnite layer R0∞ are pre-calculated and stored in look-up-tables (LUTs). Approximate equations for these functions can be used as well. In particular, a good approximation for K0(µ) is:

3 K (µ) = (1 + 2µ). (3.7) 0 7 One derives: 1 K (µ)K (µ ) τ = 0 0 0 − a . (3.8) b R∞(µ, µ0, φ) − R(µ, µ0, φ) This equation is used in SACURA for cloud optical thickness retrievals. It follows from this equation that retrievals of τ for very thick clouds (R → R∞) are highly uncertain and small errors (e.g., calibration errors) in the measured reﬂection function will lead to large errors in the retrieved cloud optical thickness. Often, a limiting value of cloud optical thickness is used in the retrieval process e.g., 100, as most clouds, but certainly not all have optical thickness below 100.

3.2. Cloud Fraction

3.2.1. Introduction

The basic idea in OCRA (Optical Cloud Recognition Algorithm Loyola and Ruppert, 1998) is to decompose optical sensor measurements into two components: a cloud-free background and a remainder expressing the inﬂuence of clouds, i.e.

R = RCF + RC (3.9)

where R is the reflectance and the indices CF and C indicate the cloud-free component and the cloudy component respectively. The reflectance for a given location (x, y), and wavelength λ can be defined as

I(x, y, λ) R(x, y, λ) = (3.10) I0(λ) · cos θ0 · cos θ

with

I Upwelling radiance into the ﬁeld of view

I0 Solar irradiance θ Viewing Zenith angle

θ0 Solar Zenith Angle

The key to the algorithm is the construction of the cloud-free composite that is invariant with respect to the atmosphere, and to topography and solar and viewing angles. The comparison of the spatially resolved cloud- free scenes and the measurement gives then the cloud fraction for a given scene. As for the GOME processor, in SCIAMACHY PMD data are used to determine the cloud fraction. These have the advantage that they have a higher spatial resolution than the spectrally resolved measurements (≈ 7 x 30 km). The steps for the calculation of the cloud fraction are 1. Generation of a database with the cloud-free composite by a) Extracting PMD signals and calculating a relative reﬂectance b) Find cloud-free scenes

SGP OL1b-2 ATBD Version 7 Page 17 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

c) Grid data

2. Calculation of cloud fraction a) Determine offset and scaling factor b) For each measurement, sum up signal of all PMDs and compare with cloud-free composite to determine the cloud fraction c) For each measurement, average over subpixels

3.2.2. Generation of the cloud-free composite

In previous versions of the SCIAMACHY data processor the GOME cloud-free composite database was used. Starting from the version 5 of the processor PMD measurements of SCIAMACHY were used to build a database of cloud-free measurements. First the PMD values of the Earth and Sun measurements were extracted from Level 1b data (version 6.03, the latest available version) in the time range from 2003 to 2007. Earth measurements are downsampled to 32 Hz in the Level 1b from the original 40 Hz while the sun measurements still have the original sampling. Additionally, the degradation of the instrument has to be corrected:

PMDEarth mPMD 32Hz RPMD = · · (3.11) PMDSun mP MDcal 40Hz with PMDEarth/Sun as the signal of PMD Earth and Sun measurements and mPMD(cal) as the degradation correction factor for Earth and Sun PMD measurements. For version 6 of the processor the new m-factors derived from the scan mirror model (Slijkhuis and Lichtenberg(2014)) were used. The above equation holds for all PMDs. In order to determine if the observed scene is cloud-free, the saturation S of the PMD reﬂectances is calculated :

max(R ,R .R ) − min(R ,R ,R ) S = PMD2 PMD3 PMD4 PMD2 PMD3 PMD4 max(RPMD2,RPMD3,RPMD4) min(R ,R ,R ) = 1 − PMD2 PMD3 PMD4 (3.12) max(RPMD2,RPMD3,RPMD4) with RP MDn as the signal in PMD number n. The saturation was preferred over the colour distance (that is used for the GOME cloud-free composite), because it can be easily expanded for a future separation of cloudy scenes from ice or snow covered surfaces. Both methods are completely equivalent, since they are a measure for the difference between a coloured scene and achromatic (white/grey/black) scene. Totally clouded scenes have a low saturation near 0, because they are white in the wavelength range measured by PMDs 2-4 and the signals in all of them will be similar. A cloud-free scene is indicated by a large saturation. The database with the cloud-free composite is build on a grid with a resolution of 0.36◦ × 0.36◦ (latitude x longitude). Each grid point is initialised with a saturation value of 0 (fully clouded). The database contains the following values: 1. Latitude and longitude of the grid point 2. Centre latitude and longitude of the measurement with the highest saturation so far 3. Reﬂectance values for all PMDs divided by the solar zenith angle 4. Solar Zenith angle 5. Saturation 6. Colour distance from the white point (see Loyola, 2000)

SGP OL1b-2 ATBD Version 7 Page 18 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

For each month in the considered time range the saturation values of the observations is sequentially calculated. If the measured saturation is larger than the one on the nearest grid point in the database and thus indicating that the observed scene is less cloudy than the one currently in the database, all values in the database for this grid point are replaced with those of the current measurements. The resulting monthly databases are combined to one database after all data are processed. In future versions of the processor the monthly databases can be used to build a seasonally dependent cloud-free composite to decrease the impact of colour variations of the surface over time.

3.2.3. Determination of the cloud fraction

For a given PMD observation the cloud fraction cfrac can now be determined as

v u 4 2 uX cf cfrac = t RP MDi − RP MDi − αi · βi (3.13) i=2

with

RP MDi Observed reﬂectance of PMD i cf RP MDi Reﬂectance for the cloud-free case from the database

αi Offset for PMD i

βi Scaling factor for PMD i

The offsets correct for the fact that the measured reflectances can be smaller than the cloud-free reflectance in the database. Possible reasons can be seasonal change of vegetation or ocean colour. Since the sign of cf the difference RP MDi − RP MDi is lost, cloud-free areas with lower brightness will give the same cloud fraction as slightly clouded scenes. By setting the offset to a large enough value the cloud fraction can be corrected for this effect. Using a small subset of the reflectance data set, the offset αi was set to the largest negative value of the reflectance difference found in the set. Note that a future extension of the database to include seasonal variations can minimise the influence of the colour changes.

The scaling factors βi correct for the fact that the albedo of observed clouds is not uniform but depends the illumination geometry, the thickness of the cloud, the size of the water droplets and other parameters. In effect this leads to different reﬂectances for scenes with the same cloud cover and eventually to different cloud fractions. In the current algorithm, only one scaling factor is available for each PMD, independent of season and geographical coordinates. In order to minimise errors in the calculation of the cloud fraction, the scaling factors were determined by a comparison with FRESCO+ (Wang et al., 2008). Version 6 also contains the cloud ice discrimination SPICI using the algorithm as described in Krijger et al. (2005). With this algorithm cloud free subpixels above ice are now properly detected1. SPICI employs a similar method as OCRA to determine the colour of the sub-pixel. It uses additionally PMD D (800 -900 nm) to discriminate clouds from ice.

3.2.4. Future improvements

Since the calculation and extraction of SCIAMACHY data from several years is rather time consuming, possible future algorithm improvements were already considered during the generation of the cloud-free composite. These improvements include: Determination of the offset using the complete data set and not only a subset Independent determination of the scaling factors using a composite of totally clouded scenes Introduction of seasonal and geographically dependent offsets and scaling factors

1ice and cloud were not distinguishable by their colour using only PMDs in the visible range

SGP OL1b-2 ATBD Version 7 Page 19 of 137 4. Absorbing Aerosol Index

4.1. Overview

The Absorbing Aerosol Index (AAI) indicates the presence of elevated absorbing aerosols in the Earth’s atmosphere. The aerosol types that are mostly seen in the AAI are desert dust and biomass burning aerosols. The AAIs described in this ATBD are derived from the reﬂectances measured by SCIAMACHY at 340 and 380 nm.

4.1.1. Retrieval Settings

The wavelength pair used is [340, 380] nm. Both wavelengths originate from cluster 9 of the SCIAMACHY spectrum. The associated reﬂectances at these two (centre) wavelengths are determined by taking the average of all the reﬂectances that fall inside a 1-nm wavelength bin around the respective centre wavelengths.

4.2. Detailed Description

4.2.1. Deﬁnition of used geometry

Figure 4.1 deﬁnes, in a graphical way, the angles that are used to specify the viewing and solar directions. An imaginary volume element that is responsible for the scattering of incident sunlight is located in the origin of the sphere. The solar direction is described by the solar zenith angle θ0 and the solar azimuth angle φ0. The viewing direction is deﬁned by the viewing zenith angle θ and the viewing azimuth angle φ.

local zenith

φ-φ0 local θ0 meridian plane Sun θ

φ0

φ scattered local light horizon scattering plane incident Θ sunlight

Figure 4.1.: Deﬁnition of viewing and solar angles.

20 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

We make use of a right-handed coordinate frame, meaning that φ − φ0 as sketched in figure 4.1 is positive. We followed the same definition of φ and φ0 as is present in the SCIAMACHY data. However, there is no need to specify it because only the azimuth difference φ − φ0 is of relevance. For completeness, we mention that the single scattering angle, under this definition of angles, can be calculated from

cos θ = − cos θ cos θ0 + sin θ sin θ0 cos(φ − φ0) (4.1)

4.2.2. Deﬁnition of the Residue

The Absorbing Aerosol Index (AAI) is a dimensionless quantity which was introduced to provide information about the presence of UV-absorbing aerosols in the Earth’s atmosphere. It is derived directly from another quantity, the residue, which is deﬁned by Herman et al.(1997) as

R R r = −100 ·{10log( λ )obs −10 log( λ )Ray} (4.2) Rλ0 Rλ0

In this equation, Rλ denotes the Earth’s reflectance at wavelength λ. The superscript obs refers to reflectances which are measured by, in this case, SCIAMACHY, while the superscript Ray refers to modelled reflectances. These modelled reflectances are calculated for a cloud-free and aerosol-free atmosphere in which only Rayleigh scattering, absorption by molecules, Lambertian surface reflection as well as surface absorption can take place. The reflectance is defined as

πI R = (4.3) µ0E where I is the radiance reflected by the Earth atmosphere (in W m−2nm−1sr−1), E is the incident solar irradi- −2 −1 ance at the top of the atmosphere perpendicular to the solar beam (in W m nm ), and µ0 is the cosine of the solar zenith angle θ0. As for the wavelengths involved, the wavelengths λ and λ0 must lie in the UV, and were set to 340 and 380 nm, respectively, for SCIAMACHY. The Rayleigh atmosphere in the simulations is bounded below by a Lambertian surface having a wavelength independent surface albedo As, which is not meant to represent the actual ground albedo. It is obtained from requiring that the simulated reflectance equals the measured reflectance at λ0 = 380 nm:

obs Ray (4.4) Rλ0 = Rλ0 (As)

Ray The surface albedo found in this way is used to calculate Rλ , so one assumes that the surface albedo is constant in the wavelength interval [λ, λ0], which is true for most cases. Note that equation (4.2) can now be reduced to

obs ! 10 Rλ r = −100 · log Ray (4.5) Rλ

4.2.3. Deﬁnition of the AAI

The importance of the residue, as defined above, lies in its ability to effectively detect the presence of absorbing aerosols even in the presence of clouds. When a positive residue (r > 0) is found, absorbing aerosols were detected. Negative or zero residues on the other hand (r ≤ 0), suggest an absence of absorbing aerosols. For that reason, the AAI is defined as equal to the residue r where the residue is positive, and it is not defined where the residue has a negative value.

SGP OL1b-2 ATBD Version 7 Page 21 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

4.2.4. Calculation of the Residue

The first step in the calculation of the residue involves finding the artificial surface albedo As for which the measured SCIAMACHY reflectance at the reference wavelength λ0 = 380 nm equals the simulated reflectance at the same wavelength (cf. equation 4.4). The simulated reflectances are found from a collection of Look-Up Tables (LUTs). These LUTs are described in section 4.2.5. The simulations basically describe a cloud-free, homogeneous atmosphere which is bounded below by a Lambertian surface. The contribution of the surface to the TOA reflectance may be separated from that of the atmosphere according to the following formula (Chandrasekhar, 1960):

0 Ast(µ)t(µo) R(µ, µo, φ − φ0,As) = R (µ, µo, φ − φ0) + ∗ (4.6) 1 − Ass

In this equation, the first term R0 is the path reflectance, which is the atmospheric contribution to the reflectance. The second term is the contribution of the surface with an albedo As. The parameter t is the total atmospheric transmission for the given zenith angle, s∗ is the spherical albedo of the atmosphere for illumination from below, µ is the cosine of the viewing zenith angle θ, and µ is the cosine of the solar zenith angle defined earlier. Using equation (4.6) and by demanding that the simulated reflectance Ray equals the θ0 Rλ0 measured reflectance obs at 380 nm, we find the following expression for the surface albedo : Rλ0 As

Robs − R0 A = λ0 λ0 (4.7) s ∗ obs − 0 t(µ)t(µ0) + s (Rλ0 Rλ0 )

In this equation, 0 denotes the simulated reﬂectance for the actual atmosphere, but without surface reﬂection. Rλ0 R0 can be expanded in a Fourier series. In our case, with a Rayleigh atmosphere, the expansion is exact with only three terms in φ − φ0 . We have

2 0 X R = a0 + 2ai(µ, µ0) cos i(φ − φ0) (4.8) i=1

∗ The idea is that with a proper LUT of ai, t(µ) and s , we can easily calculate R0, As and Rλ. Some interpolation is necessary. In this case we have to interpolate over µ and µ0 and over the surface height hs.

4.2.5. Look-up Tables

Using the radiative transfer code DAK (Doubling-Adding KNMI, de Haan et al., 1987), Look-Up Tables (LUTs) ∗ were created. The LUTs contain the parameters ai, t(µ) and s as well as the total Ozone column, as defined in section 4.2.4. The definition of the coefficients ai is such that the path reflectance R0 may be reconstructed from them according to

R0 = a0 + 2a1 cos(φ − φ0) + 2a2 cos 2(φ − φ0) (4.9)

Adding the surface contribution requires equation (4.6) and the parameters t(µ) and s∗ . The simulations were done for cloud-free situations. Aerosols were not included either. The LUTs were prepared as a function of µ and µ0, surface height hs and total ozone column. The coefﬁcients were calculated for 42 Gaussian distributed µ and µ0 points, and for 9 surface heights hs ranging from 0 to 8 km in 1 km steps. The ozone total column retrieved by SCIAMACHY for the given observation is used for the look-up of the correct values in the LUT.

4.2.6. Degradation Correction

Since the algorithm relies on absolute values for the reﬂectance, the degradation has to be corrected. The residue increased by, on average, more than 2.5 index points over the last years, and an analysis indicated

SGP OL1b-2 ATBD Version 7 Page 22 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18 that the increase could be completely explained by the instrument degradation which was recorded by instrument Light Path Monitoring (LPM) measurements (which are taken on a regular basis). In the processor, the degradation is now corrected during the internal Level 1b to 1c step . Details of the degradation correction can be found in Slijkhuis and Lichtenberg(2014) and Krijger(2011).

SGP OL1b-2 ATBD Version 7 Page 23 of 137 5. SDOAS Algorithm General Description

The vertical column retrieval algorithm GDOAS was developed at BIRA-IASB speciﬁcally for GOME on the basis of the well-established and widely used DOAS method (Platt and Stutz, 2008). GDOAS was implemented into the GOME data processor (GDP) version 4. The GDOAS adaptation to the SCIAMACHY instrument was renamed SDOAS and has also been implemented in the SCIAMACHY Ground Processor L12 (from version 3) to have an accuracy similar to the one obtained with GDP4. This section presents a brief description of the DOAS method that involves two steps: the slant column retrieval and the conversion into vertical columns using calculated air mass factors. The detailed description of GDOAS is provided in (Spurr et al., 2004; Van Roozendael et al., 2006; Lerot et al., 2009).

5.1. Slant column retrieval

The Lambert-Beer extinction law applied to the earthshine measurements is the basis for the slant column retrieval from satellite measurements. A direct solar irradiance measurement void of atmospheric absorption structures is used as the background spectrum. The validity of this law is limited to the weak absorption, which is generally true in case of the atmospheric trace gas absorbers. However, in very particular conditions, the DOAS approximation may be not veriﬁed (e.g. in the UV for high solar zenith angles due to the strong ozone absorption or in case of high SO2 concentrations during volcanic eruptions). In SDOAS it is generally the logarithms of intensities (so-called "optical densities") that are being used. The effects of the broadband molecular and aerosol scattering are represented in the form of the low-order polynomials (typically of the 2nd or 3rd degree). The main equation used for the DOAS retrieval is then

Iλ(Θ) Y (λ) = ln 0 Iλ(Θ) n X X ∗ j = − Eg(Θ) · σg(λ) − αj (λ − λ ) − αRσR(λ) (5.1) g j=0 with

Iλ(Θ) Earth radiance at wavelength λ for the geometrical path Θ

0 Iλ Solar irradiance

Eg Effective slant column density of gas g

σg Differential absorption cross-section of gas g λ∗ Reference wavelength for closure polynomial

σR Ring reference spectrum

αR Scaling parameter for Ring spectrum

The DOAS retrieval consists in a minimization of the weighted least squares difference between measured and calculated optical densities Y(λ) in the spectral interval appropriately selected for the trace gas under study. To calculate optical density, reference absorption cross-sections are provided by independent measurements or are pre-calculated. For instance, the Ring effect caused by the inelastic Raman scattering is treated as a

24 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18 pseudo-absorber. The corresponding Ring spectrum is usually deﬁned as a (logarithmic) ratio between the radiance spectra with and without inelastic Raman scattering (Chance and Spurr, 1997). Two measurements of the solar irradiance are realized daily using the two diffusers (ESM and ASM) mounted on the instrument. The algorithm uses the most recent update of the sun spectrum measured by one of the two diffusers, depending on the trace gas to retrieve. The wavelength calibration of the irradiance spectrum is further improved using a cross-correlation procedure involving a high-resolution reference solar spectrum. In addition, to correct for possible distortion of the wavelength grid (e.g. due to the Doppler effect caused by the satellite movement), the wavelength grid of the earthshine spectrum is allowed to be shifted in the DOAS procedure.

The retrievals performed in the channel 2 of SCIAMACHY (BrO, SO2, OClO) revealed a problem, which is due to the presence of a strong polarization-related anomaly occurring right in the middle of the channel. This anomaly interferes with the absorption structures of the retrieved gases that leads to the non-physical results. To reduce this polarization-related artefact, a polarization vector η (often called ETA), extracted from the SCIAMACHY key data set, is included into the SDOAS ﬁt procedure for some of the trace gases. Also, to account for possible intensity offsets in the spectra, a supplementary cross-section calculated as the inverse earthshine spectrum is ﬁtted in the DOAS procedure. This term also corrects for some limitations of the Ring spectrum, e.g. over bright surfaces or in regions where vibrational Raman scattering in ocean water is relevant and is used for most minor trace species.

5.2. Vertical column retrieval

The conversion from slant to vertical column amounts is done via the AMF computation. The AMF deﬁnition used in the SGP is the common one

Inog ln I A = g (5.2) τvert with A Air mass factor

Inog Simulated backscatter radiances without main gas

Ig Simulated backscatter radiance with main gas

τvert Vertical optical depth of the main gas

These AMFs are directly calculated on the fly with the multiple scattering radiative transfer model LIDORT (Linearized Discrete Ordinate Radiative Transfer), without the need for large look-up tables. LIDORT assumes the atmosphere as a stratified structure with a given number of optically homogeneous layers. For more information about LIDORT see Spurr et al.(2001). Since the AMF depends not only on viewing geometry and solar position but also on the wavelength and the concentration vertical profile shape, it is calculated for each retrieved gas individually. The concentration profile shape is generally provided by an external climatology. AMF calculation demands also an information about a surface albedo. In the SGP it is taken from the TOMS albedo data base. Two types of calculations are possible with the data processor: 1. An iterative approach when the total column is one of the input parameters for the selection of the profile within the climatology. Currently, this method is only used for ozone (see below) as it is only needed for strong absorbers. 2. A standard approach when the climatology is only based on seasonal and geographical parameters.

SGP OL1b-2 ATBD Version 7 Page 25 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

For strongly varying absorbers such as SO2, the use of climatologies leads to large errors. Therefore, in the current processor two standard AMF are calculated, one for a volcanic eruption scenario and one for a boundary layer pollution situation. Both results are provided to the user who has to decide which value is more appropriate for his application. For OClO, no vertical column is calculated at all as rapid photochemistry and the change of solar zenith angle along the light path through the stratosphere at low sun make application of the concept of vertical columns difﬁcult.

In general, two types of AMF are calculated: for clear sky (Aclear; from the top of the atmosphere to the ground) and for cloudy (Acloud; from the top of the atmosphere to the cloud-top level) scenarios. The cloudy air mass factor calculation is based on cloud parameters generated off-line by dedicated algorithms. The vertical column is then calculated as

E + cfrac · G · Acloud V = MR (5.3) (1 − cfrac) · Aclear + cfrac · Acloud with

V Vertical column density E Slant column density of main gas

MR Molecular Ring correction scale factor

cfrac Intensity weighted cloud fraction G “Ghost column” of main gas below cloud top height

Acloud AMF to the cloud top level

Aclear AMF to the ground level

The molecular Ring correction MR is specific to total ozone retrievals (see below)). For other trace gases, this factor is fixed to 1. The "ghost column" is the part of the column hidden below the cloud and thus not detectable by the satellite instrument. Therefore, for the cloudy pixels, a climatological gas column comprised between the ground and the cloud top needs to be added to the retrieved one. The "ghost column" concept works reasonably well for trace gases with more or less constant tropospheric distributions and prevents the cloud-induced "jumps" in retrieved vertical column fields. For strongly varying species such as SO2, no ghost column is applied.

5.3. Nadir Ozone Total Column Retrieval

5.3.1. Motivation

The role of ozone in the Earth’s atmosphere is very important since it absorbs solar UV photons, protecting the biosphere from these harmful radiations. Also, the radiation absorption by the high concentrations of ozone in the stratosphere leads to an heating of this part of the atmosphere. This temperature increase in the stratosphere makes this region very stable and inﬂuences the dynamic of the whole atmosphere. The discovery of the ozone hole above Antarctica during spring time was one of the most obvious changes observed in Earth’s atmosphere. Although the Montreal Protocol and its amendments have regulated the production and release of ozone depleting substances, the ozone recovery expected to occur in the next decades has to be monitored on a long-term basis. Furthermore, climate change will have an impact on the atmospheric dynamics and chemistry which inﬂuence the ozone concentrations. Global measurements of ozone columns can help to better understand these interactions and consequently to better predict the timing of ozone recovery.

SGP OL1b-2 ATBD Version 7 Page 26 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Although 90% of ozone is in the stratosphere, tropospheric ozone concentrations can locally be important. In the troposphere, ozone is a pollutant and a constituent of smog. It is also a powerful oxidizing agent and an important greenhouse gas. Total ozone measurements may be combined to stratospheric ozone measurements to derive information on tropospheric ozone content.

5.3.2. Retrieval Settings

Ozone absorption is strongly temperature-dependent in the UV Huggins bands, and in this spectral range an effective temperature is derived from the DOAS ﬁtting in addition to the ozone slant column itself.

Level 1b-1c Settings Calibration All calibrations applied SMR D0 (Sun over ESM diffuser) DOAS Main Settings Fitting Interval 325 - 335 nm Polynomial Degree 3rd order Wavelength Shift Wavelength calibration of sun reference optimized over ﬁtting interval by NLLS adjustment on pre-convolved NEWKPNO atlas Absorption Cross Sections/Fitted Curves

O3 Bogumil et al.(2003)@243 K, shifted by 0.02nm, scaled by 1.03

NO2 Bogumil et al.(2003)@243 K 243K 223K O3 Difference Spectrum Difference σO3 − σO3 , shift allowed, scaled by 1.03 Ring Spectrum Calculated by convolution of the Chance and Spurr(1997) solar atlas with RRS cross-sections of molecular N2 and O2. Total Column Calculation: Proﬁles/AMF AMF ref. Wavelength 325.5 nm

O3 Profile Column-classified ozone profile climatology TOMS V.8 Radiative Transfer Model LIDORT Cloud Top Height SACURA Cloud Fraction OCRA

5.3.3. Additional Details

5.3.3.1. Molecular Ring correction

In addition to the ﬁlling-in of Fraunhofer absorption solar lines, rotational Raman scattering also causes the ﬁlling-in of atmospheric absorption features. This has to be considered for precise total ozone retrievals. It is shown in Van Roozendael et al.(2006) that a simple correction for this molecular Ring effect can be realized by dividing the slant column by a scale factor MR calculated with the following equation

sec(θ0) MR = 1 + αRσR 1 − (5.4) (1 − cfrac) · Aclear + cfrac · Acloud with symbols as in equation 5.3 and

SGP OL1b-2 ATBD Version 7 Page 27 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

αR fractional intensity of Raman light (from DOAS ﬁt)

σR average Ring cross section calculated over the ﬁtting interval

θ0 Solar zenith angle

It is calculated “on the ﬂy” during the vertical column retrieval and usually not available separately. Following a request from users, the slant column provided in the SGP product is the Ring corrected slant column (i.e. E/MR ).

5.3.3.2. Vertical column with iterative approach

For ozone, the total column is a good proxy for the ozone profile, and column-classified ozone profile climatology is useful for selecting profiles required for the radiative transfer. In GDOAS, the ozone AMF is adjusted iteratively to reflect the actual column content as expressed by the slant column.

In the ﬁrst iteration, an initial AMF A0 is computed for a given initial guess of vertical column. The vertical column V0 is obtained, which serves as the second guess for the vertical column. This process runs until the convergence criterion is reached, i.e. until the relative difference between consequent iterations is less than some predeﬁned small number.

SGP OL1b-2 ATBD Version 7 Page 28 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

5.4. Nadir NO2 Total Column Retrieval

5.4.1. Motivation

The main source of NO2 in the stratosphere is microbiological soil activity. The diurnal, seasonal, and latitudinal variation of NO2 is dominated by the equilibrium between NOx and the reservoir substances (mainly N2O5, HNO3, ClONO2). In the troposphere, the main sources of NO2 are anthropogenic in origin, e.g. industry, power plants, trafﬁc and forced biomass burning. Other, but slightly less important origins comprise natural biomass burning, lightning and microbiological soil activity. NO2 emissions have increased by more than a factor of 6 since pre-industrial times with concentrations being highest in large urban areas.

Global monitoring of NO2 emissions is a crucial task since nitrogen oxides play a central role in atmospheric chemistry. For example, it catalyses ozone production, contributes to acidiﬁcation and also adds to radiative forcing. The high spatial resolution of SCIAMACHY allows us performing very detailed observations of polluted regions.

5.4.2. Retrieval Settings

Level 1b-1c Settings Calibration All calibrations except radiometric SMR A0 (Sun over ASM diffuser without radiometric calibration) DOAS Main Settings Fitting Interval 426.5 - 451.5 nm Polynomial Degree 2nd order Wavelength Shift Wavelength calibration of sun reference optimized over ﬁtting interval by NLLS adjustment on pre-convolved NEWKPNO atlas Absorption Cross Sections/Fitted Curves

NO2 Bogumil et al.(2003)@243 K

O3 Bogumil et al.(2003)@243 K, shifted 0.025 nm

O2-O2 Greenblatt et al.(1990); Wavelength axis corrected by Burkholder

H2O Generated from HITRAN database Ring Spectrum Calculated by convolution of the Chance and Spurr(1997) solar atlas with RRS cross-sections of molecular N2 and O2. Empirical Functions Offset and Slope Correction Inverse Earthshine spectrum (radiance) Total Column Calculation: Proﬁles/AMF AMF ref. Wavelength 439 nm

NO2 Proﬁle HALOE (Lambert et al., 2000) Radiative Transfer Model LIDORT Cloud Top Height SACURA Cloud Fraction OCRA

SGP OL1b-2 ATBD Version 7 Page 29 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

5.5. Nadir SO2 Total Column Retrieval

5.5.1. Retrieval Settings

Level 1b-1c Settings Calibration All calibrations except radiometric SMR A0 (Sun over ASM diffuser without radiometric calibration) DOAS Main Settings Fitting Interval 315 - 327 nm Polynomial Degree 3rd order Absorption Cross Sections/Fitted Curves

SO2 Vandaele et al.(1994)

O3 Bogumil et al.(2003)@243 K 243K 223K O3 Difference Spectrum Difference σO3 − σO3 Ring Spectrum Vountas et al.(1998) Empirical Functions Offset and Slope Correction Inverse Earthshine spectrum (radiance) Background Correction SCD from reference sector over the Paciﬁc (180◦ - 220◦) Total Column Calculation: Proﬁles/AMF AMF ref. Wavelength 315 nm

SO2 Proﬁle Anthropogenic Pollution scenario: 1 DU present from surface to 1 km height

SO2 Proﬁle Volcanic Eruption scenario: 10 DU present in layer between 10 and 11 km Radiative Transfer Model LIDORT Cloud Top Height Not applied, ghost column set to 0 Cloud Fraction Not applied

5.5.2. Additional Details

For both, the slant column computation and the total column retrieval the standard method described above was adjusted.

5.5.2.1. Slant Column

In the slant column determination, a spectral interference at large ozone absorption (high solar zenith angle, large ozone column) leads to unrealistic slant columns in the SO2 retrieval. To correct for this offset, the SO2 ◦ ◦ columns from a reference sector over Paciﬁc (180 −220 longitude) are subtracted from the retrieved SO2 slant columns. For determination of the offset it is assumed that no SO2 is present in the so called reference sector over the Paciﬁc and that the interfering effect (mainly ozone absorption) does not depend strongly on longitude. The values from the reference sector (i.e. the assumed offsets) are stored in a database (BG_DB) as background values.

Database Filling and Content For optimum correction, the BG_DB should contain those measurements over the reference sector which are closest in time to the actual measurement analysed at other longitudes. It should also have no gaps, should not be affected by volcanic SO2 plumes or outliers and should be available in NRT.

SGP OL1b-2 ATBD Version 7 Page 30 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

The orbit duration of ENVISAT is roughly 100 minutes leading to 14 orbits per day. Of these 14 orbits typically two cross the background reference sector near the equator. For higher latitudes more crossings occur. How- ever, as result of the gaps during limb measurements and from missing data, not all of the reference sector is covered on each day. The BG_DB contains the nadir SO2 slant columns retrieved by the off-line processor. It is continuously updated during processing. The values are retrieved with the same algorithm as the “regular” slant columns, their only difference is the geolocation of the ground-pixel. New values are written to the database if The centre longitude is in the reference sector (currently all longitudes between 180◦ and 220◦) The ground-pixel is in the descending node of the orbit (i.e. on the sunlit side of the orbit). The RMS of the retrieval is below a limit (currently 0.007) The fractional cloud cover is below a limit (currently 0.5) At least one latitudinal bin with quality > 0 was found The orbit was not already used to add data to the BG_DB (duplicate entries would distort the quality value and the averaged value) All values obtained on the same day for a given latitude bin are averaged, weighted with their quality. Each latitudinal bin spans 5◦ leading to 36 bins per day. In order to judge the quality of the background value, the data base also contains a quality ﬂag that can have values between 0 (no data) and 255 (best quality). The quality is determined from the number of data points available for each bin and the standard deviation of the SO2 slant columns. The quality of the background correction is written to the SCD FITFLAG. If the Background correction is used, bit #8 is set . The quality itself is coded into three bits (#9-11), resulting into 8 values : the value 0 means no background correction was performed, the value 7 marks the best possible correction and values in between represent intermediate quality.

Application of the Background Values

After the SO2 retrieval the slant columns are written to the MDS of the product with the offset from BG_DB applied. The offsets depend on latitude and time. Since the database is ﬁlled continuously during the processing, at the start of the processing there will always be measurements that have no corresponding (in time and latitude) background value. In this case the following method is used to obtain a background correction: 1. Search the database alternately backward and forward in time, until values from the neighbouring latitude bins are found. 2. Interpolate between the 2 neighbouring SCD offsets to the current latitude. 3. Interpolate between the 2 corresponding quality ﬂags; a latitude near a missing latitude bin thus will automatically result in an SCD offset and a quality of or near 0. 4. Subtract the offset from the measurement.

5.5.2.2. Vertical Column

As already mentioned, VCD calculation was adjusted as well. Since SO2 is present only sporadically, the use of climatological values for the vertical proﬁles is not possible. Therefore, the approach was taken to assume a constant proﬁle shape for two typical scenarios:

a proﬁle with 1 DU of SO2 between a 0 and 1 km simulating an anthropogenic pollution event;

a proﬁle with 10 DU of SO2 between a 10 and 11 km simulating a volcanic eruption.

Accordingly, two types of SO2 vertical columns - anthropogenic and volcanic - are computed. Both results are written into two different MDSs of the level 2 product. The second adjustment concerns the treatment of clouds in the retrieval. The ghost column approach that corrects for column parts hidden below clouds (see section 5.2) can lead to meaningless results in case of SO2. Contrary to O3, NO2 and BrO, the vertical atmospheric SO2 distribution is neither known nor predictable.

SGP OL1b-2 ATBD Version 7 Page 31 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Both of the main sources of SO2, the volcanic eruptions and the anthropogenic pollution events, are sporadic. Therefore, assuming that SO2 is present below every cloud will give incorrect results. Consequently, no ghost column is added to retrieved SO2 vertical columns. In order to stay consistent with this approach, for the AMF factor always Aclear is used. The users should be advised that in some measurements SO2 might be hidden below a cloud resulting in too low SO2 vertical columns.

5.6. Nadir BrO Total Column Retrieval

5.6.1. Motivation

Bromine monoxide (BrO) is a key atmospheric trace gas playing a major role as catalyst of the ozone de- struction in both the stratosphere and the troposphere. The main sources of bromine in the stratosphere are long-lived and short-lived brominated organic compounds of natural and anthropogenic origin (CH3Br and halons). In the troposphere, large emissions of inorganic bromine have been observed in the polar boundary layer when frost ﬂowers on the surface of the ocean vanish at the end of the winter. These bromine explosion events also result in efﬁcient tropospheric ozone depletion. Finally, an other source of inorganic bromine in the troposphere and possibly in the stratosphere is the release by active volcanoes.

5.6.2. Retrieval Settings

Level 1b-1c Settings Calibration All calibrations except radiometric SMR A0 (Sun over ASM diffuser without radiometric calibration) DOAS Main Settings Fitting Interval 336 - 351 nm Wavelength Shift Wavelength calibration of sun reference optimized over ﬁtting interval by NLLS adjustment on pre-convolved NEWKPNO atlas Polynomial Degree 3rd order Absorption Cross Sections/Fitted Curves

NO2 Bogumil et al.(2003)@ 243 K

O3 Bogumil et al.(2003)@ 243 K, shifted

O2-O2 Greenblatt et al.(1990), wavelength axis corrected by Burkholder BrO Fleischmann et al.(2004)@223 K 243K 223K O3 Difference Spectrum Difference σO3 − σO3 , shifted Ring Spectrum Vountas et al.(1998) Empirical Functions Polarisation Feature Correction η Nadir included in the fit Offset and Slope Correction Inverse Earthshine spectrum Total Column Calculation: Profiles/AMF AMF ref. Wavelength 343.5 nm BrO Profile Climatology based on the 3-D CTM BASCOE from BIRA (Theys et al., 2008) Radiative Transfer Model LIDORT Cloud Top Height SACURA Cloud Fraction OCRA

SGP OL1b-2 ATBD Version 7 Page 32 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

5.7. Nadir OClO Slant Column Retrieval

5.7.1. Retrieval Settings

Level 1b-1c Settings Calibration All calibrations except radiometric SMR A0 (Sun over ASM diffuser without radiometric calibration) DOAS Main Settings Fitting Interval 365 - 389 nm Polynomial Degree 4th order Absorption Cross Sections/Fitted Curves

NO2 Bogumil et al.(2003)@223 K

O4 Hermans et al.(1999) OClO Kromminga et al.(2003) Ring Spectrum Vountas et al.(1998) Undersampling Constant Undersamlping Spectrum calculated by IUP Empirical Functions Offset and Slope Correction Inverse Earthshine spectrum (radiance) Polarisation Feature Correction η Nadir included in the ﬁt Intensity Correction Ratio Ratio of cloudy and cloud-free measurements calculated by IUP

5.7.2. Additional Details

The OClO retrieval in SDOAS is also changed with respect to the standard approach: an interference, which so far could not be explained, can lead to unrealistically large OClO columns over bright surfaces and clouds. Possible reasons are insufﬁcient correction of the Ring effect of problems in the lv1 data, e.g. from instrument stray light, detector non-linearities or detector memory effects. To correct for this artefact, an empirical function (the ratio between two measurements - one over a bright cloud and another from a close clear pixel) is included into the ﬁt procedure. Experience shows, that inclusion of this function can reduce artefacts over bright surfaces to a high degree. This empirical intensity correction ratio spectrum is sometimes referred to as "magic ratio".

5.8. Nadir HCHO Total Column Retrieval

5.8.1. Motivation

Formaldehyde (H2CO) and glyoxal (CHOCHO) are formed during the oxidation of the volatile organic compounds (VOCs) emitted by plants, fossil fuel combustion, and biomass burning. Due to a rather short lifetime of formaldehyde and glyoxal, their distribution maps represent the emission ﬁelds of their precursors, VOCs. Additionally, HCHO could also be used as a tracer for carbonaceous aerosols. This information will contribute to understanding of biogenic emissions, biomass burning, and pollution.

SGP OL1b-2 ATBD Version 7 Page 33 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

5.8.2. Retrieval Settings

Level 1b-1c Settings Calibration All calibrations except radiometric SMR A0 (Sun over ASM diffuser without radiometric calibration) DOAS Main Settings Fitting Interval 328.5 - 346 nm Polynomial Degree 5th order Absorption Cross Sections/Fitted Curves

NO2 Vandaele et al.(1998)

O3 Brion et al.(1998)@228 & 243 K HCHO Meller and Moortgat(2000)@298 K BrO Fleischmann et al.(2004)@223 K OClO Bogumil et al.(2003)@293 K SCIAMACHY irradiance of 20030329, KPNO solar spectrum, Gaussian slit Ring Spectrum function Undersampling Constant Undersamlping Spectrum calcculated by BIRA Empirical Functions Polarisation Feature Correction η and ς Nadir included in the fit Offset and Slope Correction Inverse Earthshine spectrum (radiance) Background Correction SCD from reference sector over the Pacific (180◦ - 220◦) Total Column Calculation: Profiles/AMF AMF ref. Wavelength 340 nm HCHO Profile IMAGES (Müller and Brasseur(1995)) Radiative Transfer Model LIDORT Cloud Top Height SACURA Cloud Fraction OCRA

5.8.3. Additional Details

Three successive steps are needed to retrieve HCHO total columns from SCIAMACHY nadir spectra: the slant column retrieval, the air mass factor calculation and a post-correction based on the slant columns retrieved over the Pacific Ocean and on the a-priori HCHO profiles. For the AMF calculation, a-priori concentrations profiles are taken from the CTM IMAGES (resolved monthly and 2° X 1.25°). Zonally and seasonally dependent artefacts due to interferences with BrO and ozone are strongly reduced by the reference sector method, which is applied in the same way as for the SO2. After the reference sector correction slant columns are converted into a vertical column

SCD − SCD VCD = 0 + VCD (5.5) AMF 0

VCD0 represents the climatological HCHO vertical column in the reference sector at the latitude of the measurement. The climatological vertical column database is built by integrating the appropriate climatological concentration proﬁles within the reference sector. In this database, the different climatological columns coming

SGP OL1b-2 ATBD Version 7 Page 34 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18 from different proﬁles with different longitudes for the same latitude are averaged. VCD0 is obtained via an interpolation scheme in the a-priori column database (as a function of the latitude and time). Contrary to a majority of other nadir products formaldehyde measurements are not provided for pixels with solar zenith angle larger than 60°.

5.8.4. User warning

Formaldehyde vertical columns from all pixels are written into the product, regardless of their cloud fraction. It is strongly advised to apply a cloud ﬁltering to the formaldehyde vertical columns. It is to be emphasized that vertical columns of pixels with cloud fraction ≤ 40% represent actual concentration of formaldehyde. For larger cloud fractions the information content below the cloud altitude is weak because the column is dominated by a climatological information.

5.9. Nadir CHOCHO Total Column Retrieval

5.9.1. Retrieval Settings

Level 1b-1c Settings Calibration Uncalibrated data: only dark current (leakage), memory effect and etalon correction applied SMR A0 (Sun over ASM diffuser without radiometric calibration) DOAS Main Settings Fitting Interval 435 - 457 nm Polynomial Degree 4th order Absorption Cross Sections/Fitted Curves CHOCHO Volkamer et al.(2005)

NO2 Bogumil et al.(2003)@223 K

O3 Bogumil et al.(2003)@273 K

O4 Greenblatt et al.(1990)

H2O liquid Pope and Fry(1997) Ring Spectrum Vountas et al.(1998) Empirical Functions Offset and Slope Correction Inverse Earthshine spectrum (radiance) Background Correction SCD from reference sector over the Pacific (180◦ - 200◦) Total Column Calculation: Profiles/AMF AMF ref. Wavelength 446 nm CHOCHO Profile IMAGES (Müller and Brasseur(1995)) Radiative Transfer Model LIDORT Cloud Top Height SACURA Cloud Fraction OCRA

SGP OL1b-2 ATBD Version 7 Page 35 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

5.9.2. Additional Details

For the retrieval of glyoxal total columns, the DOAS method is used including an offset correction by means of the reference sector method (as in cases of sulphur dioxide and formaldehyde total columns). The only difference is that the offset correction value for glyoxal does not depend on latitude. Another important peculiarity of the glyoxal post-correction scheme is the size of the reference sector: whereas it extents from pole to pole and between 140°W and 180°W for SO2 and HCHO, it extents only between 160°W and 180°W for glyoxal. Vertical proﬁles of glyoxal for AMF calculations are taken from the IMAGES climatology. This climatology contains proﬁles of formaldehyde as well as those of glyoxal with a temporal resolution of 1 month and a spatial resolution of 2°lat x 1.25°lon.

5.9.3. User warning

For glyoxal the same data policy as for formaldehyde is applied: vertical columns from all pixels are written into the product. In case of glyoxal it is recommended to use the vertical columns of pixels with cloud fraction ≤ 20%. For larger cloud fractions the information content is dominated by a climatological information.

SGP OL1b-2 ATBD Version 7 Page 36 of 137 6. Water Vapour Retrieval

This chapter describes the Air Mass Corrected Differential Optical Absorption Spectroscopy (AMC-DOAS) algorithm used for the retrieval of total water vapour columns from measurements SCIAMACHY. The algorithm has also been successfully applied to measurements of the GOME and GOME-2 instruments.

6.1. Inversion Algorithm

The AMC-DOAS algorithm (Noël et al., 1999) is based on the well-known Differential Optical Absorption Spec- troscopy (DOAS) approach (Platt, 1994) which has been modiﬁed to handle effects arising from the strong differential absorption structures of water vapour. The general features of this modiﬁed DOAS method are that 1. saturation effects arising from highly structured differential spectral features which are not resolved by the measuring instrument are accounted for, and

2.O 2 absorption features are ﬁtted in combination with H2O to determine a so-called air mass factor (AMF) correction which compensates to some degree for insufﬁcient knowledge of the background atmospheric and topographic characteristics, like surface elevation and clouds. The main equation of the Air Mass Corrected DOAS method is given by:

I b ln = P − a τO2 + c · CV (6.1) I0

with

I,I0 Earthshine radiance and solar irradiance P Polynomial to correct for broadband contributions (resulting e.g. from Rayleigh and Mie scattering or surface albedo)

τO2 Optical depth of O2

CV Vertical column amount of water vapour b, c Spectral quantities describing saturation effect and absorption

c contains the effective reference absorption cross section and the air mass factor. The scalar parameter a is

the above mentioned AMF correction factor. The quantities τO2 , b, and c are determined from radiative transfer calculations performed for different atmospheric conditions and solar zenith angles (see Section 6.2). CV and a are then derived from a non-linear fit. The error of the vertical column is calculated from the covariance matrix also resulting from the fit. A typical fit result is shown in Figure 6.1

37 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

SCIAMACHY AMC-DOAS H2O Fit Results (27 Jan 2003 10:22) 0.15 0.14

0.13 Data I/I0 0.12 Polynomial Polynomial+O2 Absorption 0.11 Fit 688 690 692 694 696 698 700 Wavelength, nm 4

-2

-4 688 690 692 694 696 698 700 Residual (Data-Fit)/Data, % Wavelength, nm

Figure 6.1.: Example for AMC-DOAS ﬁt results. Top: Spectral ﬁt. Bottom: Residual

The AMC-DOAS method is applied to the spectral region between 688 nm and 700 nm because both O2 and water vapour absorb in this region. They are the main absorbers in this spectral range, having slant optical depths of similar strength (see Figure 6.2). The main purpose of the AMF correction factor is to correct the retrieved water vapour column, but beside this the AMF correction factor can be used as an inherent quality check for the retrieved data. The AMC- DOAS retrieval assumes a cloud-free tropical background atmosphere and does not consider different surface elevations. If the derived AMF correction is too large, this is an indication that these assumptions are not valid (most likely because the observed scene is too cloudy or contains a high mountain area). Experience has shown that retrieved water vapour columns for an AMF correction factor of 0.8 or higher are reliable; only these data are distributed.

Figure 6.2.: Typical slant optical depths of water vapour and O2 in the spectral region around the AMC-DOAS ﬁtting window.

SGP OL1b-2 ATBD Version 7 Page 38 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18 6.2. Forward Model and Databases

The τO2 , b, and c parameters are spectral quantities which are taken from a precalculated data base. This data base has been derived using the radiative transfer model (RTM) SCIATRAN (see e.g. Rozanov et al.,

2002) as a forward model. The term τO2 is determined using radiative transfer calculations with and without O. The parameters b and c are determined from radiative transfer calculations assuming different water vapour columns CV (see Noël et al., 1999, for further details). ◦ ◦ ◦ The spectra for τO2 , b and c have been calculated for a set of solar zenith angles (SZAs), namely 0 , 20 , 40 , ◦ ◦ ◦ ◦ ◦ ◦ 50 , 60 , 70 , 80 , 85 , and 88 . Based on this data set the τO2 , b, and c spectra are then interpolated during the retrieval for the actual SZA. 88◦ is therefore the maximum SZA for which the retrieval produces reliable results. The following assumptions have been made for the radiative transfer calculations: tropical background atmosphere (MODTRAN proﬁle) no clouds surface elevation 0 km

An example for τO2 , b and c is shown in Figure 6.3. Deviations of the real conditions from these assumptions are handled via the retrieved AMF correction factor a.

SCIAMACHY AMC-DOAS H2O RTM Parameters (27 Jan 2003 10:22) 1 0.9 0.8 b 0.7 0.6 b 0.5 688 690 692 694 696 698 700 Wavelength, nm 0.1 0.08 0.06 c 0.04 0.02 c 0 688 690 692 694 696 698 700 Wavelength, nm 0.3 0.25 0.2

O2 0.15 τ 0.1 τ 0.05 O2 0 688 690 692 694 696 698 700 Wavelength, nm

Figure 6.3.: Example for RTM parameters τO2 , b and c.

6.3. Auxiliary Data

Except for the RTM data base described in Section 6.2, the AMC-DOAS method does not rely on any other external data, e.g. calibration factors derived from comparisons with ground based radio sonde measurements as it is the case for e.g. Special Sensor Microwave Imager (SSM/I) data. The retrieved water vapour columns therefore provide a completely independent data set.

SGP OL1b-2 ATBD Version 7 Page 39 of 137 7. Infra-red Retrieval using BIRRA

7.1. Introduction

Nadir sounding of vertical column densities of atmospheric gases is well established in atmospheric remote sensing. For UV instruments such as SCIAMACHY (Bovensmann et al., 1999) the analysis is traditionally based on a DOAS–type methodology. This approach has also been successfully applied for analysis of SCIA- MACHY (Scanning Imaging Absorption spectroMeter for Atmospheric CHartograghY) near infra-red (IR) channels (see e.g. Buchwitz et al., 2007; Gloudemans et al., 2005; Frankenberg et al., 2005) and was the basis of the BIAS (Basic Infra-red Absorption Spectroscopy) non-linear least squares algorithm (Spurr, 1998). In addition to the column scale factors for the molecular densities a one-parameter “closure term” is fitted to account for any other effects such as (single or multiple) scattering, surface reflection or instrumental artefacts. However, recent sensitivity studies have shown the importance of adequate modelling the instrumental slit function; in particular an under- or overestimate of the slit function half width can have a significant impact on the retrieved columns. Furthermore, in BIAS molecular absorption is taken into account using look–up tables that have been calculated for a coarse altitude grid version of the US standard atmosphere (Anderson et al., 1986) and a dated set of spectroscopic line parameters. Likewise the slant path optical depth is interpolated from look-up tables generated for a finite set of path geometries. In order to gain greater flexibility in the forward modelling and a more efficient and robust least squares inversion, a “Beer Infra-Red Retrieval Algorithm” (BIRRA) has been implemented Gimeno García et al.(2011) : Forward model based on GARLIC (Generic Atmospheric Radiative Transfer Line-by-Line Infrared Code, (Schreier et al., 2014) describing radiance according to Beer’s law; Non-linear least squares exploiting special structure of the forward model; Flexible choice of additional fit parameters besides molecular columns; Option for ’online’ line-by-line absorption cross section computation and continua.

7.1.1. A ﬁrst glimpse at near IR nadir observations

Carbon monoxide is an important atmospheric trace gas, highly variable in space and time, that affects air quality and climate. About half of the CO emissions come from anthropogenic sources (e.g. fossil fuel combustion), and further significant contributions are due to biomass burning. CO is a target species of several space-borne instruments, e.g. AIRS, MOPITT, and TES from NASA’s EOS satellite series, and SCIAMACHY. However, carbon monoxide retrieval from SCIAMACHY nadir observations is rather challenging: Only channel 8 from 2259 to 2386 nm features CO absorption signatures, albeit very weak and superposed by stronger absorption lines of concurrent gases, i.e. H2O and CH4, see Figure7.1. Additionally, an ice layer on the detector modifies the measured signal. Even worse, degradation of the detector increasingly reduces the number of reliable pixels, i.e. only about 50 of 1024 pixels in channels 8 are useful for CO retrieval, when using the WFM-DOAS bad/dead pixel mask. The “default” atmospheric model used in the original BIAS code is based on the US standard atmosphere (Anderson et al.(1986)). The influence of the atmospheric model on the vertical path transmission is shown in 7.2. Aerosols and clouds can be important in NIR radiative transfer modelling, cf. 7.3.

40 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

0.8

0.6 Transmission total H O 0.4 2 CH4

0.2 1

0.998

0.996 CO

N2O

0.994 NH3 Transmission

0.992

0.99 4200 4250 4300 4350 4400 Wavenumber [cm−1] Mon Feb 18 14:06:13 2008

Figure 7.1.: Atmospheric transmission in channel 8. MODTRAN band model, US standard atmosphere, vertical path 0 – 100km, no aerosols. The yellow bars indicate the micro-window 4282.686 – 4302.131 cm−1 as deﬁned by the WFM–DOAS bad/dead pixel mask.

SGP OL1b-2 ATBD Version 7 Page 41 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

0.8

0.6

tro mls

Transmission mlw 0.4 sas saw US

0.2

0 4200 4250 4300 4350 4400 Wavenumber [cm−1]

Figure 7.2.: Comparison of vertical transmission (0 – 100km) assuming various atmospheric models.

Influence of Aerosols

fascode, downlooking, clear sky vs haze vs haze+stratoBackground aerosol 1.50e−11

US7_75.0km_180.0dg.r US7_75.0km_180.0dg_H1.r US7_75.0km_180.0dg_H1_S1.r )]

−1 1.00e−11 sr cm 2

5.00e−12 Radiance [W/(cm

0.00e+00 4270 4280 4290 4300 4310 4320 Wavenumber [cm−1] Fri Aug 31 13:14:58 2007

Figure 7.3.: Comparison of vertical transmission assuming various aerosol models.

SGP OL1b-2 ATBD Version 7 Page 42 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18 7.2. Forward Modelling

7.2.1. Radiative Transfer

7.2.1.1. General Considerations: Schwarzschild and Beer

For an arbitrary slant path, the intensity (radiance) I at wavenumber ν received by an instrument at position s = 0 is given by (neglecting scattering and assuming local thermodynamic equilibrium) the (integral form of the) Schwarzschild equation(Liou, 1980):

sb Z ∂T (ν, s0) I(ν) = I (ν) T (ν, s ) − B (ν, T (s0)) ds0, (7.1) b b ∂s0 0

where B is the Planck function at temperature T , and Ib is the background contribution at position sb. The monochromatic transmission T is given according to Beer’s law by

 s  Z −τ ν,s 0 0 T (ν, s) = e ( ) = exp − α(ν, s ) ds  (7.2) 0  s  Z 0 X 0 0 0 = exp − ds km (ν, p(s ),T (s )) nm(s ) ,  (7.3) m 0

where τ is the optical depth, α is the absorption coefﬁcient, p is the atmospheric pressure, nm is the number density of molecule m and km is its absorption cross section. In high resolution line–by–line models the absorption cross section is obtained by summing over the contributions from many lines, X k(ν, z) = Sl(T (z)) gl(ν;ν ˆl, γl(p(z),T (z))) (7.4) l

where each individual line is described by the product of the temperature–dependent line strength Sl and a normalized line shape function g describing the broadening mechanism (for simplicity corrections due to continuum effects (Clough et al., 1989) are neglected here). For the infra-red and microwave the combined effect of pressure broadening (corresponding to a Lorentzian line shape) and Doppler broadening (corresponding to a Gaussian line shape) can be represented by a Voigt line profile. To compute the spectral radiance and transmission (or equivalently, the optical depth) the integrals in equations 7.1 and 7.2 have to be discretized. Note that in general the atmospheric pressure, temperature, and concentration profiles are given only as a finite sets of data points with a typical altitude range up to about 100 km (see e.g. Anderson et al., 1986). A common approach is to subdivide the atmosphere in a series of homogeneous layers, each described by appropriate layer mean values for pressure, temperature, and concentrations; then transmission T is given by the product of the layer transmissions, and the radiance can be calculated recursively (Clough et al., 1988; Edwards, 1988) (“Curtis–Godson approach”). Obviously the integrals in Equations 7.1 and 7.2 can also be evaluated easily by application of standard quadrature schemes (Kahaner et al., 1989).

7.2.1.2. Radiative transfer modelling of nadir near infra-red observations

In the near infra-red contributions from reﬂected sunlight become important, whereas thermal emission is negligible. Furthermore scattering can be neglected for clear sky (cloud free) observations. Thus the radiative transfer equation simpliﬁes to I(ν) = r Isun(ν) T↑(ν) T↓(ν) (7.5)

where r is the reflection factor and T↓ and T↑ denote the transmission between reflection point (e.g. Earth surface) and observer and between sun and reflection point, respectively.

SGP OL1b-2 ATBD Version 7 Page 43 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

The monochromatic transmission T (relative to the observer) is given according to Beer’s law by Z s T (ν; s) = exp[− α(ν, s0) ds0] , (7.6) 0 X (c) α(ν; s) = km(ν, s) nm(s) + α (ν, s) (7.7) m

where α is the volume absorption coefficient, km and nm are the absorption cross section and density of molecule m, and α(c) the continuum absorption coefficient. Note that the absorption cross section is a function of (altitude dependant) pressure and temperature, i.e. k(ν, z) = kν, p(z),T (z). In the infra-red and microwave spectral range molecular absorption is due to radiative transitions between rotational and vibrational states of the molecules. A single spectral line is characterized by its position νˆ, line strength S, and line width γ, where the transition wavenumber (or frequency) is determined by the energies Ei, Ef of the initial and final state, |ii, |fi, 1 νˆ = (E − E ) (7.8) hc f i

7.2.2. Molecular Absorption

7.2.2.1. Line strength and partition functions

The monochromatic absorption cross section for a single line is deﬁned as the product of the line strength S and a normalized line proﬁle function g essentially determined by line broadening,

∞ Z+ k(ν;ν, ˆ S, γ) = S · g(ν;ν, ˆ γ) with g dν = 1 . (7.9) −∞

For electric dipole transitions the line strength is determined by the square of the temperature dependent matrix element of the electric dipole moment and by further factors accounting for the partition function, Boltzmann- distribution, and stimulated emission,

3 8π giIa −Ei/kT −hcν/kTˆ −36 S(T ) = νˆ e [1 − e ] Rif · 10 (7.10) 3hc Q(T ) here gi is the degeneracy of the nuclear spin of the lower energy state, Ia is the relative abundance of the iso- 1 tope , Q(T ) is the total partition sum, Rif is the transition probability given by the matrix element of the electric 2 ~ dipole operator Rif = hf|D|ii . A similar expression is found for the line strength of magnetic quadrupole transitions. In both cases the ratio of line strength at two different temperatures is given by

Q(T0) exp (−Ei/kT ) 1 − exp (−hcν/kTˆ ) S(T ) = S(T0) × . (7.11) Q(T ) exp (−Ei/kT0) 1 − exp (−hcν/kTˆ 0)

Q(T ) is the product of the rotational and vibrational partition functions, Q = Qrot · Qvib, whose temperature dependence are calculated from

T β Qrot(T ) = Qrot(T0) , (7.12) T0 N Y −di Qvib(T ) = [1 − exp(−hcωi/kT )] , (7.13) i=1

where β is the temperature coefﬁcient of the rotational partition function, and N is the number of vibrational

1 In the HITRAN– and GEISA databases the abundances of the Earth atmosphere are used.

SGP OL1b-2 ATBD Version 7 Page 44 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18 modes with wavenumbers ωi and degeneracies di. Data required to calculate the vibrational partition sums have been taken from Norton and Rinsland(1991).

7.2.2.2. Line broadening

Even with an ideal spectrometer with full spectral resolution no infinite narrow ”delta function” absorption line could be observed. However, the natural line width which results from the finite lifetime of the excited state and is described by a Lorentz profile, can be neglected in atmospheric spectroscopy.

Pressure (collision) broadening - Lorentz proﬁle In case of pure pressure broadening the cross section for a single radiative transition is essentially given by a Lorentzian line proﬁle

γ /π L (7.14) gL(ν) = 2 2 . (ν − νˆ) + γL

The Lorentz half width (at half maximum, HWHM) γL is proportional to pressure p and decreases with in- creasing temperature. In case of a gas mixture with total pressure p and partial pressure ps of the absorber molecule the total width is given by the sum of a self broadening contribution due to collisions between the absorber molecules and an air-broadening contribution due to collisions with other molecules, n (0,air) p − ps (0,self) ps T0 γL(p, ps,T ) = γL + γL × . (7.15) p0 p0 T

The exponent n specifying the dependence of temperature is so far known for only a few transitions of the most 1 important molecules. The kinetic theory of gases (collision of hard spheres) yields the classical value n = 2 . (self) The self-broadening coefficient γL is so far known for only a few transitions and will otherwise be set to the (air) air-broadening coefficient γL (mostly specified for N2 and/or O2), i.e. n (air) p T0 γL(p, T ) = γL × (7.16) p0 T

Typical values of air-broadening coefﬁcients are γL ≈ 0.1p [cm/atm](see Table 2 in Rothman et al., 1987).

Doppler broadening The thermal motion of the molecules leads to Doppler broadening of the spectral lines, which is described by a Gaussian line shape " # 1 ln 21/2 ν − νˆ2 gD(ν) = · exp − ln 2 . (7.17) γD π γD

The half width (HWHM) is essentially determined by the line position νˆ, the temperature T , and the molecular mass m, r 2 ln 2 kT γ =ν ˆ . (7.18) D mc2 For a typical atmospheric molecule one ﬁnds

−8 p γD ≈ 6 · 10 νˆ T [K] for m ≈ 36 amu.

Combined Pressure and Doppler broadening

gV (ν) = gL ⊗ gD (7.19)

SGP OL1b-2 ATBD Version 7 Page 45 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

1 Several empirical approximations for the half width (HWHM) of a Voigt line (deﬁned by gV (ˆν ± γV ) = 2 gV (ˆν)) have been developed (Olivero and Longbothum, 1977). For the approximation q 1 2 2 γV = c γ + c γ + 4γ with c = 1.0692, c = 0.86639 (7.20) 2 1 L 2 L D 1 2

a accuracy of 0.02% has been speciﬁed, with c1 = c2 = 1 the accuracy is in the order of one percent. A comparison of Lorentzian, Doppler, and Voigt half width is given in Figure 7.4.

Figure 7.4.: Half widths (HWHM) for Lorentz-, Doppler- and Voigt-Proﬁle as a function of altitude for a variety of line positions ν. The Lorentz width is essentially proportional to pressure and hence decays approximately exponentially with altitude. In contrast the Doppler width is only weakly altitude dependent. In the troposphere lines are generally pressure broadened, the transition to the Doppler regime depends on the spectral region. The dotted line indicated atmospheric temperature. (Pres- sure and temperature: US Standard atmosphere, molecular mass 36 amu).

Numerical√ Aspects — The Voigt function It is convenient to introduce the Voigt function K(x, y) (normalized to π) deﬁned by p ln 2/π gV(ν;ν, ˆ γL, γD) = K(x, y) , (7.21) γD 2 y Z ∞ e−t (7.22) K(x, y) = 2 2 dt , π −∞ (x − t) + y

where the dimensionless variables x, y are deﬁned in terms of the distance from the line centre, ν − νˆ0, and the Lorentzian and Doppler half–widths γL, γD: √ ν − νˆ √ γ x = ln 2 0 and y = ln 2 L . (7.23) γD γD

The Voigt function represents the real part of the complex function W (z) with z = x + iy that, for y > 0, is

SGP OL1b-2 ATBD Version 7 Page 46 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

identical to the complex error function (probability function) deﬁned by Abramowitz and Stegun(1964)

z 2 2i Z 2 2 2 w(z) = e−z 1 + √ et dt = e−z 1 − erf(−iz) = e−z erfc(−iz) . (7.24) π 0 The complex error function satisﬁes the differential equation

2i w0(z) = −2z · w(z) + √ (7.25) π and the series and asymptotic expansions

∞ X (iz)n w(z) = (7.26) Γ n + 1 n=0 2 ∞ 1 √ i X Γ k + i π w(z) = 2 = + ... (7.27) π z2k+1 π z k=0

2 2 For vanishing arguments x or y one has K(0, y) = ey 1 − erf(y) and K(x, 0) = e−x respectively, where erf is the error function. Truncating the asymptotic expansion of the complex error function readily shows that the wing of the Voigt proﬁle is approximated by a Lorentzian. Further mathematical properties and relationships of the Voigt function and complex error function can be found in the extensive review by Armstrong(1967) or in Abramowitz and Stegun(1964).

Approximation of an arbitrary function by a rational function, i.e. the quotient PM /QN of two polynomials of degree M and N is generally superior to polynomial approximations (Ralston and Rabinowitz, 1978). For the computation of the Voigt function GARLIC and BIRRA uses a combination of the Humlícekˇ (1982) and the Weideman(1994) rational approximations (Schreier, 2011).

Numerical Aspects — Computational Challenges The computational challenge of high resolution atmospheric radiative transfer modelling is due to several facts. The summation in Equation 7.4 has to include all relevant lines contributing to the spectral interval considered. In many line-by-line codes a cut-off wavenumber of 25 cm−1 from line centre is frequently employed for truncation of line wings. Note that the widely used HITRAN and GEISA spectroscopic databases (Rothman et al., 2003; Jacquinet-Husson et al., 2008) list more than a million lines of about 40 molecules in the microwave, infra-red, to ultraviolet regime, whereas the JPL spectral line catalogue (Pickett et al., 1998) covering the submillimetre, millimetre, and microwave only has almost 2 million entries. Furthermore the wavenumber grid has to be set in accordance with the line widths γ, i.e. the grid spacing is typically chosen in the order of δν ≈ γ/4. Typical line widths due to pressure broadening are in the order of γ(p) ≈ (p/p0) 0.1cm with p0 = 1013 mb. In the atmosphere the pressure decays approximately exponentially with altitude z, and the line width decreases accordingly until Doppler broadening (proportional to line position and the square root of the temperature over molecular mass ratio) becomes dominant (cf. Figure 7.4). For a spectral interval of width ∆ν = 10cm in the region of the CO2 ν2 band around 500 cm the number of spectral grid points is in the order of 105. A variety of approaches has been developed to speed–up the calculation and an essential difference between different line-by-line codes is the choice of the line profile approximation, wavenumber grid, and interpolation. Some of the algorithms are specifically designed for the individual functions to be calculated, e.g. the (Clough and Kneizys, 1979) algorithm used in FASCODE (Clough et al., 1988): The Lorentzian (or Voigt function) is decomposed using three or four even quartic functions, each of them is then calculated on its individual grid (a similar technique using quadratic functions has been developed by Uchiyama, 1992). GENLN2 (Edwards, 1988) performs the line–by–line calculation in two stages, i.e., the entire spectral interval of interest is first split in a sequence of “wide meshes”; contributions of lines with their centre in the current wide mesh interval are computed on a fine mesh, and the contribution of other lines is computed on the wide mesh. Fomin(1995) defines a series of grids and evaluates line wing segments of larger distance to the line centre on increasingly

SGP OL1b-2 ATBD Version 7 Page 47 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

coarse grids. Sparks(1997) also uses a series of grids with 2k + 1 grid points (k = 1, 2,... , where the coarsest grid with 3 points spans the entire region) and uses a function decomposition similar to ours.

7.2.3. Geometry

Figure 7.5.: Observation geometry for SCIAMACHY nadir.

The line–of–sight (LoS) in an one-dimensional spherical atmosphere is usually deﬁned by the observer position (altitude) and the viewing angle (zenith angle), with α = 0 for a vertical uplooking, and α = 180◦ for a vertical donwlooking (nadir) path. For calculation of the path geometry it is more appropriate to use radii instead of altitudes, e.g.

robs = rEarth + zobs (7.28)

rend = rEarth + zend (7.29)

for the observer point and the path end point. For all observation geometries the radius (altitude) of the LoS tangent point to the observer is given by rt = robs · sin α (7.30)

7.2.3.1. Uplooking

By convention, uplooking paths are characterized by zenith angles α < 90◦, see Figure 7.6. The distance of the tangent point to the observer and to the path end point (usually at top-of-atmosphere, ToA) are given by

t = robs · cos α (7.31) q 2 2 s = rend − rt − t (7.32)

The angle at the path end point and the earth centred angle are r β = arcsin t (7.33) rend r π ψ = arccos t + α − (7.34) rend 2

SGP OL1b-2 ATBD Version 7 Page 48 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Alternatively the angles can be computed according to the sine theorem r r s obs = end = (7.35) sin β sin(π − α) sin ψ

β s

rToA α

robs

rEarth

Figure 7.6.: Geometry of uplooking path.

7.2.3.2. Downlooking

◦ Downlooking paths are deﬁned by zenith angles α > 90 and a tangent point altitude ht < 0, see Figure 7.7. The length of the path is given by q 2 2 s = robs − rt − t (7.36) with

t = robs · cosα ¯ (7.37)

and angles are given by r β = π − arcsin t (7.38) rend r r ψ = arccos t − arccos t (7.39) robs rend

SGP OL1b-2 ATBD Version 7 Page 49 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

α¯ s

robs β t

rEnd rt

rEarth ψ

Figure 7.7.: Geometry of downlooking path.

7.2.3.3. Refraction

Because of refraction the line of sight is curved towards the Earth centre. For a atmospheric path deﬁned by observer position and viewing angle (w.r.t. zenith) this results in a lower tangent point of the path an increased path length Path refraction is characterized by an refractive index η different from 1.0 (vacuum). Assuming a layered atmosphere with constant values of pressure, temperature, densities, and hence refractive index in each layer, the path geometry can be deduced from Snell’s law ηl sin βl = ηl+1 sin αl+1 (7.40)

Triangle geometry r r l = l+1 (7.41) sin βl sin αl

The refractive index is calculated according to the recipe given for the MIPAS NRT processor (Carlotti et al., 1998) 19 3 η(n) − 1 = 0.000272632 ∗ n/n0 with n0 = 2.54683 · 10 molec/cm (7.42)

SGP OL1b-2 ATBD Version 7 Page 50 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Figure 7.8.: Refraction for an uplooking path

For an uplooking path starting at an altitude h0 = r0−rEarth with zenith angle α0, the refracted path is calculated in a step-wise fashion, cf. Figure 7.8. First the path incident angle at the top of the current layer, i.e. just below next level is obtained from rl βl = arcsin( sin αl) rl+1

The length of the path segment between altitudes hl and hl+1 is given by p sl = (rl+1 − rl sin αl) · (rl+1 + rl sin αl) − rl cos αl

Using Snell’s law (Equation 7.40) the next zenith angle is computed as η¯l,l+1 αl+1 = arcsin · sin βl+1 η¯l+1,l+2 where η¯ is the mean refractive index of the atmospheric layer between rl and rl+1. For SCIAMACHY nadir observations, refraction is only taken into account for the Sun — Earth surface path element, that can be modelled as an uplooking path for a ground based “observer” with viewing angle deﬁned by the solar zenith angle (SZA), see Figure 7.5. For the “downlooking” path segment (satellite — Earth surface) with viewing angles ≤ 30◦ refraction can be neglected.

7.2.4. Instrument - Spectral Response

The instrumental response is taken into account by convolution of the monochromatic intensity spectrum (7.1) with a spectral response function S (a.k.a. instrumental line shape function ILS) Z ∞ I[(ν) ≡ (I ⊗ S)(ν) = I (ν) × S (ν − ν0) dν0 (7.43) −∞ SGP OL1b-2 ATBD Version 7 Page 51 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

(in general a further convolution will be required to account for the finite field of view, however, this is usually negligible for nadir viewing). For SCIAMACHY NIR measurements Gaussian, hyperbolic or Voigt profiles are commonly used, " # 1 ln 21/2 ν 2 SG(ν, γ) = · exp − ln 2 (7.44) γ π γD √ 2γ3/π SH (ν, γ) = (7.45) [ν4 + γ4]

SV (ν, γL, γG) = SL(ν, γL) ⊗ SG(ν, γG) (7.46) where the Voigt proﬁle 7.19 is a convolution of the Gaussian with a Lorentzian proﬁle

2 2 SL(ν, γ) = γ/ π(ν + γ ) .

∞ Note that all proﬁles deﬁned here are normalized to unity, i.e. R S(ν) dν = 1. −∞

7.2.5. Input data for forward model

Modelling high resolution infra-red radiative transfer is usually done by means of line-by-line models reading molecular spectroscopic data from databases such as Hitran (Rothman et al., 2009; Gordon et al., 2017) or Geisa (Jacquinet-Husson et al., 2016), see Figure 7.11. Whereas spectroscopic data of methane and carbon monoxide have not been changed in recent versions of these databases, water spectroscopic data have been updated recently, see Figure 7.9.

H2O

−22 Hitran 2004: 625 lines with S<1.13e−22 10 Geisa 2003: 689 lines with S<1.24e−22 )] 2

10−24 /(molec.cm −1 10−26 S [cm

10−28 )] −22 −2 10

10−24

10−26

−28 Cross section [1/(molec.cm 10 4200 4250 4300 4350 4400 position [cm−1] Mon Oct 16 13:30:50 2006

Figure 7.9.: Water lines in SCIAMACHY channel 8 — Inter-comparison of recent Hitran 2004 and Geisa 2003 line data and corresponding cross sections. (H2O data in Hitran 2000 are approximately identical to Geisa).

SGP OL1b-2 ATBD Version 7 Page 52 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Temperature Dependence of Line Strength

Hitran 2004; Tref=296K −−> T=200K 10−20

10−21

10−22

10−23

10−24

10−25

−26 Line Strength 10

10−27 H2O −28 10 CO CH4 10−29

10−30 4280 4290 4300 4310 Wavenumber [cm−1]

Figure 7.10.: Temperature dependence of molecular spectroscopic lines in SCIAMACHY channel 8 (Hitran 2004 database) indicated by the length of the vertical “error” bars.

Figure 7.11.: Molecular spectroscopic lines in SCIAMACHY channel 8 according to the Geisa database.

SGP OL1b-2 ATBD Version 7 Page 53 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

120.0 H2O 80

40 100.0 Tropical Midlatitude summer 20 Midlatitude winter 0 Subarctic summer 108 1010 1012 1014 1016 1018 CH4 Subarctic winter 80 80.0 US standard 60

40 Altitude [km] 60.0 20

0 106 108 1010 1012 1014 CO Altitude (km) 80

40.0 60

20 20.0 0 108 109 1010 1011 1012 1013 VMR [ppm]

0.0 150.0 200.0 250.0 300.0 Temperature (K)

Figure 7.12.: Temperature proﬁles (left) and volume mixing ratios (right) listed in the AFGL model atmosphere data set.

Kurusz Planck 6000K Planck 6250K

4.5 )] −1 cm 2

4 W/(cm µ Irradiance [

3.5

3 4000 4200 4400 4600 4800 5000 Wavenumber [cm−1]

Figure 7.13.: Solar irradiance

SGP OL1b-2 ATBD Version 7 Page 54 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18 7.3. Inversion

7.3.1. Non-linear least squares

The objective of SCIAMACHY near infra-red measurements in nadir viewing geometry is to retrieve information on the vertical distribution of trace gases such as N2O, CH4 or CO, e.g. the volume mixing ratio qX (z) or number density nX (z) = qX (z) · nair(z) of molecule X. The standard approach to estimate the desired quantities ~x from a measurement ~y (a vector of m components) relies on (in general non-linear) least squares ﬁt

2 min ~y − F~ (~x) . (7.47) x

Here F~ denotes the forward model Rn → Rm essentially given by the radiative transfer and instrument model. Because of the ill-posed nature of vertical sounding inverse problems, it is customary to retrieve column densities Z ∞ NX (z0) ≡ nx(z) dz , (7.48) z0

where z0 is the ground elevation (surface altitude).

Denoting by αm the scale factors to be estimated and by n¯m(z) the reference (e.g. climatological) densities of molecule m, the up-welling monochromatic radiance 7.5 can be written as

 ∞  Z 0 dz X 0 0 I(ν) = rµ Isun(ν) exp − αmn¯m(z ) km(ν, z ) µ m 0  ∞  Z 00 dz X 00 00 exp − αmn¯m(z ) km(ν, z ) (7.49) µ m 0

where for simplicity we have used the plane–parallel approximation with µ ≡ cos θ for an observer zenith angle θ and µ for the solar zenith angle θ ; furthermore continuum is neglected here. Introducing the total optical depth of molecule m (for the reference proﬁles and the entire path),

F~ (~x) ≡ I[(ν) (7.50) P − αmτm(ν) = rµ Isun(ν) e m ⊗ S(ν, γ) + b ,

where the state vector ~x is comprised of geophysical and instrumental parameters ~α, γ, r and an optional baseline correction b (generally a polynomial). For the solution of the non-linear least squares problem 7.47 BIRRA uses solvers provided in the PORT Opti- mization Library(Dennis, Jr. et al., 1981) based on a scaled trust region strategy. BIRRA provides the option to use a non-linear least squares with simple bounds (e.g. non-negativity) to avoid non-physical results, e.g.

2 min ~y − F~ (~x) . (7.51) x>0

7.3.2. Separable non-linear least squares

Note that the surface reﬂectivity r and the baseline correction b enter the forward model F~ ≡ I(\ν; ... ), 7.50, linearly and the least squares problem 7.47 can be reduced to a separable non-linear least squares problem (Golub and Pereyra, 2003). Splitting the vector ~x of parameters to be ﬁtted into a vector ~α of non-linear parameters and a vector β~ of linear parameters, i.e.

n p q ~x −→ (~α, β~) with ~x ∈ R , ~α ∈ R , β~ ∈ R and n = p + q, (7.52)

SGP OL1b-2 ATBD Version 7 Page 55 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

the forward model can be written as q X F~ (~x) −→ βlf~l(~α) . (7.53) l=1 ~ m ~ m where F ∈ R and fl ∈ R for l = 1, . . . , q. Combining these functions in a matrix ~ ~ ~ m×q A(α) ≡ f1(~α), f2(~α),..., f2(~α) with A ∈ R (7.54)

2 ~ ~ 7.47 is a linear least squares problem minβ ~y − Aβ for the vector β, that is (formally) solved by

−1 β~ = AT A AT~y. (7.55)

Inserting this solution in 7.53, the original least squares problem 7.47 becomes

2 X T −1 T min ~y − A A A ~y f~l(~α) . (7.56) α l l

This is a non-linear least squares problem for ~α independent of β~ and can be solved in the usual way by means of Gauss–Newton or Levenberg–Marquardt algorithms. Once the optimum ~α is found, the linear parameter vector β~ is obtained from 7.55. The main advantages of this approach are The non-linear least squares solver has to iterate only for a reduced ﬁt vector ~α no initial guess is required for the linear parameters β~ The size of the Jacobian matrix is reduced

7.4. CO Retrieval

7.4.1. Detailed Description

Carbon monoxide (CO) is an important trace gas affecting air quality and climate. It is highly variable in space and time. About half of the CO comes from anthropogenic sources (e.g. fuel combustion), and further significant contributions are due to biomass burning. CO is a target species of several space-borne instruments, i.e. for AIRS, MOPITT, and TES from NASA’s EOS satellite series, and MIPAS and SCIAMACHY on ESA’s EN- VISAT. For the retrieval of carbon monoxide the spectral interval 4282.68615 to 4302.13102 cm−1, i.e. 2.3244 µm to 2.3350 µm was used. Pressure, temperature, and H2O profiles were taken from the NCEP reanalysis providing profiles four times per day with a latitude / longitude resolution of 2.5 dg (Kistler et al., 2001). An US standard atmosphere was assumed for the molecular density profiles. Surface reflectivity was modelled with a second order polynomial, baseline was ignored. Correction factors for CO, CH4 and H2O are simultaneously retrieved. The product contains the correction factors, the resulting columns from the multiplication of the correction factors with the starting values and a "CH4 corrected" value of the total CO column. The underlying assumption for the latter is that CH4 is homogeneously distributed compared to CO. The division of the correction factor for CO by the correction factor for CH4 corrects approximately for remaining instrument effects, clouds in the FoV, etc:

αCO xCO = VCDCO,ref · (7.57) αCH4

Additionally the product also contains the directly retrieved CO column:

CO = VCDCO,ref · αCO (7.58)

SGP OL1b-2 ATBD Version 7 Page 56 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

CO is a very weak absorber. Additionally, bad pixels and a growing ice layer in channel 8 hamper the retrieval. Thus, the most likely use case for the CO retrieved values are averages over a larger data set (e.g. monthly means). The quality ﬂags in the products should be examined.

7.4.2. Retrival Settings

Level 1b-1c Settings Calibration All calibrations except polarisation and radiometric SMR A0 (Sun over ASM diffuser without radiometric calibration) Main Settings Fitting Interval 2324.4 - 2335.0 nm

Absorbers Fitted CO,CH4,H2O Polynomial Degree Albedo 2 Slitfunction Gaussian

Proxy for xCO CH4

7.5. CH4 Retrieval

7.5.1. Detailed Description

The CH4 retrieval uses 2 spectral windows in channel 6. As a proxy correction to take into account the effect of clouds and transmission changes CO2 is used, since its variations are small compared to methane. As for CO two columns can be found in the product, the total column

CH4 = VCDCH4ref · αCH4 (7.59) and the total column corrected by the CO2 proxy

αCH4 xCH4 = VCDCH4ref · (7.60) αCO2

7.5.2. Retrieval Settings

Level 1b-1c Settings Calibration All calibrations except polarisation and radiometric SMR A0 (Sun over ASM diffuser without radiometric calibration) Main Settings Fitting Interval 1557.18 - 1594.13 nm & 1628.93 - 1670.56 nm

Absorbers Fitted CO2,CH4,H2O Polynomial Degree Albedo 2 Slitfunction Gaussian

Proxy for xCH4 CO2

SGP OL1b-2 ATBD Version 7 Page 57 of 137 Part III.

Limb Retrieval Algorithms

58 8. Limb Algorithm General Description

8.1. Summary of Retrieval Steps

Before we describe the mathematical background of the retrieval in detail we give a short description of the processing and relate the steps to the following sections. Input for the retrieval are fully calibrated Level 1c data ratioed with a measured spectrum at a reference height. Geometry of the problem: Section 8.2.2. In order to calculate the radiation ﬁeld for the forward model, we ﬁrst formulate the line-of-sight geometry and the coordinate system.

8.2. Forward model radiative transfer

8.2.1. General considerations

The radiative transfer equation for the diffuse radiation is:

dI (r, Ω) = −σ (r) I (r, Ω) + J (r, Ω) , ds ext where I (r, Ω) is the radiance at the point r in the direction characterized by the unit vector Ω, ds is the differential path length in the direction Ω, σext is the extinction coefﬁcient and J (r, Ω) is the source function. In the following discussion we omit for simplicity speciﬁc reference to wavelength dependence. The direction Ω is characterized by the zenith and azimuth angles θ and φ, respectively, and we indicate this dependency by writing Ω = Ω (θ, φ). The source function can be decomposed into the single and multiple scattering source terms J (r, Ω) = Jss (r, Ω) + Jms (r, Ω) .

The single scattering source function contains the solar pseudo-source term and the thermal emission as determined by a Planck function B (T ),

σscat (r) −τ sun |r−r r | J (r, Ω) = P (r, Ω, Ω ) F e ext ( TOA( ) ) + σ (r) B (T (r)) , ss 4π sun sun abs while the multiple scattering source function is given by Z σscat (r) 0 0 0 Jms (r, Ω) = P (r, Ω, Ω ) I (r, Ω ) dΩ . 4π 4π

In the above relations, Ωsun = Ω (θsun, φsun) is the unit vector in the sun direction (or the solar direction), Z 0 0 τext (|r1 − r2|) = σext (r ) ds , |r1−r2|

is the extinction optical depth between the points r1 and r2, rTOA (r) is the point at the top of the atmosphere corresponding to r, rTOA = r−|rTOA − r| Ωsun, Fsun is the incident solar ﬂux, σscat is the scattering coefﬁcient and P (r, Ω, Ω0) is the effective scattering phase function with Ω and Ω0 being the incident and scattering directions, respectively.

59 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

In practical applications it is more convenient to introduce the function

0 0 S (r, Ω, Ω ) = σscat (r) P (r, Ω, Ω ) and to express the single scattering contribution as

Fsun −τ sun |r−r r | J (r, Ω) = S (r, Ω, Ω ) e ext ( TOA( ) ) + σ (r) B (T (r)) ss 4π sun abs and the multiple scattering term as Z 1 0 0 0 Jms (r, Ω) = S (r, Ω, Ω ) I (r, Ω ) dΩ . 4π 4π

Beginning at a reference point rr and performing an integration along the path |r − rr|, the following integral form of the radiative transfer equation can be obtained:

−τ(|r−rr|) I (r, Ω) = I (rr, Ω) e

Z 0 + J (r0, Ω) e−τ(|r−r |)ds0. |r−rr|

At this stage of our presentation we consider a discretization of the line of sight as shown in Figure 8.1. To compute the limb radiance at the top of the atmosphere on the near side of the tangent point we use the integral form of the radiative transfer equation and derive a recurrence relation for the diffuse radiance at a set Np of discrete points along the line of sight. With {rp}p=1 being the set of intersection points of the line of sight with a sequence of spherical surfaces of radii rp, i = 1, 2, ..., Np, we apply the integral form of the radiative transfer equation to a layer p, with boundary points rp and rp+1, and obtain

−σ¯ext,p4p Ip (ΩLOS) = Ip+1 (ΩLOS) e

Z 0 0 −σ¯ext,p|rp−r | 0 + J (r , ΩLOS) e ds . (8.1) |rp−rp+1|

In the above relation Ip (ΩLOS) = I (rp, ΩLOS) is the intensity at the point Mp, 4p = |rp − rp+1| is the limb/nadir path and σ¯ext,p is the extinction coefﬁcient on the layer p. To transform (8.1) into a computational expression we have to describe more precisely the scattering geometry, to characterize the optical properties of the atmosphere and to evaluate the source terms.

Z sun

 LOS , p LOS e A rp p e p TOA e  p p M p

M p1 sun , p r p sun Earth r p1

Figure 8.1.: Integration along the line of sight

SGP OL1b-2 ATBD Version 7 Page 60 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

8.2.2. Geometry of the scattering problem

In this section we derive some geometrical parameters under the assumption that the input data are

1. the zenith angle of the line of sight θLOS,

2. the zenith angle of the sun direction θsun and

3. the relative azimuthal angle between the line of sight and the sun direction ϕ0.

Line of sight 2 Z er ,2 Line of sight 1 T 2 A sun ,2 A' 2 2 LOS er ,1

esun A T 1 1 sun ,1 A'1 LOS

esun

rTOA r sin  TOA LOS ,1 LOS ,2

LOS ,1 X

Figure 8.2.: For a sequence of limb scans characterized by different tangent levels, the solar zenith angle and the zenith angle of the line of sight are scan dependent.

By convention, we assume that these angles are speciﬁed at the top of the atmosphere in a coordinate system attached to the near point of the line of sights. We choose the Earth coordinate system such that the X-axis is parallel to the line of sight ΩLOS , that is, ΩLOS = ex.

The spherical coordinate system (er, eθ, eϕ) correspond to the near point A on the line of sight, while the spherical coordinate system (er, eθ1, eϕ1) is obtained by a plane rotation of the coordinate system (er, eθ, eϕ) with the angle ϕ0. Note that for a sequence of limb scans characterized by different tangent levels, the solar zenith angle θsun, the zenith and azimuthal angles of the line of sight θLOS and ϕ0, respectively, are scan dependent. The spherical unit vectors are given by the relations:

er = cos θLOSex + sin θLOSez,

eθ = sin θLOSex − cos θLOSez,

eϕ = ey, and

eθ1 = cosϕ0 sin θLOSex + sinϕ0ey − cosϕ0 cos θLOSez (8.2)

eϕ1 = −sinϕ0 sin θLOSex + cosϕ0ey + sinϕ0 cos θLOSez (8.3)

SGP OL1b-2 ATBD Version 7 Page 61 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

while the unit vector of the sun direction esun = −Ωsun, read as

esun = cosθsuner + sinθsuneθ1

= (sinθsuncosϕ0sinθLOS + cosθsuncosθLOS) ex + sinθsunsinϕ0ey

− (sinθsuncosϕ0cosθLOS − cosθsunsinθLOS) ez (8.4)

We deﬁne the solar coordinate system such that the Zsun-axis is parallel to the sun direction esun and the Ysun-axis is parallel to the azimuthal unit vector eϕ1, i.e.

ez,sun = esun, ey,sun = eϕ1, ex,sun = ey,sun × ez,sun.

In view of (8.3) and (8.4) it is apparent that the Cartesian unit vectors ex,sun, ey,sun and ez,sun can be expressed in terms of the Cartesian unit vectors ex, ey and ez. The unit vector of the line of sight ΩLOS can be computed in the solar coordinate system as

ΩLOS = ex = nx,sunex,sun + ny,suney,sun + nz,sunez,sun (8.5) with the coordinates being given by

nx,sun = ex · ex,sun = −sinθsuncosθLOS + cosθsunsinθLOScosϕ0

ny,sun = ex · ey,sun = sinϕ0sinθLOS

nz,sun = ex · ez,sun = sinθsunsinθLOScosϕ0 + cosθsuncosθLOS

Solar direction A' Line of sight T

90−LOS

Y sun Y

e 1 A O e r e  sun X sun LOS LOS esun e1 X e

sun 0 Z sun

Top of the atmosphere

Sun

Figure 8.3.: The Earth coordinate system is such that the X-axis is parallel to the line of sight ΩLOS. The solar coordinate system is such that the Zsun-axis is parallel to the sun direction esun = −Ωsun and the Ysun-axis is parallel to the azimuthal unit vector eϕ1. The spherical coordinate systems (er, eθ, eϕ) and (er, eθ1, eϕ1) are rotated by ϕ0.

Let Mp be a point on the line of sight situated at the distance sp from the near point A. The Cartesian

SGP OL1b-2 ATBD Version 7 Page 62 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

coordinates of the near point A in the solar coordinate system are given by

xA,sun = −r1sinθsun, yA,sun = 0, zA,sun = r1cosθsun. (8.6)

In this context we use the representation

rA = xA,sunex,sun + yA,suney,sun + zA,sunez,sun, to express the position vector of the point Mp on the line of sight as

rMp = rA − spΩLOS. (8.7)

In view of (8.5)-(8.7) it is apparent that we can compute the Cartesian coordinates of the point Mp in the solar coordinate system: xM ,sun, yM ,sun and zM ,sun, while a standard transformation routine enable us to calculate p p p the polar coordinates rlev(p), Φsun,p, Ψsun,p from the Cartesian coordinates xMp,sun, yMp,sun, zMp,sun . The signiﬁcance of rlev(p) will be clariﬁed latter.

Z Top of the atmosphere Line of sight

s p T A  A' LOS

M p

Earth r p r =r T p r1

LOS X

O 2 2 2 Case : pp r r sin  p− 1 LOS

r1 cos LOS

Z Top of the atmosphere Line of sight

stage index : 2 p−2 p p−1 1 A' A point index : 2 p−1 2 p−2 ... p1 p p−1 ... 2 1 layer index : 1 p−1 p−1 1

Figure 8.4.: Limb viewing geometry. Top: The point Mp is situated at a distance sp from the point A. The depicted situation corresponds to the case p < p¯, where p¯ is the tangent level index. Bottom: The points on the line of sight are 1, ..., 2¯p − 1, the stages on the line of sight are 1, ..., 2¯p − 2 and the traversed limb layers are 1, ..., p¯ − 1.

SGP OL1b-2 ATBD Version 7 Page 63 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

In the radiative transfer calculation we are interested in the calculation of the spherical coordinates θLOS,p and ϕLOS,p of the line of sight vector ΩLOS in the spherical coordinate system attached to the point Mp. Using the representation (8.5), we have

ΩLOS ≡ nx,sunex,sun + ny,suney,sun + nz,sunez,sun

= mLOS,rp erp + mLOS,Φp eΦp + mLOS,Ψp eΨp , which gives

mLOS,rp = −sinΨsun,pnx,sun + cosΨsun,pny,sun

mLOS,Φp = cosΦsun,pcosΨsun,pnx,sun + cosΦsun,psinΨsun,pny,sun − sinΦsun,pnz,sun

mLOS,Ψp = sinΦsun,pcosΨsun,pnx,sun + sinΦsun,psinΨsun,pny,sun + cosΦsun,pnz,sun Finally we pass from the Cartesian coordinates mLOS,rp , mLOS,Φp , mLOS,Ψp to the spherical coordinates (1, θLOS,p, ϕLOS,p) by using a standard transformation routine. The cosine of the scattering angle between the line of sight and the sun direction is given by

cos Θsun = Ωsun · eLOS = −esun · eLOS = sinΦsun,pmLOS,Φp − cosΦsun,pmLOS,rp .

Obviously, cos Θsun should have the same values at all points on the line of sight and this property should be used as an internal check of the code. To define some parameters of calculation we first introduce the tangent level p¯ as that level for which it holds that rp¯ = rtg, where rtg = rearth + htg, with htg being the tangent altitude. In this context, we define point 1. number of points on the limb path, NLOS = 2¯p − 1; stage point 2. number of stages on the limb path, NLOS = NLOS − 1 = 2¯p − 2; level 3. number of levels above and including the limb path NLOS =p ¯. layer level 4. number of layers above the limb path, NLOS = NLOS − 1 =p ¯ − 1; 5. the limb paths computed in the earth coordinate system at all stages,

stage 4p = sp − sp+1, p = 1, ..., NLOS ,

where the distances from a generic point Mp on the limb path to the near point A are computed by

s1 = 0 , s2¯p−1 = 2r1 cos θLOS, q 2 2 2 sp = r1 cos θLOS − rp − r1 sin θLOS, p = 2, ..., p¯ − 1, q 2 2 2 s2¯p−p = r1 cos θLOS + rp − r1 sin θLOS, p = 2, ..., p¯ − 1,

sp¯ = r1 cos θLOS;

point 6. the limb zenith angles of the point Mp, θLOS,p, p = 1, ..., NLOS ; point 7. the limb azimuthal angles of the point Mp, ϕLOS,p, p = 1, ..., NLOS ; point 8. the solar zenith angles of the point Mp, Φsun,p, p = 1, ..., NLOS ;

9. the cosine of the solar scattering angle cos Θsun.

SGP OL1b-2 ATBD Version 7 Page 64 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Z sun esun sun , p e rp A'  M p A LOS e p sun , p e p r p sun Y O sun

sun , p X sun

erp sun , p

LOS , p LOS esun

LOS⊥

esun⊥ e p

M p

LOS , p e p sun

Figure 8.5.: Top: The spherical coordinates of the generic point Mp in the solar coordinate system rlev(p), Φsun,p, Ψsun,p . Note that the near point A is situated in the XsunZsun-plane. Bottom: Spherical angles θLOS,p and ϕLOS,p of ΩLOS in the spherical coordinate system attached to Mp.

Due to the peculiarities of the limb scattering geometry, two important maps can be deﬁned: point 1. the map lev (p) which map the point index p, p = 1, ..., NLOS , into the level index ( p, 1 ≤ p ≤ p¯ lev (p) = ; 2¯p − p, p¯ ≤ p ≤ 2¯p − 1

stage 2. the map lay (p) which map the stage index p, p = 1, ..., NLOS , into the layer index ( p, 1 ≤ p ≤ p¯ − 1 lay (p) = . 2¯p − p − 1, p¯ ≤ p ≤ 2¯p − 2

The map lev (p) will be used when computing the solar paths 4sun,p,s and the multiple scattering term (see below), while the map lay (p) will be used when deriving a recurrence relation for intensities and Jacobian.

SGP OL1b-2 ATBD Version 7 Page 65 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

When computing the single scattering term we need to evaluate the optical depth between a point Mp and the top of the atmosphere (along the sun direction) Z sun 0 0 τext,p = σext (r ) ds . |rp−rTOA|

The computational relation is trav Nsun,p sun X τext,p = σ¯ext,u4sun,p,u, u=1 trav where Nsun,p is the effective number of solar traversed layers at the point Mp and σ¯ext,u is extinction coefﬁcient on the layer u. We are now concern with the calculation of the solar paths 4sun,p,u at all points Mp on the line trav point of sight, that is, for all u = 1, ..., Nsun,p and p = 1, ..., NLOS . Considering the point Mp in the solar coordinate system we are faced with 2 situations which we have to consider:

1. Φsun,p < π/2. In this case no layers below Mp appears in the calculation of the optical depth and the number of solar traversed layer is trav Nsun,p = lev (p) − 1.

2. Φsun,p > π/2. In this case additional layers appear below Mp and we compute the index t as the ﬁrst index for which the inequality rlev(p)+t+1 < rlev(p) sin Φsun,p holds true. The number of of solar traversed layer is trav Nsun,p = lev (p) + t,

Z sun

1 1 2 ... lev−1 lev−1 lev r 2 2 2 r 2 r −r sin  1  1 lev sun , p M p

sun , p r lev

sun

Case :sun , p/2 O

Z sun

1 1 2 ... lev lev lev1 r lev ... lev t levt sun , p

O sun r levt1 levt levt1

rlev ... r levt1r lev sin sun , p lev2 t lev lev2 t1 Case :sun , p/2 M p

Figure 8.6.: Solar traversed layers in the cases Φsun,p < π/2 (top) and Φsun,p > π/2 (bottom).

SGP OL1b-2 ATBD Version 7 Page 66 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

8.2.3. Optical properties

We consider a discretization of the atmosphere in Nlev levels: r1 > r2 > ... > rNlev with the convention r1 = rTOA and rNlev = rs. A layer j is bounded at above by the level rj and below by the level rj+1; the number of layers is given by Nlay = Nlev − 1. The gas law for air molecules states that p = nkT, or equivalently that R p = n T, NA

with k = R/NA. The hydrostatic equilibrium law for air molecules is gpM dp = − dr, RT whence accounting of the gas law we see that

M dp = −g n (r) dr. NA In terms of ﬁnite variations, the hydrostatic equilibrium law for air molecules takes the form

N n (r) 4r = A 4p gM and further molec n (r) 4r = 2.120156 · 1022 · 4p [mb] cm2

8.2.3.1. Partial columns

The partial column of the gas g on the layer j is given by

Z rj Z rj Z rj kT0 kT0 T0 p (r) X¯g,j = ng (r) dr = VMRg (r) n (r) dr = VMRg (r) dr, (8.8) p0 rj+1 p0 rj+1 p0 rj+1 T (r)

where the gas law p = nkT and the expression relating the number density of the gas g to the air number density n, ng = VMRg · n, have been employed. The computational relation then takes the form −3 7 VMRg[.]p [mb] X¯g,j 10 cm = 2.69578 · 10 (r) 4rj, T [K] j

with −6 VMRg [.] = 10 · VMRg [ppm] . Another relation can be derived if we assume that the hydrostatic equilibrium law holds true. In this case we obtain

Z rj Z rj kT0 kT0 X¯g,j = VMRg (r) n (r) dr = VMRg,j n (r) dr p0 rj+1 p0 rj+1

kT0 kT0NA = VMRg,jn¯j4rj = VMRg,j4pj p0 p0gM

SGP OL1b-2 ATBD Version 7 Page 67 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

and the computational relation read as

−3 6 X¯g,j 10 cm = 0.789087 · 10 · VMRg,j [.] 4pj [mb] . (8.9)

8.2.3.2. Number density and VMR

Because the retrieved quantities are the partial columns it is important to relate the partial columns to the number density and VMR. From the relation (cf.(8.8))

Z rj kT0 kT0 X¯g,j = ng (r) dr = n¯g,j4rj, p0 rj+1 p0 we get molec X¯ 10−3cm · 11 g,j n¯g,j 3 = 2.6868 10 . cm 4rj [km] To compute the VMR we use again (8.8) which yields

Z rj T0 p (r) T0 p X¯g,j = VMRg (r) dr = VMRg,j (r) 4rj p0 rj+1 T (r) p0 T j and further −3 X¯g,j 10 cm VMRg,j [.] = . 7 p[mb] 2.69578 · 10 · T K (r) 4rj [km] [ ] j When the hydrostatic equilibrium law is assumed to holds true, the computational relation is (8.9) and the result is, −3 X¯g,j 10 cm VMRg,j [.] = 6 . 0.789087 · 10 · 4pj [mb]

8.2.3.3. Scattering optical depth of air molecules

The scattering optical depth of air molecules on the layer j is deﬁned by

Z rj Z 4rj mol p (r) τ¯scat,j (λ) = Cscat (λ) n (r) dr = Cscat (λ) dr rj+1 0 kT (r)

and the computational expression is mol 24 2 p [mb] τ¯scat,j (λ) = 0.724311 · 10 · Cscat (λ) cm (r) 4rj [km] T [K] j

Assuming that the the hydrostatic equilibrium law holds true, we obtain

Z rj mol NA τ¯scat,j (λ) = Cscat (λ) n (r) dr = Cscat (λ)n ¯j4rj = Cscat (λ) 4pj rj+1 gM

and further mol 22 2 τ¯scat,j (λ) = 2.120156 · 10 · Cscat (λ) cm 4pj [mb] For the scattering coefﬁcient we get mol mol τ¯scat,j (λ) σ¯scat,j (λ) = 4rj

SGP OL1b-2 ATBD Version 7 Page 68 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

The Rayleigh scattering coefﬁcient Cscat (λ) is computed by using the relation

−4 4 2 3.99 · 10 (1/λ) −24 Cscat (λ) cm = · 10 1 − 1.06 · 10−2 (1/λ)2 − 6.68 · 10−5 (1/λ)4 with λ [µm].

8.2.3.4. Absorption optical depth of gas molecules

The absorption optical depth of gas molecules on the layer j is a sum over all constituents

mol X mol τ¯abs,j (λ) = τ¯abs,g,j (λ) , g where the optical depth of the gas component g is given by

Z rj mol ¯ ¯ p0 ¯ τ¯abs,g,j (λ) = Cabs,g,j (λ) ng (r) dr = Cabs,g,j (λ) Xg,j rj+1 kT0 or equivalently, by mol 16 ¯ 2 ¯ −3 τ¯abs,g,j (λ) = 2.6868 · 10 · Cabs,g,j (λ) cm Xg,j 10 cm . The absorption coefﬁcient is then computed accordingly to the relation

mol mol τ¯abs,j (λ) σ¯abs,j (λ) = . 4rj

8.2.3.5. Scattering optical depth of aerosols

The scattering optical depth of aerosols on the layer j depends on the aerosol optical thickness taer,j and the wavelength λ, aer aer aer τ¯scat,j (λ) = taer,j a¯3,j + (λ − λref)a ¯4,j , For the extinction optical depth we have a similar relation

aer aer aer τ¯ext,j (λ) = taer,j a¯1,j + (λ − λref)a ¯2,j . The scattering and extinction coefﬁcients are then given by

aer aer τ¯scat,j (λ) σ¯scat,j (λ) = 4rj and aer aer τ¯ext,j (λ) σ¯ext,j (λ) = , 4rj respectively.

8.2.3.6. Total extinction optical depth

Since

extinction = scattering by air molecules + absorption by gas molecules + extinction by aerosols

SGP OL1b-2 ATBD Version 7 Page 69 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

the total extinction optical depth on the layer j is computed as

mol mol aer τ¯ext,j (λ) = τ¯scat,j (λ) +τ ¯abs,j (λ) +τ ¯ext,j (λ) , while the extinction coefﬁcient is given by

τ¯ext,j (λ) σ¯ext,j (λ) = . 4rj

Because only the absorption of gas molecules depends on partial columns, the partial derivative of the extinction and absorption coefﬁcients with respect to the partial columns of the gas g are given by

mol ∂σ¯ext,j ∂σ¯abs,j ∂σ¯abs,j 16 1 (λ) = (λ) = (λ) = 2.6868 · 10 · C¯abs,g,j (λ) ∂X¯g,j ∂X¯g,j ∂X¯g,j 4rj

while the partial derivative of the extinction and scattering coefﬁcients with respect to the aerosol optical thickness read as ∂σ¯ext,j 1 aer aer (λ) = a¯1,j + (λ − λref)a ¯2,j ∂taer,j 4rj and ∂σ¯scat,j 1 aer aer (λ) = a¯3,j + (λ − λref)a ¯4,j ∂taer,j 4rj

8.2.3.7. Phase functions

For a scattering atmosphere, the effective scattering phase function accounts of Rayleigh scattering by air molecules (molecular scattering) P and Mie scattering by aerosols (particle scattering),

0 0 S (r, Ω, Ω , λ) = σscat (r, λ) P (r, Ω, Ω , λ) mol 0 aer 0 = σscat (r, λ) PRay (Ω, Ω , λ) + σscat (r, λ) PMie (r, Ω, Ω , λ) .

If cos Θ = Ω · Ω0 is the cosine of the angle Θ between the incident and scattering directions, we have the simpliﬁed representation

S (r, cos Θ, λ) = σscat (r, λ) P (r, cos Θ, λ) mol aer = σscat (r, λ) PRay (cos Θ, λ) + σscat (r, λ) PMie (r, cos Θ, λ) .

In practical applications it is important to compute

PRay (cos Θsun, λ)

and P¯Mie,p (cos Θsun, λ) stage for all stages p = 1, ..., NLOS . The Rayleigh phase function possesses the representation

2 PRay (cos Θ, λ) = A (λ) + B (λ) cos Θ with 3 + 3ρ (λ) 3 − 3ρ (λ) A (λ) = ,B (λ) = . 4 + 2ρ (λ) 4 + 2ρ (λ) The depolarization ratio is computed as 6F (λ) − 6 ρ (λ) = K , 7FK (λ) + 3

SGP OL1b-2 ATBD Version 7 Page 70 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18 while the King factor is given by

2 4 −4 1 −5 1 FK (λ) = 1.04695 + 3.25031 · 10 + 3.86228 · 10 λ λ with λ expressed in µm. The Mie scattering phase function takes the form

1 − g¯2 (λ) ¯ lay(p) PMie,lay(p) (cos Θ, λ) = 3/2 , 2 1 +g ¯lay(p) (λ) − 2¯glay(p) (λ) cos Θ where the asymmetry factor g¯lay(p) on the layer lay (p) is computed as

aer aer g¯lay(p) (λ) =a ¯4,lay(p) + (λ − λref)a ¯5,lay(p).

8.2.4. Recurrence relation for limb radiance and Jacobian

We are now well prepared to transform equation (8.1) into a computational expression. For two points on the line of sight Mp and Mp+1 characterized by the position vectors rp and rp+1 and being the boundaries of the layer lay (p), the integral form of the radiative transfer equation takes the form

−σ¯ext,lay(p)4p Ip (ΩLOS) = Ip+1 (ΩLOS) e

Z 0 0 −σ¯ext,lay(p)|rp−r | 0 + J (r , ΩLOS) e ds , |rp−rp+1| with Ip (ΩLOS) = I (rp, ΩLOS) and 4p = |rp − rp+1|. Note that the position vector of the point Mp is rp, while the radial distance is r (Mp), that is, in the solar coordinate system we have the representation rp = point (r (Mp) , Φsun,p, Ψsun,p). The downward recurrence relation starts at the point index NLOS , where for limb viewing geometries, I point (ΩLOS) = 0. NLOS

p s' level index : lev  p1 lev  p point index : p1 M p p1 M p stage index : p p=1,... ,2 p−2 layer index : lay  p r' rp1 rp , rp=r lev p ,sun p ,sun p

Figure 8.7.: Stage index p, point indices p and p + 1, layer index lay (p) and the level indices lev (p) and lev (p + 1).

SGP OL1b-2 ATBD Version 7 Page 71 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

8.2.4.1. Recurrence relation for limb radiance

In this Section we are concerned with the single and multiple scattering term integrations.

Single scattering term integration stage On the layer lay (p), where p is the stage index ranging downward from NLOS to 1, the single scattering term can be approximated by

0 1 −τ sun r0−r r0 J (r , Ω ) = S (r, Ω , Ω ) F e ext (| TOA( )|) +σ ¯ B¯ , ss LOS 4π LOS sun p sun abs,lay(p) lay(p) where 1 B¯ = [B (Tp) + B (Tp )] , lay(p) 2 +1

with Tp and Tp+1 being the intensities at the points Mp and Mp+1, respectively. We deﬁne the solar optical depths at the boundary points Mp and Mp+1 by

N trav Z sun,p sun 0 0 X τext,p = σext (r ) ds = σ¯ext,q4sun,p,q, |rp−rTOA,p| q=1 N trav Z sun,p+1 sun 0 0 X τext,p+1 = σext (r ) ds = σ¯ext,q4sun,p+1,q, |rp+1−rTOA,p+1| q=1

trav where the 4sun,p,q are the solar optical depths at the point Mp for all q = 1, ..., Nsun,p and the 4sun,p+1,q are the trav solar optical depths at the point Mp+1 for all q = 1, ..., Nsun,p+1. We then use the linear approximations sun 0 0 s sun s sun τext (|r − rTOA (r )|) = τext,p + 1 − τext,p+1 4p 4p to obtain

Z 0 0 −σ¯ext,lay(p)|rp−r | 0 Jss (r , ΩLOS) e ds |rp−rp+1| Z 4p h i F − s τ sun + 1− s τ sun sun 4p ext,p 4p ext,p+1 −σ¯ext,lay(p)(4p−s) = S (r, ΩLOS, Ωsun)p e e ds 4π 0 Z 4p ¯ −σ¯ext,lay(p)(4p−s) +¯σabs,lay(p)Blay(p) e ds. 0 The result of integration is

Z 0 0 −σ¯ext,lay(p)|rp−r | 0 Iss (rp, rp+1, ΩLOS) = Jss (r , ΩLOS) e ds |rp−rp+1| Fsun = S (r, Ω , Ω ) 4pb (τp) + a (τp) 4pσ¯ B¯ 4π LOS sun p 0 abs,lay(p) lay(p) where

Z 1 −x −x(1−ξ) 1 − e a0 (x) = e dξ = 0 x 1 − τ sun +x −τ sun Z sun sun e ( ext,p+1 ) − e ext,p −[ξτext,p+(1−ξ)τext,p+1] −x(1−ξ) b (x) = e e dξ = sun sun 0 τext,p − τext,p+1 − x and τp =σ ¯ext,lay(p)4p.

SGP OL1b-2 ATBD Version 7 Page 72 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

The term S (r, ΩLOS, Ωsun)p is computed by using the relation

mol aer ¯ S (r, Ω, Ωsun)p =σ ¯scat,lay(p)PRay cos Θsun,p + σscat,lay(p)PMie,lay(p) cos Θsun,p ¯ where the stage values PRay cos Θsun,p and PMie,lay(p) cos Θsun,p are calculated by the optical module for all stage p = 1, ..., NLOS .

Multiple scattering term integration stage On the layer lay (p), where p is the stage index ranging downward from NLOS to 1, the multiple scattering term read as

Z 0 1 0 0 0 0 Jms (r , ΩLOS) = S¯p (ΩLOS, Ω ) I (r , Ω ) dΩ 4π 4π ¯ 0 and note that the calculation of S (r, ΩLOS, Ωsun)p and Sp (ΩLOS, Ω ) is different. Using the linear approximation 0 0 s 0 s 0 I (r , Ω ) = Ip (Ω ) + 1 − Ip+1 (Ω ) , 4p 4p

where Ip and Ip+1 are the intensities at the points Mp and Mp+1, respectively, we obtain

Z 0 0 −σ¯ext,lay(p)|rp−r | 0 Jms (r , ΩLOS) e ds |rp−rp+1| " # Z 4p s −σ¯ext,lay(p)(4p−s) ¯1 = e ds Jms,p 0 4p " # Z 4p s −σ¯ext,lay(p)(4p−s) ¯2 + 1 − e ds Jms,p 0 4p

with Z 1 0 0 0 J¯ms,p = S¯p (ΩLOS, Ω ) Ip (Ω ) dΩ 4π 4π Z 1 0 0 0 J¯ms,p+1 = S¯p (ΩLOS, Ω ) Ip+1 (Ω ) dΩ 4π 4π

To compute J¯ms,p we use the relation Z 1 0 0 0 J¯ms,p = S¯p (ΩLOS, Ω ) Ip (Ω ) dΩ 4π 4π 2M−1 Z 1 1 X 0 0 0 = sm σ¯ , µ ,p, µ Im (r (Mp) , µ ) dµ cos mϕ ,p 2 scat,lay(p) LOS LOS m=0 −1 2M−1 " M 1 X X + + + = w sm σ¯ , µ ,p, µ Im r (Mp) , µ 2 k scat,lay(p) LOS k k m=0 k=1 M # X − − − + wk sm σ¯scat,lay(p), µLOS,p, µk Im r (Mp) , µk cos mϕLOS,p, k=1 where θLOS,p and ϕLOS,p are the zenith and azimuthal angles of the point Mp, µLOS,p = cos θLOS,p and

2M−1 0 X m m 0 sm σ¯scat,lay(p), µLOS,p, µ = ξn σ¯scat,lay(p) Pn (µLOS,p) Pn (µ ) , n=m mol mol aer aer ξn σ¯scat,lay(p) =σ ¯scat,lay(p)χn +σ ¯scat,lay(p)χ¯n,lay(p).

SGP OL1b-2 ATBD Version 7 Page 73 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

The Im (r (Mp) , θ) are the azimuthal components of the intensity at the the point Mp

2M−1 X Ip (Ω) = I (r (Mp) , θ, ϕ) = Im (r (Mp) , θ) cos mϕ m=0 ± ¯ and are computed at a set of discrete angles θk , k = 1, ..., M by the multiple scattering module. For Jms,p+1 we have a similar relation Z 1 0 0 0 J¯ms,p+1 = S¯p (ΩLOS, Ω ) Ip+1 (Ω ) dΩ 4π 4π 2M−1 " M 1 X X + + + w sm σ¯ , µ ,p , µ Im r (Mp ) µ 2 k scat,lay(p) LOS +1 k +1 k m=0 k=1 M # X − − − + wk sm σ¯scat,lay(p), µLOS,p+1, µk Im r (Mp+1) , µk cos mϕLOS,p+1, k=1 where Im (r (Mp+1) , θ) are the azimuthal components of the intensity at the the point Mp+1 and

2M−1 0 X m m 0 sm σ¯scat,lay(p), µLOS,p+1, µ = ξn σ¯scat,lay(p) Pn (µLOS,p+1) Pn (µ ) . n=m

Performing the integrals we arrive at

Z 0 0 −σ¯ext,lay(p)|rp−r | 0 Ims (rp, rp+1, ΩLOS) = Jms (r , ΩLOS) e ds |rp−rp+1|

= a2 (τp) 4pJ¯ms,p + a1 (τp) 4pJ¯ms,p+1 with

Z 1 1 − e−x(1−ξ) − e−x a1(x) = (1 ξ) dξ = 2 1 (1 + x) 0 x Z 1 1 e−x(1−ξ) − e−x a2(x) = ξ dξ = 2 x 1 + . 0 x

Collecting all results we are led to the recurrence relation

−τp Fsun Ip (Ω ) = Ip (Ω ) e + S (r, Ω , Ω ) 4pb (τp) LOS +1 LOS 4π LOS sun p ¯ ¯ ¯ +a0 (τp) 4pσ¯abs,lay(p)Blay(p) + a2 (τp) 4pJms,p + a1 (τp) 4pJms,p+1

8.2.4.2. Recurrence relation for the Jacobian matrix

The derivative of the radiance at the top of the atmosphere Ip (ΩLOS) with respect to the partial columns X¯g,i, with i = 1, ..., Nd and g = 1, ..., Ngas ∂Ip (ΩLOS) , ∂X¯g,i can be computed in terms of the derivatives with respect to the extinction coefﬁcient

∂Ip (ΩLOS) ∂σ¯ext,i by using the relation

SGP OL1b-2 ATBD Version 7 Page 74 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

∂Im ∂Im ∂σ¯ext,i (ΩLOS) = (ΩLOS) ∂X¯g,i ∂σ¯ext,i ∂X¯g,i with mol ∂σ¯ext,i ∂σ¯abs,i 16 1 = · 2.6868 · 10 · C¯abs,g,i. ∂X¯g,i ∂X¯g,j 4ri

For the partial derivatives with respect to the extinction coefﬁcient we have the recurrence relation

∂Ip (ΩLOS) ∂σ¯ext,i

∂ −τp −τp ∂Ip+1 = Ip+1 (ΩLOS) e + e (ΩLOS) ∂σ¯ext,i ∂σ¯ext,i

Fsun ∂ n o Fsun ∂b (τp) + 4pb (τp) S (r, ΩLOS, Ωsun)p + S (r, ΩLOS, Ωsun)p4p 4π ∂σ¯ext,i 4π ∂σ¯ext,i ¯ ∂ ¯ ∂σ¯abs,lay(p) +4pσ¯abs,lay(p)Blay(p) {a0 (τp)} + a0 (τp) 4pBlay(p) ∂σ¯ext,i ∂σ¯ext,i

∂J¯ms,p ∂ +a2 (τp) 4p + 4pJ¯ms,p {a2 (τp)} ∂σ¯ext,i ∂σ¯ext,i

∂J¯ms,p+1 ∂ +a1 (τp) 4p + 4pJ¯ms,p+1 {a1 (τp)} ∂σ¯ext,i ∂σ¯ext,i

For our calculation it is important to observe that the extinction coefﬁcient σ¯ext,i which is computed as

mol mol aer σ¯ext,i = σ¯scat,i +σ ¯abs,i +σ ¯ext,i

aer (note that σ¯scat,i does not appear explicitly) enters: 1. explicitly,

2. in the expression of τsun and

3. in the expression of σ¯abs,lay(p) multiplying the Planck function B,

σ¯abs,lay(p) =σ ¯ext,lay(p) − σ¯scat,lay(p),

but does not enter in the expression of S (or sm). The partial derivative terms are computed as follows: 1. The partial derivative of e−τp is given by

∂σ¯ ∂ −τp −τp ext,lay(p) −τp e = −4pe = −4pe δlay(p),i ∂σ¯ext,i ∂σ¯ext,i

2. Since S does not depend on σ¯ext,i , it is apparent that

∂ n o S (r, ΩLOS, Ωsun)p = 0. ∂σ¯ext,i

3. To compute the term ∂b (τ ) p , ∂σ¯ext,i

SGP OL1b-2 ATBD Version 7 Page 75 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

we ﬁrst compute

N trav  sun sun,p ∂τext,p ∂  X  = σ¯ext,q4sun,p,q ∂σ¯ext,i ∂σ¯ext,i  q=1   4 , i ≤ N trav  sun,p,i sun,p =  trav 0, i > Nsun,p

and

N trav  sun sun,p+1 ∂τext,p+1 ∂  X  = σ¯ext,q4sun,p+1,q ∂σ¯ext,i ∂σ¯ext,i  q=1   4 , i ≤ N trav  sun,p+1,i sun,p+1 = .  trav 0, i > Nsun,p+1

Then we have sun ∂b (τp) ∂τext,p+1 = −b (τp) ∂σ¯ext,i ∂σ¯ext,i sun sun 1 ∂τext,p ∂τext,p+1 + sun sun − τext,p − τext,p+1 − τp ∂σ¯ext,i ∂σ¯ext,i −4pδlay(p),i 1 sun ∂ −τext,p+1 −τp + sun sun e e τext,p − τext,p+1 − τp ∂σ¯ext,i sun sun −τ sun ∂τext,p ∂τext,p+1 +e ext,p − ∂σ¯ext,i ∂σ¯ext,i

4. The partial derivative of a0 (τp) takes the form

∂ 0 {a0 (τp)} = a0 (τp) 4pδlay(p),i, ∂σ¯ext,i

where e−xx − (1 − e−x) xe−x + e−x − 1 a0 (x) = = . 0 x2 x2

Similarly, the partial derivatives of a2 (τp) and a1 (τp) are given by

∂ 0 {a2 (τp)} = a2 (τp) 4pδlay(p),i ∂σ¯ext,i

with (1 − e−x) x2 − 2x (x − 1 + e−x) a0 (x) = 2 x4 x2 − x2e−x − 2x2 + 2x − 2xe−x = x4 −x2e−x − x2 + 2x − 2xe−x = x4 (x − 2) + e−x (x + 2) = − , x3

SGP OL1b-2 ATBD Version 7 Page 76 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

and ∂ 0 {a1 (τp)} = a1 (τp) 4pδlay(p),i ∂σ¯ext,i with 0 0 0 a1(x) = a0 (x) − a2(x).

5. The partial derivative of the absorption coefﬁcient is identical to the partial derivative of the extinction coefﬁcient ∂σ¯abs,lay(p) ∂σ¯ext,lay(p) = = δlay(p),i. ∂σ¯ext,i ∂σ¯ext,i

6. To compute the partial derivatives of the multiple scattering terms J¯ms,p and J¯ms,p+1 we take into account that the sm does not depend on σ¯ext,i and derive

∂J¯ms,p ∂σ¯ext,i 2M−1 " M 1 X X + + ∂Im + = wk sm σ¯scat,lay(p), µLOS,p, µk r (Mp) , µk 2 ∂σ¯ext,i m=0 k=1 M # X − − ∂Im − + wk sm σ¯scat,lay(p), µLOS,p, µk r (Mp) , µk cos mϕLOS,p ∂σ¯ext,i k=1 and

∂J¯ms,p+1 ∂σ¯ext,i 2M−1 " M 1 X X + + ∂Im + = wk sm σ¯scat,lay(p), µLOS,p+1, µk r (Mp+1) , µk 2 ∂σ¯ext,i m=0 k=1 M # X − − ∂Im − + wk sm σ¯scat,lay(p), µLOS,p+1, µk r (Mp+1) , µk cos mϕLOS,p+1 ∂σ¯ext,i k=1

8.2.4.3. Input parameters for limb radiance and Jacobian calculation

The input parameters for radiance and Jacobian calculation are listed below: mol aer 1. the extinction coefficient σ¯ext,j (λ), the molecular and aerosol scattering coefficients σ¯scat,j (λ) and σ¯scat,j (λ), mol and the molecular absorption coefficient σ¯abs,j (λ) on all layers, that is, for all j = 1, ..., Nlay; 2. the partial derivative of the extinction coefficient

∂σ¯ ext,j , ∂X¯g,j

for all gases and all on all layers, that is, for all g = 1, ..., Ngas and j = 1, ..., Nlay; 3. the Rayleigh and Mie phase functions PRay cos Θsun,p and P¯Mie,p cos Θsun,p on all stages on the line of stage sight, that is, for all p = 1, ..., NLOS ; stage 4. the limb paths 4p of all stages on the line of sight, that is, for all p = 1, ..., NLOS ; trav 5. the number of solar traversed layers Nsun,p at all points Mp on the line of sight, that is, for all p = point 1, ..., NLOS ; trav 6. the solar optical depths 4sun,p,q at all points Mp on the line of sight, that is, for all q = 1, ..., Nsun,p and point p = 1, ..., NLOS ;

SGP OL1b-2 ATBD Version 7 Page 77 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

7. the multiple scattering terms J¯ms,p and J¯ms,p+1 on all stages on the line of sight, that is for all p = stage 1, ..., NLOS ; 8. the partial derivatives of the multiple scattering terms

∂J¯ ∂J¯ ms,p and ms,p+1 , ∂σ¯ext,i ∂σ¯ext,i

stage for all stages on the line of sight and all layers, that is, for all p = 1, ..., NLOS and i = 1, ..., Nlay.

8.2.5. Pseudo-spherical discrete ordinate radiative transfer equation

The discrete ordinate method converts the radiative transfer equation into a linear system of differential equations by discretizing the angular variation of the phase function and radiance. In the conventional eigenvalue approach, the general solution of the linear system of differential equations consists of a linear combination of all the homogeneous solutions plus the particular solutions for the assumed sources. For a multi-layered medium, the expansion coefﬁcients of the homogeneous solutions are the unknowns of the discretized radiative transfer problem and are computed by imposing the continuity condition for the radiances across the layer interfaces. In the matrix-exponential formalism, the linear system of differential equations is treated as a boundary value problem. For each layer this classical mathematical procedure yields a so called layer equation which relates the level values of the radiance. The discretized radiative transfer problem then reduces to a system of linear algebraic equations for the unknown level values of the radiance. The purpose of this Section is to present a stable discrete ordinate algorithm for vertically inhomogeneous layered media using the matrix-exponential solution. Conceptually, the algorithm is similar to a ﬁnite-element algorithm and involves the following steps: discretization of the atmosphere into a number of distinct but vertically uniform layers, derivation of the layer equation using the matrix-exponential solution, assemblage of the layer equation into the system matrix of the entire atmosphere, solution of the assembled system of equations.

8.2.5.1. Requirement for a pseudo-spherical model

∂J¯ms,p The calculation of the multiple scattering terms J¯ms,p and J¯ms,p and of their partial derivatives and +1 ∂σ¯ext,i ¯ ∂Jms,p+1 on the stage index p, with p = 1, ..., N stage require the knowledge of the azimuthal components of the ∂σ¯ext,i LOS intensities and their derivatives at the boundary points Mp and Mp+1, that is,

± ∂Im ± Im r (Mp) , θk and r (Mp) , µk ∂σ¯ext,i

and ± ∂Im ± Im r (Mp+1) , θk and r (Mp+1) , µk ∂σ¯ext,i

for all k = 1, ..., M, m = 0, ..., 2M − 1 and i = 1, ..., Nd. The above requirements are equivalent with the knowledge of ± ∂Im ± Im r (Mp) , θk and r (Mp) , µk ∂σ¯ext,i

at all points Mp on the line of sight, that is for all k = 1, ..., M, m = 0, ..., 2M − 1, i = 1, ..., Nd and p = point 1, ..., NLOS . Because the point Mp is situated on a radial direction which encloses the angle Φsun,p with the Zsun-axis and assuming that for a solar zenith angle

θsun = π − Φsun,p,

SGP OL1b-2 ATBD Version 7 Page 78 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

we are able to compute the intensities

± ∂Im ± Im rj, θk and rj, µk ∂σ¯ext,i for all k = 1, ..., M, m = 0, ..., 2M − 1, i = 1, ..., Nd and j = 1, ..., N by a pseudo-spherical model. To achieve that we may use a simple interpolation procedure and derive ± s ± s ± Im r (Mp) , θk = Im r¯j, θk + 1 − Im r¯j+1, θk , 4rj 4rj ∂Im ± s ∂Im ± s ∂Im ± r (Mp) , µk = r¯j, θk + 1 − r¯j+1, θk ∂σ¯ext,i 4rj ∂σ¯ext,i 4rj ∂σ¯ext,i ¯ where the index j is such that r¯j+1 ≤ rlev(p) ≤ r¯j, s = rlev(p) − r¯j+1 and 4rj = r¯j − r¯j+1. It is important to note ± ∂Im ± that Im rj, θ is a 3-dimensional array, while rj, µ is a 4-dimensional array. k ∂σ¯ext,i k

8.2.5.2. General considerations

In a pseudo-spherical atmosphere the boundary-value problem for the diffuse radiance consists of the inhomogeneous differential equation

dI µ (r, Ω) = −σ (r) I (r, Ω) + J (r, Ω) + J (r, Ω) , (8.10) dr ext ss ms the top-of-atmosphere boundary condition (r = rTOA)

− I rTOA, Ω = 0 (8.11) and the surface boundary condition (r = rs)

+ + A + −τ sun(|r −r |) I r , Ω = Ω B (r ) + F |µ | ρ Ω , Ω e ext s TOA (8.12) s s sun π sun norm sun Z A − − + − − + I rs, Ω µ ρnorm Ω , Ω dΩ π 2π

Assuming the azimuthal expansion of the diffuse radiance Im (ϕsun = 0)

2M−1 X I (r, Ω) = I (r, µ, ϕ) = Im (r, µ) cos mϕ, m=0 the radiative transfer equation can be expressed in the discrete ordinate space as:

dI µ± m r, µ± k dr k ± = −σext(r)Im r, µk + δm0σabs(r)B (r)

sun Fsun ± −τ (|r−rTOA|) + (2 − δm ) σ (r) pm r, µ , µ e ext (8.13) 0 4π scat k sun M 1 X ± + + ± − − + wlσ (r) pm r, µ , µ Im r, µ + pm r, µ , µ Im r, µ , 2 scat k l l k l l l=1

where pm are the azimuthal expansion coefﬁcients of the scattering phase function

2M−1 0 X 0 0 P (r, Ω, Ω ) = (2 − δm0) pm (r, µ, µ ) cos [m (ϕ − ϕ )] . m=0

SGP OL1b-2 ATBD Version 7 Page 79 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

In equation (8.13) µ is the cosine of the zenith angle; our convention is that µ = 1 for upward radiation and − for downward radiation. The set + is the set of Gauss-Legendre quadrature points and µ = 1 µk , wk k=1,M weights in the interval , while − with − − + is the set of quadrature points and weights (0, 1) µk , wk k=1,M µk = µk in the interval (−1, 0). Deﬁning the radiance vector function in the discrete ordinate space by " # i+ (r) i m m (r) = − im (r)

± ± with [im (r)]k = Im r, µk , k = 1, ..., M we are led to the linear system of differential equations

dim (r) = Am (r) im (r) + bm (r) . (8.14) dr The entries of the layer matrix " # A11 (r) A12 (r) A (r) = m m m 21 22 Am (r) Am (r) are given by

11 1 + + Am (r) kl = + wlσscat (r) pm r, µk , µl − 2σext(r)δkl , 2µk 12 1 + − Am (r) kl = + wlσscat (r) pm r, µk , µl , 2µk 21 1 − + Am (r) kl = − wlσscat (r) pm r, µk , µl , 2µk 22 1 − − Am (r) kl = − wlσscat (r) pm r, µk , µl − 2σext(r)δkl 2µk for all k, l = 1, ..., M. The source vector is decomposed into a solar and a thermal contribution

sun −τ (|r−rTOA|) bm (r) = bsun,m (r) e ext + bth,m (r) ,

where 1 F b± − sun ± sun,m (r) k = ± (2 δm0) σscat (r) pm r, µk , µsun µk 4π and 1 b± th,m (r) k = ± δm0σabs(r)B (r) µk for all k = 1, ..., M.

8.2.5.3. Integral form of the layer equation

Let us consider a discretization of the atmosphere in N levels: r1 > r2 > ... > rN with the convention r1 = rTOA and rN = rs, where rs is the lowest (surface) point of the atmosphere. A layer j is bounded above by the level rj and below by the level rj+1; the number of layers is N − 1. The optical coefﬁcients and the phase function are assumed to be constant within each layer and for the layer j with geometrical thickness 4rj = rj − rj+1 we denote by A¯ m,j the average value of Am (r). In this regard equation (8.14) reduces to a linear system of differential equations with constant coefﬁcients

dim (ρ) = A¯ m,jim (ρ) + bm (ρ) , (8.15) dr

SGP OL1b-2 ATBD Version 7 Page 80 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

where the layer coordinate ρ is deﬁned by ρ = r − rj+1 and 0 ≤ ρ ≤ 4rj. Solving equation (8.15) with the boundary condition im (0) = im,j+1 yields Z 4rj A¯ m,j 4rj A¯ m,j (4rj −ρ) im,j = e im,j+1 + e bm (ρ) dρ (8.16) 0 with im,j and im,j+1 being the radiances at the boundary levels j and j + 1, respectively. Using the property of the matrix exponential: eaAebA = e(a+b)A, which holds true for two arbitrary scalars a and b, we left multiply ¯ the above equation by the matrix e−Am,j 4rj and obtain the integral form of the layer equation

Z 4rj −A¯ m,j 4rj −A¯ m,j ρ im,j+1 = e im,j − e bm (ρ) dρ. (8.17) 0

8.2.5.4. Layer equation

In principle, the exponential of a matrix can be computed by methods involving the matrix eigenvalues, approximation theory, differential equations and the matrix characteristic polynomial. In the present analysis we concentrate on the matrix eigenvalue method and the Padé approximation. Furthermore, the computation of the integral of the matrix exponential in equation (8.17) requires a parametrization of the source term within each layer. In this regard, the thermal layer vector bth,m (ρ) is linearly interpolated between the level values b¯th,m,j and b¯th,m,j+1, ρ ρ bth,m (ρ) = b¯th,m,j + 1 − b¯th,m,j+1, 4rj 4rj although a higher-order polynomial approximation can also be considered. The solar layer vector is computed accordingly to the average secant approximation, that is,

sun sun −τ (|r−rTOA|) −τ (ρ) bsun,m (ρ) e ext = b¯sun,m,je ext

and sun ρ sun ρ sun τext (ρ) = τext,j + 1 − τext,j+1, 4rj 4rj ¯ sun sun where bsun,m,j is a layer quantity, τext,j and τext,j+1 are the solar optical depths at the boundary levels j and j + 1. For most practical applications encountered in atmospheric remote sensing, the accuracy of the average secant approximation is reasonable; for extreme situations involving optically thick layers and high solar zenith angles, improved exponential-polynomial parametrization can also be employed.

8.2.5.5. Matrix eigenvalue method

In the eigenvalue solution method the matrix exponential and the integral terms involving matrix exponentials are computed by using the spectral decomposition of the matrix A¯ m,j. This method is especially efﬁcient due to the special structure of the matrix A¯ m,j, written for convenience as " # A11 A12 A = . −A12 −A11

The eigenvalues of A are real and occur in pairs, in which case the order of the algebraic eigenvalue problem can be reduced by a factor of 2. The steps of computing the eigenstate of the matrix A can be summarized as follows: 1. Compute A+ = A−A+, where A+ = A11 + A12 and A− = A11 − A12 , and determine the eigenstate w+ of the matrix A+. µk, k k=1,M + 2. Normalize the vectors wk for k = 1, ..., M.

SGP OL1b-2 ATBD Version 7 Page 81 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

3. Compute the eigenvectors of the matrix A− = A+A−

− 1 + + wk = √ A wk , k = 1, ..., M. (8.18) µk

4. Set 1 1 v+ = w+ + w− , v− = w+ − w− , k = 1, ..., M. k 2 k k k 2 k k 5. Construct the eigenvectors of A as " # " # v+ v− v+ k v− k ¯k = − , ¯k = + , k = 1, ..., M. vk vk

+ + + It should be remarked that if Av¯k = λk v¯k with

+ √ λk = µk, k = 1, ..., M, that is, " # " # A11v+ + A12v− v+ k k = λ+ k , 12 + 11 − k − −A vk − A vk vk then there holds " # " # A11v− + A12v+ v− Av¯− = k k = −λ+ k = −λ+v¯−. k 12 − 11 + k + k k −A vk − A vk vk + Also note that the normalization condition imposed on wk is required for an explicit analytic determination of the partial derivatives of the radiance ﬁeld. The spectral decomposition of the matrix A is then

A = VΛV−1 with + + − − V = ¯v1 , ..., ¯vM , ¯v1 , ..., ¯vM and  +  λ1 ... 0 0 ... 0    ...     0 ... λ+ 0 ... 0   M  def diag + − + Λ =  +  = λk ; λk .  0 ... 0 −λ1 ... 0     ...    + 0 ... 0 0 ... −λM

Returning to our conventional notation, that is, setting A¯ m,j for A and Vm,j for V, we express the matrix exponential in equation (8.17) as −A¯ m,j 4rj 0 −1 e = Vm,jΛm,jVm,j, (8.19) 0 where the diagonal matrix Λm,j is given by

0 + + Λm,j = diag a0 λk 4rj ; a0 −λk 4rj with −x a0 (x) = e .

To compute the integrals involving the source term, we use the basic result Z Z −A¯ m,j ρ + + −1 e f (ρ) dρ = Vm,j diag a0 λk ρ ; a0 −λk ρ f (ρ) dρ Vm,j

SGP OL1b-2 ATBD Version 7 Page 82 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

with f being some scalar function of ρ and obtain

Z 4rj −A¯ m,j ρ e bm (ρ) dρ 0 1 −1 ¯ 2 −1 ¯ = Vm,jΛm,jVm,j 4rjbth,m,j + Vm,jΛm,jVm,j 4rjbth,m,j+1 3 −1 ¯ +Vm,jΛm,jVm,j 4rjbsun,m,j . (8.20) The diagonal matrices entering in equation (8.20) are given by

1 + + Λm,j = diag a1 λk 4rj ; a1 −λk 4rj , 2 + + Λm,j = diag a2 λk 4rj ; a2 −λk 4rj , (8.21) 3 + + Λm,j = diag b1 λk 4rj ; b1 −λk 4rj , while the interpolation functions read as

Z 1 1 − (1 + x) e−x e−xξ d a1 (x) = ξ ξ = 2 , 0 x Z 1 x − 1 + e−x − e−xξ d a2 (x) = (1 ξ) ξ = 2 , 0 x 1 − τ sun +x −τ sun Z sun sun e ( ext,j ) − e ext,j+1 −[ξτext,j +(1−ξ)τext,j+1] −xξ b1 (x) = e e dξ = sun sun . 0 τext,j+1 − τext,j − x

Inserting equations (8.19) and (8.20) into equation (8.17), we establish a ﬁrst representation of the layer equation

0 −1 1 −1 ¯ im,j+1 = Vm,jΛm,jVm,jim,j − Vm,jΛm,jVm,j 4rjbth,m,j 2 −1 ¯ 3 −1 ¯ −Vm,jΛm,jVm,j 4rjbth,m,j+1 − Vm,jΛm,jVm,j 4rjbsun,m,j . (8.22)

For negative eigenvalues of the matrix A¯ m,j the arguments of the exponential functions in equation (8.21) are positive and as a result the layer equation (8.22) is numerically unstable. This instability can be circumvented −1 if we left multiply equation (8.22) by Vm,j and scale the resulting equation by the diagonal matrix

+ Dm,j = diag 1; a0 λk 4rj . We then obtain

−1 Dm,jVm,jim,j+1 ¯ 0 −1 ¯ 1 −1 ¯ = Λm,jVm,jim,j − Λm,jVm,j 4rjbth,m,j ¯ 2 −1 ¯ ¯ 3 −1 ¯ −Λm,jVm,j 4rjbth,m,j+1 − Λm,jVm,j 4rjbsun,m,j (8.23) ¯ p p with Λm,j = Dm,jΛm,j for all p = 0, ..., 3. Accounting of the identities

−x −x e a1 (−x) = a2 (x) , e a2 (−x) = a1 (x) ,

we ﬁnd the following numerically stable representations of the diagonal matrices in equation (8.23):

¯ 0 + Λm,j = diag a0 λk 4rj ; 1 , ¯ 1 + + Λm,j = diag a1 λk 4rj ; a2 λk 4rj , ¯ 2 + + Λm,j = diag a2 λk 4rj ; a1 λk 4rj , (8.24) ¯ 3 + + Λm,j = diag b1 λk 4rj ; b2 λk 4rj

SGP OL1b-2 ATBD Version 7 Page 83 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

with sun sun −(τext,j+1+x) −τext,j −x e − e b2 (x) = e b1 (−x) = sun sun . τext,j − τext,j+1 − x In compact notations the scaled layer equation (8.23) can be expressed as

1 2 Am,jim,j + Am,jim,j+1 = bm,j,

where the layer quantities are given by

1 ¯ 0 −1 Am,j = Λm,jVm,j, 2 −1 Am,j = −Dm,jVm,j, ¯ 1 −1 ¯ ¯ 2 −1 ¯ bm,j = Λm,jVm,j 4rjbth,m,j + Λm,jVm,j 4rjbth,m,j+1 ¯ 3 −1 ¯ +Λm,jVm,j 4rjbsun,m,j .

1 2 It should be remarked that the scaling process has a symmetry effect on the layer matrices Am,j and Am,j: the ¯ 0 matrices Λm,j and Dm,j have a “complementary” diagonal structure.

8.2.5.6. Padé approximation

For optically thin layers the Padé approximation to the matrix exponential simpliﬁes the algorithm implementation. Essentially, the pth diagonal Padé approximation to the exponential of the matrix A is given by

−1 Qp (−A) Qp (A) ,

where Qp (X) is a polynomial in X of degree p deﬁned by

p X k (2p − k)!p! Qp (X) = ckX , ck = . (2p)!k!(p − k)! k=0

For an efﬁcient application of the Padé approximation and under the assumption of optically thin layers, we suppose a linear approximation of the source term

ρ ρ bm (ρ) = b¯m,j + 1 − b¯m,j+1 4rj 4rj

with the radiance level values being given by

−τ sun b¯m,j = b¯th,m,j + b¯sun,m,je ext,j , −τ sun b¯m,j+1 = b¯th,m,j+1 + b¯sun,m,je ext,j+1 .

In this context the ﬁrst-order Padé approximation

4 −1 4 −A¯ m,j 4rj rj rj e ≈ I + A¯ m,j I − A¯ m,j 2 2

together with integral approximations

Z 4rj 2 1 ρ ¯ 1 4rj 4rj −Am,j ρ ¯ ¯ 2 e dρ ≈ I − Am,j + Am,j (8.25) 4rj 0 4rj 2 3 8 and Z 4rj 2 1 ρ ¯ 1 4rj 4rj −Am,j ρ ¯ ¯ 2 1 − e dρ ≈ I − Am,j + Am,j (8.26) 4rj 0 4rj 2 6 24

SGP OL1b-2 ATBD Version 7 Page 84 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

give the layer equation 1 2 Am,jim,j + Am,jim,j+1 = bm,j with

1 4rj A = I − A¯ m,j, m,j 2 2 4rj A = − I + A¯ m,j , m,j 2 1 4rj bm,j = I − A¯ m,j 4rjb¯m,j 2 12 1 4rj + I + A¯ m,j 4rjb¯m,j . 2 12 +1

The integral approximations (8.25) and (8.26) have been obtained by considering a Taylor expansion of the matrix exponential function, while the layer equation has been derived by left multiplying equation (8.17) with I + (4rj/2) A¯ m,j. As in the case of the matrix eigenvalue method, this process leads to a symmetric form of 1 2 the layer matrices Am,j and Am,j. The second-order Padé approximation

!−1 ! 4 2 4 2 4rj rj 2 4rj rj 2 exp −A¯ m,j4rj ≈ I + A¯ m,j + A¯ I − A¯ m,j + A¯ 2 6 m,j 2 6 m,j

in conjunction with the integral approximations (8.25) and (8.26) yield a similar equation with the layer quantities

4 2 1 4rj rj 2 A = I − A¯ m,j + A¯ m,j 2 6 m,j ! 4 2 2 4rj rj 2 A = − I + A¯ m,j + A¯ , m,j 2 6 m,j ! 4 2 1 4rj rj 2 bm,j = I − A¯ m,j + A¯ 4rjb¯m,j 2 12 24 m,j ! 4 2 1 4rj rj 2 + I + A¯ m,j + A¯ 4rjb¯m,j . 2 12 24 m,j +1

The algorithm based on the ﬁrst-order Padé approximation is essentially equivalent to the ﬁnite-difference method.

8.2.5.7. System matrix of the entire atmosphere

The radiative transfer problem must be solved subject to the boundary conditions at the top of the atmosphere and at the lower surface of the atmosphere. At the top of the atmosphere the down-welling diffuse radiation vanishes and the boundary condition can be expressed in matrix form as

[0, I] im,1 = 0, (8.27)

where dim ([0, I]) = M × 2M. The surface boundary condition also possesses a matrix representation

[I, Rm] im,N = rm, (8.28)

where the entries of the reﬂection matrix and of the reﬂection vector are given by

− norm + − [Rm]kl = −2Awl µl ρm µk , µl (8.29)

SGP OL1b-2 ATBD Version 7 Page 85 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

and

+ [rm]k = δm0 µk B (rs)

sun Fsun −τ (|rs−rTOA|) norm + + (2 − δm ) A |µ | e ext ρ µ , µ , (8.30) 0 π sun m k sun norm respectively. In equations (8.29) and (8.30) is the surface emissivity, A is the surface albedo and ρm are azimuthal expansion coefﬁcients of the normalized bi-directional reﬂection function. Assembling the layer equations together with the boundary conditions (8.27) and (8.28) into the global matrix of the entire atmosphere, leads to the matrix equation

Amim = bm with dim (Am) = 2MN × 2MN. The organization of the system matrix Am and of the source vector bm is shown in Table 1. As in the conventional approach the matrix Am has 3M − 1 sub- and super-diagonals and it may be compressed into band-storage and then inverted using standard methods.

Am bm row\column 2M 2M 2M ... 2M 2M M [0, I] 0 0 ... 0 0 0 1 2 2M Am,1 Am,1 0 ... 0 0 bm,1 1 2 2M 0 Am,2 Am,2 ... 0 0 bm,2 ...... 1 2 2M 0 0 0 ... Am,N−1 Am,N−1 bm,N−1 M 0 0 0 ... 0 [I, Rm] rm

Table 8.1.: Organization of the system matrix and of the source vector for the entire atmosphere.

8.2.5.8. Weighting functions calculation

The retrieval of atmospheric constituents from satellite measurements requires the knowledge of the weighting functions, i.e. the partial derivatives of the measured radiance with respect to the atmospheric parameters being retrieved. The process of obtaining the set of partial derivatives which constitute the matrix of weighting functions is commonly referred to as linearisation analysis. The radiance measured by a satellite instrument can be expressed in the framework of the source integration technique in terms of the solution of the radiative transfer equation In this regard the derivatives calculation can be performed by linearising the radiative transfer equation with respect to the desired parameters. The signal measured by a satellite instrument can be modelled by integrating the radiative transfer equation along the line of sight. For a nadir viewing geometry with the line of sight bounded by the surface point S and the point at the top of the atmosphere A, the diffuse radiance in the measurement direction Ωm can be expressed as I (rA, Ωm) = Iss (rA, Ωm) + Ims (rA, Ωm) , (8.31) where Z −τext(|rA−rs|) −τext(|rA−r|) Iss (rA, Ωm) = Iss (rs, Ωm) e + Jss (r, Ωm) e ds (8.32) |rA−rs| and Z −τext(|rA−rs|) −τext(|rA−r|) Ims (rA, Ωm) = Ims (rs, Ωm) e + Jms (r, Ωm) e ds (8.33) |rA−rs|

SGP OL1b-2 ATBD Version 7 Page 86 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

are the single and the multiple scattering contributions, respectively, and Z 0 0 τext (|r1 − r2|) = σext (r ) ds |r1−r2|

is the extinction optical depth between the points r1 and r2. The single and the multiple scattering source functions are deﬁned by

σscat (r) −τ sun |r−r | J (r, Ω) = F P (r, Ω, Ω ) e ext ( TOA ), (8.34) ss sun 4π sun Z σscat (r) 0 0 0 Jms (r, Ω) = P (r, Ω, Ω ) I (r, Ω ) dΩ , (8.35) 4π 4π while the surface values of the radiance ﬁelds are given by

+ A + −τ sun(|r −r |) I r , Ω = F |µ | ρ Ω , Ω e ext s TOA , (8.36) ss s sun π sun norm sun Z + A − − + − − Ims rs, Ω = I rs, Ω µ ρnorm Ω , Ω dΩ . (8.37) π 2π For a limb viewing geometry the boundary conditions (8.36) and (8.37) have to be replaced by homogeneous boundary conditions. For a numerical computation of the path integrals in equations (8.32) and (8.33) the atmosphere is discretized in homogeneous layers. In this context the measured radiance I (rA, Ωm) can be regarded as a function of some vector parameters, e.g. the layer values of the extinction and the scattering coefﬁcients or of some scalar parameters, e.g. the surface albedo. If ςi is an atmospheric parameter in the layer i, then equation (8.31) yields

∂I ∂Iss ∂Ims (rA, Ωm) = (rA, Ωm) + (rA, Ωm) . ∂ςi ∂ςi ∂ςi

The single scattering radiance satisfies the radiative transfer equation with the source function (8.34) and the boundary condition (8.36). Integrating the differential equation along all paths of the line of sight bounded by adjacent layers, we are led to a recurrence relation for the single scattering radiance. This recurrence relation depends analytically on the layer values of the optical coefficients and on the surface albedo. As a result a recurrence relation for the partial derivative of the single scattering radiance can be derived in a straightforward manner. In view of equations (8.33), (8.35) and (8.37) the multiple scattering radiance can be computed on a recursive basis, which in turn leads to a recurrence relation for the partial derivatives ∂Ims/∂ςi along the boundary points of the line of sight. This recursion requires the knowledge of the partial derivative of the radiance field in a plane-parallel atmosphere and at all levels j, ∂Ij/∂ςi. To compute the level quantities ∂Ij/∂ςi we will use the radiance solution computed in the framework of the discrete ordinate method with matrix exponential. The matrix exponential formalism operates with the concept of the layer equation

1 2 Am,jim,j + Am,jim,j+1 = bm,j, (8.38)

which relates the level values of the radiance ﬁeld

" + # im,j im,j = − im,j

with i± (r) = I (r , ±µ ), k = 1, ..., M. m,j k m j k The layer equation together with the boundary conditions at the top and the bottom of the atmosphere are assembled into the global matrix of the entire atmosphere and the solution of the resulting system of equations yields the level values of the radiance ﬁeld.

SGP OL1b-2 ATBD Version 7 Page 87 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

To compute the partial derivative with respect to the atmospheric parameter ςi we linearise the layer equation (8.38) and obtain

A1 A2 1 ∂im,j 2 ∂im,j+1 ∂bm,j ∂ m,j ∂ m,j Am,j + Am,j = − im,j − im,j+1. (8.39) ∂ςi ∂ςi ∂ςi ∂ςi ∂ςi As for radiance calculation all sets of derivative layer equations (8.39) are assembled into a global system of equations for the entire atmosphere. It is worth to notice, that the system matrix for derivative calculation coincides with the system matrix for radiance calculation; only the right-hand sides are different. In order to increase the efﬁciency of the method we compute the partial derivatives with respect to all atmospheric parameters of interest ςi, i = 1, ..., Nd, that is, we solve a system of equations with multiple right-hand sides. To 1 2 compute the partial derivatives of the layer quantities Am,j, Am,j and bm,j we apply the chain rule. In the case of Padé approximation, the derivatives calculation is trivial, but in the case of the matrix eigenvalue method we −1 are faced with the calculation of the partial derivatives of the eigenvector matrix Vm,j and of the eigenvalues λk. −1 + To compute ∂Vm,j/∂ςi and ∂λk/∂ςi we consider the eigenvalue problem for the matrix A

+ + + A wk = µkwk .

For a ﬁxed discrete ordinate index k we take the derivative with respect to ςi and obtain

+ + + ∂A + + ∂wk ∂µk + ∂wk wk + A = wk + µk . (8.40) ∂ςi ∂ςi ∂ςi ∂ςi

+ Equation (8.40) is a system of M equations with M + 1 unknowns: the scalar ∂µk/∂ςi and the vector ∂wk /∂ςi. + Since the eigenvectors wk are normalized, we derive an additional equation

+ +T ∂wk wk = 0, (8.41) ∂ςi which implies the compatibility of the system of equations. Using equations (8.40) and (8.41) the resulting system of equations can be written in matrix form as

+ " + + #" ∂µk # " ∂A + # w µkI − A w k ∂ςi ∂ςi k T ∂w+ = . 0 w+ k 0 k ∂ςi

It is important to observe that we can solve the above system of equations for all atmospheric parameters ςi, i = 1, ..., Nd, that is, we can solve the matrix equation   " # ∂µk ∂µk " + + # w+ I − A+ ... ∂A w+ ... ∂A w+ k µk ∂ς1 ∂ςNd ∂ς1 k ∂ςN k  + +  = d . +T ∂wk ∂wk 0 wk ... 0 ... 0 ∂ς1 ∂ςNd

+ √ If ∂µk/∂ςi is known, the partial derivative of λk = µk with respect to ςi follows immediately

+ ∂λk 1 ∂µk = + . ∂ςi 2λk ∂ςi

− To compute the partial derivative of wk we use the deﬁnition (8.18) and apply the chain rule to obtain

− + + ∂wk 1 ∂λk + + 1 ∂A + 1 + ∂wk = − +2 A wk + + wk + + A . ∂ςi λk ∂ςi λk ∂ςi λk ∂ςi Further calculations give

∂v+ 1 ∂w+ ∂w− ∂v− 1 ∂w+ ∂w− k = k + k , k = k − k ∂ςi 2 ∂ςi ∂ςi ∂ςi 2 ∂ςi ∂ςi

SGP OL1b-2 ATBD Version 7 Page 88 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

and " ∂v+ # " ∂v− # ∂v¯+ k ∂v¯− k k ∂ςi k ∂ςi = ∂v− , = ∂v+ . ∂ςi k ∂ςi k ∂ςi ∂ςi

Using now the deﬁnition of the Vm,j matrix we obtain

∂V ∂v¯+ ∂v¯+ ∂v¯− ∂v¯− m,j = 1 , ..., M , 1 , ...., M ; ∂ςi ∂ςi ∂ςi ∂ςi ∂ςi

−1 whence, taking into account that Vm,jVm,j = I, we end up with

V−1 ∂ m,j −1 ∂Vm,j −1 = −Vm,j Vm,j. ∂ςi ∂ςi

The calculation of the partial derivatives for Nd atmospheric parameters and a speciﬁc solar zenith angle requires the solution of a system of equations with Nd + 1 right-hand sides; while, in the matrix eigenvalue method, a system of equations with M + 1 unknowns has to be solved additionally for each layer.

8.2.6. Picard iteration

Different methods have been developed to solve the multiple scattering problem in a spherical atmosphere. These include an order of scattering solution method, ﬁnite difference method in conjunction with a combined differential-integral approach and the Monte Carlo method. In this Section we analyse several versions of the Picard iteration for solving the radiative transfer equation in a spherical atmosphere. To compute the limb radiance at the top of the atmosphere we use the integral form of the radiative transfer equation

−τext(|r−rr|) I (r, Ω) = I (rr, Ω) e + Iss (r, rr, Ω) + Ims (r, rr, Ω) , (8.42) where rr is a reference point and derive a recurrence relation for the diffuse radiance at a set of discrete points along the line of sight. The recurrence relation then takes the form

I (rp, Ω) = I (rp+1, Ω) exp (−τp) + Iss (rp, rp+1, Ω) + Ims (rp, rp+1, Ω) , (8.43)

Np where {rp}p=1 is the set of intersection points of the line of sight with a sequence of spherical surfaces. In a single scattering model the multiple scattering contribution Ims is neglected, while in a multiple scattering model this term has to be included in the computation. Our further analysis is focused on the computation of the multiple scattering contribution Ims, which in turns requires the computation of the diffuse radiance at the limb points rp, p = 1, 2, ..., Np. We choose a global coordinate system by directing the Z-axis along the solar direction, since this choice leads to an axis-symmetric radiation field. At each point on the line of sight the local coordinate system is chosen as the local spherical coordinate system. The polar angles θ and ϕ of the direction Ω are specified in the local coordinate system. By convention the first quadrant of the local coordinate system corresponds to θ ∈ (0, π/2) and ϕ ∈ (−π/2, π/2), the second quadrant to θ ∈ (0, π/2) and ϕ ∈ (π/2, 3π/2), the third quadrant to θ ∈ (π/2, π) and ϕ ∈ (π/2, 3π/2) and finally, the fourth quadrant to θ ∈ (π/2, π) and ϕ ∈ (−π/2, π/2). Because the problem is axis-symmetric, i.e. I (r, Θ, Ψ, Ω) = I (r, Θ, Ω), a two-dimensional Picard iteration can be used to compute the radiances at a set of discrete points in the azimuthal plane Ψ = 0. The domain of analysis is shown in Fig. 8.8 and is given by:

D = {(r, Θ, Ψ) / r ∈ [rs, rTOA] , Θ ∈ [0, Θshd] , Ψ = 0} ,

where rs is the lowest (surface) point of the atmosphere, rTOA is the radius at the top of the atmosphere and Θshd is the polar angle at which the shadow region begins. Along the radial line Θ = 0 we consider axis-symmetric boundary conditions I (r, Θ = 0, θ, ϕ) = I (r, Θ = 0, θ, ϕ + π) , ϕ ∈ (−π/2, π/2) , (8.44)

SGP OL1b-2 ATBD Version 7 Page 89 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18 while along the radial line Θ = Θshd we assume homogeneous boundary conditions (for the radiances pointing into the domain) I (r, Θ = Θshd, θ, ϕ) = 0, ϕ ∈ (π/2, 3π/2) . (8.45)

The domain of analysis is discretized in Nr optically homogeneous spherical shells with radii ri in decreasing order, i.e. r1 = rTOA > r2 > ... > rNr = rs, while the zenith direction is discretized in NΦ equidistant radial lines. The local direction of the radiance in each grid point is discretized in NθNϕ discrete ordinates, where Nθ is the number of zenith directions and Nϕ is the number of azimuthal directions.

TOA axisymmetric BC independent BC Earth min

max X O

shd

homogeneous BC

Figure 8.8.: Domain of analysis and boundary conditions.

The radiative transfer equation

dI (r, Ω) = −σ (r) I (r, Ω) + J (r, Ω) + J (r, Ω) ds ext ss ms with

Fsun −τ sun |r−r r | J (r, Ω) = S (r, Ω, Ω ) e ext ( TOA( ) ), ss 4π sun Z 1 0 0 0 Jms (r, Ω) = S (r, Ω, Ω ) I (r, Ω ) dΩ 4π 4π is of the form I = f (I) and the Picard iteration technique based on a ﬁxed-point iteration is appropriate for its numerical solution: if the sequence I(n+1) = f I(n) converges to I and the function f is continuous, then there holds f I(n) → f (I), which, in turn, yields I = f (I). Thus, at the iteration step n the recurrence relation for computing the radiance at the generic point r and along the characteristic Ω read as

(n+1) (n+1) −τext(|r−rr|) (n) I (r, Ω) = I (rr, Ω) e + Iss (r, rr, Ω) + Ims (r, rr, Ω) , (8.46)

SGP OL1b-2 ATBD Version 7 Page 90 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

where essentially r stands for the grid point rij = (ri, Θj, Ψ = 0) with i = 1, ..., Nr and j = 1, ..., NΦ and Ω for the discrete ordinate direction Ωkl = (θk, ϕl) with k = 1, ..., Nθ and l = 1, ..., Nϕ. In the long characteristic method the reference point is the intersection point of the characteristic with the model boundary D (Fig. 8.9). The radiance at the generic grid point is computed on a recursive basis by Nq considering the intersection of the characteristic with a set of spherical surfaces {rq}q=1, for which it holds true that r1 = rr > r2 > ... > rNq = r. The recurrence relation is as in equation (8.43), but with p replaced by q. The (n+1) radiance at the reference point I (rr, Ω) is used to initialize the recursion and is speciﬁed by the model (n) boundary radiance. The multiple scattering radiance Ims (rq, Ω) at the characteristic point rq is computed by (n) a linear interpolation in the spatial and the discrete-ordinate domains using the grid point values Ims (rij, Ωkl). The single scattering contribution Iss (rq, rq+1, Ω) is calculated at all stages of the characteristic without any interpolation. In the short characteristic method the reference point is the intersection point of the characteristic with the cell boundary (Fig. 8.9). The radiance at the generic point is computed accordingly to equation (8.46), where the (n+1) radiance at the reference point I (rr, Ω) is determined via a linear interpolation of the grid point values of (n) the face pierced by the characteristic. As before the multiple scattering radiance Ims (rr, Ω) is computed by linear interpolation, while the single scattering contribution Iss (r, rr, Ω) is calculated in an exact manner. The number of iterations required to attain convergence can be reduced by an appropriate choice of the initial estimate. For each radial line Θj ≤ Θlim, where Θlim / 90°, we compute the initial radiance ﬁeld by using a one-dimensional model. Because the calculations are performed individually for each radial line (or solar zenith angle), this model is equivalent to the independent pixel approximation for three-dimensional radiative transfer in clouds.

long characteristic short characteristic

M r e r M r M r r  e r r r

Figure 8.9.: Long and short characteristics.

The short and the long characteristic Picard iteration involves the following steps: 1. Consider a two-dimensional discretization of the domain of analysis D.

2. Compute the initial estimate I (rij, Ωkl) at all radial lines Θj ≤ Θlim by using the independent pixel approximation and initialize the radiances at all radial lines Θshd ≥ Θj > Θlim to zero.

3. Determine the single scattering contribution Iss at all grid points and in all discrete ordinate directions.

4. Calculate the grid values of the multiple scattering radiance Ims (rij, Ωkl).

SGP OL1b-2 ATBD Version 7 Page 91 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

5. Beginning downward for each layer i = 1, ..., Nr − 1 compute the radiances at the lower nodes for all discrete ordinates in the third and fourth quadrant and at all radial lines. For the boundary cells take into account the boundary conditions (8.44) and (8.45). 6. Compute the upward radiances at the bottom of the domain by using the boundary conditions for a Lambertian surface. 7. Working upwards repeat the calculations of step 5, but compute the radiances at the upper nodes for all discrete ordinates in the ﬁrst and second quadrant.

8. Interpolate the radiances at each point rp of the line of sight by using the grid radiance values of the cell containing the point.

9. Compute the multiple scattering contribution Ims (rp, rp+1, Ω) at all stages of the line of sight. 10. Repeat steps 4-9 until the maximum change in the multiple scattering contributions is smaller than some preassigned error values at all stages of the line of sight. Steps 4-7 represent one Picard iteration. A speciﬁc feature of the above method is that the grid values of the multiple scattering radiance Ims (rij, Ωkl) are computed before starting the downward and the upward recur- rences, and are implicitly determined by the grid radiance values I (rij, Ωkl) at the previous Picard iteration. This technique works well for the long characteristic method, but yields a slow convergence rate for the short characteristic method. To remedy this drawback we use an iterative scheme, in which we update the multiple scattering radiances at each layer calculation. For example, in the case of the downward recurrence we ﬁrst compute the multiple scattering radiances at the top of the layer and then proceed to calculate the radiances at the bottom of the layer. The multiple scattering radiances then include the values of the grid radiances in the third and fourth quadrant at the actual Picard iteration. By convention this version of the Picard iteration will be called the accelerated short characteristic method. A further reduction of the computation time can be achieved by using the following approximate model: instead of solving the radiative transfer problem on the domain D, we solve the problem on the reduced domain

Dr = {(r, Θ, Ψ) / r ∈ [rs, rTOA] , Θ ∈ [Θmin, Θmax] , Ψ = 0} with Θmin < Θp < Θmax for all p = 1, 2, ..., Np. The reduced domain of analysis is illustrated in Fig. 8.8. The values of the diffuse radiance pointing into the domain (the boundary values along the radial lines Θ = Θmin and Θ = Θmax) are computed by using the pseudo-spherical model and remain unchanged during the iterative process. Thus, the boundary lines become independent of the interior cells and the radiance at their grid points serves as the boundary radiance for the interior points. This type of boundary conditions is known as open boundaries. Therefore this method will be referred to as the Picard iteration method with open boundaries.

8.3. Inversion methods

Most of the inverse problems arising in atmospheric remote sensing are non-linear. In this chapter we discuss the practical aspects of Tikhonov regularization for solving the non-linear problem

F (x) = y, (8.47) where, due to the complexity of the radiative transfer, the forward model is computed by using a numerical model. The non-linear system of equations (8.47) is called a discrete ill-posed problem because the underlying continuous problem is ill-posed. If we accept a characterization of ill-posedness via linearisation, the condition number of the Jacobian matrix K (x) = F0 (x) may serve as a quantiﬁcation of ill-posedness. The right-hand side y of (8.47) is supposed to be contaminated by measurement errors and we have the representation

yδ = y + δ, where yδ is the noisy data and δ is the noise vector. In a deterministic setting the error is characterized by the noise level ∆, while in a semi-stochastic setting δ is assumed to be a discrete white noise vector with the 2 covariance matrix Cδ = σ Im.

SGP OL1b-2 ATBD Version 7 Page 92 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

8.3.1. Tikhonov regularization

The formulation of Tikhonov regularization for non-linear problems is straightforward: the equation (8.47) is replaced by a minimization problem involving the objective function

1 h δ 2 2i F (x) = F (x) − y + α kL (x − xa)k . (8.48) 2 For a positive regularization parameter minimizers of the Tikhonov function always exists (but are not unique) δ and a global minimizer xα is called a regularized solution. In this Section we review appropriate optimization methods for minimizing the Tikhonov function, discuss practical algorithms for computing the new iterate and characterize the solution error. Finally, we analyse the performances of the Tikhonov regularization with a priori, a posteriori and error-free parameter choice methods.

8.3.1.1. Inversion models

For retrieval problems in the visible spectral region we consider two inversion models. The first inversion model is the radiance model Rmeas (λ, h) ≈ Pampl λ, pampl, h Rsim (λ, x, h) , (8.49) where λ is the wavelength, Pampl is a polynomial of low order with coefficients pampl and R stands for the “scan-ratioed” radiance ratio, that is, the radiance profile normalized with respect to a reference tangent height

I (·, h) R (·, h) = . (8.50) I (·, href) The normalization procedure removes the need to measure the exoatmospheric solar irradiance and the absolute instrument calibration. This is similar to the self-calibration feature of the limb occultation method, where the unattenuated solar radiation is measured outside the atmosphere. In addition, there is a reduction in the effect of the surface reﬂectance and clouds that can control the diffuse radiation even at high altitudes. The normalization procedure does not completely remove the effect of the surface albedo, but does reduce the accuracy to which the algorithm must model this effect. The closure term Pampl is intended to account on the contribution from aerosols with smooth spectral signature. The second inversion model is the differential radiance model ln R¯meas (λ, h) ≈ ln R¯sim (λ, x, h) (8.51) with ln R¯sim (λ, x, h) = ln Rsim (λ, x, h) − Psim (λ, psim, h) and ln R¯meas (λ, h) = ln Rmeas (λ, h) − Pmeas (λ, pmeas, h) .

For any x the simulated smoothing polynomial Psim of coefficients psim = psim (x) and the measurement smoothing polynomial Pmeas of coefficients pmeas, are defined by

2 psim = arg min kln Rsim (·, x, h) − Psim (·, p, h)k p and 2 pmeas = arg min kln Rmeas (·, h) − Pmeas (·, p, h)k , p respectively. In general,a smoothing polynomial is assumed to account on the low frequency structure due to the scattering mechanism, in which case ln R¯ will mainly reﬂects the absorption process due to the gas molecules. The choice of the inversion model is a very important task of the retrieval process, because an appropriate formulation may considerably reduce the non-linearity of the problem. In a stochastic framework the degree of non-linearity can be examined by comparing the forward model with its linearisation within the a priori variability.

SGP OL1b-2 ATBD Version 7 Page 93 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

For this purpose we assume that x is a random vector characterized by a Gaussian a priori probability density function with mean xa and covariance matrix Cx. In the x-space the ellipsoid

T −1 (x − xa) Cx (x − xa) = 1 represents the contour of the a priori covariance, outlining the region within which the state vector is likely to lie. Considering the linear transformation

−1/2 T z = Σx Vx (x − xa) ,

T where Cx = VxΣxVx , we observe that in the z-space the contour of the a priori covariance is a sphere of T T ± h i radius 1 centred at the origin, that is, z z = 1. The points zk = 0, ..., ±1, ..., 0 represents the intersection of the sphere with the coordinate axes and delimit the region to which the state vector belongs. In the x-space these boundary points are given by

± 1/2 ± xk = xa + VxΣx zk = xa ± ck,

1/2 where the vectors ck, deﬁned by the decomposition VxΣx = [c1, ..., cn], represent the error patterns for the covariance matrix Cx. The size of the linearisation error

R (x) = F (x) − F (xa) − K (xa)(x − xa)

can be evaluated through the quantity

1 2 ε k = kR (x ± ck)k . lin mσ2 a

If εlink ≤ 1 for all k, then the inverse problem is said to be linear to the accuracy of the measurements within the normal range of variation of the state. Numerical simulations have shown that the differential radiance model (8.51) yields a smaller linearisation error than the radiance model (8.49). For this reason the differential radiance model will be adopted in our simulations.

8.3.1.2. Optimization methods for the Tikhonov function

δ In the framework of Tikhonov regularization the regularized solution xα is computed by minimizing the function

1 h δ 2 2i F (x) = F (x) − y + α kL (x − xa)k , (8.52) 2 where the factor 1/2 has been included in order to avoid the appearance of a factor two in the derivatives. The general minimization problem can be formulated as the least-squares problem

1 F (x) = kf (x)k2 , (8.53) 2 where the augmented vector f is given by " # F (x) − yδ f (x) = √ . αL (x − xa)

The regularized solution can be computed by using optimization methods for unconstrained minimization problems. Essentially, optimization tools are iterative methods, which use the Taylor expansion to compute approximations to the objective function at all points in the immediate neighbourhood to the current iterate. For Newton–type methods the quadratic model 1 M (p) = F (x) + gT (x) p + pT G (x) p (8.54) 2

SGP OL1b-2 ATBD Version 7 Page 94 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

is used as a reasonable approximation to the objective function. In (8.54) g and G are the gradient and the Hessian of F, that is, T g (x) = ∇F (x) = Kf (x) f (x) and 2 T G (x) = ∇ F (x) = Kf (x) Kf (x) + Q (x) , where " # 0 K (x) Kf (x) = f (x) = √ αL is the Jacobian matrix of f and X Q (x) = fi (x) Gi (x) i 2 with Gi = ∇ fi being the Hessian of fi is the second–order derivative term. Although the objective function (8.53) can be minimized by a general method, in most circumstances the special forms of the gradient and the Hessian make it worthwhile to use methods designed speciﬁcally for least-squares problems. Non-linear optimization methods can be categorized into two broad classes: “step-length-based methods” and “trust-region methods”.

Step–length methods For an iterative method it is important to have a measure of progress in order to decide whether a new iterate δ δ xk+1α is better than the current iterate xkα. A natural measure of progress is to require a decrease of F at every iteration and to impose the descent condition

δ δ F xk+1α < F xkα . A method that impose this condition is termed a descent method. A step–length–based algorithm requires the δ computation of a vector pkα called the search direction and the calculation of a positive scalar τk, the step length, for which it holds that δ δ δ F xkα + τkpkα < F xkα . th δ To guarantee that the objective function F can be reduced at the k iteration step, the search direction pkα δ should be a descent direction at xkα, that is, the inequality

T δ δ g xkα pkα < 0

should hold true. In the steepest-descent method for general optimization the objective function is approximated by a linear model and the search direction is taken as

δ δ pkα = −g xkα .

δ The negative gradient −g xkα is termed the direction of steepest descent and, evidently, the steepest-descent direction is a descent direction (unless the gradient vanishes) since

T δ δ δ 2 g xkα pkα = − g xkα < 0.

The convergence rate of the steepest-descent method is linear and a method yielding a super-linear convergence rate is the Newton method. In the Newton method the objective function is approximated by the quadratic model (8.54) and the search δ direction pkα, which minimizes the quadratic function, solves the so called Newton equation

δ δ G xkα p = −g xkα . (8.55)

δ If the Hessian G xkα is positive deﬁnite, only one iteration is required to reach the minimum of the model function (8.54) from any starting point and we expect good convergence from Newton’s method when the

SGP OL1b-2 ATBD Version 7 Page 95 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18 quadratic model is accurate. For a general non-linear function Newton’s method converges quadratically to δ δ δ the minimizer xα if the initial guess is sufficiently close to xα, the Hessian matrix is positive definite at xα and δ the step lengths {τk} converges to unity. Note that when G xα is positive definite, the solution of (8.55) is a descent direction, since T δ δ T δ −1 δ δ g xkα pkα = −g xkα G xkα g xkα < 0 holds true. T In the Gauss-Newton method for least–squares problems it is assumed that the first-order term Kf Kf in the expression of the Hessian dominates the second-order term Q. This assumption is not justified when the δ residuals at the solution are very large, i.e. roughly speaking, when the residual f xα is comparable to the T δ δ largest eigenvalue of Kf xα Kf xα . For small residual problems the search direction solves the equation

T δ δ T δ δ Kf xkα Kf xkα p = −Kf xkα f xkα (8.56) and possesses the variational characterization

δ δ δ 2 pkα = arg min Kf xkα p + f xkα . (8.57) p

The vector that solves (8.56) or (8.64) is called the Gauss-Newton direction and if Kf is of full column rank, δ then the Gauss-Newton direction is uniquely and approaches the Newton direction as Q xα tends to zero. δ δ Consequently, if f xα is zero and the columns of K xα are linearly independent, the Gauss-Newton method can ultimately achieve a quadratic rate of convergence, despite the fact that only ﬁrst derivatives are δ used to compute pkα. δ For large–residual problems the term f xα is not small and the second–order term Q cannot be neglected. δ In fact, a large-residual problem is one in which the optimal residual f xα is large relative to the small T δ δ eigenvalues of Kf xα Kf xα , but not with respect to its largest eigenvalue. One possible strategy for large- ¯ δ residual problems is to include a quasi-Newton approximation Q xkα to the second–order derivative term δ Q xkα and to compute the search direction by solving the equation

T δ δ ¯ δ T δ δ Kf xkα Kf xkα + Q xkα p = −Kf xkα f xkα . (8.58)

Quasi-Newton methods are based on the idea of building up curvature information as the iterations proceed using the observed behaviour of the objective function and of the gradient. The initial approximation of the second–order derivative term is usually taken as zero and, with this choice, the ﬁrst iteration of the quasi- δ Newton method is equivalent to an iteration of the Gauss–Newton method. After xk+1α has been computed, ¯ δ ¯ δ a new approximation of Q xk+1α is obtained by updating Q xkα to take into account the newly-acquired curvature information. An update formula read as

¯ δ ¯ δ Q xk+1α = Q xkα + Uk, where the update matrix Uk is usually chosen as a rank–one matrix. The standard condition for updating Q¯ is known as the quasi–Newton condition and requires that the Hessian should approximate the curvature of the objective function along the change in x during the kth iteration. The most widely used quasi-Newton scheme, which satisﬁes the quasi–Newton condition and possesses the property of hereditary symmetry, is the Broyden-Fletcher-Goldfarb-Shanno (BFGS) update

¯ δ ¯ δ 1 δ T δ 1 T Q xk+1α = Q xkα − T δ W xkα sksk W xkα + T hkhk , (8.59) sk W xkα sk hk sk where δ δ sk = xk+1α − xkα is the change in x during the kth iteration

δ δ hk = g xk+1α − g xkα

SGP OL1b-2 ATBD Version 7 Page 96 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18 is the change in gradient and

δ T δ δ ¯ δ W xkα = Kf xk+1α Kf xk+1α + Q xkα .

A step-length procedure is frequently included in Newton–type methods because a step of unity along the Newton direction will not necessarily reduce the objective function, even though it is the step to the minimum of the function. The main requirements of a step-length algorithm can be summarized as follows: if x and p denote the actual iterate and the search direction, respectively, then 1. the average rate of decrease from F (x) to F (x + τp) should be at least some prescribed fraction of the initial rate of decrease in that direction

T F (x + τp) ≤ F (x) + εfτg (x) p, εf > 0;

2. the rate of decrease of F in the direction p at x + τp should be larger than some prescribed fraction of the rate of decrease in the direction p at x

T T g (x + τp) p ≥ εgg (x) p, εg > 0.

The first condition guarantees a sufficient decreases in F values relative to the length of the step, while the second condition avoids too small steps relative to the initial rate of decrease of F. The condition εg > εf implies that both conditions can be satisfied simultaneously. In practice, the second condition is not needed because the use of a backtracking strategy avoids excessively small steps. Since computational experience has shown the importance of taking a full step whenever possible, the modern strategy of a step-length algorithm is to start with τ = 1 and then, if x+p is not acceptable, “backtrack” (reduce τ) until an acceptable x+τp is found. The implemented backtracking step-length algorithm uses only condition (1) and is based on quadratic and cubic interpolation. On the first backtracking the new step length is selected as the minimizer of the quadratic interpolation function mq (τ), satisfying the conditions

0 T mq (0) = F (x) , mq (0) = g (x) p, mq (1) = F (x + p) but being constrained to be larger than ε1 = 0.1 of the old step length. On all subsequent backtracks the new step length is chosen by using the values of the objective function at the last two values of the step length. Essentially, if τ and τprev are the last two values of the step length, the new step length is computed as the minimizer of the cubic interpolation function mc (τ), satisfying the conditions

0 T mc (0) = F (x) , mc (0) = g (x) p, and mc (τ) = F (x + τp) , mc τprev = F x + τprevp , but being constrained to be larger than ε1 = 0.1 and smaller than ε2 = 0.5 of the old step length.

Trust–region methods

In a trust region method the step length τk is taken as unity, so that the new iterate is deﬁned by

δ δ δ xk+1α = xkα + pkα.

δ For this reason the term “step” is often used to designate the search direction pkα. In order to ensure that the δ descent condition holds, it must be necessary to compute several trial vectors before ﬁnding a satisfactory pkα. δ The most common mathematical formulation of this idea computes the trial step pkα by solving the constrained minimization problem min Mk (p) subject to kpk ≤ Γk, (8.60) p

SGP OL1b-2 ATBD Version 7 Page 97 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

δ where Mk is the quadratic model (8.54) at the current iterate xkα and Γk is the trust region radius. Assuming that the solution occurs on the boundary of the constrained region, the minimization of the Lagrangian

1 1 L (p, λ) = F xδ + gT xδ p + pT G xδ p + λ kpk2 − Γ2 kα kα 2 kα 2 k yields the ﬁrst–order optimality conditions

δ δ G xkα + λIn pλ = −g xkα (8.61)

and 2 2 kpλk = Γk. (8.62) Particularizing the trust-region method for general minimization to least-squares problems with a Gauss–Newton Hessian approximation, we deduce that the search direction solves the equation

T δ δ T δ δ Kf xkα Kf xkα + λIn pλ = −Kf xkα f xkα , (8.63)

while the Lagrange parameter λ solves the equation (8.62). For comparison with the Gauss-Newton method we note that the solution of (8.63) is a solution of the linear regularized least-squares problem

δ h δ δ 2 2i pkα = arg min Kf xkα p + f xkα + λ kpk . (8.64) p

δ δ If λ is zero, pkα is the Gauss-Newton direction as λ → ∞, pkα becomes parallel to the steepest-descent δ direction −g xkα . Generally, a trust-region algorithm uses the predictive reduction in the linearised model (8.54)

δ ∆Fpredk = Mk (0) − Mk pkα (8.65)

and the actual reduction in the objective function

δ δ δ ∆Fk = F xkα − F xkα + pkα (8.66)

δ to decide whether the trial step pkα is acceptable and how the next trust-region radius is chosen. The heuristic to update the size of the trust region usually depends on the ratio of the expected change in F to the predicted change ∆Fk rk = . ∆Fpredk

The trust-region algorithm ﬁnds a new iterate and produces a trust-region radius for the next global iteration. The algorithm starts with the calculation of the trial step p for the actual trust-region radius (c.f. equations (8.62) and (8.63) ) and with the computation of the prospective iterate xnew = x + p and of the objective function F (xnew). Then, depending on the average rate of decrease of the objective function, the following situations may appear: T 1. If F (xnew) ≥ F (x)+εfg (x) p, then the step is unacceptable. In this case, if the trust-region radius is too small, the algorithm terminates with xnew = x. If not, the step length τmin is computed as the minimizer of the quadratic interpolation function mq (τ), satisfying the conditions

0 T mq (0) = F (x) , mq (0) = g (x) p, mq (1) = F (x + p)

and the new radius is chosen as τmin kpk but constrained to be between ε1Γ = 0.1 and ε2Γ = 0.5 of the old radius. T 2. If F (xnew) < F (x) + εfg (x) p, then the step is acceptable and the reduction of the objective function predicted by the quadratic model

T 2 4Fpred = −g (x) p − 0.5 kKf (x) pk

is computed. If ∆F and 4Fpred agree to within a prescribed tolerance or negative curvature is indicated,

SGP OL1b-2 ATBD Version 7 Page 98 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

then the trust-region radius is increased and the while-iteration loop is continued. If not, xnew is accepted as the new iterate and the trust-region radius is updated for the next global iteration.

Stopping criteria In a deterministic setting the standard stopping criteria for unconstrained minimization are: 1. the relative gradient test

δ δ g xkα i maxi xkα i , typ [x]i max ≤ εg, i δ max F xkα , typF

2. the absolute function convergence test δ F xkα < εfa,

3. the relative function convergence test

δ δ F xkα − F xk+1α δ ≤ εfr, F xkα

4. the X-convergence test n o max xδ − xδ i k+1α i kα i n o ≤ εx. max xδ , typ [x] i k+1α i i

It should be mentioned that the problem of measuring relative change when the argument z is near zero is addressed by substituting z with max {|z| , typz}, where typz is an user’s estimate of a typical magnitude of z. Also note that in the PORT optimization routines the X-convergence test is expressed as n o max xδ − xδ i k+1α i kα i ≤ ε . (8.67) n o x max xδ + xδ i k+1α i kα i

In a stochastic framework the absolute and relative function convergence test are formulated as

2 δ δ χ F xkα − y ≤ m and χ2 F xδ − yδ − χ2 F xδ − yδ kα k+1α ≤ ε , 2 δ δ fr χ F xkα − y respectively, while the X-convergence test read as

δ δ T − δ δ x − x Cb 1 x − x k+1α kα x k+1α kα ≤ ε , n x where 2 δ δ δ δT −1 δ δ χ F xkα − y = F xkα − y Cδ F xkα − y ,

2 Cδ is the measurement error covariance and Cb x is the a posteriori covariance. If Cδ = σ Im, then the absolute function convergence test is equivalent to the discrepancy principle for the square residual norm

δ 2 δ δ 2 rkα = F xkα − y .

Because in practical applications the estimation of the covariance matrices is problematic, we suggest to use the X-convergence test as in (8.67) and the relative function convergence test as in a deterministic setting, but δ 2 δ with the residual norm rkα in place of the Tikhonov function F xkα . For a data model with white noise this test will be also valid in a stochastic setting.

SGP OL1b-2 ATBD Version 7 Page 99 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

8.3.1.3. Practical methods for computing the new iterate

A step-length based method for minimizing the Tikhonov function is of the form of the following model algorithm: 1. compute the search direction; 2. compute the step length (see 8.3.1.2); 3. terminate the iterative process accordingly to the X-convergence criterion or the relative function convergence test. The step-length procedure is optional, but our experience demonstrates that this step improves the stability of the method and reduce the number of iterations. In this Section we concern with the computation of the search δ δ δ δ direction pkα or, more precisely, with the computation of the new iterate xk+1α = xkα + pkα. Certainly, if a δ step-length procedure is a part of the inversion algorithm, then xk+1α is the prospective iterate, but we prefer to use the term “new iterate” because it is frequently encountered in remote sensing community.

Using the explicit expressions of the augmented vector f and of its Jacobian Kf, we deduce that the Gauss- δ Newton step pkα solves the equation (cf. (8.56))

T T T δ δ T δ KkαKkα + αL L p = −Kkα F xkα − y − αL L xkα − xa δ with Kkα = K xkα . The new iterate then takes the form

δ δ δ † δ xk+1α = xkα + pkα = xa + Kkαyk, (8.68) where † T T −1 T Kkα = KkαKkα + αL L Kkα and δ δ δ δ yk = y − F xkα + Kkα xkα − xa (8.69) are the generalized inverse and the noisy data at the iteration step k, respectively. In order to give a more practical interpretation of the Gauss-Newton iterate (8.68), we consider a linearisation δ of F about xkα δ δ δ F (x) = F xkα + Kkα x − xkα + R x, xkα . If x† is a solution of the equation F (x) = y, then x† also solves the equation

† Kkα x − xa = yk, where δ δ † δ yk = y − F xkα + Kkα xkα − xa − R x , xkα is the exact data at the iteration step k. Because yk is unknown, we consider the equation

δ Kkα (x − xa) = yk (8.70)

δ with yk being given by (8.69). Evidently, the noise in the data is due to the measurement noise and the linearisation error and we have the representation

δ † δ yk − yk = δ + R x , xkα .

Because the non-linear problem is ill-posed, its linearisation is also ill-posed and we solve the linear equation (8.70) by means of Tikhonov regularization with a penalty term of the form Ω(x) = kL (x − xa)k. The Tikhonov function for the linear sub-problem takes the form

δ 2 2 Flk (x) = yk − Kkα (x − xa) + α kL (x − xa)k

δ † δ and as in (8.68) the new iterate is given by xk+1α = xa + Kkαyk. Thus, the solution of a non-linear ill–posed problem by means of Tikhonov regularization is equivalent to the solution of a sequence of ill–posed linearisation of the forward model about the current iterate.

SGP OL1b-2 ATBD Version 7 Page 100 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

The new iterate can be computed by using the GSVD of the matrix pair (Kkα, L). Although the GSVD is of great theoretical interest for analysing general-form regularization problems, it is of computational interest only for small- and medium-sized problems. The reason is that the computation of the GSVD of the matrix 2 3 pair (Kkα, L) is quite demanding; the conventional implementation requires about 2m n+15n operations. For practical solutions of large-scale regularization problems it is much simpler to deal with standard-from problems in which L = In. The regularization in standard form relies on the solution of the linear equation

¯ δ Kkα 4 x¯ = yk (8.71)

−1 with K¯ kα = KkαL and −1 4x = x − xa = L 4 x¯ T by means of Tikhonov regularization with L = In. Considering the SVD of the Jacobian matrix K¯ kα = UΣV , the solution of the standard-form problem

¯ T ¯ ¯ T δ KkαKkα + αIn 4 x¯ = Kkαyk (8.72) is given by n δ X σi T δ 4¯x = u y vi. k+1α σ2 + α i k i=1 i

An efﬁcient implementation of Tikhonov regularization for large-scale problems, which also take into account that we wish to solve (8.72) several times for various regularization parameters, is described below. In this approach the standard-form problem (8.72) is treated as a least-squares problem of the form

" # " # 2 K¯ yδ kα k √ 4 x¯ − . αIn 0

The matrix K¯ kα is transformed into an upper bidiagonal matrix J " # J T K¯ kα = U V 0 by means of orthogonal transformations from the left and from the right with U ∈ Rm×m, J ∈ Rn×n and V ∈ Rn×n. The orthogonal matrices U and V are represented by series of orthogonal transformations, which are usually stored in appropriate arrays and later used when matrix-vector multiplications, e.g. UT x and Vx, are needed. Making the changes of variables

ξ = VT 4 x¯ and " # zδ zδ UT yδ zδ 1 zδ ∈ n = k, = δ , 1 R , z2 we are led to an equivalent minimization problem involving the objective function

" # " # 2 J zδ 1 √ ξ − . αIn 0

The above minimization problem can be solved very efﬁciently by means of O (n) operations. Essentially, for each value of the regularization parameter we compute the QR factorization " # " # J Tα √ = Qα (8.73) αIn 0

n×n 2n×2n by means of 2n − 1 Givens rotations, where Tα ∈ R is an upper bidiagonal matrix and Qα ∈ R is a

SGP OL1b-2 ATBD Version 7 Page 101 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

product of Givens rotations. Further, deﬁning the vector " # zδ ζδ = QT 1 α α 0

and partitioning ζδ as α " # ζδ δ α1 δ ∈ n ζα = δ , ζα1 R , ζα2 we obtain δ −1 δ ξα = Tα ζα1 and ﬁnally, δ δ 4¯xk+1α = Vξα.

The standard-form problem (8.72) can be formulated as the normal equation

¯ T ¯ ¯ T ¯ Kf Kf4x¯ = Kf f (8.74) with " δ # " ¯ # ¯ yk ¯ Kkα f = , Kf = √ 0 αIn and iterative methods, as for instance the CGNE and the LSQR algorithms, can be used for computing the new iterate. For large-scale problems the computational efﬁciency can be increased by using an appropriate pre-conditioner. Note that for the normal equation (8.74) the pre-conditioner M should be chosen such that T ¯ T ¯ the condition number of M Kf KfM is small. A pre-conditioner constructed by using the close connection T between the Lanczos algorithm and the conjugate gradient method is usually used. If K¯ kα = UΣV is a singular value decomposition of the Jacobian matrix, then there holds h i ¯ T ¯ 2 T Kf Kf = V diag σi + α n×n V

and for a ﬁxed index r the preconditioned matrix can be constructed as   1 diag √ 0r×(n−r) σ2+α  i  T M = V  r×r  V . √1 0(n−r)×r diag α (n−r)×(n−r)

We then obtain   diag (1) 0r× n−r T T r×r ( ) T ¯ ¯ 2 M Kf KfM = V  σi +α  V 0(n−r)×r diag α (n−r)×(n−r) T T 2 2 and the condition number of M Kf KfM is 1 + σr+1/α. If σr+1 is not much larger than α, then the condition number is small and very few iterations are required to compute the new iterate. Turning now to practical implementation issues we mention that iterative algorithms are coded without explicit reference to M; only the matrix–vector product Mx is involved. Since

r ! 1 X 1 1 T Mx = √ x + − √ v x vi α p 2 α i i=1 σi + α

we observe that the calculation of Mx requires the knowledge of the ﬁrst r singular values and vectors of K¯ kα and these quantities can be efﬁciently computed by the Lanczos algorithm . The steps of computing the r singular values and the right singular vectors of a m × n matrix A can be summarized as follows: 1. apply r steps of the Lanczos bi-diagonalization algorithm with Householder orthogonalization to produce a lower (r + 1) × r bidiagonal matrix B, a n × r matrix V¯ containing the right singular vectors and a

SGP OL1b-2 ATBD Version 7 Page 102 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

m × (r + 1) matrix U¯ containing the left singular vectors

AV¯ = UB¯ ;

2. compute the QR factorization of the bidiagonal matrix B " # R B = Q , 0

where Q is a (r + 1) × (r + 1) unitary matrix and R is an upper r × r bidiagonal matrix; 3. compute the singular value decomposition of the bidiagonal matrix R

T R = URΣVR ;

4. the ﬁrst r singular values are the diagonal entries of Σ, while the corresponding right singular vectors are the columns of the n × r matrix V = VV¯ R.

The standard-form problem (8.72) can be also expressed as

A 4 x¯ = b (8.75)

with ¯ T ¯ A = KkαKkα + αIn and ¯ T δ b = Kkαyk. The symmetric system of equations (8.75) can now be solved by using standard iterative solvers, as for instance the Conjugate Gradient Squared (CGS) or the Bi-conjugate Gradient Stabilized (Bi-CGSTAB). A relevant practical aspect is that for iterative methods the matrix A is never formed explicitly as only matrix-vector products with A and eventually with AT are required. The calculation of the matrix-vector product Ax de- ¯ T ¯ ¯ T ¯ ¯ T ¯ mands the calculation of KkαKkαx and KkαKkαx should be computed as Kkα Kkαx and not by forming the ¯ T ¯ cross-product matrix KkαKkα. The reasons for avoiding explicit formation of the cross-product matrix is the loss of information due to round-off and the excessively large computational time. A right pre-conditioner for the system (8.75), i.e. 0 0 AMa 4 x¯ = b, Ma 4 x¯ = 4x¯ can also be constructed by using the Lanczos algorithm. The right pre-conditioner is given by

 1  diag 2 0r×(n−r) σi +α T Ma = V  r×r  V , 0 1 (n−r)×r diag α (n−r)×(n−r)

2 in which case the condition number of AMa is 1 + σr+1/α and r 1 X 1 1 T M x = x + − v x vi. a α σ2 + α α i i=1 i

8.3.1.4. Error characterisation

An important part of a retrieval is to assess the accuracy of the regularized solution by performing an error analysis. The most commonly used methods to calculate the errors in the non-linear case are based on a linearisation of the forward model and of the gradient of the Tikhonov function.

SGP OL1b-2 ATBD Version 7 Page 103 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Linearisation of the forward model function δ The Gauss–Newton iterate xk+1α is the regularized solution of the linear problem (8.70) and its expression is given by (8.68). For the exact data vector y the Gauss–Newton iterate possesses a similar representation

† xk+1α − xa = Kkαyk,

where, in order to avoid an abundance of notations, yk is now given by

yk = y − F (xkα) + K (xkα)(xkα − xa) .

As in the linear case we consider the decomposition

† δ † δ x − xk+1α = x − xk+1α + xk+1α − xk+1α (8.76)

δ and try to estimate each term in the right-hand side of (8.76). Using the linearisation about xkα and xkα

† † y = F (xkα) + K (xkα) x − xkα + R x , xkα

and δ † δ † δ y = F xkα + Kkα x − xkα + R x , xkα ,

respectively, and assuming that Kkα ≈ K (xkα), we express the ﬁrst term in (8.76) as

† † † † † † x − xk+1α = x − xa − Kkαyk = (In − Akα) x − xa − KkαR x , xkα and the second term as

δ † δ † † † δ † xk+1α − xk+1α = Kkα yk − yk = −Kkαδ − Kkα R x , xkα − R x , xkα with † Akα = KkαKkα being the averaging kernel. Inserting the above relations into (8.76) we ﬁnd that

† δ † † † † δ x − xk+1α = (In − Akα) x − xa − Kkαδ − KkαR x , xkα . (8.77)

δ δ Assuming that the sequence xkα converges and denoting by xα the limit of this sequence, we let k → ∞ in (8.77) and obtain δ δ δ eα = esα + enα + elα, (8.78) where δ † δ eα = x − xα is the error vector at the solution, † esα = (In − Aα) x − xa is the smoothing error vector, δ † enα = −Kαδ is the noise error vector and δ † † δ elα = −KαR x , xα † is the non-linearity error vector. Note that the generalized inverse Kα and the averaging kernel Aα, which δ enter in the expressions of the error components, are evaluated at xα. δ The expression of the total error can also be derived by using the fact that xα is a minimizer of the Tikhonov δ function. The stationary condition for F at xα

δ δ g xα = ∇F xα = 0

SGP OL1b-2 ATBD Version 7 Page 104 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

δ implies that xα satisﬁes the Euler equation

T δ δ T δ Kα F xα − y + αL L xα − xa = 0; (8.79)

δ whence considering the linearisation of F about xα

δ † δ † δ y = F xα + Kα x − xα + R x , xα , (8.80)

we ﬁnd that

T T † δ T † T T † δ Kα Kα + αL L x − xα = αL L x − xa − Kα δ − Kα R x , xα .

Further, using the identity

T T −1 T T T −1 T K K + αL L αL L = In − K K + αL L K K,

we obtain † δ † † † † δ x − xα = (In − Aα) x − xa − Kαδ − KαR x , xα , (8.81) which is the explicit form of (8.78). Thus, the error representations in the non-linear and the linear case are δ similar, excepting an additional term, which represents the non-linearity error. If the minimizer xα is sufﬁciently close to the exact solution x†, the non-linearity error can be neglected and the agreement is perfect. δ In a semi–stochastic framework we suppose that Kα is deterministic and, as a result, the error vector eα is 2 † †T stochastic with mean esα and covariance Cen = σ KαKα . As in the linear case, we may deﬁne the mean square error matrix

T Sα = esαesα + Cen † † T T 2 † †T = (In − Aα) x − xa x − xa (In − Aα) + σ KαKα (8.82)

δ † to quantify the dispersion of the regularized solution xα about the true solution x . The one-rank matrix † † T x − xa x − xa can be approximated by

† † T δ δ T x − xa x − xa ≈ xα − xa xα − xa (8.83) or by

2 T σ −1 x† − x x† − x ≈ LT L . (8.84) a a α

The approximation (8.83) yields the so called semi-stochastic representation of Sα, while the approximation (8.84) yields the stochastic representation of Sα, since in this case Sα coincides with the a posteriori covariance matrix in a statistical inversion. Accounting on all assumptions made, we deduce that a linearised error analysis can be performed when δ 1. the sequence of iterates xkα converges; δ † δ 2. the forward model can be linearised about xα in the sense that the non-linearity residual R x , xα is small; 3. the data error model is correct. If one of these assumptions is violated the error analysis is erroneous. The convergence of iterates can be guaranteed by using an appropriate termination criterion, but the second and third assumptions require more attention. The linearity assumption can be verified at the boundary of a confidence region for the solution. For this T purpose we consider a singular value decomposition of the means square error matrix Sα = VΣV and 1/2 define the normalized error patterns sk for Sα from the decomposition VΣ = [s1, ..., sn]. The linearisation error δ δ R (x) = F (x) − F xα + Kα x − xα

SGP OL1b-2 ATBD Version 7 Page 105 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18 can be estimated by comparing 1 δ 2 εlink = R x ± sk ≈ 1 mσ2 α for all k = 1, ..., n. The validity of the data error model assumption is perhaps the most important problem of an error analysis. If the data error δy contains only the noise error component δ and if δ is a Gaussian random vector with zero 2 δ 2 mean and covariance matrix Cδ = σ Im, then the square residual norm rα is Chi-squared distributed with 2 δ 2 variance σ and trace Im − Ab α degrees of freedom. In the limit α → 0 rα is Chi-squared distributed with variance σ2 and m − n degrees of freedom and it must holds

δ 2 δ δ 2 2 rα = F xα − y ≈ (m − n) σ , α → 0.

If the contribution of the systematic error δsys in the data error δy is signiﬁcant, we have instead δ δ 2 1 2 2 F x − y ≈ (m − n) δsys + σ , α → 0. α m

The presence of the systematic errors introduce an additional bias in the solution and to handle with this type of errors we may proceed as in the linear case, that is, we may replace the data error δy by an equivalent white noise δe such that n 2o n 2o E kδek = E δy .

The variance of the white noise δe is then given by

2 1 2 2 σ = δsys + σ e m and the estimate 2 1 δ δ 2 σ ≈ F x − y , α → 0 e m − n α 2 can be used to perform an error analysis with the equivalent white noise covariance matrix Cδe = σe Im. It is apparent that by this equivalence we increase the noise variance and eliminate the bias due to the systematic errors.

Linearisation of the gradient of the Tikhonov function The error representation (8.81) has been derived by assuming a linearisation of the forward model about the minimizer of the Tikhonov function. This derivation does not make use of the Gauss approximation of the Hessian, since the Euler equation (8.79) holds true for the Gauss-Newton method as well as for the Newton method. An alternative error representation can be derived for a general Hessian matrix by using the linearisation of the gradient of the objective function. Because the objective function depends on the data yδ we will indicate this dependence by writing F x, yδ in place of F (x).

† δ δ δ † Setting x = xα −4xα and using the representation y = y −δ we expand the gradient at x , y in a ﬁrst-order δ δ Taylor series about xα, y

∂F ∂F x†, y = xδ − 4xδ , yδ − δ ∂x ∂x α α ∂F ∂2F = xδ , yδ − xδ , yδ 4xδ ∂x α ∂x2 α α ∂2F − xδ , yδ δ + R x†, xδ . ∂x∂yδ α α At the minimum we have ∂F xδ , yδ = 0, (8.85) ∂x α

SGP OL1b-2 ATBD Version 7 Page 106 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

whence taking into account that

∂F x†, y = g x†, y = αLT L x† − x ∂x a and that ∂2F ∂2F xδ , yδ = G xδ , xδ , yδ = −KT xδ , ∂x2 α α ∂x∂yδ α α we obtain † δ −1 T † −1 T −1 † δ x − xα = αGα L L x − xa − Gα Kα δ − Gα R x , xα (8.86) δ with Gα = G xα . Thus, neglecting the non-linearity error, the smoothing and the noise errors take the forms

−1 T † esα = αGα L L x − xa

and δ −1 T enα = −Gα Kα δ, respectively. The representations (8.81) and (8.86) coincide only for a Gauss approximation of the Hessian and the difference between them stems from the different linearisation employed. The mean and the covariance δ matrix of the error vector eα are given by the smoothing error vector esα and the retrieval noise covariance matrix 2 −1 T −1 Cen = σ Gα Kα KαGα .

When the quasi–Newton method is used to compute the minimizer of Tikhonov function, the Hessian approximation can be used for error estimation. However, for the Gauss–Newton method, an additional computational step involving the calculation of the Hessian at the solution has to be performed. For this purpose we consider δ the Taylor expansion about the minimizer xα

δ 1 δ T δ F (x) ≈ F x + x − x Gα x − x , (8.87) α 2 α α where by deﬁnition the entries of the Hessian are given by

2 ∂ F δ [Gα]ij = xα (8.88) ∂xi∂xj with xi = [x]i. Equations (8.87) and (8.88) suggest that we may use ﬁnite differences for approximating Gα. δ th Setting for convenience x = xα and denoting by ∆xi the displacement in the i component of x, we may compute the diagonal elements by using (8.87)

F (xi + ∆xi) − F (xi) [Gα]ii = 2 2 (8.89) (∆xi)

and the off-diagonal elements from (8.88) with central differences

[Gα]ij = [F (xi + ∆xi, xj + ∆xj) − F (xi − ∆xi, xj + ∆xj) −F (xi + ∆xi, xj − ∆xj) + F (xi − ∆xi, xj − ∆xj)] / (4∆xi∆xj) . (8.90)

The calculation of the Hessian by using finite differences requires an adequate choice of the step sizes ∆xi. The difficulty associated with the selection of the step size stems from the fact that the objective function may varies slowly in some directions of the x-space and rapidly in other. The implemented iterative algorithm significantly improves the reliability of the Hessian matrix calculation and is based on the following result: if Gα T h 2 i is the exact Hessian and Gα = VΣV with Σ = diag σi n×n is a singular value decomposition of Gα, then the linear transformation x = VΣ−1/2z (8.91)

SGP OL1b-2 ATBD Version 7 Page 107 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

implies that in the z-space the surfaces of constant F are spheres, i.e. 1 F (z) − F (z ) = (z − z )T (z − z ) . (8.92) 0 2 0 0 The computation of the pseudo-Hessian by ﬁnite differences is performed in the z-space by using (8.89) and (8.90) and this process is more stable than a Hessian calculation in the x-space. The step sizes ∆zi are chosen such that, when computing the diagonal elements, the variations F (zi + ∆zi) − F (zi) are approximately one.

8.3.1.5. Parameter choice methods

The choice of the regularization parameter plays an important role in computing a reliable approximation of the solution. In this section we present selection criteria with variable and constant regularization parameters. In the ﬁrst case the regularization parameter is estimated at each iteration step, while in the second case the minimization of the Tikhonov function is done a few times with different regularization parameters.

A priori parameter choice methods The expected error estimation can be formulated as an a priori parameter choice method. The idea is to perform a random exploration of a domain to which the solution is suppose to lie and for each state vector † n δ 2o realization xi to compute the optimal regularization parameter for error estimation αopti = arg minα E eαi p¯ and the exponent pi = ln αopti/ ln σ. The regularization parameter is then chosen as αe = σ , where p¯ = PN (1/N) i=1 pi is the sample mean exponent and N is the number of events. The error estimation method can be formulated for non-linear problems by representing the expected error at the solution as n δ 2o 2 n δ 2o E eα = kesαk + E enα with n † X α † T −1 e α = (In − Aα) x − x = x − x w wi (8.93) s a γ2 + α a i i=1 i and n 2 2 n δ 2o 2 † †T 2 X γi 1 2 E enα = σ trace KαKα = σ 2 kwik . (8.94) γ + α σi i=1 i th In (8.93) and (8.94) γi are the generalized singular values of the matrix pair (Kα, L) , wi is the i column −1 th −1 of the non-singular matrix W and wi is the i line of the inverse matrix W . Because the Jacobian matrix Kα is evaluated at the solution, a non-linear minimization method has to be employed for computing n δ 2o αopt = arg minα E eα for each state vector realization. The resulting algorithm is extremely computational expensive and in order to alleviate this drawback we approximate the Jacobian matrix at the solution by the Jacobian matrix at the a priori state. This is a realistic assumption because many ill-posed problems arising in atmospheric remote sensing are nearly linear (with an appropriate choice of the forward model their degree of non-linearity is of moderate size). The a priori parameter choice method is then equivalent to the expected error estimation method applied to a linearisation of the forward model about the a priori state. Another version of the expected error estimation method can be designed by assuming a semi-stochastic error representation and by using an iterative process for minimizing the expected error. Two relevant features reduce the performances of the so called iterated expected error estimation method: 1. The semi-stochastic error representation is valid if the Tikhonov solution is an acceptable approximation of the exact solution, that is, if the regularization parameter is in the neighbourhood of the optimal value of the regularization parameter. 2. The minimizer of the expected error is in general larger than the optimal value of the regularization parameter.

SGP OL1b-2 ATBD Version 7 Page 108 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Selection criteria with variable regularization parameters Since the solution of a non-linear ill-posed problem by means of Tikhonov regularization is equivalent to the solution of a sequence of ill-posed linearisation of the forward model about the current iterate, parameter choice methods for the linear problem (8.70) can be used to compute the regularization parameter at each iteration step. The noise in the right–hand side of the linearised equation (8.70) is due to the measurement noise in the data and the linearisation error. At the iteration step k the noise level can be estimated by the minimum value of the δ linearised residual corresponding to the smallest singular value rlkmin . In the framework of the discrepancy principle the regularization parameter is then selected as the solution of the equation

δ 2 δ 2 rlkα = τ rlkmin with τ > 1. Due to the difﬁculties associated with the noise level estimation, error-free parameter choice methods (based only on information about the noisy data) are more attractive. For the linearised equation (8.70), the following parameter choice methods can be considered: 1. the generalized cross-validation δ αgcvk = arg min υkα α with 2 rδ l,kα υδ = , (8.95) kα h i2 trace Im − Ab kα

2. the maximum likelihood estimation δ αmlek = arg min λkα α with δT δ y Im − Ab kα y δ k k λkα = r , (8.96) m det Im − Ab kα

3. the L-curve method δ αlck = arg max κ kα α lc with 00 0 0 00 δ xk (α) yk (α) − xk (α) yk (α) κlckα = 3/2 h 0 2 0 2i (xk (α)) + (yk (α)) and δ 2 δ 2 xk (α) = log rlkα , yk (α) = log ckα .

In the above relations the linearised residual and the constrained vector are given by

δ δ δ δ rlkα = yk − Kkα xk+1α − xa = Im − Ab kα yk and δ δ ckα = L xk+1α − xa respectively. In practice the following recommendations for choosing the regularization parameter has to be taken into account: 1. at the beginning of the iterations large α-values should be used to avoid local minima and to get well- conditioned least squares problems to solve;

SGP OL1b-2 ATBD Version 7 Page 109 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

2. during the iterations the regularization parameter should be decreased slowly to achieve a stable solution.

Numerical experiments have shown that a brutal use of the regularization parameter αoptk computed by one of the above parameter selection criteria may lead to an oscillation sequence of α-values. A heuristic formula that deals with this problem can be formulated as follows: the regularization parameter is the weighted sum between the previous α-value and the proposed regularization parameter αoptk ( ξαk−1 + (1 − ξ) αoptk, αoptk < αk−1 αk = . αk−1, αoptk ≥ αk−1

This selection rule guarantees a descending sequence of regularization parameters and the resulting method is very similar to the iteratively regularized Gauss-Newton method.

Selection criteria with constant regularization parameters The numerical realization of these parameter choice methods requires to solve the non-linear minimization problem several times for different regularization parameters. Each minimization is solved with a regulariza- δ tion parameter α and a solution xα is obtained. If the solution is satisfactory as judged by these selection criteria, then the inverse problem is considered to have been solved. The discrete values of the regularization pi parameters are chosen as αi = σ with pi > 0. In the framework of the discrepancy principle the regularization parameter is the solution of the equation

δ δ 2 2 y − F xα = τ∆ (8.97)

with τ > 1. Because of non-linear problems, the discrepancy principle equation has only a solution under very strong restrictive assumptions, we use a simpliﬁed version of this selection criterion: if the sequence {αi} is in descending order, we choose the largest regularization parameter αi? for which the residual norm is below the noise level, that is, 2 2 yδ − F xδ ≤ 2 yδ − F xδ ≤ ? αi? τ∆ < αi , 0 i < i .

The generalized discrepancy principle can also be formulated as an a posteriori parameter choice method for the non-linear Tikhonov regularization. A heuristics justiﬁcation of this parameter choice method can be given in a deterministic setting by using the error norm estimate

δ 2 2 δ 2 eα ≤ 2 kesαk + enα

together with the noise error bound 2 δ 2 2τ∆ e < , τ > 1. nα α

To estimate the smoothing error we assume L = In and consider the unperturbed solution xα corresponding to the exact data y. The stationary condition for the Tikhonov function at xα yields

T Kα [F (xα) − y] + αIn (xα − xa) = 0 (8.98) with Kα = K (xα). We then obtain

† † † † x − xα = (In − Aα) x − xa − KαR x , xα

and further † esα = (In − Aα) x − xa . Taking into account that for any x there holds

n 2 n 2 X α T 2 X T 2 2 k(In − Aα) xk = v x ≤ v x = kxk , σ2 + α i i i=1 i i=1 we deduce that an upper bound for the error norm is given by

SGP OL1b-2 ATBD Version 7 Page 110 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

2 1 † 2 ∆ M (α) = 4 x − xα + τ . 2 α To derive the necessary condition for a minimum of the estimate M (α) we consider the function

1 † 2 f (α) = x − xα (8.99) 2 and ﬁnd that df † T dxα (α) = − x − xα . (8.100) dα dα Formal differentiation of the Euler equation (8.98) with respect to α yields

T dKα T dxα dxα [F (xα) − y] + K Kα + α = − (xα − x ) (8.101) dα α dα dα a and the neglect of the ﬁrst term in (8.101) gives

dxα T −1 1 † ≈ − K Kα + αIn (xα − x ) = K [F (xα) − y] . dα α a α α The linearisation † y ≈ F (xα) + Kα x − xα and the matrix identity T −1 T T T −1 Kα Kα + αIn Kα = Kα KαKα + αIm then yield

df 1 † T † (α) ≈ x − xα K [y − F (xα)] dα α α 1 † T T −1 = Kα x − xα KαK + αIm [y − F (xα)] α α 1 T T −1 ≈ [y − F (xα)] KαK + αIm [y − F (xα)] . α α

δ δ If we now replace xα by xα and y by y we obtain the generalized discrepancy principle equation in the form

δ δ T T −1 δ δ 2 α y − F xα KαKα + αIm y − F xα = τ∆ .

Error-free methods with ﬁxed regularization parameter are natural extensions of the corresponding criteria for linear problems and the most popular are the generalized cross validation, the maximum likelihood estimation and the non-linear L-curve method. To formulate the generalized cross validation and the maximum likelihood estimation we employ some heuristic δ arguments. At the iteration step k the generalized cross validation function υkα and the maximum likelihood δ function λkα, as given by (8.95) and (8.96), respectively, depend on the inﬂuence matrix Ab kα, the linearised residual δ δ δ rlkα = yk − Kkα xk+1α − xa , and the noisy data vector δ δ δ δ yk = y − F xkα + Kkα xkα − xa .

δ δ Assuming that the iterates xkα converge to xα, then Ab kα converges to the inﬂuence matrix at the solution † δ Ab α = KαKα, rlkα to the non-linear residual

δ δ δ rα = y − F xα

δ and yk to δ δ δ δ yα = y − F xα + Kα xα − xa .

SGP OL1b-2 ATBD Version 7 Page 111 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Thus, in the limit k → ∞ the generalized cross validation function takes the form

δ δ 2 y − F x υδ = α α h i2 trace Im − Ab α and the maximum likelihood function read as δT δ m 2 yα Im − Ab α yα P T δ 2 i=1 ui yα / γi + α λα = = , r mpQm 2 m i=1 1/ (γi + α) det Im − Ab α

where (γi; wi, ui, vi) is a generalized singular system of the matrix pair (Kα, L).

δ 2 δ 2 The non-linear L-curve is the plot of the constraint norm cα = L xα − xa against the residual norm δ 2 δ δ 2 rα = y − F xα for a range of values of regularization parameter α. This curve is monotonically decreasing and convex. In a computational sense the non-linear L-curve consists of a number of discrete points corresponding to the different values of the regularization parameter and, in practice, the following techniques can be used for choosing the regularization parameter: 1. As for linear iterative regularization methods, we ﬁt a cubic spline curve to the discrete points of the δ δ L-curve (x (αi) , y (αi)) with x (α) = 2 log rα and y (α) = 2 log cα , and determine the point on the original discrete curve that is closest to the spline curve’s corner. 2. In the framework of the minimum distance function approach we compute

2 αlc = arg min d (αi) i with 2 2 2 d (α) = [x (α) − x0] + [y (α) − y0] ,

x0 = mini x (αi) and y0 = mini y (αi). 3. Relying on the deﬁnition of the corner of the L-curve as given by Reginska, we determine the regularization parameter as αlc = arg min {x (αi) + y (αi)} , i that is, we detect the minimum of the logarithmic L-curve rotated by π/4 radians.

8.4. Relation Number Density and VMR

The following deﬁnitions are used for the Limb MDS

Number of elements of the retrieval grid: nmain (= number of proﬁle entries and number of layers) State vector: x

Element index per layer: k = 1 . . . nmain

State vector elements assigned to partial columns per layer: xk, k = 1 . . . nmain (vertical columns per layer = partial columns per layer) Retrieval grid: ◦ Ordering: top to down ◦ Top of atmosphere is ﬁxed to 100 km and used for the height grid ◦ Information at TOA is used for height and pressure grid

◦ Height at the lower boundary of each layer k : zk

SGP OL1b-2 ATBD Version 7 Page 112 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

◦ Pressure at the lower boundary of each layer k : pk

◦ Altitude grid element: ∆zk = zk−1 − zk with z0 = 100 km

◦ Pressure grid element: ∆pk = pk−1 − pk

◦ Pressure is read from climatology with p0 at top of atmosphere

Number density per layer: n with nk, k = 1 . . . nmain

Volume mixing ratio per layer: vmr with vmrk , k = 1 . . . nmain

Since it is very often desirable to represent the retrieved proﬁle in number density or volume mixing ratio, the relations with respect to the chosen state vector representation (partial columns per layer) are given in the following. Note that the conversion is related to quantities which describe the result for a given proﬁle layer. Addi- tionally, note that the information provided for pressure and height is implicitly extended by the entries at TOA which are not given in the product.Because the retrieved quantities are the partial columns it is important to relate the partial columns to the number density and VMR. Using molec x 10−3cm · 11 k nk 3 = 2.6868 10 . cm 4zk [km] and −3 xk 10 cm vmrk [.] = 6 , 0.789087 · 10 · 4pk [mb]

we express nk and vmrk as nd nk = sk xk and vmr vmrk = sk xk, respectively, where

11 nd 2.6868 · 10 sk = , 4zk [km]

vmr 1 sk = 6 . 0.789087 · 10 · 4pk [mb]

If nk, vmrk and xk are the entries of the number density vector n, the VMR vector vmr, and the state vector x, respectively, we express the above relations in matrix form as:

n = Sndx, vmr = Svmrx. nd vmr Here, Snd and Svmr are diagonal matrices deﬁned by [Snd]ij = si · δij and [Svmr]ij = si · δij. The conversion can also be applied to the representation of the covariance matrix, the correlation matrix, the relative error and ﬁnally the averaging kernel. The computed covariance matrix is in fact the means square error matrix in a semi-stochastic framework, i.e.,

T † † T T 2 † †T C(x) = E{xx } = (In − A) x − xa x − xa (In − A) + σ K K ,

SGP OL1b-2 ATBD Version 7 Page 113 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

† † T where the one-rank matrix (x − xa)(x − xa) is approximated by

† † T T x − xa x − xa ≈ (x − xa)(x − xa) .

The covariance matrix C(x) can be then converted into number density representation by

C(n) = SndC(x)Snd and into VMR representation by

C(vmr) = SvmrC(x)Svmr.

The relative error i at the solution is deﬁned by

p p p [C(x)]ii [C(n)]ii [C(vmr)]ii i = · 100 = · 100 = · 100, xi ni vmri while the off-diagonal elements of the correlation matrix of the ﬁt are given by

[C(x)]ij [C(n)]ij [C(vmr)]ij ρij = p p = p p = p p [C(x)]ii · [C(x)]jj [C(n)]ii · [C(n)]jj [C(vmr)]ii · [C(vmr)]jj with i, j = 1 . . . nmain and i =6 j. † † If xj = [x ]j represents the “true” proﬁle in partial columns on layer j, then the averaging kernel is deﬁned by

∂xi [A(x)]ij = † ∂xj The averaging kernels for the number density and VMR are then given by

nd ∂ni si [A(n)]ij = † = nd [A(x)]ij, ∂nj sj and vmr ∂vmri si [A(vmr)]ij = † = vmr [A(x)]ij, ∂vmrj sj respectively.

8.5. Common Characteristics of the Proﬁle retrieval

For all profile retrievals the results of the Limb cloud detection are taken into account. In order to avoid cloud contamination, the first measurement data used for the profile retrieval is taken at the first cloud free height above the configured minimum height, i.e.

( hmin for max(hcloud) < hmin h0 = (8.102) hi+1 for max(hcloud) > hmin with i = Index of max(hcloud)

Retrievals below 20 km and above the reference height are expected to have larger errors.

SGP OL1b-2 ATBD Version 7 Page 114 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18 8.6. Limb Ozone Proﬁle Retrieval Settings

Input spectra Calibration All calibrations applied Retrieval Height Range 13.5 km (or lowest cloud free height) - 65 km Reference Height 46 km (for 520 - 590 nm); 65 km (for 283 - 310 nm) Fit Settings Fitting Interval 283 - 310 nm and 520 - 590 nm Polynomial Degree 4th order Number of Layers 33 Aerosol Model LOWTRAN Absorption Cross Sections/Fitted Curves

O3 Bogumil et al.(2003)@243 K

NO2 Bogumil et al.(2003)@243 K Proﬁles

O3 McLinden et al.(2002)

NO2 McLinden et al.(2002)

8.7. Limb NO2 Proﬁle Retrieval Settings

Input spectra Calibration All calibrations applied Retrieval Height Range 13.5 km (or lowest cloud free height) - 46 km Reference Height 43 km Fit Settings Fitting Interval 420 - 470 nm Polynomial Degree 3rd order Number of Layers 33 Absorption Cross Sections/Fitted Curves

O3 Bogumil et al.(2003)@243 K

NO2 Bogumil et al.(2003)@243 K Proﬁles

O3 McLinden et al.(2002)

NO2 McLinden et al.(2002)

SGP OL1b-2 ATBD Version 7 Page 115 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18 8.8. Limb BrO Proﬁle Retrieval Settings

Input spectra Calibration All calibrations applied Retrieval Height Range 10.5 km (or lowest cloud free height) - 35 km Reference Height 35 km Fit Settings Fitting Interval 337 - 357 nm Polynomial Degree 4th order Number of Layers 33 Absorption Cross Sections/Fitted Curves

O3 Bogumil et al.(2003)@243 K BrO Bogumil et al.(2003)@243 K Proﬁles

O3 McLinden et al.(2002) BrO McLinden et al.(2002)

SGP OL1b-2 ATBD Version 7 Page 116 of 137 9. Limb Cloud Retrieval

9.1. Background

9.1.1. Motivation

In limb mode SCIAMACHY measures light scattered along the line-of-sight. If the line-of-sight intersects a cloud at a certain height, the spectrally resolved measurements differ from cloud free measurements. To eliminate systematic uncertainties and to enhance the sensitivity of the cloud determination, differences and/or ratios of spectral measurements in a certain wavelength region are used. Here we use radiance ratios at 750 nm and 1090 nm (and of 1550 nm and 1685 nm, see Section 9.3) for the detection of clouds. In an ideal Rayleigh scattering atmosphere, populated only by molecules, the difference between radiances at two wavelengths about 300 nm apart is large (I ∼ λ−4), while for larger particles like cloud droplets this difference is reduced (e.g., I ∼ λ−1, Section 9.1.2). The basic geometry and the principle of the SCIAMACHY cloud top height detection from limb measurements is shown in Figure 9.1. The satellite instrument detects different light from the cloud and above, which is then derived as the cloud top height (CTH) above the tangent point.

9.1.2. Theory

In earth’s atmosphere radiation is scattered non-isotropically depending on the wavelength of the incident radiation, the atmospheric density as well as on the size and shape of the scattering particles. The distribution of the intensity I of scattered radiation is described by Rayleigh scattering: wavelength particle size (molecules like nitrogen or oxygen) and/or Mie scattering: wavelength particle size (aerosol, cloud particles as can be found in tropospheric clouds and polar stratospheric clouds (PSCs)) In the Rayleigh scattering case we ﬁnd I ∼ λ4, whereas in the case of Mie scattering I is proportional to λ−α, with a value of α varying from 0 to 4. Due to the dependence of the spectral signature on the size of the scattering particles it is possible to derive information about the physical properties of clouds from comparisons of radiation at different wavelengths.

Figure 9.1.: Illustration of the principle of the cloud top height detection using scattered solar radiation at clouds for two wavelengths in the near infra-red (red and brown darts). SCIAMACHY scans the limb in tangent height steps with a difference ∆TH=3.3 km. The cloud top height CTH is calculated for the tangent height (the vector being perpendicular to the ground and the line-of-sight).

117 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

9.1.3. Product description

Using this method a detection tropospheric clouds (i.e. stratus, cumulus and cirrus), cirrus clouds at the tropical tropopause, stratospheric clouds in polar winters and noctilucent clouds near the polar summer mesopause is straightforward. While for every cloud type the detection is based on the same physical principles, each cloud type detection requires speciﬁc details. For example, detecting PSCs appropriately requires knowledge about the stratospheric aerosol content to avoid erroneous detection due to high levels of aerosol.

9.2. Algorithm Description

9.2.1. Physical Considerations

As cloud particles (diameter between 2 µm to 1.5 mm) are larger than the wavelength used in the UV-to-IR measurement range of SCIAMACHY (around 0.8 µm), scattered radiation tends to be scattered not uniformly in all directions, as is the case for the much smaller background air molecules (~0.1 nm). In general, the phase function of scattered radiation from water cloud droplets has a strong forward peak. On the other hand cloud droplets scatter radiation at different wavelengths more uniformly than molecules. Furthermore molecules absorb radiation in certain wavelength region. To identify wavelength regions, which are suitable for radiance comparisons to detect clouds, certain requirements have to be fulfilled 1. Radiation at wavelengths below 400 nm is not suitable because the atmosphere gets optically thick for Rayleigh scattering in the upper troposphere at about 15-18 km; 2. Spectral windows with strong molecular absorption like for ozone, oxygen or water vapour are not suitable because of the height dependence of the absorption profiles. For the identification (‘flagging’) of cloud contaminated pixels in the SCIAMACHY limb data and the retrieval of cloud top heights we choose two wavelength pairs of 750 nm and 1090 nm, and of 1550 nm and 1685 nm. The ratio of 750 nm and 1090 nm is used for the general detection of clouds. Radiance at 750 nm (SCIAMACHY channel 4) is basically free of absorption signatures, as ozone absorption is comparatively small (see Figure 9.2). It is situated between the H2O absorption band at 725 nm and the O2-A band at 760 nm. The 1090 nm radiance (SCIAMACHY channel 6) is just outside a H2O absorption band ranging from 1100 nm to 1170 nm. The wavelength pair of 1550 nm and 1685 nm is chosen for the detection of ice clouds, as the spectral signatures of ice is more pronounced in this wavelength region Kokhanovsky et al.(2005). The imaginary part of the refractive index is shown in Figure 9.3 taken from Acarreta et al.(2004). The 1550 nm region is outside the strong water vapour absorption band around 1400 nm.

SGP OL1b-2 ATBD Version 7 Page 118 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Figure 9.2.: Calibrated Limb spectra for the SCIAMACHY channel 4 (top) and channel 6 (bottom) for an example measurement (von Savigny et al.(2005)).

Figure 9.3.: Taken from (Acarreta et al.(2004))and updated. The blue lines give wavelengths of SCIAMACHY channel 6+ suitable for ice cloud detection. The red dots indicate a spectral behaviour of a pure Rayleigh scattering atmosphere, showing the strong decrease of radiance.

9.2.2. Mathematical Description of the Algorithm

To improve the signal-to-noise ratio SCIAMACHY measurements are integrated over the following wavelength windows: 750 – 751 nm, 1088 – 1092 nm, 1550 – 1553.2 nm and 1683 – 1687 nm. For the detection of cloudy pixels the height dependent colour index Rc is created from the ratio of the wavelength dependent intensities

SGP OL1b-2 ATBD Version 7 Page 119 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

I(λ1,TH) Rc(TH) = (9.1) I(λ2,TH)

TH denotes the tangent height for the SCIAMACHY line-of-sight, λ the wavelength, and I the intensity. For the determination of the cloud top, the colour index ratio proﬁle is computed as follows Starting with the lowest height, pairs of two adjacent colour indices are divided. The maximum of the colour index ratio will then show the cloud top. The colour index ratio Θ(TH) is deﬁned as

R (TH) Θ(TH) = c (9.2) Rc(TH + ∆TH)

For SCIAMACHY limb measurements the tangent height difference ∆TH is 3.3 km, which also gives the vertical resolution. Taking this difference into account, a limb measurement at e.g. 20 km tangent height has a horizontal resolution along track of about 400 km and due to the scan cycle of 240 km across track.

9.3. Application and Determination of Cloud Types

9.3.1. Tropospheric Clouds

The main reason for flagging cloudy pixels is to support a successful trace gas retrieval. A fully clouded scene at high altitudes will influence the quality of the retrieved trace gas profiles. For absorbers peaking in the troposphere like water vapour, cloudy scenes will foil a reliable retrieval. Thus it is mandatory to know if there is cloud contamination in the line-of-sight and what the cloud top height is.

Figure 9.4.: Colour index proﬁles of SCIAMACHY Limb radiances (left) and the corresponding colour index ratio proﬁles (right).

An example for colour index proﬁles is shown in Figure 9.4 for a measurement in the tropics at 9.2◦ N and 83.7◦ E on 1 July 2005. The depicted lines in the altitude range from 0 to 30 km show different ratios 1090/750 nm (black line), 1685/1550 nm (red line). The ratios 1550/750 nm (blue line) and 1685/750 nm (green line) are shown only for testing purposes. These ratios are redundant and are not used for the detection of clouds.

SGP OL1b-2 ATBD Version 7 Page 120 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

The near infra-red colour ratios may be used for the distinction of water and ice clouds. To have the same starting value at the bottom, the radiance profiles are divided by the lowermost radiance measured by SCIA- MACHY. The right part of Figure 9.4 shows the colour index ratios for the same measurement. The cloud top can be found at the maximum value at a height of about 14.5 km. In Figure 9.5 the colour index ratios (CIR) along an entire SCIAMACHY orbit are shown. To flag a fully cloudy scene in one CIR profile a value of Θ = 2 is taken (solid circles), while partly cloudy scenes (black rings) are assigned for values between 1.4 and 2. CIR below the threshold of 1.4 are considered as cloud free (depicted as full circles at 0 km altitude). The values are empirical and still preliminary. They will be verified using a larger dataset in the frame of an ongoing verification. In the upper part of the figure the cloud free region in the tropics shows slightly elevated values at 20 km, which could hint to cirrus clouds. This also is supported by the ice cloud detection as shown in the lower plot, where clouds are detected at these altitudes. The NIR gradients tend to have outliers at the upper boundary possibly due to small signal to noise ratios.

Figure 9.5.: Colour index ratios for a SCIAMACHY orbit as a function of latitude and tangent height. The circles depict the cloud top heights. Full cloud coverage is given by solid circles above the ground, while partially cloudy scenes are illustrated by the black rings in the upper part of the plot. In the lower part solid circles depict ice phase clouds.

9.3.2. Polar Stratospheric Clouds (PSCs)

While general tropospheric clouds are easy to detect, more sophisticated detection criteria are needed for PSCs (see Figure 9.4) to reduce false results. Currently the stratospheric aerosol content is low due to the absence of volcanic eruptions for longer time periods. The wavelength ratio of 750 nm and 1090 nm is used for PSC detection. To distinguish between an aerosol loaded stratosphere and an optically thin PSC, a threshold of the colour index ratio of Θ(TH) > 1.35(for the northern hemisphere); > 1.3 (for the southern one) was chosen. This value is a result of model studies of von Savigny et al.(2005). High altitude cirrus clouds can also be confused with PSCs, so as a further constraint, Θ(TH) should be taken from measurements above the climatological tropopause. A simple constraint is to choose a tangent height between 15 and 30 km and to look for PSCs only above 50° latitude.

SGP OL1b-2 ATBD Version 7 Page 121 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Figure 9.6.: Colour index proﬁle (a) and colour index ratios (b) in case of a PSC contaminated measurement (blue) and a background proﬁle (black).

9.3.3. Noctilucent Clouds (NLCs)

Noctilucent clouds (NLCs) are a mesospheric phenomenon (80 and 85 km), occurring at high latitudes during summertime, when the mesopause region is very cold. A typical season starts about 3–4 weeks before the summer solstice and persists for approximately 3 months (Robert et al.(2009)). These clouds can frequently be detected from ground. UV-radiation at 265 nm and 291 nm is used for the detection of clouds at these heights, because multiple scattering and reﬂection from the ground can be neglected for the limb scattering geometry. Two mechanisms have been implemented to establish an NLC detection method. 1) For stronger NLCs differencing is a feasible way for detection:

ΘD(TH) = I(TH) − I(TH + 4TH) (9.3)

A negative difference (Eq.9.3 ) at tangent heights between 75 km and 90 km indicates NLCs in the line-of-sight (see Fig. 9.7). 2) A division of the radiance is used to get the colour ratio for weaker NLCs:

R (TH) Θ(TH) = c (9.4) Rc(TH + ∆TH)

If both wavelengths give ratios Θ(TH) > 3, NLCs are detected (von Savigny et al.(2004)). This is for weak NLCs that do not have a peak in radiance at these altitudes, but only a bump.

SGP OL1b-2 ATBD Version 7 Page 122 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Figure 9.7.: Detection of NLC due to the increase in backscattered limb radiance above 80 km (purple) and a background measurement (red).

9.4. Summary and Implementation

Cloud detection can substantially influence the limb results, if a limb profile is sensitive to cloud contamination with respect to the spectral measurement. A possible drawback could be that results may be biased towards an atmosphere that is too dry. This is because clouds need moisture to evolve. This moisture gets transported in higher altitudes. If cloudy scenes are removed the resulting picture of the atmosphere represents a too dry environment. On average the altitude coverage of profiles may change because, due to cloud, indexing profiles from lower altitudes may be skipped as they are flagged cloud contaminated. On the other hand, a cloud index may improve results significantly by sorting out cloud contaminated scenes. A cloud index in principle also inherits the possibility to distinguish between different cloud types. Currently it has not been investigated what influence the coverage of the field of view (FOV) of SCIAMACHY has (i.e. partial coverage, horizontal coverage, vertical coverage). We recommend to carefully choose the wavelength region for setting up the cloud index. The wavelength regions shown in this ATBD are not the only wavelengths where a detection of clouds is possible but they have proven to work. A final determination of the quality of the detection with this set up is currently work in progress. We generally recommend using a cloud index, because sorting out obstacles from the FOV is essential for limb sounding.

9.4.1. Exception Rules

As the SCIAMACHY data is not perfect, especially in channel 6 at wavelengths near 1685 nm , it is necessary to sort out unexpected, non physical radiances. Thus a few exception rules were implemented to avoid extreme colour index ratios due to high or negative radiances.

SGP OL1b-2 ATBD Version 7 Page 123 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

1. As it very unusual to have negative radiances in the three wavelength bands (750 nm, 1090 nm, 1550 nm), the retrieval is stopped, if this shall happen. 2. For the band 1685 nm two tests are made:

a) The wavelengths where negative radiances occur at any tangent height below the maximum height of 35 km are not used for the calculation of the mean. b) If unusually high radiances were found within the range, they will also be excluded. This is done by calculating the mean and the standard deviation of the sum of the radiances at the tangent heights up to the maximum height.

SGP OL1b-2 ATBD Version 7 Page 124 of 137 Part IV.

Tropospheric Products Algorithms

125 10. Tropospheric NO2

10.1. Motivation

Tropospheric NO2 is an air pollutant negatively affecting ecosystems and human health. The main anthropogenic sources of NO2 in the troposphere are fossil fuel combustion and biomass burning. There are natural sources as well. These are microbial production in soils, wildﬁres and lightning.

10.2. Retrieval Settings

Nadir Settings see Section 5.4.2 Limb Settings see Section 8.7 Limb-Nadir Matching Settings Tropopause height ECMWF ERA-Interim re-analysis Cloud limits radiance cloud fraction < 50% (corresponds to effective CF < 0.2) Background Ref. Sector 180 – 220◦ (Paciﬁc) AMF Settings

Stratospheric LIDORT (HALOE NO2 Proﬁles Climatology) Tropospheric Look-up table of the box AMFs used in the operational processing of the GOME-2 data Tropospheric NO2 proﬁles from the MOZART CTM

The retrieval of tropospheric NO2 columns from SCIAMACHY measurements is performed in several steps making use of various SCIAMACHY products (NO2 nadir slant columns, NO2 limb profiles) as well as several external data sets needed for the calculation of tropospheric air mass factors. Additional normalization is necessary using a background data base of values over the Pacific reference sector. As NO2 cross-sections depend on temperature and in the total column retrieval temperatures in the stratosphere are assumed, a correction needs to be applied for AMFtropospheric calculation accounting for differing temperature in the troposphere. In principle, the retrieval approach is simple. Nadir observations provide columns integrated over the full atmosphere down to the ground (note that only almost cloudless nadir pixels - CF < 0.2 - are processed). Limb measurements on the other hand provide a stratospheric profile, which can be integrated to the stratospheric column. The difference of the two measurements provides the tropospheric column. In practice, the subtraction of the stratospheric contribution has to be performed on the slant columns in order to account for the different sensitivity of the nadir measurements in different altitudes. Therefore, the limb profiles are integrated and for each nadir pixel converted to the stratospheric slant columns using AMFstratospheric accounting for the given viewing conditions (SZA, LOS, relative azimuth). Since many more nadir measurements than limb profiles are performed within one state, the limb data need to be interpolated to the position of the nadir pixels. This is

126 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

done by first computing all limb columns for a complete orbit, then interpolating them in latitude for each of the 4 viewing angles separately. This yields four VCstrat values for every nadir state. These values are then linearly interpolated and extrapolated in viewing angle to the respective nadir position. A further complication arises from small systematic differences between limb and nadir columns, which need to be corrected for. This is done by computing the average nadir and limb slant columns over the Pacific reference sector at all latitudes and correcting the limb data with the difference between the two values. The overall retrieval for each measurement can be broken down in the following steps: 1. Computation of latitudinal dependent offset between limb and nadir slant columns 2. Computation of stratospheric nadir slant column using offset-corrected limb profiles 3. Computation of tropospheric slant column using nadir slant columns and calculated stratospheric slant columns

4. Addition of a tropospheric background to account for tropospheric NO2 in the Paciﬁc reference sector

5. Computation of tropospheric vertical column applying AMFtropospheric The schematic chart visualizing the retrieval steps is depicted in Figure 10.1. L2 input is colored green, databases blue, intermediate and ﬁnal results yellow and processing steps red.

SGP OL1b-2 ATBD Version 7 Page 127 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Figure 10.1.: Flow chart of the tropospheric NO2 retrieval algorithm.

SGP OL1b-2 ATBD Version 7 Page 128 of 137 11. Tropospheric BrO

11.1. Motivation

Bromine monoxide plays a signiﬁcant role in the tropospheric chemistry, especially in the polar regions. It was discovered that during spring time large amounts of bromine species are released in the boundary layer of both polar regions. This phenomenon was named “polar bromine explosion”. So far it is known that bromine is emitted from sea salt during a number of photochemical and heterogeneous reactions. This mechanism, however, is still not completely understood, in particular roles of wind strength or sea ice age. But it is absolutely evident that this phenomenon leads to severe ozone depletion in the polar troposphere. On a smaller scale tropospheric bromine emissions have been observed also over salt lakes, in the marine boundary layer and in volcanic plumes. The tropospheric BrO retrieved from the SCIAMACHY measurements provides a very useful data set, which will undoubtedly be helpful in further studies of polar bromine explosions. It will also improve our knowledge about the existence of a tropospheric BrO background at the global scale. The operational tropospheric BrO retrieval is based on the scientiﬁc algorithm developed by BIRA (?).

11.2. Retrieval Settings

11.2.1. Splitting of the tropospheric and tropospheric parts

Details of the BrO total column retrieval are described in Section 5.6. Here a splitting of the total column into the tropospheric and the stratospheric parts is explained. The stratospheric BrO profiles are calculated using the dynamical BrO climatology (Theys et al., 2008). The climatology is based on the output of the 3-D chemical transport model BASCOE. The impact of the atmospheric dynamics on the stratospheric BrO distribution is treated by means of Bry/O3 correlations, while photochem- BrO ical effects are taken into account using stratospheric NO2 columns as an indicator for the /Bry ratio. In practice, for each pixel, a stratospheric BrO profile is calculated from the climatology using the total O3 (Nadir product) and stratospheric NO2vertical columns (intermediate result from the tropospheric NO2 product). The stratospheric BrO vertical columns (VCDstrato ) are derived by integrating climatological stratospheric BrO profiles between the tropopause and the top- of-atmosphere. VCDstrato is than converted into the SCDstrato:

SCDstrato = V CDstrato ∗ AMF strato (11.1)

Tropospheric slant column is calculated by subtracting the stratospheric part from the total slant columns:

SCDtropo = SCDtotal − SCDstrato (11.2)

During the verification phase it has been discovered that there is an offset between the operational and the scientific (BIRA) BrO slant columns. Besides the operational slant columns also demonstrate a time trend not confirmed by any satellite or ground-based sensors. In order to get rid of this offset an equatorial correction was recommended by the scientific product developers. For a given day, all SCDs (from all orbits of this day) in the equatorial latitudinal band (5°S-5°N) are taken and the average SCD in this region is estimated. This average SCDs is then subtracted from all SCDs of the day. Finally, a constant SCD value of 7.5e13 molec/cm² is added to compensate for the equatorial value. By doing so, any constant offset is automatically removed.

129 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

The equatorial correction values are pre-calculated from the latest completely processed L2 version. Here, an assumption is made that equatorial BrO SCDs do not change from version to version. This assumption has been conﬁrmed by comparison of the equatorial corrections based on L2v6.01 with those computed from the veriﬁcation data set of the current L2 version (L2v7). Finally, the tropospheric column is computed as follows:

SCDtropo V CDtropo = (11.3) AMF tropo

Nadir Settings see Section 5.6 Stratospheric Part see Subsection 11.2.1 Limits Cloud limit cloud fraction < 0.4 AMFtropo limit AMFtropo > 0.5 (to ensure that measurements are sensitive enough to the tropospheric BrO) SZA limit 80° Used Look-up Tables AMF Stratospheric Look-up table of the box AMFs used in the BIRA scientific processor Stratospheric BrO profiles from the BASCOE CTM AMF Tropospheric Look-up table of the box AMFs used in the BIRA scientific processor Gaussian tropospheric BrO profile with a maximum at 6 km and a full width half maximum of 2 km Tropopause height ECMWF ERA-Interim re-analysis

The schematic chart visualizing the retrieval steps is depicted in Figure 11.1. L2 input is colored green, databases blue, intermediate and ﬁnal results yellow and processing steps red. This tropospheric BrO will become a part of the current total BrO product. Following the new approach the total BrO column results from the sum of BrOstrato and BrOtropo.

SGP OL1b-2 ATBD Version 7 Page 130 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Figure 11.1.: Flow chart of the tropospheric BrO retrieval algorithm.

SGP OL1b-2 ATBD Version 7 Page 131 of 137

Bibliography

Abramowitz, M. and Stegun, I.: Handbook of Mathematical Functions, National Bureau of Standards, AMS55, New York, 1964. Acarreta, J. R., Stammes, P., and Knap, W. H.: First retrieval of cloud phase from SCIAMACHY spectra around 1.6 @mm, Atmospheric Research, 72, 89–105, 2004. Anderson, G., Clough, S., Kneizys, F., Chetwynd, J., and Shettle, E.: AFGL Atmospheric Constituent Profiles (0 – 120 km), Tech. Rep. TR-86-0110, AFGL, 1986. Armstrong, B.: Spectrum Line Profiles: The Voigt Function, J. Quant. Spectrosc. & Radiat. Transfer, 7, 61–88, doi:10.1016/0022-4073(67)90057-X, 1967. Bogumil, K., Orphal, J., Homann, T., Voigt, S., Spietz, P., Fleischmann, O. C., Vogel, A., Hartmann, M., Krom- minga, H., Bovensmann, H., Frerick, J., and Burrows, J. P.: Measurements of molecular absorption spectra with the SCIAMACHY pre-flight model: instrument characterization and reference data for atmospheric remote-sensing in the 230-2380 nm region, Journal of Photochemistry and Photobiology, 157, 167–184, 2003. Bovensmann, H., Burrows, J. P., Buchwitz, M., Frerick, J., Noël, S., Rozanov, V. V., Chance, K. V., and Goede, A. P. H.: SCIAMACHY: Mission Objectives and Measurement Modes, Journal of Atmospheric Sciences, 56, 127–150, doi:10.1175/1520-0469(1999)056, 1999. Brion, J., Chakir, A., Charbonnier, J., Daumont, D., Parisse, C., and Malicet, J.: Absorption spectra measurements for the ozone molecule in the 350-830 nm region, J. Atmos. Chem., 30, 291–299, 1998. Buchwitz, M., Khlystova, I., Bovensmann, H., and Burrows, J. P.: Three years of global carbon monoxide from SCIAMACHY: comparison with MOPITT and first results related to the detection of enhanced CO over cities, 7, 2399–2411, 2007. Carlotti, M., Höpfner, M., Raspollini, P., Ridolfi, M., and Carli, B.: High level algorithm definition and physical and mathematical optimisations, Tech. Rep. TN-IROE-RSA9601, IROE, Florence, Italy, 1998. Chance, K. and Spurr, R.: Ring effect studies: Rayleigh scattering, including molecular parameters for rotational Raman scattering, and the Fraunhofer spectrum., Applied optics, 36, 5224–5230, 1997. Chance, K. V. and Spurr, R. J. D.: Ring effect studies: Rayleigh scattering, including molecular parameters for rotational Raman scattering, and the Fraunhofer spectrum, App. Opt., 36, 5224–5230, doi:10.1364/AO.36. 005224, 1997. Chandrasekhar, S.: Radiative Transfer, Dover, Mineola, New York, 1960. Clough, S. and Kneizys, F.: Convolution Algorithm for the Lorentz function, Appl. Opt., 18, 2329–2333, 1979. Clough, S., Kneizys, F., Anderson, G., Shettle, E., Chetwynd, J., Abreu, L., Hall, L., and Worsham, R.: FAS- COD3: spectral simulation, in: IRS’88: Current Problems in Atmospheric Radiation, edited by Lenoble, J. and Geleyn, J., pp. 372–375, A. Deepak Publishing, 1988. Clough, S., Kneizys, F., and Davies, R.: Line Shape and the Water Vapor Continuum, Atmos. Res., 23, 229– 241, 1989. de Haan, J., Bosma, P., and Hovenier, J.: The adding method for multiple scattering calculations of polarised light, Astron. Astrophys., 183, 371–391, 1987. Dennis, Jr., J., Gay, D., and Welsch, R.: An Adaptive Nonlinear Least–squares Algorithm, 7, 348–368, 1981. Deschamps, P., Breon, F., Leroy, M., Podaire, A., Bricaud, A., Buriez, J., and Seze, G.: The POLDER mission: instrument characteristics and scientific objectives, IEEE Transactions on Geoscience and Remote Sensing, 32, 598–615, doi:10.1109/36.297978, 1994.

133 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Edwards, D.: Atmospheric transmittance and radiance calculations using line–by–line computer models, in: Modelling of the Atmosphere, vol. 928, pp. 94–116, 1988. Fleischmann, O., Hartmann, M., Burrows, J. P.,and Orphal, J.: New ultraviolet absorption cross-sections of BrO at atmospheric temperatures measured by time-windowing Fourier transform spectroscopy, J. Photochem. Photobiol. A, 168, 117–132, 2004. Fomin, B.: Effective Interpolation technique for line–by–line calculation of radiation absorption in gases, J. Quant. Spectrosc. & Radiat. Transfer, 53, 663–669, 1995. Frankenberg, C., Platt, U., and Wagner, T.: Iterative maximum a posteriori (IMAP)-DOAS for retrieval of strongly absorbing trace gases: Model studies for CH4 and CO2 retrieval from near infrared spectra of SCIAMACHY onboard ENVISAT, 5, 9–22, 2005. Gimeno García, S., Schreier, F., Lichtenberg, G., and Slijkhuis, S.: Near infrared nadir retrieval of vertical column densities: methodology and application to SCIAMACHY, Atmos. Meas. Tech., 4, 2633–2657, doi: 10.5194/amt-4-2633-2011, 2011. Gloudemans, A., Schrijver, H., Kleipool, Q., van den Broek, M., Straume, A., Lichtenberg, G., van Hees, R., Aben, I., and Meirink, J.: The impact of SCIAMACHY near-infrared instrument calibration on CH4 and CO total columns, 5, 2369–2383, 2005. Golub, G. and Pereyra, V.: Separable nonlinear least squares: the variable projection method and its applications, 19, R1–R26, 2003. Gordon, I., Rothman, L., et al.: The HITRAN2016 molecular spectroscopic database, J. Quant. Spectrosc. & Radiat. Transfer, 203, 3 – 69, doi:10.1016/j.jqsrt.2017.06.038, 2017. Greenblatt, G. D., Orlando, J. J., Burkholder, J. B., and Ravishankara, A. R.: Absorption measurements of oxygen between 330 and 1140 nm, J. Geophys. Res., 95, 18 577–18 582, doi:10.1029/JD095iD11p18577, 1990. Herman, J. R., Bhartia, P. K., Torres, O., Hsu, C., Seftor, C., and Celarier, E.: Global distribution of UV- absorbing aerosols from Nimbus 7/TOMS data, Journal of Geophysical Research, 102, 16 911–16 922, doi: 10.1029/96JD03680, 1997. Hermans, C., Vandaele, A. C., Carleer, M., Fally, S., Colin, R., Jenouvrier, A., Coquart, B., and Mèrienne, M.-F.: Absorption cross-sections of atmospheric constituents: NO2,O2, and H2O, Environ. Sci. Pollut. R., 6, 151–158, 1999. Humlícek,ˇ J.: Optimized computation of the Voigt and complex probability function, J. Quant. Spectrosc. & Radiat. Transfer, 27, 437–444, doi:10.1016/0022-4073(82)90078-4, 1982. Jacquinet-Husson, N., Scott, N., Chedin, A., Crepeau, L., Armante, R., Capelle, V., Orphal, J., Coustenis, A., Boonne, C., Poulet-Crovisier, N., Barbe, A., Birk, M., Brown, L., Camy-Peyret, C., Claveau, C., Chance, K., Christidis, N., Clerbaux, C., Coheur, P., Dana, V., Daumont, L., Backer-Barilly, M. D., Lonardo, G. D., Flaud, J., Goldman, A., Hamdouni, A., Hess, M., Hurley, M., Jacquemart, D., Kleiner, I., Kopke, P., Mandin, J., Massie, S., Mikhailenko, S., Nemtchinov, V., Nikitin, A., Newnham, D., Perrin, A., Perevalov, V., Pinnock, S., Regalia-Jarlot, L., Rinsland, C., Rublev, A., Schreier, F., Schult, L., Smith, K., Tashkun, S., Teffo, J., Toth, R., Tyuterev, V., Auwera, J. V., Varanasi, P., and Wagner, G.: The GEISA spectroscopic database: Current and future archive for Earth and planetary atmosphere studies, J. Quant. Spectrosc. & Radiat. Transfer, 109, 1043–1059, doi:10.1016/j.jqsrt.2007.12.015, 2008. Jacquinet-Husson, N. et al.: The 2015 edition of the GEISA spectroscopic database, 327, 31 – 72, doi:10. 1016/j.jms.2016.06.007, 2016. Kahaner, D., Moler, C., and Nash, S.: Numerical Methods and Software, Prentice–Hall, Englewood Cliffs, NJ, 1989. Kistler, R., Collins, W., Saha, S., White, G., Woollen, J., Kalnay, E., Chelliah, M., Ebisuzaki, W., Kana- mitsu, M., Kousky, V., van den Dool, H., Jenne, R., and Fiorino, M.: The NCEP-NCAR 50-Year Re- analysis: Monthly Means CD-ROM and Documentation, Bull. Am. Met. Soc., 82, 247–267, doi:10.1175/ 1520-0477(2001)082<0247:TNNYRM>2.3.CO;2, 2001.

SGP OL1b-2 ATBD Version 7 Page 134 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Kokhanovsky, A. A. and Rozanov, V. V.: The physical parameterization of the top-of-atmosphere reflection function for a cloudy atmosphere-underlying surface system: the oxygen A-band case study, Journal of Quantitative Spectroscopy and Radiative Transfer, 85, 35–55, doi:10.1016/S0022-4073(03)00193-6, 2004. Kokhanovsky, A. A., Rozanov, V. V., Zege, E. P., Bovensmann, H., and Burrows, J. P.: A semianalytical cloud retrieval algorithm using backscattered radiation in 0.4-2.4 µm spectral region, Journal of Geophysical Research (Atmospheres), 108, 4008–+, doi:10.1029/2001JD001543, 2003. Kokhanovsky, A. A., Rozanov, V. V., Burrows, J. P., Eichmann, K.-U., Lotz, W., and Vountas, M.: The SCIA- MACHY cloud products: Algorithms and examples from ENVISAT, Advances in Space Research, 36, 789– 799, doi:10.1016/j.asr.2005.03.026, 2005. Kondratyev, K. Y. and Binenko, V.: Impact of Cloudiness on Radiation and Climate, Leningrad: Gidrometeoiz- dat, 1984. Kretschel, K.: ENVISAT-1 SCIAMACHY Level 1b to 2 Offline Processing, Architectural Design Document, issue 2 (ENV-IS-DLR-SCI-2200-0005), Tech. rep., DLR, 2006. Krijger, J. M.: SCIAMACHY degradation model ATBD, SRON-SQWG-TN-2011-001, draft, Tech. rep., SRON, 2011. Krijger, J. M., Aben, I., and Schrijver, H.: Distinction between clouds and ice/snow covered surfaces in the identification of cloud-free observations using SCIAMACHY PMDs, Atmospheric Chemistry & Physics, 5, 2729–2738, 2005. Kromminga, H., Orphal, J., Spietz, P., Voigt, S., and Burrows, J. P.: The temperature dependence (213- 293 K) of the absorption cross-sections of OClO in the 340-450 nm region measured by Fourier-transform spectroscopy, Journal of Photochemistry and Photobiology A, 157, 149–160, 2003. Lambert, J.-C., Granville, J., Van Roozendael, M., Müller, J.-F., Goutail, F., Pommereau, J. P., Sarkissian, A., Johnston, P. V., and Russell III, J. M.: Global Behaviour of Atmospheric NO2 as Derived from the Integrated Use of Satellite, Ground-based Network and Balloon Observations, in: Atmospheric Ozone - 19th Quad. Ozone Symp., Sapporo, Japan, edited by NASDA, pp. 201–202, 2000. Lerot, C., Van Roozendael, M., van Geffen, J., van Gent, J., Fayt, C., Spurr, R., Lichtenberg, G., and von Bar- gen, A.: Six years of total ozone column measurements from SCIAMACHY nadir observations, Atmospheric Measurement Techniques, 2, 87–98, 2009. Lichtenberg, G. and Kretschel, K.: SCIAMACHY Level 1b-2 Off-line Data Processing Configuration Manage- ment Of Level 1b-2 Auxiliary Data Files, issue 4 (ENV-CMA-DLR-SCIA-0061), Tech. rep., DLR, 2009. Liou, K.-N.: An Introduction to Atmospheric Radiation, Academic Press, Orlando, 1980. Loeb, N. G. and Davies, R.: Observational evidence of plane parallel model biases: Apparent dependence of cloud optical depth on solar zenith angle, J. Geophys. Res., 101, 1621–1634, doi:10.1029/95JD03298, 1996. Loyola, D.: Cloud retrieval for SCIAMACHY, in: Proc. of ERS-ENVISAT Symposium, Gothenburg, Sweden, 16-20 Oct 2000, vol. SP-641, ESA, 2000. Loyola, D. and Ruppert, T.: A new PMD cloud-recognition algorithm forGOME, ESA Earth Observation Quar- terly, 58, 45–47, 1998. McLinden, C. A., McConnell, J., Griffioen, E., and McElroy, C.: A vector radiative–transfer model for the Odin/OSIRIS project, Can J Phys., 80, 375–93, 2002. Meller, R. and Moortgat, G. K.: Temperature dependence of the absorption cross section of HCHO between 223 and 323K in the wavelength range 225-375 nm, J. Geophys. Res., 105, 7089–7102, doi: 10.1029/1999JD901074, 2000. Müller, J.-F. and Brasseur, G.: IMAGES: A three-dimensional chemical transport model of the global troposphere, J. Geophys. Res., 100, 16 445–16 490, 1995. Noël, S., Buchwitz, M., Bovensmann, H., Hoogen, R., and Burrows, J. P.: Atmospheric water vapor amounts retrieved from GOME satellite data, Geophys. Res. Lett., 26, 1841–1844, doi:10.1029/1999GL900437, 1999. Norton, R. and Rinsland, C.: ATMOS data processing and science analysis methods, Appl. Opt., 30, 389–400, 1991.

SGP OL1b-2 ATBD Version 7 Page 135 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Olivero, J. and Longbothum, R.: Empirical Fits to the Voigt Line Width: a brief Review, J. Quant. Spectrosc. & Radiat. Transfer, 17, 233–236, doi:10.1016/0022-4073(77)90161-3, 1977. Pickett, H., Poynter, R., Cohen, E., Delitsky, M., Pearson, J., and Müller, H.: Submillimeter, millimeter, and microwave spectral line catalog, J. Quant. Spectrosc. & Radiat. Transfer, 60, 883–890, 1998. Platt, U.: Differential optical absorption spectroscopy (DOAS): in Air Monitoring by Spectroscopic Techniques, John Willey, New York, 1994. Platt, U. and Stutz, J.: Differential Optical Absorption Spectroscopy: Principles and Applications, Springer Verlag, 2008. Pope, R. M. and Fry, E. S.: Absorption spectrum (380-700 nm) of pure water. II. Integrating cavity measurements, Appl. Opt., 36, 8710–8723, 1997. Ralston, A. and Rabinowitz, P.: A First Course in Numerical Analysis, McGraw–Hill Book Company, second edn., 1978. Robert, C. E., von Savigny, C., Burrows, J. P., and Baumgarten, G.: Climatology of noctilucent cloud radii and occurrence frequency using SCIAMACHY, Journal of Atmospheric and Solar-Terrestrial Physics, 71, 408–423, doi:10.1016/j.jastp.2008.10.015, 2009. Rodgers, C. D.: Inverse methods for atmospheric sounding: theory and practice, Atmos. Oceanic Planet. Phys., World Scientiﬁc Publishing, Singapore, 2000. Rossow, W. B.: Measuring Cloud Properties from Space: A Review., Journal of Climate, 2, 201–213, doi: 10.1175/1520-0442(1989)002<0201:MCPFSA>2.0.CO;2, 1989. Rothman, L., Gamache, R., Goldman, A., Brown, L., Toth, R., Pickett, H., Poynter, P., Flaud, J.-M., Camy- Peyret, C., Barbe, A., Husson, N., Rinsland, C., and Smith, M.: The HITRAN database: 1986 edition, Appl. Opt., 26, 4058, 1987. Rothman, L., Barbe, A., Benner, D. C., Brown, L., Camy-Peyret, C., Carleer, M., Chance, K., Clerbaux, C., Dana, V., Devi, V., Fayt, A., Flaud, J.-M., Gamache, R., Goldman, A., Jacquemart, D., Jucks, K., Lafferty, W., Mandin, J.-Y., Massie, S., Nemtchinov, V., Newnham, D., Perrin, A., Rinsland, C., Schroeder, J., Smith, K., Smith, M., Tang, K., Toth, R., Auwera, J. V., Varanasi, P., and Yoshino, K.: The HITRAN molecular spectroscopic database: edition of 2000 including updates through 2001, J. Quant. Spectrosc. & Radiat. Transfer, 82, 5–44, doi:10.1016/S0022-4073(03)00146-8, 2003. Rothman, L., Gordon, I., Barbe, A., Benner, D. C., Bernath, P., Birk, M., Boudon, V., Brown, L., Campargue, A., Champion, J.-P., Chance, K., Coudert, L., Dana, V., Devi, V., Fally, S., Flaud, J.-M., Gamache, R., Goldman, A., Jacquemart, D., Kleiner, I., Lacome, N., Lafferty, W., Mandin, J.-Y., Massie, S., Mikhailenko, S., Miller, C., Moazzen-Ahmadi, N., Naumenko, O., Nikitin, A., Orphal, J., Perevalov, V., Perrin, A., Predoi-Cross, A., Rinsland, C., Rotger, M., Simecková, M., Smith, M., Sung, K., Tashkun, S., Tennyson, J., Toth, R., Vandaele, A., and Auwera, J. V.: The HITRAN 2008 molecular spectroscopic database, J. Quant. Spectrosc. & Radiat. Transfer, 110, 533 – 572, doi:10.1016/j.jqsrt.2009.02.013, 2009. Rozanov, V. V. and Kokhanovsky, A. A.: Semianalytical cloud retrieval algorithm as applied to the cloud top altitude and the cloud geometrical thickness determination from top-of-atmosphere reﬂectance measurements in the oxygen A band, Journal of Geophysical Research (Atmospheres), 109, 5202–+, doi: 10.1029/2003JD004104, 2004. Rozanov, V. V., Kurosu, T., and Burrows, J. P.: Retrieval of atmospheric constituents in the UV-visible: a new quasi-analytical approach for the calculation of weighting functions., Journal of Quantitative Spectroscopy and Radiative Transfer, 60, 277–299, doi:10.1016/S0022-4073(97)00150-7, 1998. Rozanov, V. V., Buchwitz, M., Eichmann, K.-U., de Beek, R., and Burrows, J. P.: Sciatran - a new radiative transfer model for geophysical applications in the 240-2400 NM spectral region: the pseudo-spherical version, Advances in Space Research, 29, 1831–1835, doi:10.1016/S0273-1177(02)00095-9, 2002. Scherbakov, D.: SCIAMACHY Command Line Tool SciaL1c Software User’s Manual, issue 2B (ENV-SUM- DLR-SCIA-0071), Tech. rep., DLR, 2008. Schreier, F.: Optimized Implementations of Rational Approximations for the Voigt and Complex Error Function, J. Quant. Spectrosc. & Radiat. Transfer, 112, 1010–1025, doi:10.1016/j.jqsrt.2010.12.010, 2011.

SGP OL1b-2 ATBD Version 7 Page 136 of 137 Docnr.: ENV-ATB-QWG-SCIA-0085 Issue : 3 Date : 16.04.18

Schreier, F., Gimeno García, S., Hedelt, P., Hess, M., Mendrok, J., Vasquez, M., and Xu, J.: GARLIC – A General Purpose Atmospheric Radiative Transfer Line-by-Line Infrared-Microwave Code: Implementation and Evaluation, J. Quant. Spectrosc. & Radiat. Transfer, 137, 29–50, doi:10.1016/j.jqsrt.2013.11.018, 2014. Slijkhuis, S. and Lichtenberg, G.: ENVISAT-1 SCIAMACHY Level 0 to 1c Processing, Algorithm Theoretical Basis Document, issue 6 (ENV-ATB-DLR-SCIA-0041), Tech. rep., DLR, 2014. Sparks, L.: Efﬁcient Line–by–Line Calculation of Absorption Coefﬁcients to High Numerical Accuracy, J. Quant. Spectrosc. & Radiat. Transfer, 57, 631–650, doi:10.1016/S0022-4073(96)00154-9, 1997. Spurr, R.: SCIAMACHY Level 1b to 2 Off-line Processing, Tech. Rep. ENV-ATB-SAO-SCIA-2200-0003, DLR– DFD and Harvard–CfA–SAO, 1998. Spurr, R., Kurosu, T., and Chance, K.: A Linearized discrete Ordinate Radiative Transfer Model for Atmospheric Remote Sensing, Journal of Quantitative Spectroscopy and Radiative Transfer, 68, 689–735, 2001. Spurr, R., van Roozendael, M., and Loyola, D. G.: Algorithm Theoretical Basis Document for GOME Total Column Densities of Ozone and Nitrogen Dioxide, Tech. rep., DLR, 2004. Theys, N., van Roozendael, M., Errera, Q., Hendrick, F., Daerden, F., Chabrillat, S., Dorf, M., Pfeilsticker, K., Rozanov, A., Lotz, W., Burrows, J. P.,Lambert, J.-C., Goutail, F., Roscoe, H. K., and de Mazière, M.: A global stratospheric bromine monoxide climatology based on the BASCOE chemical transport model, Atmospheric Chemistry & Physics Discussions, 8, 17 581–17 629, 2008. Uchiyama, A.: Line–by–Line Computation of the Atmospheric Absorption Spectrum Using the Decomposed Voigt Line Shape, J. Quant. Spectrosc. & Radiat. Transfer, 47, 521–532, 1992. Van Roozendael, M., Loyola, D., Spurr, R., Balis, D., Lambert, J., Livschitz, Y., Valks, P., Ruppert, T., Kenter, P., Fayt, C., et al.: Ten years of GOME/ERS-2 total ozone data – The new GOME data processor (GDP) version 4: 1. Algorithm description, Journal of Geophysical Research-Atmospheres, 111, D14 311, 2006.

Vandaele, A. C., Simon, P. C., Guilmot, J. M., Carleer, M., and Colin, R.: SO2 absorption cross section measurement in the UV using a Fourier transform spectrometer, J. Geophys. Res., 99, 25 599–25 606, doi: 10.1029/94JD02187, 1994. Vandaele, A. C., Hermans, C., Simon, P. C., Carleer, M., Colin, R., Fally, S., Merienne, M. F., Jenouvrier, A., −1 −1 and Coquart, B.: Measurements of the NO2 absorption cross-section from 42 000 cm to 10 000 cm (238-1000 nm) at 220K and 294 K, J. Quant. Spectrosc. Ra., 59, 171–184, 1998. Volkamer, R., Spietz, P., Burrows, J., and Platt, U.: High-resolution absorption cross-section of glyoxal in the UV–vis and IR spectral ranges, Journal of Photochemistry and Photobiology A: Chemistry, 172, 35–46, 2005. von Savigny, C., Kokhanovsky, A., Bovensmann, H., Eichmann, K.-U., Kaiser, J. W., Noël, S., Rozanov, A. V., Skupin, J., and Burrows, J. P.: NLC Detection and Particle Size Determination: First Results from SCIA- MACHY on ENVISAT, Adv. Space Res., 34, 851–856, 2004. von Savigny, C., Ulasi, U. P., Eichmann, K.-U., Bovensmann, H., and Burrows, J. P.: Detection and mapping of polar stratospheric clouds using limb scattering observations, Atmos. Chem. Phys., 5, 3071–3079, 2005. Vountas, M., Rozanov, V. V., and Burrows, J. P.: Ring effect: impact of rotational Raman scattering on radiative transfer in Earth’s atmosphere., Journal of Quantitative Spectroscopy and Radiative Transfer, 60, 943–961, doi:10.1016/S0022-4073(97)00186-6, 1998.

Wang, P., Stammes, P., van der A, R., Pinardi, G., and van Roozendael, M.: FRESCO+: an improved O2 A-band cloud retrieval algorithm for tropospheric trace gas retrievals, Atmospheric Chemistry & Physics, 8, 6565–6576, 2008. Weideman, J.: Computation of the Complex Error Function, 31, 1497–1518, doi:10.1137/0731077, 1994. Yamomoto, G. and Wark, D. Q.: Discussion of letter by A. Hanel: Determination of cloud altitude from a satellite, J. Geophys. Res., 66, 3596, 1961.

SGP OL1b-2 ATBD Version 7 Page 137 of 137