Laboratpry Comparison Experiment Implementation Plan

Fiducial Reference Measurements for validation of Surface Temperature from Satellites (FRM4STS) - Laboratory Calibration of Participants Radiometers and Blackbodies

Archive of the FRM TIR radiometer calibration and verification data

ESA Contract No. 4000113848_15I-LG

Evangelos Theocharous & Nigel Fox

OCTOBER 2015

Reference OFE-110-V1-Iss-1-Ver-1-DRAFT Issue 1 Revision 1 Date of Issue 30 October 2015 Status DRAFT Document Type D110- ARCH

NPL - Commercial OFE-D-110-V1-Iss-1-Ver-1-DRAFT

INTENTIONALLY BLANK

Fiducial Reference Measurements for validation of Surface Temperature from Satellites (FRM4STS): Laboratory Calibration of Participants Radiometers and Blackbodies

D-110 Archive of the FRM TIR radiometer calibration and verification data

Evangelos Theocharous & Nigel Fox Environment Division

 Queen’s Printer and Controller of HMSO, 2015

National Physical Laboratory Hampton Road, Teddington, Middlesex, TW11 0LW

CONTENTS

DOCUMENT VERSION HISTORY DOCUMENT APPROVAL

NPL - Commercial OFE-D-110-V1-Iss-1-Ver-1-DRAFT 19

DOCUMENT MANAGEMENT

Issu Revisio Date of Description of Changes e n Issue/revision 1 1 30-Oct-15 Creation of document

DOCUMENT APPROVAL

Contractor Approval

Name Role in Project Signature & Date (dd/mm/yyyy)

Dr Nigel Fox Technical Leader

Mr David Gibbs Project Manager

CUSTOMER APPROVAL

Name Role in Project Signature Date (dd/mm/yyyy)

C Donlon ESA Technical Officer

APPLICABLE DOCUMENTS

AD Ref. Ver. Title /Iss. EOP- 1 Fiducial Reference Measurements for Thermal Infrared Satellite SM/2642 Validation (FRM4STS) Statement of Work

ACRONYMS AND ABBREVIATIONS

AMBER Absolute Measurements of Black-body Emitted Radiance CEOS Committee on Earth Observation Satellites DMI Danish Meteorological Institute IR Infra-Red ISO International Organization for Standardization KIT Karlsruhe Institute of Meteorology LST Land Surface Temperature NIST National Institute of Standards and Technology (USA) NMI National Metrology Institute NPL National Physical Laboratory PTB Physikalisch Technische Bundesanstalt RMS Root Mean Square SST Sea Surface Temperature SI Système Internationale WGCV Working Group for Calibration and Validation WST Water Surface Temperature

1. INTRODUCTION

The measurement of the Earth’s surface temperature is a critical product for meteorology and an essential parameter/indicator for climate monitoring. Satellites have been monitoring global surface temperature for some time, and have established sufficient consistency and accuracy between in-flight sensors to claim that it is of “climate quality”. However, it is essential that such measurements are fully anchored to SI units and that there is a direct correlation with “true” surface/in-situ based measurements.

The most accurate of these surface based measurements (used for validation) are derived from field deployed IR radiometers. These are in principle calibrated traceably to SI units, generally through a reference radiance blackbody. Such instrumentation is of varying design, operated by different teams in different parts of the globe. It is essential for the integrity of their use, to provide validation data for satellites both in-flight and to provide the link to future sensors, that any differences in the results obtained between them are understood. This knowledge will allow any potential biases to be removed and not transferred to satellite sensors. This knowledge can only be determined through formal comparison of the instrumentation, both in terms of its primary “lab based” calibration and its use in the field. The provision of a fully traceable link to SI ensures that the data are robust and can claim its status as a “climate data record”.

The “IR Cal/Val community” is well versed in the need and value of such comparisons having held highly successful exercises in Miami and at NPL in 2001 [1, 2] and 2009 [3, 4]. However, six years will have passed and it is considered timely to repeat/update the process. Plans are in place for the comparisons to be repeated in 2016. The 2016 comparison will consist of five stages:

i. Laboratory comparisons of the radiometers and reference radiance blackbodies of the participants. ii. Field comparisons of Water Surface Temperature (WST) scheduled to be held at Wraysbury fresh water reservoir, near NPL. iii. Field comparisons of Land Surface Temperature (LST) scheduled to be held on the NPL campus. iv. Field comparisons of Land Surface Temperature (LST) scheduled to be held at two sites (Gobabeb Training and Research Centre on the Namib plain and the “Farm Heimat” site in the Kalahari bush) in Namibia in 2016. v. Field comparisons of Ice Surface Temperature (IST) scheduled to be held in the Arctic.

During the 2016 comparison, NPL will act as the pilot laboratory and will provide traceability to SI units during laboratory and “field” comparisons at and around NPL. NPL will be supported by the PTB providing traceability to SI units during laboratory measurements. This document describes how the calibration and verification data of the laboratory FRM TIR radiometers and blackbodies will be established and archived, following good data stewardship practices, which will provide access to records by research on request.

2. ORGANIZATION

2.1 PILOT

NPL, the UK national metrology institute (NMI) will serve as pilot for the 2016 laboratory comparisons, supported by the PTB, the NMI of Germany. NPL, the pilot, will be responsible for inviting participants and for the analysis of data, following appropriate processing by individual participants. NPL, as pilot, will be the only organisation to have access and to view all data from all participants. This data will remain confidential to the participants and NPL at all times, until the publication of the report showing results of the comparison to participants.

2.2 PARTICIPANTS

The list of the potential participants, based on current contacts and expectation who will be likely to take part can be found elsewhere [5]. Dates for the comparison activities are also provided in ref 5. A full invitation to the international community through CEOS and other relevant bodies will be carried out to ensure full opportunity and encouragement is provided to all. All participants should be able to demonstrate independent traceability to SI of the instrumentation that they use, or make clear the route of traceability via another named laboratory.

By their declared intention to participate in this key comparison, the participants accept the general instructions and the technical protocols written down in this document and commit themselves to follow the procedures strictly. Once the protocol and list of participants have been reviewed and agreed, no change to the protocol may be made without prior agreement of all participants. Where required, demonstrable traceability to SI will be obtained through participation of PTB and NPL as pilot.

The laboratory comparison of the radiometers and blackbodies of the participants will take place at NPL during the week beginning 20th June 2016. Full details of this comparison can be found elsewhere [5]. During the laboratory comparison, the participating radiometers will measure the brightness temperature of the NPL ammonia heat-pipe reference blackbody, while the NPL transfer radiometer (AMBER) [6] and the PTB broad band infrared radiometer will be used to measure the brightness temperature of the participating blackbodies at 10.1 µm and in the 8 µm to 14µm wavelength regions, respectively.

3. PARTICIPANTS’ INFORMATION

Each organisation which will be participating in this comparison will have its own directory on the website which will include the following:

A. Name of the organisation: e.g. the Mediterranean Centre for Environmental Studies (Fundación Centro de Estudios Ambientales del Mediterraneo, CEAM)

B. Contact person(s) within that organisation responsable for the measurements: e.g. Dr Raquel Niclos

C. Contact information/Address: e.g. 14 Charles Darwin, 46980 PATERNA, Valencia, Spain.

D. Website of organisation: e.g. www.ceam.es

E. Email of contact person(s): e.g. [email protected]

F. Summary of the activities of the organisation in this field:

4. RADIOMETER COMPARISON

4.1 TECHNICAL DESCRIPTION OF THE RADIOMETERS

A. Technical descriptions of the radiometers which will take part in the 2016 comparisons will be included in this Section E.g.: The radiometer uses a thermopile detector to provide a millivolt output which is dependent on the temperature difference between the target being viewed and the radiometer

body temperature. A thermistor is used to measure the temperature of the sensor body, which is used to reference the target temperature. The radiometer operates in the 8 m to 14 m spectral band. The radiometer uses a germanium lens to image the target being viewed and has a field of view of 18º half-angle.

Radiometer type and model APOGEE Model IRR-PN

FOV 18° half angle Spectral band (μm) 8-14

Full information on this radiometer can be found at: http://www.apogeeinstruments.com/infrared_temperature_sensors.htm

B. References of description and work done using this radiometer

E.g.: Currently, the radiometer is being used for field measurements (hand held and mounted on meteorological towers) of land surface temperature and emissivity, both for research on surface thermal emission properties and for testing thermal infrared products. References to the work done using this radiometer:

1. Campbell Scientific and Apogee Instruments, 2007, “IRR-P Precision Infrared Temperature Sensor”, Instruction manual. 10pp. 2. Bugbee, B., Blonquist, M. and Lewis, N., “Design and Calibration of a New Infra-red Radiometer (website)”

C. Description of the traceability route for the primary calibration of the radiometer. E.g. According to the manufacturer of the radiometer (Apogee Instruments Inc.), the radiometer was calibrated using a blackbody, with k = 1 combined uncertainties shown below: Combined uncertainty of ±0.2 K in the 263 K to 338 K range and Combined uncertainty of ±0.5 K in the 233 K to 343 K range. The custom calibration by the manufacturer resulted with a unique calibration coefficient for the radiometer (calibration date: February 2016). Because of the limited detail about the traceability of this primary calibration, the institute carried out its own uncertainty analysis which was based on laboratory measurements using a blackbody Model P80P http://landinstruments.net/infrared/products/calibration_sources/p80p.htm during April, 2016. Calibration measurements were performed in the laboratory following the technical procedures described in the Draft Protocol. The Landcal Blackbody Source P80P was set at four temperatures (278 K, 283 K, 293 K and 303 K) in 3 different runs. Enough time was allowed for the blackbody to reach equilibrium at each temperature. Radiometers were aligned with the blackbody cavity and placed at a distance so that the field of view of the radiometer under-filled the blackbody aperture. The values of the component uncertainties reported in Table 1 are typical for the range of blackbody temperatures being considered.

D. Date of the last primary calibration of the radiometer.

E.g. 18th February 2016

E. Uncertainty budget associated with measurements completed with the radiometer.

Table 1 tabulates the uncertainty contributions associated with measurements completed with the APOGEE IRR-PN radiometer at k=1: - Repeatability: Typical value of the standard deviation of 180 measurements (1 per second, the radiometer response time is shorter than 1 s) at a fixed blackbody temperature without radiometer re-alignment. A repeatability value of ±0.05 K was measured in the 263 K to 338 K range while the value of repeatability increased to ±0.1 K in the 238 K to 343 K temperature range. - Reproducibility: Typical value of the standard deviation of three measurements of the reference blackbody maintained at constant temperature, including re-alignment between successive measurements. - Primary calibration: Combined uncertainty of the primary calibration of the APOGEE IRR- PN radiometer as given in the calibration certificate of the primary calibration. - Drift since calibration: 0.01 K estimated since last primary calibration of the APOGEE IRR- PN radiometer.

Table 1, is indicative of the component uncertainties associated with the calibration of the APOGEE IRR-PN radiometer. It should be noted that some of these components may sub-divide further depending on their origin. The Root Mean Square (RMS) total refers to the usual expression i.e. square root of the sum of the squares of all the individual uncertainty terms as shown in the example for Type A uncertainties.

Table 1: Uncertainty budget for the IRR-PN - APOGEE Instruments radiometer

Uncertainty Contribution Type A Type B Uncertainty in Uncertainty in Uncertainty in Brightness temperature Value / % Value / K (appropriate units)

Repeatability of URepeat measurement (1) 0.009K / 0.003%

Reproducibility of URepro measurement (2) 0.06K / 0.019%

(3) Primary calibration 0.15 K UPrim

Linearity of radiometer 0.04 K ULin

Drift since calibration 0.01 K UDrift

Ambient temperature 0.007 K Uamb fluctuations

Atmospheric 0.003 K Uatm absorption/emission

2 2 ½ RMS total ((Urepeat) +(URepro) ))

(1) Typical value of the standard deviation of 180 measurements at fixed black body temperature without re-alignment of radiometer. (2) Typical value of the standard deviation of three measurements of the reference blackbody maintained at constant temperature, including re-alignment between successive measurements. (3) Combined uncertainty of the primary calibration of the APOGEE IRR-PN radiometer as given in the calibration certificate.

F. Sea surface temperature measurements The Type A and Type B uncertainty contributions of the APOGEE IRR-PN radiometer in measuring the land surface temperature are shown below:

Operating Uncertainty Uncertainty wavelength Type A Type B µm K K

Apogee IRR-PN 11.5 µm 0.02 to 0.18 0.21

4.2 REPORTING OF PARTICIPANTS’ MEASUREMENTS AND RESULTS ON THE LABORATORY CALIBRATION OF RADIOMETERS

The results of the measurements of the brightness temperature of the reference blackbody by the participating radiometers should be reported on the form shown in Appendix A of this document. The measurements should also be provided in an excel file in the form shown in Table 2 shown below. These results will be available on the project’s website along with the corresponding measurements of the brightness temperature of the reference blackbody based on the temperature measurements made using Platinum Resistance Thermometers. Table 3 shows how the brightness temperature of the reference blackbody made by the pilot will be reported on the website.

Table 2: Format with which the measurement of the brightness temperature of the reference blackbody made by a participant should be reported. The table caption should include the date of the measurement (e.g. 22nd June 2016)

Time Radiometer measurement Combined uncertainty UTC K K

13:22:01 20.158 0.031 13:22:31 20.159 0.031 13:23:01 20.166 0.031 13:23:33 20.159 0.035 13:24:12 20.151 0.031 13:24:42 20.151 0.033 13:25:12 20.146 0.033 13:25:42 20.146 0.033 13:26:12 20.146 0.034 13:26:42 20.148 0.033 13:27:12 20.145 0.033 13:27:42 20.151 0.033 13:28:12 20.151 0.029 13:28:42 20.147 0.030 13:29:12 20.140 0.034 13:29:42 20.134 0.033 13:30:12 20.140 0.032 13:30:42 20.150 0.032 13:31:12 20.152 0.035 13:31:42 20.149 0.033 13:32:12 20.148 0.031 13:32:42 20.143 0.032 13:33:12 20.144 0.030 13:33:42 20.123 0.030 13:34:12 20.131 0.033 13:34:42 20.136 0.032 13:35:12 20.136 0.035 13:35:42 20.131 0.034 13:36:12 20.129 0.033 13:36:42 20.130 0.031 13:37:12 20.132 0.030 13:37:42 20.133 0.033 13:38:12 20.140 0.033 13:38:42 20.145 0.033 13:39:12 20.145 0.033 13:39:42 20.140 0.033 13:40:12 20.133 0.034 13:40:42 20.132 0.033

Table 3: Format with which the measurement of the brightness temperature of the reference blackbody made by the pilot will be reported. The table caption should include the date of the measurement (e.g. 22nd June 2016).

Blackbody brightness Combined Time temperature uncertainty UTC K K

13:22:32 20.138 0.035 13:23:20 20.130 0.038 13:24:08 20.109 0.041 13:24:56 20.093 0.043 13:25:44 20.101 0.041 13:26:32 20.116 0.05 13:27:20 20.117 0.038 13:28:08 20.122 0.034 13:28:56 20.115 0.042 13:29:43 20.097 0.036 13:30:31 20.107 0.03 13:31:19 20.119 0.027 13:32:07 20.117 0.031 13:32:55 20.127 0.039 13:33:43 20.103 0.043 13:34:31 20.112 0.037 13:35:19 20.108 0.031 13:36:07 20.090 0.029 13:36:55 20.093 0.029 13:37:43 20.107 0.036 13:38:31 20.104 0.044 13:39:19 20.095 0.036 13:40:07 20.083 0.043 13:40:55 20.100 0.035 13:41:43 20.104 0.036 13:42:31 20.117 0.034 13:43:19 20.132 0.033

The website will also show graphical plots of the measurements made by each participant as a function of time. These plots will include the actual brightness temperature of the reference blackbody, as measured by the test radiometer, as shown in Figure 1. The average value of the measurements made by the test radiometer (20.109 oC in this example) and the corresponding average value of the actual blackbody temperature (20.141 oC) will also be given on the website. The difference between the two average values (0.032 oC in this example) will also be given, to indicate the “error” with which the test radiometer measures the brightness temperature of the reference blackbody at that particular temperature setting.

Figure 1: Plots of the measurements made by every participant radiometer along with the actual brightness temperature of the blackbody which was being monitored, will be given on the website for all laboratory measurements completed in this comparison. Figure 1 shows how the measurements will be presented graphically.

The website will also show tables which summarise the differences/“errors” of the measurements made by participating radiometers of the brightness temperature of the reference blackbody at different temperatures from the actual blackbody cavity temperature. Table 4 shows an example of such a Table from the 2009 radiometer comparison which was completed at NPL [3]. Finally the website will show these differences in a graphical presentation; Figure 2 shows the plot of the differences of the mean of the radiometer readings from the mean of the temperature of the NPL reference blackbody during the same period (blue circles). The blackbody was maintained at a nominal temperature of 10 oC during the acquisition of the measurements shown in Figure 2. The red squares in the same Figure show the points corresponding to when the University of Miami blackbody (also maintained at a nominal temperature of 10 oC) was being viewed by the same radiometer.

Table 4: Difference of the mean of the measurement of the brightness temperature of the NPL reference blackbody completed by the participants during the 2009 comparison at NPL from the actual blackbody temperature. Similar tables will be created for the 2016 laboratory comparison sctivities.

Set NPL VTBB NPL VTBB Set NPL VTBB NPL VTBB Temperature 1st measur. 2nd measur. Temperature 1st measur. 2nd measur. oC mK mK oC mK mK GOTA 30 1147 Valencia DEPT 30 -108 8-14 μm 20 9 79 CE312-1-Unit 1 20 118 79 10 -1276 -1165 10.7 μm 10 321 292 5 -2137 5 416 GOTA 30 67 Valencia DEPT 30 -160 11.35 μm 20 189 189 CE312-1-Unit 1 20 221 160 10 254 235 12 μm 10 464 442 5 293 5 701

GOTA 30 87 Valencia DEPT 30 66 10.65 μm 20 199 139 CE312-1-Unit 1 20 82 101 10 204 185 10.8 μm 10 140 145 5 243 5 208

GOTA 30 -43 Valencia DEPT 30 372 9.1 μm 20 169 109 CE312-1-Unit 1 20 83 104 10 54 185 8.8 μm 10 -179 -154 5 253 5 -209 GOTA 30 -93 Valencia DEPT 30 -57 8.7 μm 20 169 129 CE312-1-Unit 2 20 189 134 10 74 185 10.7 μm 10 423 366 5 253 5 529 Canary 30 47 Valencia DEPT 30 14 8.3 μm 20 149 169 CE312-1-Unit 2 20 233 243 10 44 125 12 μm 10 467 383 5 133 5 557 IPL 30 -393 -298 Valencia DEPT 30 104 CE312-2 20 162 190 CE312-1-Unit 2 20 204 168 10.54 μm 10 (uncorrected) 691 683 10.8 μm 10 289 237 10 (corrected) 40 14 5 328 5 (uncorrected) 956 5 (corrected) -104 Valencia DEPT 30 -12 CE312-1-Unit 2 20 225 153 IPL 30 -278 -283 8.8 μm 10 514 392 CE312-2 20 136 154 5 605 11.29 μm 10 (uncorrected) 644 633 10 (corrected) 74 62 RAL 30 -24 5 (uncorrected) 860 SISTER 20 -25 -33 5 (corrected) 30 105 -16-6 -23-19 IPL 30 -333 -178 CE312-2 20 150 240 Southampton 30 6 10.57 μm 10 (uncorrected) 716 655 ISAR 20 126 28 10 (corrected) 26 -40 10 87 69 5 (uncorrected) 888 5 77 5 (corrected) -16 OUC 30 59 48 IPL 30 -438 -507 ISAR 20 94 64 CE312-2 20 74 228 10 145 136 9.15 μm 10 (uncorrected) 881 934 5 237 279 10 (corrected) 80 125 5 (uncorrected) 1434 KIT 30 190 152 5 (corrected) 206 Heidronics 20 30 KT - 15 10 -208 IPL 30 -492 -413 5 -632 CE312-2 20 169 225 8.69 μm 10 (uncorrected) 896 773 10 (corrected) -93 -223 5 (uncorrected) 1502 5 (corrected) 238 IPL 30 -402 -197 CE312-2 20 173 296 8.44 μm 10 (uncorrected) 983 809 10 (corrected) 291 127 5 (uncorrected) 1538 5 (corrected) 486

o 1.5 10 C (corrected)

) NPL C o

( RSMAS 1 e r u t a r 0.5 e p m e

e -0.5 r e f e r -1 m o r f

e -1.5 c n e r

e -2 f f i D -2.5 ) ) ) ) ) ) ) ) ) ) ) ) ) 1 3 5 L T R C 5 5 1 7 3 9 5 4 4 7 8 8 I 2 R R R . . . R . . . A L 3 6 2 1 4 U 1

1 K . . 9 8 8 . . . A 0 0 8 ( ( R D ( ( ( O 1 0 1 9 8 1 1 M M M o S ( ( ( ( 2 t I 1 1 1

A A A A A A ( ( (

1 1 8 L L l T T T E E E

a ( L l l a A A

P P O C C C O O V I I a a T T P V A I G G G V V T O O O G G G Radiometer-participant Figure 2: Plot of the mean of the differences of the radiometer readings from the temperature of the NPL reference blackbody (blue circles), maintained at a nominal temperature of 10 oC. The red squares show the points corresponding to the University of Miami blackbody. Similar plots corresponding to the 2016 comparison will be shown on the comparison website.

4.3 REPORTING OF PARTICIPANTS’ MEASUREMENTS AND RESULTS ON THE FIELD CALIBRATION OF RADIOMETERS

The results of the measurements of the Water Surface Temperature (WST) at Wraysbury reservoir and Land Surface Temperature (LST) should be reported by the participants on the forms shown in Appendix B of this document. The measurements should also be provided in excel files in the form shown in Table 5, shown below. These results will be available on the project’s website along with the corresponding measurements of other participants. The website will also include excel summary files showing the processed measurements.

Table 5: Format with which the WST and LST measurements made during the field comparison of the radiometers should be reported by a participants. The table caption should include the angle of view of the radiometer from the Nadir and the wavelength and bandwidth at which the radiometer is operating.

Sky Temp . Unce WST rtaint Time WST Uncertainty Sky Temperature y UTC oC oC oC oC

12/05/2009 18:00 29.212 0.123 -20.788 0.198 12/05/2009 18:10 29.156 0.121 -20.844 0.196 12/05/2009 18:20 29.162 0.103 -20.838 0.178 12/05/2009 18:30 29.195 0.096 -20.805 0.171 12/05/2009 18:40 29.218 0.128 -20.782 0.203 12/05/2009 18:50 29.249 0.113 -20.751 0.188 12/05/2009 19:00 29.303 0.115 -20.697 0.190 12/05/2009 19:10 29.294 0.119 -20.706 0.194 12/05/2009 19:20 29.231 0.112 -20.769 0.187 12/05/2009 19:30 29.288 0.090 -20.712 0.165 12/05/2009 19:40 29.313 0.111 -20.687 0.186 12/05/2009 19:50 29.346 0.107 -20.654 0.182 12/05/2009 20:00 29.326 0.109 -20.674 0.184 12/05/2009 20:10 29.324 0.131 -20.676 0.206 12/05/2009 20:20 29.346 0.144 -20.654 0.219 12/05/2009 20:30 29.320 0.122 -20.680 0.197 12/05/2009 20:40 29.251 0.122 -20.749 0.197 12/05/2009 20:50 29.330 0.117 -20.670 0.192 12/05/2009 21:00 29.269 0.101 -20.731 0.176 12/05/2009 21:10 29.279 0.139 -20.721 0.214 12/05/2009 21:20 29.281 0.115 -20.719 0.190 12/05/2009 21:30 29.242 0.132 -20.758 0.207 12/05/2009 21:40 29.264 0.117 -20.736 0.192 12/05/2009 21:50 29.213 0.129 -20.787 0.204 12/05/2009 22:00 29.172 0.124 -20.828 0.199 12/05/2009 22:10 29.107 0.130 -20.893 0.205 12/05/2009 22:20 29.074 0.117 -20.926 0.192 12/05/2009 22:30 29.067 0.147 -20.933 0.222 12/05/2009 22:40 28.978 0.158 -21.022 0.233 12/05/2009 22:50 28.913 0.127 -21.087 0.202 12/05/2009 23:00 28.486 0.158 -21.514 0.233 12/05/2009 23:10 28.532 0.149 -21.468 0.224 12/05/2009 23:20 28.578 0.177 -21.422 0.252 12/05/2009 23:30 28.565 0.165 -21.435 0.240 12/05/2009 23:40 28.747 0.152 -21.253 0.227 12/05/2009 23:50 28.818 0.146 -21.182 0.221 13/05/2009 00:00 28.884 0.128 -21.116 0.203 13/05/2009 00:10 28.942 0.167 -21.058 0.242 13/05/2009 00:20 28.909 0.159 -21.091 0.234 13/05/2009 00:30 28.994 0.074 -21.006 0.149 13/05/2009 00:40 28.974 0.128 -21.026 0.203

The website will also show graphical plots of the measurements made by each participant as a function of time. These plots will include the actual brightness temperature of the target, as measured by the test radiometer, as shown in Figure 3.The website will also include measurements made by every participant together plotted together in the same Figure. Figure 4 shows an example of the measurements of four continuously measuring radiometers during the 2009 comparison. Figure 5 shows the difference of the four continuously measuring radiometers from their mean value. Similar plots will be produced for the measurements made by radiometers during the 2016 comparison.

Figure 3: Plots of the WST measurements made by a particular participant radiometer, as shown in this Figure, will be given on the website for all field measurements completed during the 2016 comparison. Measurements are shown for two angles of view, 25o and 55o to the nadir.

29.4 MAERI ISAR 29.2 KIT 29 RAL

28.8 )

C 28.6 o (

S 28.4 S

27.6 12/05/2009 12/05/2009 13/05/2009 13/05/2009 13/05/2009 13/05/2009 13/05/2009 14:24 19:12 00:00 04:48 09:36 14:24 19:12 Time (UT) Figure 4: Plots of the measurements of the four continuously measuring radiometers made during the 2009 Miami comparison. Similar plots will be plotted for the measurements made during the 2016 comparison.

0.4 MAERI ISAR 0.3 KIT

C RAL o

/ 0.2 n a e 0.1 m

n -0.1 e r e f f

i -0.2 D -0.3

-0.4 12/05/2009 12/05/2009 13/05/2009 13/05/2009 13/05/2009 13/05/2009 13/05/2009 14:24 19:12 00:00 04:48 09:36 14:24 19:12 Time (UT)

Figure 5: Difference of the four continuously measuring radiometers from their mean value.

The radiometers do not provide continuous field measurements for a number of reasons. Moreover, measurements are taken at different times, so linear interpolation will be used to calculate the values the measurements of each radiometer at 5 minute intervals. The output of radiometers which have to be operated manually has to be treated differently. Figure 6 shows plots of the measurements made by all the manually operating radiometers during the 2009 comparison. Similar plots corresponding to measurements made during the 2016 comparison will also be included in the website. Finally the difference of all the field measurements made by all the radiometers (both continuously monitoring and manual radiometers) from the mean will also be presented. Figure 7 shows the difference of the measurements made by every radiometer taking part from the corresponding measurements made by ISAR during the 2009 SST radiometer comparison. The ISAR radiometer was chosen because it was the only radiometer which produced continuous measurements during the entire duration of the 2009 SST comparison.

29.5 8 to 14 um 29.3 8.7 um 29.1 8.3 um 9.1 um 28.9 10.65 um C o

/ 11.35 um e 28.7 r

u RA1 t

a 28.5 r RA2 e p 28.3 RA3 m e DLR data T 28.1

27.9 27.7

27.5 12/05/200 13/05/200 13/05/200 14/05/200 14/05/200 15/05/200 15/05/200 16/05/200 9 12:00 9 00:00 9 12:00 9 00:00 9 12:00 9 00:00 9 12:00 9 00:00 Time (UT)

Figure 6: Plots of the measurements made by all the manually operating radiometers during the 2009 comparison. Similar plots corresponding to measurements made during the 2016 comparison will be included in the new website.

0.40 MAERI KIT )

C RAL o ( 0.00 R CEAM RA1 A S

I CEAM RA2

m CEAM RA3 o -0.40 r f GOTA 8to14 e c

n GOTA 8.7 e r

e -0.80 GOTA8.3 f f i GOTA9.1 D GOTA10.65 -1.20 GOTA DLR

-1.60 12/05/2009 12/05/2009 13/05/2009 13/05/2009 13/05/2009 13/05/2009 13/05/2009 14:24 19:12 00:00 04:48 09:36 14:24 19:12 Time (UT)

Figure 7: Difference of the measurements made by every radiometer from the corresponding measurement made by ISAR during the 2009 SST radiometer comparison.

5 BLACKBODY COMPARISON

5.1 TECHNICAL DESCRIPTION OF THE BLACKBODIES

A. Blackbody used for the comparison E.g. Land Infrared Landcal Blackbody Source P80P. (http://landinstruments.net/infrared/products/calibration_sources/p80p.htm).

B. Technical description of the radiometer E.g. the radiometer was made of Aluminium and had a black, high temperature refractory coating. The body of the radiometer was cylindrical with a 50 mm diameter and 155 mm in length. The blackbody cavity had a 120o cone at the closed end and a cavity emissivity was estimated to be greater than 0.995. The temperature of the cavity is measured using an internal Platinum Resistance Thermometer (PRT) whose output was displayed on a digital display with 0.01 K resolution. The temperature of the blackbody cavity was also measured using an external PRT-100 which was calibrated traceable to National Standards (UKAS calibration certificate available).

C. Establishment of traceability route and breakdown of uncertainty:

The following error analysis is based on laboratory measurements (April 6 and 7, 2016).

Type A uncertainty: - Repeatability: 0.01% or 0.03 K (corresponding to variations of 0.01 Ω in external PRT-100 measurements). - Reproducibility: 0.01% or 0.03 K (corresponding to variations of 0.01 Ω in external PRT-100 measurements). Total Type A uncertainty (RSS): 0.015% or 0.04 K

Type B uncertainty - Emissivity: Uncertainty due to emissivity <0.005 (value provided by the blackbody manufacturer). This corresponds to a temperature error of 0.3 K in the 8 m to14 m wavelength range. - Blackbody thermometer calibration: 0.1 K (External PRT calibration at the ice point). Also comparison with an identical blackbody from the University of La Laguna, Spain, which has a UKAS certificate. - Blackbody isothermal variance: 0.3 K (value provided by manufacturer). - Primary source: 0.6 K. Primary source calibration is not available for this blackbody. The assigned value is the same for the identical University of La Laguna blackbody (calibrated with UKAS certificate) corresponding to the uncertainty (coverage factor k=1) of the standard radiation thermometer used in the blackbody calibration for temperature range from 10 oC to 30 oC.

Total Type B uncertainty: 0.7 K.

Combined Type A and Type B uncertainty: 0.7 K.

D. Operational methodology during measurement campaign

E.g. the blackbody was set to a fixed temperature (ice point at 0.00 oC) and sufficient time was allowed for the source to reach the fixed point. Temperatures were read from the internal thermometer, the external PRT-100 and another available PRT. An ice point apparatus was used to calibrate the external PRTs.

E. Uncertainties associated with the Landcal P80P blackbody

The table 5 shown below shows the uncertainty budget for the brightness temperature of the test blackbody. Uncertainty contributions include Type A uncertainty contributions (repeatability and reproducibility) as well as Type B uncertainty contributions, blackbody emissivity, stability, PRT calibration, temperature variations in the cavity bottom and cylindrical wall.

5.2 REPORTING OF PARTICIPANTS’ MEASUREMENTS AND RESULTS ON BLACKBODIES

The brightness temperature of the participating blackbodies should be provided by the participants on the form shown in Appendix A of this document. The measurements should also be provided in an excel file, in the form shown in Table 6. These results will be available on the project’s website along with the corresponding measurements of the brightness temperature of the blackbodies made by the NPL AMBER reference radiometer and PTB broadband reference radiometers. Table 7 shows how the brightness temperature of a participant blackbody made by AMBER, the NPL reference radiometer, and by the PTB broadband radiometer will be reported on the website.

Table 6: Format with which the participants should report the brightness temperature of their blackbodies, along with the combined uncertainty of each measurement. The table caption will include the date of the measurement (e.g. 23rd June 2016)

Time from start Blackbody Temperature Combined Uncertainty Minutes K K

0.500 11.075 0.034 1.501 11.101 0.036 2.501 11.128 0.032 3.502 11.154 0.036 4.502 11.181 0.032 5.503 11.207 0.034 6.503 11.234 0.035 7.504 11.261 0.033 8.504 11.288 0.035 9.505 11.315 0.038 10.505 11.342 0.032 11.506 11.369 0.036 12.506 11.396 0.034 13.507 11.423 0.033 14.507 11.451 0.034 15.508 11.478 0.036

Table 7: Format with which the measured brightness temperature of the test blackbody measured by the reference radiometers during measurements of the test blackbodies. The combined uncertainty of each measurement will also be provided. The table caption will include the date of the measurement (e.g. 23rd June 2016)

Time from start Blackbody Temperature Combined Uncertainty minutes K K

0.500 11.089 0.054 1.501 11.115 0.053 2.501 11.141 0.053 3.502 11.168 0.054 4.502 11.194 0.052 5.503 11.221 0.053 6.503 11.248 0.055 7.504 11.275 0.058 8.504 11.301 0.056 9.505 11.329 0.053 10.505 11.356 0.052 11.506 11.384 0.052 12.506 11.414 0.059 13.507 11.438 0.054 14.507 11.465 0.058 15.508 11.493 0.052

The same measurements will also be shown in plots as a function of time. Figure 8 shows the values of brightness temperature measured by AMBER when it was monitoring the cavity of a test blackbody during the 2009 comparison. The average value of the measurements made by AMBER will also be given (9.836 oC in this example). The corresponding value of the blackbody brightness temperature provided by the participant will also be plotted on the same Figure, as a function of time. In the example shown in Figure 8, the participant stated that the brightness temperature of the test blackbody was 10.000 oC at all times during the period of the test. The difference between the average value of the blackbody brightness temperature during the monitoring period and the average value measured by the reference radiometer will also be given (0.164 oC in the case of the data shown in Figure 3) to indicate the “error” with which the blackbody “reads” its own brightness temperature at that particular temperature setting.

GOTA BB at 10 oC, 1st run, mean=9.836 oC, Difference=-164 mK 9.846

9.842 ) C o (

9.84 e r u t

a 9.838 r e p 9.836 m e T 9.834

9.83 0 2 4 6 8 10 12 Time (min) Figure 8: Plot of the brightness temperature values of the test blackbody, as measured by the AMBER radiometer, when the test blackbody temperature was set to 10 oC. The average blackbody brightness temperature measured by the reference radiometer was 164 mK lower than the set value.

The website will also show tables which summarise the differences/“errors” of the measurements made by the reference radiometer and the brightness temperature values of the test blackbodies reported by the participants. Table 8 shows an example of such a Table from the 2009 radiometer comparison which was completed at NPL [4]. Note that there will be two sets of Tables 6, 7 and 8. The first set will correspond to the measurements made by AMBER, the NPL reference radiometer, while the second set will correspond to the measurements made with the PTB broadband infrared radiometer. Similarly, there will be two version of Figure 3; the first version will correspond to the measurements made by AMBER, the NPL reference radiometer, while the second version will correspond to the measurements made with the PTB broadband infrared radiometer on the test blackbody under identical conditions.

Table 8: Difference between the brightness temperature of a blackbody provided by the participant and the temperature of the same blackbody measured by the AMBER radiometer at different blackbody set temperatures in the 5 oC to 30 oC temperature range.

Set temperature Temperature "error" Temperature "error" Participant oC 21st April run 22nd April run mK mK

RAL 30 14 6 SISTeR BB 20 -8 -5 10 -15 -14

Southampton 30 -7 3 ISAR BB 20 -16 -14 10 -19 -18

GOTA 30 -176 -188 La Laguna Univ. 20 -152 -181 Canary Island 10 -164 -177

DEPT 30 -167 -185 Valencia University 20 -143 -166 LAND P80P 10 -74 -87

6. REFERENCES

1. Barton, I. J., Minnett, P. J., Maillet K. A., Donlon, C. J., Hook, S. J., Jessup, A. T. and Nightingale, T. J., 2004,” The Miami 2001 infrared radiometer calibration and intercomparison: Part II Shipboard results”, Journal of Atmospheric and Oceanic Technology, 21, 268-283.

2. Rice, J. P., Butler, J. I., Johnson, B. C., Minnett, P. J., Maillet K. A., Nightingale, T. J, Hook, S. J., Abtahi, A., Donlon, and. Barton, I. J., 2004, ”The Miami 2001 infrared radiometer calibration and intercomparison. Part I: Laboratory characterisation of blackbody targets”, Journal of Atmospheric and Oceanic Technology, 21, 258-267.

3. Theocharous, E., Usadi, E. and Fox, N. P., “CEOS comparison of IR brightness temperature measurements in support of satellite validation. Part I: Laboratory and ocean surface temperature comparison of radiation thermometers”, NPL REPORT OP3, July 2010.

4. Theocharous E. and Fox N. P., “CEOS comparison of IR brightness temperature measurements in support of satellite validation. Part II: Laboratory comparison of the brightness temperature comparison of blackbodies”, NPL Report COM OP4, August 2010.

5. Theocharous, E. and Fox, N. P., 2016, “Fiducial Reference Measurements for validation of Surface Temperature from Satellites (FRM4STS) - Laboratory Calibration of Participants Radiometers and Blackbodies”, Protocol for the FRM4STS LCE (LCE-IP), ESA Contract No. 4000113848_15I-LG, NPL report OFE-D-90A-V1-Iss-1-Ver-1

6. Theocharous, E., Fox, N. P., Sapritsky, V. I., Mekhontsev, S. N. and Morozova, S. P., 1998, “Absolute measurements of black-body emitted radiance”, Metrologia, 35, 549-554.

APPENDIX A: REPORTING OF MEASUREMENT RESULTS

The attached measurement summary should be completed by each participant for each completed set of laboratory measurements. A complete set being one, which may include multiple measurements on, or using the same instrument but does not include any realignment of the instrument. For each realignment a separate measurement sheet should be completed.

For clarity and consistency the following list describes what should be entered under the appropriate heading in the tables.

Time The time of the measurements should be UTC.

Measured Brightness temperature Brightness temperature measured or predicted by participant.

Measurement uncertainty Combined/total uncertainty of the measurement.

Uncertainty The total uncertainty of the measurement of brightness temperature separated into Type A and Type B. The values should be given for a coverage factor of k=1.

Wavelength This describes the assigned centre wavelength used for the measured brightness temperature. For the case of Fourier Transform spectrometers, the wavelength range and wavelength resolution should be specified.

Bandwidth This is the spectral bandwidth of the instrument used for the comparison, defined as the Full Width at Half the Maximum.

Standard Deviation The standard deviation of the number of measurements made to obtain the assigned brightness temperature without realignment

Number of Runs The number of independent measurements made to obtain the specified standard deviation.

Measurement Laboratory Results: Blackbody Comparison

Instrument Type ...…… …………………………. Identification No ………………………….

Date of measurement: …………………………… Ambient temperature …………………….

Blackbody BB Brightness Time of measurement Brightness Temperature Uncertainty (UTC) Temperature Uncertainty K mK A % B

Participant: …………………………………………………………………………………… Signature: …………………………….. Date: ……………………………

Measurement Laboratory Results: Radiometer Comparison

Instrument Type ...…… …………………………. Identification No ………………………….

Date of measurement: …………………………… Ambient temperature …………………….

Measured Combined No. Time of Brightness Measurement Wave- Band- Uncertainty of measurement Temperature Uncertainty length width (UTC) K mK m nm A % B Runs

APPENDIX B: REPORTING OF MEASUREMENT RESULTS

The attached measurement summary should be completed by each participant for each completed set of WST field measurements. A complete set being one, which may include multiple measurements on, or using the same instrument but does not include any realignment of the instrument. For each realignment a separate measurement sheet should be completed. A separate measurement sheet should also be completed if a different view angle from nadir, or a different wavelength or bandwidth is used by the same radiometer.

For clarity and consistency the following list describes what should be entered under the appropriate heading in the tables.

Time The time of the measurements should be UTC.

Measured Water Brightness temperature measured or predicted by participant. Surface Temperature

Measurement uncertainty Combined/total uncertainty of the measurement.

Measured Sky Temperature Brightness sky temperature measured or predicted by participant.

Uncertainty The total uncertainty of the measurement of brightness temperature separated into Type A and Type B. The values should be given for a coverage factor of k=1.

Wavelength This describes the assigned centre wavelength used for the measured brightness temperature. For the case of Fourier Transform spectrometers, the wavelength range and wavelength resolution should be specified.

Bandwidth This is the spectral bandwidth of the instrument used for the comparison, defined as the Full Width at Half the Maximum.

Standard Deviation The standard deviation of the number of measurements made to obtain the assigned brightness temperature without realignment

Number of Runs The number of independent measurements made to obtain the specified standard deviation.

View angle from Nadir The angle of view of the radiometer to the surface of the water from Nadir.

WST Measurement Results at Wraysbury Reservoir Instrument Type ...………… Identification Number ……… Ambient temperature …………

Date of measurement: …………………… View angle from nadir (degrees)………………

Wavelength (µm) …………………………… Bandwidth (µm) …………………………….

Measured Combined Measured Uncert. in No. Time WST WST sky sky Uncertainty of (UTC) Uncertainty temperature temperature K K K K A % B Runs

LST Measurement Results on the at NPL site

Instrument Type ...………… Identification Number ……… Ambient temperature …………

Date of measurement: …………………… View angle from nadir (degrees)………………

Wavelength (µm) …………………………… Bandwidth (µm) …………………………….

Measured Combined Measured Uncert. in Target Uncert. Time LST LST sky sky Uncertainty No. emissivity in (UTC) Uncertaint. temperat. temper. of used emissiv. K K K K A % B Runs

-END OF DOCUMENT-

NPL - Commercial