INTERNATIONAL JOURNAL OF CLIMATOLOGY, VOL. 16, 1197-1226 (1996) RESISTANT, ROBUST AND NON-PARAMETRIC TECHNIQUES FOR THE ANALYSIS OF CLIMATE DATA: THEORY AND EXAMPLES, INCLUDING APPLICATIONS TO HISTORICAL RADIOSONDE STATION DATA JOHN R. LANZANTE Geophysical Fluid Dynamics LaboratorylNOAA, Princeton University, Princeton, NJ 08542, USA email:
[email protected] Received 16 August 1995 Accepted 7 February I996 ABSTRACT Basic traditional parametric statistical techniques are used widely in climatic studies for characterizing the level (central tendency) and variability of variables, assessing linear relationships (including trends), detection of climate change, quality control and assessment, identification of extreme events, etc. These techniques may involve estimation of parameters such as the mean (a measure of location), variance (a measure of scale) and correlatiodregression coefficients (measures of linear association); in addition, it is often desirable to estimate the statistical significance of the difference between estimates of the mean from two different samples as well as the significance of estimated measures of association. The validity of these estimates is based on underlying assumptions that sometimes are not met by real climate data. Two of these assumptions are addressed here: normality and homogeneity (and as a special case statistical stationarity); in particular, contamination from a relatively few ‘outlying values’ may greatly distort the estimates. Sometimes these common techniques are used in order to identify outliers; ironically they may fail because of the presence of the outliers! Alternative techniques drawn from the fields of resistant, robust and non-parametric statistics are usually much less affected by the presence of ‘outliers’ and other forms of non-normality. Some of the theoretical basis for the alternative techniques is presented as motivation for their use and to provide quantitative measures for their performance as compared with the traditional techniques that they may replace.