This is a repository copy of Significance testing testate amoeba water table reconstructions. White Rose Research Online URL for this paper: http://eprints.whiterose.ac.uk/99925/ Version: Accepted Version Article: Payne, RJ, Babeshko, KV, van Bellen, S et al. (17 more authors) (2016) Significance testing testate amoeba water table reconstructions. Quaternary Science Reviews, 138. pp. 131-135. ISSN 0277-3791 https://doi.org/10.1016/j.quascirev.2016.01.030 © 2016, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/ Reuse Unless indicated otherwise, fulltext items are protected by copyright with all rights reserved. The copyright exception in section 29 of the Copyright, Designs and Patents Act 1988 allows the making of a single copy solely for the purpose of non-commercial research or private study within the limits of fair dealing. The publisher or other rights-holder may allow further reproduction and re-use of this version - refer to the White Rose Research Online record for this item. Where records identify the publisher as the copyright holder, users can verify any specific terms of use on the publisher’s website. Takedown If you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing [email protected] including the URL of the record and the reason for the withdrawal request. [email protected] https://eprints.whiterose.ac.uk/ 1 Significance testing testate amoeba water table reconstructions 1,2 2 3 4 5 2 Richard J Payne , Kirill V Babeshko , Simon van Bellen , Jeffrey J Blackford , Robert K Booth , Dan J 3 Charman6, Megan R Ellershaw7, Daniel Gilbert8, Paul D M Hughes9, Vincent EJ Jassey8,10 Ł 4 Lamentowicz11, Mariusz Lamentowicz11, Elena A Malysheva2, Dmitri Mauquoy12, Yuri Mazei2,13, 5 Edward A D Mitchell14,15, Graeme T Swindles16, Andrey N Tsyganov2, T Edward Turner16, Richard J 6 Telford17 7 1 Environment, University of York, Heslington, York, YO105DD, United Kingdom. 8 2 Department of Zoology and Ecology, Penza State University, Krasnaya str. 40, 440026 Penza, 9 Russia. 10 3 GEOTOP-Université du Québec à Montréal, 201 Avenue Président-Kennedy, Montréal, Québec, 11 H2X 3Y7, Canada. 12 4 Department of Geography, Environment and Earth Sciences, University of Hull, Cottingham Road, 13 Hull, HU6 7RX, United Kingdom. 14 5 Department of Earth and Environmental Sciences, Lehigh University, 1 West Packer Avenue, 15 Bethlehem, PA 18015-3001, USA. 16 6 Geography, College of Life and Environmental Sciences, University of Exeter, Amory Building, 17 Rennes Drive, Exeter, EX4 4RJ, United Kingdom. 18 7 Natural England, Northminster House, Peterborough, PE1 1UA, United Kingdom. 19 8 Laboratoire Chrono-Environnement, UMR 6249 Université de Franche-Comté/CNRS, UFR Sciences 20 et Techniques, 16 route de Gray, 25030 Besançon, France. 21 9 Geography and Environment, University of Southampton, University Road, Southampton, SO17 22 1BJ, United Kingdom. 23 10 Swiss Federal Institute for Forest, Snow and Landscape Research (WSL), Site Lausanne, Station 2, 24 1015 Lausanne, Switzerland. 25 11 Department of Biogeography and Palaeoecology, Faculty of Geographical and Geological Science, 26 Adam Mickiewicz University, Poznan, ul. Dziegielowa 27, 61-680 Poznan, Poland. 27 12 School of Geosciences, University of Aberdeen, Elphinstone Road, Aberdeen AB24 3UF, United 28 Kingdom. 29 13 Department of Hydrobiology, Lomonosov Moscow State University, Leninskiye gory, 1, Moscow 30 119991, Russia. 31 14 Laboratory of Soil Biology, University of Neuchâtel, Rue Emile-Argand 11, CH-2000 Neuchâtel, 32 Switzerland. 33 15 Jardin Botanique de Neuchâtel, Pertuis-du-Sault 58, CH- 2000 Neuchâtel, Switzerland 34 16 School of Geography, University of Leeds, Leeds, LS2 9JT, United Kingdom. 35 17 EECRG, Department of Biology, University of Bergen, Allegaten 41, N-5007 Bergen, Norway. 36 ABSTRACT 37 Transfer functions are valuable tools in palaeoecology, but their output may not always be 38 meaningful. A recently- TF the potential to distinguish 39 among reconstructions which are more likely to be useful, and those less so. We applied this test to 40 a large number of reconstructions of peatland water table depth based on testate amoebae. 41 Contrary to our expectations, a substantial majority (25 of 30) of these reconstructions gave non- 42 significant results (P>0.05). The underlying reasons for this outcome are unclear. We found no 43 significant correlation between randomTF P-value and transfer function performance, the properties 44 of the training set and reconstruction, or measures of transfer function fit. These results give cause 45 for concern but we believe it would be extremely premature to discount the results of non- 46 significant reconstructions. We stress the need for more critical assessment of transfer function 47 output, replication of results and ecologically-informed interpretation of palaeoecological data. 48 KEYWORDS: Testate amoeba; Protist; Palaeoecology; Palaeohydrology; Transfer function; randomTF 49 INTRODUCTION 50 Testate amoebae are widely-used proxies in palaeoecological studies; in particular for the 51 reconstruction of water table depth in peatlands (Charman, 2001; Mitchell et al., 2008). Over the last 52 25 years palaeoecology has been revolutionised by the use of statistical models (transfer functions) 53 to quantitatively reconstruct environmental variables. However, questions are increasingly being 54 raised about the reliability and robustness of transfer function results (Belyea, 2007; Juggins, 2013). 55 A transfer function will always give an output but that output may not always be meaningful. The 56 only way to establish whether the output of a transfer function is s by comparing the results 57 to independent data, but such data are not always available and even in such cases correlations are 58 complicated by temporal autocorrelation and the limitations of the chronology. 59 Although we cannot realistically assess whether all reconstructions are correct we can conceivably 60 test whether they are potentially useful. Telford and Birks (2011) propose a pragmatic solution: that 61 a reconstruction can be considered statistically significant if it explains more of the variance in the 62 fossil data than those of transfer functions trained on randomly-generated data. Telford and Birks 63 (2011) propose a method, TF, in which: 64 1. The transfer function is applied to the fossil data to derive a reconstruction (using any 65 commonly-applied method). 66 2. The proportion of variance in the fossil data explained by the reconstruction is determined 67 using constrained ordination. 68 3. Multiple new transfer functions are derived using the established modern species data but 69 with the environmental data replaced by uniformly distributed random variables. 70 4. These transfer functions are applied in turn to the fossil data and the variance they explain 71 tested. This is repeated a large number of times, typically 999. 72 5. A reconstruction is considered statistically significant when the proportion of variance 73 explained is greater than that of 95% of the transfer functions based on randomly-generated 74 data. 75 We would expect reliable reconstructions to explain more variance in the fossil data than transfer 76 functions trained on random data, and therefore to give significant results. However, a significant 77 randomTF value is not proof of accuracy and a non-significant result does not necessarily imply 78 inaccuracy. Non-significant results do however give cause for concern and suggest that transfer 79 function output should be treated with caution. randomTF tests can potentially tell us which 80 reconstructions we should trust more, which less, and whether we can predict more than one 81 environmental variable from the same fossil dataset. Telford and Birks (2011) also propose an 82 alternative test based on the correlation of optima values with axis species scores from a 83 constrained ordination of the fossil data. This test is not applicable to all transfer functions methods 84 and is not considered here. The randomTF test has been applied in a few studies (Amesbury et al., 85 2013; Lamarre et al., 2013; Swindles et al., 2015a) but is not yet routinely used in testate amoeba 86 palaeoecology. Here we apply this test to a large number of published and unpublished records with 87 the aim to identify the characteristics which are likely to lead to better reconstructions, giving better 88 randomTF results. 89 METHODS 90 We identified 30 published and unpublished testate amoeba palaeoecological records (Table 1). 91 These records span a large range of regions, mire types, analysts, time periods, and sampling 92 resolutions, and form a large and reasonably representative sample of testate amoeba 93 palaeoecological research. Reconstructions of water table depth were produced using either the 94 transfer function used in the original study, the most geographically-appropriate model where a 95 transfer function was not previously applied, or in a few cases transfer functions which have been 96 produced since the data were originally published. Taxonomy was harmonised between the fossil 97 data and training set, which in many instances required the grouping or deletion of some taxa 98 (performance statistics may therefore differ slightly from those previously published). Transfer 99 functions were applied based on the model selected by the original authors with