Expert Assessment of TLSB’s monitoring proposal for Atlantic salmon and sea trout, as set out in the Adaptive Environmental Management Plan (AEMP) Prof. Carlos Garcia de Leaniz, Dr. Luca Börger, Dr. Jim Bull, Dr. Sofia Consuegra Swansea University, College of Science Department of BioSciences Singleton Park, Swansea SA2 8PP Swansea 17 June 215 1 In relation to the Tidal Lagoon Swansea Bay development, this report comprises 1. A critical assessment of the power analysis done by THA on behalf of TLSB 2. An outline of some forecasting simulations , along with some pilot results, to address some of the question being asked in the AEMP 3. An outline of alternative ways of monitoring that would better fulfil some of the AEMP’s objectives 2 1. Critical assessment of the power analysis done by THA on behalf of TLSB In their Adaptive Environmental Management Plan (Revision 4 of 25 November 2014), and then on Document No. 580N0601 dated 20 February 2015, THA presented the results of a power analysis based of annual rod catches for salmon and sea trout in the rivers Tawe, Afan and Neath which were claimed to indicate that 3 years of monitoring might be enough to detect an impact on rod catches. Some relevant statements of these two reports are reproduced below: The proportional representation data was less variable because the proportion that any one year made up of the total count for the period of record is likely to be more consistent from year to year than the raw rod count for many reasons, e.g. ocean conditions . We disagree with these statements which are, in our view, flawed. There are several problems with the ‘proportional’ power analysis done, namely: 1. Proportions are bound between 0 and 1, they will always be less variable than raw data. This has nothing to do with data ‘reliability’, and even les so with ‘ocean conditions’ 2. As used, the yearly proportional contribution to total catch will ALWAYS decrease as one increases the time series, as shown in simulations (Figures 1-2). In fact, the variability in proportional representation approaches zero as the number of years in the time series increases, which will give biased estimates if used in power analysis. 3. It is unclear how one would assess a ‘15% reduction’: For how long? Just on one year?... A catch at least 15% below the mean or lower could happen c. 41% of the time just by chance alone, as shown by the rod catch data! (Table 1). 4. A power test does not really address the question being asked, which is (or should have been) how can we be sure that if an impact occurs we can detect it 3 Table 1. Rod catch for Atlantic salmon in the three study rivers. A catch 15% below the series mean (indicated by red circles) has happened on 5 years (45% of the time) in the River Afan, 3 years on the River Neath (27% of the time) and 4 years (36% of time) in the River Tawe. For a stationary population, one would expect a 15% or greater reduction in the series mean to occur with c. 41% probability on any given year entirely by chance alone (i.e. in the absence of any additional impact). 4 400 0.10 0.09 0.08 0.07 300 0.06 0.05 0.04 SD fish counts SD fish 200 0.03 0.02 SD Yearly proportions SD Yearly 0.01 100 0.00 0 20 40 60 80 100 120 0 20 40 60 80 100 120 No. YEARS in time series No YEARS in time series Figure 1. Random catch data simulated from 0 to 100 years, SD calculated for both raw data and proportional yearly representation for time series varying from 0 to 100 years. While the magnitude of the standard deviation remains constant when fish counts are used (left panel), it decreases sharply when proportional representation are employed (right panel) even though one uses exactly the same data. Tawe For count data, the mean is equal to the variance, 20 contrary to ‘Gaussian’ (normally distributed) data. 15 E(X) = Var(X) = λ. 10 Thus, statistical methods, including power calculations, need to be adapted for count data 5 Afan Neath 20 SD (fish counts) SD (fish 15 10 5 11 30 50 70 90 100 11 30 50 70 90 100 No. YEARS in time series Figure 2. Simulations of sampled standard variation with increasing number of years in the time series for Atlantic salmon rod catches in the rivers Tawe, Afan and Neath. For raw data, sampling uncertainty decreases with sample size, as expected. 5 Figure 3. Simulations of sampled standard variation with increasing number of years in the time series for log transformed Atlantic salmon rod catches in the rivers Tawe, Afan and Neath. As with raw data, uncertainty in the standard deviation of log–transformed data also decreases with sample size. To carry out a correct power analysis, one would need to take the following into consideration: 1. A power analysis relies crucially on having correct estimates of variance/SD. 2. The correct mean-variance relationship is violated when counts or proportion data are used in a power analysis based on parametric assumptions for Gaussian data, as was done in the power analysis by THA. 3. The assumption of constant/stationary population dynamics is violated – there may also be nonlinear trends and the spatial structure (source sinks) and age structure of the populations need to be considered. 6 4. A power analysis is not really suitable for deciding for how long one would need to monitor a fishery to detect an impact on rod catches. 5. A better approach to detect a reduction in catch data would be to carry out simulations on count data and estimate how likely a certain reduction is likely to be observed. This leads to more modern forecasting simulation methods, presented next. 7 2. An outline of some forecasting simulations , along with some pilot results, to address some of the question being asked in the AEMP Figure 4. Generalized Additive Modelling (GAM) of rod catch trends through time, 2001 - 2011. Shown are cubic smoothing splines with a log-link function and Poisson likelihood. Red ribbons depict 95% confidence intervals. Figure 5. Autocorrelation functions (ACF) of trends through time, 2001 - 2011. Shown are second-differenced time series used to stabilize quadratic trends, blue dotted lines represent 95% confidence intervals. 8 Analysis of the short –term (2001-2011) rod catch data by GAM (Fig 4) suggests: 1) Recent trends are far from linear, making prediction very hard. 2) The width of the confidence intervals suggests that the variance is approximately stable over time. On the other hand, examination of autocorrelation plots (Fig 5) indicates that: 3) There is little evidence of temporal autocorrelation in detrended time series. 4) Holt-Winters forecasting might provide valid, if weak, forecasts Figure 6. Holt-Winters forecasting for 10 years beyond the short-term time series (2001- 2011). Black line shows mean catch, red line shows 15% reduction in mean catch, blue line shows most likely forecast, mid-blue ribbon shows 80% prediction intervals, light grey-blue ribbon shows 95% prediction intervals. Results of Holt-Winters forecasting in the short–time series (Fig 6) indicates that 1) Existing time series contain nowhere near enough information to forecast. 2) Additional sources of data are needed to make valid predictions. Analysis of more extensive data catch is examined next: 9 Figure 7. Holt-Winters forecasting for 10 years beyond the long-term time series (1990- 2013). Black line shows mean catch, red line shows 15% reduction in mean catch, blue line shows most likely forecast, mid-blue ribbon shows 80% prediction intervals, light grey-blue ribbon shows 95% prediction intervals. Figure 8. Autocorrelation functions (ACF) of trends through time, 1990 – 2013. Shown are second-differenced time series used to stabilize quadratic trends, blue dotted lines represent 95% confidence intervals. Negative auto-correlations (red circles) at lag=2 may be indicative of density dependence processes where the abundance of one year class may depress the abundance of the next one. 10 Figure 9. Partial autocorrelation functions (PACF) of trends through time, 1990 – 2013. This shows the relationship of rod catches to those of preceding years after statistically partialing out the influence of intervening years. Dotted lines are 95 CI. Negative auto-correlations (red circles) continue to be evident mostly at lag=2 , which may be indicative of density dependence processesCross Correlation. Plot Cross Correlation Plot Salmon Sea trout 1.0 1.0 0.5 0.5 0.0 0.0 Correlation Correlation -0.5 Cross Correlation Plot -0.5 Cross Correlation Plot Tawe-Neath Tawe-Neath -1.01.0 -1.01.0 -20 -10 0 10 20 -20 -10 0 10 20 Lag Lag 0.5 0.5 0.0 0.0 Correlation Correlation -0.5 Cross Correlation Plot -0.5 Cross Correlation Plot Tawe-Afan Tawe-Afan -1.01.0 -1.01.0 -20 -10 0 10 20 -20 -10 0 10 20 Lag Lag 0.5 0.5 0.0 0.0 Correlation -0.5 Correlation -0.5 Neath-Afan Neath-Afan -1.0 -1.0 -20 -10 0 10 20 -20 -10 0 10 20 Lag Lag Figure 10. Cross-correlation plots (CCF) of trends through time, 1990 – 2013 showing the relationship of rod catches between pairs of rivers a various time lags. Red envelopes represent 95 CI. Significant CCF values in some pairwise comparisons suggest that rod catches should not be assumed to be independent.