Submission to Senate Inquiry

“The identification of leading practices in ensuring evidence-based regulation of farm practices that impact water quality outcomes in the Great Barrier Reef”

Dr. Lachlan Stewart

Lachlan Stewart (PhD) is the Principal of Science Audit Service, a consultancy providing advice on the strengths or shortcomings of published scientific work. He holds the following tertiary qualifications: BSc (Mathematics), Class 1 Honours (Earth Science) and PhD (Earth Science), and his contributions to science have been recognised by the inclusion of his biography in Marquis Who’s Who in the World.

Most of his career has been spent in the employment of James Cook University (JCU) and the CSIRO. He has held various paid research roles at JCU within the then departments of Physics (Radar Oceanography), Mathematics (Spinor and Tensor research software), Earth Sciences (Cooperative Research Centre Task 1.3.1/2 Sediment Accumulation and Dynamics on the Great Barrier Reef (GBR)), and Civil and Systems Engineering (Crown of Thorn Starfish dispersal modelling) and this experience has given him a very broad base of scientific understanding.

Within the CSIRO he was heavily involved in the modelling and monitoring of sediment and nutrient delivery to the GBR lagoon from the Tully and Murray River catchments in North Queensland. In addition, he was also involved in the modelling of sugarcane crop growth and nitrate delivery to ground water under sugarcane cropping in the Burdekin delta.

He is an author on 18 scientific papers published in international and national science research journals and numerous reports.

Executive Summary

Science Quality Assurance

Based on my experience in auditing Great Barrier Reef (GBR) related science it is my opinion that there is a marked problem with the standard of quality assurance provided by the peer review process. I have conducted partial audits of two Great Barrier Reef related scientific papers that call for increased environmental regulation of industry, and both contained claims that were demonstrably incorrect. In addition, in preparing this submission I have detected a major flaw in the planning of the Queensland and Federal Government’s Reef 2050 Water Quality Improvement Plan (WQIP) that could have major implications for reef ecosystem health (see below). I believe that there should be established an independent Office of Science Quality Assurance (see Appendix 1) to audit scientific work before it is considered as a basis for policy or regulation.

In addition to issues with Quality Assurance I wish to address what I see as a tendency in the scientific literature to put forward alarming or dire interpretations of scientific data. Within scientific institutions there is an old adage, “publish or perish”. The “citation impact” and the number of scientific publications that a scientist produces will contribute towards any assessment for career progression.

The more a scientist’s publications are cited by other scientists, or the media, then the greater their chances for career progression and receiving research funding. This, I believe, sets up an unconscious bias in scientists to look for interpretations of their data that will garner the most impact or attention. Unfortunately, gloom and doom interpretations are the most likely to achieve this end.

Only two percent of GBR reefs are exposed to nutrients and suspended sediment from river runoff, and such exposure is transient.

It is widely accepted that river borne suspended sediment and nutrient will primarily affect inshore reefs only during the high flow events of the wet season, and that such exposure will last from a couple of days to perhaps three weeks. These inshore reefs represent approximately two percent of total reefal area.

The impact of exposing inshore reefs to transient, high concentrations of nutrients is likely to be, at worst, “generally sub lethal and subtle”.

Results of the benchmark ENCORE experiment that exposed in-situ GBR patch reefs to high concentrations of nutrients for 6 hours a day, every day for 430 days found the impacts to be “generally sub-lethal and subtle and the treated reefs at the end of the experiment were visually similar to control reefs.”

In contrast to the treated ENCORE patch reefs the inshore reefs of the GBR adjacent to agricultural areas are likely to spend, annually, between 94.2 to 99.5 percent of days in ambient, low-nutrient water conditions.

Crown of Thorns Starfish (COTS, Acanthaster planci)

A numerical model used to support the case for water quality influences on COTS outbreaks employs a statistical model that does not represent the experimentally observed high rates of survivability of COTS larvae in low chlorophyll conditions. The modelling should be repeated using an appropriate statistical model.

There is a major flaw in the Queensland and Federal Government’s Reef 2050 Water Quality Improvement Plan.

In the quest to improve water quality for corals and seagrass the WQIP targets reducing agriculture derived nutrient inputs to the GBR lagoon by in excess of 50% by 2025 within four out of six Queensland regions. Incredibly, the WQIP does not consider the wider ecosystem consequences to the food web of the proposed reductions in nutrient inputs. These consequences were investigated and detailed in a CSIRO report that describes food-web modelling conducted under the National Research Flagship – Water for a Healthy Country. The CSIRO found that the food-web implications of reducing nutrient inputs to the GBR by just 10% over 20 years were that populations of many marine species, including those of protected animals, would be reduced by up to more than 50% in some coastal areas. This would be a conservation disaster. Nowhere in the Reef Scientific Consensus Statement upon which the WQIP is based has the CSIRO report been considered or discussed. The WQIP should be suspended while the CSIRO report and its implications are properly considered.

Body of Submission

Office of Science Quality Assurance

Science is currently in the midst of a replication crisis. Revelations from the scientific literature indicate that major systemic failings exist in science Quality Assurance (Ioannidis, 2005, 2014; Larcombe and Ridd, 2018). Authors and industry have reported the frequency of irreproducibility of scientific results at around 50% (Hartshorne and Schachner, 2012; Vasilevsky et al., 2013; The Economist, 19/10/2013; Larcombe and Ridd, 2018). That is, around 50% of published scientific results are not reproducible. This situation creates a significant impediment to decision making for those looking to incorporate scientific findings into policy, industry activities or investment decisions. The utilisation of invalid science represents risk in terms of inappropriate policy and wasted expenditure.

There are certainly Quality Assurance issues surrounding published environmental science in Australia. I have performed partial audits upon two GBR related scientific papers that call for increased regulation of the Australian fishing industry and found profound errors in both.

The first (Edgar et al., 2018) asserts that rapid declines have occurred across Australian fishery stocks, including the GBR. The graphs displayed in the Edgar et al., 2018 and displayed in Figure 1 claim to show statistically significant declines in large fish biomass in ‘Limited fishing’ and ‘Open access’ fishing areas around Australia.

However, the decline in ‘Open access’ areas is clearly not statistically significant because the confidence band (red shaded area) can contain the null hypothesis (i.e., a line where the population slope = 0, Fig. 3). This is a fundamental concept in statistics and this error should have been detected in peer review. Edgar et al. (2018)’s paper received national media attention that created public concern and it was also, I understand, discussed heatedly in the Senate. An independent Office of Science Quality Assurance (Appendix 1) auditing or checking policy-relevant scientific papers could have detected this error very early and avoided the propagation of unnecessary public concern and debate.

Figure 1. (Modified after Edgar et al., 2018) Best fit trends in the total biomass of large fishes in differing fishing zones around Australia. The 95% confidence bands are shown by shading. In the case of the “Open access” chart, it is clear that a line of zero slope (the green line that I have added) is contained in the confidence band. This indicates a lack of statistical significance for the trend (Yamane, 1967).

The second, Purcell et al. (2016), called for increased regulation of the sea-cucumber fishing industry. One of the several arguments put forward to support their case was that sea cucumbers excrete materials that result in an increase in pH and total alkalinity of surrounding water, and therefore may buffer at local scales the effects of ocean acidification and facilitate coral calcification.

This is an erroneous claim, the work they cite (Schneider et al., 2011, 2013) actually shows the complete opposite; that the presence of sea-cucumber will decrease the local pH of surrounding water

(i.e., it will become more acidic). Schneider et al. (2013) expressly state that “The CaCO3 saturation state in the incubation seawater decreased markedly due to a greater increase in dissolved inorganic carbon (DIC) relative to total alkalinity (AT) as a result of respiration by the animals.“ When the CaCO3 saturation state in seawater decreases so does the carbonate ion concentration and this reduces the rate of calcification of marine organisms (Hoegh-Guldberg et al., 2007).

It may seem surprising that the erroneous claim and diagram detailed in Fig. 2 can make it past four well credentialed authors and three peer reviewers active in the field. Especially in a paper calling for increased regulation and control of an industry where such recommendations, if implemented, would impose unnecessary costs, red tape and potential losses of livelihood. Clearly, there is evidence that all is not well in the environmental sciences and that there is a need for an independent Office of Science Quality Assurance (Appendix 1).

Figure 2. (a) Reproduction of Fig. 3 from Purcell et al., (2016), and (b) Figure 3 of Purcell et al. (2016) modified to correctly portray the potential influence of the animals upon local seawater CaCO3 saturation state and pH.

In addition to the above, in the brief time available to prepare this submission, I have detected a serious flaw in the rationale behind the WQIP (see below) and what I perceive to be an issue with the statistical model used to inform COTS Coral modelling of Fabricius et al. (2010, see COTS section), a paper that is often cited by scientists in regard to water quality impacts. Published scientific papers should not be automatically assumed sound and error free. I believe government has a duty to ensure that the highest standards of science are being met in any scientific publication that may form the basis for the implementation of policy. The content of any scientific paper being so considered should undergo a rigorous science audit process through an independent Office of Science Quality Assurance. Unnecessary regulation based on invalid science reduces the competitiveness of Australian industry and wastes public funds.

Terrigenous Sediment and Nutrient impacts upon the GBR

Sediments

On the impacts of sediment and nutrient upon the Great Barrier Reef – More than 2900 reefs comprise the Great Barrier Reef (Hopley, 1989) some lying up to 160 km off shore. Sediment transport dynamics within the GBR lagoon were studied extensively under Task 1.3.1/2 (Sediment accumulation and dynamics), Cooperative Research Centre (CRC) for the Ecologically Sustainable Development of the Great Barrier Reef, upon which I worked as a Research Fellow. We found that sediment, fine and coarse, delivered to the GBR lagoon by rivers did not pose a threat to most coral reefs. We found that sediment largely settles out of river waters as they enter the marine waters of the GBR lagoon and into a nearshore sediment wedge (store). Then by a combination of prevailing South Easterly winds, Northward longshore current, and wave induced resuspension the finer sediments are carried northward along the coast (Larcombe and Woolfe, 1999). These sediments are eventually deposited in northward facing embayments that are protected from wave action thereby allowing the sediment to settle out of suspension (Larcombe and Woolfe, 1999). Thus the inner-shelf and fringing reefs are the primary areas affected by terrigenous (land derived) sediment delivery.

We found it highly unlikely that under normal conditions any significant amount of terrigenous sediment could be moved offshore beyond the 20m isobath, the limit of wave induced resuspension under the typical annual wave regime (Orpin et al., 1999). Figure 3, created from the analysis of seabed sediment samples collected near the mouth of the Herbert River in North Queensland (Woolfe et al, 2000), shows how land derived sediment dominates out to approximately 20m depth and beyond that depth the seafloor sediment was dominated by a biogenic carbonate (i.e. derived from broken pieces of shell, coral) sand.

Mid to outer shelf reefs typically occur in waters of 50m depth. If significant amounts of contemporary land derived sediment could somehow get onto the mid and outer shelf reef platforms we would expect to find evidence of a geological history of delivery to those areas. I am not aware of a single study demonstrating this to be the case. Indeed Devlin et al. (2001), describing the entry of river flood water into the marine zone, note that “In the initial mixing zone, water velocity is reduced and changes in salinity, pH and eH promote flocculation of particulate matter. Most of the river-derived particulate matter settles from the plume in this zone. This is most clearly shown in the results from the Burdekin for cyclone Sid where suspended solid and particulate phosphorus concentrations drop to very low levels only a few kilometres from the river mouth at salinity of approximately 10. However benthic sediment distribution information shows that the area off the mouth of the Burdekin River has a low proportion of fine sediments. This apparent inconsistency is best explained by the resuspension and northward transport and deposition in northerly facing bays of fine sediments which occurs throughout the year under the influence of the south-east wind regime on the inner shelf. Reductions in suspended sediment with increasing salinity in the plume are less clear in some of the other plumes but this is complicated by resuspension during the plume event in stronger wind conditions on these occasions.” While terrigenous sediment is not found on reef platforms it should be noted that it can be found off the shelf in the Queensland Trough, but to date the delivery mechanism has not been definitively established.

Terrigenous Biogenic Figure 3. Modified after Woolfe et al.(2000). (a) Contours show the number of days per year upon which the critical bed shear stress for sediment resuspension is exceeded offshore from the Herbert River mouth. (b) A simplified cross-section along a shore normal sampling transect relating the depth and grainsize distributions of samples, and origin of sediment.

The highest concentrations of sediment and nutrient in river waters entering the GBR lagoon occur in flood plumes during periods of riverine high-flow normally associated with the tropical wet season (Devlin et al. 2001, 2010). Thus, the areas of the Great Barrier Reef primarily affected by flood waters carrying high concentrations of nutrients and sediment are nearshore inner-shelf areas that host fringing, incipient fringing and nearshore reefs. Using the reef type and area data of Hopley, et al., (1989), we find that fringing and incipient fringing reefs represent approximately 3.3 % of the total reefal area. Smithers et al.,(2006) proposed that, while difficult to estimate, the number of inshore reefs probably exceeds 100. Assuming these inshore reefs have approximately the same mean area as fringing reefs (1.0 km2) and that we generously assume that 200 exist then we arrive at an estimate of at most 4.3% of the total reefal area. With only 50% of those inshore reefs near agricultural areas the proportion of GBR reefal area that can be directly affected by high concentrations of sediment and nutrient during river plumes is approximately 2%. This is not the popular perception.

Nutrients

Devlin et al. (2001) studied river plumes and sediment, nutrient and pesticide delivery to the GBR lagoon during periods of riverine high flow and noted that high levels of sediment and nutrient delivery were predominantly confined to inshore areas. Flood plumes with high concentrations of nutrients are documented reaching a small portion of mid shelf reefs. While visually spectacular these plumes also contain little in the way of land derived sediment (e.g., Orpin et al., 1999, Devlin et al., 2001). They are typically short lived and can have return periods of years between occurrences (Devlin et al., 2001).

Using data obtained from inshore waters around the Frankland Island reefs (Round, Russell and Normandy Islands), High Island, Goold Island, Pandora Reef and Keppel Islands Devlin et al. (2001) found that these reefs were exposed to NH4, NO3 and DIP concentrations between 1 to 8 µM, 2 to 9 µM and 0.1 to 2.5 µM. Long term ambient concentrations are 0–0.01 µM, 0.1–0.4 µM and 0.1–0.15 µM, respectively (Devlin et al, 2001; Furnas & Brodie, 1996). Using these reefs as an example they state that they may be exposed to high levels of suspended sediment and nutrients for periods of days to several weeks in the wet season.

The results of the ENCORE study (Koop et al., 2001), the only in-situ study of the impacts of nutrient supplementation of GBR reefs, is used as evidence that exposure to elevated levels of nutrients can impact coral and in support of arguments to reduce nutrient flux to the GBR lagoon. The ENCORE study exposed insitu coral-patch-reefs to elevated levels of nutrients at every low tide for a period of 30 months.

In the initial 465 day low loading phase patch reefs were exposed to an average initial concentration of

11.45 M of NH4-N that gradually depleted over the low-tide period, sometimes quite rapidly, to near ambient levels (0.91+/- 0.79 M). After the 465 day low loading phase completed the patch reefs were then exposed to a high loading phase for a further 430 days. In this phase peak concentration of NH4-N over the low tide period averaged 36.21 +/-21.87 M with an average concentration of 11.3 +/- 10.2

M NH4-N at the end of the low tide. The average exposure period was 6 hours per day. In both loading phases statistically significant impacts were detected, but by far the majority were associated with the high-loading phase.

I must address here a semantic point that can lead to some confusion for the non-scientist reading a scientific report. Used in everyday conversation the word “significant” can mean “large”, for example, “He received a significant pay rise”. The word “significant” can have a different meaning in scientific use. It can be used to indicate the detection of a subtle, non-obvious effect or response by a statistical test. For example, the sentence “The study found a significant increase in coral mortality in response to elevated nutrients.” can indicate the detection of a subtle increase in the level of coral mortality rather than a large die-back.

So in reports we often see sentences like this, for example, from Devlin et al. (2001) describing the ENCORE results “…there was [sic] significant biotic responses, include [sic] coral mortality, stunted coral growth with increase [sic] nitrogen and reduced skeletal density with increase [sic] P.”, which is perfectly normal scientific jargon but would sound rather bleak to the non-scientist. The authors of the ENCORE study noted that, “Impacts were dependent on dose level, whether Nitrogen and/or Phosphorus were elevated and were often species specific.” They also went on to say that in relation to the overall extent and severity of the detected impacts, “The impacts were generally sub-lethal and subtle and the treated reefs at the end of the experiment were visually similar to control reefs.” So, even where corals were exposed twice daily to nutrient enriched waters for about 6 hours per day, every day for 30 months the detected impacts were “generally sub-lethal and subtle”.

That notwithstanding, I am presently unconvinced that it is even a valid proposition for scientists to relate the impacts observed on corals over the extended ENCORE experiment, to corals receiving an annual, transient exposure to nutrient enriched waters of perhaps days to weeks. It does not seem valid to relate the impacts seen on corals that resulted from more than a year of twice daily exposure to highly nutrient enriched waters to inshore corals that will spend 94.2 to 99.5% of a year in ambient, low nutrient waters. This is analogous to suggesting that no matter how long you spend unprotected in the bright sun, whether it is two minutes or 5 hours, you will get the same sunburn.

The ENCORE experiment is however is the best information we have to go on in relation to the direct impacts of nutrients on coral because it was conducted under natural conditions rather than in a laboratory or aquarium where potentially important factors, such as the full range of wave activity and associated water turbulence, are not present.

Suspended Sediment and ambient GBR water quality

After evaluating the relative influence of river plumes and wave induced resuspension on suspended sediment concentrations (SSC) around reef communities Orpin and Ridd, (2012) found that the latter was the most dominant mechanism controlling water quality. They demonstrate that resuspension events are capable of generating higher ambient SSCs than those measured in flood waters. Wave induced resuspension of fine sediment is possible because at many nearshore areas there is an abundance of fine-grained bed sediment, consistent with millennial-scale geological evidence of sediment dispersal and deposition prior to European settlement and catchment impacts (Orpin and Ridd, 2012)

Once sediment enters the marine environment and settles out of suspension it can only re-enter suspension to create turbid water through either disturbance or wave/tide induced bed shear stresses. Resuspension is entirely dependent on the near-bed water velocity profile (e.g., Orpin et al., 1999).

Changing the amount of sediment delivered to the GBR lagoon cannot affect the amount of sediment resuspended because the resuspension energy comes from a finite amount of near-bed wave or tide energy and is not influenced by the depth of sediment present (Larcombe and Woolfe, 1999). Clearly, any reduction in the amount of sediment delivered to GBR lagoon is highly unlikely to impact ambient water quality in inshore areas. Even if the rivers could be switched off entirely the inshore waters of the GBR would remain turbid on windy days as the nearshore sediment deposits, accumulated over millennia and up to 30 m thick (Larcombe et al., 1996), will still be present and providing material for resuspension.

The wave regime during and immediately after a flood event may influence post event turbidity patterns up to the first strong wind event that remobilises and redistributes sediment that has accumulated near a river mouth. Fabricius et al., (2014) claim that seasonal river floods can impact water quality across the central GBR for months afterwards. However, their study warrants much deeper inspection for potential issues because the claim is inconsistent with drifter data that indicates that the GBR is continually flushed by Pacific Ocean waters, and that the residence time of water in the GBR is just a few weeks (Choukroun et al., 2010). In the southern GBR, it takes about 8 hours to flush to the Coral Sea the equivalent volume of water delivered by rivers in an entire year (Larcombe and Ridd, 2018).

Crown of Thorns Starfish Outbreaks

Fabricius et al. (2010), assert that there are three lines of evidence to link COTS outbreaks to water quality. These are: a) results from laboratory experiments where larvae were reared on natural phytoplankton, b) field data of river floods, chlorophyll concentrations and COTS outbreaks on the Great Barrier Reef, and c) results from COTS – coral population model simulations that examined the relationship between larval food availability and outbreak frequency.

They stated that experiments showed that at higher chlorophyll concentrations such as during flood plumes larval survivability was enhanced. However, the experiments also showed that high levels of larval survivability could occur at low chlorophyll concentrations. Of the 27 experimental results detailed by the authors 12 showed above zero levels of completion of development at 22 days. Of those 12, 25% were in water containing low values of chlorophyll a (<0.3 g L-1). In addition, water low in chlorophyll a provided two of the six highest survival rates at completion.

In developing their statistical model of the relationship between chlorophyll and the portion of larvae achieving successful completion, Fabricius et al.(2010) omit the data associated with the two highest rates of survivorship for low-chlorophyll a conditions, describing them as “severe outliers”. They offer no other justification for the omission of these low-chlorophyll a /high-survivorship data points.

Generally, outliers should be only omitted if there is a valid reason. For example, the U.S. National Institute of Standards and Technology website states:

“Outliers should be investigated carefully. Often they contain valuable information about the process under investigation or the data gathering and recording process. Before considering the possible elimination of these points from the data, one should try to understand why they appeared and whether it is likely similar values will continue to appear.“

Fabricius et al.(2010)’s omission of those two data points means that the statistical model informing their COTS-coral model did not include the experimentally derived observation that high levels of survivability of COTS larvae also occurs at low chlorophyll concentrations. In addition, there are field observations supporting the fact that primary outbreaks of COTS appear to have occurred under low chlorophyll conditions (i.e., at reefs not affected by flood plumes), for example Fairfax Island Reef (Miller et al. 2014) In the light of the omission of data relevant to the COTS-coral modelling, the simulations should be repeated using a statistical model that better represents the experimental results and incorporates the high survivorship of larvae at both high and low chlorophyll a concentrations.

Reducing Nutrient Inputs to the GBR

There is a glaring and serious flaw in the Queensland and Federal Government’s Reef 2050 Water Quality Improvement Plan (WQIP) to reduce riverine nutrient inputs to the GBR lagoon. The WQIP, based on the 2017 Scientific Consensus Statement – Land use impacts on Great Barrier Reef water quality and ecosystem condition, asserts that large reductions in the delivery of particulate nutrients and dissolved inorganic Nitrogen (DIN) to the GBR must be made by 2025 in order to improve coral and seagrass health (Table 1).

Table 1. After Reef 2050 WQIP. Regional end of catchment water quality targets for 2025.

However, the CSIRO stated in a report (Gerhke, 2007) prepared under the CSIRO National Research Flagships (Water for a Healthy Country Flagship) that food-web/ecosystem modelling indicated that a reduction of just 10% in nutrient inputs to the GBR lagoon over 20 years had dire implications for the populations of many species living in inshore coastal areas.

The CSIRO developed four ecosystem models for coastal areas adjacent to the natural resource management regions of the Wet Tropics, Burdekin, Fitzroy and Burnett-Mary. Their modelling indicated that populations of many species, including protected animals such as turtles, non-commercial and commercial fish and crustacean species, seabirds, and seagrass, would decline by large amounts with just a 10% reduction in nutrient inputs to coastal areas over 20 years (Gerhke, 2007). For example, Fig. 4, extracted from that report, shows that their modelling predicts the biomass (effectively population) of many species, to fall by in excess of 50% over 20 years in Burdekin region. This would be catastrophic from a conservation standpoint and also have grave implications for recreational and commercial fishers, tourism and other industries. Figure 4.

After Gehrke, 2007.

The obvious implication of the CSIRO report is that the current populations of inshore marine species are being supported by the contemporary levels of nutrient input to the GBR lagoon. It is frankly quite astounding to me that the CSIRO modelling, undertaken within a National Research Flagship, has not been acknowledged or considered in development of the WQIP. I speculate that the planners may have had tunnel vision in regard to coral reef and seagrass water quality and therefore did not consider the ecosystem wide implications of reducing nutrient inputs to the GBR lagoon. The WQIP is based on the 2017 Scientific Consensus Statement (SCS). Examining references detailed in the SCS it is readily seen that not a single chapter considered or cited the CSIRO report that, if correct, has profound implications for reef management strategies and overall reef ecosystem health. The current situation is analogous to a building engineer using a design that the CSIRO says is likely to lead to a building crumbling over 20 years. Any such engineer would be called to explain their decision making.

I believe the WQIP and further tightening of water quality regulation needs to be paused until this glaring deficiency in the plan is addressed. Without doing so, if the CSIRO modelling is correct, it is likely that our nearshore populations of protected and other marine species will be decimated by well- intentioned but misguided efforts in regard to water quality improvement. Additionally, some form of oversight in policy science is clearly needed, such as an independent Office of Science Quality Assurance (Appendix 1).

Should the WQIP continue without addressing this issue, then it is entirely possible that any associated population declines in marine species would be speculatively attributed to other causes such as over fishing, ecosystem collapse due to marine pollution, climate change, ocean acidification, etc.

References

Brodie, J.E. & Furnas, M. 1996, ‘Cyclones, river flood plumes and natural water extremes in 102 the central Great Barrier Reef’, pp. 367–375 in Downstream Effects of Land Use, eds. H.M. Hunter, A.G. Eyles and G.E. Rayment, Queensland Department of Natural Resources, Brisbane.

Choukroun S, Ridd PV, Brinkman R, and McKinna I, 2010. On the surface circulation in the western Coral Sea and residence times in the Great Barrier Reef. J. Geophys. Res., 115, C06013.

Devlin, M., McKinna, L., and Harkness, P., 2010. Annual Report, Flood Plume Monitoring for 2009/10. Reef Rescue Marine Monitoring Program: Terrestrial discharge into the Great Barrier Reef (project 3.7.2b). Department of the Environment, Water, Heritage and the Arts.

Devlin, M., Waterhouse, J., Taylor, ., and Brodie, J., 2001. Flood Plumes in the Great Barrier Reef: Spatial and Temporal Patterns in Composition and Distribution. Research Bulletin 68, GBRMPA. ISSN 1037-1508, ISBN 0 642 23097 8.

Edgar, G. J., Ward T. J., and Stuart‐Smith, R.D., (2018). Rapid declines across Australian fishery stocks indicate global sustainability targets will not be achieved without an expanded network of ‘no‐fishing’ reserves. Aquatic Conservation: Marine and Freshwater Ecosystems, Volume 28, Issue 6. https://doi.org/10.1002/aqc.2934

Fabricius, K.E., Okaji, K., De’ath, G., 2010. Three lines of evidence to link outbreaks of the crown-of- thorns seastar Acanthaster planci to the release of larval food limitation. Coral Reefs, 29(3), 593–605.

Hoegh-Guldberg, O., Mumby, P.J., Hooten, A.J., Steneck, R.S., Greenfield, P., Gomez, E., Harvell, C.D., Sale, P.F., Edwards, A.J. & Caldeira, K. 2007. Coral reefs under rapid climate change and ocean acidification. Science 318, 1737-1742.

Hopley, D., Parnell, K. E., Isdale, P. J., 1989. The Great Barrier Reef Marine Park: Dimensions and Regional Patterns. Australian Geographical Studies, Volume 27, Issue 1.

Ian Miller, Hugh Sweatman, Alistair Cheal, et al., “Origins and Implications of a Primary Crown-of- Thorns Starfish Outbreak in the Southern Great Barrier Reef,” Journal of Marine Biology, vol. 2015, Article ID 809624, 10 pages, 2015. https://doi.org/10.1155/2015/809624. Hartshorne, J.K., Schachner, A., 2012. Tracking replicability as a method of post-publication open evaluation. Front. Comput. Neurosci. 6, 1–13. http://dx.doi.org/10. 3389/fncom.2012.00001 PMID: 22291635.

Ioannidis, J.P.A., 2005. Why most published research findings are false. PLoS Med. 2 (8), e124 Retrieved from. http://www.plosmedicine.org/article/info:doi/10.1371/ journal.pmed.0020124.

Ioannidis, J.P.A., 2014. How to make more published research true. PLoS Med. 11 (10). http://dx.doi.org/10.1371/journal.pmed.1001747.

Koop, K., et al. 2001, ‘ENCORE: The effect of nutrient enrichment on coral reefs. Synthesis of results and conclusions’, Marine Pollution Bulletin, 42(2), 91–120.

Larcombe, P., Woolfe, K., and Purdon, R., 1996 (Editors). Great Barrier Reef: Terrigenous Sediment Flux and Human Impacts – Second Edition – November 1996. CRC Reef Research Centre, Current Research, Townsville; Australia, 174 pp.

Larcombe, P., and Woolfe, K.J., 1999. Increased sediment supply to the Great Barrier Reef will not increase sediment accumulation at most coral reefs. Coral Reefs 18, 163-169.

Larcombe, P., and Ridd, P.V., 2018. The need for a formalised system of Quality Control for environmental policy-science. Marine Pollution Bulletin 126, 449–461

Miller, I., Sweatman, H., Cheal, A., et al., “Origins and Implications of a Primary Crown-of-Thorns Starfish Outbreak in the Southern Great Barrier Reef,” Journal of Marine Biology, vol. 2015, Article ID 809624, 10 pages, 2015. https://doi.org/10.1155/2015/809624.

Orpin, A. and Ridd, P.V., and Stewart, L.K., 1999. Assessment of the relative importance of major sediment transport mechanisms in the central Great Barrier Reef lagoon. Australian Journal of Earth Sciences, 46, 883-896.

Orpin, A. and Ridd, P.V., 2012. Exposure of inshore corals to suspended sediments due to wave- resuspension and river plumes in the central Great Barrier Reef: A reappraisal. Continental Shelf Research, Volume 47, 55-67. https://doi.org/10.1016/j.csr.2012.06.013

Purcell, S.W., Conand, C., Uthicke, S., Byrne, M. (2016) Ecological roles of exploited sea cucumbers. Oceanography and Marine Biology: An Annual Review 54: 367-386

Schneider, K., Silverman, J., Kravitz, B., Rivlin, T., Schneider-Mor, A., Barbosa, S., Byrne, M. & Caldeira, K. 2013. Inorganic carbon turnover caused by digestion of carbonate sands and metabolic activity of holothurians. Estuarine, Coastal and Shelf Science 133, 217-223.

Schneider, K., Silverman, J., Woolsey, E., Eriksson, H., Byrne, M. & Caldeira, K. 2011. Potential influence of sea cucumbers on coral reef CaCO3 budget: a case study at One Tree Reef. Journal of Geophysical Research G: Biogeosciences 116, G04032 S. G. Smithers, D. Hopley, and K. E. Parnell. "Fringing and Nearshore Coral Reefs of the Great Barrier Reef: Episodic Holocene Development and Future Prospects." Journal of Coastal Research 22, no. 1 (2006): 175-87. http://www.jstor.org/stable/4300274.

Vasilevsky, N.A., Brush, M.H., Paddock, H., Ponting, L., Tripathy, S.J., et al., 2013. On the reproducibility of science: unique identification of research resources in the biomedical literature. PeerJ 1, e148. http://dx.doi.org/10.7717/peerj.148. 24032093.

Woolfe, K., Larcombe, P., and Stewart, L., 2000. Shelf sediments adjacent to the Herbert River delta, Great Barrier Reef, Australia. Australian Journal of Earth Sciences, 47, 301-308.

Yamane, T, 1967. Statistics; An Introductory Analysis. Harper & Row, New York and John Weatherhill, Inc., Tokyo, 919 pp.

Appendix 1 Office of Science Quality Assurance: Application to Great Barrier Reef Science.

Proposal by Mr Michael Kern, Manager, Invicta Cane Growers Organisation Ltd, Ayr and Dr Lachlan Stewart, Principal, Science Audit Service, Cairns

Executive Summary

The Problem: Some of the science upon which governments base expensive decisions is not as reliable as it needs to be. This problem is particularly acute for the science of the Great Barrier Reef where it is likely that some of the funds to save the reef will not be spent on the most urgent environmental problems. It is also possible that some legislation that is based on questionable science will result in little environmental benefit but will cause significant costs to industry.

The Solution:

Set aside less than 1% of the recently announced $500 million “Reef Rescue” funds to set up an “Office of Science Quality Assurance” that would do truly independent checks on GBR Science.

The Cost: Nil. Funds are already allocated.

It should be noted that the Dutch and US equivalents of the Australian Research Council, (Royal Netherlands Academy of Arts and Sciences, and the National Science Foundation) have both recently instituted policies to do science checking and replication studies.

Extended Summary

There is growing evidence that some of the scientific results upon which large public policy decisions are based, have serious flaws due to problems with the quality control methods being used in many scientific institutions. This proposal seeks to improve the reliability of the science upon which large public policy decisions are based (defined as Policy-Science) with a specific proposal for the science underpinning the plans to save the Great Barrier Reef (GBR). It should be noted that;

 Peer review is the primary Quality Control (QC) system for most Policy-Science. Although most of the public believe that peer review is an exhaustive process involving many scientists spending perhaps months checking results, it is in fact a very cursory process that may take a couple of reviewers only a few hours to complete.  Checks of peer reviewed work regularly find that around half is flawed. This remarkable fact, which has only come to light over the last decade, has become a major cause for concern in the mainstream scientific community. It is now called “The Replication Crisis”.  Far more rigorous QC processes than peer review should be used for policy decisions worth many millions or billions of dollars.  Unlike governments, industry rarely relies solely upon peer review for expensive decisions due to its unreliability.

In the next few years the Federal Government will spend around $500 million to save the GBR. It will be important to spend this money as efficiently as possible to maximise protection for the GBR.

 Checks already undertaken on a significant fraction of the science underpinning “reef rescue” indicate that there may be serious flaws in some of the results.  Further checks are thus advisable so that funds can be spent most efficiently and appropriately to save the GBR.  It is estimated that it would cost less than $5 million to institute replication tests on the most important scientific results.  This represents just 1% of “reef rescue” funds.  We propose that an Office of Science Quality Assurance (OSQA) should be commissioned to undertake this task. It will be essentially a scientific auditing organisation.  The OSQA should be independent of the scientific organisations whose work it will check.  The OSQA should be run through the Auditor General’s office as that organisation, although not familiar with science, understands how to ensure audits are independent.

Background

The Science “Replication Crisis” and “Policy-Science”

Since the early beginnings of Science in the time of the ancient Greeks, the scientific method has completely revolutionized human existence and almost always for the better. Science has progressed by constant checking, replication, argument and improvement. In some areas of science, such as Newton’s laws of motion, checks are effectively done billions of times every day when people fly in a plane, drive a car or walk across a bridge. Newton’s laws of motion are so well tested, checked and replicated that we stake our lives on them. But some science results are not massively validated in this way and are thus not as reliable.

Here we focus on the extent to which “Policy-Science” is checked, tested and replicated. We define the term “Policy-Science” to mean all science used as the basis for making expensive or important decisions by governments to make and deliver their policies. The critical distinction between policy-science and the rest of science is the active use by government to make expensive and important decisions on behalf of the public.

The “Replication Crisis” is the recent revelation from a wide range of the scientific literature that there may be major systemic failing in science Quality Control (QC) (Ioannidis, 2005, 2014). Perhaps the most high-profile example comes from the biomedical sciences where in checks made on peer-reviewed science, around half of important papers are found to be wrong. Prinz et al (2011) of the German drug company Bayer found that 75% of the literature used for potential drug discovery targets is unreliable. This issue has come to some international prominence:

“A rule of thumb among biotechnology venture-capitalists is that half of published research cannot be replicated. Even that may be optimistic. Last year researchers at one biotech firm, Amgen, found they could reproduce just 6 of 53 “landmark” studies in cancer research. (The Economist, 19/10/2013).

Other authors have reported the frequency of irreproducibility at around 50% (Hartshorne and Schachner, 2012; Vasilevsky et al, 2013). It has also been suggested that false or exaggerated findings in the literature are partly responsible for up to 85% of research funding resources being wasted (Chalmers and Glasziou, 2009; Ioannidis, 2014; Macleod et al., 2014). Despite replication studies being fundamental to establishing science reliability, such studies are rarely funded, and are not generally seen as a way of advancing a scientific career (Ioannidis, 2014).

A concern over reproducibility is shared by editors of major journals. Marcia Angell, a former editor of the New England Journal of Medicine, stated

“It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine.” (Angell, 2009).

The editor of The Lancet stated that

“The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.” (Horton, 2015).

The financial costs of irreproducible biomedical research are significant. Freedman et al (2015) estimated that the cumulative prevalence of irreproducible preclinical research exceeds 50%, and, in the United States alone, results in approximately US$28 billion per annum spent on research that is not reproducible. In the light of the replication crisis it would be prudent to examine whether similar problems occur in the environmental sciences, and in particular for the policy science for the Great Barrier Reef. Indeed, a call for “organized skepticism” to improve the reliability of the environmental marine sciences has already been made by Duarte et al. (2015) and Browman (2016). In particular, Duarte et al. (2015) argue that some of the major threats to ocean ecosystems may not be as severe as is portrayed in some scientific accounts, and that

“the scientific community concerned with problems in the marine ecosystem [should] undertake a rigorous and systematic audit of ocean calamities, with the aim of assessing their generality, severity, and immediacy. Such an audit of ocean calamities would involve a large contingent of scientists coordinated by a global program set to assess ocean health.”

What Quality Control Processes are used in Science?

Peer Review in Science

Peer review is the primary QC procedure used for most science results including for policy- science. The public often think that peer review involves a long and thorough process where perhaps a dozen scientists work for months to check another scientist’s results. But in reality, peer review may only be a quick read of the work for a few hours by a couple of anonymous referees selected by a journal editor. This is clearly not sufficient QC upon which to base decisions that may be worth millions of dollars. The cursory nature of peer review is at least partly responsible for the Replication Crisis and some eminent scientists are very scathing about its effectiveness. For example, an editor of The Lancet, Horton (2000), commented that

“the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong’”

Whilst this may read as an over-dramatic appraisal, the general point is clear - it is hardly a credible system upon which to base important public policy decisions worth millions or billions of dollars

“Industry Science” QC processes.

Industry and private enterprise rarely use solely peer review as its QC process. The possibility of considerable financial losses arising from an ill-informed decision is likely to drive a far more rigorous analysis to check the data and replicate results. Consider a pharmaceuticals company wishing to take a promising laboratory discovery to produce a new prescription drug for the market. On average, this costs 2.5 billion US dollars (figures for 2014) and takes a decade or more (DiMasi et al., 2016), so drug companies take great care at the beginning of a programme to make sure the initial information upon which they are basing the investment is sound. It is not accepted that a peer-reviewed journal article is adequate, partly because when checks are made, the original work is found to be wrong at least half the time (Prinz et al, 2011), and so by identifying the errors early, any waste of resources is minimised.

Engineering is far more advanced than most sciences in developing rigorous QC processes – indeed there are international standards that describe such processes in great detail. One reason that Engineering is generally more reliable than the sciences is that when an Engineer makes a mistake, there is the possibility of loss of life or massive financial loss. Consider the hypothetical case of an engineer who claimed to have invented an advanced metal alloy that would improve aircraft jet engine performance. An aircraft engine manufacturer would not regard a peer reviewed publication on the metal alloy as sufficient evidence to immediately use the metal alloy in passenger aircraft. The consequences of failure are disastrous and life threatening. Instead the engine manufacturer would subject the metal alloy to very stringent quality tests, under the guidance of strict regulations, and after many years it might ultimately be certified to use the metal for general use. This process is far more stringent and reliable than peer review.

The differences between the governments innocent trust in peer review and the far more rigorous systems of QC used by industry and engineering is remarkable. Given the level of government spending and significance of associated policy decisions, we believe that governments need to subject policy-science to greater scrutiny than the standard peer-review processes.

Great Barrier Reef Policy-Science

Both the Queensland and Australian Governments have already spent considerable sums on the Great Barrier Reef region, including AUD $375 million between 2008 to 2013 (Reef Water Quality Protection Plan Secretariat, 2013) and are expected to spend a further AUD $575 million in water quality initiatives between 2015 and 2020 (Great Barrier Reef Water Science Taskforce, 2015; Great Barrier Reef Marine Park Authority & Queensland Government (2015); Kroon et al., 2016). More recently the Commonwealth government has announced another $500 million to be spent on reef rescue. These costs do not include those borne by industry in meeting environmental legislation or the opportunity costs of preventing some forms of development in GBR river catchments or at the coast. Such costs are difficult to estimate, but by itself, the government expenditure of AUD $1 billion or more warrants rigorous scrutiny of the science well beyond peer review.

Significant concerns over some GBR policy-science

Some of the most highly cited policy-science papers have been examined (Larcombe and Ridd, 2018) which have asserted damage to the GBR. Some of these policy-science papers make very significant claims about the health and the future of the GBR system, including dire predictions of the imminent demise of the GBR. These papers make a wide suite of conclusions directly relevant to policy, which our analysis (Larcombe and Ridd, 2018) indicates should be viewed with some doubt. These include:

(a) Riverine discharge is significantly increasing GBR water turbidity.

(b) Nutrients from agricultural runoff are largely responsible for Crown-of-Thorns starfish plagues.

(c) Pollution from agricultural runoff is affecting the species diversity of the reef.

(d) There was a 50% reduction in coral cover in the GBR from the early 1960’s to 2000.

(e) There was a 14% reduction in coral growth rates between 1990 and 2005.

(f) Coral cover will fall to 5%-10% by 2022.

(g) The outer and inner GBR are 28% and 36%, respectively, down the path to ecological extinction.

Although there are a large number of papers which claim some degree of ‘stress’ on the GBR system, associated with increased fluvial loads, dredging, higher temperatures, lower pHs, higher chlorophyll concentrations and other parameters, the number of papers which assert to document a measurable decline of the GBR system’s coral is very small. It is not suggested that all of this small number is erroneous, but in order to make reliable decisions, we must first determine what science is sound and what is not.

For the GBR, there are a range of perceived ‘threats’ and a limited financial capacity of governments and industry to address the problems. It is likely that some threats are far more important than others, and there should be carefully focussed expenditure on ‘remediation’. However, in the light of the replication crisis the risk is considerable that the present focus of remediation efforts is misdirected.

The Science Checking/Audit Process

Credible science is consistently reproducible and based on sound reasoning, in consideration of all possibilities, to explain observed phenomena. The audit of a published scientific work would involve checking that:

 prior works described in the scientific paper are correctly referenced and paraphrased;  appropriate study design and statistical methods have been employed;  all sources of error have been accounted for and considered in the interpretation of results;  confounding influences and alternative interpretations have been considered in the paper’s discussion;  the study’s authors are confident enough in their work to release the data that they have collected, using public funds, to audit scrutiny; and  the results have been independently replicated, preferably on many occasions

Detailed analysis of the science thus checked/audited will be made publicly available. These detailed reports will also be supplemented by a “report card” style summary relating performance in the areas described above.

Funding GBR Policy Science checking

For the GBR, we estimate that it would cost less than $5 million to do initial checks of many of the main scientific results upon which the “reef rescue” policies are based. This is a very small sum relative to the funds that have been allocated to save the reef such as the latest Commonwealth government allocation of $500 million. Thus only 1% of allocated funds would need to be spent on checking the underlying science. This checking could be undertaken under the oversight of the proposed new Office or alternatively be undertaken independently as a one-off project. We believe that after the checking has been completed, there would very likely be a reconsideration of the priorities from threats that are less important to those where a greater improvement in reef health can be achieved. The return on the 1% investment in terms of environmental benefit is likely to be very large.

Underpinning “reef rescue” policies is the 2017 Scientific Consensus Statement: Land use impacts on Great Barrier Reef water quality and ecosystem condition. This document plays a significant role in influencing policy decisions and is therefore a deserved primary focus of scrutiny and auditing of the science upon which it is based.

The 2017 Scientific Consensus Statement considers the impact of sediment, nutrients, and pesticides, sourced from land use, upon the Great Barrier Reef. Therefore, we will divide the overall audit of the underpinning science into the same three distinct areas of focus and also include a fourth area of focus, calcification. Coral calcification rates are connected to ocean alkalinity which in turn is believed to be related to atmospheric CO2 levels, the latter being a major policy discussion point.

Costing

Scrutiny and testing of the underlying science in each focus area will proceed, and is costed, in the following manner.

Stage 1

Identify the important scientific papers that underpin the science in each of the focus area. The progression of science based knowledge is typically an incremental process. Once science is published other scientists working in the field may take the results as given and build upon the initial premise. Therefore, an audit focus on these “foundation” papers is essential.

The 2017 Science Consensus Statement, and references found therein, would be a good starting point to determine the keystone scientific papers to be tested. It is envisaged that there will be between 10 and 20 major keystone papers.

Focus area Duration Positions Salary/oncosts Operating Total (months) (EFT) All 3 1 $200K $10K $210K

Many of the personnel for this task would already be employed in GBR science and management organisations and be lead authors of the 2017 Science Consensus Statement. A small number of the OSQA staff would be required to coordinate the decision-making process of this stage.

1 FTE at $200k/annum including on-costs. This would be 4 senior scientists (working for 3 months) with a broad range of experience and who had a commitment to a rigorous review process.

Travel etc $10K.

Stage 2

Funding allocation, estimates according to resources requested. The cost estimates below represent rough estimates as they will depend upon the outcome of stage one. The detailed comments about specific scientific work speculates on the result of stage one. Much of the detail may therefore change but the costs are probably reasonably indicative.

Focus area Duration Positions Salary/oncosts Operating Total (months) (EFT) Sediment - Replication 6 2 $400K $100K $500K - and - Desktop Nutrient + COTS - Replication 12 1 $200K $200K $460K - Desktop 3 0.25 $ 50K $ 10K Pesticide - Replication - - Desktop 6 1 $100K $10K $110K Calcification - Replication 24 3 $1,200K $1,000K $2,200K - Desktop Totals $3,270K

Sediment: Scientists with a sedimentological and/or statistical background would analyse the conclusions that sediment is a major threat to the reef.

Replication studies could include a minor sampling effort to determine non carbonate sediment content on the offshore matrix of the GBR, i.e. how much mud is actually on the GBR?

Desktop studies would look at the recent publications by Fabricius, Weeks and other authors that claim elevated sediment concentrations on the GBR.

Nutrient

Replication studies would test the conclusions that nutrient (chlorophyll) concentrations are twice as much in the central GBR compared with the pristine north. A minor sampling effort would be carried out in Princess Charlotte Bay, the main embayment in the northern Zone. The sampling could be commissioned to a wide range of institutions.

Desktop study would concentrate on whether the 2017 Consensus Statement adequately reflects the literature. This would likely be done by scientists with broad experience in dealing with large data sets.

Pesticide

This would be largely a desktop study to reanalyse the pesticide data to test the conclusion that significant pesticide concentrations occur on the GBR. Attention would be given to data on the main reef matrix where much of the data already indicates pesticide levels below detection limits.

This would likely be done by scientists with broad experience in dealing with large data sets.

Calcification (growth rates) The most important paper on this topic from the Australian Institute of Marine Science (AIMS) claims a rapid reduction in calcification rates in recent times. However, there is an argument that the analysis and data is problematic. In order to test this a new replication of the sampling and measurement program would settle the argument. This is a very expensive program requiring collection of corals. AIMS is the main institution capable of this work and would likely be commissioned.

Publishing of results and general administration Publish the results to the web. Results must be published without deference to various academic institutions and industries and both groups must be prepared for good and bad news stories.

Administrative, field work backup, office costs and contingency:

Estimate 20% of stages 1-3. Approx. $600K

Grand total of expenditure: $4,080,000

7. References.

Angell, M (January 15, 2009). Drug Companies & Doctors: A Story of Corruption . The New York Review of Books.

Browman, HI (2016) Applying organised scepticism to ocean acidification research. ICES Journal of Marine Science, 73(3), 529–536. doi:10.1093/icesjms/fsw010

Chalmers I, Glasziou P. (2009) Avoidable waste in the production and reporting of research evidence. Lancet, 374: 86–89. Commonwealth of Australia (2015) Reef 2050 Long–Term Sustainability Plan.

DiMasi JA, Grabowski HG, Hansen RA (2016) Innovation in the pharmaceutical industry: new estimates of R&D costs. Journal of Health Economics, 47:20-33.

Duarte, C. M., Fulweiler, R. W., Lovelock, C. E., Martinetto, P., Saunders, M. I., Pandolfi, J. M., Stefan, G., et al. 2014. Reconsidering ocean calamities. Bioscience, 65: 130–139.

Great Barrier Reef Water Science Taskforce (2015) Great Barrier Reef Water Science Taskforce. Full Interim Report - December 2015. Clean water for a healthy reef. pp. 67. Brisbane, Australia, Queensland Government Department of Environment and Heritage Protection.

Hartshorne JK, Schachner A (2012) Tracking replicability as a method of post-publication open evaluation. Front Comput Neurosci 6: 1–13. doi: 10.3389/fncom.2012.00001 PMID: 22291635

Horton, R., 2015. “Offline: What is medicine’s 5 sigma?” Lancet, 385: 1380

Horton, R. (2000) Genetically modified food: consternation, confusion, and crack-up. Med. J. Aust. 172, 148–149.

Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. Retrieved from http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124

Ioannidis JPA (2014) How to Make More Published Research True. PLoS Med 11(10): doi:10.1371/journal.pmed.1001747

Kroon, FJ, Thorburn P, Scaffelke B and Whitten S (2016). Towards protecting the Great Barrier Reef from land based pollution. Global Change Biology 22, 1985-2002.

Larcombe, P., Ridd, P., 2018. The need for a formalised system of quality control for environmental policy-science. Mar. Pollut. Bull. 126, 449–461.

Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, et al. (2014) Biomedical research: increasing value, reducing waste. Lancet, 383: 101–104.

Prinz, F, Schlange, T. & Asadullah, K (2011) Believe it or not: how much can we rely on published data on potential drug targets? Nature Reviews Drug Discovery, 10, 712. 10.1038/nrd3439-c1 PMID: 21892149 Vasilevsky NA, Brush MH, Paddock H, Ponting L, Tripathy SJ, et al. (2013) On the reproducibility of science: unique identification of research resources in the biomedical literature. PeerJ 1: e148. doi: 10. 7717/peerj.148 PMID: 24032093