An Evaluation of Citizen Science Smartphone Apps for Inland Water Quality Assessment

remote sensing

Article An Evaluation of Citizen Science Smartphone Apps for Inland Water Quality Assessment

Tim J. Malthus 1,* , Renee Ohmsen 2 and Hendrik J. van der Woerd 3

1 Coastal Sensing and Modelling Group, Coasts Program, CSIRO Oceans and Atmosphere, Ecosciences Precinct, 41 Boggo Road, Dutton Park, QLD 4102, Australia 2 School of Earth and Environmental Sciences, University of Queensland, St Lucia, QLD 4072, Australia; [email protected] 3 Institute for Environmental Studies (IVM), VU University Amsterdam, De Boelelaan 1111, 1081 HV Amsterdam, The Netherlands; [email protected] * Correspondence: [email protected]

Received: 28 March 2020; Accepted: 13 May 2020; Published: 15 May 2020

Abstract: Rapid and widespread monitoring of inland and coastal water quality occurs through the use of remote sensing and near-surface water quality sensors. A new addition is the development of smartphone applications (Apps) to measure and record surface reflectance, water color and water quality parameters. In this paper, we present a field study of the HydroColor (HC, measures RGB reflectance and suspended particulate matter (SPM)) and EyeOnWater (EoW, determines the Forel–Ule scale—an indication to the visual appearance of the water surface) smartphone Apps to evaluate water quality for inland waters in Eastern Australia. The Brisbane river, multiple lakes and reservoirs and lagoons in Queensland and New South Wales were visited; hyperspectral reflection spectra were collected and water samples were analysed in the laboratory as reference. Based on detailed measurements at 32 sites, covering inland waters with a large range in sediment and algal concentrations, we find that both water quality Apps are close, but not quite on par with scientific spectrometers. EoW is a robust application that manages to capture the color of water with accuracy and precision. HC has great potential, but is influenced by errors in the observational procedure and errors in the processing of images in the iPhone. The results show that repeated observations help to reduce the effects of outliers, while implementation of camera response functions and processing should help to reduce systematic errors. For both Apps, no universal conversion to water quality composition is established, and we conclude that: (1) replicated measurements are useful; (2) color is a reliable monitoring parameter in its own right but it should not be used for other water quality variables, and; (3) tailored algorithms to convert reflectance and color to composition could be developed for lakes individually.

Keywords: citizen science; smartphone; water quality; lakes; EyeOnWater; HydroColor; Australia

1. Introduction Surface freshwater is a ﬁnite resource essential for human and ecosystem existence. Adequate freshwater quality and quantity is required for sustainable development necessary for human consumption, irrigation, ﬁshing and recreational use [1,2]. Access to water with good quality is one of the 17 sustainable development goals [3]. Water quality is an important aspect of Australia’s freshwater resources and ongoing information about Australia’s water quality is vital for water resource management. Studies [4–6] have evaluated the contribution and application of remote sensing for providing freshwater monitoring of biophysical properties within the water column.

Remote Sens. 2020, 12, 1578; doi:10.3390/rs12101578 www.mdpi.com/journal/remotesensing Remote Sens. 2020, 12, 1578 2 of 20

Of all the biological, chemical and physical parameters of natural waters, the detection of the photosynthetic pigment chlorophyll a (CHL) is prominent. CHL is a molecule present within most algal groups, including cyanobacteria and is commonly used as a universal index for phytoplankton biomass [7]. If present in sufficient quantity, CHL has a very prominent fingerprint on the absorption and reflectance properties of water and can be detected by sensors that operate in the visual domain (400–720 nm) [8,9]. The two other components that leave an imprint on water reflectance are suspended particulate matter (SPM) and the colored fraction of dissolved organic matter (CDOM). SPM is a combination of algal (detritus) and non-algal (sediment) concentrations [10]. Even in small concentrations, SPM dominates the light availability (turbidity) in the water and the strength of the reflected optical signal above water [11,12]. Similarly, high concentrations of humic and fulvic acid compounds in the water strongly absorb blue light and give these waters a dark impression with a brown to yellow color [10,13]. To monitor the thousands of natural lakes, freshwater reservoirs, billabongs and river systems in Australia, a system is required that is able to access these waters with high spatial and temporal resolution. For example, harmful excess CHL levels in inland waters, also referred to as algal blooms, can develop in days to weeks and blooms can be patchy and localized (Figure1). In Australia, water quality monitoring is undertaken at the state and territory levels or by agencies such as catchment management authorities and local governments, private utilities and mining companies across privately owned waters and affected public waters. Coverage is patchy in both space and time, and is dependent on individual agency priorities and resources and uses variable methods; most importantly, public data availability is remarkably inconsistent [14]. At least four lakes (Atkinson, Dyer, Liddell and Einbunpin) reported on in this paper have experienced closure for water contact due to high toxic cyanobacteria levels. Automated or hand-held hyperspectral sensor systems may be used to derive water quality parameters and make a rapid assessment of harmful events, such as the presence of algal blooms [15]. In the last decade, multiple studies have shown that coupling of in situ spectral measurements to satellite data leads to an inland water monitoring system with high potential [16–22]. In Australia, the first steps towards such a system have been taken [23,24]. Wernand et al. [25] introduced a new parameter that can be retrieved accurately from the MERIS ocean color sensor—the Forel–Ule index (FUI). This FUI is a discrete number system developed at the start of the 20th century to quantify the color of water, as perceived by the human eye. In two subsequent articles, Van der Woerd and Wernand [26,27] introduced the hue angle as the quantitative and universal color parameters that can be derived from ocean color sensors and from moderate-resolution sensors such as Landsat-8 and Sentinel-2. In a ground-breaking article, Wang et al. [28] demonstrated that the worldwide monitoring of the eutrophic state of inland waters with the Moderate Resolution Imaging Spectroradiometer (MODIS) instrument is possible, based on the hue and the FUI. Recently, quantification of the color characteristics of inland waters by satellite-derived color has been applied at the national level for New Zealand [29] and Italy [30]. Remote Sens. 2020, 12, 1578 3 of 20

Remote Sens. 2020, 12, x FOR PEER REVIEW 3 of 20

Figure 1. (a) Lake Clarendon, with an example of surface scum caused by high concentrations of Figure 1. (a) Lake Clarendon, with an example of surface scum caused by high concentrations of cyanobacteria; (b) Lake Dyer and the impact of cyanobacteria on recreational sites, particularly how cyanobacteria; (b) Lake Dyer and the impact of cyanobacteria on recreational sites, particularly how it it impacts on aesthetic values. Varying levels of cyanobacteria in (c) Lake Liddell and (d) Lake Baroon. impacts on aesthetic values. Varying levels of cyanobacteria in (c) Lake Liddell and (d) Lake Baroon.

In 2014, 2014, the the EyeOnWater (E (EoW)oW) and HydroColo HydroColorr (HC) smartphone applications (Apps) became available to measure the color of water. The use of smartphones in the hands of concerned citizens to monitor water water quality quality may may provide provide a real-time a real- estimationtime estimation of water of quality, water complementing quality, compl laboratoryementing laboratorysample analysis sample and analysis the spectral and the systems spectral discussed systems discussed above and above meeting and meeting the need the for need greater for greater spatial spatialcoverage coverage [21]. In both[21]. Apps,In both the Apps camera, the on camera the smartphone on the smartphone is used to takeis used Red-Green-Blue to take Red-Green (RGB)- colorBlue (RGB)images color of the images water surface.of the w Modelsater surface. are then Models applied are tothen derive applied parameters to derive such parameters as hue angle, such the as FUI,hue angle,SPM or the turbidity. FUI, SPM or turbidity. The EoW AppApp (www.eyeonwater.org(www.eyeonwater.org)) was was developed developed in thein the European European Citclops Citclops project. project. It has It been has beencalibrated calibrated and testedand tested in Dutch in Dutch inland inland waters waters with with a large a large diversity diversity in composition in composition [31]. [31] The. HCThe App HC App(http: // (http://misclab.umeoce.maine.edu/research/HydroColor.phpmisclab.umeoce.maine.edu/research/HydroColor.php) was tested) was extensively tested extensively in both coastal in both and coastalinland USand waters inland [ 32US] andwaters British [32] Columbiaand British [ 33Columbia]. Besides [33] a limited. Besides test a limited in the Kesses test in Dam,the Kesses Kenya Dam, [34], Kenyaa systematic [34], acomparison systematic comparison of both Apps of has both not Apps been has published. not been published. In this this article article,, we we evaluate evaluate the the accuracy accuracy and and efficiency eﬃciency of EoW of EoW and HC and measurements HC measurements on a larger on a rangelarger range of water of water bodies bodies within within an Australian an Australian context. context. A set A set of of optical optical and and laboratory laboratory water measurements over a range of water qualities (including bloom conditions) from water bodies in South East Queensland and New South Wales was collected. The The accuracy and and precision of water color m measurementseasurements for water quality from the two smartphone applications for citizen science were referenced to to reﬂectance reflectance measurements measurements made made with with a standard a standard underwater underwater spectroradiometer. spectroradiometer. Finally, Finally,and where and appropriate, where appropriate, the potential the potential for water for color water measurements color measurements to derive to otherderive water other quality water qualityparameters parameters is discussed. is discussed.

2. Materials Materials and and Methods Methods

2.1. Study Study Area Area and and Measurement Measurement Campaign This study was carried out using a total of thirteen inland water bodies within Australia—tenAustralia—ten from South South East East Queensland Queensland (SEQ) (SEQ) and and three three from fromNorthern Northern New South New Wales South (NSW). Wales These (NSW). particular These particularwater bodies water were bodies chosen were due chosen to their due variation to their variation in visible in water visible quality, water ease quality, of access ease of and access location and location (Figure 2). With the exception of the urban Einbunpin Lagoon and Brisbane River, all other lakes sampled sit within predominantly mixed agricultural and forested catchments. In total, 32 sites were visited between February 2016 and May 2016, where reflectance measurements and water

Remote Sens. 2020, 12, 1578 4 of 20

(Figure2). With the exception of the urban Einbunpin Lagoon and Brisbane River, all other lakes sampled sit within predominantly mixed agricultural and forested catchments. In total, 32 sites were visitedRemote Sens. between 2020, 1 February2, x FOR PEER 2016 REVIEW and May 2016, where reflectance measurements and water samples4 wereof 20 collected between 10:00 and 14:00 h local time. At each of the sites, water samples were taken alongside hyperspectralsamples were reflectance collected between measurements 10:00 and made 14:00 using h local a Satlantic time. At spectroradiometer, each of the sites, water and measurementssamples were taken alongside hyperspectral reflectance measurements made using a Satlantic spectroradiometer, made using the HC and EoW Apps operated using either an iPhone 6 or iPhone 6s. Three replicates and measurements made using the HC and EoW Apps operated using either an iPhone 6 or iPhone were taken at each station to examine the variation in successive measurements. At some stations two 6s. Three replicates were taken at each station to examine the variation in successive measurements. observers operated the EoW and HC Apps simultaneously. The smartphone cameras were also used to At some stations two observers operated the EoW and HC Apps simultaneously. The smartphone record general sky conditions, horizontal visibility, surface wave action and overall variation in water cameras were also used to record general sky conditions, horizontal visibility, surface wave action color. Ancillary measurements were recorded at each site following methods in [35]. Measurements and overall variation in water color. Ancillary measurements were recorded at each site following included the condition of the sky, wind speed and direction, wave height, Secchi disk depth and bottom methods in [35]. Measurements included the condition of the sky, wind speed and direction, wave depth. The campaign stations are summarized in the first four columns of Table S1. The exact GPS height, Secchi disk depth and bottom depth. The campaign stations are summarized in the first four coordinates of all stations, time of recording and their position in the Lakes are recorded on the EoW columns of Table S1. The exact GPS coordinates of all stations, time of recording and their position website (https://www.eyeonwater.org). in the Lakes are recorded on the EoW website (https://www.eyeonwater.org).

Figure 2. The locations of the 13 water bodies sampled in New South Wales and South East Figure 2. The locations of the 13 water bodies sampled in New South Wales and South East Queensland. Queensland. Basemap courtesy of openstreetmap.org. Basemap courtesy of openstreetmap.org.

2.2.2.2. Water Water SamplingSampling WaterWater samples samples werewere storedstored inin aa darkeneddarkened cooler with ice packs packs until until filtration filtration and and processing. processing. StorageStorage time time to to the the laboratory laboratory did did not exceednot exceed 4 h and 4 h all and samples all samples were filtered were filteredwithin 24 within h of collection, 24 h of followingcollection, the following REVAMP the protocols REVAMP [ 36protocols]. For SPM, [36]. WhatmanFor SPM, Whatman GF/F filters GF/F were filters prepared were prepared and weighed and priorweighed to our prior use. to Each our sample use. Each was sample filtered inwas triplo. filtered Depending in triplo. upon Depending the turbidity upon ofthe the turbidity water, between of the 200water, and between 500 mL 200 of sample and 500 water mL of was sample filtered water under was filtered low vacuum under pressurelow vacuum to prevent pressure the to particlesprevent fromthe particles breaking from up. breaking Filters for up. SPM Filters were for stored SPM were in separate stored in dishes separate for gravimetricdishes for gravimetric analysis following analysis drying.following CHL drying. filters CHL folded filter intos folded a cryo-vial, into acovered cryo-vial, in covered aluminium in aluminium foil and stored foil and in a stored80 C in freezer. a -80 − ◦ For°C CDOM,freezer. samples For CDOM, were samplesfiltered using were the filtered Millipore using filter the units Millipore and a filter 0.22 µ unitsm polycarbonate and a 0.22 filter μm (Millipore).polycarbonate Approximately filter (Millipore). 80 mL Approximately of sample was 80 filteredmL of sample through was the filtered unit. This through water the was unit. used This to rinsewater the was filter used bottle to rinse and the unit.filter Approximatelybottle and the unit. 100 Approximately mL was then filtered; 100 mL 80 was mL then of this filtered; filtered 80 water mL of this filtered water was placed in a small acid-washed glass bottle covered in aluminium foil and was placed in a small acid-washed glass bottle covered in aluminium foil and placed in the fridge. placed in the fridge. Water quality samples, once filtered and prepared, were analysed at CSIRO Water quality samples, once filtered and prepared, were analysed at CSIRO Oceans and Atmosphere’s Oceans and Atmosphere’s Hobart laboratories for determination of, amongst others, CHL, SPM and Hobart laboratories for determination of, amongst others, CHL, SPM and CDOM. The methodology CDOM. The methodology for the analysis of water quality samples followed in detail those in [37]. for the analysis of water quality samples followed in detail those in [37]. Results are documented in Results are documented in Table S1. Table S1. 2.3. Remote Sensing Reflectance A Satlantic HyperOCR hyperspectral radiance radiometer (8.5° field of view) was used to collect reflectance data in over 136 spectral channels over the 350 to 800 nm wavelength range at a sampling rate of 3 Hz. The radiance radiometer was pointed downwards consecutively over a 10% calibrated SpectralonTM panel, 20 cm over the water surface (~3 cm footprint) and just under the surface (Figure 3). Generally, 70–90 scans were recorded over a 30 s interval for each measurement position. The

Remote Sens. 2020, 12, 1578 5 of 20

2.3. Remote Sensing Reﬂectance

A Satlantic HyperOCR hyperspectral radiance radiometer (8.5◦ field of view) was used to collect reflectance data in over 136 spectral channels over the 350 to 800 nm wavelength range at a sampling rate of 3 Hz. The radiance radiometer was pointed downwards consecutively over a 10% calibrated SpectralonTM panel, 20 cm over the water surface (~3 cm footprint) and just under the surface (Figure3). Generally, 70–90 scans were recorded over a 30 s interval for each measurement position. The three different measurement geometries are required to characterize the spectra of the main components 2 1 in theRemote light Sens. field: 2020, the12, x totalFOR PEER flux REVIEW of photons from sun and sky that enter the water (Ed in Wm5 of −20 s− ), 2 1 1 sky radiance (Ls in W m− s− sr− ) that is reflected at the water surface (at depth 0) and the radiance three different measurement2 1 geometries1 are required to characterize the spectra of the main from the water (Lw in W m− s− sr− ). components in the light field: the total flux of photons from sun and sky that enter the water (Ed in This measurement method was adopted to address the limitations faced with using a small Wm−2 s−1), sky radiance (Ls in W m−2 s−1 sr−1) that is reflected at the water surface (at depth 0) and the boat for field data collection [15] and we were fortunate enough to apply this method due to fine radiance from the water (Lw in W m−2 s−1 sr−1). weather andThis stablemeasurement water method surface. was The adopted method to address we used the has limitations recently faced been with shown using toa small ‘demonstrate boat amplefor equivalence’ field data collection to near-surface [15] and we above-water were fortunate nadir-view enough to apply measurements this method performed due to fine byweather shielding the skyand/sun stable glint water contributions surface. The [38 method], the latter we used equivalent has recently to the been ‘sky blocked’shown to approach ‘demonstrate recommended ample in [39equivalence’]. The method to near draws-surface on a above long-water heritage nadir of-view similar measurements reflectance measurementsperformed by shielding made in the inland waterssky [4/sun,40– glint44] andcontributions forms the [38 basis], the of latter many equivalent measurements to the ‘sky in blocked’ studies approach which have recommended used aggregated in reflectance[39]. The data method across draws multiple on a long campaigns heritage of similar [45,46 ].reflectance Noise wasmeasurements reduced bymade taking in inland 60–90 waters spectra, and removing[4,40–44] and outliers, forms thethereby basis removing of many measurements the effects of in changing studies which illumination have used conditions aggregated in the reflectance data across multiple campaigns [45,46]. Noise was reduced by taking 60–90 spectra, and atmosphere and ripples at the air–water interface. We checked the variability about the individual removing outliers, thereby removing the effects of changing illumination conditions in the measurements caused by the influence of wave action at a turbid and ‘wavy’ site and found coefficients atmosphere and ripples at the air–water interface. We checked the variability about the individual of variationmeasurements of ~4.2% caused for bythe the panel influence measurements of wave action and ~2% at a forturbid the and water ‘wavy’ surface site andmeasurements found (Figurecoefficients S1). These of results variation are of well ~4.2% within for the acceptable panel measurements uncertainties and for ~2% such for measurements the water surface despite whatmeasurements configuration (Figure of sensors S1). and These viewing results geometries are well within is used the [39 acceptable]. uncertainties for such 1 Huemeasurements angle is notdespite dependent what configuration upon the absolute of sensors Rrs and (sr viewing− ); it isgeometries purely determined is used [39]. by the relative distributionHue of angle Rrs overis not the dependent visible wavelengths. upon the absolute However, Rrs (sr− to1); furtherit is purely check determined the impact by the of therelative depth on measurementsdistribution of of the Rrs relative over the distribution visible wavelengths. of Rrs and However, hue angle, to further we used check HydroLight the impact ofTM themodelling depth to TM showon that measurements the impact of on the hue relative angle distribution determination of Rrs is and extremely hue angle small, we used (within HydroLight 0.2 degrees modelling for a sensor to show that the impact on hue angle determination is extremely small (within 0.2 degrees for a sensor immersed 5 cm below the water surface, Figure S2). immersed 5 cm below the water surface, Figure S2).

Figure 3. Deployment of the Satlantic spectroradiometer at Lake Dyer. Three sets of successive Figure 3. Deployment of the Satlantic spectroradiometer at Lake Dyer. Three sets of successive measurements were taken (a) over the calibrated grey panel, (b) over the water surface and (c) just measurements were taken (a) over the calibrated grey panel, (b) over the water surface and (c) just below the surface. below the surface. The above-surface remote-sensing reflectance Rrs (sr−1) is calculated in accordance with the 1 Themethods above-surface of [47]. This remote-sensingproperty is a function reﬂectance of wavelength Rrs (sr (λ)− )and is calculateddefined by dividing in accordance the upward with the λ methodswater of leaving [47]. This radiance property (Lw) by is athe function downward of wavelength irradiance (E (d),) just and above deﬁned the bysurface, dividing indicated the upward by + water(0 leaving+): radiance (LW) by the downward irradiance (Ed), just above the surface, indicated by (0 ):

Rrs(λ) = Lw (0++,λ)/ Ed(0+,+ λ) (1) Rrs(λ) = Lw (0 ,λ)/ Ed(0 , λ) (1) The Ed (0+, λ) was calculated from the grey SpectralonTM panel radiance (Ld):

Ed (0+, λ) = βπ Ld (λ) (2) where β is the calibration correction (10) for the 10% reflectance panel and π accounts for the conversion from radiance to irradiance. Because the tip of the Satlantic spectrometer was immersed in the water, we could derive the Lw (0+,λ) directly from underwater radiance Lw (0−, λ) and correcting

Remote Sens. 2020, 12, 1578 6 of 20

+ TM The Ed (0 , λ) was calculated from the grey Spectralon panel radiance (Ld):

+ Ed (0 , λ) = βπ Ld (λ) (2) where β is the calibration correction (10) for the 10% reflectance panel and π accounts for the conversion from radiance to irradiance. Because the tip of the Satlantic spectrometer was immersed in the water, + we could derive the Lw (0 ,λ) directly from underwater radiance Lw (0−, λ) and correcting for the refraction effects by dividing by n2, with n = 1.33 the refractive index of freshwater. The very small Fresnel reflection at 0◦ from water to air was ignored. To be comparable to the images collected by the HC and EoW Apps, Satlantic data of Rrs (λ) were converted to RGB [27] and references therein. The RGB spectral response of the human observer is well characterized and described by the standard colorimetric 2-degree Color-Matching Functions (CMF) of the International Commission on illumination [48]. First, both the Satlantic-derived Rrs(λ) and the CIE RGB curves were interpolated on a 1 nm grid, multiplied and integrated over all wavelengths. This resulted in the 3 chromaticity values XYZ (describing the RGB values in that order). Then, the red (x) and green (y) normalized value was derived by dividing by X + Y + Z. Grey or white light, where there is no wavelength dependency, has coordinates x = 1/3 and y = 1/3—the white point. Finally, the distance to the white point (saturation) and the hue angle were calculated. In this paper, the hue angle is expressed in degrees counter-clockwise relative to the y = 1/3, x = 1/3 to 1.0 line (hue angle zero). Further information is available in [31,49] and [28], including the transformation table from hue angle to the FUI.

2.4. The HC Application Developed by Leeuw from the University of Maine [50], HC is a water quality App that uses an iPhone’s digital camera to determine the Rrs of the water body in the RGB channels. HydroColor uses images collected by the user to measure how much light is reflected from the surface water, corrected for the reflection of sky radiance (Ls) off the surface and normalized to incoming radiation (Ed). In fact, HC assumes that the camera acts as a linear spectrometer and that the same protocol can be followed as for standard above-water radiometer settings [8,36,51]. The absolute magnitude of the reflectance in these channels are used to estimate SPM concentrations and the turbidity of the body of water using in-built algorithms [32,50]. In this paper, we used the HC App version available at the start of 2016. HC is well designe and, has clear instructions and easy to read pages. HC uses the internal GPS and clock to compute the position to the sun. With the help of the smartphone’s compass and gyroscope, the user is supported in finding the optimum viewing angles to minimize surface reflection [52]. Examples of the HC screens are given in Figure4. The observer must possess an 18% photographers’ grey card to normalize to the ambient illumination. The App compiles information for latitude, longitude, date, time, sun zenith, sun azimuth, device zenith, device azimuth, exposure values, remote sensing reflectance, turbidity, suspended particulate matter, concentration and the backscattering coefficient [32,50]. HC takes a 200 200 pixel square in the centre of the image and × averages these RGB values. The digital numbers (DNs) in each band are divided by the camera 1 exposure time (DN s− ) as an approximation to the radiance. The Rrs in RGB is then calculated using the relative radiance measured in each image:

Rrs (RGB) = (Lu(RGB) ρ.Ls (RGB))/ E (RGB) (3) − d 1 1 where Lu (RGB) is water surface radiance (DN s− ), Ls (RGB) is the sky radiance (DN s− ) and ρ is the sun/sky glint coeﬃcient at the air–water interface, taken as 0.028 from [53]. The Ed (RGB) is derived from the grey card observation using Equation (2), where in this case β = 1/(0.18). Note that although 1 1 the input parameters are measured in DN s− , the output has units (sr− ). Finally, HC uses two non-linear models to convert the Rrs in the R- band to SPM concentration and subsequently to translate Remote Sens. 2020, 12, 1578 7 of 20 the SPM concentration to turbidity; Leeuw and Boss (2018) outline the underlying assumptions and restrictions in these models. Remote Sens. 2020, 12, x FOR PEER REVIEW 7 of 20

Remote Sens. 2020, 12, x FOR PEER REVIEW 7 of 20

FigureFigure 4. The 4. The three three steps steps involved involved in in capturing capturing data on the the Brisbane Brisbane river river with with the theHydroColor HydroColor (HC) (HC) application. (a) The opening screen as ‘start here,’ suggesting the starting photo. The first photo is of application. (a) The opening screen as ‘start here,’ suggesting the starting photo. The ﬁrst photo is of a a grey card, the second photo taken is of sky and the third photo is of the water (shown in b). The grey card, the second photo taken is of sky and the third photo is of the water (shown in b). The results results are summarized in (c). are summarized in (c). Two Figurevalues 4. Theare threepresented steps involved by the in applicationcapturing data (Figureon the Brisbane 5b). One river withis the the FUI HydroColor selected (HC by) the user Two(FU_Value), valuesapplication achieved are presented. (a) byThe matching opening by screen the the application asphoto ‘start heretaken,’ suggesting (Figurewith the the5 appropriateb). starting One photo. is color the The FUI firsthue. photo selectedThe issecond of by value the user (FU_Value),(FU_Processed) achieveda grey card, is bybased the matching second on automated photo the taken photo isprocessing of sky taken and the withof thethird theuploaded photo appropriate is of theimage water by color (shown the hue.WAter in b). The The COlor second from value (FU_Processed)Digital Imagesresults is based (areWACODI summarized on automated) algorithm in (c). on processing the EoW server of the [3 uploaded1]. WACODI image assumes by that the WAterall images COlor from from Digitalcommercial ImagesTwo (WACODI) smartphones values are presented algorithm are delivered by the on application the in the EoW sRGB (Figure server standard 5b). [31 One]. format. WACODIis the FUI Conversion selected assumes by tothe that linearizeduser all images fromchromaticity commercial(FU_Value), coordinates smartphones achieved by XYZ matching are is deliveredperformed the photo inin taken thetwo with sRGBsteps: the standard1)appropriate convert format. thecolor sRGB hue. Conversion The values second in valuethe to digital linearized chromaticityimages(FU_Processed) to coordinates the XYZ CIE is based XYZsystem on is automatedusing performed the processingconversion in two of steps:matrices the uploaded (1) found convert image in [ 5by the4] the and sRGB WAter 2) Gamma values COlor fromcorrection in the digital imagesto to compensate theDigital XYZ Images CIE for ( systemWACODI non-linear using) algorithm compression the conversion on the ofEoW images server matrices in[31] the. WACODI found sRGB in system.assumes [54] and that In (2) the all Gamma images image from, 16 correction small to sectionscommercial of 40 × 40 smartphones pixels are areselected delivered and the in the resulting sRGB standard distribution format. of hue Conversion angles in to all linearized sections are compensate for non-linear compression of images in the sRGB system. In the image, 16 small sections combinedchromaticity to select coordinates the ‘best’ XYZ approximation is performed inof twothe steps:water 1) hue convert and the the sRGB corresponding values in the FUI digital [31] . As of 40 40 pixelsimages to are the selected XYZ CIE andsystem the using resulting the conversion distribution matrices of found hue anglesin [54] and in all2) Gamma sections correction are combined to images can be quite complex, with shadows, the rim of a boat or pier, the ‘best’ selection is not always × to compensate for non-linear compression of images in the sRGB system. In the image, 16 small selectthe the ‘centre’ ‘best’ section approximation of the image. of the In this water paper hue, we and used the the corresponding WACODI open FUI-source [31]. software As images [31] to can be sections of 40 × 40 pixels are selected and the resulting distribution of hue angles in all sections are quite complex, with shadows, the rim of a boat or pier, the ‘best’ selection is not always the ‘centre’ analysecombined all images to select in more the ‘best’depth. approximation In total, 6 × 6of subsections the water hue were and analysed the corresponding and four FUI hue [31] angles. As were sectionretrieved: ofimages the image.the can ‘best’ be quite In and this complex, ‘centre’ paper, with hue we shadows,angle, used assuming thethe rim WACODI of sunny a boat oror open-source pier, overcast the ‘best’ illumination selection software is conditions.not [31 always] to analyse all images inthe more ‘centre’ depth. section In of total, the image. 6 6 In subsections this paper, we were used analysedthe WACODI and open four-source hue software angles [31] were to retrieved: × the ‘best’ andanalyse ‘centre’ all images hue in angle, more depth. assuming In total sunny, 6 × 6 subsections or overcast were illumination analysed and four conditions. hue angles were retrieved: the ‘best’ and ‘centre’ hue angle, assuming sunny or overcast illumination conditions.

Figure 5. At the proper orientation, a photo is first taken and a color chart appears. The user is able to scrollFigure through 5. At the the color proper chart orientation and select, a photo which is first color taken best and matches a color chart the appears. photo of The the user water is able (a to). After Figure 5. At the proper orientation, a photo is ﬁrst taken and a color chart appears. The user is uploadingscroll throughand processing the color on chart the and EyeOnWater select which (E coloroW) bestserver, matches the photo the photo with of all the results water is(a ).available After on able to scroll through the color chart and select which color best matches the photo of the water (a). the webserveruploading (andb). processing on the EyeOnWater (EoW) server, the photo with all results is available on After uploadingthe webserver and processing (b). on the EyeOnWater (EoW) server, the photo with all results is available on3. Results the webserver (b). 3. Results

Remote Sens. 2020, 12, 1578 8 of 20

3. Results

3.1. Water Quality Conditions In Table S1, a summary of the mean laboratory results for CHL and SPM concentrations is given, together with the CDOM absorption at 440 nm. The results show that the measurements taken across the 13 water bodies have covered a wide range of water qualities from oligotrophic (poor Remote Sens. 2020, 12, x FOR PEER REVIEW 8 of 20 nutrient supply and little plant life) to hypertrophic (high nutrient content and water quality issues). The spread3.1. of CHLWater Quality levels Conditions suggests that the majority of sites included within this study have relatively low chlorophyllIn a Table concentrations. S1, a summary The of the majority mean laboratory of sites results are within for CHL the and 1.5 SPMµ gconcentrations/L (Lake St Clair)is given, to 26 µg/L (Lake Liddell)together CHL with range the CDOM with absorption sites such at 440 as nm. Lake The Dyer, results Einbunpin show that the andmeasurements Atkinson taken Dam across presenting the 13 water bodies have covered a wide range of water qualities from oligotrophic (poor nutrient elevated levels of 177, 121 and 85 µg/L, respectively. Secchi Disk depth ranges from the clearer blue supply and little plant life) to hypertrophic (high nutrient content and water quality issues). The waters of Lakespread St of Clair CHL (5levels m) suggests to the murky that the yellow-greenmajority of sites included water of within Lake this Atkinson study have (0.27 relatively m). The SPM results furtherlow reflectchlorophyll this a where concentrations SPM ranges. The majority from theof sites clearer are within waters the of1.5 Lake µg/L (Lake St Clair St Clair) (0.89 to mg 26 /L) to the µg/L (Lake Liddell) CHL range with sites such as Lake Dyer, Einbunpin1 and Atkinson Dam 1 highly turbid Brisbane River (158 mg/L). CDOM ranged from 0.34 m− at Lake Liddell to 2.30 m− at presenting elevated levels of 177, 121 and 85 µg/L, respectively. Secchi Disk depth ranges from the Einbunpin Lagoon. clearer blue waters of Lake St Clair (5 m) to the murky yellow-green water of Lake Atkinson (0.27 m). The SPM results further reflect this where SPM ranges from the clearer waters of Lake St Clair (0.89 3.2. Remotemg/L) Sensing to the Reflectance highly turbid Brisbane River (158 mg/L). CDOM ranged from 0.34 m−1 at Lake Liddell −1 Most ofto 2.30 the m watersat Einbunpin were Lagoon. calm,with no or only small ripples; the weather was calm and sky conditions ranged3.2. Remote from Sensing cloudless Reflectance to almost overcast (see Table S1). In Figure6, the Rrs ( λ) of 31 sites across 13 waterMost bodies of the are waters shown, were normalized calm, with no to or the only Rrs small (550 ripples; nm). the The weather reflectance was calm curve and sky is impacted by particlesconditions and dissolved ranged from matter cloudless (CHL, to almost SPM overcast and CDOM) (see Table within S1). In Figure the water 6, the Rrs column, (λ) of 31 insites particular suspendedacross algal 13 biomass water bodies (CHL), are shown, suspended normalized sediment to the Rrs and (550 CDOM. nm). The The reflectance spectra curve from is impacted the water bodies measured showby particles several and commondissolved matter features. (CHL, With SPM theand CDOM) exception within of the water Brisbane column, River, in particular all sites have a suspended algal biomass (CHL), suspended sediment and CDOM. The spectra from the water bodies peak betweenmeasured 560 and show 580 several nm, co correspondingmmon features. With to scattering the exception due of the to theBrisbane presence River, a ofll sites particulate have a matter and low absorption.peak between There 560 and is a580 prominent nm, corresponding reflectance to scattering peak atdue approximately to the presence of 700 particulate nm for matter the sites with the highestand chlorophyll low absorption. levels—these There is a prominent sites include reflectance Einbunpin peak at approximately Lagoon, Lake 700 nm Atkinson for the sites and with Lake Dyer. This high peakthe highest is caused chlorophyll by an levels interaction—these sites of algal include absorption Einbunpin inLagoon, the red Lake at Atkinson 676 nm, and in Lake combination Dyer. with This high peak is caused by an interaction of algal absorption in the red at 676 nm, in combination scattering bywith algae scattering and by pigment algae and [55 pigment]. The [5 absorption5]. The absorption near near 620 620 nm nm is is a a result result of of the the phycocyaninphycocyanin (PC) pigment and(PC) suggests pigment theand suggests presence the of presence blue-green of blue algae.-green algae. Sites Sites with with steeper steeper slopes slopes in the green green region of the spectrum,region of such the spectrum, as Lake such Liddell as Lake have Liddell significantly have significantly higher higher PC PC levels levels compared compared to to Lak Lakee St St Clair which has aClair gradual which slopehas a gradual indicating slope aindicating lower PC a lower value. PC Sincevalue. theSince red the bandred band (X) (X) has has a FWHMa FWHM between between 558 and 634 nm, it is clear from these spectra that waters with high CHL concentrations 558 and 634corrupt nm, it any is clearsimple from relation these between spectra X (or thatY) and waters SPM [5 with6]. The high results CHL of the concentrations conversion of these corrupt any simple relationcomplex between spectra Xto (orhue Y)angle and are SPM shown [56 in]. Table The S results1. of the conversion of these complex spectra to hue angle are shown in Table S1.

Figure 6. SatlanticFigure 6. Satlantic Rrs spectra Rrs spectra from from the the 13 13 water water bodiesbodies measured, measured, normalized normalized to 550 nm. to550 nm.

RemoteRemote Sens.Sens. 20202020,, 1212,, 1578x FOR PEER REVIEW 99 of 2020

3.3. EoW Performance 3.3. EoW Performance During the campaign, a total of 110 measurements of the 13 waters were uploaded to the EoW During the campaign, a total of 110 measurements of the 13 waters were uploaded to the EoW website, with a minimum of three replicates taken at each of the locations. These can be found on the website, with a minimum of three replicates taken at each of the locations. These can be found on the EoW website, with the location and timing provided in Table S1. Each entry shows the FU_Value EoW website, with the location and timing provided in Table S1. Each entry shows the FU_Value assigned by the user and FU_Processed from the image (see Figure 5b). In Figure 7, three examples assigned by the user and FU_Processed from the image (see Figure5b). In Figure7, three examples of of these are shown, which demonstrate the variability in water composition, illumination and these are shown, which demonstrate the variability in water composition, illumination and interfering interfering features in the image, such as shadow, sun glint and boat rim. Some images were taken features in the image, such as shadow, sun glint and boat rim. Some images were taken from the shore from the shore at 1.70 m height and sometimes the use of a small boat only allowed images to be at 1.70 m height and sometimes the use of a small boat only allowed images to be taken 30 cm above taken 30 cm above the water surface. All images were analysed and no visual inspection or selection the water surface. All images were analysed and no visual inspection or selection took place. took place.

Figure 7.7. Three examples of images collected with the EoWEoW App:App: Lake Atkinson (a),), Lake St ClairClair stationstation 66 ((bb)) andand WivenhoeWivenhoe DamDam stationstation 11 ((cc).).

A wide range of water colorscolors were measured withinwithin this study,study, withwith thethe FUIFUI rangingranging from 7 toto 18.18. An initialinitial inspection indicates that there is a higher occurrence of green waters within the sample of lakeslakes chosenchosen for thisthis study,study, withwith valuesvalues rangingranging mostlymostly fromfrom 1010 toto 16.16. LakesLakes AtkinsonAtkinson andand DyerDyer had higher FU_ValuesFU_Values ofof 1717 andand 16,16, reflectingreflecting thethe green-yellowgreen-yellow colorcolor ofof thethe waters.waters. Lostock Reservoir and Ewen Maddock Dam both presented FU_ValuesFU_Values ofof 17;17; thesethese waterswaters werewere darkerdarker inin color and had higher CDOM concentrations present.present. The Brisbane River had a FU_Value of 18 and was significantly significantly browner withwith higherhigher SPMSPM levels levels in in comparison comparison to to the the other other sites. sites. The The site site with with the the lowest lowest FU_Value FU_Value of 8of was 8 was Lake Lake St St Clair—this Clair—this water water body body was was clearer clearer and and bluer bluer than than the the other othersites sites measuredmeasured andand hadhad lowerlower CHLCHL andand SPMSPM levels.levels. The processing of 36, insteadinstead of 16,16, subsectionssubsections had limitedlimited impact on the ‘best’ huehue angleangle andand conversion toto FU_Processed.FU_Processed. InIn 2323 out of 116 images,images, thethe FU scale shifted byby plus or minus 1 unit; the overall correlation betweenbetween thethe twotwo valuesvalues hadhad aa slope of 1.01, confirmingconfirming thatthat this is not a systematic eeffect,ffect, butbut merely merely a a random random eff effectect whereby whereby a slightly a slight smallerly smaller or larger or larger hue anglehue angle is found is found that just that tips just it intotips anotherit into another FUI. In FigureFUI. In8, Figure the FUI 8 of, the WACODI FUI of (36WACODI subsection, (36 bestsubsection, value under best sunnyvalue under illumination sunny conditions)illumination is conditions) compared tois compared the FUI entered to the byFUI the entered observer. by the Two obse interestingrver. Two aspects interesting can beaspects deduced can frombe deduced this histogram: from this first,histogram: a trained first observer, a trained is indeedobserver capable is indeed of accuratelycapable of judgingaccurately the judg FUIing of thethe waterFUI of from the water the smartphone from the smartphone screen; 78% screen; of the 78% results of the gave results an FUI gave within an FUI one within unit from one theunit image from FUIthe image and only FUI 3 and out ofonly 116 3 were out of 3 units116 were off. Second, 3 units theoff. distributionSecond, the isdistribution somewhat is skewed somewhat such thatskewed the imagesuch that analysis the image overall analysis provides overall a slightly provides smaller a slightly FUI than smaller the observer.FUI than the observer.

Remote Sens. 2020, 12, x FOR PEER REVIEW 10 of 20 Remote Sens. 2020, 12, x FOR PEER REVIEW 10 of 20 Remote Sens. 2020, 12, 1578 10 of 20

Figure 8. Frequency distribution of the difference in Forel–Ule index (FUI) between the WACODI- Figureprocessed 8. 8. Frequency valueFrequency (P) distributionand distributionthe FUI ofentered the of dif thebyference the di ffobservererence in Forel (O). in–Ule Forel–Ule index (FUI index) between (FUI) the between WACODI the- WACODI-processedprocessed value (P) and value the (P) FUI and entered the FUI by entered the observer by the (O). observer (O). The precision of EoW image information is analysed as follows: all images collected at each stationThe were precisionprecision grouped of of EoW EoW together image image informationand information the average is analysed is hue analysed angle as follows: wasas follows: derived. all images all Subsequent images collected collectedly at, the each deviationat station each werestationfrom grouped thewere average grouped together was together calculated and the and average in the degrees average hue (the angle hue unit wasa ofngle hue derived. was angles). derived. Subsequently, The Subsequentse results the arely deviationshown, the deviation in Figure from thefrom9. averageFor the a averagetrained was calculatedobserverwas calculated, it in turns degrees in degreesout (thethat (the unitthe collectionunit of hue of hue angles). and angles). processing These The resultsse ofresults an are image are shown shown is a inprocess in Figure Figure that9. For9.provides For a traineda trained stable observer, observer results itwithin, it turns turns a out standardout that that the thedeviation collection collection of and3.15 and processingdegrees. processing To of testofan an the imageimage accuracy isis aa process processof WACODI, that providesthe image stable-derived results hue within angle awas standard compared deviation to the ofSatlantic 3.15 degrees. hue angle. To To test Note the th accuracy accuracyat per station of of WACODI, WACODI,, only one theaverage image image-derived -Satlanticderived hue hue angleangle angle was(Table was compared compared S1) is available, to to the the Satlantic Satlanticwhile three hue hue angle. to angle. six NoteEoW Note thimagesat that per perwerestation station, taken., only onlyWhen one oneaveragewe averageassume Satlantic Satlanticthat hueeach angle hue EoW angle (Table observation (Table S1) is S1)available, is isan available, independent while whilethree observation to three six toEoW six of images EoWa stable images were (over taken. were a period When taken. of Whenweapproximately assume we assume that each30 that min EoW each) natural EoWobservation observationwater isthat an isindependent ancharacterized independent observation by observation the Satlantic of a ofstable ahue stable angle,(over (over awe period a periodfind ofthe ofapproximatelycorrelation approximately shown 30 30min in min) )Figure natural natural 10 .water A water simple that that linearisis characterized characterized regression showsby by the the theSatlantic Satlantic slope closehue hue angle, angle,to 1.00 wewe and find find an R the2 of 2 correlation0.84. This shown shown result in in is Figure Figure similar 10 10 . to A. A the simple simple results linear linear reported regression regression for shows Dutch shows the inland the slope slope close waters, close to 1.00 basedto 1.00 and onand an RWACO anof R 0.84.2 ofDI - This0.84.processing result This isresult of similar images is similar to theand results Rrs to thespectra reported results collected forreported Dutch by a for inland set Dutch of waters,Ramses inland based spectrometers waters, on WACODI-processing based [31] on. Interestingly, WACODI of- imagesprocessingthis result and of Rrsdid images spectranot improve and collected Rrs or spectra worsen by a set collected ofif just Ramses the by hue spectrometersa set angles of Ramses of centre [31 spectrometers]. Interestingly, subsections [31] thiswere. Interestingly, result correlated did not to improvethisthe result Satlantic or did worsen not values improve if just or the if or hue the worsen angles overcast if of just centre conditions the subsectionshue angles were of takenwere centre correlated as subsections standard to the illumination. were Satlantic correlated values This orto is ifthedemonstrated the Satlantic overcast conditionsvalues in Figure or were if 1 the1 which taken overcast asshows standard conditions that illumination.overall were the taken central This as is demonstratedhue standard angle is illumination. only in Figure slightly 11 This which higher is showsdemonstrated(RC=1.06) that overallthan in theFigure the best central 1hue1 which angle. hue angleshows The outliers is that only overall slightlyin this the graph higher central are (RC hueall= related 1.06)angle than tois onlythe the complexity bestslightly hue higher angle. of the The(RC=1.06)image outliers coll thanected, in thisthe graphprovidingbest hue are angle. all problems related The tooutliers with the complexity the in imagethis graph of processing. the are image all related In collected, fact, to inthe providing the complexity new problemsWACODI of the - withimageprocessing the coll imageected, on processing. the providing EoW server, In problems fact, the in thedeviation with new WACODI-processing the of imageabsolute processing. difference on In thebest fact, EoW/centre in server, the hue new theangle deviation WACODI should of be- absoluteprocessingless than di ff10 onerence degrees the EoW best in/ centre server,order hueto the achieve angle deviation should the highestof beabsolute less quality than difference 10 flag. degrees best in/centre order tohue achieve angle the should highest be qualityless than flag. 10 degrees in order to achieve the highest quality flag.

FigureFigure 9. 9. FrequencyFrequency distribution distribution of of the the di differenceﬀerence in in hue hue angle angle (degrees) (degrees) between between individual individual measurementsFiguremeasurements 9. Frequency and and the the distribution mean mean per per station. station. of the difference in hue angle (degrees) between individual measurements and the mean per station.

Remote Sens. 2020, 12, x FOR PEER REVIEW 11 of 20

RemoteRemote Sens. Sens.2020 2020, 12, 1,2 1578, x FOR PEER REVIEW 1111 of of 20 20

Figure 10. Correlation between hyperspectral (Satlantic) and mean EoW image (WACODI) hue angle. The solid line represents the regression through the data, and the dashed line represents the 1:1 relationship.

From this large set of data, collected from inland waters covering a wide range in composition, there is some encouragement for the accuracy and precision of the observed FUI and the processed FUI results. The average standard deviation for the replicates of the user selected FU_Value is 0.34 and 0.42 for the FU_Processed value. The slightly higher variation in the processed FUI values may be because the user may be more inclined to select the same color as previously selected, unless the images are significantly different. The results suggest that there is no need to take multiple measurements at each of the sites of interest as commonly the first image will be representative of the water color unless there is an issue with how it is taken. Our field experience suggests that the application is best used under clear and still water conditions to avoid the impacts of ripple reflection and precision issues do arise with very dark waters, windy conditions and fast-changing illumination Figure 10. Correlation between hyperspectral (Satlantic) and mean EoW image (WACODI) hue angle. conditionsFigure 10. whenCorrelation it is partly between cloudy. hyperspectral (Satlantic) and mean EoW image (WACODI) hue angle.The solid The solid line represents line represents the regression the regression through through the the data, data, and and thethe dashed dashed line line represents represents the the 1:1 1:1relationship. relationship.

FigureFigure 11. 11.Relation Relation between between the the hue hue angle angle derived derived from from the center the center and best and part best of part the image.of the image. The solid The linesolid represents line represents the regression the regression through through the data, the the data, dashed the linedashed represents line represents the 1:1 relationship. the 1:1 relationship.

3.4.From HC Performance this large set of data, collected from inland waters covering a wide range in composition, there is some encouragement for the accuracy and precision of the observed FUI and the processed FUI During the field campaign, a total of 103 complete datasets were collected with the HC App. HC results. The average standard deviation for the replicates of the user selected FU_Value is 0.34 and 0.42 captures images, taken of the grey card, sky and water surface and the central 200 × 200 pixels are for the FU_Processed value. The slightly higher variation in the processed FUI values may be because processed to derive Rrs (RGB). All relevant information, including location, time, solar zenith angle, the user may be more inclined to select the same color as previously selected, unless the images are significantly different. The results suggest that there is no need to take multiple measurements at each of the sites of interest as commonly the first image will be representative of the water color unless there is an issue with how it is taken. Our field experience suggests that the application is best used under clear and still water conditions to avoid the impacts of ripple reflection and precision issues do arise with very dark waters, windy conditions and fast-changing illumination conditions when it is partly cloudy. Figure 11. Relation between the hue angle derived from the center and best part of the image. The 3.4. HCsolid Performance line represents the regression through the data, the dashed line represents the 1:1 relationship.

3.4.During HC Performance the ﬁeld campaign, a total of 103 complete datasets were collected with the HC App. HC captures images, taken of the grey card, sky and water surface and the central 200 200 pixels × are processedDuring the to derivefield campaign Rrs (RGB)., a total All relevantof 103 complete information, datasets including were collected location, with time, the solarHC App. zenith HC angle,captures viewing images, angles, taken integration of the grey times card, and sky intensity and water of the surface three images,and the pluscentral derived 200 × Rrs200 andpixels SPM are processed to derive Rrs (RGB). All relevant information, including location, time, solar zenith angle,

Remote Sens. 2020, 12, x FOR PEER REVIEW 12 of 20 viewingRemote Sens. angles,2020, 12 , integration 1578 times and intensity of the three images, plus derived Rrs and12 SPM of 20 concentration were downloaded and recorded. For this study, at least three replicates were taken at each site to determine the variability about a single measurement, although the HC app does not specifyconcentration that three were measurements downloaded should and recorded. be taken For at each this study,site to atreduce least threeerror. replicatesFigure 12 wereshows taken the variationat each site between to determine the three the replicates variability taken, about based a single on all measurement, the derived althoughRrs (RGB) the values. HC app It is does evident not thatspecify large that errors three did measurements occur between should the three be taken replicates at each made site to in reduce some error.water Figurebodies 12and shows that the the anomalies,variation between when they the threeoccurred, replicates are generally taken, based simultaneously on all the derived present Rrs in (RGB) each values.of the three It is evident bands. thatFor Lostock01large errors replicate did occur B, between Somerset the replicates three replicates A and made C and in some Borumba water replicates bodies and A that and the C anomalies, zero was returnedwhen they as occurred,a result. are generally simultaneously present in each of the three bands. For Lostock01 replicateExplanations B, Somerset for replicates the causes A of and these C and variation Borumbas inclu replicatesde both A how and the C zero photos was were returned taken as and a result. the conditionsExplanations they were for taken the causes under. ofMeasurements these variations taken include at sites both Lake how Liddell the 03, photos 05 and were 06, takenSomerset, and Borumbathe conditions and theyBaroon were Pocket taken Dam under. were Measurements made under taken patchy at sites cloudy Lake conditions. Liddell 03, Despite 05 and the 06, measurementsSomerset, Borumba being and made Baroon in rel Pocketatively Dam quick were succession, made under the variation patchy cloudy between conditions. the replicates Despite may the bemeasurements a result of moving being madeclouds in impacting relatively on quick the succession,red, blue and the green variation channels. between Clouds the replicatesimpact on may the reflectancebe a result of the moving water clouds as well impacting as the intensity on the measured red, blue in and the greensky photo. channels. Measurements Clouds impact taken onunder the clearreﬂectance sunny of conditions the water such as well as asLake the Atkinson intensity measuredand Lake inSt theClair sky resulted photo. in Measurements replicates with taken very under low variationclear sunny between conditions measurements. such as Lake Atkinson and Lake St Clair resulted in replicates with very low variation between measurements.

Figure 12. Summary of HC-derived Rrs (RGB) for all measurements, with variation between the three Figure 12. Summary of HC-derived Rrs (RGB) for all measurements, with variation between the three replicate HC RGB values expressed as bars indicating standard error. replicate HC RGB values expressed as bars indicating standard error. The accuracy of the HC information was tested in the RGB bands. In Table S1, the Satlantic red bandThe Rrs accuracy is given forof the 30 HC stations. information Also given was istested the meanin the HCRGB red bands. band In Rrs Table that S is1, simplythe Satlantic the mean red bandvalue Rrs at each is given station, for derived30 stations. from Also 3 to given 6 datasets. is the Figuremean HC 12 presents red band the Rrs Rrs that (RGB) is simply results the derived mean valuefrom theat each images station, taken derived within from HC. At3 to first 6 datasets. sight, the Figure variation 12 presents in peaks the represents Rrs (RGB) the results wide rangederived of fromwater the qualities images measured taken within within HC. this At first study; sight, indeed, the variation the locations in peaks with represents higher peaks the wide such range as Lake of waterAtkinson, qualiti Einbunpines measured Lagoon, within Brisbane this study; River, indeed Lostock, the Dam locations and Lake with Liddell higher represent peaks such sites as which Lake Atkinson,have higher Einbunpin turbidity Lagoon, or chlorophyll Brisbane levels. River, However, Lostock Dam Figure and 13 Lakedemonstrates Liddell represent that the highsites valueswhich havemight higher have a turbidity different origin,or chlorophyll because welevels. find However, wide discrepancies Figure 13 when demonstrates compared that to the the hyperspectral high values mightmeasurements. have a different The results origin, do not because indicate we any find significant wide discrepancies form of co-variation when compared for each channel to the hyperspectralbetween the two measurements. sets of measurements The results for do all not bands. indicate This indicatesany significant that the form two of measurement co-variation methodsfor each channelwere not between measuring the thetwo same sets of quantity measurements or that random for all bands. errors This are largeindicates with that a high the impacttwo measurement on the final mean values. Potential errors and remedies are discussed below.

Remote Sens. 2020, 12, x FOR PEER REVIEW 13 of 20

methodsRemote Sens. were 2020 , not12, x measuring FOR PEER REVIEW the same quantity or that random errors are large with a high impact13 of 20 on the final mean values. Potential errors and remedies are discussed below. methods were not measuring the same quantity or that random errors are large with a high impact

Remoteon the Sens. final2020 ,mean12, 1578 values. Potential errors and remedies are discussed below. 13 of 20

Figure 13. Relationship between the Satlantic and HC-derived Rrs in the Red channel. The solid line

represents the regression through the data, and the dashed line represents the 1:1 relationship. FigureFigure 13. 13.Relationship Relationship between between the the Satlantic Satlantic and and HC-derived HC-derived Rrs Rrs in in the the Red Red channel. channel. The The solid solid line line 3.5. representsHCrepresents and EoW the the regressionInformation regression through onthrough Water the the data,Composition data, and and the the dashed dashed line line represents represents the the 1:1 1:1 relationship. relationship.

3.5.3.5. HC HCThe and and HC EoW EoW App Information Information calculates on bothon Water Water turbidity Composition Composition (NTU) and concentration of suspended particular matter (SPMThe). InHC Figure App calculates 14, the relation both turbidity is shown (NTU) between and concentration the gravimetric of suspended measured particular and HC optically matter retrievedThe HC SPM A pp concentration calculates boths; again, turbidity the HC (NTU) result and is concentration the mean value of suspended of the 3–6 particular replicates matter at 30 (SPM).(SPM) In. In Figure Figure 14 1,4 the, the relation relation is is shown shown between between the the gravimetric gravimetric measured measured and and HC HC optically optically retrievedstations. SPM HC perform concentrations; well in again,some water the HC bodies result at is low the meanSPM concentrations, value of the 3–6 but replicates not others at 30 (e.g. stations., Lake Liddell).retrieved Because SPM concentration SPM is retrieveds; again, from the the HC red result channel is the Rrs, mean the error value in of the the red 3– 6Rrs replicates is obviously at 30 HCstations. perform HC well perform in some well water in some bodies water at low bodies SPM at concentrations, low SPM concentrations, but not others but not (e.g., others Lake (e.g. Liddell)., Lake Becausetranslated SPM tois errors retrieved in the from retrieved the red SPM channel concentration. Rrs, the error In answer in the red to Rrsthe isquestion obviously we translatedpostulated to at theLiddell). outset Becauseof this study SPM (is“how retrieved accurate from is thethe HCred conversionchannel Rrs, of theRrs errorto SPM in forthe the red large Rrs isvariation obviously in errorstranslated in the retrievedto errors SPMin the concentration. retrieved SPM In concentration. answer to the questionIn answer we to postulated the question at the we outset postulated of this at studyAustralian (“how lakes, accurate rivers is theand HC dams?” conversion), based of on Rrs these to SPM results for, no the firm large answer variation can inbeAustralian given. No further lakes, correlationsthe outset of could this study be established (“how accurate between is the HC resultsconversion and concentrationsof Rrs to SPM for of CHLthe large or CDOM. variation in riversAustralian and dams?”), lakes, rivers based and on thesedams?” results,), based no ﬁrmon these answer results can, beno given.firm answer No further can be correlations given. No couldfurther becorrelations established could between be established the HC results between and concentrations the HC results of and CHL concentrations or CDOM. of CHL or CDOM.

Figure 14. Relationship between in situ-measured suspended particulate matter (SPM) and Figure 14. Relationship between in situ-measured suspended particulate matter (SPM) and HydroColor-derivedHydroColor-derived SPM SPM (mg (mg/L)/L) shown shown for for individual individual lakes. lakes. The The solid solid line line represents represents the the regression regression Figure 14. Relationship between in situ-measured suspended particulate matter (SPM) and throughthrough the the data, data, and and the the dashed dashed line line represents represents the the 1:1 1:1 relationship. relationship. HydroColor-derived SPM (mg/L) shown for individual lakes. The solid line represents the regression TheTthroughhe EoW EoW appthe app data, does does and not not the provide provide dashed an linean estimate estimate represents for for the water water 1:1 relationship quality quality parameters parameters. but but it it is is through through the the FU FU scalescale that that the the relative relative health health of of the the water water can can be be estimated. estimated. The The scale scale range range of of 21 21 colors colors from from indigo indigo blue toT brown,he EoW through app does blue-green, not provide green an estimate and yellow for water colors, quality is indicative parameters of the but level it is of through nutrients the or FU algaescale present that the (1–5 relat FU),ive health dissolved of the matter water and can sediment be estimated. (6–9 FU),The scale phytoplankton range of 21 levels colors (10–13 from indigo FU), sediment (14–17 FU) and humic acids (18–21 FU). First, the observed FU_Value over the full range of values (8–18 FU) was simply correlated to in situ measurements of SDD, SPM and CHL. No signiﬁcant Remote Sens. 2020, 12, x FOR PEER REVIEW 14 of 20

blue to brown, through blue-green, green and yellow colors, is indicative of the level of nutrients or algae present (1–5 FU), dissolved matter and sediment (6–9 FU), phytoplankton levels (10–13 FU), sediment (14–17 FU) and humic acids (18–21 FU). First, the observed FU_Value over the full range of Remote Sens. 2020, 12, 1578 14 of 20 values (8–18 FU) was simply correlated to in situ measurements of SDD, SPM and CHL. No significant relationships could be established. Second, we converted each FU_Value to the average relationshipsSDD and CHL could value be established.derived at a Second, global scale we converted [57] (Tables each A3 FU_Value and A4), to and the then average correlated SDD and to CHLthe in valuesitu measurements. derived at a global Again, scale no [57 relat] (Tablesion could A3 and be established. A4), and then We correlated conclude to that the no in situgeneric measurements. relation can Again,be established no relation between could color be established. and concentrations We conclude for these that 13 no very generic different relation Australian can be inland established waters betweendue to the color absence and concentrationsof any relation forbetween these the 13 verythree diopticallyfferent Australianactive substances. inland waters due to the absence of any relation between the three optically active substances. 3.6. Spectral Integrity 3.6. Spectral Integrity The two applications are difficult to compare, as HC uses a more robust approach to measure reflectanceThe two by applications taking three are di separatefficult to measurements compare, as HC ofuses the a grey more card, robust sky approach and water to measure surface, reflectancerespectively, by taking compared three to separate the one measurements single horizontal of the sur greyface card, image sky taken and water by EoW surface,. Despite respectively, this, the comparedcommon toground the one between single horizontalthe two Apps surface was imagetested takenin a simple by EoW. exercise. Despite The this, average the common hue angle ground of the betweenHC measurements the two Apps is wasgiven tested in Table in a simpleS1 and exercise.the procedure The average to convert hue anglethe Rrs of (RGB) the HC of measurements HC to x,y and issubsequently given in Table to S1 hue and angle the procedureis described to in convert Section the 2.3. Rrs Because (RGB) ofFigure HC to 12 x,y indicated and subsequently that errors toseem hue to anglebe correlated is described for RGB, in Section it might 2.3 be. Because that the Figureshape of 12 the indicated reflection that spectrum errors seem is less to prone be correlated to errors than for RGB,the absolute it might Rrs be that values. the shapeIn Figure of the 15, reflectionthe HC hue spectrum angle isis related less prone to the to Satlantic errors than hue the angle absolute. Although, Rrs values.the R2 Inis Figureof the same15, the order HC hueas the angle EoW is relatedresults ( toFigure the Satlantic 10), the hueresulting angle. trend Although, line is the very R2 steep:is of the the sameblue orderwaters as ( thee.g. EoW, Lake results St. Clair) (Figure are 10extremely), the resulting blue and trend the line yellow is very-brown steep: waters the blue have waters an extreme (e.g., Lakered contribution, St. Clair) are resulting extremely in blue very and low thehue yellow-brown angles. The reason waters for have this warrants an extreme further red contribution, investigation. resultingFrom this in result very low, we hueconclude angles. that The HC reason has the for potential this warrants to cover further the full investigation. range of hue From angles, this but result, that wefirst conclude a correction that HC must has be the applied potential to av tooid cover overshooting the full range of the of huespectral angles, shape. but that first a correction must be applied to avoid overshooting of the spectral shape.

FigureFigure 15. 15.The The hue hue angle angle of of HC HC observations, observations, plotted plotted as as a a function function of of the the Satlantic Satlantic hue hue angle. angle. The The 1:1 1:1 referencereference line line is is shown, shown, together together with with the the steep steep linear linear trend trend line line that that ﬁts fits the the data. data. The The solid solid line line representsrepresents the the regression regression through through the the data, data, and and the the dashed dashed line line represents represents the the 1:1 1:1 relationship. relationship.

4. Discussion 4. Discussion TheThe quality quality of of Australian Australian inland inland waters waters is is under under pressure pressure and and a a dedicated dedicated monitoring monitoring e ﬀeffortort is is neededneeded to to evaluate evaluate changes changes and and detect detect rapid rapid changes. changes. Algal Algal blooms blooms of of blue-green blue-green algae, algae, harmful harmful to to humans,humans, animals animals and and the the local local ecosystem, ecosystem, occur occur frequently frequently in in the the lakes lakes close close to to major major cities cities of of New New SouthSouth Wales Wales and and Queensland. Queensland. Given Given the the scale scale of theof the Australian Australian continent, continent, frequent frequent in situ in situ sampling sampling of allof major all major inland inland waters waters is almost is almost an impossible an impossible task. Therefore,task. Therefore, satellite satellite observations, observations, supported supported by a suiteby aof suite in situ of underwater in situ underwater and above-water and above automated-water sensors, automated are a sensors good and, are welcome a good addition and welcome [6,58]. Opticaladdition close-range [6,58]. Optical instruments close-range can beinstruments applied to can derive be applied water quality to derive parameters water quality for monitoring parameters purposesfor monitoring and for purposes the validation and for of the optical validation satellite of data. optical Management satellite data. authorities Management may implementauthorities themay use of such sensors over traditional point measurements with the advantage being that estimates of water quality parameters are available almost instantly with high accuracy, enabling fast management decisions [15,59]. Remote Sens. 2020, 12, 1578 15 of 20

Citizen science offers a low-cost and potential solution for mapping environmental parameters such as water quality at a continental scale [21]. Although volunteered information has enhanced geographical and environmental data at virtually no cost, it has also prompted concerns in regards to the quality, reliability and the overall value of such data [60]. For this reason, we have tested the performance of the HC and EoW app and analysed the accuracy and precision of the collected information. Spectral measurements were taken with the Satlantic spectroradiometer as a reference surface reflectance measurement for the evaluation of the low-cost systems. A total of 13 inland waters with extreme differences in composition were visited and observations were carried out under strict professional protocols. A summary of the differences between the two apps can be found in [34]. The EoW App collects information on the water color, expressed in two related units; the hue angle and the FUI [31]. The observer reports a FUI, supported by a visual tool in the App. In the server and reported on the website, the FUI is also derived from the image taken by the observer. Overall, the results presented here suggest that the EoW app may measure the color of the water with good accuracy. In only 3% of the images, the observer and image reported a difference of more than 2 FUI units. The user-assigned values were ~0.5 FUI units higher than the processed values. Although this bias must be studied in more detail to find out its origin, we suggest five possible causes: (1) the difficulty of estimating the color of the digital image taken within the field. Although a shade cloth was used to view the phone screen in bright conditions, glare and reflectance from the water surface significantly impacted upon the ability to select a color which matched the image taken. (2) Saturation and brightness also impact upon how the color of water is perceived. One option is to take, one way or another, these effects into the color coding e.g., [29]. (3) Larger discrepancies occur at sites with patchy sky conditions. (4) It was difficult to obtain a matching FU_Value for darker waters such as Ewen Maddock Dam and Lostock Reservoir due to the relatively high contribution of reflection from the water surface. (5) The server software (WACODI) selects a section of the image that is most stable and has the lowest hue angle. Results presented in Figure 11 provide evidence that the hue angle, and thus the corresponding FUI in the image centre, where the observer has focussed, has a slightly higher value. The color and the FUI should be considered as significant additional attributes of natural waters. Optically active constituents of the water, such as CHL, SPM and CDOM, together determine the overall reflection and its color. In this paper, we have shown that for Australian inland waters, there does not exist a simple inverse relation between color and CHL or SPM or Secchi disk depth. In a study by Wernand et al. [61], it was shown that in special cases, such as for the open ocean, a simple relation can be found for the FUI and CHL. Therefore, they were able to convert trends in the FUI over decades to trends in algal concentration. For inland waters we are at present aware of two publications that connect color to the state of water quality. At a very local scale of the Kesses Dam in Kenya, Ouma et al. [34] developed a local algorithm where small changes in SPM or CHL were correlated to changes in color, based on a local calibration. At the global scale, Wang et al. [28] demonstrated how the FUI derived from MODIS 500 m resolution images can be used to estimate the trophic state of lakes worldwide. This study is a milestone in color assessment of inland waters and can help to have a better understanding of the color derived from EoW. The HC App has been developed as an ‘easy to use interface’ to calculate the water surface reflectance, using the three RGB channels of the phone camera as radiometers and following a measurement protocol that is standard for above-water hyperspectral measurement of water Rrs [32]. Despite the fact that HC is able to cover the three important components of the light field (total irradiation, sky radiation and water-leaving radiance), instead of only one in EoW, the HC Rrs compared poorly to the Satlantic spectroradiometer. The qualitative comparisons of RGB spectra from Satlantic, EoW and HC show, in many cases, large similarities but the presence of systematic and random errors in the retrieved information makes HC difficult to use as a monitoring device for SPM and turbidity. Remote Sens. 2020, 12, 1578 16 of 20

Because the whole concept of HC is physically sound, it certainly has the potential to become a key smartphone app for water managers. However, we suggest some further investigation, based on our field campaign and the results presented in [32–34]. Variation in the smartphone-derived Rrs (RGB) values may be expected due to the viewing geometry, detectors and spectral band-pass function used to record the amount of electromagnetic radiation and smartphone settings and processing methods of images. The replicate and cloud cover assessment showed that the HC App performed poorer under patchy sky conditions resulting in a higher standard deviation between the replicates taken. This is inherent to the observation sequence that takes some time between the images, during which illumination conditions might have changed. This suggests that it is important to take multiple measurements, especially under cloudy conditions to detect and filter out the major outliers. Until recently, information on smartphone spectral response functions was lacking and we were dependant on the manufacturer to make a proper conversion of the raw counts in RGB to the sRGB format. Thereby, errors can be introduced that depend on illumination and reflection properties. Recently, Burggraaff et al. [62] presented a detailed laboratory study of the characteristics of smartphone cameras, including the spectral response function. As a final remark, this study and the previous study of the EoW results [31] found no major deviations in the sRGB products provided by Android or iPhone cameras. The iPhone camera setting during our campaign sometimes resulted in extremely low or zero values in the blue band. This has also been detected by Yang et al. [33], who also could not find an explanation for this behaviour. The most important error in the overall post processing steps in the HC App seems to be the handling of the non-linearity of the image material. [32] tested the linearity of the iPhone in the lab and concluded that the impact of sRGB conversion and Gamma correction is small. However, there are a number of other findings which contradict this: [62] demonstrated that the Gamma correction is significant, varying between 2.2 and 2.45 with the color (RGB) and position in the image. Furthermore, in WACODI, a Gamma index of 2.2 is adopted for all positions and all bands; this correction is essential to retrieve reliable hue angles. The results of this campaign also demonstrate that HC Rrs derived from waters with a strong blue (hue angle larger than 90 degrees) or red (hue angle less than 40 degrees) color are exaggerated. This is exactly the expected impact of the Gamma expansion in the processing from raw image to sRGB. Finally, it was also found that for the clear blue waters between Vancouver and Vancouver Island in Canada, the dynamic range in blue/green ratios of the Ramses spectrometers was systematically different from the HC Rrs blue/green ratio [33].

5. Conclusions This analysis has been undertaken to test the usability, strengths and weaknesses of the EyeOnWater and HydroColor Apps as citizen science methods to obtain and portray information relevant for water quality monitoring. After extensive testing on multiple water bodies, there is a degree of confidence that the modern FU scale presented within the EoW App is appropriate and provides a fairly accurate estimation of the condition of the water. The testing has shown that an observer can capture images with the App, and select the corresponding color of the water body to provide an accurate estimation of the quality of the water. This analysis has shown that, although highly useful in its own right, the relationship between water color and water quality is complex and that the EoW App should not be used as a surrogate for other water quality variables of interest; this is not the intention of the FU scale. The FU scale may simply be an indication of the visual appearance of the water surface to monitor over the seasons and prompt further laboratory water quality testing. The results for the HC App were less conclusive than expected. In these difficult circumstances, with sun almost near zenith and a wide range of waters, from highly reflective (Brisbane River) to highly absorptive (Lake St Clair), the results were prone to errors. Before an assessment can be made of Remote Sens. 2020, 12, 1578 17 of 20

the applicability of the SPM algorithm in the App, it is essential to ﬁrst reduce the errors in the retrieved Rrs. It remains important to take multiple measurements, especially under cloudy conditions. Finally, it is important to note that both the HydroColor and EyeOnWater Apps were only tested by two users. In order to gauge an understanding of the inﬂuence of the user on the quality of the results, these Apps should be tested on a larger group of users.

Supplementary Materials: The following are available online at http://www.mdpi.com/2072-4292/12/10/1578/s1, Figure S1: Mean raw spectra for panel and water surface measurements made at Station 6 on Lake Liddell, 02.03.2016, presented together with standard deviation and coefficient of variation, Figure S2: Model outcomes to show changes in hue angle when a spectrometer is lowered to 5 cm depth in natural waters, Table S1: Summary of the basic measurements from all stations. Author Contributions: The general set up of the measurement campaign was developed by T.J.M. R.O. and T.J.M. carried out the field measurements and H.J.v.d.W. assisted with data handling with the support of other CSIRO staff. This work formed RO’s extended Honours study under the supervision of T.J.M. and Stuart Phinn. R.O., T.J.M. and H.J.v.d.W. all interpreted the dataset and wrote the article. All authors have read and agreed to the published version of the manuscript. Funding: This project was in part supported by funding from the New South Wales Office of the Premier in a project entitled “Early detection of algal blooms using remote sensing”. HvdW received travel support from the “Citizen Science helping improve satellite detection of water quality” project supported by the Australian Federal Department of Industry, Science, Energy and Resources (grant number CSG56397), led by Janet Anstee (CSIRO). Acknowledgments: We wish to thank Heidi Franklin, Janet Anstee and Hannelie Botha (all from CSIRO) for their assistance in the field work and laboratory training. Access to South Eastern Queensland water reservoirs was kindly granted by Seqwater (seqwater.com.au); our thanks to Deb Gale and Cameron Veal for useful discussions. We also acknowledge the support of Adrian D’Alessandro (CSIRO) in the conversion of WACODI to Python script. Conflicts of Interest: The authors declare no conflict of interest.

References

1. Khalil, B.; Ouarda, T.B.M.J.; St-Hilaire, A.; Chebana, F. A statistical approach for the rationalization of water quality indicators in surface water quality monitoring networks. 2010. J. Hydrol. 2010, 386, 173–185. [CrossRef] 2. Paerl, H.W.; Huisman, J. Climate change: A catalyst for global expansion of harmful cyanobacterial blooms. Environ. Microbiol. Rep. 2009, 1, 27–37. [CrossRef] 3. Fritz, S.; See, L.; Carlson, T.; Hakley, M.; Oliver, J.L.; Fraisl, D.; Mondardini, R.; Brocklehurst, M.; Shanley, L.A.; Schade, S.; et al. Citizen Science and the United Nations Sustainable Development Goals. Nat. Sustain. 2019, 2, 922–930. [CrossRef] 4. Dekker, A.G. Detection of Optical Water Quality Parameters for Eutrophic Waters by High Resolution Remote Sensing. Ph.D. Thesis, Vrije University, Amsterdam, The Netherland, 1993. 5. Olmanson, L.G.; Brezonik, P.L.; Bauer, M.E. Airborne hyperspectral remote sensing to assess spatial distribution of water quality characteristics in large rivers: The Mississippi River and its tributaries in Minnesota. Remote Sens. Environ. 2013, 130, 254–265. [CrossRef] 6. Hestir, E.L.; Brando, V.E.; Bresciani, M.; Giardino, C.; Matta, E.; Villa, P.; Dekker, A.G. Measuring freshwater aquatic ecosystems: The need for a hyperspectral global mapping satellite mission. Remote Sens. Environ. 2015, 167, 181–195. [CrossRef] 7. Busch, J.A.; Zielinski, O.; Cembella, A.D. Subsea Optics and Imaging. In Subsea Optics and Imaging; Elsevier: Amsterdam, The Netherlands, 2013. [CrossRef] 8. Gons, H.J.; Rijkeboer, M.; Bagheri, S.; Ruddick, K.G. Optical teledetection of chlorophyll a in estuarine and coastal waters. Environ. Sci. Technol. 2000, 34, 5189–5192. [CrossRef] 9. Gitelson, A.; Gurlin, D.; Moses, W.J.; Barrow, T. A bio-optical algorithm for the remote estimation of the chlorophyll- a concentration in case 2 waters. Environ. Res. Lett. 2009, 4, 45003. [CrossRef] 10. Matthews, M.W. A current review of empirical procedures of remote sensing in inland and near-coastal transitional waters. Int. J. Remote Sens. 2011, 32, 6855–6899. [CrossRef] 11. Dekker, A.G.; Vos, R.J.; Peters, S.W.M. Analytical algorithms for lake water TSM estimation for retrospective analyses of TM and SPOT sensor data. Int. J. Remote Sens. 2002, 23, 15–35. [CrossRef] Remote Sens. 2020, 12, 1578 18 of 20

12. Nechad, B.; Ruddick, K.G.; Park, Y. Calibration and validation of a generic multisensor algorithm for mapping of total suspended matter in turbid waters. Remote Sens. Environ. 2010, 114, 854–866. [CrossRef] 13. Al-Kharusi, E.S.; Tenenbaum, D.E.; Abdi, A.M.; Kutser, T.; Karlsson, J.; Bergström, A.-K.; Berggren, M. Large-Scale Retrieval of Coloured Dissolved Organic Matter in Northern Lakes Using Sentinel-2 Data. Remote Sens. 2020, 12, 157. [CrossRef] 14. Dekker, A.; Hestir, E. Evaluating the Feasibility of Systematic Inland Water Quality Monitoring with Satellite Remote Sensing; CSIRO: Canberra, Australia, 2012. 15. Hommersom, A.; Kratzer, S.; Laanen, M.; Ansko, I.; Ligi, M.; Bresciani, M.; Giardino, C.; Beltrán-Abaunza, J.M.; Moore, G.; Wernand, M.; et al. Intercomparison in the field between the new WISP-3 and other radiometers (TriOS Ramses, ASD FieldSpec, and TACCS). J. Appl. Remote Sens. 2012, 6, 63615. [CrossRef] 16. Hunter, P.D.; Tyler, A.N.; Carvalho, L.; Codd, G.A.; Maberly, S.C. Hyperspectral remote sensing of cyanobacterial pigments as indicators for cell populations and toxins in eutrophic lakes. Remote Sens. Environ. 2010, 114, 2705–2718. [CrossRef] 17. Li, L.; Sengpiel, R.E.; Pascual, D.L.; Tedesco, L.P.; Wilson, J.S.; Soyeux, E. Using hyperspectral remote sensing to estimate chlorophyll- a and phycocyanin in a mesotrophic reservoir. Int. J. Remote Sens. 2010, 31, 4147–4162. [CrossRef] 18. Le, C.; Li, Y.; Zha, Y.; Sun, D.; Huang, C.; Zhang, H. Remote estimation of chlorophyll a in optically complex waters based on optical classification. Remote Sens. Environ. 2011, 115, 725–737. [CrossRef] 19. O’Donnell, D.M.; Effler, S.W.; Strait, C.M.; Peng, F.; Perkins, M. Remote sensing reflectance in the Great Lakes: In situ measurements, closure analyses, and a forward model. J. Great Lakes Res. 2013, 39, 137–150. [CrossRef] 20. Kudela, R.M.; Palacios, S.L.; Austerberry, D.C.; Accorsi, E.K.; Guild, L.S.; Torres-Perez, J. Application of hyperspectral remote sensing to cyanobacterial blooms in inland waters. Remote Sens. Environ. 2015, 167, 196–205. [CrossRef] 21. Busch, J.A.; Badají, R.; Ceccaroni, L.; Friedrichs, A.; Piera, J.; Simon, C.; Thijsse, P.; Wernand, M.; Van der Woerd, H.J.; Zielinski, O. Citizen bio-optical observations from coast- and ocean and their compatibility with ocean colour satellite measurements. Remote Sens. 2016, 8, 879. [CrossRef] 22. Brando, V.E.; Lovell, J.L.; King, E.A.; Boadle, D.; Scott, R.; Schroeder, T. The Potential of Autonomous Ship-Borne Hyperspectral Radiometers for the Validation of Ocean Color Radiometry Data. Remote Sens. 2016, 150. [CrossRef] 23. Lymburner, L.; Botha, B.; Hestir, E.; Anstee, J.; Sagar, S.; Dekker, A.; Malthus, T. Landsat 8: Providing continuity and increased precision for measuring multi-decadal time series of total suspended matter. Remote Sens. Environ. 2016, 185, 108–118. [CrossRef] 24. Malthus, T.J.; Lehmann, E.; Ho, X.; Botha, E.; Anstee, J. Implementation of a satellite based inland water algal bloom alerting system using analysis ready data. Remote Sens. 2019, 11, 2954. [CrossRef] 25. Wernand, M.R.; Hommersom, A.; Van der Woerd, H.J. MERIS-based ocean colour classification with the discrete Forel-Ule scale. Ocean Sci. 2013, 9, 477–487. [CrossRef] 26. Van der Woerd, H.J.; Wernand, M.R. True colour classification of natural waters with medium-spectral resolution satellites: SeaWiFS, MODIS, MERIS and OLCI. Sensors 2015, 15, 25663–25680. [CrossRef][PubMed] 27. Van der Woerd, H.J.; Wernand, M.R. Hue angle product for low to medium spectral resolution optical satellite sensors. Remote Sens. 2018, 10, 180. [CrossRef] 28. Wang, S.; Junsheng, L.; Zhang, B.; Spyrakos, E.; Tyler, A.N.; Shen, Q.; Zhang, F.; Kutser, T.; Lehmann, M.K.; Wu, Y.; et al. Trophic state assessment of global inland waters using a MODIS-derived Forel-Ule index. Remote Sens. Environ. 2018, 217, 444–460. [CrossRef] 29. Lehmann, M.K.; Nguyen, U.; Allan, M.; Van der Woerd, H.J. Colour classification of 1486 lakes across a wide range of optical water types. Remote Sens. 2018, 10, 1273. [CrossRef] 30. Giardino, C.; Koks, K.; Bolpagni, R.; Luciani, G.; Candiani, G.; Lehmann, M.K.; Van der Woerd, H.J.; Bresciani, M. The color of water from space: A case study for Italian lakes from Sentinel-2. In Earth Observation and Geospatial Analysis; Intech Open: London, UK, 2019. [CrossRef] 31. Novoa, S.; Wernand, M.R.; Van der Woerd, H.J. WACODI: A generic algorithm to derive the intrinsic color of natural waters from digital images. Limnol. Oceanogr. Methods 2015, 13, 697–711. [CrossRef] 32. Leeuw, T.; Boss, E. The HydroColor App: Above water measurements of remote sensing reflectance and turbidity using a smartphone camera. Sensors 2018, 18, 256. [CrossRef] Remote Sens. 2020, 12, 1578 19 of 20

33. Yang, Y.; Cowen, L.L.E.; Costa, M. Is ocean reflectance acquired by citizen scientists robust for science applications? Remote Sens. 2018, 10, 835. [CrossRef] 34. Ouma, Y.O.; Waga, J.; Okech, O.; Lavisa, O.; Mbuthia, D. Estimation of reservoir bio-optical water quality parameters using smartphone sensor apps and Landsat ETM+: Review and comparative experimental results. J. Sens. 2018, 3490757. [CrossRef] 35. Fargoin, G.S.; Mueller, J.L. Ocean Optics Protocols for Satellite Ocean Color Sensor Validation, 3rd ed.; NASA Goddard Space Flight Center: Greenbelt, MD, USA, 2002. 36. Tilstone, G.H.; Moore, G.F.; Doerffer, R.; Røttgers, R.; Ruddick, K.G.; Pasterkamp, R.; Jørgensen, P.V. Regional Validation of MERIS Chlorophyll products in North Sea REVAMP Protocols. In Proceedings of the Working Meeting on MERIS and AATSR Calibration and Geophysical Validation (ENVISAT MAVT-2003), Frascati, Rome, Italy, 20–24 October 2003; ESA Special Publication WPP-233: Noordwijk, The Netherlands, 2002; pp. 1–77. 37. Clementson, L.A.; Parslow, J.S.; Turnbull, A.R.; Mckenzie, D.C.; Rathbone, C.E. Optical properties of waters in the Australasian sector of the Southern Ocean. J. Geophys. Res. Oceans 2001, 106, 31611–31625. [CrossRef] 38. Zibordi, G.; Talone, M. On the equivalence of near-surface methods to determine the water-leaving radiance. Opt. Express 2020, 28, 3200. [CrossRef][PubMed] 39. Ruddick, K.G.; Vos, K.; Boss, E.; Castagna, A.; Frouin, R.; Gilerson, A.; Hieronymi, M.; Johnson, B.C.; Kuusk, J.; Lee, Z.; et al. A Review of Protocols for Fiducial Reference Measurements of Water-Leaving Radiance for Validation of Satellite Remote-Sensing Data over Water. Remote Sens. 2019, 11, 2198. [CrossRef] 40. Simis, S.G.H.; Peters, S.W.M.; Gons, H.J. Remote sensing of the cyanobacterial pigment phycocyanin in turbid inland water. Limnol. Oceanogr. 2005, 50, 237–245. [CrossRef] 41. Ruiz-Verdu, A.; Simis, S.G.H.; De Hoyos, C.; Gons, H.J.; Peña-Martínez, R. An evaluation of algorithms for the remote sensing of cyanobacterial biomass. Remote Sens. Environ. 2008, 112, 3996–4008. [CrossRef] 42. Gitelson, A.A.; Dall’Olmo, G.; Moses, W.; Rundquist, D.C.; Barrow, T.; Fisher, T.R.; Gurlin, D.; Holz, J. A simple semi-analytical model for remote estimation of chlorophyll-a in turbid waters-Validation. Remote Sens. Environ. 2008, 112, 3582–3593. [CrossRef] 43. Li, L.; Li, L.; Song, K. Remote sensing of freshwater cyanobacteria: An extended IOP Inversion Model of Inland Waters (IIMIW) for partitioning absorption coefficient and estimating phycocyanin. Remote Sens. Environ. 2015, 157, 9–23. [CrossRef] 44. Liu, G.; Lin, L.; Song, K.; Li, Y.; Lyu, H.; Wen, Z.; Fang, C.; Bi, S.; Sun, X.; Wang, Z.; et al. An OLCI-based algorithm for semi-empirically partitioning absorption coefficient and estimating chlorophyll a concentration in various turbid case-2 waters. Remote Sens. Environ. 2020, 239, 111648. [CrossRef] 45. Ogashawara, I.; Mishra, D.R.; Mishra, S.; Curtarelli, M.P.; Stech, J.L. A Performance Review of Reflectance Based Algorithms for Predicting Phycocyanin Concentrations in Inland Waters. Remote Sens. 2013, 5, 4774–4798. [CrossRef] 46. Spyrakos, E.; O’Donnell, R.; Hunter, P.D.; Miller, C.; Scott, M.; Simis, S.G.H.; Niel, C.; Barbosa, C.; Binding, C.E.; Bradt, S.; et al. Optical types of inland and coastal waters. Limnol. Oceanogr. 2017, 63, 846–870. [CrossRef] 47. Kirk, J.T. Light and Photosynthesis in Aquatic Ecosystems, 3rd ed.; Cambridge University Press: Cambridge, NY, USA, 2011. 48. CIE. Commission Internationale de l’Éclairage Proceedings, 1931; Cambridge University Press: Cambridge, UK, 1932; pp. 19–29. 49. Novoa, S.; Wernand, M.R.; Van der Woerd, H.J. The Forel-Ule scale revisited spectrally: Preparation protocols, transmission measurements and chromaticity. J. Eur. Opt. Soc. RP 2013, 8, 13057. [CrossRef] 50. Leeuw, T. Crowdsourcing Water Quality Data Using the iPhone Camera; Paper 2118; University of Maine: Orono, ME, USA, 2014; Available online: http://digitalcommons.library.umaine.edu/etd/2118 (accessed on 24 January 2020). 51. Garaba, S.; Voss, D.; Wollschlager, J.; Zielinski, O. Modern approaches to shipborne ocean color remote sensing. Appl. Opt. 2015, 54, 3602–3612. 52. Mueller, J.L.; Fargion, G.S.; Mcclain, C.R.; Mueller, J.L.; Morel, A.; Frouin, R.; Davis, C.; Arnone, R.; Carder, K.; Steword, R.G.; et al. NASA/TM-2003-Ocean Optics Protocols for Satellite Ocean Color Sensor Validation, Revision 4, Volume III: Radiometric Measurements and Data Analysis Protocols; NASA: Washington DC, USA, 2003. 53. Mobley, C.D. Estimation of the remote-sensing reflectance from above-surface measurements. Appl. Opt. 1999, 38, 7442–7455. [CrossRef][PubMed] Remote Sens. 2020, 12, 1578 20 of 20

54. Pascale, D. A review of RGB color spaces... from xyY to R’G’B’. 2003. Babel Color. pp. 1–35. Available online: www.babelcolor.com (accessed on 24 January 2020). 55. Lindell, L.T.; Pierson, D.; Premazzi, G.; Zillioli, E. Manual for Monitoring European Lakes Using Remote Sensing Techniques; Joint Research Centre European Commission: Ispra, Italy, 1999; ISBN 978-9282853900. 56. Van der Woerd, H.J.; Pasterkamp, R. Mapping of the North Sea turbid coastal waters using SeaWiFS data. Can. J. Remote Sens. 2004, 30, 44–53. [CrossRef] 57. Pitarch, J.; Van der Woerd, H.J.; Brewin, R.J.W.; Zielinsky, O. Optical properties of Forel-Ule water types deduced from 15 years of global satellite ocean color observations. Remote Sens. Environ. 2018, 231, 111249. [CrossRef] 58. Malthus, T.J.; Lehmann, E.; Ho, X.; Gensemer, S.; Anstee, J.; Botha, E.; Brayan, J.; Bowling, L.; Shaikh, M.; Ohmsen, R.; et al. An algal bloom early warning system for NSW using satellite and near-surface observations. In Proceedings of the OzWater17 Conference, 16–18 May 2017; Australian Water Association: Municipality, Australia. Available online: https://awa.shareﬁle.com/share?#/view/s69fcd4a5d344354b (accessed on 23 February 2020). 59. Bowling, L.; Shaikh, M.; Brayan, J.; Malthus, T. An evaluation of a handheld spectroradiometer for the near real-time measurement of cyanobacteria for bloom management purposes. Environ. Monit. Assess. 2017, 189, 495. [CrossRef] 60. Flanagin, A.J.; Metzger, M.J. The credibility of volunteered geographic information. GeoJournal 2008, 72, 137–148. [CrossRef] 61. Wernand, M.R.; Van der Woerd, H.J.; Gieskes, W.W.C. Trends in Ocean Colour and Chlorophyll Concentration from 1889 to 2000, Worldwide. PLoS ONE 2013, 8. [CrossRef] 62. Burggraaﬀ, O.; Schmidt, N.; Zamorano, J.; Puly, K.; Pascual, S.; Tapia, C.; Spyrakos, E.; Snik, F. Standardized spectral and radiometric calibration of consumer cameras. Opt. Express. 2019, 19075. [CrossRef]

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).