Misidentifications in Citizen Science Bias the Phenological Estimates

Ibis (2021) doi: 10.1111/ibi.12985

Misidentiﬁcations in citizen science bias the phenological estimates of two hard-to-identify Elaenia ﬂycatchers

FABRICIO C. GORLERI*1,2 & JUAN I. ARETA1 1Laboratorio de Ecologıa, Comportamiento y Sonidos Naturales, Instituto de Bio y Geociencias del Noroeste Argentino (IBIGEO-CONICET), Salta, Argentina 2Aves Argentinas/Asociacion Ornitologica del Plata, Buenos Aires, Argentina

Citizen science initiatives contain a large volume of observations that can be useful to address ecological questions for a wide array of organisms. However, one limitation of citizen science data is the potential for species misidentification. Although recent studies have shown that citizen science data are relatively accurate for many taxa, the effect of misidentification errors in hard-to-identify species remains poorly explored. If misidentification events occur at large scales, ecological estimates can be compromised. Here, we show that misidentifications contained in citizen science databases biased phenological estimates in a pair of migratory and partially overlapping Neotropical flycatchers: the Chilean Elaenia Elaenia chilensis and the Small-billed Elaenia Elaenia parvirostris.We reviewed and re-classified 4399 photos of these species from Argentina, Chile and Uru- guay. We found that overall identification accuracy was high (c. 90%) for both species when they were allopatric, but dramatically low for Small-billed Elaenia during autumn migration (from 28.6% to 84.6%) because migrating individuals of Chilean Elaenia were systematically reported as Small-billed. The phenological estimates for both Elaenias were biased due to the large number of misidentifications concentrated towards the autumn migration period. These errors caused a 1-week advancement in the estimated arrival, and a 2-week delay in the estimated departure for Small-billed Elaenia. For Chi- lean Elaenia, errors caused a 1-week delay for the estimated spring peak passage and underestimation of the magnitude of the autumn passage. Our results highlight the importance of performing critical assessments of records when using citizen science databases to describe ecological patterns in species that are hard to identify. The large volume of information provided by citizen science initiatives is useful to describe spatio- temporal patterns in birds, particularly in those of poorly known regions. However, to further enhance the usefulness of such databases, it is imperative actively to post-process and contrast patterns derived from documented (and undocumented) records, with a special focus on misidentifications. This will only be possible through a thorough review of the documented data, together with an intimate understanding of the natural history of the study species. Keywords: Argentina, Chilean Elaenia, cryptic species, eBird, EcoRegistros, identification accuracy, phenology, Small-billed Elaenia, Tyrannidae, Uruguay.

Citizen science initiatives harbour a large volume address ecological questions for a wide array of of observations that have enormous potential to organisms (Bonney et al. 2009, Dickinson et al. 2012). These projects are increasingly recognized as valuable sources of information. However, the * Corresponding author. reliability and accuracy of the data are still poorly Email: [email protected] Twitter: @FabriGorleri explored, especially in species that pose

identification challenges. Because the ability to to check the accuracy of data on cryptic taxa by correctly identify organisms is not uniform across assessing the potential direction and magnitude of participants (Dickinson et al. 2010, Kelling et al. errors before analysis. The reconstruction of 2015), citizen science databases often contain vari- spatio-temporal patterns of species for which iden- able amounts of misidentified records. Surprisingly, tification errors can be problematic should be although identification errors are a known concern preferably based on validated or documented, veri- for data analysis (Royle & Link 2006), few studies fied data (Areta et al. 2016, Aubry et al. 2017). confront problems related to this type of error In the Neotropical region, multiple species in when using citizen science databases to describe the bird genus Elaenia (Aves: Tyrannidae) are sim- ecological patterns (Ensing et al. 2013). Therefore, ilar looking and are among the hardest birds to there is a need to further evaluate the effects of identify (Ridgely & Tudor 1994, Straube 2013), identification errors in analyses using citizen posing challenges to the study of their ecological science data. patterns using citizen science databases. Elaenias Misidentifications occur most often between are dull-toned, medium-sized flycatchers, with coexisting, similar-looking taxa. Recent studies that eye-rings of variable strength, bold wing bars, and have analysed the identification accuracy of citizen often with a semi-concealed white coronal patch science participants found that identification accu- in adult plumage (Ridgely & Tudor 1994). While racy could be lower than 70% for similar-looking visual identification is difficult, vocalizations are plants (Crall et al. 2011), mammals (Swanson generally the best aid for identification in this et al. 2016) and insects (Falk et al. 2019). Like- group (Minns 2018, Pearman & Areta 2020) and wise, the error rate increases in birds that co-occur therefore silent birds can be easily misidentified. and are similar in appearance (Hull et al. 2010, Most Elaenias are migratory and some species can Kelling et al. 2015). Although several citizen be found together in certain periods, which science projects have developed data-validation increases the risk of confusion (Pearman & Areta systems to target unusual reports and potential 2020). Because citizen science databases are misidentifications, these systems often fail to flag increasingly used to explore spatio-temporal pat- identification errors between similar-looking spe- terns in Neotropical birds (Schubert et al. 2019), cies that are likely to be found together (Bonter & there is a need to assess the quality of data and Cooper 2012). In such species, therefore, the the reliability of phenological analyses derived amount of noise resulting from species misidentifi- from them, with a special focus on cryptic species cations is generally unknown, and quantification of groups such as the Elaenias. the errors is needed to evaluate the accuracy of Here, we evaluate the accuracy of documented the data. citizen science data and their performance to Although non-systematic and sporadic identifi- describe the phenology of two completely migra- cation errors may have a negligible impact on data tory Elaenia species with partial spatio-temporal analysis, problems arise when errors are abundant overlap: the Chilean Elaenia Elaenia chilensis and and systematic (Costa et al. 2015). For example, the Small-billed Elaenia Elaenia parvirostris. The when one species is continuously confused with Chilean Elaenia breeds mostly along a thin strip in one or more similar species, this leads to the the southern Andes and is narrowly parapatric (in simultaneous addition of false-positives (i.e. reports the north) or widely allopatric (in the south) with of an absent species) and false-negatives (i.e. lack the lowland/foothill breeding Small-billed Elaenia of reports of a species that is present) to databases. (Fig. 1). However, geolocator studies indicate that If not properly addressed, these false-positives and Chilean Elaenia migrate through the lowlands of false-negatives can strongly bias ecological esti- Argentina, Paraguay and Uruguay towards north- mates (Miller et al. 2011, 2013). Patterns derived east Brazil, leading to momentary sympatry with from noisy data would be expected to shift toward Small-billed Elaenia during both autumn and the distribution of the contaminating data, there- spring migration (Jimenez et al. 2016, Bravo et al. fore providing unrealistic information. This effect 2017; see Fig. 1). Because these species are mostly may be exacerbated if the distribution of errors is allopatric, misidentifications between them are skewed in comparison with the distribution of true more likely to occur when they are found together data (Molinari-Jobin et al. 2012, Costa et al. during the migratory periods, and thus the accu- 2015). For these reasons, it is especially important racy rate may vary strongly across space and time.

Figure 1. Schematic distribution maps of Chilean Elaenia Elaenia chilensis and Small-billed Elaenia E. parvirostris, and migration routes of Chilean Elaenia. Migration routes are based on geolocator data given in Jimenez et al. (2016) and Bravo et al. (2017). Distributional maps were based on our assessment of documented records and Ridgely and Tudor (1994). Note that Chilean Elaenia migrates over the breeding range of Small-billed Elaenia.

To achieve our goals, we reviewed and classified phenological estimates, we expected a systematic as correct, wrong or uncertain photographic error of migrating Chilean Elaenia being misidenti- reports of Chilean and Small-billed Elaenias in fied as Small-billed Elaenia; therefore, we predicted Argentina, Chile and Uruguay that were uploaded that the phenological function built with the to citizen science platforms, and assessed their curated dataset would differ markedly from that year-round and weekly identification accuracy. We built with raw data, in particular by estimating subsequently compared how the original, raw more intense migratory passages for Chilean Elaenia dataset and an actively post-processed, curated and weekly differences in either the arrival or the dataset estimated different phenological attributes departure dates for Small-billed Elaenia. for each species (peak-passage dates and intensity based on the number of records for Chilean Elae- METHODS nia and arrival/departure dates for Small-billed Elaenia) at the area of overlap (i.e. within the Data collection and validation breeding range of Small-billed). Regarding identification accuracy, we predicted a lower identification We gathered photographic reports of Chilean and accuracy at times of geographical overlap during Small-billed Elaenia from eBird (Sullivan et al. migration because of reciprocal misidentifications, 2009, www.ebird.org) and EcoRegistros (www.ec and a higher identification accuracy at times of geo- oregistros.org). These platforms are popular citizen graphical segregation during breeding. For the science databases that allow the submission of bird

species lists and single observations with the identification by song (including the use of play- option to add multimedia (photo, video or audio). back when necessary). We found and observed a eBird also offers the possibility to include informa- total of 41 Chilean Elaenia and 288 individuals of tion on survey effort per list (Sullivan et al. 2014). Small-billed Elaenia in 87 survey events, complet- To target misidentifications, each platform has ing 160 h of observations (all of our sightings are developed a quality-control system. eBird primarily available at eBird). uses smart filters that automatically flag unusual Additionally, we carefully reviewed in situ more entries for expert review, and the users can also than 500 specimens of Chilean and Small-billed participate by flagging wrong reports from the Elaenia from the American Museum of Natural public output (Sullivan et al. 2014). EcoRegistros History (AMNH), United States National Museum is based on manual detection of wrong entries, (USNM) and Instituto Miguel Lillo de Ciencias where both the public and the site owners partici- Naturales (IML). We took notes of coloration and pate. We gathered the reports used in this study overall appearance of individuals with a high cer- from the public output of each platform, and tainty of identification; for example, we considered these therefore have gone through their own only Chilean Elaenias that were collected in Chi- reviewing process, being either validated or not lean and Argentine Patagonia (the only Elaenia in properly corrected. these areas) and Small-billed Elaenias collected in We filtered reports submitted in Argentina, northeast Argentina during December to January Uruguay and Chile between 1998 and 2020. We (when Chilean Elaenia is not known to be present assessed the identity of 2059 photos reported as and is far from areas of possible overlap). Chilean Elaenia, and 2340 photos reported as Finally, we examined the key literature describ- Small-billed Elaenia. We classified each photo ing both species: Belton (1985), Ridgely and record as correct, wrong or uncertain. We consid- Tudor (1994), Straube (2013), Pyle et al. (2015), ered photos to be uncertain when they did not and Pearman and Areta (2020). A primer to the allow us to reach species-level identification but identification features of Chilean Elaenia and had a possibility of being correctly identified. Even Small-billed Elaenia used in the present work is after training and specimen examination (see shown in Supporting Information Appendix S2, below), we were aware of the difficulty of identi- Table S1, and Figs S1 and S2. fying Elaenias visually in photographic reports. For this reason, any doubtful photo was conservatively Data cleaning treated as uncertain. Wrong reports were assigned to the correct species whenever possible in our Citizen science participants can upload multiple database. We also explored how wrong reports photos of one individual (bird) during a sampling were distributed among users, to evaluate whether event, and also upload the same photo(s) to differ- the misidentifications were concentrated on a few ent platforms; we therefore had to ensure the users or whether the errors were evenly distributed independence of each observation by removing among observers. Although the platforms differ in these duplicate reports. To achieve this, we their quality-control systems, we did not find sig- removed duplicate reports that contained the same nificant differences in the general misidentification information on species, locality, observer and date. ratio of photographic reports between them for However, because the locality (geographical coor- the Elaenias (for more detail see Supporting Infor- dinates) of a report and the names of the observers mation Appendix S1). Therefore, we decided to may vary from one platform to another, we stan- combine the information of both platforms to aug- dardized this information to be able accurately ment the sample size for the analyses (Fig. 2). to remove duplicates. We manually fixed and Because Chilean and Small-billed Elaenias are standardized the user names when possible; for extremely similar in appearance, we had to train example, when one user had different user names our identification skills to classify photos accu- in each platform (e.g. Pablo Capovilla in EcoRegis- rately. To achieve this, we carefully observed all tros and Pablo Hernan Capovilla in eBird), we the Elaenias we encountered during opportunistic selected one and changed the other to match. To field trips that we took in Argentina from 2018 to standardize the localities of the reports, we gener- 2020, taking field notes of the overall appearance ated a locality identifier encompassing nearby of each individual with confirmation of reports. To generate the locality identifier, we

Figure 2. The data-ﬁltering process from the citizen-science data sources to the creation of the raw and curated datasets of Chilean Elaenia Elaenia chilensis and Small-billed Elaenia E. parvirostris.

created a 10-km hexagonal grid across Argentina, phenological attributes: peak date and intensity of Chile and Uruguay with the R package dggridR migration in Chilean Elaenia based on the num- (Barnes 2017), and then assigned the value of the ber of records, and arrival/departure dates in corresponding cell as a locality identifier to each Small-billed Elaenia. The phenological attributes report, effectively grouping all records that lay were described across the overlapping area of the within the same cell under the same identifier. We two species (i.e. the breeding range of Small- chose to use hexagons rather than squares because billed Elaenia, see Fig. 1), where misidentification hexagons are the most circular of all regular poly- probably occurs at high rates. We chose to mea- gons that can be tesselated, and therefore provide sure different phenological attributes between less spatial distortion (Sahr 2011). Finally, we pro- these species because Chilean Elaenia migrates ceeded to select only one record from those con- over the breeding area of Small-billed Elaenia, taining the same information on reported species, and therefore the peak of migration better locality identifier, user name and date. The result- explains the phenology of this species across the ing dataset consisted of unique reports of Chilean area of overlap rather than the arrival or depar- and Small-billed Elaenias (Fig. 2). ture dates. We filtered the records of Chilean Elaenia from within the area of overlap for the pheno- Identification accuracy analysis logical analysis, defining the boundaries of the We calculated the year-round and weekly identifi- area using a minimum convex polygon encom- cation accuracy for Chilean and Small-billed Elae- passing all the correct reports of Small-billed nias using all their unique reports. These Elaenia during the breeding season (November calculations allowed us to verify the overall accu- to February). This area-filtered dataset consti- racy of photographic reports and whether the tuted the raw dataset used to model the phe- accuracy rate varied across the annual cycle. We nology of each Elaenia (Fig. 2, Supporting measured accuracy as the percentage of correct Information Appendix S3). Next, we created a photos over the total number of identifiable pho- curated dataset for each species and used this tos (i.e. excluding those tagged as uncertain from as a benchmark to compare against the raw the total). We used weeks rather than days as our dataset in the phenological analysis. The curated replicate unit because our sample size was too low dataset consisted of a set of reports where we to allow daily analysis; nonetheless, the use of manually added the correct observations to the weeks worked well for describing the intended corresponding species and removed those that patterns in our study. We assigned the correspond- were uncertain or wrong (Fig. 2). For example, ing week to each record with the R package when a photographic report of Small-billed lubridate (Grolemund & Wickham 2011). There Elaenia corresponded to a Chilean Elaenia, we were some weeks with few records, especially dur- removed this record from the Small-billed data- ing the beginning and end of the migratory peri- set and added it to the Chilean dataset. Because ods. Therefore, in weeks where n < 7, accuracy the amount of data was insufficient to calculate was considered null to avoid biasing estimates. yearly phenologies using the available data, we Finally, we tested whether any observed change in pooled the weekly reports for each species/data- the weekly identification accuracy was statistically set from the entire date range (1998–2020). significant with a two-proportion Z-test (two-sided The raw and curated datasets for modelling thus with continuity correction) in base R. comprised the cumulative number of photos reported by week from 1998 to 2020. As citizen science databases exhibit temporal Phenological analysis variation in sampling effort (i.e. some weeks or Because Chilean and Small-billed Elaenias are seasons being surveyed more frequently), we cor- migratory, we explored whether the phenological rected this temporal bias to ensure that any estimates in both species differed when built with observed increase or decrease in the weekly num- raw data (i.e. data directly gathered from citizen ber of reports of each Elaenia is not simply a func- science platforms) as opposed to curated data (i.e. tion of differences in sampling effort. Because we data free from identification errors to the best of were using only photographic reports, we did not our knowledge). We described the following have true absence data or information on sampling

effort for a direct correction for this bias. Nonethe- Taxonomic note less, an alternative method to correct the bias of presence-only data from citizen science initiatives Although we have adopted the taxonomy of the consists of weighting the data with some other IOC World Bird List (Gill et al. 2021), the situa- information that allows an approximation of sur- tion is complex and the species-level classification vey effort: for example, information of the total of the Chilean Elaenia remains contentious (Pear- amount of records of other species submitted dur- man & Areta 2020, Remsen et al. 2021). Molecu- ing a specific time window (Lees & Martin 2014, lar phylogenetic and vocal data suggest that Schubert et al. 2019). Therefore, assuming that chilensis either constitutes a separate species the photo submission rate in citizen science plat- (Rheindt et al. 2009) or that it can be conspecific forms increases with a more intense community with Sierran Elaenia Elaenia pallatangae (Chat- sampling, we approximated the weekly survey topadhyay et al. 2017, Tang et al. 2018), while dif- effort by using information on the total number of ferent studies agree in that chilensis is only passerine photos submitted by week in each plat- distantly related to the nominate Elaenia albiceps form across the study area, from the same date where it has been traditionally placed (Rheindt range as the reports of the two Elaenia species et al. 2009, Chattopadhyay et al. 2017, Tang et al. (1998–2020). We then used this information to 2018). weight the cumulative number of reports per week for each species/dataset to model their phenologies RESULTS (see below). We described the phenological attributes by fit- We found that 1395 of the 1430 photos of the ting generalized additive models (GAMs) using the Chilean Elaenia were correctly identified, 27 were raw and curated datasets for each species. We wrong and the remaining eight were uncertain. For chose GAMs instead of other methods to describe Small-billed Elaenia, 1348 of 1577 were correct, phenological events because of their flexibility in 173 were wrong and the remaining 56 were uncer- describing complex patterns without specific tain. Therefore, the general identification accuracy assumptions about the shape of the phenological was 98.1% for Chilean Elaenia and 88.6% for distribution (Linden et al. 2017). We fitted bino- Small-billed Elaenia (Table 1). Errors were evenly mial GAMs with a logit link function in the R distributed among users who reported misidenti- package mgcv (Wood 2017). We used the cumula- fied photographs. For Chilean Elaenia, 18 users tive number of weekly reports for each species/da- were responsible for the 27 misidentifications; 17 taset as the response variable, weighted by the of these users submitted one or two erroneous total number of passerine photos submitted by photos, and a single user submitted four (14% of week. We modelled the resulting proportion as a all errors). Likewise, for Small-billed Elaenia, 115 function of time (week of year). For each model, users were responsible for the 173 misidentifica- we estimated the smoothing parameter using the tions. In this case, 104 users made one or two restricted maximum likelihood method (REML), errors, and 11 users made from three to a maxi- which is preferred over other methods to give a mum of 10 (28% of all errors). good fit to the data when the sample size is small Most misidentifications were reciprocal (i.e. (McNeish 2017). We fitted each model with a involving confusions among the two species of cyclic cubic regression spline (CCRS). In CCRS, interest rather than with other species), with the start and end of the smoother are constrained 65.3% (n = 113) of the misidentified Small-billed to match in value; therefore, these are useful for Elaenia being Chilean and 70.3% (n = 19) of the fitting models with cyclic components such as sea- misidentified Chilean Elaenia being Small-billed sonal effects (Pedersen et al. 2019). With the (Table 1). As expected, misidentifications occurred model fits we determined the peaks of autumn/ mostly across the breeding range of Small-billed spring migration for Chilean Elaenia as the week Elaenia and during the migratory periods when with the highest fit for each autumn and spring they overlap (Fig. 3, Supporting Information Figs passage, and the arrival/departure weeks for Small- S3 and S4). Highlighting the identification prob- billed Elaenia by determining the 5th and 95th per- lems that exist when dealing with similar and centiles from the cumulative sum of the model overlapping species, the remaining 34.7% (n = 60) fits. of wrong reports in Small-billed Elaenia contained

Table 1. Classiﬁcation of photographic reports of Chilean Elaenia Elaenia chilensis and Small-billed Elaenia E. parvirostris submitted to eBird and EcoRegistros in Argentina, Chile and Uruguay. The percentage was calculated as the number of photos of each class over the total photos (excluding those classiﬁed as uncertain in the total).

Photos reported as:

Chilean Elaenia Small-billed Elaenia (Elaenia chilensis) (Elaenia parvirostris) Classiﬁed as:

1395 (98.1%) 1348 (88.6%) Correct 27 (1.9%) 173 (11.4%) Wrong – photo corresponded to: * 113 Chilean Elaenia Elaenia chilensis 19 * Small-billed Elaenia Elaenia parvirostris 3 17 Large Elaenia Elaenia spectabilis 0 3 Yellow-bellied Elaenia Elaenia ﬂavogaster 1 3 Highland Elaenia Elaenia obscura 0 1 Olivaceous Elaenia Elaenia mesoleuca 0 1 Slaty Elaenia Elaenia strepera 0 1 Other Elaenia Elaenia sp. 0 12 Southern Beardless-Tyrannulet Camptostoma obsoletum 0 1 Greenish Elaenia Myiopagis viridicata 2 0 Suiriri Flycatcher Suiriri suiriri 1 3 Tyrannulet sp. Serpophaga sp. 0 1 Plain Inezia Inezia inornata 0 4 Bran-colored Flycather Myiophobus fasciatus 1 9 Southern Scrub Flycatcher Sublegatus modestus 0 1 Plumbeous Tyrant Knipolegus cabanisi 0 2 Flycatcher sp. Myiarchus sp. 0 1 Grassland Sparrow Ammodramus humeralis 8 56 Uncertain

Asterisk(*): Does not apply. an assortment of similar looking and sympatric 7 and 17 was significantly lower than in the pre- species: five other Elaenia species, other flycatchers vious 11 weeks (weeks 48 and 6) for Small-billed (family Tyrannidae) and one record that is probably Elaenia (v2 = 131.26, df = 1, P < 0.05), but not a data-entry error and not a true misidentification for Chilean Elaenia (v2 = 1.77, df = 1, P = 0.1). (Grassland Sparrow, family Passerellidae; Table 1). Chilean Elaenia showed a relatively high accuracy Likewise, the remaining 29.7% (n = 8) misidentified throughout the year, which was always above Chilean Elaenia were confused with two Elaenia 90%. species and three other species of flycatchers The phenological function shifted when built (Table 1). with the raw and curated datasets and showed Weekly identification accuracy varied strongly weekly differences in the estimated phenological through time for Small-billed Elaenia but not for attributes for both species (Fig. 5). For Chilean Chilean Elaenia. Identification accuracy for Small- Elaenia, the curated dataset model exhibited a 1- billed Elaenia experienced a dramatic decrease week earlier peak of spring migration (week 41, during the autumn migration period (from week early October) across the study area (Fig. 5). This 7 to week 17, mid-February to late April), rang- model also showed a stronger intensity for the ing from 84.6% to a minimum of 28.6% at week peak autumn migration (week 12, mid- to late 17 (Fig. 4). During this time window, 70 of 241 March) than that of the raw dataset model, while photos (29%) reported as Small-billed corre- coinciding in the peak week (Fig. 3). For Small- sponded to Chilean Elaenia (mostly juveniles; see billed Elaenia, the curated dataset model showed a Discussion), which represents 62% of the 113 1-week later arrival (week 40, early October) and Chilean Elaenia reported as Small-billed Elaenia a 2-week earlier departure (week 11, mid-March; (Table 1). Identification accuracy between weeks Fig. 5).

Figure 3. Spatio-temporal distribution of correctly identiﬁed Chilean Elaenia Elaenia chilensis (green circles) and misidentiﬁed Chilean Elaenia reported as Small-billed Elaenia E. parvirostris (red triangles) in eBird and EcoRegistros for Argentina, Chile and Uruguay. (a) Geographical distribution of reports and (b) temporal distribution of reports as a function of longitude. Note that errors concentrate during the spring and autumn (fall) migratory periods when both species overlap in the breeding range of Small-billed Elaenia.

highlight both the ability of citizen science data- DISCUSSION bases to explore spatio-temporal patterns of migra- We assessed the identification accuracy of docu- tory birds and the need to perform critical mented citizen science data and tested the ability assessments of records when using citizen science of these data to describe phenological patterns of data to describe patterns in species that are hard two confusingly similar bird species that overlap to identify. when one (Chilean Elaenia) migrates over the Misidentifications occurred mostly when both breeding range of the other (Small-billed Elaenia). Elaenias overlapped in space and time, and We found that identification accuracy was high for although the errors were reciprocal, they occurred both Elaenias when they were allopatric (c. 90% mostly in one direction: Chilean Elaenia were far accuracy), but dramatically low for Small-billed more often identified as Small-billed than vice Elaenia (down to 28.6%) during the autumn versa (Table 1). We believe that this outcome migration period, when migrating individuals of could stem from a general lack of knowledge con- Chilean Elaenia were consistently reported as cerning the migration of Chilean Elaenia. As the Small-billed. This striking large-scale misidentifica- migration flyways of this species have only recently tion effect strongly biased the phenological esti- been described at relatively fine scales (Jimenez mates for both species, which differed noticeably et al. 2016, Bravo et al. 2017), most observers do when using raw (with identification errors) vs. not expect to find the migratory Chilean Elaenia curated (identification error-free) data. Our results on the breeding grounds of Small-billed Elaenia.

Figure 4. Weekly identiﬁcation accuracy of photographic reports of (a) Chilean Elaenia Elaenia chilensis and (b) Small-billed Elaenia E. parvirostris in eBird and EcoRegistros for Argentina, Chile and Uruguay. Numbers outside the circle indicate the week of the year (starting 1 January). The accuracy rate for each week is shown by colour and is measured as the percentage of correct photos over the total of identiﬁable photos (i.e. excluding those tagged as uncertain). Grey cells indicate null information given low sample size (n < 7). Arrows indicate either autumn (fall) or spring migratory periods. Note that the accuracy of Small-billed Elaenia drops notably between weeks 7 and 17 when Chilean Elaenia is on peak autumn migration across the study area (see Fig. 5).

Therefore, observers may simply resort to identifi- create false-positives and false-negatives; when this cation by default, in which birds are identified occurs at large scales it can lead to biased estima- based on what is expected at a given locality and tions in ecological analyses (Royle & Link 2006, date, not by critically assessing the identification. Miller et al. 2011, Guillera-Arroita et al. 2017). The ‘default identification’ could work reasonably Our phenological models built with the raw data- well in our study system at times when both spe- set contained a large number of false-positives for cies are allopatric or narrowly parapatric, but not Small-billed Elaenia and concomitant false- during spring and autumn migration, when they negatives for Chilean Elaenia. After removal of overlap. The rate of misidentifications of Chilean such false-positives in Small-billed Elaenia, the Elaenia as Small-billed was exacerbated during curated model estimated a 2-week earlier depar- autumn migration due to the passage of juveniles ture date. We expected this outcome, as most of of Chilean Elaenia. Juveniles lack the white crest the wrong reports of Small-billed Elaenia were (Pyle et al. 2015) and this feature is not properly concentrated from March to May, overestimating described in most field guides (but see Pearman & its presence towards the end of the migratory per- Areta 2020). We postulate that as a consequence iod. After converting false-negatives into true posi- of this general lack of knowledge about migration tives for Chilean Elaenia, the curated model and plumage, a striking unidirectional bias revealed that the intensity of autumn migratory occurred that involved not only citizen science passage in this species was underestimated with users but also database reviewers who were unable the raw data, and that spring peak passage was off- to detect and mitigate such bias. set by 1 week. Similar to our findings, other stud- The large number of wrong reports that were ies indicate that false-positives cause overestimates, mostly concentrated in a short window (autumn and false-negatives the opposite (Miller et al. migration) created spurious phenological patterns 2012, Cruickshank et al. 2019). for both Chilean Elaenia and Small-billed Elaenia We acknowledge that our phenological analysis when we estimated phenology with raw data. may still exhibit some temporal bias as a product Misidentifications between species simultaneously of variation in sampling effort, even after indirectly

Figure 5. Phenological GAMs of (a) Chilean Elaenia Elaenia chilensis and (b) Small-billed Elaenia E. parvirostris built with raw citizen science reports (red lines) and curated citizen science reports (green lines), across the overlapping area of both species in Argentina and Uruguay (see Fig. 3). Dotted lines indicate the estimated peak of spring and autumn (fall) migration for Chilean Elae- nia, and estimated arrival/departure dates for Small-billed Elaenia. The shaded area represents the 95% confidence interval of each model. Note that the y-axis (week of year) starts and ends at week 26 (late June). Curated models estimated a 1-week difference in the spring peak passage and a stronger autumn peak in Chilean Elaenia, and a 1-week later arrival and a 2-week earlier departure in Small-billed Elaenia. accounting for this bias by weighting the data by critical limitation of existing validation systems the ‘photographic’ effort of each platform. This is (Bonter & Cooper 2012). This is problematic because species-specific traits such as higher activ- because although some taxa seem to have a high ity during the breeding season or an increase in data accuracy (in terms of correct identifications), the abundance of migrant passerines may result in this is only true because false-negatives are not a larger quantity of photos reported, without nec- taken into account when measuring such accuracy. essarily implying more intense sampling by the In our study, this problem was very evident for contributors. Nevertheless, we believe that the Chilean Elaenia, a species that had a high identifi- output provided by the curated models is a robust cation accuracy throughout the year (almost approximation to the true phenology of both Elae- always above 90%), but a high rate of false- nias in the study area. negatives that ultimately biased its phenology. Citizen science validation systems are effective While effective detection of false-negatives is in flagging unusual records, but impractical for difficult, a practical approach would be to perform flagging errors when records lay within the thorough reviews of the documented reports of expected range of a species (Bonter & Cooper potential ‘confusion’ species (Swanson et al. 2016). 2012). This problem is exacerbated between If several of the focal species are being reported as similar-looking species (Gardiner et al. 2012), as other similar-looking taxa, this becomes a powerful was also demonstrated in our work. Although indicator that false-negatives can represent a seri- false-positives can be ‘effectively’ detected as long ous problem, probably requiring efforts to amelio- as records have some evidence attached (requiring, rate the effect of errors when analysing data. In of course, the ability of reviewers to detect such this context, it becomes imperative to question the errors), the lack of false-negative detection is a reliability of unvouchered observations. Any large-

scale mismatch between spatio-temporal patterns AUTHOR CONTRIBUTION uncovered by curated documented data vs. those using undocumented data demands careful scru- Fabricio Gorleri: Conceptualization (equal); Data tiny. For example, the lack of evidence (photo or curation (lead); Formal analysis (lead); Methodol- audio) for reports of a species during certain peri- ogy (lead); Validation (equal); Writing-original ods or across large geographical areas is a strong draft (lead); Writing-review & editing (equal). indicator that large-scale systematic errors are Juan Ignacio Areta: Conceptualization (equal); occurring, and this must be assessed before using Data curation (supporting); Formal analysis (sup- any dataset. porting); Methodology (supporting); Supervision Our work exemplifies the ability of citizen (lead); Writing-original draft (supporting); science databases to describe spatio-temporal pat- Writing-review & editing (lead). terns in birds. However, it also provides a warning about the need to undertake active post-processing Data availability statement of records before analysing data when the species The datasets used in this paper (Appendix S3) are of interest is hard to identify (see also Areta & available at Zenodo: http://doi.org/10.5281/zenod Juhant 2019). Large-scale analyses involving many o.4774417 taxa often do not allow for manual correction or testing of identification accuracy (but see Yu et al. 2014, Saoud et al. 2020). Therefore, we highlight REFERENCES the importance of citizen science initiatives to alert Areta, J.I. & Juhant, M.A. 2019. The Rufous-thighed Kite users and researchers to the most commonly Harpagus diodon is not an endemic breeder of the Atlantic misidentified species or species groups in their Forest: lessons to assess Wallacean shortfalls. Ibis 161:337– databases. While scientists are ultimately responsi- 345. ble for the data used for analysis, this warning on Areta, J.I., Mangini, G.G., Gandoy, F.A., Gomez, D., potential problems arising from species misidentifi- Depino, E.A., Jordan, E.A. & Gorleri, F. 2016. Ecology and behavior of Alder Flycatchers (Empidonax alnorum)on cations can help mitigate the misuse of raw data in their wintering grounds in Argentina. Wilson J. Ornithol. 128: phenological or other analyses, thereby increasing 830–845. the value of citizen science data. Aubry, K.B., Raley, C.M. & McKelvey, K.S. 2017. The importance of data quality for generating reliable distribution models for rare, elusive, and cryptic species. PLoS One 12:1–17. Barnes, R. 2017. dggridR: Discrete Global Grids for R. R package version 2.0.4. Available at: https://github.com/r-ba We thank Paul Sweet, Bentley Bird and Christopher rnes/dggridR/ (accessed 20 July 2020). Milensky who facilitated access to the American Belton, W. 1985. Birds of Rio Grande do Sul, Brazil. Part 2: Museum of Natural History and the Unites States Formicariidae through Corvidae. Bull. Am. Mus. Nat. Hist. National Museum where we improved our identification 180:1–242. skills by checking holdings; Juan Klavins for providing Bonney, R., Cooper, C.B., Dickinson, J., Kelling, S., further information on the identification of the species; Phillips, T., Kenneth, V., Shirk, J., Bonney, R., Cooper, Matıas Juhant, Emiliano Depino, Emilio Jordan, Ingrid C.B., Dickinson, J., Kelling, S. & Phillips, T. 2009. Citizen Holzmann, Juliana Benitez, Freddy Burgos and Juan Science: a developing tool for expanding science knowledge Amaya (all members of the ECOSON lab) for providing and scientific literacy. Bioscience 59: 977–984. writing and statistical feedback on the manuscript; Heli- Bonter, D.N. & Cooper, C.B. 2012. Data validation in citizen ana Guirado for giving us feedback on the figure designs; science: a case study from Project FeederWatch. Front. – Diego Gallegos for providing information on Ecol. Environ. 10: 305 307. identification-types that enriched the manuscript; Jorge Bravo, S.P., Cueto, V.R. & Gorosito, C.A. 2017. Migratory La Grotteria for facilitating access to all the observations timing, rate, routes and wintering areas of Chilean Elaenia submitted to EcoRegistros; Cullen Hanks for facilitating (Elaenia albiceps chilensis), a key seed disperser for Patagonian Forest regeneration. PLoS One 12:1–15. the summary of eBird records of passerines; Diego ~ fi Chattopadhyay, B., Garg, K.M., Gwee, C.Y., Edwards, S.V. Nunez for support during our eld trips; Pablo Bran- fl ı ı & Rheindt, F.E. 2017. Gene ow during glacial habitat shifts dol n and Ramiro Ram rez for sharing photos of Elaenias facilitates character displacement in a Neotropical flycatcher with us; and Mark Pearman and Paul Smith for their radiation. BMC Evol. Biol. 17: 210. fi comments on the identi cation section. Finally, we Costa, H., Foody, G.M., Jimenez, S. & Silva, L. 2015. thank all the people and institutions involved with citi- Impacts of species misidentification on species distribution zen science initiatives – their contributions allow us to modeling with presence-only data. ISPRS Int. J. Geo-Inform. understand Neotropical wildlife better. 4: 2496–2518.

Crall, A.W., Newman, G.J., Stohlgren, T.J., Holfelder, K.A., Miller, D., Nichols, J., McClintock, B., Campbell, E., Bailey, Graham, J. & Waller, D.M. 2011. Assessing citizen science L. & Weir, L. 2011. Improving occupancy estimation when data quality: an invasive species case study. Conserv. Lett. two types of observational error occur: non-detection and 4: 433–442. species misidentification. Ecology 92: 1422–1428. Cruickshank, S.S., Buhler,€ C. & Schmidt, B.R. 2019. Miller, D.A.W., Weir, L.A., McClintock, B.T., Grant, E.H.C., Quantifying data quality in a citizen science monitoring Bailey, L.L. & Simons, T.R. 2012. Experimental program: false negatives, false positives and occupancy investigation of false positive errors in auditory species trends. Conserv. Sci. Pract. 1:1–14. occurrence surveys. Ecol. Appl. 22: 1665–1674. Dickinson, J.L., Shirk, J.L., Bonter, D.N., Bonney, R.E., Crain, Minns, J. 2018. Identification of some Brazilian elaenias by R.L., Martin, J., Phillips, T. & Purcell, K. 2012. The current voice. Available at: https://www.xeno-canto.org/article/228 state of citizen science as a tool for ecological research and (accessed 10 July 2020). public engagement. Front. Ecol. Environ. 10:494–501. Molinari-Jobin, A., Kery, M., Marboutin, E., Molinari, P., Dickinson, J.L., Zuckerberg, B. & Bonter, D.N. 2010. Citizen Koren, I., Fuxjager,€ C., Breitenmoser-Wursten,€ C., Wol€ fl, science as an ecological research tool: challenges and S., Fasel, M., Kos, I., Wol€ fl, M. & Breitenmoser, U. 2012. benefits. Annu. Rev. Ecol. Evol. Syst. 41: 149–172. Monitoring in the presence of species misidentification: the Ensing, D.J., Moffat, C.E. & Pither, J. 2013. Taxonomic case of the Eurasian lynx in the Alps. Anim. Conserv. 15: identification errors generate misleading ecological niche 266–273. model predictions of an invasive hawkweed. Botany 91: Pearman, M. & Areta, J.I. 2020. Birds of Argentina and the 137–147. South-west Atlantic. Field Guide. London: Helm. Falk, S., Foster, G., Comont, R., Conroy, J., Bostock, H., Pedersen, E.J., Miller, D.L., Simpson, G.L. & Ross, N. Salisbury, A., Kilbey, D., Bennett, J. & Smith, B. 2019. 2019. Hierarchical generalized additive models in ecology: Evaluating the ability of citizen scientists to identify An introduction with mgcv. PeerJ 7: e6876. bumblebee (Bombus) species. PLoS One 14:1–21. Pyle, P., Engili Jr., A., & Kelt, D.A. 2015. Manual for ageing Gardiner, M.M., Allee, L.L., Brown, P.M.J., Losey, J.E., Roy, and sexing landbirds of Bosque Fray Jorge National Park H.E. & Smyth, R.R. 2012. Lessons from lady beetles: and North-central Chile, with notes on occurrence and accuracy of monitoring data from US and UK citizen science breeding seasonality. Special Publication of the Occasional programs. Front. Ecol. Environ. 10: 471–476. Papers of the Museum of Natural Science, Baton Rouge. Gill, F., Donsker, D. & Rasmussen, P. (eds). 2021. IOC Remsen Jr., J.V., Areta, J.I., Bonaccorso, E., Claramunt, World Bird List (v11.1). https://doi.org/10.14344/IOC.ML.11.1 S., Jaramillo, A., Lane, D.F., Pacheco, J.F., Robbins, Grolemund, G. & Wickham, H. 2011. Dates and times made M.B., Stiles, F.G. & Zimmer, K.J. 2021. A classification of easy with lubridate. J. Stat. Softw. 40:1–25. the bird species of South America. American Ornithological Guillera-Arroita, G., Lahoz-Monfort, J.J., van Rooyen, A.R., Society. Available at: http://www.museum.lsu.edu/~Remsen/ Weeks, A.R. & Tingley, R. 2017. Dealing with false positive SACCBaseline.htm (accessed 3 March 2021). and false negative errors about species occurrence at Rheindt, F.E., Christidis, L. & Norman, J.A. 2009. Genetic multiple levels. Methods Ecol. Evol. 38:42–49. introgression, incomplete lineage sorting and faulty Hull, J.M., Fish, A.M., Keane, J.J., Mori, S.R., Sacks, B.N. & taxonomy create multiple cases of polyphyly in a montane Hull, A.C. (2010) Estimation of species identification error: clade of tyrant-flycatchers (Elaenia, Tyrannidae). Zool. Scr. implications for raptor migration counts and trend estimation. 38: 143–153. J. Wildl. Manage 74: 1326–1334. Ridgely, R.S. & Tudor, G. 1994. The Birds of South America. Jimenez, J.E., Jahn, A.E., Rozzi, R. & Seavy, N.E. 2016. vol. 1. Austin: University of Texas Press. First documented migration of individual Chilean Elaenias Royle, J.A. & Link, W.A. 2006. Generalized site occupancy (Elaenia albiceps chilensis) in South America. Wilson J. models allowing for false positives and false negatives Ornithol. 128: 419–425. errors. Ecology 87: 835–841. Kelling, S., Johnston, A., Hochachka, W.M., Iliff, M., Fink, Sahr, K. 2011. Hexagonal discrete global GRID systems for D., Gerbracht, J., Lagoze, C., La Sorte, F.A., Moore, T., geospatial computing. Arch. Photogramm. Cartogr. Remote Wiggins, A., Wong, W.K., Wood, C. & Yu, J. 2015. Can Sens. 22: 363–376. observation skills of citizen scientists be estimated using Saoud, Z., Fontaine, C., Lo€ıs, G., Julliard, R. & species accumulation curves? PLoS One 10:1–20. Rakotoniaina, I. 2020. Miss-identification detection in citizen Lees, A.C. & Martin, R.W. 2014. Exposing hidden endemism science platform for biodiversity monitoring using machine in a Neotropical forest raptor using citizen science. Ibis 157: learning. Ecol. Inform. 60: 101176. 103–114. Schubert, S.C., Manica, L.T. & Guaraldo, A.D.C. 2019. Linden, A., Meller, K. & Knape, J. 2017. An empirical Revealing the potential of a huge citizen-science platform to comparison of models for the phenology of bird migration. J. study bird migration. Emu 119: 364–373. Avian Biol. 42: 255–265. Straube, F. 2013. Um incomodo^ consenso: estudo de caso McNeish, D. 2017. Small sample methods for multilevel sobre Elaenia. Atual. Ornitol. 172:37–48. modeling: a colloquial elucidation of REML and the Kenward- Sullivan, B.L., Aycrigg, J.L., Barry, J.H., Bonney, R.E., Roger correction. Multivariate Behav. Res. 52:661–670. Bruns, N., Cooper, C.B., Damoulas, T., Dhondt, A.A., Miller, D.A.W., Nichols, J.D., Gude, J.A., Rich, L.N., Dietterich, T., Farnsworth, A., Fink, D., Fitzpatrick, J.W., Podruzny, K.M., Hines, J.E. & Mitchell, M.S. 2013. Fredericks, T., Gerbracht, J., Gomes, C., Hochachka, Determining occurrence dynamics when false positives W.M., Iliff, M.J., Lagoze, C., La Sorte, F.A., Merrifield, M., occur: estimating the range dynamics of wolves from public Morris, W., Phillips, T.B., Reynolds, M., Rodewald, A.D., survey data. PLoS One 8: e65808. Rosenberg, K.V., Trautmann, N.M., Wiggins, A., Winkler,

D.W., Wong, W.K., Wood, C.L., Yu, J. & Kelling, S. 2014. Aires, Argentina, 12 December 2014. (D) Juvenile The eBird enterprise: an integrated approach to Small-billed Elaenia, Amed Hernandez, development and application of citizen science. Biol. ML199745351, El Soldado, Lavalleja, Uruguay, 11 Conserv. 169:31–40. Sullivan, B.L., Wood, C.L., Iliff, M.J., Bonney, R.E., Fink, D. January 2020. The background of the photos was & Kelling, S. 2009. eBird: a citizen-based bird observation deleted for practical purposes. For a more detailed network in the biological sciences. Biol. Conserv. 142: description of each species see Appendix S1 and 2282–2292. Fig. S2. Swanson, A., Kosmala, M., Lintott, C. & Packer, C. 2016. A Figure S2. Comparative photos of Elaenias held generalized approach for producing, quantifying, and validating citizen science data from wildlife images. Conserv. at the Macaulay Library (ML, www.macaulaylibra Biol. 30: 520–531. ry.org). Top (A–C): adults of Chilean Elaenia Elae- Tang, Q., Edwards, S.V. & Rheindt, F.E. 2018. Rapid nia chilensis, middle (D–F): juveniles of Chilean diversification and hybridisation have shaped the dynamic Elaenia, bottom: two adults (G,H) and one juve- history of the genus Elaenia. Mol. Phylogenet. Evol. 127: nile (I) of Small-billed Elaenia E. parvirostris. Pho- 522–533. Wood, S. 2017. Generalized additive models. An Introduction tos have been cropped for practical purposes. with R. , 2nd edn. New York: Chapman and Hall/CRC. Photo details (author, catalogue number, locality Yu, J., Hutchinson, R.A. & Wong, W.K. 2014. A latent and date): (A) Noah Strycker, ML51754871, Ush- variable model for discovering bird species commonly uaia, Tierra del Fuego, Argentina, 21 January fi misidenti ed by citizen scientists. In: Proceedings of the 2017; (B) Luis R Figueroa, ML204932961, Villar- National Conference on Artificial Intelligence, pp. 500–506. rica NP, Araucanıa, Chile, 25 December 2016; (C) Alec Earnshaw, ML111669271, Parque Received 27 November 2020; Revision 25 April 2021; Provincial Potrero de Yala, Jujuy, Argentina, 10 revision accepted 7 June 2021. October 2017; (D) Renato Machado de Sobral, Associate Editor: Zhijun Ma ML155014741, Santo Andre, S~ao Paulo, Brazil, 30 April 2019; (E) Patricio Saez, ML259082201, SUPPORTING INFORMATION Punilla, Cordoba, Argentina, 20 March 2020; (F) Thomas Kallmeyer, ML86493181, Ushuaia, Tierra Additional supporting information may be found del Fuego, Argentina, 4 February 2018; (G) Jorge online in the Supporting Information section at Claudio Schlemmer, ML188772911, Reserva Nat- the end of the article. ural Estricta Quebrada de Las Higueritas, San Appendix S1. Testing for differences in identifi- Luis, Argentina, 16 November 2019; (H) Jorge cation of accuracy between eBird and EcoRegis- Quiroga, ML196413051, Cerro de la Virgen, tros. Salta, Argentina, 1 January 2020; (I) Christopher Appendix S2. Identification of Chilean and Rex Prevett, ML202014181, Melo, Cerro Largo, Small-billed Elaenias. Uruguay, 9 January 2020. Table S1. Comparative table of identification Figure S3. Geographical distribution of photo- features for (A) adult Chilean Elaenia Elaenia graphic reports of Chilean Elaenia chilensis sub- chilensis, (B) juvenile Chilean Elaenia, (C) Small- mitted to eBird and EcoRegistros in Argentina, billed Elaenia E. parvirostris and (D) juvenile Chile and Uruguay. Green circles indicate those Small-billed Elaenia. Bold indicates key features. classified as correct, and red those classified as For a more detailed description of each species see wrong. The solid black line indicates the mini- Appendix S1 and Fig. S2. mum convex polygon encompassing the correct Figure S1. Cues for identification of Chilean reports of Small-billed Elaenia within which Elaenia Elaenia chilensis and Small-billed Elaenia E. phenological comparisons were made using the parvirostris. Photos were taken from the Macaulay raw and curated datasets (see Methods); note Library (ML, www.macaulaylibrary.org). (A) that most wrong reports lie within this Adult Chilean Elaenia, Pio Marshall, polygon. ML108211551, Villarrica, Araucanıa, Chile, 7 Figure S4. Geographical distribution of photo- February 2016. (B) Juvenile Chilean Elaenia, Hal graphic reports of Small-billed Elaenia Elaenia and Kirsten Snyder, ML204320101, Ushuaia, parvirostris submitted to eBird and EcoRegistros in Tierra del Fuego, Argentina, 4 February 2015. (C) Argentina, Chile and Uruguay. Green triangles Adult Small-billed Elaenia, Adrien Mauss, indicate those classified as correct, and red those ML33836461, San Clemente del Tuyu, Buenos classified as wrong. The solid black line indicates

the minimum convex polygon encompassing the the raw and curated datasets (see Methods); note correct reports of Small-billed Elaenia within that most wrong reports lie within this polygon. which phenological comparisons were made using Appendix S3. Raw data set.