Identification of Saimaa Ringed Seal Individuals Using Transfer Learning

Identification of Saimaa ringed seal individuals using transfer learning Ekaterina Nepovinnykh12, Tuomas Eerola1, Heikki Kälviäinen2, and Gleb Radchenko2 1 Machine Vision and Pattern Recognition Laboratory, Department of Computational and Process Engineering, School of Engineering Science, Lappeenranta University of Technology, Lappeenranta, Finland, [email protected] 2 School of Electrical Engineering and Computer Science, South Ural State University, Chelyabinsk, Russian Federation Abstract. The conservation efforts of the endangered Saimaa ringed seal depend on the ability to reliably estimate the population size and to track individuals. Wildlife photo-identification has been successfully utilized in monitoring for various species. Traditionally, the collected images have been analyzed by biologists. However, due to the rapid increase in the amount of image data, there is a demand for automated methods. Ringed seals have pelage patterns that are unique to each seal en- abling the individual identification. In this work, two methods of Saimaa ringed seal identification based on transfer learning are proposed. The first method involves retraining of an existing convolutional neural network (CNN). The second method uses the CNN trained for image classification to extract features which are then used to train a Support Vector Machine (SVM) classifier. Both approaches show over 90% identification accuracy on challenging image data, the SVM based method being slightly better. Keywords: animal biometrics, Saimaa ringed seals, convolutional neural networks, transfer learning, identification, image segmentation 1 Introduction The Saimaa ringed seal (Pusa hispida saimensis) is a subspecies of ringed seal (Pusa hispida) living in Lake Saimaa in Finland (Fig. 1). At present, around 360 seals inhabit the lake, and on the average 60 to 85 pups are born annually. This small and fragmented population is threatened by various anthropogenic factors, especially by-catch and climate change [13]. Therefore the long-term and accurate assessment of the population is needed for conservation purposes. Successful conservation requires constant population monitoring which is not easy to do without invasive methods. Traditional population monitoring methods include tagging that requires catching the animal and may cause stress to Fig. 1. Saimaa ringed seal. it, as well as may change its behavior or increase mortality. This makes non- invasive methods preferable for population monitoring. Wildlife Photo Identifi- cation (WPI) is a technology that allows to recognize individuals and to track the movement of animal populations over time. It is based on acquiring images of animals and further identifying individuals. Recently, camera trapping has been launched as a monitoring tool also for the Saimaa ringed seal [5, 12]. The Saimaa ringed seals have a distinctive fur pattern that is never repeated in different individuals and does not significantly change over the course of seal's life [12]. This makes photo identification based on the fur pattern suitable for non-invasive monitoring. In this work, an automatic photo identification of the Saimaa ringed seals is considered. The proposed method first segments the seal from the background and then uses the fur pattern to identify the individual. The work continues studies presented in [20] and [7] where the first steps towards automatic individual identification of the Saimaa ringed seal were taken. In this paper, new methods for both the segmentation phase and the identification phase are proposed by utilizing convolutional neural networks (CNNs) and transfer learning. 2 Related work A computational approach to the wildlife photo identification is an emergent field that aims to apply formal methods to automate the process of animal biometric identification. There are many advantages over manual identification: traditional methods are time-consuming, highly dependent on the skills of a person who performs identification, and prone to various errors such as observer errors and biases [15]. Moreover, human observers often ignore classification uncertainty, and as such misclassification is often underestimated [9]. Computer methods avoid this problem by utilizing probabilistic methods and often report classification certainty along with other possible classification results. The main advantage of utilizing the animal biometrics system is that it allows researchers to rapidly collect and to robustly analyze the extensive amount of data which ultimately improves research about the seals and their monitoring. Several approaches for automatic image-based animal identification can be found in the literature. Methods have been developed, for example, for polar bears [3], newts [11], giraffes [10], salamanders [6], and snakes [1], All of these methods use image processing and pattern recognition techniques to identify individuals. Most of the studies limit the individual identification to a certain animal species or species groups. All the above methods were developed for one species only and as such are not generalizable to the Saimaa ringed seals. In [20], the first steps towards the automatic individual identification of the Saimaa ringed seals were taken. The paper proposes a segmentation method for the Saimaa ringed seals using unsupervised segmentation and texture based superpixel classification. Furthermore, a simple texture based approach for the ringed seal identification was evaluated. In [7], the segmentation method was further developed to decrease its computation time without sacrificing the performance. Moreover, a set of post-processing operations for segmented images was proposed to make the seals easier to identify. Two existing species independent individual identification methods were evaluated to demonstrate the importance of the segmentation and post-processing operations. However, the identification performance of neither of the methods is good enough for most practical applications. There have been also research efforts towards creating a unified approach ap- plicable for identification purposes for several animal species. For example, in [8], the HotSpotter method to identify individual animals in a labeled database was presented. This algorithm is not species specific and has been applied to Grevy's and plain zebras, giraffes, leopards, and lionfish. HotSpotter uses viewpoint in- variant descriptors and a scoring mechanism that emphasizes the most distinc- tiveness keypoints and descriptors. In [19], a species recognition algorithm based on sparse coding spatial pyramid matching (ScSPM) was proposed. It was shown that the proposed object recognition techniques can be successfully used to identify animals on sequences of images captured using camera traps in nature. One of the problems with the species independent individual identification methods is that they do not provide an automatic method to detect the animals in images. Therefore, either a manual detection or development of a detection method for the studied animal is needed. Furthermore, typically higher identification performance can be obtained by tuning the identification method for one species only. 3 Proposed method In this work two Saimaa ringed seal identification methods based on transfer learning are proposed. The goal of the both methods is, given the image of a Saimaa ringed seal, to output the best suitable individual identifier for the specimen. The both proposed identification algorithms consist of two steps. In the first step, the image is segmented. The segmentation result is an image of a seal without the background or overlapping objects. This is important since most of the image material is obtained using static camera traps. Therefore, the same seal is often captured with the same background increasing the risk that a supervised Fig. 2. General seal identification algorithm. identification algorithm learns to \identify" the background instead of the actual seal if the full image or the bounding box around the seal is used. This may further lead to a system that is not able to identify the seal in a new environment. The second step is the identification using transfer learning. The first proposed method for the identification is a CNN-based method. It involves retraining a classification CNN by using image extraction layer from another, pre- existing convolutional neural network. After the initial experiments, it was con- cluded that training a CNN from the ground up is too computationally intensive given the constraint. Therefore, it was decided to use a pretrained general pur- pose CNN from [14] as the source of feature extraction layers. The second method for the identification is a Support Vector Machine (SVM) based method. For this method transfer learning is performed by using the above pretrained CNN for the feature extraction and SVM for the classification. The identification process is visualized in Fig. 2. 3.1 Segmentation Automatic segmentation of animals is often difficult due to the camouflage colors of animals, i.e., the coloration and patterns are similar to the visual background of the animal. Segmentation results, however, can have a significant impact on identification performance. Segmentation helps to reduce the overfitting by re- moving the irrelevant background from an image, allowing a standardized object rotation on different images, reducing the dataset bias by only presenting the objects of interest to the training algorithm, and allowing improved color-correction by zeroing out all background colors and only

Load more