Mobile Touchless Fingerprint Recognition: Implementation, Performance and Usability Aspects
Total Page:16
File Type:pdf, Size:1020Kb
Mobile Touchless Fingerprint Recognition: Implementation, Performance and Usability Aspects a,c, b a c a Jannis Priesnitz ∗, Rolf Huesmann , Christian Rathgeb , Nicolas Buchmann , Christoph Busch ada/sec { Biometrics and Internet Security Research Group, Hochschule Darmstadt, Sch¨offerstraße 8b, 64295 Darmstadt, Germany bUCS { User-Centered Security Research Group, Hochschule Darmstadt, Sch¨offerstraße 8b, 64295 Darmstadt, Germany cFreie Universit¨atBerlin, Takustraße 9, 14195 Berlin, Germany Abstract This work presents an automated touchless fingerprint recognition system for smartphones. We provide a comprehensive description of the entire recognition pipeline and discuss important requirements for a fully automated capturing sys- tem. Also, our implementation is made publicly available for research purposes. During a database acquisition, a total number of 1,360 touchless and touch-based samples of 29 subjects are captured in two different environmental situations. Experiments on the acquired database show a comparable performance of our touchless scheme and the touch-based baseline scheme under constrained environmental influences. A comparative usability study on both capturing device types indicates that the majority of subjects prefer the touchless capturing method. Based on our experimental results we analyze the impact of the current COVID-19 pandemic on fingerprint recognition systems. Finally, implementation aspects of touchless fingerprint recognition are summarized. Keywords: Biometrics, Fingerprint Recognition, Touchless Fingerprint, Usability, Biometric Performance 1. Introduction with a powerful processing have already been implemented in a practical way [16]. However, to the best of the au- Fingerprints are one of the most important biometric thors' knowledge no approach to mobile touchless recog- characteristic due to their known uniqueness and persis- nition scheme based on of-the-shelf components has been tence properties. Fingerprint recognition systems are not documented so far. only used worldwide by law enforcement and forensic agen- In this work, we propose a mobile touchless fingerprint cies, they are also deployed in the mobile devices as well recognition scheme based for smartphones. We provide a as in nation-wide applications. The vast majority of finger- fully automated capturing scheme with integrated quality print capturing schemes requires contact between the finger assessment in form of an Android app. All components of and the capturing device's surface. These systems suffer the processing pipeline will be made publicly available for from distinct problems, e.g. low contrast caused by dirt research purposes. The rest of the recognition pipeline is or humidity on the capturing device plate or latent finger- composed of open source algorithms for feature extraction arXiv:2103.03038v1 [cs.CV] 4 Mar 2021 prints of previous users (ghost fingerprints). Especially in and comparison. multi-user applications hygienic concerns lower the accept- To benchmark our proposed recognition pipeline we ac- ability of touch-based fingerprint systems and hence, limit quired a database under real life conditions. A number of their deployment. In a comprehensive study, Okereafor et 29 subjects have been captured by two touchless capturing al. [1] analyze the risk of an infection by touch-based fin- devices in different environmental situations. Moreover, gerprint recognition schemes and the hygienic concerns of touch-based samples have been acquired as baseline and their users. The authors conclude that touch-based fin- to measure the interoperability between both schemes. A gerprint recognition carries a high risk of an infection if a usability study, which has been conducted after the captur- previous user has contaminated the capturing device sur- ing, reviews the users' experiences with both capturing de- face, e.g. with the SARS-CoV-2 virus. vice types. This paper details on every implementation step To tackle these shortcomings of touch-based schemes, of the processing pipeline and summarizes our database touchless fingerprint recognition systems have been re- acquisition setup in detail. Further, we evaluate the bio- searched for more than a decade. Touchless capturing metric performance of our acquired database and discusses schemes operate without any contact between the finger the outcomes of our usability study. Based on our exper- and the capturing device. Several contributions to the imental results we elaborate on the impact of the current research area paved the way for a practical implementa- COVID-19 pandemic on fingerprint recognition in terms of tion of touchless capturing schemes. Specialized stationary biometric performance and user acceptance. Further we capturing devices based on multi-camera setups combined summarize implementation aspects which we consider as beneficial for mobile touchless fingerprint recognition. The rest of the paper is structured as follows: Section 2 ∗Corresponding author Email address: jannis.priesnitz@h-da.de (Jannis Priesnitz) gives an overview on touchless end-to-end schemes pro- URL: dasec.h-da.de (Jannis Priesnitz) posed in the scientific literature. In Section 3, the proposed Preprint submitted to Elsevier March 5, 2021 Table 1: Overview on selected recognition workflows with biometric performance. (Device type: P = Prototypical hardware, S = Smartphone, W = Webcam) Device Mobile / Amount Automatic Free finger On-device Biometric Authors Year type Stationary of fingers capturing positioning processing Performance Hiew et al. [2] 2007 P S 1 N N N EER: 5.63% Piuri et al. [3] 2008 W S 1 N N N EER: 0.04% Wang et al. [4] 2009 P S 1 N N N EER: 0.0% 1 Kumar and Zhou [5] 2011 W S 1 N N N EER: 0.32% Derawi et al. [6] 2011 S S 1 N N N EER: 4.5% Noh et al. [7] 2011 P S 5 Y N N EER: 6.5% Stein et al. [8] 2013 S M 1 Y Y Y EER: 1.2% Raghavendra et al. [9] 2014 P S 1 N Y N EER: 6.63% Tiwari et al. [10] 2015 S M 1 N Y N EER: 3.33% Sankaran et al. [11] 2015 S M 1 Y N N EER: 3.56% FAR = 0.01% @ Carney et al. [12] 2017 S M 4 Y N Y FRR = 1% TAR = 98.55% @ Deb et al. [13] 2018 S M 1 Y Y Y FAR = 1.0% FRR 1.2% @ Weissenfeld et al. [14] 2018 P M 4 Y Y Y FAR 0.01% Birajadar et al. [15] 2019 S M 1 Y N N EER: 1.18% Our method 2021 S M 4 Y Y Y EER: 5.36% Figure 1: Overview on the most relevant steps of our proposed method. processing pipeline is presented. In Section 4, we describe app and manually transferred to a remote device where the our experimental setup and provide details about the cap- processing is performed [10, 11]. The improvement of the tured database and the usability study. The results of our camera and processing power in current smartphones has experiments are reported in Section 5. The influence of the made it possible to capture multiple fingers in in a single COVID-19 pandemic on fingerprint recognition is discussed capture attempt and process them on the device. Stein et in Section 6. Section 7 discusses implementation aspects. al. [8] showed that it is feasible for the automated captur- Finally, Section 8 concludes. ing of a single finger image using a smartphone. Carney et al. [12] presented the first four-finger capturing scheme. 2. Related Work Weissenfeld et al. [14] proposed a system with a free po- sitioning of four fingers in a mobile prototypical hardware In this section, we present an overview on touchless fin- setup. gerprint end-to-end solutions along with reported biomet- In summary, it can be observed that the evolution of ric performance. Table 1 summarizes most relevant related touchless fingerprint technologies has moved towards out- works and their properties. For a comprehensive overview of-the-box device and a more convenient and practically on the topic of touchless fingerprint recognition the reader relevant recognition process. Table 1 also indicates that is referred to [17]. both an accurate recognition and a convenient capturing The research on touchless fingerprint recognition has are hard to achieve. It should be further noted that the evolved from bulky single finger devices to more conve- level of constraints during the capturing has a major influ- nient multi-finger capturing schemes. First end-to-end ap- ence on the recognition performance. proaches with prototypical hardware setups were presented by Hiew et al. [2] and Wang et al. [4]. Both works em- ployed huge capturing devices for one single finger acquisi- 3. Mobile Touchless Recognition Pipeline tion within a hole-like guidance. For remote user authentication, Piuri et al. [3] and Ku- An unconstrained and automated touchless fingerprint mar et al. [5] investigated the use of webcams as finger- recognition system usually requires a more elaborated pro- print capturing device. Both schemes showed a very low cessing compared to touch-based schemes. Figure 1 gives EER in experimental results. However, the database cap- an overview on the key processing steps of the proposed turing process was not reported precisely. In addition, the recognition pipeline. Our method features an on-device usability and user acceptance of such an approach should capturing, pre-processing, and quality assessment whereas be further investigated. the biometric identification workflow is implemented on a More recent works use smartphones for touchless finger- back-end system. This section describes each component print capturing. Here, a finger image is taken by a photo of the recognition pipeline in detail. The proposed method combines three implementation 1The authors only report the EER on a score level fusion of 8 aspects seen as beneficial for an efficient and convenient sequentially acquired fingerprints. recognition: 2 Figure 2: Overview of the segmentation of connected components from a continuous stream of input images. Figure 3: Overview of the coarse rotation correction, separation of fingerprint images from each other, fine rotation correction, fingertip cropping and normalization of the fingerprint size.