e-Counterfeit: a mobile-server platform for document counterfeit detection Albert Berenguel∗y, Oriol Ramos Terrades∗, Josep Llados´ ∗ and Cristina Canero˜ y ∗Computer Vision Center, Universitat Autonoma of Barcelona, Barcelona, Spain Email: faberenguel,oriolrt,[email protected] yICAR Vision Systems S.A, Barcelona, Spain Email: [email protected] Abstract—This paper presents a novel application to detect issues. Thanks to this, a large part of counterfeits just follows counterfeit identity documents forged by a scan-printing op- the procedure of scan a real document, alter the data and then eration. Texture analysis approaches are proposed to extract print the document with a common commercial printer. We validation features from security background that is usually printed in documents as IDs or banknotes. The main contribution refer to this methodology of counterfeit document generation of this work is the end-to-end mobile-server architecture, which as scan-printing procedure. Following the scan-printing pro- provides a service for non-expert users and therefore can be used cedure, we expect that the background print design will loose in several scenarios. The system also provides a crowdsourcing detail hence it will be possible to classify it as counterfeit. We mode so labeled images can be gathered, generating databases point our efforts to detect this kind of low quality counterfeit for incremental training of the algorithms. documents. It is important to note that most of the print I. INTRODUCTION design methodology that is possible to find in the security documents, were first created and used to detect counterfeit Making a fake passport is easy, making a good fake passport money banknotes. The system proposed in this work has been is very, very hard. Probably there are few criminal organiza- designed for ID documents, however, as mentioned before, tions in the world which can produce a counterfeit visa or it is generic enough so it can be used for other documents passport good enough to fool professional passport control. with security background texture as banknotes. The main Counterfeit detection has traditionally been a task for law functionality is the analysis of the document authenticity from enforcement agencies. EUROPOL and INTERPOL central of- a single image using a mobile phone camera within a non- fices are combating document and banknote counterfeiting [1], controlled environment. This process is used by services or [2]. They have destined millions of euros in funds to provides products that require a genuine identification of the client, technical databases, forensic support, training and operational such as renting a car, opening a bank account, applying for assistance to its member countries. Despite all these efforts a loan, checking-in in an hotel, etc. Once authenticity of the counterfeit detection remains an open issue. There are many document is validated, the purpose of the platform where the different strategies used to fake an identity document (ID) service is integrated can take place (reading and storing the like the alteration of a real passport, impersonation of the personal information of the document holder). legitimate owner or printing false information on a stolen blank The main contribution is a mobile-server framework to de- real paper, to cite some. An important security feature that tect counterfeit documents. This end-to-end system provides: serves against counterfeiting and manipulation of documents a low cost solution fit to be used with common smartphones, is the background/security printing. The security printing can flexibility for further updates and robust validation methods. be classified into two categories: The mobile-server framework also intends to address the arXiv:1708.06126v1 [cs.CV] 21 Aug 2017 1) Security printing techniques and printing processes used: lack of tools to generate datasets acquired from smartphone microprinting as well as security inks (e.g. optical vari- devices. We have published the framework code to help future able ink, ultraviolet or infrared ink). This security features researchers1. can only be checked with specialized equipment like The remainder of the paper is organized as follows. In ultraviolet lamp or a magnified glass, therefore are not section II, we review the existing frameworks for banknote possible to check if our target of image acquisition is the counterfeit detection. Section III describes in detail each one general smartphones cameras. of the components of the end-to-end framework proposed and 2) Print designs and security elements: guilloches or fine- IV explains in detail the Counterfeit module. In section V the line patterns, rainbow coloring, etc., discernible to the dataset is presented. At section VI we explain the current set- naked eye. We focus in this category of elements. up and the discussion of the results. Finally, in section VII The counterfeiter requires high technical specialized print- we draw some conclusions and outline rewarding avenues for ing equipment to reproduce this background printing tech- future work. niques. Having this equipment is not feasible for the majority of the counterfeiters due economic and restricted availability 1https://github.com/gitabcworld/e-Counterfeit. Fig. 1. Server-client counterfeit framework. A document is acquired with a mobile application by the user. The image is sent to the server which stores this image into a database through the web-server. Afterwards a queue manager manages several workers, which are reading simultaneously from the database to process the images with the Counterfeit module. Finally the results are sent back to the user. The WSGI module is used to connect the HTTP messages with Python Flask module that centralizes all the operations. II. RELATED WORK A. Server Framework Several private companies are specialized in ID document counterfeit using smartphone applications [3]–[6]. However We have built a server which communicates with a client there is few academic research published about this topic. mobile application, see Fig. 1. First a user acquires a document The authors in [7] developed an operational pilot project to photo which sends through JSON REST API to our server. be used in mobile context focused on document background Through post/get messages we establish a handshake protocol and photo area printing. On the other hand, the counterfeit to send the validation image and receive the response informa- analysis field applied to banknotes has been a topic of interest tion. A web server is set up on top of the operating system to for researchers the last years, being this field applicable to ID send the HTTP requests, but it could also serve static files like counterfeiting. Other researchers have focused in a banknote images, JavaScript files, HTML pages, etc. We use NGINX as validation method which uses radio frequency identification our web server for its resource efficiency and responsiveness (RFID) and an NFC-enabled smartphone [8]. Although this under load. Once the web server has the data, it process the method is computationally less expensive than other methods JSON message with Flask, which is a micro-framework for not all the documents and banknotes have the RFID chips. For Python focused at web application code. Since a web server this reason, we focus on texture background like the authors cannot communicate directly with Flask, we implement a Web in [9], where they use small patches of the banknote focused Server Gateway Interface (WSGI) to act as a proxy between in the Intaglio regions instead. A different approach is treat the server and Python/Flask. As a summary we have HTTP the problem of banknote counterfeit detection as a dictionary requests routed from the web server to Flask, which Flask problem to cover all possible background texture variations handles appropriately. The responses are then sent right back and encode the representation as sparse coding features [10]. to the web server and, lastly, back to the end user application. They validate the complete document from a single image. The The Flask module will unpack the JSON message containing Intaglio zones have been also validated by the authors in [11], the image to process. The image along with other data will [12], using Discrete Fourier Transformation (DFT) and models be stored in a MySQL database. An ID is assigned to this to describing the edge transition respectively. They acquire image and is given back to the client. The image will not be their dataset with a minimum of 1100 dpi within a controlled processed immediately, it will be managed by Celery, a queue environment. manager which will be constantly querying through multiple workers the database for new images to process. Meanwhile III. SYSTEM ARCHITECTURE AND COMPONENTS the mobile clients will establish a handshake protocol inter- We follow the document acquisition approach given by mittently asking for the result to the server using the given the authors in [10] and we build a service-oriented archi- ID. Finally each one of the queue manager workers will access tecture (SOA) end-to-end counterfeit validation framework. the counterfeit module which will process the image and store The proposed SOA application is composed by integration the result in the database. The counterfeit module integrates of distributed, separately-maintained and deployed software into this framework languages as Matlab, OpenCV, VlFeat, components. The end-to-end system developed is scalable and C++, Python and Torch, with the purpose to evaluate different provide fast responses. The image acquisition procedure is existing algorithms written in their respective languages by easy and friendly for the user. Finally, this framework provides their authors. Identity documents are also preprocessed to find a way to store and manage the new data that is being sent to blurring and highlights although these algorithms are out of the server for further improvements. the scope of this paper. (a) Adjust photo (b) Crop Fig. 3. The GrabCut algorithm and dewarping done in the mobile application.