DOCSLIB.ORG

Towards Real-Time Object Recognition and Pose Estimation in Point Clouds

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Towards Real-Time Object Recognition and Pose Estimation in Point Clouds Towards Real-time Object Recognition and Pose Estimation in Point Clouds Marlon Marcon1 a, Olga Regina Pereira Bellon2 b and Luciano Silva2 c 1Dapartment of Software Engineering, Federal University of Technology - Parana,´ Dois Vizinhos, Brazil 2Department of Computer Science, Federal University of Parana,´ Curitiba, Brazil Keywords: Transfer Learning, 3D Computer Vision, Feature-based Registration, ICP Dense Registration, RGB-D Images. Abstract: Object recognition and 6DoF pose estimation are quite challenging tasks in computer vision applications. De- spite efficiency in such tasks, standard methods deliver far from real-time processing rates. This paper presents a novel pipeline to estimate a fine 6DoF pose of objects, applied to realistic scenarios in real-time. We split our proposal into three main parts. Firstly, a Color feature classification leverages the use of pre-trained CNN color features trained on the ImageNet for object detection. A Feature-based registration module conducts a coarse pose estimation, and finally, a Fine-adjustment step performs an ICP-based dense registration. Our proposal achieves, in the best case, an accuracy performance of almost 83% on the RGB-D Scenes dataset. Regarding processing time, the object detection task is done at a frame processing rate up to 90 FPS, and the pose estimation at almost 14 FPS in a full execution strategy. We discuss that due to the proposal’s modularity, we could let the full execution occurs only when necessary and perform a scheduled execution that unlocks real-time processing, even for multitask situations. 1 INTRODUCTION Template matching approaches rely on RANSAC- based feature matching algorithms, following the Object recognition and 6D pose estimation represent pipeline proposed by (Aldoma et al., 2012b). a central role in a broad spectrum of computer vision RANSAC algorithm has proven to be one of the most applications, such as object grasping and manipula- versatile and robust. Unfortunately, for large or dense tion, bin picking tasks, and industrial assemblies ver- point clouds, its runtime becomes a significant limita- ification (Vock et al., 2019). Successful object recog- tion in several of the example applications mentioned nition, highly reliable pose estimation, and near real- above (Vock et al., 2019). When we seek a 6Dof esti- time operation are essential capabilities and current mation pose, the real-time is a more challenging task challenges for robot perception systems. (Marcon et al., 2019). In an extensive benchmark of A methodology usually employed to estimate full cloud object detection and pose estimation, (Ho- rigid transformations between scenes and objects is dan et al., 2018) reported runtime of a second per test centered on a feature-based template matching ap- target on average. proach. Assuming we have a known item or a part Deep learning strategies for object recognition and of an object, this technique involves searching all the classification problems have been extensively studied occurrences in a larger and usually cluttered scene for RGB images. As the demand for good quality (Vock et al., 2019). However, due to natural occlu- labeled data increases, large datasets are becoming sions, such occurrences may be represented only by available, serving as a significant benchmark of meth- a partial view of an object. The template is often an- ods (deep or not) and as training data for real appli- other point cloud, and the main challenge of the tem- cations. ImageNet (Deng et al., 2009) is, undoubt- plate matching approach is to maintain the runtime edly, the most studied dataset and the de-facto stan- feasibility and preserve the robustness. dard on such recognition tasks. This dataset presents more than 20,000 categories, but a subset with 1,000 a https://orcid.org/0000-0001-9748-2824 categories, known as ImageNet Large Scale Visual b https://orcid.org/0000-0003-2683-9704 Recognition Challenge (ILSVRC), is mostly used. c https://orcid.org/0000-0001-6341-1323 164 Marcon, M., Bellon, O. and Silva, L. Towards Real-time Object Recognition and Pose Estimation in Point Clouds. DOI: 10.5220/0010265601640174 In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 5: VISAPP, pages 164-174 ISBN: 978-989-758-488-6 Copyright c 2021 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved Towards Real-time Object Recognition and Pose Estimation in Point Clouds Training a model on ImageNet is quite a challeng- bled the depth of AlexNet, but exploring tiny filters ing task in terms of computational resources and time (3 × 3), and became the runner-up on the ILSVRC, consumption. Fortunately, transferring its models of- one step back the GoogLeNet (Szegedy et al., 2015), fer efficient solutions in different contexts, acting as with 22 layers. GoogLeNet relies on the Inception a blackbox feature extractor. Studies like (Agrawal architecture (Szegedy et al., 2016). Another type of et al., 2014) explore and corroborate this high ca- ConvNets, called ResNets (He et al., 2016), uses the pacity of transferring such models to different con- concept of residual blocks that use skip-connection texts and applications. Regarding the use of pre- blocks that learn residual functions regarding the in- trained CNN features, some approaches handle the put. Many architectures have been proposed based object recognition on the Washington RGB-D Object on these findings, such as ResNet with 50, 101, dataset, e.g., (Zia et al., 2017) with the VGG architec- and 152 (He et al., 2016). Also, based on devel- ture and (Caglayan et al., 2020) evaluate several pop- opments regarding the residual blocks, (Xie et al., ular deep networks, such as AlexNet, VGG, ResNet, 2017) developed the ResNeXt architecture. The ba- and DenseNet. sis upon ResNeXt blocks resides on parallel ResNet- This paper introduces a novel pipeline to deal with like blocks, which have the output summed before the point cloud pose estimation in uncontrolled environ- residual calculation. Some architectures propose the ments and cluttered scenes. Our proposed pipeline use of Deep Learning features on resource-limited de- recognizes the object using color feature descriptors, vices, such as smartphones and embedded systems. crops the selected bounding-box reducing the scenes’ The most prominent architecture is the MobileNet searching surface, and finally estimates the object’s (Sandler et al., 2018). Another family of leading net- pose in a traditional local feature-based approach. De- works is the EfficientNet (Tan and Le, 2019). Relying spite adopting well-known techniques in the 2D/3D on the use of these lighter architectures, EfficientNet computer vision field, our proposal’s novelty centers proposes very deep architectures without compromise on the smooth integration between 2D and 3D meth- resource efficiency. ods to provide a solution efficient in terms of accuracy and time. 2.2 Pose Estimation As presented in (Aldoma et al., 2012b), a compre- 2 BACKGROUND hensive registration process usually consists of two steps: coarse and fine registrations. We can produce Recognition systems work with objects, which are a coarse registration transformation by performing a digital representations of tangible real-world items manual alignment, motion tracking or, the most com- that exist physically in a scene. Such systems are mon, by using the local feature matching. Local- unavoidably machine-learning-based approaches that feature-matching-based algorithms automatically ob- use features to locate and identify objects in a scene tain corresponding points from two or multiple point- reliably. Together with the recognition, another task clouds, coarsely registering by minimizing the dis- is to estimate the location and orientation of the de- tance between them. These methods have been exten- tected items. In a 3D world, we estimate six degrees sively studied and have confirmed to be compliant and of freedom (6DoF), which refers to the geometrical computer efficient (Guo et al., 2016). After coarsely transformation representing a rigid body’s movement registering the point clouds, a fine-registration algo- in a 3D space, i.e., the combination of translation and rithm is applied to refine the initial coarse registra- rotation. tion iteratively. Examples of fine-registration algo- rithms include the ICP algorithm that perform point- to-point alignment (Besl and McKay, 1992), or point- 2.1 Color Feature Extraction to-plane (Chen and Medioni, 1992). These algorithms are suitable for matching between point-clouds of As a mark on the deep learning history, (Krizhevsky isolated scenes (3D registration) or between a scene et al., 2012) presented the first Deep Convolutional and a model (3D object recognition). This proposal Architecture employed on the ILSVRC, an 8-layer ar- adopted two approaches to generate the initial align- chitecture dubbed AlexNet. This network was the first ment: a traditional feature-based RANSAC and the to prove that deep learning could beat hand-crafted Fast Global Registration (FGR) (Zhou et al., 2016). methods when trained on a large scale dataset. Af- ter that, ConvNets became more accurate, deeper, and bigger in terms of parameters. (Simonyan and Zis- serman, 2015) propose VGG, a network that dou- 165 VISAPP 2021 - 16th International Conference on Computer Vision Theory and Applications 3 PROPOSED APPROACH step. The database is composed of information con- cerning each item, as well as the extracted features of In this section, we explain in detail our proposed ap- them. We choose a local-descriptors-based approach proach. Our proposed pipeline starts from an RGB to estimate the object’s pose. For each instance of an image and its corresponding point cloud, generated object, we store several partial views of it. Between from RGB and depth images. These inputs are sub- these views, our method will select the most likely to mitted to our three-stage architecture: color feature the correspondent object on the scene. classification, feature-based registration, and fine ad- Based on the predicted objects’ classes, we can justment.
Load more

Recommended publications
  • Template Matching Using Fast Normalized Cross Correlation
    Template Matching using Fast Normalized Cross Correlation Kai Briechle and Uwe D Haneb eck Institute of Automatic Control Engineering Technische Universitat Munc hen Munc hen Germany ABSTRACT In this pap er we present an algorithm for fast calculation of the normalized cross correlation NCC and its applica tion to the problem of template matching Given a template t whose p osition is to b e determined in an image f the basic idea of the algorithm is to represent the template for which the normalized cross correlation is calculated as a sum of rectangular basis functions Then the correlation is calculated for each basis function instead of the whole template Theresultofthe correlation of the template t and the image f is obtained as the weighted sum of the correlation functions of the basis functions Dep ending on the approximation the algorithm can by far outp erform Fouriertransform based implementations of the normalized cross correlation algorithm and it is esp ecially suited to problems where many dierent templates are to b e found in the same image f Keywords Normalized cross correlation image pro cessing template matching basis functions INTRODUCTION A basic problem that often o ccurs image pro cessing is to determine the p osition of a given pattern in an image or part of an image the socalled region of interest This problem is closely related to the determination of a received digital signal in signal pro cessing using eg a matched lter Two basic cases can b e dierentiated The p osition of the pattern is unknown estimate
    [Show full text]
  • Human Face and Facial Parts Detection Using Template Matching Technique
    International Journal of Engineering and Advanced Technology (IJEAT) ISSN: 2249 – 8958, Volume-9 Issue-4, April 2020 Human Face and Facial Parts Detection using Template Matching Technique Payal Bose, Samir Kumar Bandyopadhyay Abstract: Face recognition has become relevant in recent years But still, it is one of the most important tasks. Because face because of its potential applications. The aim of this paper is to detection is very much important for security sections in find out the relevant techniques which give not only better different places, crowd analysis and many more. There are accuracy also the efficient speed. There are several techniques several authors describe several techniques for face available for face detection which give much better accuracy but detection. In 2017, the authors, Shrinath Oza, Abdullah the execution speed is not efficient. In this paper, a normalized Khan, Nilabh Swapnil, Aayush Sharma, Jayashree cross-correlation template matching technique is used to solve [1] this problem. According to the proposed algorithm, first different Chaudhari, implemented an algorithm , the Restricted facial parts are detected likes mouth, eyes, and nose. If any of Boltzmann machine with the rules of deep learning and deep the two facial parts are found successfully then the face can be belief network for face recognition. According to this detected. For matching the templates with the target image, the algorithm, it stores the face images as input in a database template rotates at a certain angle interval. and a neural network used to ensure the efficiency and improvement of the recognition system trained the neural Keywords: Face detection, Face Recognition, Template network greedy layer-wise.
    [Show full text]
  • RESEARCH INVENTY: International Journal of Engineering and Science
    International Refereed Journal of Engineering and Science (IRJES) ISSN (Online) 2319-183X, (Print) 2319-1821 Volume 5, Issue 1 (January 2016), PP.36-40 Introduction of Pattern Recognition with various Approaches 1Tiwary, 2Shiva ramakrishna,3B Sandeep 1,2,3,4( Sreedattha Institute Of Engineering And Science) (Department Of Electronics & Communication Engineering ) ABSTRACT : The objective of this paper is To Familiarize with the fundamental concepts of pattern recognition (PR) & discusses various approaches for same. This paper deals issues: definition of PR (Pattern recognition), PR as System, approaches for PR. Keywords: About five key words in alphabetical order, separated by comma. I. INTRODUCTION In machine learning, PR( pattern recognition) is the assignment of a label to a given input value. An example of pattern recognition is classification, which attempts to assign each input value to one of a given set of classes. Pattern recognition generally aim to provide a reasonable answer for all possible inputs and to perform "most likely" matching of the inputs, taking into account their statistical variation. This is opposed to pattern matching, which look for exact matches in the input with pre-existing patterns. Our goal is to discuss and compare pattern recognition as the best possible Way. II. DEFINITION Before we define PR, we see what is mean by Pattern is a term with pair comprising an observation & a meaning.So PR as per “It is a process of taking raw data & performing action based on the “category” of the pattern.” Or Assignments of inputs to classes. Classification of PR is based on the following two: 1) Supervised classification: Here the input pattern is identified as a member of a pre- defined class.
    [Show full text]
  • OATM: Occlusion Aware Template Matching by Consensus Set Maximization
    OATM: Occlusion Aware Template Matching by Consensus Set Maximization Simon Korman1,3 Mark Milam2 Stefano Soatto1,3 1Weizmann Institute of Science 2Northrop Grumman 3UCLA Vision Lab Abstract We present a novel approach to template matching that is efficient, can handle partial occlusions, and comes with provable performance guarantees. A key component of the method is a reduction that transforms the problem of searching a nearest neighbor among high-dimensional N Figure 1. Instances of the occlusion experiment (Sec. 4.2) A tem- vectors, to searching neighbors among two sets of order plate (overlaid in green) that is 60% occluded by random blocks is √N vectors, which can be found efficiently using range searched for in an image. OATM shows the best results in dealing search techniques. This allows for a quadratic improve- with significant deformation and occlusion (use zoom for detail). ment in search complexity, and makes the method scalable in handling large search spaces. The second contribution increasing the area increases the probability that its corre- is a hashing scheme based on consensus set maximiza- spondent in the target image will be occluded, which causes tion, which allows us to handle occlusions. The resulting the correspondence to fail, unless occlusion phenomena are scheme can be seen as a randomized hypothesize-and-test explicitly taken into account. algorithm, which is equipped with guarantees regarding the In this work we model occlusions explicitly as part of a number of iterations required for obtaining an optimal so- robust template matching process where the co-visible re- lution with high probability. The predicted matching rates gion is assumed to undergo affine deformations of the do- are validated empirically and the algorithm shows a sig- main and range, up to additive noise.
    [Show full text]
  • Matchability Prediction for Full-Search Template Matching Algorithms
    Matchability Prediction for Full-Search Template Matching Algorithms Adrian Penate-Sanchez1, Lorenzo Porzi2,3, and Francesc Moreno-Noguer1 1Institut de Robotica` i Informatica` Industrial, UPC-CSIC, Barcelona, Spain 2Fondazione Bruno Kessler, Trento, Italy 3University of Perugia, Italy Abstract template matching algorithms that perform a full search on all possible transformations of the template guarantee to While recent approaches have shown that it is possible find the global maximum of the distance function, yielding to do template matching by exhaustively scanning the pa- more accurate results at the price of increasing the compu- rameter space, the resulting algorithms are still quite de- tational cost. manding. In this paper we alleviate the computational load Several approaches have focused on improving specific of these algorithms by proposing an efficient approach for characteristics of the template matching framework, such as predicting the matchability of a template, before it is actu- its speed [22, 24], its robustness to partial occlusions [4] or ally performed. This avoids large amounts of unnecessary ambiguities [26] and its ability to select reliable templates computations. We learn the matchability of templates by for rapid visual tracking [2]. Yet, most of these improve- using dense convolutional neural network descriptors that ments have been designed for the algorithms that do not per- do not require ad-hoc criteria to characterize a template. form a full search. In this work we will bring these improve- By using deep learning descriptions of patches we are able ments to algorithms that perform full search, and in particu- to predict matchability over the whole image quite reli- lar we will consider FAst-Match [15], a recent method that ably.
    [Show full text]
  • Arxiv:2007.15817V3 [Cs.CV] 7 May 2021
    Robust Template Matching via Hierarchical Convolutional Features from a Shape Biased CNN Bo Gao1[0000−0003−3930−1815] and Michael W. Spratling1[0000−0001−9531−2813] Department of Informatics, King's College London, London, UK fbo.gao,[email protected] Abstract. Finding a template in a search image is an important task underlying many computer vision applications. Recent approaches perform template matching in a deep feature-space, produced by a convolutional neural network (CNN), which is found to provide more tolerance to changes in appearance. In this article we investigate if enhancing the CNN's encoding of shape information can produce more distinguishable features that improve the performance of template matching. This investigation results in a new template matching method that produces state-of-the-art results on a standard benchmark. To confirm these results we also create a new benchmark and show that the proposed method also outperforms existing techniques on this new dataset. Our code and dataset is available at: https://github.com/iminfine/Deep-DIM. Keywords: Template match · Convolutional neural networks · VGG19. 1 Introduction Template matching is a technique to find a rectangular region of an image that contains a certain object or image feature. It is widely used in many computer vision applications such as object tracking [1, 2], object detection [3, 4] and 3D reconstruction [5, 6]. A similarity map is typically used to quantify how well a template matches each location in an image. Traditional template matching methods calculate the similarity using a range of metrics such as the normalised cross-correlation (NCC), the sum of squared differences (SSD) or the zero-mean normalised cross correlation (ZNCC) applied to pixel intensity or color values.
    [Show full text]
  • Orientation Detection Using a CNN Designed by Transfer Learning of Alexnet
    Proceedings of the 8th IIAE International Conference on Industrial Application Engineering 2020 Orientation Detection Using a CNN Designed by Transfer Learning of AlexNet Fusaomi Nagataa,*, Kohei Mikia, Yudai Imahashia, Kento Nakashimaa Kenta Tokunoa, Akimasa Otsukaa, Keigo Watanabeb and Maki K. Habibc aDepartment of Mechanical Engineering, Faculty of Engineering, Sanyo-Onoda City University Sanyo-Onoda 756-0884, Japan bGraduate School of Natural Science and Technology, Okayama University Okayama 700-8530, Japan cMechanical Engineering Department, School of Sciences and Engineering, The American University in Cairo AUC Avenue, P.O.Box 74, New Cairo 11835, Egypt *Corresponding Author: [email protected] Abstract classification in spite of only two layers. Nagi et al. designed max-pooling convolutional neural networks (MPCNN) for Artificial neural network (ANN) which has four or vision-based hand gesture recognition(1). The MPCNN could more layers structure is called deep NN (DNN) and is classify six kinds of gestures with 96% accuracy and allow recognized as a promising machine learning technique. mobile robots to perform real-time gesture recognition. Convolutional neural network (CNN) has the most used and Weimer et al. also designed a deep CNN architectures for powerful structure for image recognition. It is also known automated feature extraction in industrial inspection that support vector machine (SVM) has a superior ability for process(2). The CNN automatically generates features from binary classification in spite of only two layers. We have massive amount of training image data and demonstrates developed a CNN&SVM design and training tool for defect excellent defect detection results with low false alarm rates.
    [Show full text]
  • Template Matching Approach for Face Recognition System
    International Journal of Signal Processing Systems Vol. 1, No. 2 December 2013 Template Matching Approach for Face Recognition System Sadhna Sharma National Taiwan Ocean University, Taiwan Email: [email protected] Abstract—Object detection or face recognition is one of the Identification (one-to-many matching): Given an most interesting application in the image processing and it is image of an unknown individual, determining that a classical problem in computer vision, having application to person’s identity by comparing (possibly after surveillance, robotics, multimedia processing. Developing a encoding) that image with a database of (possibly generic object detection system is still an open problem, but there have been important successes over the past several encoded) images of known individuals. years from visual pattern. Among the most influential In general, based on the face representation face system is the face recognition system. Face recognition has recognition techniques can be divided into Appearance- become a popular area of research in computer vision and based technique (uses holistic texture features and is one of the most successful applications of image analysis and applied to either whole face or specific regions in a face understanding. Because of the nature of the problem, not image) and Feature-based technique (uses geometric only computer science researchers are interested in it, but facial features mouth, eyes, brows, cheeks etc and neuroscientists and psychologists also. It is the general geometric relationships between them). opinion that advances in computer vision research will provide useful insights to neuroscientists and psychologists A. Appearance Based into how human brain works, and vice versa.
    [Show full text]
  • Object Detection Using Template and HOG Feature Matching
    (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 11, No. 7, 2020 Object Detection using Template and HOG Feature Matching 1 Marjia Sultana Partha Chakraborty3, Mahmuda Khatun4 Department of Computer Science and Engineering Department of Computer Science & Engineering Begum Rokeya University Comilla University, Cumilla - 3506, Bangladesh Rangpur, Rangpur-5404 Bangladesh Md. Rakib Hasan5 2 Department of Information and Communication Technology Tasniya Ahmed Comilla University, Cumilla - 3506, Bangladesh Institute of Information Technology Noakhali Science and Technology University 6 Noakhali – 3814 Mohammad Shorif Uddin Bangladesh Department of Computer Science and Engineering Jahangirnagar University, Savar, Dhaka-1342, Bangladesh Abstract—In the present era, the applications of computer analysis [2]. For any vision-based image processing vision is increasing day by day. Computer vision is related to the application, object detection is the most integrated part [3]. An automatic recognition, exploration and extraction of the efficient object detection technique enables us to determine necessary information from a particular image or a group of the presence of our desired object from a random scene, image sets. This paper addresses the method to detect the desired regardless of object’s scaling and rotation, changes in camera object from an image. Usually, a template of the desired object is orientation, and changes in the types of illumination. Template used in detection through a matching technique named matching is one of the approaches of great interest in current Template Matching. But it works well when the template image times which has become a revolution in computer vision. is cropped from the original one, which is not always invariant Another widely used approach of object detection is HOG due to various transformations in the test images.
    [Show full text]
  • Deep Learning-Based Template Matching Spike Classification For
    applied sciences Article Deep Learning-Based Template Matching Spike Classification for Extracellular Recordings 1, 1, 1 1 2,3,4,5 In Yong Park y, Junsik Eom y, Hanbyol Jang , Sewon Kim , Sanggeon Park , Yeowool Huh 2,3 and Dosik Hwang 1,* 1 School of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, Korea; [email protected] (I.Y.P.); [email protected] (J.E.); [email protected] (H.J.); [email protected] (S.K.) 2 Department of Medical Science, College of Medicine, Catholic Kwandong University, Gangneung 25601, Korea; [email protected] (S.P.); [email protected] (Y.H.) 3 Translational Brain Research Center, Catholic Kwandong University, International St. Mary’s Hospital, Incheon 22711, Korea 4 Department of Neuroscience, University of Science & Technology, Daejeon 34113, Korea 5 Center for Neuroscience, Korea Institute of Science and Technology, Seoul 02792, Korea * Correspondence: [email protected]; Tel.: +82-2-2123-5771 Equally contributing authors. y Received: 12 November 2019; Accepted: 30 December 2019; Published: 31 December 2019 Abstract: We propose a deep learning-based spike sorting method for extracellular recordings. For analysis of extracellular single unit activity, the process of detecting and classifying action potentials called “spike sorting” has become essential. This is achieved through distinguishing the morphological differences of the spikes from each neuron, which arises from the differences of the surrounding environment and characteristics of the neurons. However, cases of high structural similarity and noise make the task difficult. And for manual spike sorting, it requires professional knowledge along with extensive time cost and suffers from human bias.
    [Show full text]
  • An Overview of Various Template Matching Methodologies in Image Processing
    International Journal of Computer Applications (0975 – 8887) Volume 153 – No 10, November 2016 An Overview of Various Template Matching Methodologies in Image Processing Paridhi Swaroop Neelam Sharma, PhD M.Tech(Computer Science) Asst. Professor Banasthali Vidhyapeeth Banasthali Vidhyapeeth Rajasthan Rajasthan ABSTRACT MSE(Mean Squared Error) The recognition and classification of objects in images is a ∑∑(Temporary(u, v) − Target(u, v) = emerging trend within the discipline of computer vision Number of pixels community. A general image processing problem is to decide the vicinity of an object by means of a template once the scale wherever, Temporary (u, v) and Target (u, v) are the strength and rotation of the true target are unknown. Template is values of Template image and the input image respectively. primarily a sub-part of an object that‟s to be matched amongst entirely different objects. The techniques of template matching are flexible and generally easy to make use of, that makes it one amongst the most famous strategies of object localization. Template matching is carried out in versatile fields like image processing,signal processing, video compression and pattern recognition.This paper gives brief description of applications and methods where template matching methods were used. General Terms Template matching,computer vision,image processing,object recognition. Fig 1: Example of template matching. An example which describes template matching. This figure Keywords describes template based eye detection.The template is Template Matching, image processing, object recognition. correlated with absolutely distinctive regions of a face image.The regions of a face that offers maximum correlation 1. INTRODUCTION with template refers to the eye region.This approach is Template Matching may be a high-level machine vision effective in both the instances like closed eyes along with method which determines the components of a figure which open eyes and conveys numerous simulation outcome.
    [Show full text]
  • Matching RGB and Infrared Remote Sensing Images with Densely-Connected Convolutional Neural Networks
    remote sensing Article Matching RGB and Infrared Remote Sensing Images with Densely-Connected Convolutional Neural Networks Ruojin Zhu 1, Dawen Yu 1, Shunping Ji 1,* and Meng Lu 2 1 School of Remote Sensing and Information Engineering, Wuhan University, 129 Luoyu Road, Wuhan 430079, China; [email protected] (R.Z.); [email protected] (D.Y.) 2 Department of Physical Geography, Faculty of Geoscience, Utrecht University, Princetonlaan 8, 3584 CB Utrecht, The Netherlands; [email protected] * Correspondence: [email protected] Received: 17 October 2019; Accepted: 26 November 2019; Published: 29 November 2019 Abstract: We develop a deep learning-based matching method between an RGB (red, green and blue) and an infrared image that were captured from satellite sensors. The method includes a convolutional neural network (CNN) that compares the RGB and infrared image pair and a template searching strategy that searches the correspondent point within a search window in the target image to a given point in the reference image. A densely-connected CNN is developed to extract common features from different spectral bands. The network consists of a series of densely-connected convolutions to make full use of low-level features and an augmented cross entropy loss to avoid model overfitting. The network takes band-wise concatenated RGB and infrared images as the input and outputs a similarity score of the RGB and infrared image pair. For a given reference point, the similarity scores within the search window are calculated pixel-by-pixel, and the pixel with the highest score becomes the matching candidate. Experiments on a satellite RGB and infrared image dataset demonstrated that our method obtained more than 75% improvement on matching rate (the ratio of the successfully matched points to all the reference points) over conventional methods such as SURF, RIFT, and PSO-SIFT, and more than 10% improvement compared to other most recent CNN-based structures.
    [Show full text]