DOCSLIB.ORG

Artificial Neural Networks for Pattern Recognition

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

S~dhan& Vol. 19, Part 2, April 1994, pp. 189-238. © Printed in India. Artificial neural networks for pattern recognition B YEGNANARAYANA Department of Computer Science and Engineering, Indian Institute of Technology, Madras 600036, India E-mail: yegna @ iitm. ernet, in MS received 12 April 1993; revised 8 September 1993 Abstract. This tutorial article deals with the basics of artificial neural networks (ANN) and their applications in pattern recognition. ANN can be viewed as computing models inspired by the structure and function of the biological neural network. These models are expected to deal with problem solving in a manner different from conventional computing. A distinction is made between pattern and data to emphasize the need for developing pattern processing systems to address pattern recognition tasks. After introducing the basic principles of ANN, some fundamental networks are examined in detail for their ability to solve simple pattern recognition tasks. These fundamental networks together with the principles of ANN will lead to the development of architectures for complex pattern recognition tasks. A few popular architectures are described to illustrate the need to develop an architecture specific to a given pattern recognition problem. Finally several issues that still need to be addressed to solve practical problems using ANN approach are discussed. Keywords. Artificial neural network; pattern recognition; biological neural network. 1. Introduction Human problem solving is basically a pattern processing problem and not a data processing problem. In any pattern recognition task humans perceive patterns in the input data and manipulate the pattern directly. In this paper we discuss attempts at developing computing models based on artificial neural networks (ANN) to deal with various pattern recognition situations in real life. Search for new models of computing is motivated by our quest to solve natural (intelligent) tasks by exploiting the developments in computer technology (Marcus & van Dam 1991). The developments in artificial intelligence (AI) appeared promising till a few years ago. But when the AI methods were applied to natural tasks such as in speech, vision and natural language processing, the inadequacies of the methods 189 190 B Yegnanarayana showed up. Like conventional algorithms, AI methods also need a clear specification of the problem, and mapping of the problem into a form suitable for the methods to be applicable. For example, in order to apply heuristic search methods, one needs to map the problem as a search problem. Likewise, to solve a problem using a rule-based approach, it is necessary to explicitly state the rules governing it. Scientists are hoping that computing models inspired by biological neural networks may provide new directions to solving problems arising in natural tasks. In particular, it is hoped that neural networks would extract the relevant features from input data and perform the pattern recognition task by learning from examples, without explicitly stating the rules for performing the task. The objective of this tutorial paper is to present an overview of the current approaches based on artificial neural networks for solving various pattern recognition tasks. From the overview it will be evident that the current approaches still fall far short of our expectations, and there is scope for evolving better models inspired by the principles of operation of our biological neural network. This paper is organized as follows: In § 2 we discuss the nature of patterns and pattern recognition tasks that we encounter in our daily life. We make a distinction between pattern and data, and also between understanding and recognition. In this section we also briefly discuss methods available for dealing with pattern recognition tasks, and make a case for new models of computing based on artificial neural networks. The basics of artificial neural networks are presented in § 3, including a brief discussion on the operation of a biological neural network, models of neuron and the neuronal activation and synaptic dynamics. Section 4 deals with the subject matter of this paper, namely, the use of principles of artificial neural networks to solve simple pattern recognition tasks. This section introduces the fundamental neural networks that laid the foundation for developing new architectures. In § 5 we discuss a few architectures for complex pattern recognition tasks. In the final section we discuss several issues that need to be addressed to develop artificial neutral network models for solving practical problems. 2. Patterns and pattern recognition tasks 2.1 Notion of intelligence The current usage of the terms like AI systems, intelligent systems, knowledge-based systems, expert systems etc., are intended to show the urge to build machines that can demonstrate intelligence similar to human beings in performing some simple tasks. In these tasks we look at the performance of a machine and compare it with the performance of a person. We attribute intelligence to the machine if the perfor- mances match. But the way the tasks are performed by a machine and by a human being are basically different; the machine performing the task in a step-by-step sequential manner dictated by an algorithm, modified by some known heuristics. The algorithm and the heuristics have to be derived for a given task. Once derived, they generally remain fixed. Typically, implementation of these tasks requires large number of operations (arithmetic and logical) and also a large amount of memory. The trends in computing clearly demonstrate the machine's ability to handle a large number of operations (Marcus & van Dam 1991). Artificial neural networks for pattern recognition 191 2.2 Patterns and data However, the mere ability of a machine to perform a large amount of symbolic processing and logical inferencing (as is being done in AI) does not result in intelligent behaviour. The main difference between human and machine intelligence comes from the fact that humans perceive everything as a pattern, whereas for a machine all are data. Even in routine data consisting of integer numbers (like telephone numbers, bank account numbers, car numbers), humans tend to see a pattern. Recalling the data is also normally from a stored pattern. If there is no pattern, then it is very difficult for a human being to remember and reproduce the data later. Thus storage and recall operations in humans and machines are performed by different mechanisms. The pattern nature in storage and recall automatically gives robustness and fault tolerance for a human system. Moreover, typically far fewer patterns than the estimated capacity of human memory systems are stored. Functionally also humans and machines differ in the sense that humans understand patterns, whereas machines can be said to recognize patterns in data. In other words, humans can get the whole object in the data even though there may be no clear identification of subpatterns in the data. For example, consider the name of a person written in a handwritten cursive script. Even though individual patterns for each letter may not be evident, the name is understood due to the visual hints provided in the written script. Likewise, speech is understood even though the patterns corresponding to individual sounds may be distorted sometimes to unrecognizable extents. Another major characteristic of a human being is the ability to continuously learn from examples, which is not well understood at all in order to implement it in an algorithmic fashion in a machine. Human beings are capable of making mental patterns in their biological neural network from input data given in the form of numbers, text, pictures, sounds etc., using their sensory mechanisms of vision, sound, touch, smell and taste. These mental patterns are formed even when the data are noisy, or deformed due to variations such as translation, rotation and scaling. The patterns are also formed from a temporal sequence of data as in the case of speech and motion pictures. Humans have the ability to recall the stored patterns even when the input information is noisy or partial (incomplete) or mixed with information pertaining to other patterns. 2.3 Pattern recognition tasks The inherent differences in information handling by human beings and machines in the form of patterns and data, and in their functions in the form of understanding and recognition have led us to identify and discuss several pattern recognition tasks which human beings are able to perform very naturally and effortlessly, whereas we have no simple algorithms to implement these tasks on a machine. The identification of these tasks below is somewhat influenced by the organization of the artificial neural network models which we will be describing later in this paper. 2.3a Pattern association: Pattern association problem involves storing a set of patterns or a set of input-output pattern pairs in such a way that when test data are presented, the pattern or pattern pair corresponding to the data is recalled. This is purely a memory function to be performed for patterns and pattern pairs. Typically, 192 B Yegnanarayana it is desirable to recall the correct pattern even though the test data are noisy or incomplete. The problem of storage and recall of patterns is called autoassociation. Since this is a content addressable memory function, the system should display accretive behaviour, i.e., should recall the stored pattern closest to the given input. It is also necessary
Recommended publications
  • Self-Organization Principles of Intracellular Pattern Formation

    Self-Organization Principles of Intracellular Pattern Formation

    Self-organization principles of intracellular pattern formation J. Halatek. F. Brauns, and E. Frey∗ Arnold{Sommerfeld{Center for Theoretical Physics and Center for NanoScience, Department of Physics, Ludwig-Maximilians-Universit¨atM¨unchen, Theresienstraße 37, D-80333 M¨unchen,Germany (Dated: February 21, 2018) Abstract Dynamic patterning of specific proteins is essential for the spatiotemporal regulation of many important intracellular processes in procaryotes, eucaryotes, and multicellular organisms. The emergence of patterns generated by interactions of diffusing proteins is a paradigmatic example for self-organization. In this article we review quantitative models for intracellular Min protein patterns in E. coli, Cdc42 polarization in S. cerevisiae, and the bipolar PAR protein patterns found in C. elegans. By analyzing the molecular processes driving these systems we derive a theoretical perspective on general principles underlying self-organized pattern formation. We argue that intracellular pattern formation is not captured by concepts such as \activators", \inhibitors", or \substrate-depletion". Instead, intracellular pattern formation is based on the redistribution of proteins by cytosolic diffusion, and the cycling of proteins between distinct conformational states. Therefore, mass-conserving reaction-diffusion equations provide the most appropriate framework to study intracellular pattern formation. We conclude that directed transport, e.g. cytosolic diffusion along an actively maintained cytosolic gradient, is the key process underlying pattern formation. Thus the basic principle of self-organization is the establishment and maintenance of directed transport by intracellular protein dynamics. Keywords: self-organization; pattern formation; intracellular patterns; reaction-diffusion; cell polarity; arXiv:1802.07169v1 [physics.bio-ph] 20 Feb 2018 NTPases 1 INTRODUCTION In biological systems self-organization refers to the emergence of spatial and temporal structure.
  • PATTERN RECOGNITION LETTERS an Official Publication of the International Association for Pattern Recognition

    PATTERN RECOGNITION LETTERS an Official Publication of the International Association for Pattern Recognition

    PATTERN RECOGNITION LETTERS An official publication of the International Association for Pattern Recognition AUTHOR INFORMATION PACK TABLE OF CONTENTS XXX . • Description p.1 • Audience p.2 • Impact Factor p.2 • Abstracting and Indexing p.2 • Editorial Board p.2 • Guide for Authors p.5 ISSN: 0167-8655 DESCRIPTION . Pattern Recognition Letters aims at rapid publication of concise articles of a broad interest in pattern recognition. Subject areas include all the current fields of interest represented by the Technical Committees of the International Association of Pattern Recognition, and other developing themes involving learning and recognition. Examples include: • Statistical, structural, syntactic pattern recognition; • Neural networks, machine learning, data mining; • Discrete geometry, algebraic, graph-based techniques for pattern recognition; • Signal analysis, image coding and processing, shape and texture analysis; • Computer vision, robotics, remote sensing; • Document processing, text and graphics recognition, digital libraries; • Speech recognition, music analysis, multimedia systems; • Natural language analysis, information retrieval; • Biometrics, biomedical pattern analysis and information systems; • Special hardware architectures, software packages for pattern recognition. We invite contributions as research reports or commentaries. Research reports should be concise summaries of methodological inventions and findings, with strong potential of wide applications. Alternatively, they can describe significant and novel applications
  • Guiding Self-Organized Pattern Formation in Cell Polarity Establishment

    Guiding Self-Organized Pattern Formation in Cell Polarity Establishment

    ARTICLES https://doi.org/10.1038/s41567-018-0358-7 Guiding self-organized pattern formation in cell polarity establishment Peter Gross1,2,3,8, K. Vijay Kumar" "3,4,8, Nathan W. Goehring" "5,6, Justin S. Bois" "7, Carsten Hoege2, Frank Jülicher3 and Stephan W. Grill" "1,2,3* Spontaneous pattern formation in Turing systems relies on feedback. But patterns in cells and tissues seldom form spontane- ously—instead they are controlled by regulatory biochemical interactions that provide molecular guiding cues. The relationship between these guiding cues and feedback in controlled biological pattern formation remains unclear. Here, we explore this relationship during cell-polarity establishment in the one-cell-stage Caenorhabditis elegans embryo. We quantify the strength of two feedback systems that operate during polarity establishment: feedback between polarity proteins and the actomyosin cortex, and mutual antagonism among polarity proteins. We characterize how these feedback systems are modulated by guid- ing cues from the centrosome, an organelle regulating cell cycle progression. By coupling a mass-conserved Turing-like reac- tion–diffusion system for polarity proteins to an active-gel description of the actomyosin cortex, we reveal a transition point beyond which feedback ensures self-organized polarization, even when cues are removed. Notably, the system switches from a guide-dominated to a feedback-dominated regime well beyond this transition point, which ensures robustness. Together, these results reveal a general criterion for controlling biological pattern-forming systems: feedback remains subcritical to avoid unstable behaviour, and molecular guiding cues drive the system beyond a transition point for pattern formation. ell polarity establishment in Caenorhabditis elegans zygotes phase where myosin appears to reach a steady state (Supplementary is a prototypical example for cellular pattern formation that Fig.
  • A Wavenet for Speech Denoising

    A Wavenet for Speech Denoising

    A Wavenet for Speech Denoising Dario Rethage∗ Jordi Pons∗ Xavier Serra [email protected] [email protected] [email protected] Music Technology Group Music Technology Group Music Technology Group Universitat Pompeu Fabra Universitat Pompeu Fabra Universitat Pompeu Fabra Abstract Currently, most speech processing techniques use magnitude spectrograms as front- end and are therefore by default discarding part of the signal: the phase. In order to overcome this limitation, we propose an end-to-end learning method for speech denoising based on Wavenet. The proposed model adaptation retains Wavenet’s powerful acoustic modeling capabilities, while significantly reducing its time- complexity by eliminating its autoregressive nature. Specifically, the model makes use of non-causal, dilated convolutions and predicts target fields instead of a single target sample. The discriminative adaptation of the model we propose, learns in a supervised fashion via minimizing a regression loss. These modifications make the model highly parallelizable during both training and inference. Both computational and perceptual evaluations indicate that the proposed method is preferred to Wiener filtering, a common method based on processing the magnitude spectrogram. 1 Introduction Over the last several decades, machine learning has produced solutions to complex problems that were previously unattainable with signal processing techniques [4, 12, 38]. Speech recognition is one such problem where machine learning has had a very strong impact. However, until today it has been standard practice not to work directly in the time-domain, but rather to explicitly use time-frequency representations as input [1, 34, 35] – for reducing the high-dimensionality of raw waveforms. Similarly, most techniques for speech denoising use magnitude spectrograms as front-end [13, 17, 21, 34, 36].
  • An Empirical Comparison of Pattern Recognition, Neural Nets, and Machine Learning Classification Methods

    An Empirical Comparison of Pattern Recognition, Neural Nets, and Machine Learning Classification Methods

    An Empirical Comparison of Pattern Recognition, Neural Nets, and Machine Learning Classification Methods Sholom M. Weiss and Ioannis Kapouleas Department of Computer Science, Rutgers University, New Brunswick, NJ 08903 Abstract distinct production rule. Unlike decision trees, a disjunctive set of production rules need not be mutually exclusive. Classification methods from statistical pattern Among the principal techniques of induction of production recognition, neural nets, and machine learning were rules from empirical data are Michalski s AQ15 applied to four real-world data sets. Each of these data system [Michalski, Mozetic, Hong, and Lavrac, 1986] and sets has been previously analyzed and reported in the recent work by Quinlan in deriving production rules from a statistical, medical, or machine learning literature. The collection of decision trees [Quinlan, 1987b]. data sets are characterized by statisucal uncertainty; Neural net research activity has increased dramatically there is no completely accurate solution to these following many reports of successful classification using problems. Training and testing or resampling hidden units and the back propagation learning technique. techniques are used to estimate the true error rates of This is an area where researchers are still exploring learning the classification methods. Detailed attention is given methods, and the theory is evolving. to the analysis of performance of the neural nets using Researchers from all these fields have all explored similar back propagation. For these problems, which have problems using different classification models. relatively few hypotheses and features, the machine Occasionally, some classical discriminant methods arecited learning procedures for rule induction or tree induction 1 in comparison with results for a newer technique such as a clearly performed best.
  • Unsupervised Speech Representation Learning Using Wavenet Autoencoders Jan Chorowski, Ron J

    Unsupervised Speech Representation Learning Using Wavenet Autoencoders Jan Chorowski, Ron J

    1 Unsupervised speech representation learning using WaveNet autoencoders Jan Chorowski, Ron J. Weiss, Samy Bengio, Aaron¨ van den Oord Abstract—We consider the task of unsupervised extraction speaker gender and identity, from phonetic content, properties of meaningful latent representations of speech by applying which are consistent with internal representations learned autoencoding neural networks to speech waveforms. The goal by speech recognizers [13], [14]. Such representations are is to learn a representation able to capture high level semantic content from the signal, e.g. phoneme identities, while being desired in several tasks, such as low resource automatic speech invariant to confounding low level details in the signal such as recognition (ASR), where only a small amount of labeled the underlying pitch contour or background noise. Since the training data is available. In such scenario, limited amounts learned representation is tuned to contain only phonetic content, of data may be sufficient to learn an acoustic model on the we resort to using a high capacity WaveNet decoder to infer representation discovered without supervision, but insufficient information discarded by the encoder from previous samples. Moreover, the behavior of autoencoder models depends on the to learn the acoustic model and a data representation in a fully kind of constraint that is applied to the latent representation. supervised manner [15], [16]. We compare three variants: a simple dimensionality reduction We focus on representations learned with autoencoders bottleneck, a Gaussian Variational Autoencoder (VAE), and a applied to raw waveforms and spectrogram features and discrete Vector Quantized VAE (VQ-VAE). We analyze the quality investigate the quality of learned representations on LibriSpeech of learned representations in terms of speaker independence, the ability to predict phonetic content, and the ability to accurately re- [17].
  • Training Generative Adversarial Networks in One Stage

    Training Generative Adversarial Networks in One Stage

    Training Generative Adversarial Networks in One Stage Chengchao Shen1, Youtan Yin1, Xinchao Wang2,5, Xubin Li3, Jie Song1,4,*, Mingli Song1 1Zhejiang University, 2National University of Singapore, 3Alibaba Group, 4Zhejiang Lab, 5Stevens Institute of Technology {chengchaoshen,youtanyin,sjie,brooksong}@zju.edu.cn, [email protected],[email protected] Abstract Two-Stage GANs (Vanilla GANs): Generative Adversarial Networks (GANs) have demon- Real strated unprecedented success in various image generation Fake tasks. The encouraging results, however, come at the price of a cumbersome training process, during which the gen- Stage1: Fix , Update 풢 풟 풢 풟 풢 풟 풢 풟 erator and discriminator풢 are alternately풟 updated in two 풢 풟 stages. In this paper, we investigate a general training Real scheme that enables training GANs efficiently in only one Fake stage. Based on the adversarial losses of the generator and discriminator, we categorize GANs into two classes, Sym- Stage 2: Fix , Update 풢 풟 metric GANs and풢 Asymmetric GANs,풟 and introduce a novel One-Stage GANs:풢 풟 풟 풢 풟 풢 풟 풢 gradient decomposition method to unify the two, allowing us to train both classes in one stage and hence alleviate Real the training effort. We also computationally analyze the ef- Fake ficiency of the proposed method, and empirically demon- strate that, the proposed method yields a solid 1.5× accel- Update and simultaneously eration across various datasets and network architectures. Figure 1: Comparison풢 of the conventional풟 Two-Stage GAN 풢 풟 Furthermore,풢 we show that the proposed풟 method is readily 풢 풟 풢 풟 풢 풟 training scheme (TSGANs) and the proposed One-Stage applicable to other adversarial-training scenarios, such as strategy (OSGANs).
  • Deep Neural Networks for Pattern Recognition

    Deep Neural Networks for Pattern Recognition

    Chapter DEEP NEURAL NETWORKS FOR PATTERN RECOGNITION Kyongsik Yun, PhD, Alexander Huyen and Thomas Lu, PhD Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, US ABSTRACT In the field of pattern recognition research, the method of using deep neural networks based on improved computing hardware recently attracted attention because of their superior accuracy compared to conventional methods. Deep neural networks simulate the human visual system and achieve human equivalent accuracy in image classification, object detection, and segmentation. This chapter introduces the basic structure of deep neural networks that simulate human neural networks. Then we identify the operational processes and applications of conditional generative adversarial networks, which are being actively researched based on the bottom-up and top-down mechanisms, the most important functions of the human visual perception process. Finally, Corresponding Author: [email protected] 2 Kyongsik Yun, Alexander Huyen and Thomas Lu recent developments in training strategies for effective learning of complex deep neural networks are addressed. 1. PATTERN RECOGNITION IN HUMAN VISION In 1959, Hubel and Wiesel inserted microelectrodes into the primary visual cortex of an anesthetized cat [1]. They project bright and dark patterns on the screen in front of the cat. They found that cells in the visual cortex were driven by a “feature detector” that won the Nobel Prize. For example, when we recognize a human face, each cell in the primary visual cortex (V1 and V2) handles the simplest features such as lines, curves, and points. The higher-level visual cortex through the ventral object recognition path (V4) handles target components such as eyes, eyebrows and noses.
  • Mixed Pattern Recognition Methodology on Wafer Maps with Pre-Trained Convolutional Neural Networks

    Mixed Pattern Recognition Methodology on Wafer Maps with Pre-Trained Convolutional Neural Networks

    Mixed Pattern Recognition Methodology on Wafer Maps with Pre-trained Convolutional Neural Networks Yunseon Byun and Jun-Geol Baek School of Industrial Management Engineering, Korea University, Seoul, South Korea {yun-seon, jungeol}@korea.ac.kr Keywords: Classification, Convolutional Neural Networks, Deep Learning, Smart Manufacturing. Abstract: In the semiconductor industry, the defect patterns on wafer bin map are related to yield degradation. Most companies control the manufacturing processes which occur to any critical defects by identifying the maps so that it is important to classify the patterns accurately. The engineers inspect the maps directly. However, it is difficult to check many wafers one by one because of the increasing demand for semiconductors. Although many studies on automatic classification have been conducted, it is still hard to classify when two or more patterns are mixed on the same map. In this study, we propose an automatic classifier that identifies whether it is a single pattern or a mixed pattern and shows what types are mixed. Convolutional neural networks are used for the classification model, and convolutional autoencoder is used for initializing the convolutional neural networks. After trained with single-type defect map data, the model is tested on single-type or mixed- type patterns. At this time, it is determined whether it is a mixed-type pattern by calculating the probability that the model assigns to each class and the threshold. The proposed method is experimented using wafer bin map data with eight defect patterns. The results show that single defect pattern maps and mixed-type defect pattern maps are identified accurately without prior knowledge.
  • Pattern Formation, Social Forces, and Diffusion Instability in Games

    Pattern Formation, Social Forces, and Diffusion Instability in Games

    EPJ manuscript No. (will be inserted by the editor) Pattern Formation, Social Forces, and Diffusion Instability in Games with Success-Driven Motion Dirk Helbing ETH Zurich, UNO D11, Universit¨atstr.41, 8092 Zurich, Switzerland Received: date / Revised version: date Abstract. A local agglomeration of cooperators can support the survival or spreading of cooperation, even when cooperation is predicted to die out according to the replicator equation, which is often used in evolutionary game theory to study the spreading and disappearance of strategies. In this paper, it is shown that success-driven motion can trigger such local agglomeration and may, therefore, be used to supplement other mechanisms supporting cooperation, like reputation or punishment. Success-driven motion is formulated here as a function of the game-theoretical payoffs. It can change the outcome and dynamics of spatial games dramatically, in particular as it causes attractive or repulsive interaction forces. These forces act when the spatial distributions of strategies are inhomogeneous. However, even when starting with homogeneous initial conditions, small perturbations can trigger large inhomogeneities by a pattern-formation instability, when certain conditions are fulfilled. Here, these instability conditions are studied for the prisoner's dilemma and the snowdrift game. Furthermore, it is demonstrated that asymmetrical diffusion can drive social, economic, and biological systems into the unstable regime, if these would be stable without diffusion. PACS. 02.50.Le Decision theory and game theory { 87.23.Ge Dynamics of social systems { 82.40.Ck Pattern formation in reactions with diffusion, flow and heat transfer { 87.23.Cc Population dynamics and ecological pattern formation 1 Introduction tions [17,18], which agree with replicator equations for the fitness-dependent reproduction of individuals in biology Game theory is a well-established theory of individual [19,20,21].
  • Introduction to Reinforcement Learning: Q Learning Lecture 17

    Introduction to Reinforcement Learning: Q Learning Lecture 17

    LLeeccttuurree 1177 Introduction to Reinforcement Learning: Q Learning Thursday 29 October 2002 William H. Hsu Department of Computing and Information Sciences, KSU http://www.kddresearch.org http://www.cis.ksu.edu/~bhsu Readings: Sections 13.3-13.4, Mitchell Sections 20.1-20.2, Russell and Norvig Kansas State University CIS 732: Machine Learning and Pattern Recognition Department of Computing and Information Sciences LLeeccttuurree OOuuttlliinnee • Readings: Chapter 13, Mitchell; Sections 20.1-20.2, Russell and Norvig – Today: Sections 13.1-13.4, Mitchell – Review: “Learning to Predict by the Method of Temporal Differences”, Sutton • Suggested Exercises: 13.2, Mitchell; 20.5, 20.6, Russell and Norvig • Control Learning – Control policies that choose optimal actions – MDP framework, continued – Issues • Delayed reward • Active learning opportunities • Partial observability • Reuse requirement • Q Learning – Dynamic programming algorithm – Deterministic and nondeterministic cases; convergence properties Kansas State University CIS 732: Machine Learning and Pattern Recognition Department of Computing and Information Sciences CCoonnttrrooll LLeeaarrnniinngg • Learning to Choose Actions – Performance element • Applying policy in uncertain environment (last time) • Control, optimization objectives: belong to intelligent agent – Applications: automation (including mobile robotics), information retrieval • Examples – Robot learning to dock on battery charger – Learning to choose actions to optimize factory output – Learning to play Backgammon
  • Image Classification by Reinforcement Learning with Two-State Q-Learning

    Image Classification by Reinforcement Learning with Two-State Q-Learning

    EasyChair Preprint № 4052 Image Classification by Reinforcement Learning with Two-State Q-Learning Abdul Mueed Hafiz and Ghulam Mohiuddin Bhat EasyChair preprints are intended for rapid dissemination of research results and are integrated with the rest of EasyChair. August 18, 2020 Image Classification by Reinforcement Learning with Two- State Q-Learning 1* 2 Abdul Mueed Hafiz , Ghulam Mohiuddin Bhat 1, 2 Department of Electronics and Communication Engineering Institute of Technology, University of Kashmir Srinagar, J&K, India, 190006 [email protected] Abstract. In this paper, a simple and efficient Hybrid Classifier is presented which is based on deep learning and reinforcement learning. Q-Learning has been used with two states and 'two or three' actions. Other techniques found in the literature use feature map extracted from Convolutional Neural Networks and use these in the Q-states along with past history. This leads to technical difficulties in these approaches because the number of states is high due to large dimensions of the feature map. Because our technique uses only two Q-states it is straightforward and consequently has much lesser number of optimization parameters, and thus also has a simple reward function. Also, the proposed technique uses novel actions for processing images as compared to other techniques found in literature. The performance of the proposed technique is compared with other recent algorithms like ResNet50, InceptionV3, etc. on popular databases including ImageNet, Cats and Dogs Dataset, and Caltech-101 Dataset. Our approach outperforms others techniques on all the datasets used. Keywords: Image Classification; ImageNet; Q-Learning; Reinforcement Learning; Resnet50; InceptionV3; Deep Learning; 1 Introduction Reinforcement Learning (RL) [1-4] has garnered much attention [4-13].