DOCSLIB.ORG

Reinforcement Learning for Data Mining Based Evolution Algorithm

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Reinforcement Learning for Data Mining Based Evolution Algorithm Reinforcement Learning for Data Mining Based Evolution Algorithm for TSK-type Neural Fuzzy Systems Design Sheng-Fuu Lin, Yi-Chang Cheng, Jyun-Wei Chang, and Yung-Chi Hsu Department of Electrical and Control Engineering, National Chiao Tung University Hsinchu, Taiwan, R.O.C. ABSTRACT membership functions of a fuzzy controller, with the fuzzy rule This paper proposed a TSK-type neural fuzzy system set assigned in advance. Juang [5] proposed the CQGAF to (TNFS) with reinforcement learning for data mining based simultaneously design the number of fuzzy rules and free evolution algorithm (R-DMEA). In R-DMEA, the group-based parameters in a fuzzy system. In [6], the group-based symbiotic symbiotic evolution (GSE) is adopted that each group represents evolution (GSE) was proposed to solve the problem that all the the set of single fuzzy rule. The R-DMEA contains both fuzzy rules were encoded into one chromosome in the structure learning and parameter learning. In the structure traditional genetic algorithm. In the GSE, each chromosome learning, the proposed R-DMEA uses the two-step self-adaptive represents a single rule of fuzzy system. The GSE uses algorithm to decide the suitable number of rules in a TNFS. In group-based population to evaluate the fuzzy rule locally. parameter learning, the R-DMEA uses the mining-based Although GSE has a better performance when compared with selection strategy to decide the selected groups in GSE by the tradition symbiotic evolution, the combination of groups is data mining method called frequent pattern growth. Illustrative fixed. example is conducted to show the performance and applicability Although the above evolution learning algorithms [4]-[6] of R-DMEA. can improve the traditional genetic algorithms, these algorithms may encounter one or more of the following problems: 1) all the Keywords : Neural fuzzy system, control, group-based fuzzy rules are encoded into one chromosome; 2) the numbers symbiotic evolution, reinforcement learning, FP-Growth. of fuzzy rules have to be assigned in advance; 3) the population cannot evaluate each fuzzy rule locally. In this paper, a TSK-type neural fuzzy system (TNFS) 1. INTRODUCTION with reinforcement learning for data mining based evolution Recently, a fuzzy system used for control problems has algorithm (R-DMEA) is proposed to solve the aforementioned become a popular research topic. A fuzzy system consists of a problems. The R-DMEA uses a two-step self-adaptive algorithm set of fuzzy if-then rules. Conventionally, the selection of fuzzy (TSSA) to determine the number of fuzzy rules automatically if-then rules often relies on a substantial amount of heuristic and processes the variable combination of chromosomes applied observations. Obviously, it is difficult for human experts to to the GSE. After the TSSA determines the number of fuzzy examine all the input-output data from a complex system to find rules (in this paper, the number of groups for GSE), R-DMEA proper rules. To cope with this difficulty, several supervised uses a data mining-based selection strategy (MSS) to decide the learning approaches that generate if-then rules from numerical groups selected to construct a TNFS. Data mining is a method data have been proposed [1]. of mining information in the database called transactions. One For some real-world applications, precise training data goal of data mining is to find association rules among frequent are usually difficult and to obtain. Therefore, there has been a item sets in transactions. Hang et al. [7] proposed frequent growing interest in reinforcement learning problems [2] and [3]. pattern growth (FP-Growth) to mine frequent patterns without Recently, the evolutionary method used for training the candidate generation. Since data mining successfully find the fuzzy model has become an important field. The evolutionary information within the large item sets, it should be useful to fuzzy model generates a fuzzy system by incorporating solve the problem of how to select groups to construct TNFS evolutionary learning procedure such as genetic algorithms with many different combinations of fuzzy rules. (GAs). Several genetic fuzzy models have been proposed in In the MSS, groups obtained from FP-Growth are [4]-[6]. In [4], Karr applied GAs to the design of the selected by using three different actions: normal, searching, and exploring actions. After executing MSS, the population can This work was supported in part by National Science Council, search better combination of individuals and simultaneously Taiwan, R.O.C., under Contract No. NSC 97-2221-E-009-091. examine whether the mined combination of individuals 3. A DATA MINING BASED EVOLUTION ALGORITHM encounter the local optimal condition. The DMEA consists of structure and parameter learning. The advantages of the R-DMEA are summarized as In the structure learning, the DMEA determines the number of follows: 1) the TSSA is proposed to decide the suitable number fuzzy rules automatically and processes the variable of rules; 2) the R-DMEA uses group-based symbiotic evolution combination of chromosomes. The length of combination of to evaluate the fuzzy rule locally; 3) the MSS is proposed to chromosomes denotes the number of rules sets forming a TNFS. select the suitable groups. The proposed DMEA adopts the GSE [6] for GAs evolution. In This paper is organized as follows. Section 2 introduces the the GSE [6], a population is divided to several groups. Each TNFS. The proposed data mining based evolution algorithm group represents a set of the chromosomes that belong to one (DMEA) is described in Section 3. Section 4 introduces the fuzzy rule. Each group will execute the reproduction and R-DMEA. Section 5 presents the results of simulation. The crossover operations in each generation independently. The conclusions are summarized in the last section. structure of the chromosome in DMEA is shown in Fig. 1(a). In DMEA, for solving the problem that the numbers of fuzzy rules 2. REVIEW A TSK-TYPE NEURO-FUZZY SYSTEM have to be assigned in advance, the two-step self-adaptive In this paper, a TSK-type fuzzy system with an R-DMEA algorithm (TSSA) is proposed. In TSSA, the building blocks is used to solve a nonlinear control problem. It is a five-layer (BBs) [8] determines the suitable length of groups combined network structure. The functions of the nodes in each layer are together and decides the number of selection times of each described as follows: length of combined groups applied to the TNFS. In Fig. 1(b), Layer1 (Input Node ): The node only transmits input values to the TSSA codes the probability vector into the building blocks layer 2. That is (BBs), Mk represents k rules that used to form a TNFS; V is a (1) M k = (1) ui x i Layer2 (Membership Function Node ): For an external input xi, probability vector represents the suitability of a TNFS model the following Gaussian membership function is used: with k rules. In the TSSA, the maximum number of rules ( Mmax ) 2 and the minimum number of rules ( M ) must be predefined to (1) − min ui m ij (2) u(2) =exp − ij σ 2 prevent the number of fuzzy rules from being generated beyond ij a certain bound. where mij and σij are, respectively, the center and the width of In parameter learning, for selecting suitable groups to the Gaussian membership function of the jth term of the ith construct a TNFS, this study proposes the MSS to select k input variable xi. individuals to form a TNFS by FP-Growth [7]. The goal of Layer 3 (Rule Node ): The function of each rule is FP-Growth is to find the frequent patterns without candidate (3)= (2) (3) generation. In MSS, FP-Growth is used to find the frequent uj∏ u ij i group sets from transactions, which means the collection of Layer 4 (Consequent Node ): The input to a node in layer 4 is well-performed chromosomes selected from Mk groups. After combined with the output delivered from layer 3 and inputs of the frequent group sets have been found, the MSS uses three layer 1. actions to select from Mk groups k rules to form a TNFS. The n (4)= (3) + (4) three actions defined in this paper are normal, searching, and uj uw j(0 j∑ wx ij i ) i=1 exploring action. In DMEA, the coding structure of the where the summation is over all the inputs and wij are the chromosomes must be suitable for GSE. Fig. 1(c) describes the corresponding parameters of the consequent part. coding of a fuzzy rule. Layer 5 (Output Node ): Each node in this layer corresponds to one output variable. The node integrates all the actions recommended by layers 3 and 4 and acts as a defuzzifier with Mk M k M k (4) (3) + ∑uj ∑ uw j(0 j ∑ wx ij i ) = = = (5) y= u (5) =j1 = j 1 i 1 Mk M k (3) (3) ∑uj ∑ u j j=1 j = 1 where Mk is the number of fuzzy rule. value with k rules greater than Best _ Fitness minus a M k predefined threshold value named ThreadFitnessvalues . Step 2. Determine the selection times according to the probability vectors of the BBs as follows: Rp = (Selection _ Times (*) V /Total _Velocy ) Figure 1: (a) The structure of the chromosome in DMEA. (b) M k M k for = ⋅⋅⋅ (12) Coding the probability vector into the building blocks (BBs) in MMMk min, min+ 1, , M max TSSA. (c) Coding a rule into a chromosome in DMEA. Rmax Total_ Velocy= ∑ V (13) M k =R Mk min The learning process of the DMEA in each group where Selection_Times represents total selection times in each involves four major operators: the TSSA, MMS, crossover generation; Rp is the selection times of Mk groups that use to strategy, and mutation strategy.
Load more

Recommended publications
  • A Study on Social Network Analysis Through Data Mining Techniques – a Detailed Survey
    ISSN (Online) 2278-1021 ISSN (Print) 2319-5940 IJARCCE International Journal of Advanced Research in Computer and Communication Engineering ICRITCSA M S Ramaiah Institute of Technology, Bangalore Vol. 5, Special Issue 2, October 2016 A Study on Social Network Analysis through Data Mining Techniques – A detailed Survey Annie Syrien1, M. Hanumanthappa2 Research Scholar, Department of Computer Science and Applications, Bangalore University, India1 Professor, Department of Computer Science and Applications, Bangalore University, India2 Abstract: The paper aims to have a detailed study on data collection, data preprocessing and various methods used in developing a useful algorithms or methodologies on social network analysis in social media. The recent trends and advancements in the big data have led many researchers to focus on social media. Web enabled devices is an another important reason for this advancement, electronic media such as tablets, mobile phones, desktops, laptops and notepads enable the users to actively participate in different social networking systems. Many research has also significantly shows the advantages and challenges that social media has posed to the research world. The principal objective of this paper is to provide an overview of social media research carried out in recent years. Keywords: data mining, social media, big data. I. INTRODUCTION Data mining is an important technique in social media in scrutiny and analysis. In any industry data mining recent years as it is used to help in extraction of lot of technique based reports become the base line for making useful information, whereas this information turns to be an decisions and formulating the decisions. This report important asset to both academia and industries.
    [Show full text]
  • Geoseq2seq: Information Geometric Sequence-To-Sequence Networks
    GEOSEQ2SEQ:INFORMATION GEOMETRIC SEQUENCE-TO-SEQUENCE NETWORKS Alessandro Bay Biswa Sengupta Cortexica Vision Systems Ltd. Noah’s Ark Lab (Huawei Technologies UK) London, UK Imperial College London, London, UK [email protected] ABSTRACT The Fisher information metric is an important foundation of information geometry, wherein it allows us to approximate the local geometry of a probability distribu- tion. Recurrent neural networks such as the Sequence-to-Sequence (Seq2Seq) networks that have lately been used to yield state-of-the-art performance on speech translation or image captioning have so far ignored the geometry of the latent embedding, that they iteratively learn. We propose the information geometric Seq2Seq (GeoSeq2Seq) network which abridges the gap between deep recurrent neural networks and information geometry. Specifically, the latent embedding offered by a recurrent network is encoded as a Fisher kernel of a parametric Gaus- sian Mixture Model, a formalism common in computer vision. We utilise such a network to predict the shortest routes between two nodes of a graph by learning the adjacency matrix using the GeoSeq2Seq formalism; our results show that for such a problem the probabilistic representation of the latent embedding supersedes the non-probabilistic embedding by 10-15%. 1 INTRODUCTION Information geometry situates itself in the intersection of probability theory and differential geometry, wherein it has found utility in understanding the geometry of a wide variety of probability models (Amari & Nagaoka, 2000). By virtue of Cencov’s characterisation theorem, the metric on such probability manifolds is described by a unique kernel known as the Fisher information metric. In statistics, the Fisher information is simply the variance of the score function.
    [Show full text]
  • Data Mining and Soft Computing
    © 2002 WIT Press, Ashurst Lodge, Southampton, SO40 7AA, UK. All rights reserved. Web: www.witpress.com Email [email protected] Paper from: Data Mining III, A Zanasi, CA Brebbia, NFF Ebecken & P Melli (Editors). ISBN 1-85312-925-9 Data mining and soft computing O. Ciftcioglu Faculty of Architecture, Deljl University of Technology Delft, The Netherlands Abstract The term data mining refers to information elicitation. On the other hand, soft computing deals with information processing. If these two key properties can be combined in a constructive way, then this formation can effectively be used for knowledge discovery in large databases. Referring to this synergetic combination, the basic merits of data mining and soft computing paradigms are pointed out and novel data mining implementation coupled to a soft computing approach for knowledge discovery is presented. Knowledge modeling by machine learning together with the computer experiments is described and the effectiveness of the machine learning approach employed is demonstrated. 1 Introduction Although the concept of data mining can be defined in several ways, it is clear enough to understand that it is related to information extraction especially in large databases. The methods of data mining are essentially borrowed from exact sciences among which statistics play the essential role. By contrast, soft computing is essentially used for information processing by employing methods, which are capable to deal with imprecision and uncertainty especially needed in ill-defined problem areas. In the former case, the outcomes are exact within the error bounds estimated. In the latter, the outcomes are approximate and in some cases they may be interpreted as outcomes from an intelligent behavior.
    [Show full text]
  • Data Mining with Cloud Computing: - an Overview
    International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) Volume 5 Issue 1, January 2016 Data Mining with Cloud Computing: - An Overview Miss. Rohini A. Dhote Dr. S. P. Deshpande Department of Computer Science & Information Technology HOD at Computer Science Technology Department, HVPM, COET Amravati, Maharashtra, India. HVPM, COET Amravati, Maharashtra, India Abstract: Data mining is a process of extracting competitive environment for data analysis. The cloud potentially useful information from data, so as to makes it possible for you to access your information improve the quality of information service. The from anywhere at any time. The cloud removes the integration of data mining techniques with Cloud need for you to be in the same physical location as computing allows the users to extract useful the hardware that stores your data. The use of cloud information from a data warehouse that reduces the computing is gaining popularity due to its mobility, costs of infrastructure and storage. Data mining has attracted a great deal of attention in the information huge availability and low cost. industry and in society as a whole in recent Years Wide availability of huge amounts of data and the imminent 2. DATA MINING need for turning such data into useful information and Data Mining involves the use of sophisticated data knowledge Market analysis, fraud detection, and and tool to discover previously unknown, valid customer retention, production control and science patterns and Relationship in large data sets. These exploration. Data mining can be viewed as a result of tools can include statistical models, mathematical the natural evolution of information technology.
    [Show full text]
  • Challenges in Bioinformatics for Statistical Data Miners
    This article appeared in the October 2003 issue of the “Bulletin of the Swiss Statistical Society” (46; 10-17), and is reprinted by permission from its editor. Challenges in Bioinformatics for Statistical Data Miners Dr. Diego Kuonen Statoo Consulting, PSE-B, 1015 Lausanne 15, Switzerland [email protected] Abstract Starting with possible definitions of statistical data mining and bioinformatics, this article will give a general side-by-side view of both fields, emphasising on the statistical data mining part and its techniques, illustrate possible synergies and discuss how statistical data miners may collaborate in bioinformatics’ challenges in order to unlock the secrets of the cell. What is statistics and why is it needed? Statistics is the science of “learning from data”. It includes everything from planning for the collection of data and subsequent data management to end-of-the-line activities, such as drawing inferences from numerical facts called data and presentation of results. Statistics is concerned with one of the most basic of human needs: the need to find out more about the world and how it operates in face of variation and uncertainty. Because of the increasing use of statistics, it has become very important to understand and practise statistical thinking. Or, in the words of H. G. Wells: “Statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write”. But, why is statistics needed? Knowledge is what we know. Information is the communication of knowledge. Data are known to be crude information and not knowledge by itself. The way leading from data to knowledge is as follows: from data to information (data become information when they become relevant to the decision problem); from information to facts (information becomes facts when the data can support it); and finally, from facts to knowledge (facts become knowledge when they are used in the successful completion of the decision process).
    [Show full text]
  • The Theoretical Framework of Data Mining & Its
    International Journal of Social Science & Interdisciplinary Research__________________________________ ISSN 2277- 3630 IJSSIR, Vol.2 (1), January (2013) Online available at indianresearchjournals.com THE THEORETICAL FRAMEWORK OF DATA MINING & ITS TECHNIQUES SAYYAD RASHEEDUDDIN Associate Professor – HOD CSE dept Pulla Reddy Engineering College Wargal, Medak(Dist), A.P. ABSTRACT Data mining is a process which finds useful patterns from large amount of data. The research in databases and information technology has given rise to an approach to store and manipulate this precious data for further decision making. Data mining is a process of extraction of useful information and patterns from huge data. It is also called as knowledge discovery process, knowledge mining from data, knowledge extraction or data pattern analysis. The current paper discusses the following Objectives: 1. To discuss briefly about the concept of Data Mining 2. To explain about the Data Mining Algorithms & Techniques 3. To know about the Data Mining Applications Keywords: Concept of Data mining, Data mining Techniques, Data mining algorithms, Data mining applications. Introduction to Data Mining The development of Information Technology has generated large amount of databases and huge data in various areas. The research in databases and information technology has given rise to an approach to store and manipulate this precious data for further decision making. Data mining is a process of extraction of useful information and patterns from huge data. It is also called as knowledge discovery process, knowledge mining from data, knowledge extraction or data /pattern analysis. The current paper discuss about the following Objectives: 1. To discuss briefly about the concept of Data Mining 2.
    [Show full text]
  • Machine Learning and Data Mining Machine Learning Algorithms Enable Discovery of Important “Regularities” in Large Data Sets
    General Domains Machine Learning and Data Mining Machine learning algorithms enable discovery of important “regularities” in large data sets. Over the past Tom M. Mitchell decade, many organizations have begun to routinely cap- ture huge volumes of historical data describing their operations, products, and customers. At the same time, scientists and engineers in many fields have been captur- ing increasingly complex experimental data sets, such as gigabytes of functional mag- netic resonance imaging (MRI) data describing brain activity in humans. The field of data mining addresses the question of how best to use this historical data to discover general regularities and improve PHILIP NORTHOVER/PNORTHOV.FUTURE.EASYSPACE.COM/INDEX.HTML the process of making decisions. COMMUNICATIONS OF THE ACM November 1999/Vol. 42, No. 11 31 he increasing interest in Figure 1. Data mining application. A historical set of 9,714 medical data mining, or the use of records describes pregnant women over time. The top portion is a historical data to discover typical patient record (“?” indicates the feature value is unknown). regularities and improve The task for the algorithm is to discover rules that predict which T future patients will be at high risk of requiring an emergency future decisions, follows from the confluence of several recent trends: C-section delivery. The bottom portion shows one of many rules the falling cost of large data storage discovered from this data. Whereas 7% of all pregnant women in devices and the increasing ease of the data set received emergency C-sections, the rule identifies a collecting data over networks; the subclass at 60% at risk for needing C-sections.
    [Show full text]
  • Big Data, Data Mining and Machine Learning
    Contents Forward xiii Preface xv Acknowledgments xix Introduction 1 Big Data Timeline 5 Why This Topic Is Relevant Now 8 Is Big Data a Fad? 9 Where Using Big Data Makes a Big Difference 12 Part One The Computing Environment ..............................23 Chapter 1 Hardware 27 Storage (Disk) 27 Central Processing Unit 29 Memory 31 Network 33 Chapter 2 Distributed Systems 35 Database Computing 36 File System Computing 37 Considerations 39 Chapter 3 Analytical Tools 43 Weka 43 Java and JVM Languages 44 R 47 Python 49 SAS 50 ix ftoc ix April 17, 2014 10:05 PM Excerpt from Big Data, Data Mining, and Machine Learning: Value Creation for Business Leaders and Practitioners by Jared Dean. Copyright © 2014 SAS Institute Inc., Cary, North Carolina, USA. ALL RIGHTS RESERVED. x ▸ CONTENTS Part Two Turning Data into Business Value .......................53 Chapter 4 Predictive Modeling 55 A Methodology for Building Models 58 sEMMA 61 Binary Classifi cation 64 Multilevel Classifi cation 66 Interval Prediction 66 Assessment of Predictive Models 67 Chapter 5 Common Predictive Modeling Techniques 71 RFM 72 Regression 75 Generalized Linear Models 84 Neural Networks 90 Decision and Regression Trees 101 Support Vector Machines 107 Bayesian Methods Network Classifi cation 113 Ensemble Methods 124 Chapter 6 Segmentation 127 Cluster Analysis 132 Distance Measures (Metrics) 133 Evaluating Clustering 134 Number of Clusters 135 K‐means Algorithm 137 Hierarchical Clustering 138 Profi ling Clusters 138 Chapter 7 Incremental Response Modeling 141 Building the Response
    [Show full text]
  • Deep Reinforcement Learning for Imbalanced Classification
    Deep Reinforcement Learning for Imbalanced Classification Enlu Lin1 Qiong Chen2,* Xiaoming Qi3 School of Computer Science and Engineering South China University of Technology Guangzhou, China [email protected] [email protected] [email protected] Abstract—Data in real-world application often exhibit skewed misclassification cost to the minority class. However, with the class distribution which poses an intense challenge for machine rapid developments of big data, a large amount of complex learning. Conventional classification algorithms are not effective data with high imbalanced ratio is generated which brings an in the case of imbalanced data distribution, and may fail when the data distribution is highly imbalanced. To address this issue, we enormous challenge in imbalanced data classification. Con- propose a general imbalanced classification model based on deep ventional methods are inadequate to cope with more and reinforcement learning. We formulate the classification problem more complex data so that novel deep learning approaches as a sequential decision-making process and solve it by deep Q- are increasingly popular. learning network. The agent performs a classification action on In recent years, deep reinforcement learning has been one sample at each time step, and the environment evaluates the classification action and returns a reward to the agent. The successfully applied to computer games, robots controlling, reward from minority class sample is larger so the agent is more recommendation systems [5]–[7] and so on. For classification sensitive to the minority class. The agent finally finds an optimal problems, deep reinforcement learning has served in eliminat- classification policy in imbalanced data under the guidance of ing noisy data and learning better features, which made a great specific reward function and beneficial learning environment.
    [Show full text]
  • Data Mining and Machine Learning Techniques for Aerodynamic Databases: Introduction, Methodology and Potential Benefits
    energies Article Data Mining and Machine Learning Techniques for Aerodynamic Databases: Introduction, Methodology and Potential Benefits Esther Andrés-Pérez Theoretical and Computational Aerodynamics Branch, Flight Physics Department, Spanish National Institute for Aerospace Technology (INTA), Ctra. Ajalvir, km. 4, 28850 Torrejón de Ardoz, Spain; [email protected] Received: 31 July 2020; Accepted: 15 October 2020; Published: 6 November 2020 Abstract: Machine learning and data mining techniques are nowadays being used in many business sectors to exploit the data in order to detect trends, discover certain features and patters, or even predict the future. However, in the field of aerodynamics, the application of these techniques is still in the initial stages. This paper focuses on exploring the benefits that machine learning and data mining techniques can offer to aerodynamicists in order to extract knowledge from the CFD data and to make quick predictions of aerodynamic coefficients. For this purpose, three aerodynamic databases (NACA0012 airfoil, RAE2822 airfoil and 3D DPW wing) have been used and results show that machine-learning and data-mining techniques have a huge potential also in this field. Keywords: machine learning; data mining; aerodynamic analysis; computational fluid dynamics; surrogate modeling; linear regression; support vector regression 1. Introduction In the field of aerodynamics, complex steady flows are simulated by computational fluid dynamics (CFD) daily in the industry since CFD tools have already reached an acceptable level of maturity. These simulations are usually performed over full-aircraft configurations or several aircraft components where meshes of hundreds of million points are required in order to provide precise features of the flow. In addition, simulations are performed for different parameters to properly explore the design space.
    [Show full text]
  • Journal of Theoretical & Computational Science
    Journal of Theoretical & Computational Science Theoretical aspects of data science Marcello Trovati Edge Hill University, UK Abstract Data Science is an emerging research field consisting of novel methods and approaches to identify actionable knowledge from the continuous creation of data. The applications of this research field to data-driven sciences has led to the discovery of new research directions, with ground-breaking results. In particular, new mathematical theories have contributed to enhanced Data Science frame- works, focusing on the algorithms and mathematical models to identify, extract and assess actionable information. Furthermore, these theoretical approaches have important implications to decision systems based on Big Data, and their application to a variety of multi-disciplinary scenarios, such as Criminology, eHealth and business analysis. Biography After having obtained his PhD in Mathematics at the University of Exeter in 2007, specializing in theoretical dynamical systems with singularities, he has worked for R&D organizations as well as academic institutions, including IBM Research where he was involved in a number of research projects. In 2016 Marcello joined the Computer Science Department at Edge Hill University as a senior lecturer and he was recently awarded a Readership in Computer Science. He is involved in several research themes and projects. He is co-leading the STEM Data Research centre and is actively involved in the Productivity and Innovation Lab, aiming to collaborate and support SMEs in Lancashire. Marcello’s main interests include: Mathematical Modelling, Data Science, Big Data Analytics, Network Theory, Machine Learning, Data and Text Mining. Publication 1. Big-Data Analytics and Cloud Computing, Theory, Algorithms and Applications, to appear in the Computer Communications and Networks, Springer, 2016.
    [Show full text]
  • Privacy-Preserving Reinforcement Learning
    Privacy-Preserving Reinforcement Learning Jun Sakuma [email protected] Shigenobu Kobayashi [email protected] Tokyo Institute of Technology, 4259 Nagatsuta-cho, Midoriku, Yokohama, 226-8502, Japan Rebecca N. Wright [email protected] Rutgers University, 96 Frelinghuysen Road, Piscataway, NJ 08854, USA Abstract In this paper, we consider the privacy of agents’ per- ceptions in DRL. Specifically, we provide solutions for We consider the problem of distributed rein- privacy-preserving reinforcement learning (PPRL), in forcement learning (DRL) from private per- which agents’ perceptions, such as states, rewards, and ceptions. In our setting, agents’ perceptions, actions, are not only distributed but are desired to be such as states, rewards, and actions, are not kept private. Consider two example scenarios: only distributed but also should be kept pri- vate. Conventional DRL algorithms can han- Optimized Marketing (Abe et al., 2004): Consider dle multiple agents, but do not necessarily the modeling of the customer’s purchase behavior as guarantee privacy preservation and may not a Markov Decision Process (MDP). The goal is to ob- guarantee optimality. In this work, we design tain the optimal catalog mailing strategy which max- cryptographic solutions that achieve optimal imizes the long-term profit. Timestamped histories of policies without requiring the agents to share customer status and mailing records are used as state their private information. variables. Their purchase patterns are used as actions. Value functions
    [Show full text]