AGENT INTELLIGENCE THROUGH DATA MINING MULTIAGENT SYSTEMS, ARTIFICIAL SOCIETIES, AND SIMULATED ORGANIZATIONS International Book Series Series Editor: Gerhard Weiss, Technische Universität München Editorial Board: Kathleen M. Carley, Carnegie Mellon University, PA, USA Yves Demazeau, CNRS Laboratoire LEIBNIZ, France Ed Durfee, University of Michigan, USA Les Gasser, University of Illinois at Urbana-Champaign, IL, USA Nigel Gilbert, University of Surrey, United Kingdom Michael Huhns, University of South Carolina, SC, USA Nick Jennings, University of Southampton, UK Victor Lesser, University of Massachusetts, MA, USA Katia Sycara, Carnegie Mellon University, PA, USA Michael Wooldridge, University of Liverpool, United Kingdom Books in the Series: AGENT INTELLIGENCE THROUGH DATA MINING, Andreas Symeonidis, Pericles Mitkas, ISBN 0-387-24352-6 EXTENDING WEB SERVICES TECHNOLOGIES: The Use of Multi-Agent Approaches, edited by Lawrence Cavedon, Zakaria Maamar, David Martin, and Boualem Benatallah, ISBN 0-387-23343-1 AUTONOMY ORIENTED COMPUTING: From Problem Solving to Complex Systems Modeling, Jiming Liu, XiaoLong Jin, and Kwok Ching Tsui, ISBN1-4020-8121-9 METHODOLOGIES AND SOFTWARE ENGINEERING FOR AGENT SYSTEMS: The Agent-Oriented Software Engineering Handbook, edited by Federico Bergenti, Marie- Pierre Gleizes, Franco Zambonelli AN APPLICATION SCIENCE FOR MULTI-AGENT SYSTEMS, edited by Thomas A. Wagner, ISBN: 1-4020-7867-6 DISTRIBUTED SENSOR NETWORKS, edited by Victor Lesser, Charles L. Ortiz, Jr., Milind Tambe, ISBN: 1-4020-7499-9 AGENT SUPPORTED COOPERATIVE WORK, edited by Yiming Ye, Elizabeth Churchill, ISBN: 1-4020-7404-2 AGENT AUTONOMY, edited by Henry Hexmoor, Cristiano Castelfranchi, Rino Falcone, ISBN: 1-4020-7402-6 REPUTATION IN ARTIFICIAL SOCIETIES: Social Beliefs for Social Order, by Rosaria Conte, Mario Paolucci, ISBN: 1-4020-7186-8 GAME THEORY AND DECISION THEORY IN AGENT-BASED SYSTEMS, edited by Simon Parsons, Piotr Gmytrasiewicz, Michael Wooldridge, ISBN: 1-4020-7115-9 AGENT INTELLIGENCE THROUGH DATA MINING Andreas L, Symeonidis Pericles A. Mitkas Aristotle University ofThessaloniki Greece 4ut Springer Andreas L. Symeonidis Pericles A. Mitkas Postdoctoral Research Associate Associate Professor Electrical and Computer Engineering Dept. Electrical and Computer Engineering Dept. Aristotle University of Thessaloniki Aristotle University of Thessaloniki 54124/Thessaloniki, Greece 54124,Thessaloniki, Greece Library of Congress Cataloging-in-Publication Data A C.I.P. Catalogue record for this book is available from the Library of Congress. AGENT INTELLIGENCE THROUGH DATA MINING by Andreas L. Symeonidis and Pericles A. Mitkas Aristotle University of Thessaloniki,Greece Multiagent Systems, Artificial Societies, and Simulated Organizations Series Volume 14 ISBN-10: 0-387-24352-6 e-ISBN-10: 0-387-25757-8 ISBN-13: 978-0-387-24352-8 e-ISBN-13: 978-0-387-25757-0 Printed on acid-free paper. © 2005 Springer Science+Business Media, Inc. All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, Inc., 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now know or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks and similar terms, even if the are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed in the United States of America. 987654321 SPIN 11374831, 11421214 springeronline.com Andreas L. Symeonidis dedicates this book to his sister, Kyriaki, and to the "Hurri"cane of his life... Pericles A. Mitkas dedicates this book to Sophia, Alexander, and Danae. For all the good years... Contents Dedication v List of Figures xiii List of Tables xvii Foreword xix Preface xxi Acknowledgments xxv Part I Concepts and Techniques 1. INTRODUCTION 3 1 The Quest for Knowledge 3 2 Problem Description 4 3 Related Bibliography 5 4 Scope of the Book 6 5 Contents of the Book 8 6 How to Read this Book 9 2. DATA MINING AND KNOWLEDGE DISCOVERY: A BRIEF OVERVIEW 11 1 History and Motivation 11 1.1 The Emergence of Data Mining 11 1.2 So, what is Data Mining? 13 1.3 The KDD Process 13 1.4 Organizing Data Mining Techniques 15 2 Data Preprocessing 18 2.1 The Scope of Data Preprocessing 18 2.2 Data Cleaning 18 viii A GENT INTELLIGENCE THR 0 UGH DA TA MINING 2.3 Data Integration 19 2.4 Data Transformation 19 2.5 Data Reduction 20 2.6 Data Discretization 20 3 Classification and Prediction 21 3.1 Defining Classification 21 3.2 Bayesian Classification 21 3.3 Decision Trees 22 3.3.1 The ID3 algorithm 24 4 Clustering 26 4.1 Definitions 27 4.2 Clustering Techniques 27 4.3 Representative Clustering Algorithms 28 4.3.1 Partitioning Algorithms 28 4.3.2 Hierarchical Algorithms 29 4.3.3 Density-Based Algorithms 30 5 Association Rule Extraction 32 5.1 Definitions 32 5.2 Representative Algorithms 33 6 Evolutionary Data Mining Algorithms 35 6.1 The Basic Concepts of Genetic Algorithms 35 6.2 Genetic Algorithm Terminology 36 6.3 Genetic Algorithm Operands 37 6.4 The Genetic Algorithm Mechanism 38 6.5 Application of Genetic Algorithms 38 7 Chapter review 40 3. INTELLIGENT AGENTS AND MULTI-AGENT SYSTEMS 41 1 Intelligent Agents 41 1.1 Agent Definition 41 1.2 Agent Features and Working Definitions 42 1.3 Agent Classification 44 1.4 Agents and Objects 45 1.5 Agents and Expert Systems 47 1.6 Agent Programming Languages 47 2 Multi-Agent Systems 48 2.1 Multi-Agent System Characteristics 50 2.2 Agent Communication 51 2.3 Agent Communication Languages 53 Contents ix 2.3.1 KQML 53 2.3.2 KIF 54 2.3.3 FIPA ACL 54 2.4 Agent Communities 55 Part II Methodology 4. EXPLOITING DATA MINING ON MAS 59 1 Introduction 59 1.1 Logic and Limitations 60 1.2 Agent Training and Knowledge Diffusion 62 1.3 Three Levels of Knowledge Diffusion for MAS 63 2 MAS Development Tools 63 3 Agent Academy 66 3.1 A A Architecture 67 3.2 Developing Multi-Agent Applications 68 3.3 Creating Agent Ontologies 68 3.4 Creating Behavior Types 68 3.5 Creating Agent Types 69 3.6 Deploying a Multi Agent System 69 5. COUPLING DATA MINING WITH INTELLIGENT AGENTS 71 1 The Unified Methodology 72 1.1 Formal Model 72 1.1.1 Case 1: Training at the MAS application level 72 1.1.2 Case 2: Training at the MAS behavior level 72 1.1.3 Case 3: Training evolutionary agent communities 72 1.2 Common Primitives for MAS Development 73 1.3 Application Level: The Training Framework 76 1.4 Behavior Level: The Training Framework 77 1.5 Evolutionary Level: The Training Framework 80 2 Data Miner: A Tool for Training and Retraining Agents 82 2.1 Prerequisites for Using the Data Miner 82 2.2 Data Miner Overview 82 2.3 Selection of the Appropriate DM Technique 85 2.4 Training and Retraining with the Data Miner 86 x AGENT INTELLIGENCE THROUGH DATA MINING Part III Knowledge Diffusion: Three Representative Test Cases 6. DATA MINING ON THE APPLICATION LEVEL OF A MAS 93 1 Enterprise Resource Planning Systems 93 2 The Generalized Framework 95 2.1 IRF Architecture 97 2.1.1 Customer Order Agent type 98 2.1.2 Recommendation Agent type 99 2.1.3 Customer Profile Identification Agent type 99 2.1.4 Supplier Pattern Identification Agent type 100 2.1.5 Inventory Profile Identification Agent type 100 2.1.6 Enterprise Resource Planning Agent type 100 2.2 Installation and Runtime Workflows 101 2.3 System Intelligence 103 2.3.1 Benchmarking customer and suppliers 103 2.3.2 IPIA products profile 106 2.3.3 RA Intelligence 106 3 An IRF Demonstrator 109 4 Conclusions 112 7. MINING AGENT BEHAVIORS 115 1 Predicting Agent Behavior 115 1.1 The Prediction Mechanism 115 1.2 Applying «-Profile on MAS 119 1.3 Modeling Agent Actions in an Operation Cycle 121 1.4 Mapping Agent Actions to Vectors 122 1.5 Evaluating Efficiency 123 1.5.1 Profile efficiency evaluation 123 1.5.2 Prediction system efficiency evaluation 124 2 A Recommendation Engine Demonstrator 124 2.1 System Parameters 125 2.1.1 The fuzzy variable Time 125 2.1.2 The fuzzy variable Frequency 126 2.1.3 The output fuzzy variable Weight 127 2.2 The Rules of the FIS 127 2.3 Browsing through a Web Site 130 3 Experimental Results 131 4 Conclusions 133 Contents xi 3. MINING KNOWLEDGE FOR AGENT COMMUNITIES 135 1 Ecosystem Simulation 135 2 An Overview of Biotope 138 2.1 The Biotope Environment 138 2.2 The Biotope Agents 139 2.2.1 Agent sight 139 2.2.2 Agent movement 139 2.2.3 Agent reproduction 141 2.2.4 Agent communication - Knowledge exchange 141 2.3 Knowledge Extraction and Improvement 142 2.3.1 Classifiers 143 2.3.2 Classifier Evaluation mechanism 143 2.3.3 Genetic Algorithm 144 2.4 The Assessment Indicators 145 2.4.1 Environmental indicators 145 2.4.2 Agent performance indicators 146 3 The Implemented Prototype 148 3.1 Creating a New Simulation Scenario 149 4 Experimental Results 150 4.1 Exploiting the Potential of Agent Communication 151 4.1.1 Specifying the optimal communication rate 152 4.1.2 Agent efficiency and knowledge base size 152 4.1.3 Agent communication and unreliability 153 4.2 GAs in Unreliable Environments 155 4.3 Simulating Various Environments 158 5 Conclusions 160 Part IV Extensions..