DOCSLIB.ORG

Extracting Decision Trees from Diagnostic Bayesian Networks to Guide Test Selection

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Extracting Decision Trees from Diagnostic Bayesian Networks to Guide Test Selection Extracting Decision Trees from Diagnostic Bayesian Networks to Guide Test Selection Scott Wahl 1, John W. Sheppard 1 1 Department of Computer Science, Montana State University, Bozeman, MT, 59717, USA [email protected] [email protected] ABSTRACT procedure is a natural extension of the general process In this paper, we present a comparison of five differ- used in troubleshooting systems. Given no prior knowl- ent approaches to extracting decision trees from diag- edge, tests are performed sequentially, continuously nar- nostic Bayesian nets, including an approach based on rowing down the ambiguity group of likely faults. Re- the dependency structure of the network itself. With this sulting decision trees are called “fault trees” in the sys- approach, attributes used in branching the decision tree tem maintenance literature. are selected by a weighted information gain metric com- In recent years, tools have emerged that apply an al- puted based upon an associated D-matrix. Using these ternative approach to fault diagnosis using diagnostic trees, tests are recommended for setting evidence within Bayesian networks. One early example of such a net- the diagnostic Bayesian nets for use in a PHM applica- work can be found in the creation of the the QMR knowl- tion. We hypothesized that this approach would yield edge base (Shwe et al., 1991), used in medical diagno- effective decision trees and test selection and greatly re- sis. Bayesian networks provide a means for incorpo- duce the amount of evidence required for obtaining ac- rating uncertainty into the diagnostic process; however, curate classification with the associated Bayesian net- Bayesian networks by themselves provide no guidance works. The approach is compared against three alter- on which tests to perform when. Rather, test information natives to creating decision trees from probabilistic net- is applied as evidence whenever it becomes available. In works such as ID3 using a dataset forward sampled from this paper, we compare approaches to using Bayesian the network, KL-divergence, and maximum expected networks to derive decision trees to integrate the ad- utility. In addition, the effects of using χ2 statistics and vantage of each approach.1 Our approach weights the probability measures for pre-pruning are examined. The classes based on prior probability distributions and then results of our comparison indicate that our approach pro- derives a weighted decision tree using the associated D- vides compact decision trees that lead to high accuracy matrix characterizing the structure of the Bayesian net- classification with the Bayesian networks when com- work. We hypothesize that this method will yield com- pared to trees of similar size generated by the other meth- pact decision trees (thus reducing the amount of evidence ods, thus supporting our hypothesis. required to be evaluated) that result in high classification accuracy relative to alternative methods that we evalu- ated. 1. INTRODUCTION Historically, creating decision trees is usually based Proper support of a system is wisely regarded as vital around a static set of data (Casey & Nagy, 1984; Heck- to its function. In general, support of a system involves erman, Geiger, & Chickering, 1995; Martin, 1997; both corrective and preventative maintenance. The pri- Murthy, 1997; Quinlan, 1986). It has been noted by such mary goal of corrective maintenance is to repair faults previous work that information theory and other methods while preventative maintenance attempts to avoid faults are based upon the assumption that the data set accu- or improve the useful lifetime of parts. Satisfying these rately represents the true underlying distribution of the goals requires isolating what faults have occurred or are data. To completely do so requires an infinitely large most likely to occur. data set. However, by using diagnostic Bayesian net- Decision trees have been used extensively in perform- works directly, it is possible to use the distributions and ing fault diagnosis during corrective maintenance. This information provided by the network to create a decision tree directly. Without adequate stopping criteria, how- This is an open-access article distributed under the terms of ever, the resulting trees are likely to be extraordinarily the Creative Commons Attribution 3.0 United States License, which permits unrestricted use, distribution, and reproduction 1While this paper focuses on deriving static decision trees, in any medium, provided the original author and source are the algorithms used can be adapted easily to an online setting credited. where tests are recommended dynamically. 1 Annual Conference of the Prognostics and Health Management Society, 2010 large. Another potential method for creating decision of these measures are used directly in the generation of trees from probabilistic networks is based upon using the decision trees in this paper. Specifically, information the structure of the network. The work here uses a spe- theory is used in generating the decision tree from a D- cialized adjacency matrix used for diagnostic networks, Matrix and by forward sampling the diagnostic network. called a D-matrix, to accomplish this task. Much of the prior work in creating decision trees In describing the results of our study, this paper has and test selection from Bayesian networks focuses on been organized as follows. Section 2 provides motiva- reducing the expected cost of performing a series of tion behind the current study. Section 3 describes related actions, both tests and repairs (Heckerman, Breese, & work with decision trees and Bayesian networks. Sec- Rommelse, 1995; Mussi, 2004; Skaanning, Jensen, & tion 4 provides background information on probabilistic Kjærulff, 2000; Jensen et al., 2001; Vomlel, 2003). In models and decision trees. Section 5 specifies the ap- the SACSO system, tests and actions are recommended proach used in creating the decision trees. Sections 6 based upon the expected cost of taking any action or per- and 7 give the results and discussion, respectively. Fi- forming of tests using the probability the problem will nally, we conclude in section 8. be corrected and a cost function. Given the nature of the problem, the SACSO system relies on a set of networks 2. MOTIVATION where each reported problem, e.g. spots on a page, is the root of its own tree. A similar approach was used Recent trends in developing diagnostic tools have fo- by Mussi for the generation of sequential decision sup- cused on providing online methods for selecting tests to port systems. These approaches are similar to what is evaluate and on incorporating uncertainty into the rea- defined by Koller for measures of maximum expected soning process. Legacy diagnostic systems rely on static utility (MEU) (Koller & Friedman, 2009). Given that fault trees to guide the test and diagnosis process. The performing the inference for such measures is intractable primary issue with using static trees is that they are not for large problems, Zheng, Rish, and Beygelzimer devel- able to adapt to changing conditions such as the loss of a oped an algorithm for calculating approximate entropy test resource. Even so, evaluating the performance of an (Zheng, Rish, & Beygelzimer, 2005). adaptive system is difficult in that the reasons for select- Use of probabilities in creating decision trees has been ing certain sources of evidence may be unclear. analyzed before by Casey and Nagy for use in optical Our research is focused on developing Bayesian diag- character recognition (Casey & Nagy, 1984). Based nostic systems that can be assessed through sound em- upon a data set, the probability of a single pixel having pirical analysis. In an attempt to reduce the amount of a specific color was determined for each possible letter. evidence to be evaluated and maintain control, we found This information was then used in creating the decision it beneficial to use static decision trees in the evaluation tree. process. Such trees can be used to control the amount Additional work has been performed in optimal cre- of evidence evaluated, can be examined to justify the ation of AND/OR decision trees. Research by Patti- test choices, and can be applied consistently across all pati and Alexandridis has shown that using a heuristic methods studied. Unfortunately, as we started looking search with AND/OR graphs and the Gini approach can for existing approaches to generate decision trees to be be used to create optimal and near-optimal test sequences used with Bayesian networks, we found very little direct, for problems (Pattipati & Alexandridis, 1990). comparative work had been done in this area. There- fore, we sought to evaluate alternative approaches with There has also been research performed in extract- the goal of finding one that had minimal computational ing rules and decision trees from neural networks in an burden (both in generating the tree and in using the tree) attempt to provide context to learned neural networks and still yielded accurate results once evidence was eval- (Craven. & Shavlik, 1996). Additionally, combining the uated. two approaches, decision trees and Bayesian networks, has been performed by Zhou, Zheng, and Chen (Zhou, In order to do so, three existing methods were se- Zhang, & Chen, 2006). Their research associated a prior lected based upon information theory and value of in- distribution to the leaves of the decision tree and uti- formation: forward sampling and ID3, maximum ex- lized a Monte-Carlo method to perform inference.
Load more

Recommended publications
  • ID3 ALGORITHM in DATA MINING APPLIED to DIABETES DATABASE Dr.R.Jamuna
    IJITE Vol.03 Issue-03, (March, 2015) ISSN: 2321-1776 Impact Factor- 3.570 ID3 ALGORITHM IN DATA MINING APPLIED TO DIABETES DATABASE Dr.R.Jamuna Professor, Department of Computer Science S.R.College, Bharathidasan university, Trichy. Abstract The past decade has seen a flurry of promising breakthroughs in data mining predictions. Many of these developments hold the potential to prevent dreadful diseases to improve the quality of life. The advances in medical science and technology have corresponded to the use of computer algorithms as an intermediary between the medical researchers and technocrats. Diabetes Mellitus is a major killer disease of mankind today. Data mining techniques can be used to highlight the significant factors causing such a disorder. Even though total cure is not possible for this pancreatic disorder the complications can be avoided by awareness using data mining algorithms. In this paper, eight major factors playing significant role in the Pima Indian population are analyzed. Real time data is taken from the dataset of National Institute of Diabetes and Digestive and Kidney Diseases. The data is subjected to an analysis by logistic regression method using spss 7.5 statistical software, to isolate the most significant factors. Then the significant factors are further applied to decision tree technique called the Iterative Dichotomiser-3 algorithm which leads to significant conclusions. Conglomeration of data mining techniques and medical research can lead to life saving conclusions useful for the physicians. Keywords: BMI, Diabetes, decision tree, logistic regression, plasma. A Monthly Double-Blind Peer Reviewed Refereed Open Access International e-Journal - Included in the International Serial Directories International Journal in IT and Engineering http://www.ijmr.net.in email id- [email protected] Page 10 IJITE Vol.03 Issue-03, (March, 2015) ISSN: 2321-1776 Impact Factor- 3.570 ID3 ALGORITHM IN DATA MINING APPLIED TO DIABETES DATABASE Introduction Diabetes Mellitus is a major killer disease of mankind today.
    [Show full text]
  • Decision Tree Learning an Implementation and Improvement of the ID3 Algorithm
    Decision Tree Learning An implementation and improvement of the ID3 algorithm. CS 695 - Final Report Presented to The College of Graduate and Professional Studies Department of Computer Science Indiana State University Terre Haute, Indiana In Partial Fullfilment of the Requirements for the Degree Master of Science in Computer Science By Rahul Kumar Dass May 2017 Keywords: Machine Learning, Decision Tree Learning, Supervised Learning, ID3, Entropy, Information Gain. Decision Tree Learning An implementation and improvement of the ID3 algorithm. Rahul Kumar Dass CS 695 - supervised by Professor L´aszl´oEgri. Abstract The driving force behind the evolution of computing from automating automation [3] to automating learning [8] can be attributed to Machine Learning algorithms. By being able to generalize from examples, today's computers have the ability to perform autonomously and take critical decisions in almost any given task. In this report, we briefly discuss some of the foundations of what makes Machine Learning so powerful. Within the realm of supervised learning, we explore one of the most widely used classification technique known as Decision Tree Learning. We develop and analyze the ID3 algorithm, in particular we demonstrate how concepts such as Shannon's Entropy and Information Gain enables this form of learning to yield such powerful results. Furthermore, we introduce avenues through which the ID3 algorithm can be improved such as Gain ratio and the Random Forest Approach [4]. 2 Acknowledgements There are many people that I would like to express my sincerest gratitude, without which this research course would not have been completed. Firstly, I would like start by thanking my research advisor Professor L´aszl´oEgri for his support, guidance and giving me the opportunity to study Machine Learning without any prior education or experience in the field.
    [Show full text]
  • Machine Learning: Decision Trees
    What is learning? • “Learning denotes changes in a system that ... MachineMachine Learning:Learning: enable a system to do the same task more efficiently the next time.” –Herbert Simon • “Learning is constructing or modifying DecisionDecision TreesTrees representations of what is being experienced.” Chapter 18.1-18.3 –Ryszard Michalski • “Learning is making useful changes in our minds.” –Marvin Minsky Some material adopted from notes by Chuck Dyer 1 2 Why study learning? A general model of learning agents • Understand and improve efficiency of human learning – Use to improve methods for teaching and tutoring people (e.g., better computer-aided instruction) • Discover new things or structure previously unknown – Examples: data mining, scientific discovery • Fill in skeletal or incomplete specifications about a domain – Large, complex AI systems can’t be completely derived by hand and require dynamic updating to incorporate new information. – Learning new characteristics expands the domain or expertise and lessens the “brittleness” of the system • Build agents that can adapt to users, other agents, and their environment 3 4 1 Major paradigms of machine learning The inductive learning problem • Rote learning – One-to-one mapping from inputs to stored • Extrapolate from a given set of examples to make accurate representation. “Learning by memorization.” Association-based predictions about future examples storage and retrieval. • Supervised versus unsupervised learning • Induction – Use specific examples to reach general conclusions – Learn an unknown function f(X) = Y, where X is an input example and Y is the desired output. • Clustering – Unsupervised identification of natural groups in data – Supervised learning implies we are given a training set of (X, Y) • Analogy – Determine correspondence between two different pairs by a “teacher” – Unsupervised learning means we are only given the Xs and some representations (ultimate) feedback function on our performance.
    [Show full text]
  • Course 395: Machine Learning – Lectures
    Course 395: Machine Learning – Lectures • Lecture 1-2: Concept Learning (M. Pantic) • Lecture 3-4: Decision Trees & CBC Intro (M. Pantic & S. Petridis) • Lecture 5-6: Evaluating Hypotheses (S. Petridis) • Lecture 7-8: Artificial Neural Networks I (S. Petridis) • Lecture 9-10: Artificial Neural Networks II (S. Petridis) • Lecture 11-12: Instance Based Learning (M. Pantic) • Lecture 13-14: Genetic Algorithms (M. Pantic) Maja Pantic Machine Learning (course 395) Decision Trees – Lecture Overview • Problem Representation using a Decision Tree • ID3 algorithm • The problem of overfitting Maja Pantic Machine Learning (course 395) Problem Representation using a Decision Tree • Decision Tree learning is a method for approximating discrete classification functions by means of a tree-based representation • A learned Decision Tree classifies a new instance by sorting it down the tree – tree node ↔ classification OR test of a specific attribute of the instance – tree branch ↔ possible value for the attribute in question • Concept: Good Car size large small ‹size = small, brand = Ferari, mid model = Enzo, sport = yes, brand no sport engine = V12, colour = red› yes Volvo BMW SUV engine no no yes no F12 V12 V8 no yes no Maja Pantic Machine Learning (course 395) Problem Representation using a Decision Tree • A learned Decision Tree can be represented as a set of if-then rules • To ‘read out’ the rules from a learned Decision Tree – tree ↔ disjunction () of sub-trees – sub-tree ↔ conjunction () of constraints on the attribute values • Rule: Good Car size IF (size = large AND brand = BMW) large small OR (size = small AND sport = yes mid AND engine = V12) brand no sport THEN Good Car = yes yes ELSE Good Car = no; Volvo BMW SUV engine no no yes no F12 V12 V8 no yes no Maja Pantic Machine Learning (course 395) Decision Tree Learning Algorithm • Decision Tree learning algorithms employ top-down greedy search through the space of possible solutions.
    [Show full text]
  • Decision Tree Classification and Forecasting of Pricing Time Series
    Decision Tree Classification and Forecasting of Pricing Time Series Data EMIL LUNDKVIST Master’s Degree Project Stockholm, Sweden July 2014 TRITA-XR-EE-RT 2014:017 Abstract Many companies today, in different fields of operations and sizes, have access to a vast amount of data which was not available only a couple of years ago. This situation gives rise to questions regarding how to organize and use the data in the best way possible. In this thesis a large database of pricing data for products within various market segments is analysed. The pricing data is from both external and internal sources and is therefore confidential. Because of the confidentiality, the labels from the database are in this thesis substituted with generic ones and the company is not referred to by name, but the analysis is carried out on the real data set. The data is from the beginning unstructured and difficult to overlook. Therefore, it is first classified. This is performed by feeding some manual training data into an algorithm which builds a decision tree. The decision tree is used to divide the rest of the products in the database into classes. Then, for each class, a multivariate time series model is built and each product’s future price within the class can be predicted. In order to interact with the classification and price prediction, a front end is also developed. The results show that the classification algorithm both is fast enough to operate in real time and performs well. The time series analysis shows that it is possible to use the information within each class to do predictions, and a simple vector autoregressive model used to perform it shows good predictive results.
    [Show full text]
  • Network Classification and Categorization Paper
    Network Classification and Categorization Paper James P. Canning, Emma E. Ingram, Adriana M. Ortiz and Karl R. B. Schmitt June 2017 Abstract Networks are often labeled according to the underlying phenomena that they represent, such as re- tweets, protein interactions, or web page links. Our research seeks to determine if we can use machine learning techniques to gain a better understanding of the categories of networks on the Network Repos- itory (www.networkrepository.com) and then classify the unlabeled networks into categories that make sense. It is generally believed that networks from different categories have inherently unique network characteristics. Our research provides conclusive evidence to validate this belief by presenting the results of global network clustering and classification into common categories using machine learning algorithms. The machine learning techniques of Decisions Trees, Random Forests, Linear Support Vector Classifica- tion and Gaussian Na¨ıve Bayes were applied to a 14-feature `identifying vector' for each graph. During cross-validation, the best technique, Gaussian Na¨ıve Bayes, achieved an accuracy of 92.8%. After train- ing the machine learning algorithm it was applied to a collection of initially unlabeled graphs from the Network Repository (www.networkrepository.com). Results were then manually checked by determining (when possible) original sources for these graphs. We conclude by examining the accuracy of our results and discussing how future researchers can make use of this process. 1 Introduction Network science is a field which seeks to understand the world by representing interactions and relationships between objects as graphs. By organizing data into networks, we can better understand how objects of interest are connected.
    [Show full text]
  • An Improved ID3 Decision Tree Algorithm Based on Attribute Weighted
    International Conference on Civil, Materials and Environmental Sciences (CMES 2015) An Improved ID3 Decision Tree Algorithm Based on Attribute Weighted 1 1 1 Xian Liang , Fuheng Qu , Yong Yang Hua Cai2 1 College of Computer Science and Technology, 2 College of electronic and information engineering, Changchun University of Science and Technology, Changchun University of Science and Technology, Changchun 130022, China Changchun 130022, China Abstract—ID3 decision tree algorithm uses information gain properties C has m different values{CCC , ,..., },the selection splitting attribute tend to choose the more property 1 2 m values, and the number of attribute values can not be used to data set of S is divided into m subsets Si （i=1,2...m）， measure the attribute importance, in view of the above description attribute A has v different values problems, a new method is proposed for attribute weighting, {AAA , ,..., } the idea of simulation conditional probability, calculation of 1 2 v ,description attribute A set S is partitioned the close contact between the attributes and the decision into v subsets S j （j=1,2...v）, the connection between the attributes, as the attribute weights and combination with attribute information gain to selection splitting attribute, attribute A and classification attribute C is defined as v improve the accuracy of decision results. Experiments |S j | show that compared with the improved algorithm and the FA = ∑ .Wj ,The number of samples about the traditional ID3 algorithm, decision tree model has higher j=1 |S | predictive accuracy, less number of leaves. value of attribute A is A j is|S j | ,The total number of Keywords-decision tree; ID3 algorithm; information gain samples is|S | ,the sum of probability of classification I.
    [Show full text]
  • A Convenient and Low-Cost Model of Depression Screening and Early Warning Based on Voice Data Using for Public Mental Health
    International Journal of Environmental Research and Public Health Article A Convenient and Low-Cost Model of Depression Screening and Early Warning Based on Voice Data Using for Public Mental Health Xin Chen 1,2,3 and Zhigeng Pan 1,2,3,* 1 School of Medicine, Hangzhou Normal University, Hangzhou 311121, China; [email protected] 2 Engineering Research Center of Mobile Health Management System, Ministry of Education, Hangzhou Normal University, Hangzhou 311121, China 3 Institute of VR and Intelligent System, Hangzhou Normal University, Hangzhou 311121, China * Correspondence: [email protected] Abstract: Depression is a common mental health disease, which has great harm to public health. At present, the diagnosis of depression mainly depends on the interviews between doctors and patients, which is subjective, slow and expensive. Voice data are a kind of data that are easy to obtain and have the advantage of low cost. It has been proved that it can be used in the diagnosis of depression. The voice data used for modeling in this study adopted the authoritative public data set, which had passed the ethical review. The features of voice data were extracted by Python programming, and the voice features were stored in the format of CSV files. Through data processing, a big database, containing 1479 voice feature samples, was generated for modeling. Then, the decision tree screening model of depression was established by 10-fold cross validation and algorithm selection. The experiment achieved 83.4% prediction accuracy on voice data set. According to the prediction Citation: Chen, X.; Pan, Z. A results of the model, the patients can be given early warning and intervention in time, so as to realize Convenient and Low-Cost Model of the health management of personal depression.
    [Show full text]
  • Combining Machine Learning and Social Network Analysis to Reveal the Organizational Structures
    applied sciences Article Combining Machine Learning and Social Network Analysis to Reveal the Organizational Structures Mateusz Nurek and Radosław Michalski * Department of Computational Intelligence, Faculty of Computer Science and Management, Wrocław University of Science and Technology, 50-370 Wrocław, Poland; [email protected] * Correspondence: [email protected] Received: 7 January 2020; Accepted: 21 February 2020; Published: 2 March 2020 Abstract: Formation of a hierarchy within an organization is a natural way of assigning the duties, delegating responsibilities and optimizing the flow of information. Only for the smallest companies the lack of the hierarchy, that is, a flat one, is possible. Yet, if they grow, the introduction of a hierarchy is inevitable. Most often, its existence results in different nature of the tasks and duties of its members located at various organizational levels or in distant parts of it. On the other hand, employees often send dozens of emails each day, and by doing so, and also by being engaged in other activities, they naturally form an informal social network where nodes are individuals and edges are the actions linking them. At first, such a social network seems distinct from the organizational one. However, the analysis of this network may lead to reproducing the organizational hierarchy of companies. This is due to the fact that that people holding a similar position in the hierarchy possibly share also a similar way of behaving and communicating attributed to their role. The key concept of this work is to evaluate how well social network measures when combined with other features gained from the feature engineering align with the classification of the members of organizational social network.
    [Show full text]
  • Surrogate Decision Tree Visualization Interpreting and Visualizing Black-Box Classification Models with Surrogate Decision Tree
    Surrogate Decision Tree Visualization Interpreting and Visualizing Black-Box Classification Models with Surrogate Decision Tree Federica Di Castro Enrico Bertini Università di Roma La Sapienza New York University Tandon School of Engineering Rome, Italy Brooklyn, New York [email protected] [email protected] Figure 1: Understanding the behavior of a black-box Machine Learning model using the explanatory visual interface of our proposed technique. Interacting with the user-interaction-panel (A) a user can choose a combination of model and dataset that they desire to analyze. The built model is a surrogate decision tree whose structure can be analyzed in the tree Panel (B) and the details of the leaves are featured in the Rule Panel (C). The user can interact with the tree, collapsing any node they see fit and automatically updating the performance Overview (D). ABSTRACT analysis of the level of fidelity the surrogate decision tree can reach With the growing interest towards the application of Machine with respect to the original model. Learning techniques to many application domains, the need for transparent and interpretable ML is getting stronger. Visualizations CCS CONCEPTS methods can help model developers understand and refine ML mod- • Human-centered computing → Dendrograms; User interface els by making the logic of a given model visible and interactive. toolkits; • Information systems → Data analytics. In this paper we describe a visual analytics tool we developed to ACM Reference Format: support developers and domain experts (with little to no expertise Federica Di Castro and Enrico Bertini. 2019. Surrogate Decision Tree Vi- in ML) in understanding the logic of a ML model without having sualization: Interpreting and Visualizing Black-Box Classification Models access to the internal structure of the model (i.e., a model-agnostic with Surrogate Decision Tree.
    [Show full text]
  • On the Optimization of Fuzzy Decision Trees
    Fuzzy Sets and Systems 112 (2000) 117}125 On the optimization of fuzzy decision trees Xizhao Wang!,*, Bin Chen", Guoliang Qian", Feng Ye" ! Department of Mathematics, Hebei University, Baoding 071002, Hebei, People+s Republic of China " Machine Learning Group, Department of Computer Science, Harbin Institute of Technology, Harbin 150001, People+s Republic of China Received June 1997; received in revised form November 1997 Abstract The induction of fuzzy decision trees is an important way of acquiring imprecise knowledge automatically. Fuzzy ID3 and its variants are popular and e$cient methods of making fuzzy decision trees from a group of training examples. This paper points out the inherent defect of the likes of Fuzzy ID3, presents two optimization principles of fuzzy decision trees, proves that the algorithm complexity of constructing a kind of minimum fuzzy decision tree is NP-hard, and gives a new algorithm which is applied to three practical problems. The experimental results show that, with regard to the size of trees and the classi"cation accuracy for unknown cases, the new algorithm is superior to the likes of Fuzzy ID3. ( 2000 Elsevier Science B.V. All rights reserved. Keywords: Machine learning; Learning from examples; Knowledge acquisition and learning; Fuzzy decision trees; Complexity of fuzzy algorithms; NP-hardness (k) (k) Notation pi , qi relative frequencies concerning some classes X a given "nite set [X"A] a branch of a decision tree i" X" D F(X) the family of all fuzzy subsets fj P( Ai Cj) the conditional frequency (k) de"ned on X Entri fuzzy entropy (k) A, B, C, P, N fuzzy subsets de"ned on X, i.e.
    [Show full text]
  • Selected Algorithms of Machine Learning from Examples
    Fundamenta Informaticae 18 (1993), 193–207 Selected Algorithms of Machine Learning from Examples Jerzy W. GRZYMALA-BUSSE Department of Computer Science, University of Kansas Lawrence, KS 66045, U. S. A. Abstract. This paper presents and compares two algorithms of machine learning from examples, ID3 and AQ, and one recent algorithm from the same class, called LEM2. All three algorithms are illustrated using the same example. Production rules induced by these algorithms from the well-known Small Soybean Database are presented. Finally, some advantages and disadvantages of these algorithms are shown. 1. Introduction This paper presents and compares two algorithms of machine learning, ID3 and AQ, and one recent algorithm, called LEM2. Among current trends in machine learning: similarity-based learning (also called empirical learning), explanation-based learning, computational learning theory, learning through genetic algorithms and learning in neural nets, only the first will be discussed in the paper. All three algorithms, ID3, AQ, and LEM2 represent learning from examples, an approach of similarity-based learning. In learning from examples the most common task is to learn production rules or decision trees. We will assume that in algorithms ID3, AQ and LEM2 input data are represented by examples, each example described by values of some attributes. Let U denote the set of examples. Any subset C of U may be considered as a concept to be learned. An example from C is called a positive example for C, an example from U – C is called a negative example for C. Customarily the concept C is represented by the same, unique value of a variable d, called a decision.
    [Show full text]