A Thesis Presented to

The Faculty of the

Fritz J. and Dolores H. Russ College of Engineering and Technology

Ohio University

In Partial Fulfillment

of the Requirement for the Degree

Master of Science

by

Brian 0.Bush

June 1996 0 1996

Brian 0. Bush

All Rights Reserved wasted views that's all they see blue hot blood guild optic nerve with the right attitude you will succeed blue seIfabusive recluse too late for me

-Skinny Puppy, Worlock, a never ending backdrop to this work

I would like to thank my mother and father, Pascal and especially Ryoko. ABSTRACT v

Natural language is perhaps the most powerful form of conveying infonnation for any

given problem or situation. Combining natural language and numerical infonnation into

fuzzy systems provide the framework to represent knowledge, constraints and inference procedures. Fuzzy systems are advantageous in the developtnent of systenls solutions that perform tasks such as automatic modeling, prediction, pattern recognition, and optimal decision making, control and planning. Thus, fuzzy systems are an essential tool for industrial and tnanufacturing systems engineering. Traditionally fuzzy systenl design is carried out by a time-consuming trial-and-error process, in order to incorporate expert knowledge into a fuzzy model. To date there is no systematic procedure to design fuzzy systems that has been met with wide-acceptance and usage. In this research, a fonnal methodology has been developed for the design and adaptation of effective fuzzy systems displaying both robustness and computational efficiency. The goal was to improve upon a functional fuzzy system by modifying the fuzzy sets of fuzzy rules using evolutionary computation principles. An experimental implementation was developed. The relative influence of selected factors such mutations and crossover schemes, mutations rates, and fuzzy rule structure upon the tnethodology's effectiveness within the experimental context were also addressed. Finally, the application of the methodology to function approximation, classification, regression, and the prediction of chaotic dynamics were accomplished to demonstrate the computational characteristics, complexity and inherent strength of the developed methodology. Acknowledgements ...... iv

Abstract ...... v

Table of Contents ...... vi

Listofsymbols ...... xii

ListofAcronyms ...... xiii

Chapter 1 Introduction ...... 1 1. I Fuzzy Systems ...... 1 1.2 Statement of Problem ...... 6 1.3Approach ...... 7 1.4 Outline of the Thesis ...... 9

Chapter 2 Background and Literature Review ...... 11 2.1 Background ...... 12 2.1.1 Fuzzy Algebraic Concepts ...... 13 2.1.2 Fuzzy Systems ...... 16 2.1.2.1 A Basic Fuzzy Control Algorithm ...... 19 2.1.2.2 The Fuzzifier ...... 20 2.1.2.3 The Fuzzy Inference Engine ...... 27 2.1.2.4 The Defuzzifier ...... 30 The Maximum Membership Method ...... 30 The Centroid Defuzzification Method ...... 32 The Weighted Average (Singleton) Method ...... 33 2.1.2.5 The Fuzzy Rule Base ...... 34 2.1.3 Evolutionary Computation ...... 36 2.1.3.1 A Simple Genetic Algorithm ...... 38 2.1.3.2 Genetic Operators ...... 38 Crossover ...... 39 Mutation ...... 41 2.1.3.3 Models of Evolution ...... 42 Parent Selection Techniques ...... 43 Offspring Insertion Techniques ...... 44 2.1.3.4 Underlying Theory of Genetic Algorithms ...... 45 2.2 Traditional Methods ...... 46 2.2.1 Straight Forward Brute-Force Approach ...... 46 2.2.2 The Wang and Mendel Methodology ...... 47 2.3 Soft-Computing Methods ...... 48 2.3.1 Artificial Neural Network Fuzzy Design Methods ...... 48 2.3.2 Fuzzy System Synthesis and Optimization using Genetic Algorithms ...... 54 2.4Summary ...... 57

Chapter 3 A Fuzzy System Design Methodology ...... 58 3.1 Fuzzy System Design Elements ...... 58 3.1.1 A Variation on a Theme: The Wang and Mendel Method ..... 61 3.1.2 OPTIMIST: An Evolutionary Optimization Method ...... 65 3.1.2.1 Fuzzy System Representation ...... 65 3.1.2.2 Fuzzy System Qualitative Measurement ...... 68 3.1.2.3 Mutation Mechanism ...... 68 3.1.2.4 Crossover ...... 73 3.1.2.5 Model of Evolution ...... 75

Chapter 4 System Design and Implementation Specifics ...... 77 4.1 Fuzzy System Design ...... 78 4.2 Implementation ...... 80 4.2.1 Design ...... 80 4.2.1.1 Functions ...... 81 4.2.1.2 Steps in Using DESIGN ...... 83 4.2.2 OPTIMIST ...... 84 4.2.2.1 Functions ...... 84 4.2.2.2 Steps in Using OPTIMIST ...... 87 4.2.3 Prototype Characteristics ...... 87

Chapter 5 Empirical Results and Benchmark Results ...... 89 5.1 Function Approximation and Prediction ...... 90 5.1.1 Experiment Series One ...... 92 5.1.3 Experiment Series Three ...... 108 viii

5.2 Classification ...... 114 5.2.1 The Mechanics of Abductive Modeling ...... 115 5.2.2 Experiments Series One ...... 118 5.3 Prediction of Boston Housing Prices ...... 124 5.4 Predicting Chaotic Dynamics ...... 128

Chapter 6 Conclusions and Further Research ...... 136 6.1Conclusions ...... 137 6.2 Accomplishments ...... 139 6.3 Limitations and Problems ...... 139 6.3.1 Problems in general ...... 140 6.3.2 Problems with DESIGN ...... 140 6.3.3 Problems with OPTIMIST ...... 140 6.4 Future Research ...... 141

References ...... 143

Appendix A DESIGNSource ...... 154

Appendix B OPTIMISTSource ...... 166

Appendix C FUZZY SYSTEM ENGINE Source ...... 199

Appendix D Fifth-Degree Polynomial Data Generator Source ...... 207

Appendix E Fuzzy System Definition Format ...... 210 Figure 2.1 An abstract view of a fuzzy system ...... 18

Figure 2.2 A triangular fuzzy number (TFN) ...... 22

Figure 2.3 A crisp nutnber viewed as a TFN ...... 23

Figure 2.4 A trapezoidal fuzzy number (TpFN) ...... 24 Figure 2.5 The fuzzy set Maximum ...... 26

Figure 2.6 (a) Fuzzy reasoning using Mamdani's minimum operation rule as a fuzzy implication function. (b) resulting decision in output space MI ...... 29

Figure 2.7 The centroid defuzzification method ...... 32

Figure 2.8 The evolutionary computation cycle ...... 37

Figure 2.9 An exa~npleof single-point crossover ...... 40

Figure 2.10 An example of two-point crossover ...... 41

Figure 2.11 An example of single-point mutation ...... 42

Figure 2.12 Patches representing fuzzy rules ...... 52

Figure 3.1 Architecture of the DESIGN system ...... 59

Figure 3.2 Architecture of the OPTIMIST system ...... 60

Figure 3.3 The division of the input space x into fuzzy regions ...... 62

Figure 3.4 The division of the output space y into fuzzy regions ...... 63

Figure 3.5 Individual structure within population ...... 67

Figure 3.6 A trapezoidal fuzzy number ...... 69

Figure 3.7 A linear shift mutation ...... 70

Figure 3.8 An expansion mutation ...... 70 X Figure 3.9 A nonlinear shift mutation ...... 71

Figure 3.10 Mutation depiction in Kosko's representation ...... 73

Figure 3.11 Crossover in fuzzy system reproduction ...... 74

Figure 4.1 File input and output in DESIGN ...... 81

Figure 4.2 File input and output in OPTIMIST ...... 84

Figure 5.1 The fifth-degree polynomial ...... 90

Figure 5.2 This screen capture of a fuzzy system definition ...... 94

Figure 5.3 Scattergram of training data ...... 96

Figure 5.4 Actual vs . predicted performance over [-2 ..2], 0.1 increment ...... 100

Figure 5.5 Scattergram of reduced training set ...... 101

Figure 5.6 Linear mutation operator over 1000 generations ...... 105

Figure 5.7 Expansion mutation operator over 1000 generations ...... 106

Figure 5.8 Contraction mutation operator over 1000 generations ...... 106

Figure 5.9 Nonlinear mutation operator over 1000 generations ...... 107

Figure 5.10 Dickerson-Kosko method of function approximation using six rules (with Permission) ...... 111

Figure 5.1 1 An abductive network ...... 115

Figure 5.13 A section of the Mackey-Glass chaotic time series ...... 129

Figure 5.14 Initial fuzzy system's absolute error ...... 132

Figure 5.15 Initial fizzy system's prediction performance ...... 132

Figure 5.16 Improved fuzzy system's absolute error ...... 134 xl

Figure 5.17 Improved fuzzy system's prediction performance ...... 134 Complement Intersection operator Union operator Logical AND Logical OR Member of a set Fuzzy membership function Membership or truth function in fuzzy set Element from domain of fuzzy set Summation Probability of event x occurring A fuzzy set definition (Trapezoidal Fuzzy Number) Empty or null set ANSI American National Standards Institute ASCII American Standard Code for Information Interchange AVQ Adaptive Vector Quantitizer CI Computational Intelligence EC Evolutionary Computation EPROM Erasable Programmable Read-Only Memory FAM Fuzzy Associative Memory FAT Fuzzy Approximation Theorem FIX Fuzzy Logic Controller GA Genetic Algorithm MSE Mean Squared Error NN Neural Network PR Probabilistic Reasoning RMS Root Mean Squared SC Soft Computing TFN Triangular Fuzzy Number TpFN Trapezoidal Fuzzy Number I . I Fuzzy SYSTEMS

In recent years, industrial and manufacturing systems engineers have become

interested in the use of Computational Intelligence (CI) based models to

describe the dynamic laws of the systems they study in specific when they

are ill-defined. CI-based models may serve two general purposes. First, they

may be used as tools in linguistic reasoning. Second, CI-based models may be

used to make predictions based on numerical and symbolic data, and

knowledge that cannot be acquired through sensors. For example, industrial

engineers have long been predicting the product demand and inventory levels

from the use of equations based on regression and moving average techniques,

rule-based expert systems, neural networks, and fuzzy systems. These 2 approaches offer different predictions, depending on the assumptions made about machine production rates, worker productivity and quality control levels. Computational Intelligence based models include fuzzy systems

(Zadeh 1965), neural networks (Werbos 1994) and genetic algorithms (Holland

1975). This thesis shall address fuzzy system design.

Fuzzy systems have been successfully applied as a paradigm, demonstrating the use of linguistic information in engineering systems. Industrial and manufacturing engineering are involved in production monitoring and improvements through such activities as forecasting and time-series analysis.

Fuzzy systems could be utilized as a problem solving paradigm to handle the nonlinearities beyond current techniques. In production control and scheduling, fuzzy systems are already a topic of research and application.

Applications include fuzzy constraint relaxation in scheduling (Slany 1993), fuzzy approach to dispatching rules in scheduling (Grabot et al. 1993), fuzzy family setup and machine balancing (Krucky 1994) and flow shop scheduling using fuzzy numbers (Tsujimura et al. 1993). Other aspects of industrial and manufacturing engineering, such as facility layout (Gen et al. 1995), procurement of facilities (Kashiwagi et al. 1995), quality control function deployment (Masud et al. 1993) and transportation engineering (Gen et al.

1995) have benefited from the usage of fuzzy systems. 3

Recent control applications include fuzzy memory devices (Togai et al.

1986), (Togai et al. 1987), (Watanabe 1988), (Yamakawa et al. 1986),

(Yamakawa 1987), and (Yamakawa et al. 1987); fuzzy computers (Yamakawa

1987); power systems and nuclear control (Bernard 1988), (Kinoshita et al.

1988); automobile speed control (Murakami 1985); automobile transmission control (Kasai et al. 1988); water purification process control (Yahishita et al.

1985); elevator control (Fujitec 1988); model car parking (Sugeno et al. 1985); robot control (Ciliz et al. 1987), (Isik 1987), (Scharf et al. 1985), (Tansheit et

al. 1988), and (Uragami et al. 1976); turning process (Sakai 1985); aircraft

flight control (Larking 1985); cement kiln (Larsen 1980), (Umers et al. 1980); heat exchanger (Ostergaad 1977); warm water process (Kickert et al. 1976);

activated sludge processing (Itohm et al. 1987), (Tong et al. 1980); and traffic

light control (Pappis et al. 1977). This abbreviated list of successful

applications shows an effective usage of fuzzy logic to control complex, non-

linear, and possibly ill-defined processes.

Fuzzy system design requires general (operational) knowledge of the process

to be controlled. Typically, the initial membership functions are chosen in a

trial and error fashion. These membership functions later require readjustment

(or "tuning") in order to achieve a desirable level of control. The advantage of

fuzzy systems is that linguistic fuzzy IF-THEN rules can often be obtained 4 from human experts who are familiar with the system under consideration.

However, the development of heuristics (rules of thumb) for fuzzy systems remains a difficult task as the system size increases (number of system variables).

Most realistic control issues have multiple inputs and outputs, where the relationship between inputs and outputs may not be known. There may also be more than one human operator currently controlling the system, where no one expert exists. It may be impossible to determine a complete set of membership functions and rules. Thus, determining the fuzzy rules along with the membership functions to properly describe a system's behavior is the major difficulty in fuzzy system design. Zadeh pointed out that probably the most critical issue in the design of a fuzzy system is membership calibration

(Zadeh 1978). This has led to many consequences, (1) has limited the widespread usage of fuzzy logic (mainly to small systems and problems), (2) the cost of fuzzy system development is high due to the non-systematic development cycle, and (3) on-line fuzzy logic systems have not been well- developed due to a rigid structure and lack of a formal redesign process.

Many attempts have been made to develop a formal systematic procedure for fuzzy system design. Several methods for combining numerical and linguistic 5 information in the design of fuzzy systems (Wang 1992) have also been formulated. One computationally simple method devised performs a single- pass through the sample data from the system in consideration to generate an

entire fuzzy system. This method provided a framework that allowed

linguistic rules to be later added to the fuzzy system that was generated. On

the other hand, genetic algorithms (GAS), a class of evolutionary computation

algorithms, have been used to adapt fuzzy logic controllers from sample data both off-line and on-line. Designing a fuzzy system requires general knowledge of the process under consideration and also knowledge of the effect of the membership functions on the performance of the fuzzy system. During the initial phase of fuzzy system design, the membership functions are most likely to be chosen in a trial-and-error fashion. This later requires tuning of the membership functions to achieve the proper desired behavior of the fuzzy system. At present there is no systematic procedure for the design of a fuzzy system. Current automated fuzzy system design methods do not exhibit robustness and

efficiency, thus hindering widespread usage and acceptance of such techniques. The purpose of this research work is to integrate previous and current concepts in the area of learning by examples and evolutionary computation, in order to develop a framework for the improved design of fuzzy systems. The overall goal of this research is to find a near-optimal answer to fuzzy system design. Finally, this research shows how this methodology may be applied to pattern recognition, function approximation, automatic modeling, and control.

First, a modification upon the Wang and Mendel method is used to generate an initial "rough" fuzzy system from sample data derived from the system while

"in control". The generated fuzzy system is operational, yet is not optimized.

Thus, in order to improve the fuzzy system, evolutionary computation is utilized to evolve a population of fuzzy systems toward a better solution.

Second, evolutionary computation (specifically genetic algorithms) improves upon the initial solution derived from the modified Wang and Mendel method using genetic operators specifically designed to operate on fuzzy sets. The improved fuzzy system will possess the same structure and number of fuzzy rules and therefore, we are not changing the degrees of freedom. The

improvements made will be confined to the descriptions of fuzzy sets only. 8

Goldberg defines robustness (Goldberg 1989) as the balance between efficiency and efficacy for survival in many different environments. The implication of robustness within the fuzzy system design methodology is twofold: (1) we are starting with an initial solution, (2) we are evolving that solution, thus we are guaranteed to converge to a solution. An improvement over other GA based techniques is that the solution space is reduced since we are using an excellent initial guess. Holland has stated that genetic algorithms are at their best in complex situations where improvements, not global optima, are the objective (Holland 1994). Thus, the focus shall be on fuzzy system improvement, not optimization.

The benefits of this research work includes four areas: (1) alleviate concerns about fuzzy system design for dynamic and static systems, (2) the construction of a robust method based on evolutionary computation and learning by examples for fuzzy system design, (3) the development of this

system using an experimental prototype, and (4) benchmark of the methodology to various problem solving areas. Moreover, this research work

continues the investigation of fuzzy system design that has only recently begun. 9

1 .4 OUTLINE OF THE THESIS

This thesis implements a fuzzy system design methodology composed of two independent techniques. The techniques are used in tandem, but are not limited to this framework.

In Chapter 2, we begin with a brief introduction to the concepts involved in this research. Then we look at the previous work in fuzzy system design identifying their weaknesses and strong points. Both traditional and soft computing approaches are discussed.

In Chapter 3, we describe an integrated approach that combines a simple one pass operation on the sample data with a genetic algorithm to arrive at an improved fuzzy system. An operational view of the methodology is also presented.

Chapter 4 covers the issues in software design and implementation of the proposed method on a Power Macintosh.

In Chapter 5, we tackle five different problems: function approximation (in

relation to control), classification, regression and prediction of chaotic

dynamics are described and modeled using the proposed methodology, with an analysis of results and comparison to other techniques.

Chapter 6 concludes the thesis and provides some future directions. AND

This chapter performs two functions: it develops on the topics in the introduction and provides a review of the relevant literature in fuzzy system design. In Chapters 3,4, and 5, discussions of previous research will be more problem-specific. The first section will describe in more detail fuzzy set theory and evolutionary computation. The second section reviews current developments in the area of fuzzy system design. In 1965, Lotfi A. Zadeh set forth the concept and mechanics of fuzzy sets

(Zadeh 1965), laying the foundations of fuzzy set theory, also known as

"fuzzy logic" (FL). Fuzzy logic is a generalization of traditional set theory.

In contrast to classical knowledge based systems, FL is aimed at a formalization of models of approximate reasoning, which operate on imprecise information. Essentially, fuzzy logic controllers (FLCs) are algorithms which convert a linguistic control strategy, based on expert knowledge (and experience), into an automatic control strategy.

In traditional set theory, an item is either a member of a set or not. There is no partial membership to a set. Consider a control system, with the system input x in the range of [O, 1001 which represents the temperature range of a room. This control system will control a heating element that has two possible states: on and 08 The heating element is on when the temperature falls into the cold set, which is defined as [0,68) and switched off when the temperature falls into the hot set, which is defined as (68, 1001. Note that the setpoint for this control system is set at 68 degrees, which defines the breaking point between the two sets: cold and hot. The controller's operation could then be defined by the following rules:

Rule I: If temperature is cold then heat is to be turned on. Rule 2: If temperature is hot then heat is to be turned off

Here the two defined sets: cold and hot are known as crisp sets.

Fuzzy logic on the other hand allows partial set membership; allowing gradual transitions between full membership and no membership. Let us now briefly cover fuzzy set theory.

2. 1 . I Fuzzy ALGEBRAICCONCEPTS

Fuzzy sets are a superset of traditional sets, evident when a traditional theory

Xis fuzzified and hence generalized to a fuzzy theory X. This can be seen in fields such as fuzzy arithmetic, fuzzy algebra and fuzzy probability, which have arisen from arithmetic, modern algebra and probability. This discussion of fuzzy set theory brief in comparison to the entire field of fuzzy set theory, providing only a foundation for the techniques and algorithms presented later.

If X is a collection of objects x, then a fuzzy set A c X is the set of ordered pairs: The entity lA(x) is the membership function, the value of which is the grade of membership of x in A. It is also the degree to which the deterministic measurement x is compatible with (the vague concept of) A. X is called the universe of discourse and all the elements of X for which A is nonzero comprise the support of A:

If the support of a fuzzy set is a single point, it is called a fuzzy singleton.

A fuzzy singleton is basically a fuzzy representation of a number (also referred to as a scalar). Now let us look at an example of fuzzification.

Example 2.1: Suppose U = [0,255], with x the system input and x E U. The two fuzzy sets: low and high defined by their membership functions as follows:

low(x) = 1 - (x/255), and high(x) = (~1255)

Note that in this example low(x) + high(x) = Unity (with Unity being 1.0), which is not always necessary. The truth value of the following statements shall now be evaluated:

(a) 5 is low. The membership of 5 to the set low is plOw(5)= 1 - (5/255), which approximately equals 0.980. Since the degree of membership of 5 to the fizzy set low is almost one, the element 5 has almost complete membership to the set low. Thus the statement is true to a degree of 0.980. In contrast the statement: 5 is high is true to a degree of 0.02, which was easily calculated due to the property: low(x) + high(x) = 1, so high(x) = 1 - low(x).

(b) 127.5 is low. The membership of 127.5 to the set low is plOw(127.5)= 1 - (127.5/255), which equals 0.5. Since the degree of membership of 127.5 to the fuzzy set low is 0.5, the element 127.5 has partial membership to the set low and is a crossover point of the set low. Thus the statement is true to a degree of 0.5. The statement: 127.5 is high is also true to a degree of 0.5.

(c) 255 is low. The membership of 255 to the set low is p1,,(255) = 1 - (255/255), which equals 0.0. Since the degree of membership of 255 to the fuzzy set low is zero, the element 255 has no membership to the set low. Thus the statement is true to a degree of 0.0. In contrast the statement: 255 is high is true to a degree of 1, therefore having complete membership into the fuzzy set high.

Very much like traditional set theory, there are specifically defined operations for combining and modifying fuzzy sets. The fundamental methods by which fuzzy sets are combined define the mechanics and limits of fuzzy logic. Basic fuzzy logic operations that were initially defined by Zadeh, shall be used within this work. For a detailed explanation of alternative set theoretic

operations and their justifications, see (Cox 1994).

Let A and B be two fuzzy sets in X. Then, the intersection A n B is a

fuzzy set in X with membership defined as follows:

PAn B(X)= ~~~{CLA(~),CLB(~)

Example 2.2: The statement-The capacitor is charged (with a truth value of 0.60) AND the circuit is draining (with a truth value of 0.25)-has the truth value of 0.25.

The union A u B is a fuzzy set in X with membership defined by: Example 2.3: The statement-The capacitor is charged (with a truth value of 0.75) OR the circuit is draining (with a truth value of 0.20)-has the truth value of 0.75.

The complement -A is a fuzzy set in X with membership defined as:

Example 2.4: The statement- The the capacitor is charged (with a truth value of 0.35) The complement of that statement, The capacitor is NOT charged, thus has a truth value of 0.65.

2. 1 .2 Fuzzy SYSTEMS

Systems are traditionally represented by mathematical models based on complete, consistent and exact information. However, the real-world is plagued with noise yielding partial, inconsistent and fuzzy information.

Human reasoning differs in two major aspects; first, we associate a truth value

(or degree of belief) with all information we perceive and, second, we easily

deal with inconsistent and conflicting evidence.

The concept of a fuzzy system was introduced by Zadeh in (Zadeh 1965) to

handle complex system design. Zadeh stated the principle of incompatibility

(Zadeh 1973) as being:

As the complexity of a system increases, our ability to make precise and signzficant statements about its behavior diminishes until a threshold is reached beyond which precision and signzficance (or relevance) become almost mutually exclusive characteristics... A corollary principle may be stated succinctly as, "The closer one looks at a real-worldproblem, the

fuzzier becomes its solution. "

The main area in fuzzy systems research has focused on a linguistic approach to fuzzy systems (Zadeh 1973), in which a fuzzy system idcomposed of a series of fuzzy IF-THEN rules and a fuzzy inference engine which is based upon techniques from approximate reasoning. See Figure 2.1 for an abstract view of a fuzzy system architecture. These techniques were employed to synthesize a fuzzy logic controller (Mamdani 1975), which have been applied to many practical problems as noted earlier in the introduction (Chapter 1).

This approach to fuzzy systems will be the approach taken in this thesis.

Fuzzy systems represent process control as a fuzzy relation between b~ Fuzzifier ~ 1 measured values 4 fuzzy sets in U I Rule Evaluation b ~ controlled

Fuzzy I fuzzy sets in V System I Definition I e /' I controller - Defuzzifier I output I I I

-I - -

Fuzzy 1 Crisp

Figure 2.1 An abstract view of a fuzzy system

information about the process condition, the system input {xl, x2, ...,$I, and

the system output (input for the process), y. The control algorithm is usually

given in IF-THEN expressions, called fuzzy control rules. An example of a

two input, one output control rule is:

If xl is small AND x2 is medium, then y is large.

The ifclause is the antecedent and the then clause is the consequent. The

variables, {xl,x2, ...,x,}, are called the input and y is the output. The values

small, medium, and large are fuzzy values (or fuzzy variables), which are

expressed by fizzy sets. An example of a one input, one output control rule 19 that demonstrates the embedded "expert knowledge" of the process to be

controlled is:

If boiler pressure is low, then open valve a little.

With the terms low and little representing hzzy sets. A fuzzy controller is

constructed using groups of these fuzzy control rules. The actual process of

calculating the output, according to the system input is accomplished by

means of a fuzzy inference engine.

What is of central importance here, is that the vague descriptions of the

antecedent clauses eliminates the need for a precise match with the input

pattern. In other words, one is able to produce a decision with no specific

match to the input. The output happens to be the combination of many

difference pieces of information derived from the input acting on the

antecedents of all the control rules.

A simple FLC algorithm could follow these steps:

Step 1: Preprocess inputs; This would involve tasks such as converting encoder pulses to distance of travel.

Step 2: Fuzzification; Apply system input(s) to the membership hnctions to determine the degree of membership to each fuzzy set.

Step 3: For each rule, find rule antecedent with minimum truth value. Step 4: Apply min truth value to rule consequents; when more than one rule affects the same fuzzy output, use the larger (MAXimum).

Step 5: Defuzzification; Compute the center-of-gravity of all fuzzy outputs for each system output.

Step 6: Post-processing outputs; This would involve tasks such as converting an output level to a series of pulses to drive a servo.

The process of fuzzification, through the fuzzifier, applies the system input(s) to the membership functions to determine the degree of membership of the system input to each fuzzy set. In the context of a FLC, Lee states that the fuzzifier performs the following functions (Lee 1992):

measures the values of input variables, performs a scale mapping that transfers the range of values of input variables into corresponding universes of discourse, performs the function of fuzzification that converts input data into suitable linguistic values which may be viewed as labels of fuzzy sets.

The simplest process of fuzzification applies a single system input, x, with x

E U, to a fuzzy set, F, and determines the truth value of the statement:

The fuzzy subset F, can be described in many ways. In designing our fuzzy sets, we want a function that describes the relationship of any x in U to the fuzzy set F. The choice of formula depends on the required accuracy, the 2 1 sensitivity of the system, and ease of implementation and manipulation.

We shall now present a basic method of forming fuzzy sets using fuzzy numbers. The concept of a fuzzy number is based on a natural and very simple idea. A fuzzy number can be considered an extension of the concept of the interval of confidence. Instead of considering the interval of confidence at only two unique values (0 and I), it is considered at several levels, fiom zero to one (or Unity). An example of a fuzzy number is about 5 and have the fuzzy set ((1, 0.2), (2, 0.4), (3, 0.6), (4, 0.8), (5, 1.0), (6, 0.8), (7, 0.6), (8,

0.4), (9,0.2)). One can view a fuzzy number, aboutf, as a triangle on U centered on the numberf, which has complete membership in the fuzzy set.

This important property called normality and defined in (Kaufinann 1985), should be a key attribute of a fuzzy number.

Definition 2.1: A fuzzy subset F E U is normal if and only if for all x E U, there is some x that is a complete member of the fuzzy set F, that is PF(X)=1 -

The property of normality means that the defined fuzzy set is absolutely true at some point in the system range, but this maximum may or may not be unique. Now let us formally define the concept of a Triangular Fuzzy

Number (TFN). Definition 2.2: Let F be a fuzzy subset of U and x E U. Given three points a, b, c in U with a I b I c and points a and c symmetric about b, that is (b - a) = (c - b), the degree to which x is a member of F is defined by the following linear piecewise function:

k(x) = 0 if x I a, = (x - a)l(b - a) ifa IxI b, =(c-x)/(c-b)ifb5x

This normal fuzzy set, F, describes the Triangular Fuzzy Number b.

1.0 r TFN defined by (a,b,c)

Truth Value

I

a b c Universe of Discourse Figure 2.2 A triangular fuzzy number (TFN)

A typical TFN in Figure 2.2, is displayed with its three defining points a, b

and c. All numbers can be viewed as a triangular fuzzy number, as in Figure

2.3 the crisp number b is depicted. Truth Value

0 0 ------+--- & b Universe of Discourse Figure 2.3 A crisp number viewed as a TFN

The concept of a fuzzy number can be extended to an interval, a Trapezoidal

Fuzzy Number (TpPN).

Definition 2.3: Let F be a fuzzy subset of U and x E U. Given four points a, b, c, d in U with a I b I c I d and intervals [a, b] and [c, d] symmetric about the interval [b, c], that is (b - a) = (d - c), the degree to which x is a member of F is defined by the following linear piecewise function:

pF(x)= 0 if x I a, =(x-a)l(b-a)ifaIxIb, =lifb

This normal fuzzy set, F, describes the Trapezoidal Fuzzy Number b to C.

A TpFN is displayed in Figure 2.4, note that this is a strict interpretation of

definition 2.3. TpFN defined by (a,b,c,d)

Universe of Discourse Figure 2.4 A trapezoidal fuzzy number (TpFN)

In this thesis, fuzzy sets will be represented as a Trapezoidal Fuzzy Numbers

(TpFN), which are a superset of Triangular Fuzzy Numbers (TFNs). By

definition 2.3, a TpFN is represented by four points (a,b,c,d). Thus, a TFN

can be represented as a TpFN with characteristic points (a,b,b,c). Now let us

look at an example that uses the fuzzy number concepts to model a real world

system, but slightly altered to fit the constraints of the system.

In order to properly establish our fuzzy sets we must have knowledge of the

system that we wish to control. The following are steps to properly lay the

foundations of system operation, an integral part of designing fuzzy sets for

use in control systems:

Step 1: Identify the Universe of Discourse, that is, the range of the system input.

Step 2: Identify and label the full operational system inputs and their ranges. The labels should represent some external environmental state that describes what is happening, such as hot or moist. The process of capturing all of the specific system requirements can be done by referring to an expert or to an empirical study of the system to be controlled. All systems tend to fall into four operational regions as described by Brubaker (1993): a) Normal - A system resides in the normal operating region most of the time. The normal operating region is centered about a currently commanded operating point, and includes the response to most minor perturbations due to noise. b) Extended - A system enters the extended operating region far less frequently than its normal mode, but sufficiently often that its operation must be accounted for. An example of an extended operational region is a controlled power-down. c) Abnormal - Abnormal operating regions are rarely entered during a system's lifetime, but the region is highly critical for system operation. An example of an abnormal operational region is a series of sensors fail that are critical to system stability. d) Impossible - If a given set of inputs will never occur over the lifetime of a system, then it can be considered an impossible region. In some cases, these inputs should not be excluded in critical situations.

Step 3: Now by viewing the labeled sets of the system input we can visually generate our fuzzy sets. Imagine our system input space with the universe of discourse as the x-axis and the y-axis being the range [O,Unity], where Unity is defined to be 1.O. The labelled sets have specific ranges that an empirical study or expert has shown to be valid as observed values for each of the specified sets. These ranges imply that a system input that falls within the bounds of a specific set is said to be in that set with certainty. The uncertainty is where the bounds actually start and end, which will be solved by viewing the sets as fuzzy sets with varying degrees of possible membership. One should also keep in mind that the slope of the membership functions of our fuzzy sets affect the rate at which the system input moves from membership to non-membership. The four main bounding points that define a fuzzy set is:

where the membership may be said to begin, where membership is definite, i.e. complete membership, where membership is no longer definite, and where membership no longer exists. Now membership function definition should be in such a manner as to reflect the four bounding points.

Example 2.4: Now let us consider the modeling of flow in a network (consider the substance flowing to be water). Suppose we are concerned with measuring the maximum flow of a network, where the measured quantities are not crisp (that is known to be accurate), but fuzzy (in that the number is uncertain andlor inaccurate). We are only certain that the measured quantity will fall in an interval. But since we do not know the actual flow, the endpoints of the range of our certainty are fuzzy. The following graphical representation of a fuzzy set, labeled Maximum, might represent the maximum flow of a pipe: with the membership function,

P maximum ('1 = { =O,ifx 58; =(x-8)/2, if8sx5 10; =l if,x>lO

We know that the system input (the Universe of Discourse) is [O, 121 and that 12 is the maximum capacity, but from experience (from a human expert or operator) we know that the number could be as low as 10. But we are not certain of the lower bound, hence the slope. Note the number is crisp on the top end, which we are certain that 12 is the maximum.

8 10 12 Universe of Discourse 27

Figure 2.5 The fuzzy set Maximum

The above example integrates the basic concepts of a triangular fuzzy number

along with a fuzzy interval and a crisp boundary. This altered fuzzy set

strays far from the definitions of a triangular fuzzy number and a fuzzy

interval, but the ability to generalize upon our definitions give us an added

design flexibility.

The inference mechanisms found in FLC are generally much simpler than those

in traditional expert systems, since in an FLC the consequent of a rule is not

applied to the antecedent of another, i.e. there is no chaining inference

mechanism. The fuzzy inference engine is decision making logic which

employs fuzzy rules from the fuzzy rule base to determine fuzzy outputs of a

FLC corresponding to the fuzzified inputs to the FLC. It is the fuzzy

inference engine that simulates the decision process of a human based on

fuzzy concepts and linguistic statements.

There are many different kinds of fuzzy logic used; see (Lee 1990) for a

comprehensive review. The design parameter of a fuzzy inference engine is:

which specific fuzzy logic is used. In this thesis, we shall use Mamdani's

minimum operation rule as a fuzzy implication function (Lee classifies this 28 reasoning approach as fuzzy reasoning of the First Type), which is just the minimum operation in the Cartesian product. This method is commonly used in FLC applications.

Let us now discuss fuzzy reasoning of the First Type in detail, this reasoning process is depicted in the Figure 2.6 below (adapted from Lee 1990). For simplicity, assume we have the following two control rules:

R1: Ifxl is A1 andx2 is B1 thenz is C1. R2: If xl is A2 and x2 is B2 then z is C2.

First note that a crisp value may be treated as a fuzzy singleton. For example, let

where y A (xi) and y qCYi) play the role of the degrees of partial match between the system inputs and the data in the rule base. This fuzzy reasoning uses

Mamdani's minimum operation rule R, as a fuzzy implication function. In this mode of reasoning, the ib rule leads to the control decision

pC,(w)= ai A P~,(w) 1 ,- ,- '1, ,'.'...".'.' ! c:::::::::::.i \, !.i::::::::::::: ...... ;.. L& W

Figure 2.6 (a) Fuzzy reasoning using Mamdani's minimum operation rule as a fuzzy implication function, (b) resulting decision in output space W

Essentially, the fuzzy inference engine uses the rules in the fuzzy rule base to 30 determine a mapping from the fuzzy sets in U and V to the fuzzy sets in W based on fuzzy logic operators.

The process of defuzzification, performed by the defuzzifier has two major justifications:

to decipher the meanings of vague or fuzzy actions, and to resolve possible conflicts between competing actions generated in a control rule-base.

There are many methods of determining crisp outputs from a fuzzy system, but only the three most commonly used methods will be presented here. For a detailed explanation of these and other defuzzification techniques, see (Cox

THE MAXIMUMMEMBERSHIP METHOD

The simplest approach to action combination is to find the maximum value of the degrees of membership of all actions pertaining to a single output action.

Example 2.5: Suppose we have two fuzzy sets, A and B, and the input into the system is x. To combine all the fuzzy actions that apply we find the fuzzy logical OR of two membership functions at point x as defined by Zadeh in (Zadeh 1973): Suppose that ~~(x)=0.74and pB(x)=0.54. The resultant output being the maximum value of the degrees of membership of all actions pertaining to a single output is accepted to be the single output action, therefore the output is the membership of degree 0.74 to fuzzy set A.

Now the defuzzification is composed of applying the resulting single degree of membership to its respective output fuzzy set to determine the corresponding crisp output. Continuing with the above example, the fuzzy set A is assigned some crisp value c, therefore the resulting non- fuzzy system output would be c*(the truth value of A) or ~"0.74.Note that this method produces a continuous output.

Although this method is simple, there are some problems with this approach.

The main problem is that the potentially significant members in other fuzzy sets are completely ignored. This can be viewed as votes that are ignored, thus not representing the collective opinions. The other problem is not as obvious, but still must be considered when designing a fuzzy system. There is a possibility of a discontinuity in the output values as the executed output moves from one output range to the next. This occurs when the membership in one fuzzy set becomes greater than the membership of the previously dominant range. This discontinuity occurs because, among the ranges, the actual output settings do not correlate with the corresponding membership values. The only way to overcome this is to assure that range overlap is such that outputs are equal for overlapping ranges for membership equalling 0.5. 32

THE CENTROIDDEFUZZIFICATION METHOD

This method considers the contributions from all output fuzzy sets and the

degree to which each is true. As illustrated in Figure 2.7, the centroid method

results in an output action that is associated with the center of mass of the

active outputs. Although this method does away with ambiguity that plagues

the previous methods, it is computationally intensive. Formally, for fuzzy

solution region A the centroid is

where d is the ith domain value and p(d) is the truth value for that domain

point.

Centroid Output Figure 2.7 The centroid defuzzification method

The approach is not acceptable in single rule systems (not realistic in real-

world applications), nor in systems that have no overlap in the input fuzzy 33 sets (Brubaker 1991, No 20). Thus, the designer must ensure proper overlap in input fuzzy sets to provide a smooth response. Despite its shortcomings, the centroid method is currently the best technique for combination and defuzzification.

The last approach is a special case of the centroid method. In comparison to the centroid method, the singleton approach requires less computation, but the centroid method provides smoother output transition. This technique represents each fuzzy output set as a single output value, which is computed by using a weighted average to combine multiple actions. This method is not as computationally intense, yet it still requires overlapping input fuzzy sets to avoid discontinuities in the output.

The defuzzification step consists of translating each active ~1 into its respective output. In order to to perform combination we need to associate weights to each of the i fuzzy sets. The weights represent output settings assigned to the fuzzy sets. The combination step is performed by evaluating the equation for the weighted average of the resulting outputs, shown below 34

Therefore the weighted average of all fuzzy outputs for a system output is y, which is suitable for the real world, in that y is non-fuzzy. Note that this method also produces a continuous output.

Example 2.6: Suppose we have the control rule base:

Rule 1: If temperature is cold then the heat is to be turned onhigh. Rule 2: If temperature is warmthen the heat is to be turned onl,,. Rule 3: If temperature is hot then the heat is to be turned off.

The weighted average of all fuzzy sets would then be:

(cold * onhigh)+ (warm * on,,,) + (hot * ofl Y = (cold + warm + hot)

In this example the value of onhigh and off correspond to one hundred and zero respectively, while the value onlow is set to 40 (representing 40% of the heating systems possible maximum output). Suppose that the fuzzy sets have the following membership values: cold=0.75, warm=0.25, hot=0.0. Evaluating y we get:

This would be the output of the controller, with the heating element being 85% on out of a possible maximum of 100%. Note that the output reflects the average truth value of all fuzzy sets with their respective crisp weights.

A fuzzy rule base R is a collection of linguistic statements in the form of "IF a

set of conditions are satisfied, THEN a set of consequences are inferred," 35 where the conditions and consequences are associated with linguistic terms

(represented by fuzzy sets). For example, consider a MIS0 (multiple-input, single-output) FLC with a fuzzy rule base R composed of m rules

R= [Rl, R2,... ,Rm1, with

R~:IFX,~SA~ANDX,~SA; AND ..ANDX,~SA;,THEN z is B' where xi (i = 1,2, ... ,n) are the inputs to the FLC, z is the output of the FLC,

A: and B' (j= 1,2,... ,K) are linguistic terms represented by fuzzy sets, and K is the number of fuzzy rules in the fuzzy rule base.

Although the antecedent of Rj is connected by the AND operation, Rj is general enough to include rules whose antecedent contains OR or NOT operations. Specifically, if there is an OR operation, we can decompose the rule into two rules whose antecedents contain only AND operations.

Consider the following rule: "IFx,is PIAND x,is P,OR x,is P,THEN z is c.

This rule can be decomposed into two rules:

"ZFx,isP,ANDx, isp, THEN zis~";and,"ZFx,isP,THEN zis$'. If there is a NOT operation, e.g. "x, is not PI ",we can simply define a new

fuzzy set 1P , with the membership function p ,p = 1 - pp , so that

"x ,is not PI is equivalent to "x ,is P, " . Thus, the fuzzy rule base R 36 includes many linguistic descriptors that can describe the relationships between the input x and the output z. In this thesis, we shall use only the

AND operation, since it can represent all other linguistic descriptors.

The concept of natural selection is a powerful search and optimization technique, that has been demonstrated in nature. The following evolutionary techniques are in a class of paradigms known as evolutionary computation

(ECs), algorithmic analogies to natural selection. There are three classes of evolutionary computation techniques, differing by how close the algorithm models evolution. Genetic algorithms, models evolution at the level of gene propagation. Evolution strategies, a method that models how evolution optimizes individuals in order to better exploit their environment.

Evolutionary programming, uses a model of evolution in which multiple species compete for shared resources. Despite their differing levels of modelling, all three of these ECs share a common computational form that differentiates them from other optimization/search paradigms. In this thesis we shall direct our focus to the usage of genetic algorithms.

In genetic algorithms, a population of solutions to the problem are created and a heuristic is used to decide which solutions are of higher quality. Those that

are better solutions are allowed to reproduce with their children replacing

inferior solutions. Subtle changes are made to the children, mimicry of

mutation in nature's imperfect reproduction with this diversity inherently

searching for better solutions. Once the population has been updated you

then evaluate the new population (solutions) and repeat the procedure.

7 /- -7 Population , Evaluation

ranked population

\, Selection 1 Population (

1 fit individuals

/' Reproduction New Population

Figure 2.8 The evolutionary computation cycle

A genetic algorithm consist of three mechanisms: (1) evaluation, (2) selection,

and (3) recombination. The first step in solving an optimization problem

using GAS is to describe (or "encode") the problem in a language the algorithm

can understand. This requires mapping the problem's variables into genes

which compose a chromosome of an individual that represents a complete

and valid solution to the problem. Second, we must devise a heuristic (an 38

"objective function") that measures how well an individual (solution) solves the problem. A benefit that can be noted here is that since the GA works from a population of possible answers, a GA is inherently parallel, because it searches from a population of possible solutions rather than a single point.

A simple genetic algorithm could follow these steps:

1. Initialize the population with assigned solutions. Solutions may be assigned randomly or assigned by a best guess methodology.

2. Evaluate fitness of entire population. Each chromosome or possible solution is given a value that is a function of how well that solution solves the problem. This fitness value allows one to determine the most fit solution in the current population. If (Goal is reached) or (No further improvement) goto step 6.

3. Apply selection pressure. The population is ranked according to fitness with the most fit chromosome ranked at the top. Higher ranking individuals are allowed a higher mating rate, thus allowed to propagate.

4. Reproduction and variation. This mating process exchanges basic building blocks from the parent chromosome to offspring chromosomes.

5. Goto step 2, to evaluate the fitness of the population.

6. Exit procedure.

There are two distinct genetic operators used in GAS: crossover and mutation.

There are several mechanisms at work within the GA: exploitation (from 39 selection and reproduction - i.e. the model of evolution), a combination of exploitation and exploration (crossover) and pure exploration (mutation).

Exploitation emerges from mating two highly fit parents, which exploits those chromosomes that are stronger. Exploration emerges from information exchange in the construction of new individual structures and random noise, which forces chromosomes to explore the solution space. Now we will explore basic techniques in both crossover and mutation.

CROSSOVER

There are many different types of crossover, ranging from very simple to very complex, depending on the types of data structures. Even with very simple data structures representing the chromosomes, there are a number of different types of crossover operators. Consider two parents, Parent A and Parent B, as depicted in Figure 2.9. A simple crossover, single-point crossover begins by generating a random point, called the crossover point (a genetic loci), along the chromosome (denoted in Figure 2.9 as a dashed line). At that point, the chromosome is divided into two segments. Now the right-most segment is exchanged between parent A and parent B and fused to create two offspring as follows: Parent A

Parent B

Offspring A

Offspring B

Random Point Figure 2.9 An example of single-point crossover

There is a problem with single-point crossover. Loci near the beginning and

end of the chromosome are very hard to separate from one another as are loci

that are close together. A simple solution is to have multiple point

crossover. In two-point crossover, shown in Figure 2.10, we generate two

random loci and then the offspring are copied from one parent before and after

the crossover and the other between them. &\\\\\\\\p Parent B l+l Offspring A

Offspring B

Random Points Figure 2.10 An example of two-point crossover

MUTATION

Where the crossover operations allow individuals to mix and match

information so as to allow a broad search of the search space of possible data

structures accessible to the GA, mutation is supposed to make small changes

in individuals allowing a local search and also provide the gradual introduction

of new ideas for exploration.

Simple mutation is called point mutation. A single-point mutation consists

of generating a random position within the chromosome and applying a point

mutation at that position, see Figure 2.1 1 for a depiction of single-point

mutation. A multiple-point mutation consists of generating some fixed

number of positions in the chromosome and performing point mutations to all

chosen points. A Lamarckian mutation of depth k, allows the parent to 42

pass "knowledge" to its offspring within the chromosome itself (just as

Lamark's theory suggested), whereby all possible combinations of k or fewer

points is performed to see which mutation provides the best fitness value.

In setting up a genetic algorithm you need to design a set of mutation

operators that will be used and decide how often each on will be used.The

probability a given mutation operator will be used is called the mutation rate.

Figure 2.1 1 An example of single-point mutation

The method of picking parents together with the method of inserting offspring

back into population, taken together, are called the models of evolution used

by the genetic algorithm.

These techniques are very crucial to the preservation of good building blocks.

For example, by allowing only a limited set of individuals to breed with a 43 limited set of individuals, premature convergence to local optima is prevented

(i.e. superior individuals take over more slowly). This then prevents premature diversity loss within the population.

The first step in forming a model of evolution we must devise a scheme to select parents from the population. The most common approaches are as follows:

*Tournament Selection- In tournament selection the population is shuffled randomly and divided into small groups of size m. The most fit pair in each small group are deemed "better" and granted parenthood. The parents are crossed over, with the two mutated children replacing the least fit within the small group. There are 2 main advantages to using tournament selection: (1) regardless of the fitness of an individual in comparison to the rest of the population, it can only have one offspring ensuring that a relatively good solution does not take over the entire population; and, (2) the best (m-2) individuals are guaranteed survival, thus the maximum fitness of a group cannot decline though time.

.Roulette Wheel Selection-Roulette wheel selection chooses parents in direct proportion of their fitness. If individual i has fitness f;: then the probability of being picked as a parent is fi/F, where F is the sum of the fitness values of the entire population.

*Rank Selection-Rank selection works in a similar fashion to roulette wheel selection, but the individuals are ordered by fitness and their rank is used instead of their fitness.

Below we compare the probabilities for roulette and rank selection. Individual i Fitness (fi) Rank P (Roulette) P (Rank)

1 3.16 5 0.160 0.238

2 2.93 4 0.149 0.190

3 1.72 1 0.087 0.047

4 2.81 3 0.142 0.143

5 2.22 2 0.1 12 0.095

6 6.82 6 0.346 0.286

Sum 19.66 2 1

When a fraction of a population with high fitness (or the highest) is allowed to survive the genetic algorithm is said to possess elitism, where the surviving individuals are called the elite. This keeps the population from falling below the maximum fitness, allowing the fitness only to improve through time.

OFFSPRING INSERTIONTECHNIQUES

In order to get a complete model of evolution we must pair our parent selection methods with an offspring insertion method.

One perfectly good technique is to place the offspring in the population at random, replacing anyone. This is called random replacement. If we select individuals to be replaced with a probability inversely proportional to their fitness we are using roulette wheel replacement. If we rank the individuals, in the opposite order used for rank selection, and chose them to be replaced with 45 probability proportional to their rank we are using rank replacement.

Another is to replace the least fit members in the population with the offspring which has been termed absolute fitness replacement. Yet another possible technique is to have the children replace their parents if they are more fit. In this method locally elite replacement, the two parents and their two children are examined and the two most fit are put into the population in the positions previously occupied by the parents.

Until recently, a fundamental question that has blocked a firm understanding of GAS has been elegantly stated by Goldberg (Goldberg 1993):

In a population oriented algorithm where individuals come and go, never to return, what are the quasi-invariants (characteristic descriptors) that are propagated from generation to generation.

John Holland's book Adaptation in Natural and Artificial Systems, 1975, answered this question, by stating that highly fit, relatively small sub-structures

(referred to as schemata or similarity subsets) are the building blocks of future individuals, being selected and propagated among good structures, thus forming even better structures as the population evolves. This was the result of his decomposition of the problem of genetic search by ways of his concept of a building block. David Brubaker, a well-known consultant and expert in the field of fuzzy systems, describes the typical approach (Brubaker 1994) of the development a fuzzy system consisting of:

Design. Both of the fuzzy component itself, and of the interface of the fuzzy component to its environment (the world or the rest of the system).

Simulation. Currently this replaces formal analysis. Additional care must be placed in the model of the environment with which the fuzzy module interacts. As complexity of the developed system increases, a tool with a good simulator quickly becomes invaluable.

Translate, Code, Compile and Download. Translating the design into code, typically C or your language of choice, compiling it, and downloading it to the target.

Debug. Removing any bugs that occurred in the translation from a fuzzy representation to C.

Test and Tune. Once debugged, the fuzzy module will need final tuning as it runs interacting with its runs interacting with its actual environment.

This procedure, however, leads the designer into a obvious time-consuming loop: test and tune. There have been many software systems and packages to aid a system designer in designing a fuzzy system. Let us briefly discuss some of the systems available. 47

2.2.2 THE WANG AND MENDEL METHODOLOGY

This approach to fuzzy system design is a data-driven methodology. Li-Xi

Wang and Jerry M. Mendel (Wang et al. 1992) from the University of

California, identified two sources of information available to a system designer in the design of control systems: (1) Linguistic information obtained from skilled operators, and (2) Numerical information sampled from the successful operation of the system. Thus far, fuzzy systems have focused on modeling linguistic information.

The methodology is composed of five steps that generates fuzzy rules from numerical data. The steps are briefly summarized below:

Divide the input and output spaces into fuzzy regions Generate fuzzy rules from given input-output data pairs Assign a degree to each rule Create a combined fuzzy rule base Determine a mapping based on the combined fuzzy rule base

Using a fuzzy rule base as a common framework, this methodology provides a mapping that is proved capable of approximating any real continuous function on a compact set to arbitrary accuracy. Applications to the truck backer- upper control and time series prediction have been demonstrated, exhibiting the usefulness of this general method. This method is covered in great detail in

Chapter 3, section 1.1. Soft computing differs from conventional (hard) computing in that, unlike hard computing, it is tolerant of imprecision, uncertainty and partial truth. In effect, the role model for soft computing is the human mind. The guiding principle of soft computing is: Exploit the tolerance for imprecision, uncertainty and partial truth to achieve tractability, robustness and low solution cost. the principal constituents of soft computing (SC) are fuzzy logic (FL), neural network theory (NN) and probabilistic reasoning (PR), with the latter subsuming belief networks, genetic algorithms, chaos theory and parts of learning theory. SC is not a conglomeration of FL, NN and PR.

Rather, it is a partnership in which each of the partners contributes a distinct methodology for addressing problems in its domain. In this perspective, FL,

NN and PR are complementary rather than competitive. Let us now look at soft computing based methods that perform fuzzy system design.

Artificial neural networks have been used in fuzzy system design, that approach the problem of fuzzy system design as a clustering problem, learning the relationship between system input(s) and output. Let us now look at a few notable systems.

NeuFuz (Khan et al. 1993) is the only commercially (distributed by National

Semiconductor) available automated fuzzy system design technique. NeuFuz is aimed at low end controls applications in automotive, industrial, and appliance areas. NeuFuz is a neural-fuzzy technology which uses backpropagation techniques to initially select fuzzy rules and membership functions. Initial stages of design using NeuFuz technology are performed using training data and backpropagation. The result is a fuzzy associative memory (FAM), which implements an approximation of the training data.

By implementing a FAM, rather than a multi-layer perceptron, the designer has a solution which can be understood and tuned to a particular application using fuzzy logic design techniques.

However, there are several limitations to the NeuFuz system. First, there is a usability constraint, since this system is tailored to small applications, thus making it incapable in handling large scale systems (problems). There is also a question of stability and robustness that plague neural network strategies.

There are several NeuFuz packages available:

NeuFuz4 Learning Kit NeuFuz2 Neural Network Learning Software Up to 2 inputs, 1 output 50 training patterns Up to 3 membership functions COP8 Code Generator (COP8 is National's family of 8-bit microcontrollers)

NeuFuz4 Software Package NeuFuz4 Software Neural Network Learning Software - Up to 4 inputs, 1 output and 1200 training patterns Up to 7 membership functions COP8 Code Generator

The NeuFuz4 Development System Neural Network Learning Software - Up to 4 inputs, 1 output and 1200 training patterns Up to 7 membership functions COP8 Code Generator COP8 In-Circuit Emulator "Debug Module" Real-Time Emulation Microcontroller EPROM Programming Real-Time Trace Complete Source/Symbolic Debug

NeuFuz4-C Learning Kit Up to 2 inputs, 1 output 50 training patterns Up to 3 membership functions ANSI Standard C Language Code Generator

NeuFuz4-C Software Package Up to 4 inputs, 1 output and 1200 training patterns Up to 7 membership functions ANSI Standard C Language Code Generator

Kosko captured fuzzy system representation and functionality in a truly unique way through the use of the Fuzzy Approximation Theorem (FAT).

FAT basically says that a fuzzy system can model (approximate) any continuous system. Kosko represents this approximation as a collection of 5 1 patches, in which each patch covers a portion of the system space. Each patch represents "knowledge" in the form of a fuzzy rule:

IF A THEN B.

Since a fuzzy system is a collection of IF-THEN rules, then a fuzzy system can be represented by a collection of patches. The better the fuzzy system approximates the system, the more patches are required. In short, big patches represent imprecise rules, whereas small patches represent more precise rules.

The FAT basically says that "one can always succeed," but as most abstract theorems, does not tell us how. To address this problem, Kosko approached adaptive fuzzy system from a neural network point-of-view. This neural network can be considered a black box, where data enters the box and fuzzy rules come out.

One of Kosko's students, Julie Dickerson, used Adaptive Vector Quantitizers

(AVQ networks) (Dickerson et al. 1993) to learn the input-output patterns of a system and to generate patches (fuzzy rules). An AVQ network, also called self-organizing feature map uses an unsupervised-feedforward paradigm to group input vectors in classes (in this case patches). In brief, once exposed to the input vector, lateral competition occurs between the network elements.

The winning node (i.e. the node having components closest to the input 52

vector) is then rewarded. The essence of the cycle is that the network is

changing itself to look like the data.

Each AVQ node or dot lies at the center of its own "error" (or covariance

ellipsoid), whereas the ellipsoid size is dependant on the region an AVQ dot

quantitizes. Sparse data regions result in large ellipsoids meaning large patches

(and thus more uncertain fuzzy rules). Regions where data is confined to a

small area result in small ellipsoids meaning small patches (more certain fuzzy

rules). Now that the fuzzy rules have been defined, a supervised neural

network (such as backpropagation) is used to fine tune the fuzzy system. 'r = = If x A2 Then y B2 /

A1 A2 A3 X Figure 2.12 Patches representing fuzzy rules 53

Khedkar and Berenji have shown that fuzzy rules can be generated from linear consequents from data in (Khedkar et al. 1993). Using the Takagi-Sugeno-

Kang (TSK) model of fuzzy rules, Khedkar et al. proposed a method to generate a fuzzy system from data using radial basis functions and an extended

Kohonen clustering neural network. This approach uses fuzzy symmetric spheres to partition the input space (due to the Kohonen network architecture), which limits the applicability of implementing the resulting fuzzy systems on a variety of platforms (microcontrollers as well as workstations).

Isik and Zia (Isik et al. 1993) have proposed a self-organizing controller that automatically generates and modifies the set of fuzzy control rules as the control process evolves. The system is based upon a self-organizing map

(SOM) neural network. This system also obtains membership functions and fuzzy rules from the system's input and output. However, the membership functions remain embedded within the SOM, thus requiring a run-time version of the SOM to fuzzify system inputs. This is not feasible in a run-time system that may have limited resources.

The apparent problem with artificial neural network approaches is that the system is easily learnt, but the information remains within the neural network.

It is well known that knowledge embedded within a neural network is hard if 54 not impossible to extract. Thus, the approaches to date do not adequately solve (extract fuzzy rules from neural networks) the problem of fuzzy system design.

2.3.2 Fuzzy SYSTEMSYNTHESIS AND OPT~M~ZAT~ON USING GENETICALGORITHMS

The evolutionary synthesis of fuzzy systems views the problem of fuzzy system design strictly as an optimization problem. Most of the genetic algorithm (GA) based techniques have been developed by Charles Karr (Karr

1991) and have shown promising results. The usage of GAS to select membership functions demonstrated in (Karr 199 1) have been applied to real- time systems. This technique uses a pGA (Karr 1989) (Krishnakumar 1989), which are small population, rapidly converging genetic algorithms. This technique is mainly a self-organizing system that optimizes during run-time.

Shehdeh and Lea (Shehdeh et al. 1992) used a GA approach to shorten development time by altering the membership functions. However, the major downfall of this approach is that an initial fuzzy system must be devised.

Lee and Takagi (Lee et al. 1993) developed a genetic algorithm based fuzzy system generation scheme that improves upon Karr's method by optimizing both the size of the fuzzy rule base along with its composition (i.e. fuzzy 55 sets). Thus, the number of fuzzy rules do not have to be specified a priori.

However, this method has a large chromosome representation and has a fixed number of fuzzy sets permitted on each input domain.

Cooper and Vidal (Cooper et al. 1994) further improved over Lee and

Takagi's method, but recognize that some fuzzy rules may possess irrelevant variables. An assumption made with earlier attempts at fuzzy system learning is that a fuzzy rule is dependent upon combinations of all input variables.

Their system encompasses all of Lee and Takagi's functionality, but addressed this problem by allowing rules to possess partial input variables. Another problem Cooper and Vidal addressed was the fuzzy system representation as a chromosome. Noting that Lee and Takagi's representation does not scale well for complex problems, due to its representational scheme. It is known that as the chromosome length increases, the search space size increases exponentially. They devised a compact representation scheme keeping the chromosome size as small as possible and discarding rules that are not necessary to accomplish the task at hand.

Another method of fuzzy system design implementing GAS is presented in

(Ishibuchi 1993) that uses inputloutput data to generate a fuzzy system. The genetic algorithm used has two objectives: maximize the number of correctly classified patterns, and to minimize the number of rules in the rule-base.

Here a genetic algorithm finds a minimum set of fuzzy if-then rules that correctly classifies all training patterns (input / output data). This method seems to provide the most automated method of fuzzy system synthesis.

In summary, the approach taken with GAS must, to be computationally feasible, begin with an initial solution. This is obvious when modeling a complicated system with large rule bases. Holland's statement on the use of

GAS in complex problems to make improvements, not necessarily to find the optima, is of great concern when designing real-world systems. It can be concluded from this literature review that there is no robust and computationally efficient methodology for the design of fuzzy systems.

Consequently, a fuzzy system design methodology should be composed of simple, yet robust problem solving elements. This calls for the utilization of evolutionary programming beginning from a quickly derived good initial solution. A Fuzzy SYSTEMDESIGN

This chapter presents a general description of the elements that comprise the

fuzzy system design methodology. These elements are explained: (1) A

variation upon Wang and Mendel's approach, (2) An evolutionary

computation approach.

3. 1 Fuzzy SYSTEMDESIGN ELEMENTS

The proposed approach consists of two independent methods:

DESIGN, a single-pass fuzzy system generation method based on a variation of Wang and Mendel's method, that generates a crude, yet complete functioning fuzzy system definition. See Figure 3.1. OPTIMIST, an evolutionary optimization technique using genetic algorithm (GAS), which improves (evolves) a fuzzy system definition. See Figure 3.2. 1 Create Fuzzy Regions I L- L- Sam~leData

Generate Fuzzy i Rules

I Compute Rule I Validity (Truth)

i Eliminate 1 Invalid/Duplicate ~I Rules -- - Fuzzy System Definition Figure 3.1 Architecture of the DESIGN system

In the fuzzy system design approach proposed by Wang and Mendel, the first

step is to collect all available information. As highlighted earlier, the

information is in two forms: (1) human experience and (2) sample data. The

human experience can be formulated into IF-THEN rules to explain all

possible states of system operation. The sampled data may be pre-processed

(filtered) to remove any noise. However, both data forms have a common

problem: Inconsistency. Since, interviewing a human expert does not extract

all information that one uses to perform the operation, some information will

be lost. Compute r) Fitness and -- ~ Rank I ' ~ i Fuzzy System v Definition ; I I I I Selection 1 i

Reproduction 1 I

Continue Improved Fuzzy System Definition Figure 3.2 Architecture of the OPTIMIST system

Also, the sampled data might not capture a system state that is rarely entered,

but is important in operating properly. Thus, combining these two forms of

data into a common framework allows a proper controller to be derived from

both sources of information. 3. 1 . I A VARIATIONON A THEME:THE WANG AND MENDEL METHOD

This section presents a detailed description of a variation of methodology proposed by Wang and Mendel. In their 1992 paper "Generating fuzzy rules by learning from examples," Wang and Mendel describe a five-step procedure that generates fuzzy rules from numerical data. The method determines a mapping from the input space to the output space represented by a common framework-the fuzzy rule base. This computationally simple method, performs a single-pass through sample data (derived from the system under consideration while "in control" ) to generate a complete fuzzy system.

Consider a set of m desired input-output numerical data from a SISO system:

(x17~l)9(xl,~1 h... (xm,Y",) where x is the input, and y is the output. This numerical training data will be used to generate a fuzzy rule base that defines the mapping f(x) - y. This simple one-input one-output case was chosen in order to emphasize and to clarify the algorithm. Now let us go through the DESIGN algorithm step by step-by-step. Step 1: Divide the input and output spaces into fuzzy regions.

Assuming that the domain intervals of x and y are [x-,xt] and b-,PI,where the domain interval defines a region in which the variable is most likely to fall (a value is allowed to lie outside its domain interval). The domain interval is then divided into N regions, which identifies the number of fuzzy sets (also referred to as the coarseness). The coarseness of each fuzzy region and range is defined by the user. Each region is assigned a fuzzy set. The fuzzy sets described here are triangular fuzzy numbers (TFNs), which are defined by three points-one point lies at the center of the region and has unity membership within the fbzzy set, while the other two points lie at the centers of the two neighboring regions, having no membership in the fuzzy set. Figure 3.3 shows an example where the input space (domain x) has been divided into five regions, and Figure 3.4 shows the output space (domain y) divided into three regions. Note that we ensure that there is an overlap of 50% between adjacent fuzzy sets.

Since we are dealing with only MIS0 systems (composed of n inputs and a single output) then we have n+l fuzzy regions, with each space pre- defined over a range.

X - x1 x2 Figure 3.3 The division of the input space x into fuzzy regions Y - Y' Y Y + Figure 3.4 The division of the output space y into fuzzy regions

Step 2: Generate Fuzzy Rules from the Data Vectors

The first step is to determine the degree of membership of the sample inputs and outputs to their corresponding regions. For each sample vector (composed of n+l elements) a rule is generated that maps the input to the output. For example, xl in Figure 3.3 has a truth value of 0.4 in c, 0.6 in d and no membership in the remaining sets in this region.

Then the given inputs and outputs are assigned to the region with maximum truth value. For example, xl in Figure 3.3 is considered to be d and yl in Figure 3.4 is considered to be b. Finally, a rule is obtained from the one input-output vector, e.g.,

(xl,yl) - [xl (0.6 in d, max); yl (0.9 in b, max)] - Rule 1: If x is d, Then y is b. Similarly,

(x2,y2) - [x2 (0.8 in d, max); yl (0.8 in c, max)] - Rule 2: If x is d, Then y is c.

The antecedents of the rules are combined using the AND operator, in which the conditions of the antecedent must be met simultaneously in order for the consequent to fire.

Step 3: Assign a degree of truth to each rule.

Since there are many data sets, with each data set generating one rule, it is likely that there will be some conflicting rules, that is, rules that have the same antecedent, but a different consequent. A simple solution to resolve this conflict is to assign a degree of truth to each rule, and accept only the rule from a conflict group with the highest degree of truth. A product strategy is used to rank each fuzzy rule in the rule base. For the rule: "IF x is A, THEN y is B," the degree of this rule, denoted D(rule), is defined as:

As an example, Rule 1, defined in the previous step, has a degree of truth equal to,

and Rule 2 is,

Note in step two the rules generated have the same antecedent, yet differing consequents. These rules are in conflict since we cannot make a conclusion from these rules. However, now we can tell which rule has the most truth, namely Rule 2. As illustrated, this step eliminates conflicting rules, thus reducing the rule base size.

This deterministic algorithm shall be denoted by the parameters of each domain. Using the above example,

DESIGN (2 domains) { x: (5s~-#I; Y: (5,J-,Y+); 1;

In general, we shall represent a fuzzy system definition as its DESIGN parameters, as follows

DESIGN (n+ 1 domains) { Domain 0: (N, min, max); Domain 1: (N, min, max);

Domain n+ 1: (N, min, max); 1; In summary, this element has several advantages: (1) computational efficiency, (2) reduced set of control parameters, (3) effective learning from examples, (4) conflict data resolution, (5) automated support to the knowledge acquisition and design of experiments - this allows integration to other function approximation techniques, e.g. neural networks, (6)explanation of data in terms of linguistic structures, and (7) tolerance to high-dimensional input space.

3. 1 .2 OPTIMIST: AN EVOLUTIONARYOPTIMIZATION METHOD

The problem that first must be addressed in setting-up a genetic algorithm is the representation of a fuzzy system definition. The computer data structure that represents the fuzzy system definition will play the role of an individual definition, or chromosome. The second problem is to find a heuristic or fitness function that maps the data structures onto an ordered set, such as real numbers. The fitness would indicate how "well" a fuzzy system performs at controlling the system under consideration.

3. 1 .2. 1 Fuzzy SYSTEMREPRESENTATION

Let us first discuss the problem of individual representation. A fuzzy system is composed of a set of rules. Each of these rules has an antecedent and a consequent. Thus a rule has the following general form:

R~:IFX,~SA:ANDX~~SA; AND ~iVDx,,isA~,THEN z is B'

where xi (i = 1,2,... ,n) are the inputs, z is the output, A: and B' (j= 1,2,... ,K) are fuzzy sets, and K is the number of fuzzy rules in the fuzzy rule base.

Now we must discuss how to best represent a complete fuzzy system in a computer data structure. All rules of a fuzzy system must be represented in a fashion that can be easily altered. In Figure 3.5, the individual chromosome structure is realized as a collection of rules, where each rule represents an allele of the chromosome. The structure of the rule, being the fuzzy sets combined using the AND operator, are held constant with the only alterations being on the fuzzy set definitions itself. Note that we are encoding the problem using real numbers, this is commonly referred to as real-number encoding as opposed to binary encoding, which would encode each number as binary number. individual pointer -

7 Individual F Information ~ rule pointer -- +~~[rufiformation '

fuzzy set TITI antecedent definition - 7 ~G--%+F-

1- 1- I nil fuzzy set consequent definition

Figure 3.5 Individual structure within population

Since genetic algorithms are global search techniques, the representation of

system variables, that is the chromosome length should be kept as small as

possible. This is because the search space increases exponentially as the size

of the chromosome increases. This is where an initial solution, or best guess

limits the search to primarily a local search. Thus, the membership functions

and rule structure will start from an initial best guess state derived from

DESIGN. 3. 1 .2.2 Fuzzy SYSTEMQUALITATIVE MEASUREMENT

In order to rank a population of fuzzy systems, an objective function must be devised to measure the "goodness" of a fuzzy system's performance. The main goal the fuzzy system is to minimize the root-mean-square (RMS) error over z data sets, where the RMS error is defined as follows,

To compute the fitness of an individual member of the population, test data is fed into the fuzzy system, whereby the difference between the real output and the predicted output (from the fuzzy system) is determined. This procedure allows the performance of the fuzzy system to be converted into a value in the real number domain. There are other possible methods of measuring the performance of a fuzzy system, such as average absolute error, which is just the average of the difference between predicted and actual output; mean squared error, and the maximum absolute error.

Crossover operations allows different individuals to mix and share information

as to perform a broad search of the state space. On the other hand, mutation

performs small changes on individuals that essentially allow a local search. 69

Mutation is usually considered the exploratory mechanism, since it provides

for gradual introduction of new information for crossover to exploit. Since the

basic fuzzy system definition is held constant, possible mutation operators

are limited to fuzzy set modifications. There are several possible methods of

mutating a set. In Figure 3.6, a TpFN is depicted with three possible

mutations in Figures 3.7 through 3.9.

a b,c d Figure 3.6 A trapezoidal fuzzy number

Let us now discuss each mutation operator in detail:

*Linear shift-The simplest form of mutation is to shift (either left or right)

the entire fuzzy set. The distance of maximum shifting is determined by a

setting, MaxSetShift. The actual distance the set is perturbed is the product of

a number randomly chosen between zero and one and MaxSetShift. This

mutation operator is depicted in Figure 3.7, where the mutation perturbs the

entire set to the right. a' b', c' d ' Figure 3.7 A linear shift mutation

Expansion-This mutation operator changes (expands) the width of the base

(i.e. points a and d depicted in Figure 3.6 compose the base of a TpFN) or the

width of the core (i.e. points b and c of a TpFN). The mutation upon the

base or the core is chosen randomly (with a probability of occurrence set at

0.5). In the expansion of the core and the base, the maximum shift possible is

the min(min(b-a,d-c)/2, MaxSetShift). An expansion mutation is depicted in

Figure 3.8 in which points b and c, the core are expanded an arbitrary amount.

/ .+2 a b' c' d Figure 3.8 An expansion mutation

Contraction-This is the opposite of the expansion mutation with the

maximum possible contraction computed as in the expansion mutation. Note

that here we cannot contract the core of TFN (since b=c), only a TpFN. Nonlinear- In nonlinear mutation, one point of the TpFN is mutated, thus

leaving the TpFN slightly changed. Each point (a, b,c or d) has an equal

probability of being chosen for perturbation. The maximum possible

contraction of each point is dependant upon its position:

a - has a maximum shift of min((b-a)/2, MaxSetShift).

b - has a maximum shift of min(ha1f the distance from b to the nearest point, MaxSetShift). If b is a member of a TFN, c is moved with b, else b moves independently.

c - has a maximum shift of min(ha1f the distance from c to the nearest point, MaxSetShift). If c is a member of a TFN, b is moved with c, else c moves independently.

d - has a maximum shift of min((d-c)/2, MaxSetShift). Note that each of these mutations moves a specific point in either direction, with each direction having equal probability.

A nonlinear mutation is depicted in Figure 3.9, in which point a has been

perturbed, thus decreasing the fuzzy set's base.

a a' b,c d Figure 3.9 A nonlinear shift mutation

Since there are four basic types of mutation that can be used, which one

should be used in order to rapidly evolve the population of fuzzy systems in 72 the most efficient way? This problem can be essentially viewed as a 4-armed bandit problem (Holland 1992), which will be discussed in the first case study

(Chapter 5, section 1).

Mutation moves and shapes the rule patches, then the evolutionary process selects the best move and shapes, thus improving the approximation. Linear shift mutation changes the position of the patch, but does not change the shape of the patch; whereas nonlinear mutation changes the shape of the patch. The other mutation operators described are essentially specific cases of these two operators. In Figure 3.10 below, a mutation operator is visualized as it shifts the rule patch down, thus changing the output fuzzy set's shape. If x = A2 Then y = B2 I

A 1 A2 A3 x Figure 3.10 Mutation depiction in Kosko's representation

The usage of sexual recombination in genetic algorithms allows the solution

space to be rapidly explored, in comparison to mutation-only methods.

Crossover in our population is very complicated to formulate, since each rule

covers part of the input space. Since the DESIGN algorithm eliminates

redundant and possibly conflicting rules, all of the input space is currently

covered. Thus, we cannot add and eliminate rules aimlessly. So to preserve

the general form of the rules currently present, we will exchange the content of

the rules, i.e. the fuzzy set definitions. Figure 3.1 1 depicts the generation of 74

an offspring from two parents. Note that the process of crossover here can be

viewed as a form of mutation in which a fuzzy system is copied from two

random sources (where each source has an equal probability of being chosen)

and one target. This views each rule as being the smallest building block that

can be exchanged between parent and offspring.

Fuzzy System Definition n

I rule 2 I rule 3 m rule 4 WB1

r rule 6 /////llla

Fuzzy System Definition m

rule 3 XmDr.11 rule 4 rule 5 izsssSS rule 6 j Figure 3.11 Crossover in fuzzy system reproduction There are several possible types of models of evolution, of which three were implemented within this work. The first model of evolution is Best-Only propagation. In this model the best individual of the population is kept (i.e. elitism=l) and mutated copies of this individual fill the population. The second model of evolution is Mutation-Only with "fixed" elitism (e.g, 10).

Here we keep the top ten ranking fuzzy systems and replace the remaining population with mutated copies of these top ten. Therefore, trials are allocated in an exponential fashion to the best members (those ones with the best salient features). Finally, the third model of evolution is Tournament selection with group size=4, which was covered in Chapter 2, section 1.3.3.

In tournament selection we use both crossover and mutation to introduce variation into the population (and avoid stagnation). In the first case study

(Chapter 5, section 1) we will compare the various models of evolution, as well as the effects of each mutation operator In the experiments that follow using OPTIMIST, in general we shall denote the experiment as follows:

OPTIMIST { Random seed, Population size, Number of generations, Model of evolution used, Maximum set shift, Probability of mutation, Probability of crossover (if applicable), Mutation probabilities linear expansion contraction nonlinear }

In summary, this second element has several advantages: (1) computational efficiency with implicit parallelism, (2) reduced set of control parameters, (3) exploration and exploitation tradeoff, and (4) tolerance to high-dimensional spaces. The fuzzy systeni design methodology described in Chapter 3 has been

implemented in a prototype model. This chapter provides descriptive details

of its implementation and follows both the modular and the operational

algorithms description presented previously. Chapter 5 discusses empirical results based on this prototype. The source code of this prototype (in the

language C) appear in Appendixes A and B. 4. 1 Fuzzy SYSTEMDESIGN

This section explains the design parameters required of a typical fuzzy system

and the fuzzy system definition that was used. The software design of

DESIGN and OPTIMIST is discussed in detail along with parameters used.

In designing a fuzzy system there are many parameters that are considered in

the initial stage of designing the system. The design parameters of a typical

FLC are:

The number of fuzzy sets defined in the input and output universes of discourse; Membership functions of these fuzzy sets; The number of fuzzy rules in the fuzzy system definition; The linguistic statements of the fuzzy rules; The decision making logic used in the fuzzy inference engine; and, The defuzzification method.

Now let us discuss each one of these design parameters in detail.

The number of fuzzy sets in the input and output universes (here we refer to

these as domains) is dependent upon the coarseness (hence the accuracy)

required. The more fuzzy sets in a domain, the classification of inputs is

better. The parameters used here is the number of fuzzy sets and the span of

the domain (i.e. [min,.. max]). 79

The membership functions of these fuzzy sets is defined by a simple, yet powerful trapezoidal fuzzy number (TpFN). This simple representation was chosen, since it requires only four points to represent a fuzzy number, as opposed to some of the more complex fuzzy set representations, such as Beta or Gaussian forms. Cox (Cox 1994) notes that "in modeling dynamic systems the use of triangular functions can approximate their behaviors to nearly any degree of precision."

The number of fuzzy rules in the fuzzy system definition shall be left up to the designer. There are many factors in implementation that effect the number of rules, such as number of fuzzy sets in the input domains and the amount of space available.

The linguistic statements of the fuzzy rules shall be of the form described in

Chapter 2 section 1.2.5. The general form of a rule is:

R~:IFX,~SA{A~Dx~isAi AND ANDX,,~SA:.THEN z is B'

where xi (i = 1,2,... ,n) are the inputs, z is the output, A: and B' (j = 1,2,... ,K) are fuzzy sets, and K is the number of fuzzy rules in the fuzzy rule base. 80

The decision making logic used in the fuzzy inference engine. We shall confine our system to using only Mamdani's minimum operation rule as a fuzzy implication function. The defuzzification method shall be set the centroid defuzzification scheme.

This section discusses the implementation of the two elements by using ANSI

C on the Macintosh platform.

As discussed in Chapter 3, section 1.1 there are three major steps to the variation of Wang and Mendel's algorithm (DESIGN). This algorithm performs a deterministic single-pass through the data creating a complete

(assuming there is enough data to create the system desired) fuzzy system definition. Experirn en t Sample Data

Fuzzy System Definition Figure 4.1 File input and output in DESIGN

Now let us discuss the functions used within DESIGN.

Data Structures

The basic structure that is used throughout this system is a fuzzy set. A fuzzy set as stated before is represented as a fuzzy trapezoidal number that can be represented by four points, along with a label (for identification purposes). For each system input (called a domain), there is a data structure that represents all of the fuzzy sets in that specific region. There also is a data structure that can easily be accessed that holds the sample data vectors. Since there is a one-to-one correlation between the sample data and the resulting rule, a rule with its corresponding information is also stored with the sample data. Let it be noted that an array-based data structure was used (when needed) in order to quickly access data. Now, let us go through the system that was implemented and discuss the algorithms used. Initialize System

Note that here the input and output space(s) are divided into fuzzy regions (Step 1). The first step in this system is to initialize the program variables. This is accomplished by reading an experiment file, which contains the critical parameters of the DESIGN algorithm. The values are read in this manner as to simplify the running of experiments. The variables are (in order of appearance in file):

the number of system inputs the domain information for each domain, such as -The coarseness of fuzzy sets (N) distribution of domain, i.e. how many fuzzy sets cover the domain. -Lower bound (min),being the lower bound of the domain. -Upper bound (max), being the upper bound of the domain.

Now the sample data is read from a file in vector form, i.e. (xl,x2,... ,x,, y), where xi represents the input and y represents the output. The data structure which holds the sample data, also contains the rule that is generated from the data, the truth value of the rule along with an active variable that indicates if the rule is being used.

Generate Rules (Step 2)

The second most constructive step, is to generate rules from the sample data. This step essentially loops through all data constructing rules. A rule is formed by checking each respective domain for the fuzzy set that best describes the data (i.e. has the highest membership in the set). This membership value is stored in order to compute the truth of the rule, which is the product of all membership values.

Eliminate Rules (Step 3)

This procedure is composed of two nested for-loops, which compare a rule to all other rules. Two criteria can disable a rule: (1) the two rules have the same antecedent, yet the consequent differs, or (2) two rules are exactly the same. Those rules that are deemed not useful (i.e. inconsistent, yet ruled out by higher truth rules) or duplicates are not needed and thus are set inactive. Cleanse Rule Base

This step moves the rules from an array-based data structure to a pointer- based data structure. The array is littered with unused rules which is the size of the original sample data set. A pointer-based approach to represent the rule base is used because of its simplicity and is also compatible with the representation used in OPTIMIST,

Check Rule Base

This procedure checks the rule base for consistency in representing all input space domains. If a fuzzy set that is defined in a particular region is not present in the rule base, a warning message is displayed warning the user. In general, the greater the coarseness of each fuzzy region, the more data is needed to generate an appropriate system. A designer should always start with a low value of coarseness, and slowly increase till a warning message is displayed indicating that some fuzzy sets in the input domain were not present in the sample data.

Save Fuzzy System Definition

Finally, the generated fuzzy system definition is saved, which is composed of critical fuzzy system information, such as number of system inputs, the number of rules present in the rule base, the span of each fuzzy region along with the coarseness, and of course all of the fuzzy rules. The format of a fuzzy system definition is listed in Appendix E.

4.2. 1 .2 STEPS IN USING DESIGN

The following steps are used in creating a fuzzy model from sample data:

1. Gather and prepare the data. Perform any preprocessing. Save the data in ASCII-format files. 2. Create a learning and a training set. Split the data set into learning and testing sets randomly. 3. Import the data into DESIGN'S internal format. 4. Invoke the program's synthesis procedure which deterministically processes the sample data and generates a fuzzy model. The test data set is not referenced in any way during the synthesis of the fuzzy model. 5. Evaluate the test set. The system automatically runs the fuzzy model on the test data to evaluate its performance and exports a fuzzy system definition.

4.2.2 OPTIMIST

This algorithm performs a non-deterministic search through all fuzzy system

definitions, beginning with an initial "guess" derived from the DESIGN

system.

Experiment Fuzzy System Definition

-- I Training (Selection) Data I ' OPTIMIST LC-1 'i Testing Data

1 Cross Validation Data i

Improved Fuzzy System Definition Figure 4.2 File input and output in OPTIMIST

Now let us discuss the data structures and functions used within OPTIMIST. Data Structures

In this system, we turn to pointer-based data structures to aid with the numerous complex structures needed to represent a population of fuzzy systems and the data that acts as the selection criteria (pressure). The population was represented as an array of fuzzy system pointers. A fuzzy system is essentially an initial block of information detailing its fitness and rank within the population and a pointer to a list of rules. These rules are nothing more than a list of antecedent fuzzy set pointers and one consequent fuzzy set pointer. Once again, we see that the common structure used throughout this system is a fuzzy set. Since we are using tournament selection, a structure to maintain the current groupings (of size 4) was needed. Other than the group structure, no other data structures were needed to handle reproduction and mutation facilities.

To curtail the usage of global variables, a structure call Information was created to handle all of the system state variables. In order to preserve information contained within the fuzzy system definition, the domain information is stored in a structure similar to that in DESIGN. Since their are several types of testing data used in this system, a structure to contain input (m inputs) and output data is also in place. Now, let us examine the system that was implemented and discuss the algorithms implemented, along with their side effects.

Initialize System

The first step initializes all system state variables and allocates memory, after loading the experiment file, which contains the critical parameters of the genetic algorithm. These values are (in order of appearance in file):

*Random seed *Size of population *Maximumnumber of generations *The reproduction to be used (0 indicates mutation only, 1 indicates crossover, 2 indicates best only chromosome propagation) *The maximum amount to shift fuzzy sets in mutation *The probability of mutation *The probability of crossover *The probability of linear shift mutation (note that all mutation probabilities should equal 1.O) *The probability of expansion mutation @Theprobability of contraction mutation *The probability of nonlinear mutation

Build Initial Population

Now the initial population is built by first loading the fuzzy system definition that was derived from DESIGN into the Initial fuzzy system structure. The initial fitness of the Initial is computed using the Selection data (see Figure 4.2), then the Initial fuzzy system is copied to the Best and Temp fuzzy system structures. Now the initial fuzzy system is propagated throughout the population by allocating memory for each fuzzy system and copying Initial.

Compute Fitness of Population

Here each fuzzy system in the population is exposed to the training data (selection criteria) and several error ratings are computed (RMS Error, Mean Squared Error, Average Absolute Error and Maximum Absolute Error). Note only RMS Error is used as selection criteria.

Rank Population

Each fuzzy system is ranked according to its fitness (with rank 0 indicating the best fuzzy system). This is accomplished using a crude selection sort (finding the minimum of the population iteratively). Since the rank information is stored internally to each fuzzy system, it becomes a burden to find the fuzzy system with a particular rank. Thus, an array of the size of the population is used to store individual's labels, with the index into the array indicating rank.

Interim Testing

To indicate current system status, several variables are printed out (such as generation, fitness of the best fuzzy system thus far, and etc.). The best fuzzy system is also tested for its ability to predict using a separate data set (aside from the training (selection) data). Reproduction

Reproduction is based upon the type indicated in the experiment file, with 0 indicating mutation-only, elitism=lO, a 1 indicating crossover and 2 indicating best-only propagation.

If best-only propagation is used, copies of the best fuzzy system fill the entire population. Then the function Mutate Population goes through the entire population of fuzzy systems, mutating the fuzzy sets of a fuzzy systems' rules with probability of occurrence determined by the state variable, Probability of mutation.

4.2.2.2STEPS IN USING OPTIMIST

The following steps are used in optimizing a fuzzy systems:

1. Gather and prepare the data. Perform any preprocessing, saving the data in ASCII-format files. The same data used in DESIGN can be used in this procedure. 2. Create a selection (training) and testing data set. 3. Load the fuzzy system definition. 4. Invoke the optimization procedure. 5. Evaluate the test set. The system automatically runs the fuzzy model(s) on the testing data to evaluate the performance and saves the best fuzzy system definition.

The final prototype consists of two elements:

(1) DESIGN size of application: 58K lines of code: 2 143

(2) OPTIMIST size of application: 7 1K lines of code: 3770 88

These two programs were written using Metrowerks Codewarrior Compiler version 1.3 on a Performa 6 1 15CD Power Macintosh @ 66 MHz equipped with 32 Megabytes of RAM under MacOS 7.5.3. The actual required space for execution is dependent upon the size of the fuzzy systems generated.

Therefore this system can handle thousands (e.g. 6000) of sample data vectors and large fuzzy systems (with inputs > 6). This system handles problems that even commercial software cannot facilitate. EMPIRICALRESULTS AND

The work reported herein has three objectives. The first two of these are to demonstrate the technical feasibility of the fuzzy design methodology presented in Chapter 3 and to show its results against other methodologies. A third objective, is to examine the various mutation factors with possible impact on the methodology's effectiveness through a series of experiments.

The case studies are presented in the context of real-world problems: function approximation, classification, regression and prediction of chaotic dynamics. 90

5.1 FUNCTIONAPPROXIMATION AND PREDICTION

Function approximation (i.e. fitting curves) is a very important problem in

many areas, such as forecasting, control, CAD (lines and surface generation to

visualize objects in space), and data compression (viewed in a terms of a

function approximation problem, such as transmitting 6000 data points

uncompressed, or 6 fuzzy rules). Typical tools are artificial neural networks,

polynomials and splines. Here we present a solution to function

approximation in the form of a fuzzy model. A single-input single-output

Figure 5.1 The fifth-degree polynomial (SISO) system approximates a fifth-degree polynomial:

y = 3x(x- I)@-1.9)(x+0.7)(x+1.8) where x E [-2..2], (graphed in Figure 5. l), with a fuzzy model.This curve has been selected because it was used by Julie Dickerson (Dickerson 1994)

(Dickerson 1993) as a means to develop an automated fuzzy system design methodology (see Chapter 2, section 3.1).

Three sets of experiments are required to identify characteristics to which the methodology is sensitive. This is a limited number of experiments, since it is not practical to describe "typical" or "average" curves of characteristics. Thus, no claim is made that the experiments and the curve utilized is representative.

It is recognized that the results which follow are limited to the range over which these factors have been manipulated. The first set of experiments will examine the starting solution (from DESIGN) for: accuracy, fuzzy set densities over input domains, number of examples, and speed. The second set of experiments will focus on the optimization routine, OPTIMIST, by noting the effect of various mutation operators on the evolution of a fuzzy system definition., along with a simple comparison among the two models of

evolution implemented (Mutation and Best-Only Propagation). The third set

of experiments begins the evolutionary optimization procedure from a 92 randomly defined fuzzy system definition. Finally, we benchmark our method against Kosko's approach to fuzzy system design using ellipsoidal patches.

Sample data is randomly generated from the function using the program listed in Appendix D, which randomly selects x values from the above equation. This sample data contains example sets of independent variables (x, the input) and corresponding dependent variables (y, the output). Note that the random number generator used begins with a seed value, thus if the seed is known, one can regenerate the data set. In the following experiments, this deterministic approach to conducting the fuzzy system design experiments will be used by noting the data set size and seed that generated the data set.

The training data is composed of 100 points derived from (seed = 2). The testing data was also 100 points derived from (seed = 4). The training data was used to generate an initial fuzzy system, while the testing data was used to test prediction ability of the fuzzy system. The DESIGN system was run several times with differing values to determine the best DESIGN parameters. A quick analysis of the training data reveals the bounding regions of each fuzzy domain being:

input: [-2, 1.91, and output: [-13,9.4].

Since the number of rules is determined by the number of input sets (an upper bound), we must consider the fuzzy system size required (i.e. the rule count). If we can only use 25 rules, we should not produce a fuzzy model of the above system with an input domain N = 26, since this could produce a possible 26 rules.

First, we should decide upon the initial parameters of DESIGN for this system. Let us begin with the parameters

(Experiment 1) DESIGN (2) { Domain 0: (21, -2,2); Domain 1: (21, -16, 10); 1;

Recall, that Domain 0 refers to the system input, while the last domain, i.e. Domain 1 refers to the system output. Below in Figure 5.2 is a screen capture of the resulting fuzzy system definition mentioned above. Fuzzy System Engine.out -A Enter f i l ename of fuzzy sys tern def : FSb 1 -- loading fuzzy system definition from 'FSbl'. 1 system input.. . Domain 0: 1-2 21, composed of 21 sets Domain 1: [-16 101, composed of 21 sets rulebase loaded with 21 rules. Fuzzy System Definition: rule 0: IF 20 THEN 10 rule 1: IF 18THEN 7 rule2: IF 19THEN 7 rule3: IF 1THEN 18 rule4: IF 5THEN 14 rule 5: IF 12 THEN 15 rule 6: IF 2 THEN 20 rule?: IF 6THEN 13 rule 8: IF 14 THEN 14 rule 9: IF 11 THEN 14 rule 10: IF 13 THEN 15 rule 11: IF 4THEN 17 rule 12: IF 7THEN 12 rule 13: IF 17 THEN 9 rule 14: IF 16 THEN 11 rule 15: IF 0 THEN 9 rule 16: IF 3THEN 20 rule 17: IF 8THEN 11 rule 18: IF 10 THEN 13 - rule 19: IF 9THEN 12 -v rule20: IF 15 THEN 13 m Figure 5.2 This screen capture of a fuzzy system definition.

This system has 21 rules with an RMS error of 1.76 on the training data, with

an RMS error of 2.3 on the testing data. There were no warnings when

generating the fuzzy system, that would indicate that some fuzzy sets in the

input domain were not present in the rulebase, thus we can infer that there is

enough data to produce even more rules. This would increase the accuracy of

the fuzzy system performance, however at a cost of more fuzzy rules. Let us

now explore the possibility of increasing the granularity of fuzzy sets on both the input and output domains of the fuzzy system generated.

(Experiment 2) DESIGN (2) { Domain 0: (25, -2,2); Domain 1: (25, -16, 10); 1;

This system has 25 rules with an RMS error of 1.65 on the training data and an RMS error of 2.27 on the testing data. A further increase in granularity of both the input and output domains defined by the following fuzzy system definition:

(Experiment 3) DESIGN (2) { Domain 0: (27, -2,2); Domain 1: (27, -16, 10); 1; produces a system with 26 rules and an RMS error of 1.69 on the training data, while the testing data produced an RMS error of 2.29. Here a warning was issued:

Warning: fuzzy set 14 of input 0 not found in rulebase! indicating that fuzzy set 14 defined as a TFN(O,0.1429,0.2857) has no data in the input domain. This can be resolved by asking an expert or interpolating the empty region from surrounding data. A scattergram of the training data is depicted in the graph below (see Figure 5.3). Another method would be to decrease the value of N on the domain where the warning originated. Figure 5.3 Scattergram of training data composed of 100 examples

Thus we shall decrease N, from 13 to 12 only on the input domain. The fuzzy

system was generated with the following system definition:

(Experiment 4) DESIGN (2) { Domain 0: (25, -2,2); Domain 1: (27, -16, 10); 1;

This system has 25 rules an RMS error of 2.26 on the training data, with an RMS error of 2.76 on the testing data. Obviously this fuzzy model does not provide a good mapping of the input to the output. Since the second fuzzy model was the best thus far, we should pursue usage of those parameters in our final model. However, can we improve this model? Looking at the training data, it can be noted that the fuzzy sets begin where data is present. This data however has very little if any membership in the fuzzy sets on the end of domain. So, let's change the min and max parameters on both the input and output domain. The fuzzy system generated has the following definition:

(Experiment 5) DESIGN (2) { Domain 0: (25, -3,2.2); Domain 1: (25, -15, 1 1); 1;

This variation of the fuzzy model in experiment 2 produces 21 rules with an RMS error of 1.3 on the training data, with an RMS error of 1.45 on the testing data. This improvement is tremendous in comparison to the other fuzzy models generated with the same data. There were, however, some warnings issued:

Warning: fuzzy set 0 of input 0 not found in rulebase! Warning: fuzzy set 1 of input 0 not found in rulebase! Warning: fuzzy set 2 of input 0 not found in rulebase! Warning: fuzzy set 3 of input 0 not found in rulebase!

This is due to our over-extending the lower bound (min) on the input domain. Correcting this is easily done by looking at the bound of fuzzy set 3 (defined as TFN (-2.4, -2.2, -2)) and setting the min for the input domain to a value in that set (such as -2.2). This over-extension of the input domain has no effect on the performance of the fuzzy system, for these sets are just not mentioned in the fuzzy rulebase. Can we still improve the fuzzy model's performance? A parameter that we have not altered yet is the output domain granularity. What happens if we vary the granularity? The following fuzzy system definition contains an increased number of fuzzy sets on the output domain: (Experiment 6) DESIGN (2) { Domain 0: (25, -3,2.2); Domain 1: (27, -1 5, 1 1); >; which produces 21 rules with an RMS error of 0.754 on the training data, and an RMS error of 1.03 on the testing data. Now if we change the ~zirz parameter on the input domain to better match the sample data set, we see that this has a great effect on the resulting fuzzy system. The fuzzy system definition:

(Experiment 7) DESIGN (2) ( Domain 0: (25, -2.2,2.2); Domain 1: (27, -15, 11); >; which produces 23 rules with an RMS error of 0.915 on the training data, and an RMS error of 1.27 on the testing data. Here the fuzzy sets on the input have a width of 0.176, whereas in the previous experiment (6) the width is 0.208. This decrease in set width created a gap in the rulebase, thus decreasing fuzzy system performance.

The notion of increasing performance by increasing fuzzy sets (and thus rules) on the input domain has no solid evidence. However, if the main purpose is to extract as many rules as possible from data, the warning messages may be ignored. The consequence of gaps in the fuzzy system definition is that an input in an region where there are no rules results in an output of zero. An example of this can be noted in the following fuzzy system definition:

(Experiment 8) DESZGN (2) { Domain 0: (29, -2.2,2.2); Domain 1: (29, - 15, 11); 1; which produces 26 rules with an RMS error of 0.758 on the training data and an RMS error of 1.27 on the testing data. This generates a "partial" fuzzy system since some gaps exist in the input region (i.e. are not be included in the rulebase). There were three warnings issued:

Warning: fuzzy set 0 of input 0 not found in rulebase! Warning: fuzzy set 15 of input 0 not found in rulebase! Warning: fuzzy set 28 of input 0 not found in rulebase! here the first and last warnings are the extreme bounds of the region, whereas the only real concern is the region covered by fuzzy set 15.

Clearly the fuzzy system produced in experiment 6 performs better than those in other experiments. Thus, we shall use this fuzzy system definition as an initial fuzzy system model. Note that this series of experiments was only to illustrate the impact of the DESZGN parameters upon the resulting fuzzy system. That is this procedure should not consume time, since the improvement strategy used will improve the original (initial) fuzzy system. Let's now analyze the performance of this fuzzy system. First, we can see

the actual output versus predicted output in Figure 5.4.

1...#1,..,1,,.. I.., I.... ++ t t - t - - t - t t - * - + ++ + + + + .. a* " - t - + - + + -

+ +

- + + - - - - - +

I""I'"'1"" ""I"" - 2 0 -15 -1 0 - 5 0 5 10 actual

Figure 5.4 Actual vs, predicted performance over [-2..2],0.1 increment

Suppose now that we begin with less data, only 50 points. We will use the

same seed values to generate the data, seed=2 for 50 training points and

seed=4 for 50 testing points. A scattergram of the 50 training points is

depicted in Figure 5.4 below. Figure 5.5 Scattergram of reduced training set composed of 50 examples

Now let's begin with the same N value for both domains, just as in the first

experiment with the 100 point example file. The following fuzzy system definition,

(Experiment 1') DESIGN (2) { Domain 0: (21, -3,2.2); Domain 1: (21, -1 5, 11); 1;

produces 16 rules with an RMS error of 1.55 on the training data and RMS error of 1.7 on the testing data. However, there were five warnings, of which, 2 were serious. In order to eliminate these two warnings, we shall reduce the number of fuzzy sets on the input domain. The following fuzzy system definition,

(Experiment 2') DESIGN (2) { Domain 0: (19, -3,2.2); Domain 1: (21, -15, 11); >; produces 15 rules with an RMS error of 1.27 on the training data and RMS error of 1.56 on the testing data. This fuzzy system produced 4 warnings, of which, 1 was serious. Once again, we will reduce the number of fuzzy sets.

The following fuzzy system definition,

(Experiment 3') DESIGN (2) { Domain 0: (17, -3,2.2); Domain 1: (21, -15, 11); >; produces 13 rules with an RMS error of 2.26 on the training data and RMS error of 1.68 on the testing data. Unfortunately, this also produced 4 warnings, with only one serious warning. Reducing again, the fuzzy system definition.

(Experiment 4') DESIGN (2) { Domain 0: (15, -3, 2.2); Domain 1: (21, -15, 11); >; produces 13 rules with an RMS error of 1.86 on the training data and RMS error of 2.1 on the testing data. This fuzzy system produced no serious 103 warnings, only two boundary sets were not used in the rulebase. However, we cannot let the warning message count dictate which fuzzy system to use.

We must also consider the error rating and the intended use. It might be that the fuzzy system generated in experiment 2', which has the best RMS error, might be superior in prediction.

In general, all of the above experiments had the same computational requirements and characteristics. The fuzzy system definition was derived almost instantly (a few seconds), with the slowest component of the system being the screen output.

Now that we have an initial solution to modeling the system, we shall now investigate the improvement of the fuzzy system using OPTIMIST. First, we will investigate the various effects of mutation, then we will examine the various types of reproduction. Since we have four types of mutation (linear, expansion, contraction and nonlinear), we want to know which mutation best improves fuzzy system evolution.

Independent runs of all mutation operators was performed in order to observe each mutation's effectiveness on error reduction. No interactions studies were conducted among various mutation operator types. The first mutation operator we shall investigate is linear mutation. However, let us first note the parameters for OPTIMIST used:

OPTIMIST { Random seed = 5; Population size = 80; Generations = 1000; Model of evolution = mutation only, elitism = 10 (0); Maximum set shift = 0.05; Probability of mutation = 0.25; Probability of crossover = NIA; Mutation probabilities: linear = w; expansion = x; contraction = y; nonlinear = z; 1;

The variables w, x, y and z were the only parameters changed during the various test runs. In the linear mutation only testing, w was set to 1.0 and x, y and z was set to 0.0. The graph below in Figure 5.6 depicts the linear mutation operator over 1000 generations. Training Prediction

Figure 5.6 Linear mutation operator over 1000 generations

The second test considered only expansion mutation. Thus, x=1.0 and w, y and z were 0.0. Figure 5.7 below depicts the performance of this operator, followed by the contraction mutation in Figure 5.8. - Training - Prediction

Generations

Figure 5.7 Expansion mutation operator over 1000 generations

- Training - Prediction

Generations

Figure 5.8 Contraction mutation operator over 1000 generations Finally, we tested the nonlinear mutation operator. The results were remarkable, as can be noted in Figure 5.9 below.

- Training - Prediction

Generations

Figure 5.9 Nonlinear mutation operator over 1000 generations

Obviously, the best mutation operator to use is nonlinear mutation operator. Linear mutation performs better than the other two, but not as well as nonlinear. A possible explanation of why nonlinear mutation performs well, could be that it is the smallest possible operation on a set. For example, two nonlinear mutations could compose a single contraction mutation. The contraction and expansion mutations perform poorly in comparison to the two aforementioned techniques.

The performance of two different types of models of evolution implemented:

(1) mutation with elitism = 10 and (2) best-only propagation, was examined an there was not a big difference in performance. This however, needs to be verified in many different problem contexts.

In this single experiment, we evolve a randomly generated fuzzy system definition using OPTIMIST. The fuzzy system

OPTZMZST { Random seed = 5; Population size = 80; Generations = 1000; Model of evolution = mutation only, elitism = 10 (0); Maximum set shift = 0.045; Probability of mutation = 0.25; Probability of crossover = NIA; Mutation probabilities: linear = 0.0; expansion = 0.0; contraction = 0.0; nonlinear = 1.O; 1;

The fuzzy system performed poorly, as expected initially. After 500 generations, the resulting RMS error on 6000 random training points was 4.5, while the error on 1000 randomly generated prediction points was 1.8. In

Dickerson's PhD dissertation (Dickerson 1994), it is shown that a system that starts with a random initial solution takes longer time to converge to a good solution. In addition, Wang and Mendel (Wang et al. 1992) mathematically proved that their methodology (essentially DESIGN) approximates the 1 09 observed sample data with a higher relative accuracy. The same cannot be proved or assumed with a randomly generated system.

Kosko's fuzzy system design methodology, discussed in Chapter 2, section

3.1, was used to generate a fuzzy system model of the fifth-degree polynomial. The system used 1000 samples per rule to generate a six-rule fuzzy system (6000 sample vectors in total). Figure 5.10 below shows the accuracy achieved with (a) depicting the function approximation and the ellipsoidal patches after unsupervised learning, (b) depicting the function approximation and the ellipsoidal patches after 50,000 iterations or supervised learning, and (c) depicting the optimal function approximation found with least squares and the Nelder-Meade algorithm (simplex search based method) with the Matlab routine fmins (Matlab Student Manual 1992).

The training and testing files consisted of 6000 (seed = 109) and 1000 (seed =

105) random examples, respectively, that were generated using the program listed in Appendix D. Using the following DESIGN parameters: DESIGN { Domain 0: (6, -3,2.2); Domain 1: (6, -1 5, 11); >; we designed a fuzzy system that achieved an initial MSE of 6.97 (RMS error of 2.64) on the training data, while it achieved an initial MSE of 0.534 (RMS error of 0.731) on the testing data. The six initial rules are as follows:

rule 0: IF x is in TFN(-2.257,-1.5 14,-0.7714) THEN y is in TFN(3.571,7.286,11) rule 1: IF x is in TFN(-0.02857,0.7143,1.457) THEN y is in TFN(-0.1429,3.571,7.286) rule 2: IF x is in TFN(-1.5 14,-0.7714,-0.02857) THEN y is in TFN(-3.857,-0.1429,3.571) rule 3: IF x is in TFN(0.7143,1.457,2.2) THEN y is in TFN(-7.571,-3.857,-0.1429) rule 4: IF x is in TFN(-3,-2.257,-1.514) THENy is in TFN(-15,-11.29,-7.571) rule 5: IF x is in TFN(-0.7714,-0.02857,0.7143) THEN y is in TFN(-3.857,-0.1429,3.571)

Essentially, these six rules were derived from six sample vectors. This

"information compression" technique extracts the most truthful data sets in terms of fuzzy sets predefined over all domains. This initial solution however can be improved to more appropriately model the actual (supposedly unknown) system. -40 -2 0 2 X (4 Figure 5.10 Dickerson-Kosko method of function approximation using six rules (with Permission)

Now let us improve this initial fuzzy system. The OPTIMIST parameters are

as follows:

OPTIMIST { Random seed = 5; Population size = 50; Generations = 2000; Model of evolution = mutation only, elitisin = 10 (0); Maxi~nunlset shift = 0.025; Probability of mutation = 0.25; Probability of crossover = NIA; Mutation probabilities: linear = 0.0; expansion = 0.0; contraction = 0.0; nonlinear = 1.O; 1;

The optimized fuzzy system definition is as follows:

Rule 0: IF x is in TFN(-1.95,-1.5,-0.586) THEN y is in TFN(4.68,7.38,13.2) Rule 1: IF x is in TFN(-0.245,0.705,1.36) THEN y is in TFN(-1.66,3.53,7.15) Rule 2: IF x is in TpFN(-1.29,-0.393,-0.398,0.0704) THEN y is in TpFN(-5.49,-0.149,-0.148,2.35) Rule 3: IF x is in TFN(0.78,1.46,1.83) THEN y is in TFN(-9.34,-3.87,-2.51) Rule 4: IF x is in TFN(-2.9,-2.28,- 1.6) THEN y is in TFN(-17.5,-11.3,-11.3) Rule 5: IF x is in TpFN(- 1.09,-0.0405,-0.0404,0.342) THEN y is inTFN(-4.38,-0.152,1.75)

After 1500 generations the RMS error was 0.771 (MSE of 0.595) on the training data, while an RMS error of 0.301 (MSE of 0.09) on the predeiction data. To examine the methodology's abilites further, we injected noise (p=O, o=0.1) into 6000 training points and then generated a fuzzy system definition from this data using the following DESIGN parameters:

DESIGN { Domain 0: (6, -3,2.2); Domainl: (6,-15, 11); 1;

The fuzzy system designned achieved an initial MSE of 8.87 (RMS error of

2.98) on the training data, while it achieved an initial MSE of 1.29 (RMS error of 1.13) on the testing data. Using the same OPTIMIST parameters as above, we see that after 1500 generations the MSE error was 2.57 (RMS of 1.6) on the training data, while an MSE error of 0.325 (RMS of 0.57) on the predeiction data. Classification is the most prominent and basic learning task. A classification task involves that a specific pattern of observations are associated with a specific class (also denominated by a pattern recognition perspective). A set of samples contains the data that the learning system (e.g. neural networks) will utilize to find the generalized decision rules for the classifier. This data is based on features (also called attributes). The benchmark data set used to examine the classification abilities of the proposed approach is Fisher's Iris data (Fisher 1936). This is a well-known benchmark utilized in the pattern recognition community. The data set contains characteristics of three flowers,

Iris Setosa (class I), Iris Versicolor (class 2) and Iris Virginica (class 3). Each group contains four descriptors of each flower, petal (1) length and (2) width, sepal (3) length and (4) width. Thus, a fuzzy system would have four inputs, with a single output denoting the class to which the input is a member. In this section we shall compare the fuzzy system methodology against neural networks based on the back-propagation (Rummelhart et al. 1986) and a new technique that used hierarchical polynomials (including multiple linear regression). This innovative technique was developed by extensive research and marketed by Abtech Corp. as Abductive Modeling (AIMTMUser

Manual). This section shall be briefly explained in the following section preceeding the experiments. Abductive modeling uses mathematical functions to represent complex and

uncertain relationships, and networks to break the problem to manageable size.

The abductive networks are feed-forward networks of powerful polynomial

functions. By integrating advanced regression techniques with networking

concepts from neural networks, the Abductive Information Modeler (AIMTM)

enables the development of an accurate, robust, and real-time systems for

complex applications (AIMTMUser's Manual). The model emerging from the

AIMTMsynthesis is a robust and compact transformation implemented as a

layered network of feed-forward functional elements as shown in Figure 5.1 1.

Normalizers First Layer Second Laver Third Laver Unitizers

Double -- I Input-a B

3 Double I

Figure 5.11 An abductive network

The functional element coefficients, and the connectivity are learned inductively. AIMTM has following seven different types of nodes (or elements); the algebraic form of these elements is (Montgomery 1989):

1. Singles: Wo + (w,x,>+ (~2':) + 6473x3

2. Doubles:

3. Triples:

where XI,'2, and x3 are node inputs and the w,'s are the coefficients learned by AIMTM.

Singles, doubles and triples are elements whose names are based on the number of input variables. Note that these elements are third degree polynomial equations and that the doubles and triples have cross-terms, allowing interaction among the element input variables.

4. White Elements: (~1'1) + (~2'2) + (~3'3) + ... + (wnxn) The white element consists of the linear weighted sums of all the outputs of the previous layer. A white element has as many inputs as there are outputs in the previous layer.

5. Normalizers: Wo + (~1'1)

Normalizers transform the original input variables into a relatively common region with a mean of zero and a variance of one using mean- sigma normalization. 6. Unitizers: w, + (~1x1)

A unitizer converts the range of the network outputs to a range with the mean and variance of the output values used to train the network.

7. Wire Elements: The wire element is used for a network that consists only of a nonnalizer and a unitizer.

The output of any given element can feed subsequent layers, together with the original input variables. Networks are synthesized from layer-to-layer until the network model ceases to improve. In synthesizing each layer, every type of element is computed and scored for all combinations of eligible inputs (e.g.,

After testing each independent variable alone, polynomials of all pairs, then all triplets are tested). The best polynomials found in the first pass are further combined into polynomials of polynomials. This process is repeated for several levels in a search for the best polynomial solution (i.e., hierarchical modeling). At each stage, the goodness of fit of the model to the learning set is measured (very similar to a evolutionary computation algorithm). A complexity penalty which is small for simple expressions and large for complicated expressions is added to the fitting square error (FSE), rewarding simple expressions and penalizing complicated ones (to develop a tradeoff between training and generalization errors). Therefore, the modeling criterion,

Predicted Square Error (PSE) could be expressed as follows:

PSE = FSE + KP l I8 where KP is the term utilized to penalize complex networks and could be further decomposed as:

KP= CPM *-2K 2 N S~ where CPM is a complexity penalty multiplier, K is the total number of network coefficients, N is the number of training samples, and si is an a prior1 estimate of the optimal model's error variance based on each variable.

Building a model with AIMTM involves four simple steps: importing a database of examples, synthesizing a model, analyzing its performance, and using the resulting model. The user interface leads the user through this process logically. Once the user has imported a database, and the user specifies specific input and unused-input, and one-output parameter, AIMTM automatically develops abductive network models.

The data was divided into two data sets, training was composed of 100 sample vectors, whereas the testing had 50 sample vectors. The training data had the following characteristic bounds:

sepal length: [4.3,7.9] sepal width: [2, 4.41 petal length: [I, 6.91 petal width: [0.1, 2.51 with the output domain requiring at least three distinct regions (or sets) to classify the three types of flowers. The first fuzzy system definition we shall try has the following parameters:

(Experiment 1) DESIGN (4) { Domain 0: (1 l,4, 8); Domain 1: (1 1,2,4.5); Domain 2: (1 1, 1, 7); Domain 3: (1 1,0,2.5); Domain 4: (3, - 1,3); 1;

This fuzzy system possessed 86 rules and had an RMS error of 0.302 on the training data and 0.48 on the testing data. More importantly, it classified 42 out of the 50 data vectors of the testing data set. This fuzzy system displays good performance considering this was the first attempt at modeling this system. There were, however, four warnings issued. This is not crucial in classification systems, but is of concern. The fuzzy system consistently classified class 1 correctly. The eight misclassified points, were members of class 2 and 3, thus we can say that class 1 was successfully distinguished. It is well known that class 1 is rather separated from the other two classes, with the other two classes NOT linearly separable from each other. Note that thus far, the fuzzy system generated has not been improved. Let's see what happens when we optimize this fuzzy system. The following OPTIMIST parameters define the experiment: OPTIMIST { Random seed = 5; Population size = 40; Generations = 200; Model of evolution = mutation only, elitism = 10; Maximum set shift = 0.05; Probability of mutation = 0.25; Probability of crossover = NIA; Mutation probabilities: linear = 0.0; expansion = 0.0; contraction = 0.0; nonlinear = 1.O; 1;

After 200 generations the fuzzy system had an RMS error of 0.100058 on the training data and 0.331931 on the testing data, while classifying 45 out of the

50 vectors.

The next attempt at producing a fuzzy classifier, focused on eliminating some warnings, which might possible improve performance. The fuzzy system definition,

(Experiment 2) DESIGN (4) { Domain 0: (15,4, 8); Domain 1: (7,2.1,4.5); Domain 2: (9,0.8,7); Domain 3: (9,0,2.6); Domain 4: (3, - 1,3); >; proved successful. This fuzzy system possessed 83 rules (less than in the first experiment) and had an RMS error of 0.108 on the training data and 0.256 on 121 the testing data. This system classified 45 out of the 50 data vectors (2 of the misclassifications being uncertain classifications) of the testing data set, with only two warnings. As in the previous experiment, class 1 was consistently classified.

We then used the OPTIMIST routine to improve this fuzzy system, the parameters were the same as before. After 200 generations the fuzzy system had an RMS error of 0.100 on the training data and 0.238 on the testing data, while correctly classifying 47 out of the 50 vectors.

Note that the three output fuzzy sets are centered on three distinct points: 0, 1, 2. The prediction displays a unique feature of fuzzy systems in classification, for example, in one test vector was a member of class 2 (set centered on I), but the prediction was midway between class 2 and 3 (1.73). This quantifies

vagueness between membership of specific classes, since it depicts the vagueness of classification between two classes and inherently eliminates the first class. In summary, the fuzzy system designed in these experiments shows the diverse ability of the fuzzy systems.

One neural network was utilized to compare approaches, using the same training and testing data. The architecture consisted on four input units, four hidden neurons, and a single output. AIMTMwas also trained and tested with the same data. The results are as follows:

The initial fuzzy solution (from experiment 2) confussion matrix:

Class 1 Class 2 Class 3

Class 1 16 1 1

Class 2 15 2

Class 3 1 14

The final fuzzy solution (from experiment 2) confusion matrix:

Class 1 Class 2 Class 3

Class 1 16 1

Class 2 16 1

Class 3 1 15

The neural network confusion matrix:

The abductive modeling confusion matrix: Class I Class 2 Class 3

Class 1 16

Class 2 16 1

Class 3 1 16 This data set is a benchmark for statistical algorithms: the prediction of

Boston housing prices from 13 factors (Breiman et al.). This data set contains

506 exemplars. This data is considered very noisy, "but have only mild nonlinearity" (Moody andyarvin, 1992). The data consists of 14 variables measured for each of 506 census tracts in the Boston area. These variables, by tract, are as follows:

y: median value of homes in thousands of dollars (MV) xl : crime rate (CRIM) x2: percent of land zoned for lots (ZN) x3: percent of nonretail business (INDUS) xq: 1 if on Charles River, 0 otherwise (CHAS) x5: nitrogen oxide concentration, pphm (NOX) xg: average number of rooms (RM) x7: percent built before 1940 (AGE) xs: weighted distance to employment centers (DIS) x9: accessibility to radial highways (RAD) xlo: tax rate (TAX) xl 1: pupillteacher ratio (PIT) ~12:percent black (B) XI 3: percent lower-status population (LSTAT) The solution obtained by regression is represented as follows:

log(MV) = a1 + a2 (RM)2 + a3 (AGE) + a4 log(DIS) + as log(RAD) + a6 (TAX) + a7 (PIT) + a8 (B-63)2 + a9 log(LSTAT) + a10 (TAX) + a1 1 (ZN) + a12 (INDUS) + a13 (CHAS) + a14 (N0X)b + E where b is a parameter to be estimated. This data set has been successfully modeled using a least squares regression equation after performing various transformations on both the dependent (y) and independent variables, see

(Breiman 1984). The same points that were used in the regression tree by

Breiman: xl (CRIM), x6 (RM), x8 (DIS), xi3 (LSTAT). The training and testing data are composed of 380 (75%) points and 126 (25%) points, respectively.

A quick analysis of the training data reveals the bounding regions of each fuzzy domain being:

xl bounded by [0.01,74] X6 bounded by [3.9,8.8] x8 bounded by [1.1,12] x13 bounded by [1.7,37] y bounded by [5.6,50] The first fuzzy system definition we shall try has the following parameters:

(Experiment 1) DESIGN (4) { Domain 0: (9, 0,75); Domain 1: (1 1, 3.85, 9); Domain 2: (1 1, 1, 12.1); Domain 3: (1 1, 1.65, 38); Domain 4: (1 1, 5.5, 5 1); >;

This fuzzy system possessed 182 rules and had an RMS error of 4.18 on the training data and 3.28 on the testing data. This fuzzy system generated only one warning from the 380 examples. The improved fuzzy system (using the parameters to OPTIMIST below) exhibited an RMS error of 3.44 on the training data and 3.28 on the testing data (after 90 generations).

OPTIMIST ( Random seed = 5; Population size = 50; Generations = 100; Model of evolution = mutation only, elitism = 10; Maximum set shift = 0.025; Probability of mutation = 0.25; Probability of crossover = NIA; Mutation probabilities: linear = 0.0; expansion = 0.0; contraction = 0.0; nonlinear = 1.0;

Neural networks and abductive modeling were also trained with the same training and testing data. Please note the table below for the results:

trainin rediction trainin rediction

MSE 29.84 16.1 11.945 13.35 , AvgAbsErr 3.41 2.52 2.56 2.75 2.34 1.26

1 -MaxAbsErr-. 2 6 11 17.83 10.56 29.3 27.5 Predicting the future is the primary encouragement behind the search for laws that explain certain phenomena (Chakraborty et al. 1992). Weigend et al.

(1990) observed that the problem of predicting the future is based on two types of knowledge: (I) knowledge of underlying laws, and (2) the discovery of strong regularities (i.e. periodicities) in observations of a given system.

Time series analysis is an important statistical tool to study the behavior of time-dependent and forecast future values depending on the history of variations in the data. A time-series is a sequence of values measured over time, in discrete or continuous time units. This is a practical problem with applications found in such areas as economic modeling, inventory control, weather forecasting, signal processing and control. 0 x(t)

Figure 5.13 A section of the Mackey-Glass chaotic time series

Here we apply our approach to fuzzy system design to the Mackey-Glass

chaotic time-series (see figure 5.14 above) (Lapedes et al. 1987). The

Mackey-Glass chaotic time series is generated from the following delay

differential equation:

where t > 17.

The Mackey-Glass time-series is a good benchmark because it has a simple definition and its elements are hard to predict due to the chaotic

characteristics. The solution will be compared against a neural network based

on backpropagation with 40 hidden-units and abductive modeling. Let z(k)

with (k = 1, 2, 3, ... ) be a time series. The problem of time-series prediction

can be formulated as: given z(k-m+l), z(k-m+2), ... , z(k), determine z(k+l),

where m and 1 are fixed positive integers, i.e. determine a mapping from [z(k-

m+l), z(k-m+2) ,... , z(k)] E %m to z(k+l) E 3.

We chose m = 9 and 1 = 1 in to model the system, just as in (Wang 1992), i.e.

nine points in the series were used to predict the value of the next point.

The training data is x(501) to x(700), where the testing data was x(701) to

~(1000).

The DESIGN parameters were as follows:

DESIGN ( 10) { Domain 0: (7,0.3, 1.5); Domain 1: (7, 0.3, 1.5); Domain 2: (7,0.3, 1.5); Domain 3: (7,0.3, 1.5); Domain 4: (7,0.3, 1.5); Domain 5: (7,0.3, 1.5); Domain 6: (7,0.3, 1.5); Domain 7: (7,0.3, 1.5); Domain 8: (7,0.3, 1.5); Domain 9: (7,0.3, 1.5); 1; The result of this initial fuzzy system were excellent. This can be noted in

Figure 5.15. The RMS error was 0.0311 (MSE of 0.000965) on the training data, while it had an RMS error of 0.0428 (MSE of 0.00183) on the testing data. Below is a section of the rule base developed, where the antecendent is composed of fuzzy set labels (not actual fuzzy sets, but representative thereof). Note the pattern within the set of rules, where the system

"discovered" regularities in the system. rule 128: IF 112222333THEN rule 129: IF 111222233THEN rule 130: IF 1111 2 2 2 2 3THEN rule 131: IF 011112222THEN rule 132: IF 1OOOOO111THEN rule 133: IF 11OOOOO11THEN rule 134: IF 111OOOOO1THEN rule 135: IF 2111OOOOOTHEN rule 136: IF 2 2 11100 0 OTHEN rule 137: IF 322111OOOTHEN rule 138: IF 3 3 2 2 1110OTHEN rule 139: IF 33322111OTHEN rule 140: IF 433322111THEN rule 141: IF 443332211THEN rule 142: IF 444333221THEN rule 143: IF 444433322THEN rule 144: IF 444443332THEN rule 145: IF 4 4 4 4 4 4 3 3 3THEN rule 146: IF 4 4 44 444 3 3THEN rule 147: IF 444444443THEN Figure 5.14 Initial fuzzy system's absolute error

- actual - predicted

Figure 5.15 Initial fuzzy system's prediction performance The initial fuzzy system was then improved using OPTIMIST with the following parameters:

OPTIMIST { Random seed = 5; Population size = 50; Generations = 75; Model of evolution = mutation only, elitism = 10; Maximum set shift = 0.025; Probability of mutation = 0.25; Probability of crossover = NIA; Mutation probabilities: linear = 0.0; expansion = 0.0; contraction = 0.0; nonlinear = 1.O; >;

After 75 generations the best fuzzy system at prediction was discovered with an RMS error of 0.0165 (MSE of 0.000272) on the training data, while it had an RMS error of 0.0332 (MSE of 0.0011) on the testing data. After 75 generations the prediction error diverges, thus not exhibiting good prediction ability. Figure 5.16 lmproved fuzzy system's absolute error

- actual - predicted

Figure 5.17 lmproved fuzzy system's prediction performance Neural networks and abductive modeling were also trained with the same training and testing data. Please note the table below for the results:

NN AIM FS training prediction training prediction training prediction 0.01085 0.038979 0.0084 0.0083 0.0165 0.0332 0.0001 18 0.001 5 19 0.000071 0.00007 0.000272 0.001 1 AvgAbsErr 0.008132 0.028599 0.0007 0.007 0.0129 0.0302 0.030135 0.083621 0.019 0.019 0.063 -- - .- -- 0.092 AND

The major focus of this research was to develop a fuzzy system design methodology that possessed robustness and computational efficiency. This methodology utilized two major elements: (1) a variation of the Wang and

Mendel methodology, and (2) an evolutionary computation based optimization procedure. This methodology was tested using a prototype and benchmark against real-world problems. In this chapter, conclusions of this research effort are presented. The chapter also describes the contributions of this research in the field of Computational Intelligence and its applications. It closes by discussing future improvements to both the methodology and the prototype. There are several conclusions to be drawn from this work. In general, the fuzzy system design methodology developed through this research showed that a combined approach using an excellent initial solution with an optimization / improvement scheme can be applied to fuzzy system design.

An initial solution based on learning by examples provides an approach that differs from the existing systems that begin with random starting solutions.

The variation of the Wang and Mendel method offers speed, learning capabilities and convenient ways to represent knowledge. On the other hand, the optimization / improvement mechanism represented by the evolutionary computation method developed provides a simple, yet robust strategy in improving fuzzy systems within both static and dynamic environments. The fuzzy system design methodology is the first attempt to integrate the elements mentioned above.

More specifically, the fuzzy system design methodology was benchmarked using many different problems. This was further extended by comparing the fuzzy systems designed against both neural networks and abductive modeling.

In the first case study, we considered the problem of function approximation and showed that a functional fuzzy system can be improved using this 138 methodology. Not only did we show the usefulness, we benchmarked the methodology obtaining good results against the Dickerson-Kosko approach.

In the second case study, we considered classification in order to demonstrate the usefulness of fuzzy systems in various problem contexts. The fuzzy system designed had a 94% correct classification rate. Neural networks are suited more for this application. However, the methodology provides an inference engine that explains how the system derived the output from the provided input.

In the third case study, we considered a standard benchmark for regression to examine the methodology's ability to predict using small exemplar data sets.

This particular problem was modeled best by the fuzzy system designed, in comparison to both neural networks and abductive modeling.

Finally, we considered the prediction of chaotic dynamics using the Mackey-

Glass time-series, which is very popular amongst the neural network community. The fuzzy system designed performed well, as well as the neural network and abductive modeling.

Overall, the fuzzy system design methodology allows one to produce a fuzzy 139

model of many problem contexts. As with other paradigms it is always good

to compare techniques to note the abilities and weaknesses of various

techniques. We can conclude that the approach developed in this research is

another tool in the industrial engineering arsenal to support their multiple

modeling activities.

The fuzzy system implemented within this thesis is kind of a caricature

because I have kept the mechanisms (fuzzy logic, rule structure, etc.) simple

and quite primitive. This simplicity however, has an elegance about itself in

describing complex systems, thus allowing an understanding of complicated

dynamical processes and systems. This is however in comparison to modem

techniques, such as the use of partial differential equations to model systems,

which underly most of today's technology.

6.3 LIMITATIONSAND PROBLEMS

The present implementation of the fuzzy system design methodology consists

of two independent techniques. 140

6.3.1 PROBLEMSIN GENERAL

The method by which a fuzzy system is defined has a crude and basic structure. The code that was written has not been optimized or analyzed to note possible points within its structure that are inefficient (in both memory and execution speed measures).

6.3.2 PROBLEMSWITH DESIGN

The fuzzifier cannot accept rules that contain only a subset of possible input variables, since the fuzzy system definition has a fixed format. Aside from the problems inherent within the fuzzification process, the design system needs only "good" data, thus a proper analysis and preprocessing of the data (similar to that in neural networks) needs to be done. In running several hundred runs of the DESIGN procedure, I realized a change in the resultant fuzzy system performance when the min and max bounds were changed slightly for each system input. This could be alleviated with the addition of a small genetic algorithm to evolve the parameters to get the best possible initial fuzzy system. Each experiment must be now specified independently. It would be convenient to be able to perform batch processing on several different experiments to investigate other possible fuzzy system configurations.

This thesis provides a sound foundation for further research in automated fuzzy system design strategies. The neutral fuzzy system definition allows other tools to be added in the design stage. An advantage to DESIGN is that it is very easy to modify the fuzzy system definition as new data becomes available. When a new data vector becomes available, a new rule can be created and then added to the fuzzy system definition. This updating procedure can be then used for prediction. Possible additions to the methodology:

An interface module that allows experts to add fuzzy rules to existing system.

*An updating mechanism that creates rules and eliminates conflicting ones.

*Formation of species-Using Darwin's continent cycle theory, we could segregate the individual fuzzy systems into small randomly chosen populations. This would prove, in terms of long term improvement of a population, that an individual who is suboptimal in one sense may have something that will be useful later on after it is crossed over with other individuals.

*An ensemble approach using MacTCP to evolve a population of fuzzy system design techniques (such as artificial neural network approaches), where the individual computers are controlled via simple message passing. Include a model explanatory facility, since fuzzy models are extremely difficult to explain.

Include a rule editor to the design methodology, where one could enter expert knowledge in the form of linguistic (fuzzy) rules and specify if they are volatile or not (i.e. can be altered by future techniques). This would however require an intuitive GUI.

Recall Cooper and Vidal's work (Cooper et al. 1994), which recognizes that some fuzzy rules may possess irrelevant input variables. A possible improvement would allow fuzzy rules which only contain a subset of the actual input variables.

Experiments conducted in this research form just a single branch of analysis. An in-depth study of the performance of the methodology under different conditions should be initiated (e.g. a formal design of experiments).

The development of procedures to allocate trials to each mutation operator to improve speeds of convergence (also known as the n-armed bandit problem (Goldberg 1989)).

Developments of a better scheme to split the population of fuzzy systems, i.e. adaptive elitism, this would also improve the convergence.

New developments in computing technology call for the implementation of the methodology in a networked environment (e.g. agent-based). Concurrent operation of several modules would be possible. This would require a solution of other problems, such as synchronization and data consistency, possible modification to the algorithms. This section contains my sources of information utilized in this thesis.

Extensive use of internet resources and personal contact was used to answer specific questions and keep informed about current developments in the area of fuzzy logic, evolutionary computation and fuzzy system design techniques. Bezdek, James C. "Computing with Uncertainty." IEEE Communications Ma~azine.September 1992, pp24-36.

Bernard, J.A. "Use of rule-based system for process control," IEEE Contr. Syst. Mag., Vol. 8, No. 5, pp3-13, 1988.

Brubaker, David I. "Fuzzy Centroids and Singletons," Huntington Technical Brief; November 1991, No. 20.

Brubaker, David I. "Fuzzy Rules and Membership Functions from Data," Huntington Technical Brief; July 1993, No. 40.

Brubaker, David I. "Fuzzy Tools," Huntington Technical Brief; November 1994, No. 56.

Buhusi, Catalin V. "Adding Dynamic Rules to Self-organizing Fuzzy Systems." Proceedings of the North American Fuzzy Information Processing, Mexico, December 1992, Vol. 2, pp360-367.

Chakraborty, K., K. Mehrothra, C. Mohan, S. Ranka. "Forecasting the behavior of Multivariate Time Series Using Neural Networks," Neural Networks, Vol. 5, No. 6, pp961-970.

Ciliz, K., J. Fei, K. Usluel, and C. Isik, "Practical aspects of the knowledge-based control of a mobile robot motion," in Proceedings of the 30th Midwest Symposium on Circuits and Systems, Syracuse, NY, 1987.

Cooper, Mark G. and Jacques J. Vidal. "Genetic Design of Fuzzy Controllers". The Second International Conference on Fuzzy Theory and Technology, 1993.

Cox, Earl. The Fuzzy Svstems Handbook: A Practitioner's Guide to build in^ Using. and Maintaining Fuzzy Svstems. Academic Press, New York, 1994.

Daugherity, Walter C., Balaji Rathakrishnan, and John Yen. "Performance Evaluation of a Self-Tuning Fuzzy Controller." Proceedings of the 1992 IEEE International Conference on Fuzzy Systems. San Diego, March, Vol. 1, pp389-397.

Dayhoff, Judith E. Neural Network Architectures: An Introduction. Van Nostrand Reinhold, New York, 1990.

Dickerson, Julie A. and Bart Kosko. "Fuzzy Function Learning with Covariance Ellipsoids." Proceedings of the 1993 International Joint Conference on Neural Networks, San Fransisco, Vol. 3, pp1162-1167.

Dickerson, Julie A. "Fuzzy Function Approximation with Ellipsoidal Rules." Ph.D. dissertation, University of Southern California, Los Angeles, CA, November 1993.

Dickerson, Julie A. and Kosko, Bart. "Fuzzy Function Approximation with Supervised Ellipsoidal Learning." Proceedings of the 1993 International Neural Networks Society, July, Vol. 2, pp9- 17.

Fisher, R. A. "The Use of Multiple Measurements in Taxonomic Problems." Annals of Eugenics, 7, 1936, pp 179-188.

Fogel, David. discussions UseNet beginning Wed, 27 Jan 1996 23:35:29 GMT within the newsgroup comp.ai.genetic, concerning evolutionary computation entitled "Classification of an Evolutionary Algorithm," originally posted by the author.

Fujitec, F. "FLWX-8800 series elevator group control system," Fujitec Co., Ltd., Osaka, Japan, 1988.

Gen, Mitsuo, Kenichi Ida, and Chunhui Cheng. "Multirow machine layout problem in fuzzy environment using genetic algorithms." Proceedings of the 17th International Conference on Computers and Industrial Engineering, pp5 19-523, 1995.

Gen, Mitsuo, Kenichi Ida, Yinzhen Li, and Erika Kubota. "Solving bicriteria solid transportation problem with fuzzy numbers by a genetic algorithms." Proceedings of the 17th International Conference on Computers and Industrial Engineering, pp537-541, 1995. Goldberg, David E. Genetic Al~orithmsin Search. O~timization.and Machine Learning. Addison-Wesley, Reading, Massachusetts, 1989.

Goldberg, David E. "Making Genetic Algorithms Fly: A Lesson from the Wright Brothers," Advanced Technologyfor Developers. Vol. 2, February 1993.

Grabot, B. and L. Geneste. Dispatching Rules in Scheduling: A Fuzzy Approach. International Journal of Production Research. Vol. 32, No. 4, 1994, pp903-915.

Holland, John H. Hidden Order: How Adaptation Builds Complexity. Addison-Wesley, Reading, Massachusetts, 1995.

Holland, John H. posted to UseNet Wed, 13 July 1994 13:07:35 MET DST on the newsgroup comp.ai.genetic.

Ishibuchi, Hisao, Ken Nozaki and Naohisa Yamamoto. "Selecting Fuzzy Rules by Genetic Algorithm for Classification Problems." Proceedings of the Second IEEE Conference on Fuzzy Systems. San Fransisco, March 1993, Vol2, pp1119-1124.

Isik, C. "Identification and fuzzy rule-basd control of a mobile robot motion," in Proceedings of the IEEE Int. Symposium on Intelligent Control, Philadelphia, PA, 1987.

Isik, Can and Farrukh Zia. "Fuzzy Logic Control Using a Self-organizing Map." Proceedings of the 1993 International Neural Network Society, July, Vol. 2, pp56-65.

Itohm, O., K. Gotoh, T. Nakayama, and S. Takamizawa, "Application of fuzzy control to activated sludge processes," in Proceedings of the 2nd IFSA Congress, Tokyo, Japan, July, 1987, pp282-285.

Kaufmann, Arnold, and Gupta, Madan M., Introduction to Fuzzy Arithmetic: Theory and Applications, Van Nostrand Reinhold, New York, 1985. Karr, Charles L. "Analysis and Optimization of an Air-injected Hydrocyclone." Ph.D. dissertation, University of Alabama, Tuscaloosa, AL, 1989.

Karr, Charles L. "Design of an Adaptive Fuzzy Logic Controller Using a Genetic Algorithm." Proceedings of the Fourth International Conference on Genetic Algorithms. San Diego, July 1991, pp450-457.

Karr, Charles L. "Genetic Algorithms in Adaptive Fuzzy Control." Proceedings of the 1992 North American Fuzzy Logic Processing Society. Mexico, December 1992, Vol. 2, pp506-514.

Karr, Charles L. and Edward J. Gentry. "Fuzzy Control of pH Using Genetic Algorithms." IEEE Transactions on Fuzzy Systems. Vol. 1, No. 1, February 1993.

Kasai, Y ., and Y. Morimoto "Electonically controlled continuous variable transmission," in Proceedings on the Int. Congress of Transportation Electronics, Dearborn, MI, 1988.

Kashiwagi, Dean T., William C. Moor. "Application of backward chaining, "fuzzy logic," and the management of information to procurement of facility systems/services ." Proceedings of the 17th International Conference on Computers and Industrial Engineering, pp399-403, 1995.

Khan, E., Venkatapuram, P. NeuFuz: Neural Network Based Fuzzy Logic Design Algorithms. Proceedings of the FUZZ-IEEE93, pp647-654, Vol. 1, March 1993.

Khedkar, Pratap S. and Hamid R. Berenji. "Generating Fuzzy Rules with Linear Consequents from Data." Proceedings of the 1993 International Neural Network Society, July, Vol. 2, pp 18-21.

Kickert, W.J.M. and H.R. Van Nauta Lemke, "Application of a fuzzy controller in a warm water plant," Autmat. Vol. 12, No. 4, pp. 301-308, 1976.

Kinoshita, M., T. Fukuzaki, T. Satoh, and M. Miyake "An automatic operation method for control rods in BWR plants," in Proc. Specialists' Meeting on In-Core Instrumentation and Reactor Core Assesment, Cadarache, France, 1988.

Kosko, Bart. "Stochastic Competetive Learning." Proceedings of the 1990 International Joint Conference on Neural Networks, San Diego, Vol. 2, pp215-226.

Kosko, Bart. Neural Networks and Fuzzy Svstems. Prentice-Hall, Englewood Cliffs, 1992.

Kosko, Bart. "Fuzzy Function Approximation." Proceedings of the 1992 International Joint Conference on Neural Networks, Vol. I, pp209-2 13.

Krishnakumar, K. S. "Micro genetic algorithms for stationary and nonstationary function optimization." Proceedings of the 1989 SPIE Intelligent Control and Adaptive Systems Conference. Philadelphia, 1989, pp289-296.

Krucky, Jan. "Fuzzy Family Setup Assignment and Machine Balancing." Hewlett-Packard Journal, June 1994.

Lapedes, A. and R. Farber, "Nonlinear signal processing using neural networks: prediction and system modeling,'' LA-UR-87-2662, 1987.

Larking, L. I. "A fuzzy logic controller for aircraft flight control," in Industrial Applications of Fuzzy Control, M. Sugeno, Ed. Amsterdam: North-Holland, 1985, pp87-104.

Larsen, P. M. "Industrial applications of fuzzy logic control," Int. J. Man Mach. Studies, Vol. 12, No. 1, pp3-10, 1980.

Lee, Chuen Chien. "Fuzzy Logic in Control Systems: Fuzzy Logic Controller Part I." IEEE Transactions on Systems, Man, and Cybernetics. Vol. 20, No. 2, March 1990, pp404-418.

Lee, Chuen Chien. "Fuzzy Logic in Control Systems: Fuzzy Logic Controller Part 11." IEEE Transactions on Systems, Man, and Cybernetics. Vol. 20, No. 2, March 1990, pp419-435. Lee, Michael A. and Hideyuki Takagi. "Integating Design Stages of Fuzzy Systems using Genetic Algorithms." Proceedings of the Second IEEE International Conference on Fuzzy Systems. New York, NY: Institute of Electrical and Electronics Engineers, pp612-617.

Mamdani, E. H. and S. Asssilian, "An experiment in linguistic synthesis with a fuzzy logic controller," Int. J. Man Mach. Studies, Vol. 7, No. 1, ppl-13, 1975.

Masud, Abu S.M., Edwin B. Dean. "Using Fuzzy Sets in Quality Function Deployment." Proceedings of the 2nd Industrial Engineering Research Conference. pp270-274, 1993.

The Student Edition of Matlab, The Mathworks, Inc., Prentice-Hall, Englewood Cliffs, New Jersey, 1992.

Mcneill, Daniel, and Freiberger, Paul, Fuzzy Logic, Simon and Schuster, 1992.

Montgomery, G., "Abductive Diagnostics," American Institute of Aeronautics and Astronautics, Inc., 1989, by AbTech Corporation.

Moody, John, and Norman Yarvin. "Networks with learned unit response functions." Advances in Neural Information Processing Systems: Proceedings of the 1990 Conference. pp1048-1055, 1990.

Murakami, S."Application of fuzzy controller to automobile speed control system," in Industrial Applications of Fuzzy Control, M. Sugeno, Ed. Amsterdam: North-Holland, 1985, pp 105- 124.

Nguyen, Hung T., Michio Sugeno, Richard Tong, and Ronald R. Yager (editors). Theoretical Aspects of Fuzzv Control. John Wiley & Sons, Inc.., New York, 1995.

Ostergaad, J.J. "Fuzzy logic control of heat exchange process," in Fuzzy Automata and Decision Processes, M.M. Gupta, G.N. Saridis, and B.R. Gaines, Eds. Amsterdam: North-Holland, 1977, pp285-320.

Pappis, C.P., and E.H. Mamdani, "A fuzzy logic controller for a traffic junction," IEEE Trans. Syst. Man Cybern., Vol. SMC-7, No. 10, pp707-7 17, 1977.

Rumelhart, D. E. G.E. Hinton, R.J. Williams. Learning internal representation by error propagation. In Parallel Distributed Processing, Rumelhard, McClelland and the PDP Group, Chapter 8, pp3 18-362, 1986. Cambridge, Massachusetts: MIT Press.

Sakai, Y. "A fuzzy controller in turning process automation," in Industrial Applications of Fuzzy Control, M. Sugeno, Ed. Amsterdam: North-Holland, 1985, pp139-152.

Scharf, E.M. and N.J. Mandic, "The application of a fuzzy controller to the control of a multi-degree-freedom robot arm," in Industrial Applications of Fuzzy Control, M. Sugeno, Ed. Amsterdam: North-Holland, 1985, pp4 1-62.

Shehdeh, Hana and Lea, Robert N. "A Genetic Algorithm Approach for Altering the Membership Functions in Fuzzy Logic Controllers," NAFIPS Vol. 2, 1992.

Sin, Sam-Kit and Rui J. P. deFigueiredo. "Fuzzy System Design through Clustering and Optimal Predefuzzification." Proceedings of the Second IEEE Conference on Fuzzy Systems. San Fransisco, March 1993, Vol. 1, pp190-195.

Simpson, Patrick K. "Fuzzy Min-Max Neural Networks-Part 2: Clustering." IEEE Transactions on Fuzzy Systems. Vol. 1, No. 1, February, 1993, pp32-45.

Slany, Wolfgang. "Fuzzy Constraint Relaxation Techniques for Knowledge-Based Scheduling." Proceedings of the First European Congress on Fuzzy and Intelligent Technologies, Hans-Jiirgen Zimmerman (ed.), Sept. 1993, Aachen, Germany, pp1124-1127.

Sugeno, M. and K. Murakami "An experimental study on fuzzy parking control using a model car," in Industrial Applications of Fuzzy Control, M. Sugeno, Ed. Amsterdam: North-Holland, 1985, pp125-138. Tanscheit, R. and E.M. Scharf, "Experiments with the use of rule-based self-organising controller for robotics applications," Fuzzy Sets Systems, Vol. 26, pp195-214, 1988.

Thrift, Philip. "Fuzzy Logic Synthesis with Genetic Algorithms." Proceedings of the Fourth International Conference on Genetic Algorithms. San Diego, July 199 1, pp509-5 13.

Togai, M. and H. Watanabe "Expert system on a chip: An engine for real-time approximate reasoning," IEEE Expert Syst. Mag., Vol. 1, ~~55-62,1986.

Togai, M. and S. Chiu "A fuzzy accelerator and a programming environment for real-time fuzzy control," in Proceedings of the 2nd IFSA Congress, Tokyo, Japan, July 19 87, pp 147- 15 1.

Tong, R. M., M. B. Beck, and A. Latten, "Fuzzy control of the activated sludge wastewater treatment process," Automat. Vol. 16, No. 6, pp695-701, 1980.

Tsujimura,Yasuhiro, Seung Hun Park, In Seong Chang and Mitsuo Gen. "An Effective Method for Solving Flow Shop Scheduling Problems with Fuzzy Processing Times." Computers and Industrial Engineering, Vol. 25, No 1-4. pp239-242, 1993.

Uragami, M., M. Mizumoto, and K. Tananka, "Fuzzy robot controls," Cybern., Vol. 6, pp39-64, 1976.

Umbers, I. G. and P.J. King, "An analysis of human-decision making in cement kiln control and the implications for automation," Int. J. Man Mach. Studies, Vol. 12, No. 1, pp11-23, 1980.

Wang, Li-Xin "Analysis and Design of Fuzzy Systems." Ph.D. Dissertation, University of Southern California, Los Angeles, CA, 1992.

Wang, Li-Xin and Jerry M. Mendel. "Generating Fuzzy Rules by Learning from Examples." IEEE Transactions on Systems, Man, and Cybernetics, Vol. 22, No. 6, 1992, pp1414-1427. Watanabe, H. and W. Dettloff "Reconfigurable fuzzy logic processor: A full custom digital VLSI," in Int. Workshop on Fuzzy System Applications, Iizuka, Japan, Aug. 1988, pp49-50.

Weigend, A., D. Rumelhart, and B. Huberman, "Back-Propagation, Weight-Elimination and Time Series," Proceedings of the 1990 Summer School, pp105-116.

Werbos, P. J., The Roots of Backpro~arration:From Ordered Derivatives to Neural Networks and Political Forecasting, 1994, John Wiley & Sons, Inc.

White, Ray H. "Learning Fuzzy Rules with Competitive Hebbian Learning 2." Proceedings of the International Neural Networks Society, San Diego, June 1994, Vol. 1, pp743-748.

Yagishita, O., 0. Itoh, and M. Sugeno. "Application of fuzzy reasoning to the water purification process," in Industrial Applications of Fuzzy Control, M. Sugeno, Ed. Amsterdam: North-Holland, 1985, pp 19-40.

Yamakawa, T. and T. Miki "The current mode fuzzy logic intregrated circuits fabricated by the standard CMOS process," IEEE Trans. Computer, Vol. C-35, No. 2, pp161-167, 1986.

Yamakawa, T. and Sasaki, K."Fuzzy memory device," in Proceedings of the 2nd IFSA Congress, Tokyo, Japan, July 1987, pp551-555.

Yamakawa, T. "Fuzzy Microprocessors-Rule chip and defuzzifier chip," in Int. Workshop on Fuzzy Systems Applications, Iizuka, Japan, Aug. 1988, pp51-52.

Yamakawa, T. "A simple fuzzy computer hardware system employing min and max operations-A challenge to the 6th generation computer," in Proceedings of the 2nd IFSA Congress, Tokyo, Japan, July 1987.

Zadeh, Lotfi A. "Fuzzy Sets." Information & Control, Vol. 8, 1965, pp338-353.

Zadeh, Lotfi A. "Outline of a New Approach to the Analysis of Complex Systems and Decision Processes." IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-3, No. 1, January 1973, pp28-43.

Zadeh, Lotfi A. "Fuzzy sets as a basis for a theory of possibility," Fuzzy Sets Syst., Vol. 1, pp3-28, 1978.

Zadeh, Lotfi A., "Syllogistic reasoning in fuzzy logic and its application to usuality and reasoning with dispositions," IEEE Trans. on Systems, Man, and Cynern., Vol. SMC-15, No. 6, pp754-763, 1985.

Zurada, Jacek M. Artificial Neural Svstems. West Publishing Co., New York, 1992.

AIM User's Manual, AbTech Corporation, Charlottesville, Virginia.

L. Breiman et. al, Classification and Regression Trees, Wadsworth and Brooks, 1984, pp217-220. DESIGN SOURCE

/*------The first step in the prototyped system, Mobius title: DESIGN-A Fuzzy System Design Method based upon a variation of Wang and Mendel's work date: August 1995 file: design.^ author: Brian 0. Bush components: design.c design.h language : Metrowerks v1.3 C for the PowerPC 60x RISC processors system: My ~acintoshPowerPC Performa 6115CD purpose : Research for my Thesis

Copyright O 1996 by Brian 0. Bush, all rights reserved ...... * /

#define debug 0 #define kMaxTrainingVectors 6000 #define kMaxTestVectors 3000 #define kunity 1.0

RulePtr gRuleBase; DomainPtr g~omain[kMaxInputs +I]; DatumPtr g~ata[kMax~rainingVectors]; Test gTestingData[kMaxTestVectors]; Information gInf o ; double CalculateMu(fl0at input, FuzzySetPtr Set); void CheckRuleBaseO; void CleanseRuleBaseO; void ComputeRuleTruth(); void CreateSets(D0mainPtr Cryogen); void Create~ntervals(shortN, double min, double max, ~omain~trdomain); void DetermineBounds ( ) ; void ~isplaySet(FuzzySetPtrx, Boolean Verbose); void ~isplayInterval(FuzzySetPtrx); void EliminateRulesO; void GenerateRulesO; void Ini tRun ( ) ; void LoadExperimentO; void LoadTrainingDataO; void LoadTestData ( ) ; void PrintData ( ) ; void PrintFS ( ) ; void SaveFS ( ) ;

//------/ / // Main / / //------void main() {

InitRun ( ) ; DetermineBoundsO;

GenerateRules ( ) ; // step 2 ComputeRuleTruthO; EliminateRules ( ) ; // step 3 CleanseRuleBaseO; CheckRuleBaseO; SaveFS ( ) ; PrintFS ( ) ;

, , / / // Initialize runtime variables and allocate memory / / //------void InitRun ( )

FILE *fp; char filename [311 ; short N, CurDom; double min, max ; printf("\nEnter filename of experiment:"); scanf("%s',filename); if ((fp = fopen(filename,"r"))== NULL printf("\nError: cannot open experiment file"); exit (1); 1 printf("\nloading experiment from '%s'.\n\nH,filename); fscanf(fp, "%hdl',&gInfo.NumInputs); printf("\n%hd system inputs defined",gInfo.NumInputs); if(gInfo.NumInputs > kMaxInputs) ( printf ( " \nError: Too many inputs ! " ) ; exit (1); // exit program! } for (CurDom=O; CurDom < (gInfo.NumInputs + 1) ; CurDom++) { gDomain[CurDom] = (Domain~tr)NewPtr(sizeof(D0main) ); if(gDomain[CurDom] != nil) I gDomain[CurDom]->Label = CurDom; gDomain [CurDoml ->N = 0 ; gDomain[CurDom]->LowerBound = 0.0; g~omain[ CurDom] ->UpperBound = 0.0; gDomain[CurDom]->SetHalfWidth = 0.0; gDomain [CurDom]->Set = nil; 1 else i printf ( " \memory Error : Init Run ( ) " ) ; exit (1); // exit program! 1 1 for (CurDom=O; CurDom < (gInfo.NumInputs + 1); CurDom++) ( // Step 1 fscanf (fp,"%hd %lg %1gN,&N, &min, &max) ; printf("\nDomain: %hd \tN:%hd \tmin:%lg \tmax:%lg",CurDom,N,min,max); CreateIntervals (N, min, max, gDomain [CurDom]) ; // main part of step 1 1 fclose (fp);

/ / // CreateIntervals divides the input and output spaces into // fuzzy regions, step 1 of Wang and Mendels Fuzzy Sytem Design Methodology / / //------void CreateIntervals(short N, double rnin, double max, DomainPtr domain) I domain->N = N; domain->LowerBound = min; domain->UpperBound = max;

// compute set half width, the width of half of a uniform fuzzy set domain->SetHalfWidth = ((domain->UpperBound - domain->LowerBound)/(domain->N + 1.0));

CreateSets(d0main); // now create N fuzzy sets in fuzzy region i

//------/ / // Load sample data vectors from file specified by user / / //------void LoadTrainingDataO ( FILE *fp; char filename [ 3 1 I ; double temp; short 1; printf("\nEnter filename of training data:"); scanf ("%s",filename); if ((fp = fopen(filename,"r"))== NULL ) { printf("\nError: cannot open training file"); exit (1); } printf("\nloading training data from '%s'.\n\nW,filename) ; gInfo .NumVectors = 0;

for(i=O; iDataVector[i]); // INPUT fscanf(fp,"%lg",&gData[gInfo.NumVectorsl-~DataVector[gInfo.NumInputsl); gInfo.NumVectors++; I, else { printf("\nError: Allocation memory failed in LoadTrainingDataO "); exit (1); 1 1 while (!feof(fp) && (gInfo.NumVectors < kMaxTrainingVectors)); printf("\n%hd sample vectors loaded",gInfo.NumVectors); fclose (fp); 1

//------/ / // Print sample data vectors / / void PrintData ( ) { short i,j ;

/ / // determine the bounds of the system input data / / void DetermineBoundsO short i,j; double lower, upper;

lower = ~in(gData[O]->DataVector[i],g~ata[l]->~ata~ector[i]); upper = Max(g~ata[O]->~ataVector[i],~~ata[ll->~ata~ector[i]);

for(j=2; jDatavector [i], lower) ; upper = Ma~(~~ata[jl->Data~ector[il,upper); } printf ("\nDomain%hd bounded by [%1.21g,%1.21gl", gDomain [ i I ->Label, lower, upper) ; 1 I

//------/ / // Displayset prints the four defining points of a TpFN to screen / / //------void ~isplaySet(FuzzySetPtr Set, Boolean verbose) { if (Set != nil) { if (Verbose) { if (Set->b == Set->c) printf (" %hd = ~FN(%1.31g,%1.31g,%l.31g)", Set->Label, Set->a, Set->b, Set->d);

else printf(" %hd = ~pFN(%1.3lg,%l.3lg,%1.31g,%1.31g)", Set->Label, Set->a, Set->b, Set->c, Set->d);

else { if(Set->Label < 10) printf(" %hdU,Set->Label); else printf (" %hdl',Set->Label) ; 1 } else printf("\nError accessing fuzzy set definition"); >

/ / // Display Interval walks the linked list of a domain's fuzzy sets / / //------void DisplayInterval(FuzzySetPtr x) { print£ ( " \nfuzzy region definition") ; if(x != nil) i for(y=x; y!=nil; y = y->next) { printf('\nset: %hd ",y->Label); if (x->b == X->c) // TFN printf("TFN(%1.4lg,%1.41g,%1.41g)",Y->a,~->b, Y->~); else // TpFN printf ("TpFN(%1.4lg,%1.41g,%1.41g,%l.41g) " ,y->a, Y->b, Y->C, y->d); } printf ( " \n") ; else printf("\n Error accessing fuzzy set definition");

//------. / / // Createsets allocates space for 2N + 1 fuzzy sets under a domain / / //------void CreateSets(D0mainPtr Cryogen) { short CurSet ; double a,b,c,d; double Scratch=O.O; FuzzySetPtr Temp, Head; // Head is the first fuzzy set

// compute first fuzzy set bounding points a = Cryogen->LowerBound; b = (a + Cryogen->SetHalfWidth); c = b; d = (a + (2 * Cryogen->SetHalfWidth));

Head = (FuzzySetPtr) New~tr(sizeof(FuzzySet)); Head->next = nil; for(CurSet=O; CurSetN; Curset++) { Temp = (FuzzySetPtr) NewPtr (sizeof(FuzzySet) ) ; if (Temp != nil) { Temp->Label = CurSet; Scratch = (CurSet * Cryogen->SetHalfWidth); Temp->a = (a + Scratch); Temp->b = (b + Scratch); Temp->c = (c + Scratch) ; Temp->d = (d + Scratch); Temp->next = Head->next; // insert node into list Head->next = Temp;

else { printf("Memory Error in Createset0 "); exit (1); }

//------/ / // Compute the Degree of truth for each rule, part of step 3. / / //------void ComputeRuleTruth() t short Curvector, CurElement; for(CurVector=O; CurVector < gInfo.NumVectors; Curvector++) I gData[CurVectorl->Truth = 1; for(CurElement=O; CurElement < (gInfo.NumInputs + 1); CurElement++) gData[CurVector]->Truth *= g~ata[CurVector]->MuVector[CurElement]; 1

//------/ / // Calculate Mu computes the truth value of the statement // "x is in A" where A is a Trapezoidal Fuzzy Number (a superset // to Triangular Fuzzy Numbers) defined by four points (a,b,c,d) / / //------double CalculateMu(f1oat Input, FuzzySetPtr Set) I double Output=O .O, MidLow=O.O, HighMid=O.O; if ( (Input <= Set->a) I / (Input >= Set->d)) Output = 0.0; else { if(1nput > Set->c) Output = ((Set->d - Input)/HighMid); else { if((1nput >= Set->b) && (Input <= Set->c)) Output = 1.0; else Output = (Input - Set->a)/MidLow; return Output;

//------/ / // Eliminate conflicting and equal rules from rulebase / / //------void EliminateRules ( ) { short CurVector, VectorX, CurDom, NumTrashedRules=O;

Boolean conflicting = false; for(CurVector=O; CurVector < gInfo.NumVectors; Curvector++) { if(gData[CurVector]->Active) ( for(VectorX=O; VectorX < gInfo.NumVectors; VectorX++) if((Cur~ector!= VectorX) && g~ata[VectorX]->Active&& gData[CurVectorl->Active) I

for(CurDom=O;CurDom < gInfo.NumInputs; CurDom++) { if(gData[CurVectorl->Rule[CurDoml == gData[VectorXl->Rule[CurDoml) conflicting = true; else ( conflicting = false; break; ) 1

if (conflicting) { if(g~ata[Curvectorl->Truth>= g~ata[~ector~] ruth) gData[Vector~]->Active= false; // rule VectorX eliminated

else gData[CurVector]->Active = false; // rule CurVector eliminated NumTrashedRules++; conflicting = false; ) 1 // else we ignore inactive rule CurVector

//------/ / // Generate fuzzy rules / /

void GenerateRules ( ) { short Curvector, CurDom; double MU ; FuzzySetPtr Set;

for(CurVector=O; Curvector < gInfo.NumVectors; Curvector++) { gData[CurVector]->Active = true; for (CurDom=O; CurDom < (gInfo.NumInputs + 1) ; CurDom++) I g~ata[CurVector]->MuVector[CurDom]= 0.0; gData[CurVector]->Rule[CurDom] = gDomain[CurDoml->Set->Label; for(Set = gDomain[CurDom]->Set; Set != nil; Set = Set->next) { Mu = ~alculate~u(g~ata[Cur~ector]->DataVector[CurDom],Set); if (Mu > gData[CurVector]-~MuVector[CurDom]){ gData[curVector]->Rule[CurDom] = Set->Label; gData[CurVector]->MuVector[CurDom] = Mu; 1

/ / // Check rulebase to determine if it is complete or not, i.e. an input fuzzy set // is not represented in the rulebase. / / //------void CheckRuleBase ( ) I RulePtr Temp ; short Dom, Set; Boolean found;

found = false;

for (Dom=O; Dom < gInfo.NumInputs; Dam++) // loop through all system inputs (if side)

for(Set=O; Set < gDomain[Dom]->N; Set++) i found=false; for(Temp = gRuleBase; Temp != nil; Temp=Temp->next) if (Temp->If[Doml == Set) ( found=true; break; 1 if(!found) I printf("\nWarning:fuzzyset %hd of input %hd not found in rulebase! " , Set,Dom) ; 1 1

//------/ / // Initialize rule base with rules from training session detaching // the data vectors from rules. / / //------

void CleanseRuleBase ( ) i short Dom, Possible, i, Set; RulePtr Temp ;

gRuleBase->next = nil; gInf o . Rulecount = 0 ;

for(Dom=O; Dom i g1nfo.NumVectors; Dom++) { if(gData[Doml->Active) i Temp = (RulePtr) NewPtr (sizeof(Rule) ) ; if (Temp != nil) { Temp->Label = g~nfo.RuleCount++;

// IF for(Set=O; Set < gInfo.NumInputs; Set++) { Temp->If [Set] = gData [Doml ->Rule[Set] ; 1

// THEN Temp->Then=gData[Dom]->Rule[Set]; Temp->next = gRuleBase->next; gRuleBase->next = Temp; I else { printf("Memory Error in CleanseRuleBase()"); exit(1); 1 1 1 gRuleBase = Temp; printf("\n%hd rules in rulebase",gInfo.~uleCount); for(i=O, Possible=l; i i gInfo.NumInputs; i++) Possible *= gDomain[il->N; printf ( "\t(Possible rule combinations=%hd)" ,Possible);

//------/ / // Print Fuzzy Rule Base / / //------void PrintFS () { short j ; FuzzySetPtr CurSet; RulePtr CurRule; printf ("\nThe fuzzy rule base generated from sample data.");

for(CurRule=gRuleBase;CurRule != nil; CurRule = CurRule->next) I printf("\nrule %hd:\tIFU,CurRule->Label); for(j=O; j < gInfo.NumInputs; j++) for(CurSet=gDomain[j]->Set;CurSet != nil; CurSet=CurSet->next) if(CurSet->Label == CurRule->If[jl) DisplaySet(CurSet,O); printf("\t\t\tTHENN); for(CurSet=gDomain[j]->Set; CurSet != nil; CurSet=CurSet->next) if(CurSet->Label == CurRule->Then) DisplaySet(CurSet,O); 1

//------/ / // Save the fuzzy system definition to file. / / //------

void SaveFS ( ) ( short j; RulePtr CurRule; FuzzySetPtr CurSet; FILE

if ((fp = fopen("Fuzzy System Definition","w")) == NULL ) ( printf("\nError: cannot open fuzzy system def file for writing"); return; )

for(CurRu1e = gRuleBase; CurRule != nil; CurRule=CurRule->next) // loop through rules t fprintf(fp,"%hd\n",CurRule->Label); for(j=O; j < gInfo.NumInputs; j++) // IF side r for(CurSet=gDomain[j]->Set; CurSet != nil; CurSet=CurSet->next) 1 if(CurSet->Label == CurRule->If[j]) fprintf(fp,"%hd $1.319 %1.31g %1.31g %l.31g\nU, CurSet->Label, CurSet->a, CurSet->b, CurSet->c, CurSet->d); // TpFN notation } 1 for(CurSet=gDomain[j]->Set; CurSet != nil; CurSet=CurSet->next) // THEN side 1 if(CurSet->Label == CurRule->Then) fprintf(fp,"%hd %1.31g %1.31g %1.31g %1.31g\nU, CurSet->Label, CurSet->a, CurSet->b, Curset->c, Curset->d); // TpFN notation

1 fclose (fp);

//------i/ EOF /*------The first step in the prototyped system, Mtjbius title: DESIGN-A Fuzzy System Design Method based upon a variation of Wang and Mendel ' s work date: ~ugust1995 file: design.h author: Brian 0. Bush components: design.c design.h language : Metrowerks v1.3 C for the PowerPC 60x RISC processors system: My Macintosh PowerPC Performa 6115CD purpose : Research for my Thesis

Copyright @ 1996 by Brian 0. Bush, all rights reserved ...... * /

//------/ / // FuzzySet defines the structure of a fuzzy set / / //------struct FuzzySet { short Label; // a unique ID for the set, generated by the machine double a, // four points define a trapezoidal fuzzy interval b, c, d; struct FuzzySet *next; 1 ; typedef struct FuzzySet FuzzySet; typedef FuzzySet *FuzzySetPtr; //------/ / // Domain defines the structure of Fuzzy Region / / struct Domain { short Label; short N; // number of fuzzy sets over region double LowerBound, UpperBound, SetHalfWidth; // half the width of a fuzzy set FuzzySetPtr Set; ? ; typedef struct Domain Domain; typedef Domain *DomainPtr; // Rule definintion

struct Rule { short Label; double Truth; short If[kMaxInputsl; short Then; struct Rule *next; > ; typedef struct Rule Rule; typedef Rule *RulePtr;

//------/ / // Data holds the training data / / //------struct Data { double DataVector[kMaxInputs + 11; double MuVector[kMaxInputs + 11; short Rule [kMaxInputs + 11 ; double Truth; // product of MuDataVector Boolean Active; 1; typedef struct Data Data; typedef Data *DatumPtr;

//------/ / // Testing Data / / //------struct Test{ double Input[kMaxInputsl; double Output; 1; typedef struct Test Test;

//------/ / // System status information / / //------struct Information I short NumVectors, NumInputs, Rulecount; >; typedef struct Information Information;

//------// EOF OPTIMIST SOURCE

/*------The second step in the prototyped system, Mobius title: OPTIMIST-An Improvement Strategy for Fuzzy Systems Using Evolutionary Computation date: November 19 95 file: 0ptimize.c author : Brian 0. Bush components: 0ptimize.c optimize.h compiler: Metrowerks v1.3 C for the PowerPC 60x RISC processors system: My Macintosh PowerPC Performa 6115CD purpose: Research for my Thesis

Copyright @ 1996 by Brian 0. Bush, all rights reserved ...... * /

#define kunity 1.0 #define kMaxPopulationSize 200 // must be a multiple of four #define kMaxValidationVectors 1000 #define kMaxTestVectors 6000 #define kMaxElite 10 #define debug 0

FILE *histfp;

SystemPtr gPopulation[kMaxPopulationSize+ll; // population of fuzzy system defs

SystemPtr gBestFS, // holds the best fuzzy system gInitialFS, // holds the initial guess fuzzy system gTempFS ; // temporary fuzzy system, used as temp in crossover

Group gGroupInfo[(kMaxPopulationSize/4) + 11; Domain gIntervals[kMaxSystemInputs + 11; Test gTestingData[kMaxTestVectors]; Test g~alidationData[kMaxValidationVectorsl; short g~ank~nfo[kMaxPopulationSizel; static Information gInfo;

void AlterSet(FuzzySetPtr Set, MutationType type); void BuildInitialFS(SystemPtr fs); void BuildInitialPop ( ) ; double CalculateMu(float Input, FuzzySetPtr Set); void Cleanup ( ) ; void ConsistencyCheck(doub1e lower, double upper, double increment); void ContractSet(FuzzySetPtr Set); void ContractBase(FuzzySetPtr Set); void ContractCore(FuzzySetPtr Set); double ComputeCentroid(FuzzySetPtr X, double Height); void ComputeFitness(SystemPtr fs) ; void ComputeFitnessOfPop(); void CopyFS(SystemPtr To, SystemPtr From); void DeleteRule(Ru1ePtr rule); void Defuzzification(SystemPtr fs); void DisplaySet(FuzzySetPtr x); void EliteSelectionO; void EliteReplacementO; void ExpandSet(FuzzySetPtr Set); void ExpandBase(FuzzySetPtr Set); void ExpandCore(FuzzySetPtr Set); void FinalTestingO; void FindMin(short x); void GenerateOffspring(short GroupNum); void GroupSelection(short NumParents); void GroupFormationO; void InitRun ( ) ; void InterimTestingO; void LoadExperimentO; void LoadFS ( ) ; void LoadTestDataO; void LoadValidationDataO; void LoadFinalTestDataO; void LocallyEliteReplacement(short group); void MutateFS(SystemPtr fs); void MutatePointA(FuzzySetPtr Set); void MutatePointB(FuzzySetPtr Set); void MutatePointC(FuzzySetPtr Set); void MutatePointD(FuzzySetPtr Set); void MutateSet(FuzzySetPtr Set); void Mutatepop () ; void NonlinearShiftSet(FuzzySetPtr Set); void PrintFS(SystemPtr fs); void PrintPopulationO; void PrintFSInfo(SystemPtr fs); void PrintPopInfo () ; void Printstatus () ; void PropagateBestFSO; void PropagateInitialFS(SystemPtr fs); void RuleEvaluation(SystemPtr fs); double RandomZeroOneO; void Rankpopulation(); void ReproductionO; void SaveFS (SystemPtr fs, char *s); void SelectParent(sh0rt group); void ShiftSet(FuzzySetPtr Set); void ~estPrediction(SystemPtr fs, Boolean Verbose); void WriteHistory (1 ;

//------/ / // Main / / //------void main0

~vent~ecordMainEventRec ; Boolean EventFlag;

Ini tRun ( ) ; Printstatus () ; BuildInitialPopO; while (gInfo.~ax~enerations> gInfo.Generation) {

FinalTestingO; SaveFS(gBestFS,"OptFS"); // save best fuzzy system definition and print out info printf("\nFinal RMS error with selection data is: %lgN,gBestFS->Fitness); printFS(gBestFS); Cleanup ( ) ;

//------/ / / / Load experiment / / //------void ~oad~xperimento FILE *fp; char filename[321 ; printf("\nEnter filename of experiment:"); scanf("%sV,filename);

if ((fp = fopen(filename,"r")) == NULL ) { printf ( " \nError: cannot open experiment file") ; exit (1); 1 printf("\nloading experiment from '%s'.\n\nl',filename);

fscanf(fp, fscanf(fp, fscanf (fp, fscanf(fp, fscanf (fp, fscanf(fp, fscanf (fp, fscanf (fp, fscanf (fp, fscanf (fp, fscanf(fp,

/ / // Initialize runtime variables and allocate memory for population / / //------

void InitRun ( )

short i,j;

if ( (histfp = fopen( "History", "w")) == NULL) { printf ("\nError : cannot open history file for writing" ) ; exit (1); i g1nfo.ReadTestingData = false; gInfo.~eadValidationData = false; gInfo .NumVectors = 0; g1nfo.NurnTestVectors = 0; gInfo.NumValidationVectors = 0; gInfo.NurnSystemInputS = 0; gInf o . Rulecount = 0 ; gInfo .Generation = 0; gInfo.BestPred = 999;

g1nitialFS = (SystemPtr) NewPtr(sizeof(System)); if (gInitialFS != nil) { gInitialFS->root = nil; g1nitialFS->Label = 0; gInitialFS->Rank = 0; g~nitial~S->Fitness= 0.0; gInitialFS->RMSErr = 0.0; gInitialFS->MSE = 0.0; g1nitialFS->AvgAbsErr = 0.0; gInitialFS->MaxAbsErr = 0.0; } else { printf("\nError: cannot allocate memory for initial FS"); exit (1); i gBestFS = (Systemptr) NewPtr(sizeof(System)); if(gBestFS != nil) { gBestFS->root = nil; gBestFS->Label = 0; gBestFS->Rank = 0; g~estFS->Fitness= 69.0; // init to a large value gBestFS->RMSErr = 0.0; gBestFS->MSE = 0.0; gBestFS->AvgAbsErr = 0.0; gBestFS->MaxAbsErr = 0.0; else { printf ( Iq\nError:cannot allocate memory for best FS") ; exit (1); i gTempFS = (SystemPtr) NewPtr(sizeof(System)); if(gTempFS != nil) { gTempFS->root = nil; gTempFS->Label = 0; gTempFS->Rank = 0; gTempFS->Fitness = 0.0; gTempFS-zRMSErr = 0.0; gTempFS->MSE = 0.0; g~empFS->AvgAbsErr= 0.0; gTempFS->MaxAbsErr = 0.0; i else { printf ("\nError: cannot allocate memory for temp FS") ; exit (1); i for(i=0;iroot = nil; gPopulation[il->Label = i; gPopulation[il ->Rank = 0 ; gPopulation[il->Fitness = 0.0; gpopulation [ i] ->RMSErr = 0.0 ; gPopulation[il->MSE = 0.0; gPopulation[i]->AvgAbsErr = 0.0; gPopulation[i]->MaxAbsErr = 0.0; gPopulation[il->GroupNum = 0; I else { printf("\nError in initializing population array"); exit(1) ; I I

//------/ / // Interim Testing / / //------. void InterimTestingO short i ; double sum; // sum of all RMS errors in population

// compute average RMS error of population for(i=O, sum=O; i~gInfo.NumOfPopulation; I++) sum += gPopulation[il->Fitness; gInfo.PopAvg = (sum / gInfo.NumOfPopulation);

printf("\n(%hd Prediction) RMSErr:%1.3lg MSE:%1.3lg AvgAbsErr:%1.3lg MaxAbsErr : %l.3lg" , gInfo.Generation, gBestFS->PredRMSErr, gBestFS->PredMSE, gBestFS->PredAvgAbsErr, gBestFS->PredMaxAbsErr);

//------/ / // Final Testing / / void FinalTesting ( ) I LoadFinalTestDataO; ~est~rediction(g~nitialFS,false); TestPrediction(gBestFS, false); printf("\nPredictability Results are:"); //------/ / // Clean up when finished with memory, files, etc. . . / / void Cleanup ( ) I

fclose (histfp);

//------/ / // Build Initial Population / / //------void BuildInitialPop ( ) I short i;

LoadFS ( ) ; // load intial guess ~om~uteFitness(g1nitialFS); ~uild~nitialFS(gBestFS); // build gBestFS and save gInitialFS to gBestFS ~uildInitialFS(gTempFS); // also build gTempFS computeFitness(gBestFS); ~estPrediction(gBestFS, false); // test gBestFS

for(i=0;i~gInfo.Num0fPopu~ation;i++) { PropagateInitialFS(gPopulation[i]); if (i>O) ~utate~~(g~opulation[i]);// mutate ail in population except first-elitism g~opulation[il->Label = i;

WriteHistory () ; // show initial state

// Show performance ratings of initial fuzzy system.. printf("\nThe Initial Fuzzy System has..."); printf("\n(Training Data) RMSErr:%1.3lg MSE:%1.31g AvgAbsErr:%1.3lg MaxAbsErr:%1.31gN, gBestFS->RMSErr, gBestFS->MSE, gBestFS->AvgAbsErr, gBestFS->MaxAbsErr); printf("\n(Prediction) RMSErr:%1.3lg MSE:%i.3lg AvgAbsErr:%1.3lg MaxAbsErr:%1.31g\nn, gBestFS->PredRMSErr, gBestFS->PredMSE, gBestFS->PredAvgAbsErr, gBestFS->PredMaxAbsErr); }

//------/ / // Write history of RMS error / / //------

void WriteHistory () { / / // Print status information

void Printstatus () ( printf("\nCritical system variables are:"); printf("\nRandom Seed is:%hdV,gInfo.RandomSeed); printf("\nNumber of Individuals in Population: %hd",gInfo.NumOfPopulation); printf("\nMaximum generations: %hdV,gInfo.MaxGenerations);

if(gInfo.Repr~ductionType== 0) printf("\n~utationOnly!"); else ( if (gInfo.ReproductionType == 1) printf("\nCrossover and ~utationused."); else printf ("\nBest only propagation used. ") ; I printf("\nMaximum Set Shift: %lg",gInfo.MaxSetShift); printf("\nProbability of mutation: %lg",gInfo.PMutation); printf("\nProbability of a Linear shift: %lg",gInfo.Plinear); printf("\n~robabilityof an Expansion: %IgM,gInfo.Pexpand); printf("\nProbability of a Contraction: %lg",gInfo.Pcontract); printf("\nProbability of a Nonlinear shift: %lg",gInfo.Pnonlinear);

//------/ / // Check the consistency of output on a specific domain, for single // input systems ... the fifth-degree polynomial / / //------

void ConsistencyCheck(double lower, double upper, double increment) { FILE *check; double error,interval;

check = fopen("check.datN,"wv;

if (upper > lower ) 1 for(interval=lower; interval 0) I I (gInfo.Predicted <= 0)) fprintf(check,"\n%l.41g\t %1.4lg",gInfo.Actual,gInfo.Predicted);

else printf("\nerror in parameters passed to Consistency Check."); printf("\nconsistency check completed");

£close (check);

//------/ / // Load Fuzzy System definition into memory -> gInitialFS / / //------void LoadFS ( ) { short i,j;

FILE *f~; FuzzySetPtr TempSet, BaseSet; // root RulePtr TempRule , BaseRule ; char filename[321; printf ("\nEnter filename of fuzzy system def: ") ; scanf ("%sM,filename) ;

if ((fp = fopen(filename,"rU))== NULL ) { printf("\nError: cannot open fuzzy system definition file"); exit (1); i printf("\nloading fuzzy system definition from '%sl.",filename);

fscanf(fp,"\n%hd%hdn, &gInfo.NumSystemInputs, &gInfo.RuleCount); printf("\n%hd system input(s)... with %hd rules",gInfo.NumSystemInputs, gInfo.RuleCount);

for(i = 0; i < (gInfo.NumSystemInputs+l); i++) { // save the domain interval information fscanf(fp,"%lg %lg %hdn, &gIntervals[il .LowerBound, &gIntervals[il .UpperBound, &gIntervals [il .NumSets); printf("\nInterval %hd defined:[%lg, %lgl composed of %hd sets.", i, gIntervals[i] .LowerBound, gIntervals[il .UpperBound, gIntervals[il .NumSets);

printf (" \tThe set width is %lg.", ((gIntervals[i].UpperBound- gIntervals[i].LowerBound)/gIntervals[il .NumSets ); 1

BaseSet = (FuzzySetPtr) NewPtr(sizeof(FuzzySet)); if (BaseSet == nil) { printf("\nError: in LoadFS() creating initial fuzzy set"); exit (1); 1

BaseRule->next = nil;

for(i = 0; i < gInfo.RuleCount; i++) { TempRule = (RulePtr) NewPtr(sizeof(Ru1e) ); if (TernpRule != nil) // create ok { fscanf(fp,"%hd",&TempRule->Label); BaseSet->next = nil; // init base set for antecedent of rule i

// process IF side for(j = 0; j < gInfo.NumSystem1nputs; j++) TempSet = (FuzzySetPtr) NewPtr(sizeof(FuzzySet)); if (TempSet != nil) { fscanf(fp,"%hd %lg %lg %lg %1gu,&TempSet->Label, &~empSet->a, &Tempset->b, &Tempset->c, &Tempset->d);

TempSet->next = BaseSet->next; // insert Set into list BaseSet->next = TempSet; } else { printf("Mem0ry Error in LoadFSO Creating Antecedent Set"); exit (1); I

// process THEN side TempSet = (FuzzySetPtr) NewPtr(sizeof(FuzzySet) ); if (TempSet != nil) { fscanf(fp,"%hd %lg %lg %lg %lgU,&TempSet->Label, &Tempset->a, &Tempset->b, &Tempset->c, &Tempset->d);

TempSet->next = nil; TempRule->Then = TempSet; 1 else { printf("Mem0ry Error in LoadFSO : Consequent Set Creation"); exit (1); I

TempRule->next = BaseRule->next; // insert rule into list BaseRule->next = TempRule; I else { print£ ( "~emoryError in LoadFS () when Creating New Rule") ; exit(1); } } g1nitial~S->root = BaseRule->next; // assign rule base pointer printf("\nrulebase loaded into gInitialFS structure. \n%hd rules in rulebase.",gInfo.RuleCount); fclose (fp);

/ / // Calculate Mu computes the truth value of the statement // "x is in A" where A is a Trapezoidal Fuzzy Number (a superset // to Triangular Fuzzy Numbers) defined by four points (a,b,c,d) / / //------double CalculateMu(f1oat Input, FuzzySetPtr Set) { double Output=O . 0, MidLow=O . 0, HighMid=O.O;

if ((Input <= Set->a) 11 (Input >= Set->dl) Output = 0.0; else { if (Input > Set->c) Output = ( (Set->d - Input)/HighMid) ; else r 1 if((1nput >= Set->b) && (Input <= Set->c)) Output = 1.0; else Output = (Input - Set->a)/MidLow; > 1 return (Output); } //------/ / // RandomZeroOne () Generates random value between 0 and 1 / / //------double RandomZeroOneO 1 static long ixl, ix2, ix3; static double r[981 ; double temp; static short iff = 0; short j; if (gInfo.RandomSeed < 0 I I iff == 0) { iff = 1; ixl = (1c1 - (gInfo.RandomSeed)) % MI; ixl = (IA1 * ixl + IC1) % MI; ix2 = ixl % M2; ixl = (IA1 * ixl + IC1) %MI; ix3 = ixl % M3; for (j = 1; j <= 97; j++) { ixl = (IA1 * ixl + IC1) % MI; ix2 = (IA2 * ix2 + IC2) % M2; r[j] = (ixl + ix2 * RM2) * RM1; 1 1 ixl = (IA1 * ixl + IC1) % MI; ix2 = (IA2 * ix2 + IC2) % M2; ix3 = (IA3 * ix3 + IC3) % M3; j = 1 + ((97 * ix3) / M3); if (j > 97 /I j < 1) ( printf("Error: random ... this should never happen! exit (1);

temp = r[j]; rljl = (ixl + ix2 * RM2) * RM~; return (MIN + (MAX - MIN) * temp) ; 1

//------/ / // Compute Centroid / / //------double ComputeCentroid(FuzzySetPtr X, double ~eight) iouble Area, Base, // base of TpFN Top, Slopel, Slope2, Runl, Run2 ;

Base = (X->d - X->a); Slope1 = kUnity/(X->b - X->a); Slope2 = kUnity/ (X->d - X->c); Runl = Height/Slopel; Run2 = Height/Slope2; Top = Base - Runl - Run2; Area = Height * (Base + Top)/2;

//------/ / // Print Fuzzy System / / void PrintFS(SystemPtr fs) printf("\nFuzzy System Definition:"); if (fs->root != nil) { printf("\ngen:%hd FS:%hd\tfitness:%1.31g\trank:%hdN, gInfo.Generation, fs->Label, fs->Fitness, fs ->Rank);

for(CurRule=fs->root;CurRule != nil; CurRule = CurRule->next) { printf("\nRule %hd:\nIFn,CurRule->Label);

for(CurSet = CurRule->If; CurSet != nil; CurSet=CurSet->next) Displayset (Curset);

J 1 else { printf ( " \nError: PrintFS ( ) -Somehow a fuzzy system could not be accessed!") ; exit (1); 1

//------/ / // Print entire population to TraceFP file or screen / / //------void PrintPopulationO I short i;

//------/ / // Reproduction Type 2-Propagate Best by Copying best FS, gBestFS // to entire population with the best solution from the previous / / generation. / / //------void PropagateBestFS() // this is kind of like hillclimbing { short i;

for(i=0;i~gInfo.Num0fPopu~ation;i++){ ~op~~~(g~opulation[i],gBestFS); gPopulation[il->Label = i; // renumber all fuzzy systems 1

//------/ / // Copy an individual, gInitialFS to gPopulation[Tol. Used to // fill population. This should only be used to initialize the // population, not again. / / //------

void PropagateInitialFS(SystemPtr fs) { FuzzySetPtr CurSet, TempSet, BaseSet ; RulePtr CurRule, TempRule , BaseRule ;

BaseSet = (FuzzySetPtr) NewPtr(sizeof(FuzzySet)); if (BaseSet == nil) { printf("\nError: in PropagateInitialFSO creating initial fuzzy set"); exit (1);

for(CurRule = gInitialFS->root; CurRule != nil; CurRule = CurRule->next) { TempRule = (RulePtr) ~ew~tr(sizeof(Ru1e)); if (TempRule != nil) // create ok { TempRule->Label = CurRule->Label; BaseSet->next = nil;

// process IF side for(CurSet = CurRule->If; CurSet != nil; CurSet = CurSet->next) { TempSet = (FuzzySetPtr) NewPtr(sizeof(FuzzySet)); // allocate fuzzy set if (TempSet != nil) { TempSet->Label = CurSet->Label; TempSet->a = CurSet->a; TempSet->b = CurSet->b; TempSet->c = CurSet->c; TempSet->d = CurSet->d;

~emp~et->next= ~aseset->next;// insert Set into list BaseSet->next = TempSet; ) else { printf("Memory Error in PropagateInitialFSO: IF"); exit (1); 1

// process THEN side CurSet = CurRule->Then; TempSet = (Fuzzysetptr) NewPtr(size0f (FuzzySet)) ; // allocate fuzzy set if(TempSet != nil) TempSet->Label = CurSet->Label; TempSet->a = CurSet->a; TempSet->b = CurSet->b; TempSet->c = Curset->c; TempSet->d = CurSet->d;

TempSet->next = nil; TempRule->Then = TempSet; 1 else { printf("Memory Error in PropagateInitialFSO: THEN"); exit(1); }

~emp~ule->next= ~ase~ule->next;// insert Rule into list BaseRule->next = TempRule; 1 else { printf ( "Memory Error in PropagateInitialFS () : Rule") ; exit (1); } 1

//------/ / / / Mutate Population. / / //------

void Mutatepop () 1 short i;

for(i=l;i~gInfo.Num0fPopulation;i++) ( // do not mutate individual 0 //------/ / // Mutate FS. / / //------void MutateFS(SystemPtr fs) I FuzzySetPtr CurSet ; RulePtr CurRule ; for (CurRule = fs->root; CurRule != nil; CurRule = CurRule->next) {

for(CurSet = CurRule->If; CurSet != nil; Curset = CurSet->next) { Mutateset (Curset); // mutate IF ) MutateSet(CurRu1e->Then); // mutate THEN 1

/ / // Mutate a fuzzy set in one of four ways: / / 1. no mutation / / 2. linear shift / / 3. expansion/contraction of base and core / / 4. nonlinear shift / / //------void MutateSet (FuzzySetPtr Set) double Type ;

Type = RandomZeroOneO;

if (Type < gInfo.Plinear) { AlterSet(Set,linear); return; 1

if (Type < (gInfo.Plinear + gInfo.Pexpand) ) { AlterSet(Set,expand); return; 1

if (Type < (gInfo.Plinear + gInfo .Pexpand + gInfo.Pcontract) ) { AlterSet(Set,contract); return; 1

if(Type <= (gInfo.Plinear + gInfo.Pexpand + gInfo.Pcontract + gInfo.Pnonlinear)) AlterSet(Set,nonlinear); return; I ) else return; / / // Alter a fuzzy set void AlterSet(FuzzySetPtr Set, MutationType type) {

case none: break; case linear: ShiftSet (Set); break; case expand: ExpandSet(Set); break; case contract: Contractset (Set); break; case nonlinear: NonlinearShiftSet(Set); break;

/ / // Shift fuzzy set / / //------void ShiftSet (FuzzySetPtr Set) double Shift=O.O;

Shift = (gInfo.MaxSetShift * RandomZeroOneO);

if (RandomZeroOne( ) < 0.5 ) Shift = -Shift;

Set->a = (Set->a + Shift); Set->b = (Set->b + Shift); Set->c = (Set->c + Shift); Set->d = (Set->d + Shift) ;

//------/ / // Expand fuzzy set // This function expands of the core of a fuzzy set / / //------

void ExpandSet(FuzzySetPtr Set) i

if (Randomzeroone() < 0.5) ExpandBase (Set); else Expandcore (Set);

// Expand core of fuzzy set / / //------void ExpandCore(FuzzySetPtr Set) ( double Shift=O.O; //maximum shift is the minimum of Min(b-a,d-c)/2 and MaxSetShift

Shift = (RandomZeroOneO * ~in(gInfo.MaxSetShift,(Min((Set->b- Set->a),(Set->d - Set->c))/2))); Set->b = (Set->b - Shift) ; Set->c = (Set->c + Shift);

/ / // Expand base of fuzzy set / / void ExpandBase(FuzzySetPtr Set) 1 double Shift=O.O;

//maximum shift is the minimum of Min(b-a,d-c) /2 and MaxSetShift

Shift = (Randomzeroone0 * Min(gInfo.MaxSetShift, (Min((Set->b - Set->a),(Set->d - Set->c))/2))); Set->a = (Set->a - Shift); Set->d = (Set-zd + Shift);

//------/ / // Contract fuzzy set / / //------void ContractSet(FuzzySetPtr Set)

if(RandomZeroOne0 < 0.5) ContractBase(Set); else Contractcore (Set);

//------/ / // Contract base of fuzzy set / / //------

void ContractBase(FuzzySetPtr Set) 1 double Shift=O.O:

// maximum contraction of a TFN is Min(d-c,b-a)/2 or MaxSetShift

Shift = (RandomZeroOne() * ~in(gInfo.~axSetShift,Min((Set->d- Set->c),(Set->b - Set->a))/2)); Set->a = (Set->a + Shift); Set->d = (Set->d - Shift); #if def kdebug print£ ("\nContracting Base of Set by %1.4lgV,Shift) ; #endi f

//------/ / // Contract core of fuzzy set void ContractCore(FuzzySetPtr Set) { double Shif t=O . 0; if(Set->b != Set->c) { // modify core (b,c) of TpFN

// maximum contraction of a TpFN is c-b/2 // (transforming the set into a TFN) or MaxSetShift

Shift = (RandomZeroOne() * Min(gInfo.MaxSetShift,(Set->c - Set->b)/2)); Set->b = (Set->b + Shift); Set->c = (Set->c - Shift); I else (:

// cannot contract core of a TFN !

#if de f kdebug printf("\nContracting Core of Set by %1.4lg",Shift); #endi f

// Follow the Black Valley Trail of Death, Into the Beautiful Sunshine... //------/ / // Nonlinear shift fuzzy set / / //------void NonlinearShiftSet(FuzzySetPtr Set) { double Shift=O.O, Point;

Point = RandomZeroOne ( ) ;

if (Point < .25) MutatePointA(Set); return; >

if (Point < .5) { MutatePointB (Set); return; I

if (Point < .75) { ~utatePointC(Set); re turn ; >

if (Point <= 1.00) { MutatePointD (Set); return; I

//------/ / // Mutate point A / / //------

void MutatePointA(FuzzySetPtr Set) { double Shift=O.O;

// maximum shift is the minimum of (b-a)/2 and MaxSetShift

Shift = (Randomzeroone() * ~in(gInfo.MaxSetShift,((Set->b- Set->a)/2))); // compute shift if (Randomzeroone() < 0.5) Shift = -Shift; // change sign Set->a = (Set->a + Shift);

//------/ / // Mutate point B / / void MutatePointB(FuzzySetPtr Set) double Shift=O.O: if(Set->b == Set->c) ( // Set is a TFN Shift = (Randomzeroone() * Min(gInfo.MaxSetShift, ((Set->b - Set->a)/2))); Set->b = (Set->b - Shift); // Set is now a TFN if Shift is > 0 1 else ( // Set is a TpFN

// maximum is smallest of (half the distance from b to the nearest point) and MaxSetShift Shift = (RandomZeroOneO * Min(gInfo.MaxSetShift,(Min((Set->c- Set->b),(Set->b - Set->a))/2))); if (RandomZeroOneO < 0.5) Shift = -Shift; // change sign Set->b = (Set->b + Shift); 1

//------/ / // Mutate point C / / //------void MutatePointC(FuzzySetPtr Set) { double Shift=O.O;

if (Set->b == Set->c) { // Set is a TFN Shift = (Randomzeroone() * ~in(g~nfo.MaxSetShift,((Set->d - Set->c)/2))1; Set->b = (Set->c + Shift); // Set is now a TFN if Shift is > 0 I

else ( // Set is a TpFN

// maximum is smallest of (half the distance from c to the nearest point) and MaxSetShift Shift = (Randomzeroone0 * ~in(gInfo.MaxSetShift,(Min((Set->c- Set->b),(Set->d - Set->c))/2))); if (~andom~ero~ne( ) < 0.5 ) Shift = -Shift; // change sign Set->c = (Set->c + Shift) ; 1

#if def kdebug printf("\nNonlinear Shifting Set ...(c) by %1.41gM,Shift); #endif / / // Mutate point D void MutatePointD(FuzzySetPtr Set) I 1 double Shift=O.O;

// maximum shift is the minimum of (d-c)/2 and MaxSetShift Shift = (Randomzeroone() * Min(gInfo.MaxSetShift,((Set->d - Set->c)/2))); // compute shift if (RandomZeroOne() < 0.5) Shift = -Shift; // change sign

Set->d = (Set->d + Shift) ;

/ / // Reproduction

void Reproduction( ) ( short i,j ; if(gInfo.~eproductionType== 0) { // *REPRODUCTION 0* mutation only

Eliteselection0 ; // select those that are above average EliteReplacement() ; return; 1

GroupFormationO; GroupSelection(2); // selects 2 parents out of each group for(i=O; i<(gInfo.NumOfPopulation/4);i++) I GenerateOffspring(i); ComputeFitness(gTempFS); MutateFS(gTempFS); ~ocally~liteReplacement(i); 1 return; 1

if (gInfo.ReproductionType == 2) { // *REPRODUCTION 2* do propagation of best only PropagateBestFSO; Mutatepop ( ) ; return; 1

/ / // Locally elite replacement into group i with TempFS / /

void LocallyEliteReplacement(short group) { short i,j, worst;

// loop through all members in group GroupNum worst=O; for(i=O; i<4; i++) { if(gPopulation[gGroupInfo[groupl.member[ill-Fitness > g~opulation[gGroupIn£o[groupl.member[worst]]->Fitness)worst = i; )

//------/ / // Form Groups / / //------void GroupFormationO i short i,j, guess; Boolean done;

for(i=0;iGroupNum = 0; gPopulation[il->Grouped = false; gpopulation [ il ->Parent = false; ) for(i=O; i~(gInfo.NumOfPopulation/4);i++) { gGroupInf o [ i I .Label = i ; / / printf ( " \nGroup %hd has : " , i) ;

for(j=O; j<4; j++) { guess = (short) (RandomZeroOneO * gInfo.~um~f~o~ulation); done=false;

while ( !done) I if(!g~opulation[guessl->Grouped)done = true; else { if(guess >= (gInfo.NmOf~o~ulation-1)) guess = 0; else guess++; I I g~opulation[guess]-~Grouped= true; g~roupInfo[ i I .member[ j 1 = guess; I >

//------/ / // Select best two individuals in group for parenthood / / //------void GroupSelection(short NumParents) { short i,j; //------/ / // Select parent in ith group / / //------void SelectParent(short group) { short i,best; best = 0; for(i=O; i<4; i++) ( if((gPopu~ation[gGroupInfo[group].memberi]-Fitness < g~opulation[gGroupInfo[groupl.member~bestll-Fits&& (!g~opulation[gGroupInfo[group].rnember[i]]-Parent))best = i; 1 gPopulation[gGroupInfo[group] .member[best -Parent = true; if(debug) printf("\nGroup %hd parent %hd (fitness=%lg)",group, gGroupInfo[group] .member[bestl, g~opulation[gGroupInfo[groupl.member[best]]->Fitness);

/ / // Generate Offspring from parent A and parent B placing child // in TempFS (called C here. . . ) / / //------void GenerateOffspring(short GroupNum) { FuzzySetPtr SetA, SetB, SetC; RuleA, RuleB, RuleC; short i,j;

for(i=O, j=O; i<4; i++) ( // loop through all members in group GroupNum if(gPopulation[gGroupInfo[GroupNuml .member[il]->Parent) {

if(j=O) ( // Parent A RuleA = gPopulation[gGroupInfo[GroupNuml.mernber[ill-~root; gPopulation[gGroupInfo[GroupNum].member[i]]->Parent = false; j=23; I

else { // Parent B RuleB = gPopulation[gGroupInfo[GroupNuml .rnernber[ill->root; g~opulation[gGroupInfo[GroupNum].member[i]]-~Parent= false; 1

if (Randomzeroone( ) < gInfo.PCrossover) { printf("\nperforming crossover... no SEX until marriage!!! "); for(Ru1eC = gTempFS->root /* RuleA and RuleB already initialized */ ; RuleA != nil, RuleB != nil, RuleC != nil; RuleA = RuleA->next, RuleB = RuleB->next, RuleC = RuleC->next) {

if(RandomZeroOne() < 0.5) { // copy current rule from parent A to C // process IF side for(SetA = RuleA->If, SetC = RuleC->If; SetA != nil, SetC != nil; SetA = SetA->next, SetC = SetC->next) {

if (SetA != nil I / SetC != nil) { SetC->Label = SetA->Label; SetC->a = SetA->a; SetC->b = SetA->b; SetC->c = SetA->c; SetC->d = SetA->d; 1 1 // process THEN side SetA = RuleA->Then; SetC = RuleC->Then; if (SetA ! = nil / I SetC ! = nil) { SetC->Label = SetA->Label; SetC->a = SetA->a; Setc->b = SetA->b; SetC->c = SetA->c; SetC->d = SetA->d; 1

else { // copy current rule from parent B to C

// process IF side for(SetB = rule^->If, SetC = RuleC->If; SetB != nil, SetC != nil; SetB = SetB->next, SetC = SetC->next) {

1 // process THEN side SetB = RuleB->Then; SetC = RuleC->Then; if (SetB ! = nil I I SetC ! = nil) { setC->Label = SetB->Label; SetC->a = SetB->a; setc->b = SetB->b; SetC->c = SetB->c; Setc->d = SetB->d; 1 1

//------/ / // Select those that are better than average... just as Holland's // schema theorem suggests. Those that are above average are elite. / / //------void EliteSelectionO

short i,j ; for(i=O; i~gInfo.NumOfPopulation; i++) ( if(g~opulation[g~ank~nfo[i]l->Fitness> gInfo.PopAvg) { gInfo.EliteCount = i; break; 1 1

//------/ / // replace those that are not elite with variants of elite / / //------void EliteReplacementO ( short i,j,k;

/ / // Builds a Fuzzy System derived from gInitialFS void ~uildInitialFS(SystemPtr fs) ( FuzzySetPtr CurSet, TempSet, BaseSet; RulePtr CurRule, TempRule , BaseRule ;

BaseSet = (FuzzySetPtr) NewPtr(sizeof(FuzzySet)); if (BaseSet == nil) { printf("\nError in BuildInitialFSO when allocating new fuzzy set"); exit (1); 1

for (CurRule = gInitialFS->root; CurRule ! = nil; CurRule = CurRule->next) ( TempRule = (RulePtr) NewPtr(sizeof(Ru1e)); // allocate space for rule if (TempRule != nil) // create ok r 1 TempRule->Label = CurRule->Label; BaseSet->next = nil; // init base set for antecedent

// process IF side for(CurSet = CurRule->If; CurSet != nil; CurSet = CurSet->next) { TempSet = (FuzzySetPtr) NewPtr(sizeof(FuzzySet)); // allocate fuzzy set if (TempSet ! = nil) TempSet->Label = CurSet->Label; TempSet->a = CurSet->a; TempSet->b = CurSet->b; TempSet->c = CurSet->c; TempSet->d = CurSet->d;

~emp~et->next= ~ase~et->next;// insert Set into list BaseSet->next = TempSet; I else { printf("Memory Error in BuildInitialFSO: IF"); // process THEN side CurSet = CurRule->Then; TempSet = (FuzzySetPtr) NewPtr(sizeof(FuzzySet)); // allocate fuzzy set if (TempSet != nil) { TempSet->Label = CurSet->Label; TempSet->a = CurSet->a; TempSet->b = CurSet->b; TempSet->c = CurSet->c; TempSet-zd = CurSet->d; ~empset->next = nil;

~emp~ule->Then= TempSet; 1 else { printf("Memory Error in Build~nitialF~():THEN"); exit(1); }

TempRule->next = BaseRule->next; // insert Rule into list. BaseRule->next = TempRule;

else { printf ("Memory Error in BuildInitialFS () : Rule"); exit (1); 1 } fs->root = BaseRule->next;

//------/ / // Copy Fuzzy System (To, From) / / //------void CopyFS (SystemPtr To, SystemPtr From) FuzzySetPtr FromSet, ToSet; RulePtr FromRule, ToRule; for(FromRu1e = From->root, ToRule = To->root; FromRule != nil, ToRule != nil; FromRule = FromRule->next, ToRule = ToRule->next) {

// process IF side for(FromSet = FromRule->If, ToSet = ToRule->If; FromSet ! = nil, ToSet != nil; FromSet = FromSet->next, ToSet = ToSet->next) {

if (ToSet ! = nil I I FromSet ! = nil) { ~oset->Label= FromSet->Label; ~oset->a= FromSet->a; ~oset->b= FromSet->b; ~oset->c= FromSet->c; ~oset->d= FromSet->d; 1

// process THEN side FromSet = FromRule->Then; ToSet = ToRule->Then; if (ToSet ! = nil I j FromSet ! = nil) { ~oset->Label= FromSet->Label; ToSet->a = FromSet->a; ToSet->b = FromSet->b; ToSet->c = FromSet->c; ToSet->d = FromSet->d; } / / // Save the best fuzzy system definition. / / //------void SaveFS (SystemPtr fs, char *s) I FuzzySetPtr CurSet; RulePtr CurRule ; FILE *,fp;// description file for entire fuzzy system short 3 ; if ((fp = fopen(s,"w"))== NULL ) { printf ("\nError: cannot open fuzzy rulebase file for writing") ; exit (1); >

for(~ur~ule=fs->root;CurRule != nil; CurRule = CurRule->next) t fprintf(fp,"%hd\n",CurRule->Label); for(CurSet = CurRule->If; CurSet != nil; CurSet=CurSet->next) I // use TpFN notation fprintf(fp,"%hd $1.319 %1.31g %1.31g %1.31g\nn, Curset->Label, CurSet->a, Curset->b, CurSet->c, Curset->d); I

CurSet = CurRule->Then; // use TpFN notation fprintf (fp," \n%hd %1.31g %1.31g %1.31g %1.31g\n1', Curset->Label, CurSet->a, CurSet->b, CurSet->c, Curset->d); ] // end for CurRule

fclose (fp); I

//------/ / // Rule Evaluation and Fuzzification / / //------

void RuleEvaluation(SystemPtr fs) // of individual i of population i short interval; double Mu=O .O; RulePtr CurRule ; FuzzySetPtr CurSet; for(CurRule=fs->root; CurRule != nil; CurRule = CurRule->next) { CurRule->Truth = kUnity; // max rule strength allowed

for(CurSet = CurRule->If, interval = 0; CurSet != nil; CurSet=CurSet->next,interval++){ Mu = CalculateMu(gInfo.RunTimeData[intervall, CurSet); CurRule->Truth = Min(CurRu1e->Truth,Mu);// take minimum I

/ / // Defuzzification converts fuzzy output to crisp / / //------void Defuzzification(SystemPtr fs) { double SumOfAreas=O.O, SumOfProducts=O.O, Area=O . 0 , Centroid=O.O; RulePtr CurRule; FuzzySetPtr CurSet, Set; for(CurRule=fs->root; CurRule != nil; CurRule = CurRule->next) Set=CurRule->Then; Area = ComputeCentroid(Set,CurRule->Truth); Centroid = (Set->a + (Set->d - Set->a)/2); SumOfProducts += (Area * Centroid); SumOfAreas += Area; 1 if(Sum0fAreas == 0) { // Sum of areas 0 division error. . . setting output to zero gInfo.Predicted=O; return;

//------/ / // read in test data from file / / //------void LoadTestData ( )

FILE *fp; // numerical sample data file char filenamer321 ; short i;

// now open file to read in data points printf("\nEnter filename of training data:"); scanf("%sN,filename);

if ((fp = fopen(filename,"r"))== NULL ) { printf("\nError: cannot open training file"); exit (1);

printf("\nloading training data from '%s'.",filename); // read system input vectors from file for(i=O; i

// Compute the fitness of the population / / void ComputeFitnessOfPop() { short i;

for(i=O; i~gInfo.NumOfPopulation;i++) // loop through all members of the population ComputeFitness(gPopulation[i]);

//------/ / // Compute the fitness of a fuzzy system / / //------void ComputeFitness(SystemPtr fs) { short i,j; double SqrErr = 0.0, AbsErr = 0.0, SumAbsErr = 0.0, MaxErr = 0.0;

if(!gInfo.ReadTestingData) { // load test data the first time! LoadTestData ( ) ; gInfo.ReadTestingData = true; 1

for(i=O; i

for(j=O; j

gInfo.Actua1 = gTestingDataLi1 .Output; RuleEvaluation(fs); Defuzzification(fs); SqrErr += ((gInfo.Predicted - gInfo.Actua1) * (gInfo.~redicted- g~nfo.~ctual)); AbsErr = gInfo.Predicted - gInfo.Actua1; if (~bsErr< 0) AbsErr = -AbsErr; SumAbsErr += AbsErr; if (AbsErr > MaxErr) MaxErr = AbsErr; 1

fs->RMSErr = sqrt(SqrErr/gInfo.NumTestVectors); ~s->MSE= (Sqr~rr/g~nfo.NumTestVectors); fs->AvgAbsErr = (SumAbsErr/gInfo.NumTestVectors); fs->MaxAbsErr = MaxErr;

fs->Fitness = fs->RMSErr; // we are using RMS error as selection criteria (fitness)

//------/ / // Test the best fuzzy system for predictability void TestPrediction(SystemPtr fs, Boolean Verbose) { short i,j; double SqrErr = 0.0, AbsErr = 0.0, SumAbsErr = 0.0, MaxErr = 0.0, Fitness = 0.0; if(!gInfo.~ead~alidationData){ // load test data the first time! LoadValidationData(); gInfo.ReadValidationData = true; 1

for(i=O; i

for(j=O; j

/ / // print out actual and predicted with error, if Verbose / / if(Verbose) printf("\n\ty:%1.3lg\ty':$1.31g\te:%1,3lg", gInfo.Actual,gInfo.Predicted, gInfo.Actua1-gInfo.Predicted);

SqrErr += ((gInfo.Predicted- gInfo.Actua1) * (gInfo.Predicted - gInfo.Actua1)1; AbsErr = gInfo.Predicted - gInfo.Actua1; if (AbsErr < 0) AbsErr = -AbsErr; SumAbsErr += AbsErr; if (AbsErr > MaxErr) MaxErr = AbsErr; }

fs->PredRMSErr = sqrt(SqrErr/gInfo.NumTestVectors); fs->PredMSE = (SqrErr/gInfo.NumTestVectors); fs->PredAvgAbsErr = (SumAbsErr/gInfo.NumTestVectors); fs->PredMaxAbsErr = MaxErr;

/ / // Load validation data

void LoadValidationDataO { FILE *fp; // numerical sample data file char fllename [ 32 I ; short i ;

printf("\nEnter filename of cross-valid (used in prediction) data (e.g.100~~):"I; scanf ("%sM,filename) ;

// now open file to read in data points if ((fp = fopen(filename,"r")) == NULL ) { printf("\nError: cannot open validation file"); exit (1); 1 printf("\nloading cross-validation data from '%s'.",filename); // read system input vectors from file for(i=O; i~gInfo.NumSystemInputs;i++) // read the system input data fscanf(fp, "%lgH,&gValidationData[gInfo.NumValidationVectorsl .Input[i]); fscanf(fp, "%lgU,&gValidationData[gInfo.NumValidationVectorsl .Output); gInfo.NumValidationVectors++;

printf("\nread in %hd validation vectors.",gInfo.Num~alidationVectors); fclose (fp); 1

//------/ / // Load final testing data / / //------void LoadFinalTestDataO t FILE *fp; // numerical sample data file char fllename[32]; short i;

printf("\nEnter filename of final independent testing data (such as 'lootst'):"I; scanf("%sH,filename);

// now open file to read in data points if ((fp = fopen(filename,"rn))== NULL ) { printf ( " \nError: cannot open final testing file") ; exit (1); printf("\nloading cross-validation data from '%sl.",filename); while(!feof(fp) && (gInfo.NumValidationVectors < kMaxValidationVectors)) { // read system input vectors from file for(i=O; i

//------/ / // Find minimum fitness of population / / //------void FindMin (short x) I short Position, i ; doub 1e Min;

// perform an intial guess for(i=0;i~gInfo.NumOfPopu1ation;i++) { if(!gPopulation[i]->Ranked) { // if not ranked Min = gPopulation[il->Fitness; Position = i; break; // a valid guess was found at Position with a value of Min for(i=0;iRanked && (Position != i)) { // if not ranked if (gpopulation[i] ->Fitness < Min) { Min = gPopulation[i]->Fitness; Position = i; 1 ) ) gPopulation[Positionl->Rank = x; gPopulation[Positionl->Ranked = true; gRankInfo [XI = Position;

, , // Rank the population according to fitness / / void Rankpopulation0 { short i ;

// initialize population rank information... for(i=0;i~gInfo.NumOfPopu1ation;i++) { gPopulation[il->Ranked = false; // set all members of population to "not ranked" gPopulation[il ->Rank = 0; ) for(i=0;i~gInfo.Num0fPopulation;i++) { FindMin(i); // a crude selection sort 1 for(i=0;i~gInfo.Num0fPopu~ation;i++) { if (gPopulation[il->Rank == 0) { if(gPopulation[i]->Fitness i gBestFS->Fitness) { CopyFS(gBestFS, gPopulation[il);

// else do not replace best FS

//------/ / // print population information / / //------void PrintPopInfo ( ) { short i,j ;

// print out currrent generation, label, fitness and rank of each FS for(j=0;j

, , / / // print fuzzy system information: / / Current Generation / / Label / / Fitness / / Rank / / void PrintFSInfo(SystemPtr fs) i

//------/ / // DisplayFuzzySet prints the four defining points of a TpFN / / void DisplaySet(FuzzySetPtr x) I if(x != nil) {

if (x->b == x->c) printf ("\t%hd = TFN(%1.31g,%1.31g, %l. 31g) \nu, x->Label, x->a, x->b, x->d);

else printf("\t%hd = TpFN(%1.3lg,%l.3lg,%1.31g,%1.31g)\n", x->Label, x->a, x->b, X->C, x->d); 1 else { printf ( " \nError: Displayset () was passed a nil fuzzy set!" ) ; exit (1); }

//------// EOF /*------The second step in the prototyped system, Mobius title: OPTIMIST-An Improvement Strategy for Fuzzy Systems Using Evolutionary Computation date : November 19 95 file: 0ptimize.h author : Brian 0. Bush components: optimize.^ optimize.h compiler: Metrowerks v1.3 C for the PowerPC 60x RISC processors system: My Macintosh PowerPC Performa 6115CD purpose: Research for my Thesis

Copyright @ 1996 by Brian 0. Bush, all rights reserved ------* /

//------/ / // FuzzySet defines the structure of a fuzzy set / / //------struct FuzzySet { short Label ; double a,b,c,d;// four points define a trapezoidal fuzzy interval struct FuzzySet *next; I ; typedef struct FuzzySet FuzzySet; typedef FuzzySet *FuzzySetPtr;

//------/ / / / Rule definition

struct Rule { short Label ; double Truth; // truth value of the antecedent FuzzySetPtr If; // "antecedent" -up to m inputs FuzzySetPtr Then; // "consequent" -only 1 output struct Rule *next; >; typedef struct Rule Rule; typedef Rule *RulePtr;

//------/ / // Fuzzy System Definition with individual info / / //------

struct System I short Label ; short Rank;

double Fitness, // default fitness is RMS error RMSErr, MSE, AvgAbsErr , MaxAbsErr, Prediction, PredRMSErr, PredMSE, PredAvgAbsErr, PredMaxAbsErr:

Boolean Ranked, // for ranking Grouped, Parent;

short GroupNum ; RulePtr root; 1 ; typedef struct System System; typedef System *SystemPtr;

// Out of Light Cometh Darkness... //------/ / // Group information / / //------struct Group { short Label ; // group label short member[4]; //groupsizeissetat four 1 ; typedef struct Group Group ;

/ / // Domain information / / //------struct Domain { short NumSets ; double LowerBound; double UpperBound; 1 ; typedef struct Domain Domain;

//------/ / // Testing Data / / //------struct Test{ double Input[kMaxSystemInputsl; double Output ; } ; typedef struct Test Test;

/ / // System status information / / struct Information { Boolean Read~estingData, ReadValidationData; short NumVectors, NumTestVectors, NumValidationVectors, NumSystemInputs, NumOfPopulation, ReproductionType, Rulecount, MaxGenerations, Generation, Randomseed; double Predicted, Actual ,

BestPred, PopAvg, Elitecount, PMutation, Plinear , Pexpand, Pcontract, Pnonlinear, PCrossover,

double RunTimeData[kMaxSystemInputs]; 1 ; typedef struct Information Information;

enum MutationType { none=O, linear, expand, contract, nonlinear 1 ; typedef enum MutationType MutationType;

//------// end of file 0ptimize.h FUZZY SYSTEM ENGINE

/*------title: A Fuzzy System Engine file: fuzzy.c author: Brian 0. Bush components: fuzzy.c main, initializations and just about everything fu2zy.h global variables, constants and structures language: Metrowerks v1.3 C for the PowerPC 60x RISC processors system: My Macintosh PowerPC Performa 6115CD purpose: Research for my Thesis

Copyright O 1996 by Brian 0. Bush, all rights reserved ...... * /

short gRuleCount=O, gNumTestVectors, gNumSystemInputs=l; double gRunTimeData[kMaxSystemInputsl; // used to hold system input double gpredictedoutput, gActualOutput, gRmsError; double ~alculateMu(f1oat input, FuzzySetPtr Set); double ComputeCentroid(FuzzySetPtr X, double Height); void ConsistencyCheck(double lower, double upper, double increment); void Defuzzification(); void DisplaySet(FuzzySetPtr Set, Boolean Verbose); void LoadFS ( ) ; void PrintFS () ; void ReadInTestDataO; void RuleEvaluation(); void Testing(Boo1ean Verbose);

//------/ / // Main Program / / //------

void main() {

LoadFS ( ) ; PrintFS ( ) ; Testing(true);

/ / // Load Fuzzy System Degfinition into memory. / /

void LoadFS () short i,l; double lower,upper; short numberofsets; // not used here, but needed FILE *fp; FuzzySetPtr TempSet, BaseSet ;

char filename[31] ;

printf("\nEnter filename of fuzzy system def:"); scanf ("%s",filename);

if ((fp = fopen(filename,"r"))== NULL ) { printf("\nError: cannot open fuzzy system definition file"); exit (1); 1 printf("\nloading fuzzy system definition from '%sl.",filename); fscanf(fp,"%hd %hd",&gNumSystemInputs,&gRuleCount); printf("\n%hd system input(s) ...",gNumSysternInputs) ;

for(i = 0; i < (gNumSystemInputs+l); i++) { fscanf(fp,"%lg%lg %hdN, &lower, &upper, &numberofsets); printf("\n\t~omain%hd:[%1.3lg %1.31g], composed of %hd sets",i,lower,upper, numberofsets); }

BaseSet = (FuzzySetPtr) NewPtr(sizeof(FuzzySet)); if (BaseSet == nil) { printf ( " \nError: in LoadFS ( ) creating initial fuzzy set"); exit(1); 1

~ase~ule->next= nil;

for (i = 0; i < gRuleCount; i++) { TempRule = (RulePtr) NewPtr(sizeof (Rule)) ; if (TempRule != nil) // create ok

fscanf ( fp,"%hdU , &TempRule->Label); BaseSet->next = nil; // init base set for antecedent of rule i

// IF for(j = 0; j < gNumSystemInputs; j++) { TempSet = (FuzzySetPtr) NewPtr(sizeof(FuzzySet)); // allocate fuzzy set if (TempSet ! = nil) { fscanf(fp,"%hd%lg %lg %lg %1gu, &Tempset->Label, &Tempset->a, &Tempset->b, &Tempset->c, &Tempset->d); TempSet->next = BaseSet->next; // insert node into list BaseSet->next = TempSet; ) // end if ! nil else { printf("Memory Error in LoadFSO Creating ~ntecedentSet"); exit (1); T

// THEN TempSet = (FuzzySetPtr) NewPtr(sizeof(FuzzySet)); // allocate fuzzy set if(TempSet != nil) ( fscanf(fp,"%hd %lg %lg %lg %1g8', &Tempset->Label,

TempSet->next = nil; TempRule->Then = TempSet;

else ( printf("Memory Error in LoadFSO: Consequent Set Creation"); exit (1); 1 TempRule->next = BaseRule->next; // insert rule into list BaseRule->next = TempRule; I else ( printf("Mem0ry Error in LoadFSO when Creating New Rule"); exit(1); 1 } gRuleBase = BaseRule->next; // assign rule base pointer printf("\nrulebase loaded with %hd rules.",gRuleCount); fclose (fp); }

/ i // read in test data from file: testing.dat / / //------

void ReadInTestData ( ) I FILE ?fp; // numerical sample data file short 1; char filename[31]; printf("\nEnter filename of testing data:"); scanf("%sU,filename); if ((fp = fopen(filename,"rH))== NULL ) { printf("\nError: cannot open testing file"); exit (1); > printf("\nloading testing data from '%sl.",filename);

// read system input vectors from file for(i=O; i

//------/ / // Testing / / //------void Testing(Boo1ean Verbose) short i,j; double SqrErr = 0.0, AbsErr = 0.0, SumAbsErr = 0.0, MaxErr = 0.0; double MaxAbsErr,AvgAbsErr, MSE, RMSErr;

1 / // print out actual and predicted with error, if Verbose / /

SqrErr += ((gpredictedoutput - gActualOutput) * (gpredictedoutput - gActual0utput)); AbsErr = (gPredictedOutput - gActual0utput); if (AbsErr < 0) AbsErr = -AbsErr; SumAbsErr += AbsErr; if (AbsErr > MaxErr) MaxErr = AbsErr; I

RMSErr = sqrt(SqrErr/gNumTestVectors); MSE = (Sqr~rr/gNumTestVectors); AvgAbsErr = (SurnAbsErr/gNumTestVectors); MaxAbsErr = MaxErr; printf("\n\nPerformance of this Fuzzy System"); printf("\n\tRMSErr:%1.41gU,RMSErr); printf("\n\tMSE:%1.4lg",MSE); printf("\n\tAvgAbsErr:%1.4~g",AvgAbsErr); printf ( "\n\tMaxAbsErr:%l.4lg",MaxAbsErr) ; 1

//------/ / // Calculate Mu computes the truth value of the statement // "x is in A" where A is a Trapezoidal Fuzzy Number (a superset // to Triangular Fuzzy Numbers) defined by four points (a,b,c,d) / /

double ~alculateMu(f1oat Input, FuzzySetPtr Set) { double Output=O.O, MidLow=O.O, HighMid=O.O;

if ((Input <= Set->a) 11 (Input >= Set->d)) Output = 0.0; else 1 if (Input > Set->c) Output = ( (Set->d - Input)/HighMid) ; else { if((1nput >= Set->b) && (Input <= Set->c)) Output = 1.0; else Output = (Input - Set->a)/MidLow; I return (Output); I

//------/ / // Rule Evaluation and Fuzzification / / //------void RuleEvaluation()

double Mu = 0.0; RulePtr CurrentRule; FuzzySetPtr Set; short interval ;

for(CurrentRu1e = gRuleBase; CurrentRule != nil; CurrentRule = CurrentRule->next) { ~urrentRule->Truth= kunity; // max rule strength allowed

for(Set = CurrentRule->If, interval = gNumSystemInputs-1; Set != nil; Set = Set->next, interval--) I Mu = CalculateMu (gRunTimeData[interval] , Set) ; CurrentRule->Truth = Min(CurrentRu1e->Truth,Mu); // take minimum 1

/ / // Defuzzification converts fuzzy output to crisp using the centroid strategy / /

void Defuzzification() I RulePtr CurrentRule; double SumOfAreas=O.O, SumOfProducts=O.O, Area=O . 0, for(CurrentRu1e = gRuleBase; CurrentRule != nil; CurrentRule = CurrentRule->next) { Area = ComputeCentroid(CurrentRule-~Then,CurrentRule-~Truth); Centroid = (CurrentRule->Then->a+ (CurrentRule->Then->d- CurrentRule->Then->a)/2); SumOfProducts += (Area * Centroid); SumOfAreas += Area; J if (SumOfAreas == 0) { // Warning: Sum of areas is 0 will cause division error. gPredictedOutput=O; return:

//------/ / // Compute Centroid / / //------double ComputeCentroid(FuzzySetPtr X, double Height) { double Area, Base, // base of TpFN TOP, Slopel, Slope2, Runl, Run2 ;

Base = (X->d - X->a); Slopel = kUnity/ (X->b - X->a); Slope2 = kUnity/ (X->d - X->c); Runl = Height/Slopel; Run2 = Height/Slope2; Top = Base - Runl - Run2; Area = Height * (Base + Top)/2; return (Area); > //------/ / // Check the consistency of output on a specific domain, for single // input systems / / //------void ConsistencyCheck(doub1e lower, double upper, double increment) FILE *check; double error,interval; check = fopen("check.dat","w"); if (upper > lower)

for(interval=lower; interval 0) I I (gpredictedoutput <= 0)) fprintf(check,"\n%l.41g\t %1.4lg",gActualOutput,gPredicted0utput); > 1 else printf("\nerror in parameters passed to Consistency Check."); printf("\nconsistency check completed"); £close (check); //------/ / // Print Fuzzy System / / //------void PrintFS ( ) { FuzzySetPtr CurSet ; RulePtr CurRule ; printf("\nFuzzy System Definition:");

for(CurRule=gRuleBase; CurRule != nil; CurRule = CurRule->next) { printf("\nrule %hd: IF ",CurRule->Label);

for(CurSet = CurRule->If; CurSet != nil; CurSet=CurSet->next) ~isplaySet(CurSet,false);

printf (I' THEN ");

DisplaySet(CurRu1e->Then, false); ) 1 else { printf ("\nError: PrintFS()-Somehow a fuzzy system could not be accessed! ") ; exit (1); 1

/ / // DisplayFuzzySet prints the four defining points of a TpFN / / if gInfo.TraceFi1e true mirrors output to TraceFP file / / //------

void DisplaySet(FuzzySetPtr Set, Boolean Verbose) {

if (Set != nil) { if (Verbose) { if(set->b == Set->c) print£(" %hd = TF~(%1.31g,%1.31g,%l.31g)", Set->Label, Set->a, Set->b, Set->d);

else printf (" %hd = TpFN(%1.31g,%1.31g,%1.31g,%1.31g)", Set->Label, Set->a, Set->b, Set->c, Set->d);

else { if(set->Label < 10) print£(" %hdH,Set->Label); else printf ( " %hd",Set->Label) ; ) 1 else printf("\nError accessing fuzzy set definition");

//------// EOF /*------title: A Fuzzy System Engine date: July 1995 file: fuzzy.h author : Brian 0. Bush components: fuzzy.c main, initializations and just about everything fuzzy.h global variables, constants and structures language: Metrowerks v1.3 C for the PowerPC 60x RISC processors system: My Macintosh PowerPC Performa 6115CD purpose: Research for my Thesis

Copyright O 1996 by Brian 0. Bush, all rights reserved ------* /

//------/ / // FuzzySet defines the structure of a fuzzy set / / struct FuzzySet i short Label ; double a,b,c,d;// four points define a trapezoidal fuzzy interval struct FuzzySet *next; 1 ; typedef struct FuzzySet FuzzySet; typedef FuzzySet *FuzzySetPtr;

/ / // Rule definition / / //------

struct Rule { short Label ; double Truth; // truth value of the antecedent FuzzySetPtr If; // "antecedent" -up to m inputs FuzzySetPtr Then; / / "consequentn -only 1 output struct Rule *next; I; typedef struct Rule Rule; typedef Rule *RulePtr;

/ / // Testing Data

struct Testingdata { double Input[kMaxSystemInputsl; double Output ; } ; struct Testingdata gTestingData[kMaxTestVectorsl; //------. // EOF title: Test Gen-A random point generator for the fifth-degree polynomial. date: November 1995 file: test.c author : Brian 0. Bush components: itself compiler: Metrowerks v1.3 C for the PowerPC 60x RISC processors system: My Macintosh PowerPC Performa 6115CD purpose: Research for my Thesis

Copyright @ 1996 by Brian 0. Bush, all rights reserved ...... * /

short gSeed=4;

double RandomZeroOneO; double Function(doub1e x);

//------/ / // RandomZeroOne () Generates random value between 0 and 1 / / //------

double RandomZeroOne ( ) ( static long ixl, ix2, ix3; static double r[981 ; double temp ; static short iff = 0; short j; if (gSeed< 0 /I iff == 0) ( iff = 1; ixl = (IC1 - (gSeed)) % MI; ixl = (IA1 * ixl + IC1) % MI; ix2 = ixl % M2; ixl = (IA1 * ixl + IC1) % MI; ix3 = ixl % M3; for (j = 1; j <= 97; j++) i ixl = (IA1 * ixl + IC1) % MI; ix2 = (IA2 * ix2 + IC2) % M2; r[j] = (ixl + ix2 * RM2) * RM1;

1 ixl = (IA1 * ixl + IC1) % MI; ix2 = (IA2 * ix2 + IC2) % M2; ix3 = (IA3 * ix3 + IC3) % M3; j = 1 + ((97 * ix3) / M3); if (j > 97 11 j < 1) printf("Error: random ... this cannot happen!"); temp = r[jl; r[j] = (ixl + ix2 * RM2) * RM1; return (MIN + (MAX - MIN) * temp); I //------/ / // Fifth-Degree Polynomial Function / / double ~unction(doub1e x) {

/ / // Main / / void main() short i, iterations, counter; double y=O .O, x=o . 0 ; FILE *fp; char filename [ 3 1 I ; printf("\nFifth-degree polynomial random point generator.\nEnter filename:"); scanf("%s",filename);

if ((fp = fopen(filename,"wU))== NULL ) { printf ("\nError: cannot open data file for writing'') ; exit (1); 1

printf ( " \nloading training data from ' %s ' . " , filename); printf("\nEnter random number generator seed:"); scanf("%hd",&gSeed); printf("\nEnter number of data points required:"); scanf("%hd",&iterations); for(i=O; kiterations; i++) { if(i!=O) fprintf(fp,"\nU); x = 2*RandomZeroOneO; if(RandomZeroOne0 >= .5) x = -x; // change sign y = Function(x); fprintf(fp, "%2.41g\t%2.41gn,x,y); 1

//------// EOF Fuzzy SYSTEMDEFINITION

The following is a fuzzy system definition format for a n-input fuzzy system

composed of r rules.

[Number of System Inputs nl [Number of Rules rl [Span of Input 11 [Granularity of Input 11 [Span of Input 21 [Granularity of Input 21 [Span of Input nl [Granularity of Input nl

[Rule 1 label] [input set 1 label] [input set definition a,b,c,dl [input set 2 label] [input set definition a,b,c,dl [input set n label] [input set definition a,b, c, dl [output set label] [output set definition a,b, c,dl [Rule 2 label] [input set 1 label] [input set definition a,b,c,dl [input set 2 label] [input set definition a,b,c,dl ... [input set n label] [input set definition a,b, c,dl [output set label] [output set definition a,b, c,dl

[Rule r label] [input set 1 label] [input set definition a,b, c,dl [input set 2 label] [input set definition a,b, c,dl ... [input set n label] [input set definition a,b, c,dl [output set label] [output set definition a,b, c, dl