sign in sign up
<<
Home , Data mining

for Mining Based Evolution

Algorithm for TSK-type Neural Fuzzy Systems Design

Sheng-Fuu Lin, Yi-Chang Cheng, Jyun-Wei Chang, and Yung-Chi Hsu Department of Electrical and Control Engineering, National Chiao Tung University Hsinchu, Taiwan, .O.C.

ABSTRACT membership functions of a fuzzy controller, with the fuzzy rule This paper proposed a TSK-type neural fuzzy system set assigned in advance. Juang [5] proposed the CQGAF to (TNFS) with reinforcement learning for data mining based simultaneously design the number of fuzzy rules and free evolution (R-DMEA). In R-DMEA, the group-based parameters in a fuzzy system. In [6], the group-based symbiotic symbiotic evolution (GSE) is adopted that each group represents evolution (GSE) was proposed to solve the problem that all the the set of single fuzzy rule. The R-DMEA contains both fuzzy rules were encoded into one chromosome in the structure learning and parameter learning. In the structure traditional . In the GSE, each chromosome learning, the proposed R-DMEA uses the two-step self-adaptive represents a single rule of fuzzy system. The GSE uses algorithm to decide the suitable number of rules in a TNFS. In group-based population to evaluate the fuzzy rule locally. parameter learning, the R-DMEA uses the mining-based Although GSE has a better performance when compared with selection strategy to decide the selected groups in GSE by the tradition symbiotic evolution, the combination of groups is data mining method called frequent pattern growth. Illustrative fixed. example is conducted to show the performance and applicability Although the above evolution learning [4]-[6] of R-DMEA. can improve the traditional genetic algorithms, these algorithms may encounter one or more of the following problems: 1) all the Keywords : Neural fuzzy system, control, group-based fuzzy rules are encoded into one chromosome; 2) the symbiotic evolution, reinforcement learning, FP-Growth. of fuzzy rules have to be assigned in advance; 3) the population cannot evaluate each fuzzy rule locally. In this paper, a TSK-type neural fuzzy system (TNFS) 1. INTRODUCTION with reinforcement learning for data mining based evolution Recently, a fuzzy system used for control problems has algorithm (R-DMEA) is proposed to solve the aforementioned become a popular research topic. A fuzzy system consists of a problems. The R-DMEA uses a two-step self-adaptive algorithm set of fuzzy if-then rules. Conventionally, the selection of fuzzy (TSSA) to determine the number of fuzzy rules automatically if-then rules often relies on a substantial amount of heuristic and processes the variable combination of chromosomes applied observations. Obviously, it is difficult for human experts to to the GSE. After the TSSA determines the number of fuzzy examine all the input-output data from a complex system to find rules (in this paper, the number of groups for GSE), R-DMEA proper rules. To cope with this difficulty, several supervised uses a data mining-based selection strategy (MSS) to decide the learning approaches that generate if-then rules from numerical groups selected to construct a TNFS. Data mining is a method data have been proposed [1]. of mining information in the called transactions. One For some real-world applications, precise training data goal of data mining is to find association rules among frequent are usually difficult and to obtain. Therefore, there has been a item sets in transactions. Hang et al. [7] proposed frequent growing interest in reinforcement learning problems [2] and [3]. pattern growth (FP-Growth) to mine frequent patterns without Recently, the evolutionary method used for training the candidate generation. Since data mining successfully find the fuzzy model has become an important field. The evolutionary information within the large item sets, it should be useful to fuzzy model generates a fuzzy system by incorporating solve the problem of how to select groups to construct TNFS evolutionary learning procedure such as genetic algorithms with many different combinations of fuzzy rules. (GAs). Several genetic fuzzy models have been proposed in In the MSS, groups obtained from FP-Growth are [4]-[6]. In [4], Karr applied GAs to the design of the selected by using three different actions: normal, searching, and exploring actions. After executing MSS, the population can This work was supported in part by National Science Council, search better combination of individuals and simultaneously Taiwan, R.O.C., under Contract No. NSC 97-2221-E-009-091. examine whether the mined combination of individuals 3. A DATA MINING BASED EVOLUTION ALGORITHM encounter the local optimal condition. The DMEA consists of structure and parameter learning. The advantages of the R-DMEA are summarized as In the structure learning, the DMEA determines the number of follows: 1) the TSSA is proposed to decide the suitable number fuzzy rules automatically and processes the variable of rules; 2) the R-DMEA uses group-based symbiotic evolution combination of chromosomes. The length of combination of to evaluate the fuzzy rule locally; 3) the MSS is proposed to chromosomes denotes the number of rules sets forming a TNFS. select the suitable groups. The proposed DMEA adopts the GSE [6] for GAs evolution. In This paper is organized as follows. Section 2 introduces the the GSE [6], a population is divided to several groups. Each TNFS. The proposed data mining based evolution algorithm group represents a set of the chromosomes that belong to one (DMEA) is described in Section 3. Section 4 introduces the fuzzy rule. Each group will execute the reproduction and R-DMEA. Section 5 presents the results of simulation. The crossover operations in each generation independently. The conclusions are summarized in the last section. structure of the chromosome in DMEA is shown in Fig. 1(a). In DMEA, for solving the problem that the numbers of fuzzy rules 2. REVIEW A TSK-TYPE NEURO-FUZZY SYSTEM have to be assigned in advance, the two-step self-adaptive In this paper, a TSK-type fuzzy system with an R-DMEA algorithm (TSSA) is proposed. In TSSA, the building blocks is used to solve a nonlinear control problem. It is a five-layer (BBs) [8] determines the suitable length of groups combined network structure. The functions of the nodes in each layer are together and decides the number of selection times of each described as follows: length of combined groups applied to the TNFS. In Fig. 1(b), Layer1 (Input Node ): The node only transmits input values to the TSSA codes the vector into the building blocks layer 2. That is (BBs), Mk represents k rules that used to form a TNFS; V is a (1) = (1) M k ui x i

Layer2 (Membership Function Node ): For an external input xi, probability vector represents the suitability of a TNFS model the following Gaussian membership function is used: with k rules. In the TSSA, the maximum number of rules ( Mmax )

2 and the minimum number of rules ( M ) must be predefined to (1) −   min ui m ij  (2) u(2) =exp  −  ij σ 2  prevent the number of fuzzy rules from being generated beyond ij   a certain bound. where mij and σij are, respectively, the center and the width of In parameter learning, for selecting suitable groups to the Gaussian membership function of the jth term of the ith construct a TNFS, this study proposes the MSS to select k input variable xi. individuals to form a TNFS by FP-Growth [7]. The goal of Layer 3 (Rule Node ): The function of each rule is FP-Growth is to find the frequent patterns without candidate

(3)= (2) (3) generation. In MSS, FP-Growth is used to find the frequent uj∏ u ij i group sets from transactions, which means the collection of

Layer 4 (Consequent Node ): The input to a node in layer 4 is well-performed chromosomes selected from Mk groups. After combined with the output delivered from layer 3 and inputs of the frequent group sets have been found, the MSS uses three layer 1. actions to select from Mk groups k rules to form a TNFS. The n (4)= (3) + (4) three actions defined in this paper are normal, searching, and uj uw j(0 j∑ wx ij i ) i=1 exploring action. In DMEA, the coding structure of the where the summation is over all the inputs and wij are the chromosomes must be suitable for GSE. Fig. 1(c) describes the corresponding parameters of the consequent part. coding of a fuzzy rule. Layer 5 (Output Node ): Each node in this layer corresponds to one output variable. The node integrates all the actions recommended by layers 3 and 4 and acts as a defuzzifier with

Mk M k M k (4) (3) + ∑uj ∑ uw j(0 j ∑ wx ij i ) = = = (5) y= u (5) =j1 = j 1 i 1 Mk M k (3) (3) ∑uj ∑ u j j=1 j = 1 where Mk is the number of fuzzy rule.

value with k rules greater than Best _ Fitness minus a M k

predefined threshold value named ThreadFitnessvalues . Step 2. Determine the selection times according to the probability vectors of the BBs as follows:

Rp = (Selection _ Times (*) V /Total _Velocy ) Figure 1: (a) The structure of the chromosome in DMEA. (b) M k M k for = ⋅⋅⋅ (12) Coding the probability vector into the building blocks (BBs) in MMMk min, min+ 1 , , M max TSSA. (c) Coding a rule into a chromosome in DMEA. Rmax Total_ Velocy= ∑ V (13) M k =R Mk min The learning process of the DMEA in each group where Selection_Times represents total selection times in each involves four major operators: the TSSA, MMS, crossover generation; Rp is the selection times of Mk groups that use to strategy, and mutation strategy. The whole learning process is M k described step-by-step as follows: select k chromosomes for constructing a TNFS. a. Two-step self-adaptive algorithm (TSSA): Step 3. After step 2, the selection times of each number of rules After every group is initialized, this paper proposed forming TNFS are obtained. Then the Rp times are used to TSSA to decide the suitable selection times of each number of M k rules (in this paper the number of rules are between [ Mmax , select k chromosomes form Mk groups to construct a TNFS.

Mmin ]). The TSSA is a procedure that decides the selection times Step 4. In the proposed TSSA, for avoiding the selected suitable of Mk groups forming a TNFS with k rules. Mk groups falling in the local optima solution, the TSSA The TSSA determines the selection times for each M to k proposed two different actions to update the V . Different reproduce chromosomes. After the TSSA, the selection times of Mk more suitable Mk will increase and the selection times of actions are taken according to the following equations: unsuitable M groups decrease. The details of the TSSA are if Rp Accumulator≤ TSSATimes k M k (14) shown as follows: then do Step 1, Step 2,and Step 3 if Best _ Fitness= Best _ Fitness Step 0. Initialize the probability vectors of the BBs: g (15) then Accumulator= Accumulator + 1; V=0.5, where MMM = , + , ⋅⋅⋅ , M (6) Mk k min min 1 max > if Accumulator TSSATimes (16) Accumulator = 0. (7) then do Step 0 and Accumulator = 0;

Step 1. Update the probability vectors of the BBs according to where TSSATimes is a predefined value; Best_Fitness g the following equations: represent the best fitness value of the best combination of chromosomes in gth generation; Best_Fitness represent the V= V +( Upt _ value *λ ), if Avg ≤ fit  MMk k M k M k (8)  best fitness value of the best combination of chromosomes in V= V − ( Upt _ value *λ ), otherwise  Mk M k M k current generations. Eqs. (14)- (16) represent that if the fitness for = ⋅⋅⋅ MMMk min, min1+ , , M max doesn't change for a sufficient long generations, the suitable Mk M max groups may fall in the local optima solution. Avg=∑ fit/( M − M ) (9) M k max min = Mk M min b. The mining-based selection strategy (MSS):

M max = (10) After TSSA, the selection times of each rule numbers in Upt_ valueM fit M∑ fit M k k k = Mk M min a TNFS are decided. The DMEA then execute the selection if Fitness≥ ( Best _ Fitness − ThreadFitnessvalue ) Mk M k (11) step. This paper proposes the MSS to decide suitable groups. In then fit= fit + Fitness ; Mk M k M k the MSS, the groups are selected according to the groups that frequently yield the better performance. For finding the groups where V is the probability vector in the BBs and represents M k that frequently yield the better performance, the FP-Growth [7] the suitability of k rules; λ is a predefine threshold value; Avg is adopted. FP-Growth is used to find the frequent groups from represents the average fitness value in the whole population; a transaction (in this paper a transaction means the set of Mk group indexes that perform well). After the frequent group Best _ Fitness represents the best fitness value with k rules; M k indexes have been found, the MSS selects the Mk group indexes according to the frequent group indexes. For avoiding fit is the sum of fitness value with k rules when the fitness M k the groups that frequently appear from falling in the local optima solution, the MSS proposed three actions to select k To construct the FP-tree, the first step is to scan the chromosomes to form TNFS from Mk groups: normal, transactions and retrieve the frequent 1-group index in searching, and exploring actions. The detail of MSS is shown transactions. Then, the retrieved frequent 1-group indexes are as follows: ordered by descending order of their supports. The ordered list Step 0. The transactions are built as follows equation: is called the F-list. After obtaining the F-list, the next step is to ≥ − discard the infrequent 1-group indexes and sort the remaining if FitnessM Best _ Fitness M ThreadFitnessvalue k k 1-group indexes as the same order of F-list in each transaction. then Transaction [ i ]= TNFSRuleSet [ i ] j M k (17) The ordered transactions are then used to construct the FP-tree. where i= 1, 2, ⋅⋅⋅ , M ; k First, construct the prefix-tree by scanning each transaction. MMM= , , ⋅⋅⋅ , M ; k min min1+ max Each node of the prefix-tree is composed of node-name, count j= 1,2, ⋅⋅⋅ , TransactionNum ; of the frequent 1-group index, and node-link. In steps of scanning single transaction, call for one group index by turns where the Fitness represents the fitness value of TNFS with M k and then travel from current node to next node. If the k rules; ThreadFitnessvalue is the predefined value; node-name of linked node matches the group index, then TransactionNum is the total number of transactions; accumulate the count of linked node. Otherwise, create a new

Transaction j[i] represents the ith item in the jth transaction; and node with its node-name copied from current group index. Complete prefix-tree is then done after all transactions are TNFSRuleSet[ i ] represents ith group index in Mk group M k processed. Then the complete FP-tree is created by the indexes that are selected to form a TNFS with k rules. The combination of the prefix-tree and the header-table of the transactions have the form shown in Table 1. Every transaction frequent 1-group index. represents Mk group indexes. For example, as shown in Table 1, (b) FP-growth: the first transaction of the transaction set means the 3 rules are FP-growth algorithm is achieved by following steps: selected from 1st group, 4th group, and 8th group has a good construction of conditional group index base, constructing of performance. corresponding conditional FP-tree, mining frequent group Table 1: Transactions in a FP-Growth. indexes on conditional FP-tree, concatenation of suffix group Transaction index Group indexes index and frequent group indexes on conditional FP-tree. 1 1, 4, 8 Start from select each frequent 1-group index as a suffix 2 2, 4, 7, 10 group index, and find the corresponding set of paths connecting … … to the root in FP-tree. The set of prefix paths is called the TransactionNum 1, 3, 4, 6, 8, 9 conditional group index base. Then accumulate the count for Step 1. Normal action: each group index in the base to construct the conditional FP-tree After building up the transactions, MSS selects group of the corresponding suffix group index. After mining the according to different actions. If the action type is normal action, frequent group indexes on conditional FP-tree, FP-growth is MSS selects the group as following equation: completed by the concatenation of suffix group index with the if Accumulator≤ NormalTimes generated frequent group indexes. Frequent group indexes (18) then GroupIndex [ i ]= Random [1, PulationSize ] generated by FP-growth are then thrown into the pool that’s = ⋅⋅⋅ = ⋅⋅⋅ where i 1, 2, , MMMMk ; k min , min+ 1 , , M max ; named FrequentPool . The detail of FP-Growth can be found in where Accumulator is defined in Eq. (18); GroupIndex[i] [7]. Step 3. Select the group indexes according to different actions: represents the selected ith group from Mk groups and PopulationSize represents there are PopulationSize groups in (a) In the searching action, the group indexes are selected from the population. the frequent item as follows: Step 2. Find the frequent groups: if NormalTimes< Accumulator ≤ SearchingTimes then GroupIndex [ i ]= w , If the action is searching or exploring action, MSS uses (19) where FP-Growth to find frequent groups in transactions. The frequent w= Random [1, PulationSize ] and w ∈ FrequentItemSet [ q ]; groups are found according to the predefined Minimum_Support FrequentItemSet [ q ]= Random [ FrequentPool ]; = ⋅⋅⋅ which means the minimum fraction of the group index in q 1, 2, , FrequentPoolNum ; i= 1, 2, ⋅⋅⋅ , MMMM ; = , + , ⋅⋅⋅ , M . transactions. FP-Growth algorithm can be viewed as two parts: k k min min 1 max construction of the FP-tree and FP-Growth. where SearchingTimes is a predefined value providing guidance (a) Construction of FP-tree: to the searching action; FrequentPool represents the sets of frequent item set obtained from FP-Growth; FrequentPoolNum action according to the current input vector (environment state). presents the total number of sets in FrequentPool and The input to the TNFS is the state of the plant, and the output is FrequentItemSet [i] presents a frequent item set selected from a control action of the state, denoted by f. The only available

FrequentPool randomly. In the Eq. (19), if Mk is greater than the feedback is a reinforcement signal that notifies the TNFS only size of FrequentItemSet [i], remaining groups are selected by Eq. when a failure occurs. An accumulator is a performance (19). . It accumulates the number of time steps before a (b) In the exploring action, the group indexes are selected failure occurs and feedbacks as the fitness of the R-DMEA. according to the frequent item as follows: The fitness function is defined by: if SearchingTimes< Accumulator ≤ ExploringTimes Fitness Value (i) =TIME-STEP(i) (14) = then GroupIndex [ i ] w , (20) where TIME-STEP(i) represents how long the experiment is a where w= Random [1, PulationSize ] and w ∉ FrequentItemSet [ i ]; “success” with the ith population. FrequentItemSet [ i ]= Random [ FrequentPool ]; 5. ILLUSTRATED EXAMPLES i = 1, 2, ⋅⋅⋅ , MMM;= , M ,, ⋅⋅⋅ M . k k min min+ 1 max The simulation is the ball and beam system. The initial where ExploringTimes is a predefined value provides judgment parameters are given in Table 2, which are determined by to the exploring action. practical experimentation or trial-and-error test.

Step 4. After selecting Mk group indexes, the k chromosomes The beam is made to rotate in vertical plane by applying are selected from Mk group as follows: a torque at the center of rotation and the ball is free to roll along ChromosomeIndex[ i ]= q , the beam. where (21) Table 2: The initial parameters before training q= Random [1, N ]; c Parameters Value Parameters Value i= 1, 2, ⋅⋅⋅ , k ; PopulationSize 16 Crossover Rate 0.5 where Nc represents the number of chromosomes in each group; NC 10 Mutation Rate 0.3

ChromosomeIndex [i] represents the index of a chromosome that Selection_Times 280 [mmin, mmax ] [0, 2] select from ith group. NormalTimes 10 [σmin, σmax ] [0, 2] c. Crossover strategy: SearchingTimes 15 [wmin, wmax ] [-30, 30] In this paper, the two-point crossover strategy [9] is used ExploringTimes 20 to perform the crossover. d. Mutation strategy: The equations of the ball and beam system can be found In this paper, a uniform mutation [9] is adopted, and the in [10]. A control strategy is deemed successful if it can balance mutated gene is drawn randomly from the domain of the the ball and beam system for 100,000 time steps. In R-DMEA, corresponding variable. the fitness function is defined in Eq. (14). The ball and beam system fails when the beam deviates beyond a certain angle 4. A REINFORCEMENT LEARNING FOR DMEA (| θ |> 12 °) or the ball reaches the end of the beam ( | r |> 2m ). The reinforcement learning algorithm for DMEA is The parameters of this example are defined as: 1) introduced in this section. The reinforcement DMEA and its ThreadFitnessvalue =100; 2) TSSATimes =40; 3) training environment are shown in Fig. 2. The reinforcement Minimum_Support=TransactionNum /2. A total of thirty runs signal indicates whether a success or a failure occurs. were performed in this example. Each run started in the same initial state. After thirty runs, the average optima number of rules is 4. Fig. 3(a) shows the TNFS balances the ball at the 64th generation averagely. When the learning process stops, the best combination of groups found at the final generation are applied to the test run. The test phase runs thirty times. The successful controlled beam angle is shown in Fig. 3 (d). Fig. 3 (d) shows the result for the first 1,000 time steps. As shown in Fig. 3 (d), the R-DMEA in thirty runs successfully controls the Figure 2: Schematic diagram of the R-DMEA for the TNFS. ball and beam system. The results show trained TNFS has good ability in controlling the ball and beam balancing system. As shown in Fig. 2, the proposed R-GCSE consists of a TNFS, which acts as the control network to determine a proper proposed to select the suitable combinations of individuals. Simulations also shows that the proposed R-DMEA has a good performance in controlling the ball and beam system.

7. REFERENCES [1] G. G. Towell and J. W. Shavlik, Extracting refined rules from (a) (b) knowledge-based neural networks, , vol. 13, pp. 71-101, 1993. [2] A. G. Barto, R. S. Sutton, and C. W. Anderson, “Neuron like adaptive elements that can solve difficult learning control problem,” IEEE Trans. Systems Man Cybern. , vol. 13, no. 5, pp. 834-847, 1993. [3] H. R. Berenji and P. Khedkar, “Learning and tuning fuzzy

logic controllers through reinforcements,” IEEE Trans. (c) (d) Neural Networks , vol. 3, no. 5, pp. 724-740, 1992. Figure 3: The performance of time steps of (a) the R-DMEA, (b) R-SE, [4] C. L. Karr, “Design of an adaptive fuzzy logic controller and (c) R-GA and control results of the ball and beam balancing system using a genetic algorithm,” in Proceedings of the Fourth using the R-DMEA of (d) Angle of the beam. International Conference on Genetic Algorithms , pp. In this example, the R-DMEA is compared with the R-SE 450-457, 1991. [10] and R-GA [4]. Four rules are set to the R-SE and R-GA. [5] C. F. Juang, “Combination of online clustering and Q-value Fig. 3 (b)-(c) shows the [10] and [4] balanced the ball at the based GA for reinforcement fuzzy system design,” IEEE 217th and 386th generation averagely. This results shows the Trans. Fuzzy Systems , vol. 13, no. 3, pp.289-302, 2005. R-DMEA has better performance than [4] and [10] in [6] C. J. Lin and Y. J. Xu, “A Self-Adaptive Neural Fuzzy controlling the ball and beam system. Network with Group-Based Symbiotic Evolution and Its Table 3 shows the performance compared with various Prediction Applications,” Fuzzy Sets and Systems , vol. 157, existing models ([4], [5], [10], and [11]) in this example. From no. 8, pp. 1036-1056, 2006. Table 3, the proposed R-DMEA obtains better performance and [7] J. Han, J. Pei, and Y. Yin, “Mining frequent patterns without smaller CPU time. candidate generation,” in Proceeding on ACM-SIGMOD , Dallas, Tx., pp. 1-12, 2000. Table 3: Performance comparison of various existing models. [8] C. J. Lin and Y. C. Hsu, “Reinforcement Hybrid Method Mean Best Worst Evolutionary Learning for Recurrent Wavelet-Based Steps Time Steps Time Steps Time Neurofuzzy Systems,” IEEE Trans. Fuzzy Systems, vol. 15, [11] 697 50.26 52 10.73 912 74.52 no. 4, pp. 729-745, 2007. [4] 386 30.21 37 8.05 536 42.36 [9] O. Cordon, F. Herrera, F. Hoffmann, and L. Magdalena, [10] 217 28.51 21 6.87 358 39.91 “Genetic fuzzy systems evolutionary tuning and learning of [5] 148 21.51 18 5.27 264 34.21 fuzzy knowledge bases,” Advances in Fuzzy R-DMEA 121 14.29 14 3.04 218 26.71 Systems-Applications and Theory , vol. 19, NJ: World Scientific Publishing, 2001. 6. CONCLUSIONS [10] C. J. Lin and Y. J. Xu, “Efficient Reinforcement Learning In this paper, a TSK-type neural fuzzy system (TNFS) Through Dynamical Symbiotic Evolution for TSK-Type with reinforcement learning for data mining based evolution Fuzzy Controller Design,” International Journal of algorithm (R-DMEA) is proposed. The proposed R-DMEA has General Systems , vol. 34, no. 5, pp. 559-578, 2005 structure-and-parameter learning ability. That is, it can [11] D. E. Moriarty and R. Miikkulainen, “Efficient determine the suitable number of fuzzy rules and efficient tune reinforcement learning through symbiotic evolution,” the free parameters in the TNFS model. The proposed learning Machine Learning , vol. 22, pp. 11-32, 1996. method also processes variable lengths of the combination of chromosomes in several groups. The advantages of the proposed R-DMEA are summarized as follows: 1) the TSSA is proposed to decide the suitable number of rules; 2) the MSS is