Fuzzy Sets and Rough Sets

n Introduction n History and definition n Fuzzy Sets n Membership function n Fuzzy set operations n Rough Sets n Approximation n Reduction

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 “Fuzziﬁcaon is a kind of scienﬁc permisiveness; it tends to result in socially appealing slogans unaccompanied by the discipline of hard work.” R. E. Kalman, 1972

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Introduction “Number of things of the same kind, that belong together because they are similar or complementary to each other.” The Oxford English Dictionary Set (collections) of various objects of interest Set Theory: George Cantor (1893)

completely new, elegant approach to vagueness

Fuzzy Set an element can belong to a set to a degree k (0 ≤ k ≤ 1) Fuzzy Set theory: Lotfi Zadeh(1965)

another approach to vagueness imprecision is expressed by a boundary region of Rough Set a set Rough Set Theory: Zdzisaw Pawlak(1982)

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Lotfi Zadeh

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Introduction

n Early computer science • Not good at solving real problems • The computer was unable to make accurate inferences • Could not tell what would happen, give some preconditions • Computer always seemed to need more information

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Lotfi Zadeh n “Fuzzy Sets” paper published in 1965 n Comprehensive - contains everything needed to implement FL n Key concept is that of membership values: extent to which an object meets vague or imprecise properties n Membership function: membership values over domain of interest n Fuzzy set operations n Awarded the IEEE Medal of Honor in 1995

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 History for fuzzy sets and system

n First fuzzy control system, work done in 1973 with Assilian (1975) n Developed for boiler-engine steam plant n 24 fuzzy rules n Developed in a few days n Laboratory-based n Served as proof-of-concept

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Early European Researchers

Hans Zimmerman, Univ. of Aachen •Founded first European FL working group in 1975 •First Editor of Fuzzy Sets and Systems •First President of Int’l. Fuzzy Systems Association Didier Dubois and Henri Prade in France •Charter members of European working group •Developed families of operators •Co-authored a textbook (1980)

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Early U. S. Researchers

K. S. Fu (Purdue) and Azriel Rosenfeld (U. Md.) (1965-75) Enrique Ruspini at SRI •Theoretical FL foundations •Developed fuzzy clustering James Bezdek, Univ. of West Florida •Developed fuzzy pattern recognition algorithms •Proved fuzzy c-means clustering algorithm •Combined fuzzy logic and neural networks •Chaired 1st Fuzz/IEEE Conf. in 1992 and others •President of IEEE NNC 1997-1999 INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 The Dark Age

•Lasted most of 1980s •Funding dried up, in US especially “...Fuzzy logic is based on fuzzy thinking. It fails to distinguish between the issues specifically addressed by the traditional methods of logic, definition and statistical decision- making...” - J. Konieki (1991) in AI Expert •Symbolics ruled: “fuzzy” label amounted to the ‘kiss of death’

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Michio Sugeno

•Secretary of Terano’s FL working group, est. in 1972

•1974 Ph.D. dissertation: fuzzy measures theory

•Worked in UK

•First commercial application of FL in Japan: control system for water purification plant (1983)

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Other Japanese Developements

n 1st consumer product: shower head using FL circuitry to control temperature (1987)

n Fuzzy control system for Sendai subway (1987)

n 2d annual IFSA conference in Tokyo was turning point for FL (1987)

n Laboratory for Int’l. Fuzzy Engineering Research (LIFE) founded in Yokohama with Terano as Director, Sugeno as Leading Advisor in 1989.

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzzy Systems Theory and Paradigms

n Variation on 2-valued logic that makes analysis and control of real (non-linear) systems possible n Crisp “first order” logic is insufficient for many applications because almost all human reasoning is imprecise n fuzzy sets, approximate reasoning, and fuzzy logic issues and applications

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzziness is not probability

• Probability is used, for example, in weather forecasting • Probability is a number between 0 and 1 that is the certainty that an event will occur • The event occurrence is usually 0 or 1 in crisp logic, but fuzziness says that it happens to some degree • Fuzziness is more than probability; probability is a subset of fuzziness • Probability is only valid for future/unknown events • Fuzzy set membership continues after the event

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Probability

• Probability is based on a closed world model in which it is assumed that everything is known • Probability is based on frequency; Bayesian on subjectivity

• Probability requires independence of variables

• In probability, absence of a fact implies knowledge

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzzy Sets

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzzy Set Membership

•In fuzzy logic, set membership occurs by degree •Set membership values are between 0 and 1 •We can now reason by degree, and apply logical operations to fuzzy sets We usually write .

µA (x) = m or, the membership value of x in the fuzzy set A is m, where 0 ≤ m ≤1

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzzy Set Membership Functions

• Fuzzy sets have “shapes”: the membership values plotted versus the variable • Fuzzy membership function: the shape of the fuzzy set over the range of the numeric variable Can be any shape, including arbitrary or irregular Is normalized to values between 0 and 1 Often uses triangular approximations to save computation time

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzzy Sets Are Membership Functions

from Bezdek INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Representations of Membership Functions

⎧ 0 .50 1 ⎫ TAMPBP = ⎨ + + ⎬ ⎩1.75 1.95 2.15⎭ ⎧ 0 .5 1 .5 0 ⎫ Warm = ⎨ + + + + ⎬ ⎩50 60 70 80 90⎭

−( p−80)2 / 50 µ FAIR _ PRICE (p) = e

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Two Types of Fuzzy Membership Function

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Equality of Fuzzy Sets

• In traditional logic, sets containing the same members are equal: {A,B,C} = {A,B,C} • In fuzzy logic, however, two sets are equal if and only if all elements have identical membership values: {.1/A,.6/B,.8C} = {.1/A,.6/B,.8/C}

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzzy Union

• In traditional logic, all elements in either (or both) set(s) are included • In fuzzy logic, union is the maximum set membership value If mA(x) = 0.7 and mB(x) = 0.9 then mA∪B(x) = 0.9

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzzy Relations and Operators

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015

Summary: FUZZY SETS Membership function and Fuzzy set operations

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Examples on fuzzy concepts

Tom is rather tall, but Judy is short. If you are tall, than you are quite likely heavy.

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Examples on fuzzy concepts

• The description of a human characteristic such as healthy. • The classification of patients as depressed. • The classification of certain objects as large. • The classification of people by age such as old. • A rule for driving such as “if an obstacle is close, then brake immediately”.

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Concept and set

intension (内涵 )： attributes of the object concept

extension (外延 )： all of the objects defined by the concept（ set）

G. Cantor (1887) AaPa= {| ()}

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 If a fuzzy concept can be rigidly described by Cantor’s notion of sets or the bivalent (true/false or two-valued)…

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 模糊概念能否用Cantor集合来刻画？秃头悖论⼀位已经谢顶的⽼教授与他的学⽣争论他是否为秃头问题。教授：我是秃头吗？学⽣：您的头顶上已经没有多少头发，确实应该说是。教授：你秀发稠密，绝对不算秃头，问你，如果你头上脱落了⼀根头发之后，能说变成了秃头了吗？学⽣：我减少⼀根头发之后，当然不会变成秃头。教授：好了，总结我们的讨论，得出下⾯的命题： ‘如果⼀个⼈不是秃头，那么他减少⼀根头发仍不是秃头 ’，你说对吗？学⽣：对！教授：我年轻时代也和你⼀样⼀头秀发，当时没有⼈说我秃头，后来随着年龄的增⾼，头发⼀根根减少到今天的样⼦。但每掉⼀根头发，根据我们刚才的命题，我都不应该称为秃头，这样经有限次头发的减少，⽤这⼀命题有限次，结论是： ‘我今天仍不是秃头 ’。

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Postulate: If a man with n (a nature number) hairs is baldheaded, then so is a man with n+1 hairs.

Baldhead Paradox：Every man is baldheaded.

 Cause： due to the use of bivalent logic for inference, whereas in fact, bivalent logic does not apply in this case。

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzzy Sets：Membership Functions Fuzzy membership function: the shape of the fuzzy set over the range of the numeric variable > Can be any shape, including arbitrary or irregular > Is normalized to values between 0 and 1 > Often uses triangular approximations to save computation time

from Bezdek INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Crisp sets VS Fuzzy Sets

C={Lines longer than 4cm} C={Long lines}

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 For contiguous data:

C={MEN OLDER THAN 50 YEARS OLD} C={OLD MEN}

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Example of Fuzzification

Assume inside temperature is 67.5 F, change in temperature last five minutes is -1.6 F, and outdoor temperature is 52 F.

Now find fuzzy values needed for our four example rules:

For InTemp,

µ cool (67.5) = 0.25, µcomfortable (67.5) = 0.75, and µtoo _ warm (67.5) = 0.0

. For DeltaInTemp, µsmall _ negative(−1.6) = 0.8, µnear _ zero (−1.6) = 0.2, and µl arge _ positive (−1.6) = 0.0

For OutTemp, . µchilly (52) = 0.9

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzzy …… Crisp

•Fuzzy logic comprises fuzzy sets and approximate reasoning

• A fuzzy “fact” is any assertion or piece of information, and can have a “degree of truth”, usually a value between 0 and 1 • Fuzziness: “A type of imprecision which is associated with ... Classes in which there is no sharp transition from membership to non-membership” - Zadeh (1970)

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzziness ……probability

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzzy relations and operations

Realons：Equality and Containment

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Equality of Fuzzy Sets

* In traditional logic, sets containing the same members are equal: {A,B,C} = {A,B,C} * In fuzzy logic, however, two sets are equal if and only if all elements have identical membership values: {.1/A,.6/B,.8C} = {.1/A,.6/B,.8/C}

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzzy Containment

• In traditional logic, A ⊂ B if and only if all elements in A are also in B. • In fuzzy logic, containment means that the membership values for each element in a subset is less than or equal to the membership value of the corresponding element in the superset. • Adding a hedge can create a subset or superset.

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzzy Intersection

* In standard logic, the intersection of two sets contains those elements in both sets.

* In fuzzy logic, the weakest element determines the degree of membership in the intersection

If mA(x) = 0.5 and mB(x) = 0.3 then mA∩B(x) ≡ 0.3

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzzy Union

• In traditional logic, all elements in either (or both) set(s) are included * In fuzzy logic, union is the maximum set membership value

If mA(x) = 0.7 and mB(x) = 0.9 then mA∪B(x) = 0.9

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Fuzzy Complement

• In tradional logic, the complement of a set is all of the elements not in the set.

• In fuzzy logic, the value of the complement of a membership is (1 - membership_value)

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Examples: Intersecon, union, complement

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015

U={u1,u2,u3,u4,u5} A=0.2/u1+0.7/u2+1/u3+0.5/u5 B=0.5/u1+0.3/u2+0.1/u4+0.7/u5

A …?... B =

B =

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Rough Sets Rough Sets: Background • vagueness • boundary region approach(Gottlob Frege ) • existing of objects which cannot be uniquely classified to the set or its complement • another approach to vagueness • imprecision in the approach is expressed by a boundary region of a set • defined quite generally by means of topological operations, interior and closure, called approximations

lower approximaon upper approximaon

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Rough Sets: Introduction • Human knowledge about a domain is expressed by classification • Rough set theory treats knowledge as an ability to classify perceived objects into categories • Objects belonging to the same category are considered to be indistinguishable to each other. • The primary notions of rough set theory are the approximation space: lower and upper approximations of an object set • The lower approximation of an object set (S) is a set of objects surely belonging to S, while its upper approximation is a set of objects surely or possibly belonging to it • An object set defined through its lower and upper approximations is called a rough set INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Introducon

• Research on rough set theory and applications in China began in the middle 1990s. • Chinese researchers achieved many significant results on rough set theory and applications. • both the quality and quantity of Chinese research papers are growing very quickly • many topics being investigated by Chinese researchers: fundamental of rough set, knowledge acquisition, granular computing based on rough set,extended rough set models, rough logic, applications of rough set, et al.

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Basic Concepts 1. preliminary

• Knowledge • Indiscernibility Relation • lower and upper approximations

2. secondary

• Reduct • Indiscernibility Matrix • Attributes Significance

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Basic Concepts

PART I: preliminary

knowledge approximate space:

K=(U,R)

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Basic Concepts PART I: preliminary patients Attributes Decision Attribute Condition Attribute

Patient Headache Muscle-pain Temperature Flu p1 yes yes normal no p2 yes yes high yes p3 yes yes very high yes p4 no yes normal no p5 no no high no p6 no yes very high yes IS（Information System/Tables）

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Basic Concepts PART I: preliminary

Basic concepts of rough set theory : • lower approximation of a set X with respect to R : is the set of all objects, which can be for certain classified as X with respect to R (are certainly X with respect to R). • upper approximation of a set X with respect to R: is the set of all objects which can be possibly classified as X with respect to R (are possibly X in view of R). • boundary region of a set X with respect to R : is the set of all objects, which can be classified neither as X nor as not-X with respect to R.

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Basic Concepts PART I: preliminary

U Headache Temp. Flu U1 Yes Normal No U2 Yes High Yes The indiscernibility classes defined by U3 Yes Very-high Yes R = {Headache, Temp.} are U4 No Normal No {u1}, {u2}, {u3}, {u4}, {u5, u7}, {u6, u8}. U5 NNoo HHiiigghh NNoo U6 No Very-high Yes U7 NNoo HHiiigghh YYeess U8 No Very-high No

X1 = {u | Flu(u) = yes} X2 = {u | Flu(u) = no} = {u2, u3, u6, u7} = {u1, u4, u5, u8} RX1 = {u2, u3} RX2 = {u1, u4} R X1 = {u2, u3, u6, u7, u8, u5} R X2 = {u1, u4, u5, u8, u7, u6}

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Basic Concepts PART I: Lower preliminary & Upper Approximations (4)

R = {Headache, Temp.} U/R = { {u1}, {u2}, {u3}, {u4}, {u5, u7}, {u6, u8}} X1 = {u | Flu(u) = yes} = {u2,u3,u6,u7} X2 = {u | Flu(u) = no} = {u1,u4,u5,u8}

RX1 = {u2, u3} X1 X2 R X1 = {u2, u3, u6, u7, u8, u5} u2 u7 u5 u1

RX2 = {u1, u4} u6 u8 u4 R X2 = {u1, u4, u5, u8, u7, u6} u3

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Basic Concepts PART I: preliminary

|()|BX− • accuracy of approximation: αB ()X = |()|BX− where |X| denotes the cardinality of X ≠ φ.

Obviously 0 ≤αB ≤1.

If α B ( X ) = 1 , X is crisp with respect to B.

If α B ( X ) < 1 , X is rough with respect to B.

INTRODUCTIONDept. TO of COMPUTATIONAL Computer Science INTELLIGENCE, and Technology, Nanjing Nanjing University University Spring 2015 Basic Concepts PART II: secondary Reduct Patient Headache Muscle-pain Temperature Flu p1 yes yes normal no p2 yes yes high yes p3 yes yes very high yes p4 no yes normal no p5 no no high no p6 no yes very high yes BAÃ is a reduct of information system if Reducts: {Headache, Temperature} IND() B= IND () A or {Muscle-pain, Temperature} and no proper subset of B has this property Core: CORE(P)=∩RED(P)

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Basic Concepts PART II: secondary Attributes Reduct Patient Headache Muscle-pain Temperature Flu p 1 yes yes normal no p 2 yes yes high yes p 3 yes yes very high yes p 4 no yes normal no p 5 no no high no p 6 no yes very high yes

，if ，then S is the Reduct of D。 SP⊂ POSSP (D)=POS (D) 其中， ∈ POSP (D)= U P_(X) , X U/D POS{M ，T}={p1,p2,p3,p4,p5,p6}

Positive region

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Basic Concepts PART II: secondary

Patient Headache Muscle-pain Temperature Flu p1 yes yes normal no p2 yes yes high yes p3 yes yes very high yes p4 no yes normal no p5 no no high no p6 no yes very high yes

Patient Headache Temperature Flu Patie Muscle- Temperat Flu nt pain ure p1 no high yes p1 yes high yes p2 yes high yes p2 no high yes p3 yes very high yes p3 yes very high yes p4 no normal no p4 yes normal no p5 yes high no p5 no high no p6 no very high yes p6 yes very high yes

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Basic Concepts PART II: secondary Patient Headache Muscle-pain Temperature Flu

A.Skowron: p1 yes yes normal no Indiscernibility Matrix p 2 yes yes high yes p 3 yes yes very high yes p 4 no yes normal no ， M(S)=[cij]n×n p 5 no no high no ∈ ：， cij={a A a(xi)≠a(xj) i,j=1,2,…,n} p 6 no yes very high yes

p1 p2 p3 p4 p5 p6 H H,T p1 % T T T p2 % p3 % H,T H,M,T ％ p4 % ％ T p5 % M,T

p6INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University % Spring 2015 Basic Concepts PART II: secondary

Patient Headache Muscle-pain Temperature Flu p 1 yes yes normal no p 2 yes yes high yes p 3 yes yes very high yes p 4 no yes normal no p 5 no no high no p 6 no yes very high yes

headache muscle-pain temperature flu

Which is more important? Definition: σ (Headache) = 0, (γ (C,D)-γ (C-{a},D)) γ (C-{a},D) σ (a)= =1- (C,D) γ (C,D) γ (C,D) σ (Muscle-pain) = 0, σ (Temperature) = 0.75

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Theory and Applications

in the view of logic Theory in the view of algebra in the view of information theory

medical data analysis finance Applications voice recognition image processing …

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Advantages and Disadvantages

• Good at… discrete values uncertainty

Disadvantages: discrete values sensitive

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Trends and Challenges

• Models • Data • Algorithms • Application

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015 Some cases

• Classifications

INTRODUCTION TO COMPUTATIONAL INTELLIGENCE, Nanjing University Spring 2015