Formal Concept Analysis Part I Radim BELOHLˇ AVEK´ Dept. Computer Science Palacky University, Olomouc [email protected] Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 1 / 111 Introduction to Formal Concept Analysis (FCA) Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 2 / 111 Introduction to Formal Concept Analysis { Formal Concept Analysis (FCA) = method of analysis of tabular data (Rudolf Wille, TU Darmstadt), { alternatively called: concept data analysis, concept lattices, Galois lattices, . { used for data mining, knowledge discovery, preprocessing data { input: objects (rows) × attributes (columns) table y1 y2 y3 y1 y2 y3 x1 1 1 1 x1 XXX 1 1 1 x2 1 0 1 or x2 XX or 1 0 1 0 1 1 x3 0 1 1 x3 XX ... ... ... ... Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 3 / 111 Introduction to Formal Concept Analysis { output: 1 hierarchically ordered collection of clusters: { called concept lattice, { clusters are called formal concepts, { hierarchy = subconcept-superconcept, 2 data dependencies: { called attribute implications, { not all (would be redundant), only representative set Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 4 / 111 Output 1: Concept Lattices input data: output concept lattice: y1 y2 y3 x1 XXX x2 XX x3 XX concept lattice = hierarchically ordered set of clusters cluster (formal concept) = hA; Bi, A = collection of objects covered by cluster, B = collection of attributes covered by cluster, example of formal concept: hfx1; x2g; fy1; y3gi, clusters = nodes in the Hasse diagram, Hasse diagram = represents partial order given by subconcept-superconcept hierarchy concept lattice = all potentially interesting concepts in data Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 5 / 111 Output 2: Attribute Implications input data: attribute implications: y1 y2 y3 A ) B like x1 XXX fy2g ) fy3g, fy1; y2g ) fy3g, x2 XX but not fy1g ) fy2g, x3 XX attribute implication = particular data dependency, large number of attribute implications may be valid in given data, some of them redundant and thus not interesting (fy2g ) fy2g), reasonably small non-redundant set of attribute dependencies (non-redundant basis), connections to other types of data dependencies (functional dependencies from relational databases, association rules). Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 6 / 111 History of FCA Port-Royal logic (traditional logic): formal notion of concept Arnauld A., Nicole P.: La logique ou l'art de penser, 1662 (Logic Or The Art Of Thinking, CUP, 2003): concept = extent (objects) + intent (attributes) G. Birkhoff (1940s): work on lattices and related mathematical structures, emphasizes applicational aspects of lattices in data analysis. Barbut M., Monjardet B.: Ordre et classification, algbre et combinatoire. Hachette, Paris, 1970. Wille R.: Restructuring lattice theory: an approach based on hierarchies of concepts. In: I. Rival (Ed.): Ordered Sets. Reidel, Dordrecht, 1982, pp. 445{470. Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 7 / 111 Literature on FCA books Ganter B., Wille R.: Formal Concept Analysis. Springer, 1999. Carpineto C., Romano G.: Concept Data Analysis. Wiley, 2004. conferences ICFCA (Int. Conference of Formal Concept Analysis), Springer LNCS, http://www.isima.fr/icfca07/ CLA (Concept Lattices and Their Applications), http://www.lirmm.fr/cla07/index.htm ICCS (Int. Conference on Conceptual Structures), Springer LNCS, http://www.iccs.info/ conferences with focus on data analysis, information sciences, etc. web keywords: formal concept analysis, concept lattice, attribute implication, concept data analysis, Galois lattice Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 8 / 111 Selected Applications of FCA clustering and classification (conceptual clustering), information retrieval, knowledge extraction (structured view on data, structured browsing), machine learning, software engineering G. Snelting, F. Tip: Understanding class hierarchies using concept analysis. ACM Trans. Program. Lang. Syst. 22(3):540{582, May 2000. U. Dekel, Y. Gill: Visualizing class interfaces with formal concept analysis. In OOPSLA'03, pp. 288{289, Anaheim, CA, October 2003. preprocessing method: e.g., Zaki M.: Mining non-redundant association rules. Data Mining and Knowl. Disc. 9(2004), 223{248. closed frequent itemsets instead of frequent itemsets ) non-redundant association rules (<< number) mathematics (new results in math. structures related to FCA) Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 9 / 111 State of the art of FCA Ganter, B., Stumme, G., Wille, R. (Eds.): Formal Concept Analysis Foundations and Applications. Springer, LNCS 3626, 2005, development of theoretical foundations, development of algorithms, applications: increasingly popular (information retrieval, software engineering, social networks, . ), FCA as method of data preprocessing, interaction with other methods of data analysis, several software packages available. Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 10 / 111 Concept Lattices Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 11 / 111 What is a concept? central notion in FCA = formal concept but what is a concept? many approaches, including: psychology (approaches: classical, prototype, exemplar, knowledge) Murphy G. L.: The Big Book of Concepts. MIT Press, 2004. Margolis E., Laurence S.: Concepts: Core Readings. MIT Press, 1999. logic (rare, but Transparent Intensional Logic) Tichy P.: The Foundations of Frege's Logic. W. De Gryuter, 1988. Materna P.: Conceptual Systems. Logos Verlag, Berlin, 2004. artificial intelligence (frames, learning of concepts) Michalski, R. S., Bratko, I. and Kubat, M. (Eds.), Machine Learning and Data Mining: Methods and Applications, London, Wiley, 1998. conceptual graphs (Sowa) Sowa J. F.: Knowledge Representation: Logical, Philosophical, and Computational Foundations. Course Technology, 1999. \conceptual modeling", object-oriented paradigm, . traditional/Port-Royal logic Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 12 / 111 Traditional (Port-Royal) view on concepts The notion of a concept as used in FCA | inspired by Port-Royal logic (traditional logic): Arnauld A., Nicole P.: La logique ou l'art de penser, 1662 (Logic Or The Art Of Thinking, CUP, 2003): concept (according to Port-Royal) := extent + intent extent = objects covered by concept intent = attributes covered by concept example: DOG (extent = collection of all dogs (foxhound, poodle, . ), intent = fbarks, has four limbs, has tail,. g) concept hierarchy subconcept/superconcept relation DOG ≤ MAMMAL ≤ ANIMAL concept1=(extent1,intent1) ≤ concept2=(extent2,intent2) , extent1 ⊆ extent2 (, intent1 ⊇ intent2) Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 13 / 111 Formal Contexts (Tables With Binary Attributes) Definition (formal context (table with binary attributes)) A formal context is a triplet hX ; Y ; I i where X and Y are non-empty sets and I is a binary relation between X and Y , i.e., I ⊆ X × Y . interpretation: X . set of objects, Y . set of attributes, hx; yi 2 I . object x has attribute y formal context can be represented by table (table with binary attributes) hx; yi 2 I ... ×in table, hx; yi 62 I . blank in table, I y1 y2 y3 y4 x1 × × × × x2 × × × x3 × × × x4 × × × x5 × Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 14 / 111 Concept-forming Operators " and # Definition (concept-forming operators) For a formal context hX ; Y ; I i, operators " : 2X ! 2Y and # : 2Y ! 2X are defined for every A ⊆ X and B ⊆ Y by A" = fy 2 Y j for each x 2 A : hx; yi 2 I g; B# = fx 2 X j for each y 2 B : hx; yi 2 I g: operator ": assigns subsets of Y to subsets of X , A" . set of all attributes shared by all objects from A, operator #: assigns subsets of X to subsets of Y , B" . set of all objects sharing all attributes from B. To emphasize that " and # are induced by hX ; Y ; I i, we use "I and #I . Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 15 / 111 Concept-forming Operators " and # Example (concept-forming operators) For table I y1 y2 y3 y4 x1 × × × × x2 × × × x3 × × × x4 × × × x5 × we have: " " fx2g = fy1; y3; y4g, fx2; x3g = fy3; y4g, " fx1; x4; x5g = ;, X " = ;, ;" = Y , # # fy1g = fx1; x2; x5g, fy1; y2g = fx1g, # # fy2; y3g = fx1; x3; x4g, fy2; y3; y4g = fx1; x3; x4g, # # ; = X , Y = fx1g: Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 16 / 111 Formal Concepts Definition (formal concept) A formal concept in hX ; Y ; I i is a pair hA; Bi of A ⊆ X and B ⊆ Y such that A" = B and B# = A. A . extent of hA; Bi, B . extent of hA; Bi, verbal description: hA; Bi is a formal concept iff A contains just objects sharing all attributes from B and B contains just attributes shared by all objects from A, mathematical description: hA; Bi is a formal concept iff hA; Bi is a fixpoint of h"; #i. Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 17 / 111 Formal Concepts Example (formal concept) For table I y1 y2 y3 y4 x1 × × × × x2 × × × x3 × × × x4 × × × x5 × the highlighted rectangle represents formal concept hA1; B1i = hfx1; x2; x3; x4g; fy3; y4gi because " fx1; x2; x3; x4g = fy3; y4g, # fy3; y4g = fx1; x2; x3; x4g. Radim Belohlavek (UP Olomouc) Formal Concept Analysis 2011 18 / 111 Example (formal concept (cntd.)) But there are further formal concepts: I y1 y2 y3 y4 I y1 y2 y3 y4 x1 × × × × x1 ×× × × x2 × × × x2 × × × x3 × × × x3 × × × x4 × × × x4 × × × x5 × x5 × I y1 y2 y3 y4 x1 ×××× x2 × ×× x3 × × × x4 × × × x5 × i.e., hA2; B2i = hfx1;