Symbolic AI for XAI: Evaluating LFIT Inductive Programming for Fair and Explainable Automatic Recruitment Alfonso Ortega, Julian Fierrez, Aythami Morales, Zilong Wang School of Engineering, Universidad Autonoma de Madrid falfonso.ortega,julian.fierrez,[email protected], [email protected] Tony Ribeiro Laboratoire des Sciences du Numerique´ de Nantes National Institute of Informatics Japan tony [email protected] Abstract learning process. A deep review about this question can be found in [2]. Machine learning methods are growing in relevance for Another characteristic of these machine learning algo- biometrics and personal information processing in domains rithms is that they rely on data, and therefore reflect those such as forensics, e-health, recruitment, and e-learning. In data. This could be an advantage in general, but in some these domains, white-box (human-readable) explanations of particular domains it could be an important drawback. Con- systems built on machine learning methods can become cru- sider, for example, automatic recruitment systems, or algo- cial. Inductive Logic Programming (ILP) is a subfield of rithms to authorize financial products. In these domains, symbolic AI aimed to automatically learn declarative the- ethic behavior is mandatory and biased data are unaccept- ories about the process of data. Learning from Interpreta- able. Appropriate measures have to be taken for guaran- tion Transition (LFIT) is an ILP technique that can learn teeing ethical AI behavior sometimes contradictory to the a propositional logic theory equivalent to a given black- possibly biased training data. These questions are receiving box system (under certain conditions). The present work increasing interest [1, 9, 25, 39, 40, 20]. takes a first step to a general methodology to incorporate On the other hand, logic programming is a declarative accurate declarative explanations to classic machine learn- programming paradigm with a high level of abstraction. It ing by checking the viability of LFIT in a specific AI ap- is based on a formal model (first order logic) to represent plication scenario: fair recruitment based on an automatic human knowledge. Inductive Logic Programming (ILP) tool generated with machine learning methods for ranking has been developed for inductively learning logic programs Curricula Vitae that incorporates soft biometric informa- from examples [22]. Roughly speaking, given a collection tion (gender and ethnicity). We show the expressiveness of of positive and negative examples and background knowl- LFIT for this specific problem and propose a scheme that edge, ILP systems learn declarative (symbolic) programs can be applicable to other domains. [24, 6], which could even be noise tolerant [7, 23], that en- arXiv:2012.00360v1 [cs.AI] 1 Dec 2020 tail all of the positive examples but none of the negative examples. 1. Introduction One of the ILP most promising approaches for us is Learning From Interpretation Transition (LFIT) [30]. Statistical and optimization-based machine learning al- LFIT learns a logic representation (digital twin) of dy- gorithms have achieved great success in various applica- namical complex systems by observing their behavior as tions such as speech recognition [38], image classification a black box under some circumstances. The most general [8], machine translation [43], and so on. Among these ap- of LFIT algorithms is GULA (General Usage LFIT Algo- proaches, deep neural networks have shown the most re- rithm). PRIDE is an approximation to GULA with poly- markable success, especially in speech and image recog- nomial performance. GULA and PRIDE generate a propo- nition. Although deep learning methods usually have good sitional logic program equivalent to the system under con- generalization ability on similarly distributed new data, they sideration. These approaches will be introduced in depth in have some weaknesses including the lack of explanations the following sections. and the poor understandability by humans of the whole Figure 1 shows the architecture of our proposed ap- Classifier seen as black-box system (input/outputs) 1 Input features (variables) 풱 = v1, v2, v3 Output classes (target) = {0,1} v1 binary, v2 and v3 ∈ ℕ Examples (two inputs 풱퐴 and 풱퐵) Declarative explanation v1 = 0 v1 = 1 (propositional logic fragment) 풱퐴 = v2 = 5 풱퐵 = v2 = 3 LFIT (PRIDE) 3 v = 2 v = 0 Logical equivalent system 3 3 v 0 . (White-box Digital Twin of target(0) :- 1 the Black-box Classifier) target(1) :- v1 1 . 2 Note that the explanation reveals that the output only depends on v1 Figure 1: Architecture of the proposed approach for generating an explanation of a given black-box Classifier (1) using PRIDE (2) with a toy example (3). Note that the resulting explanations generated by PRIDE are in propositional logic. proach for generating white-box explanations using PRIDE automatically generating LISP expressions for given tasks of a given black-box classifier. expressed as pairs input/output. This is, in fact, a typical The main contributions of this work are: machine learning scenario. GP was extended by the use of formal grammars to allow to generate programs in any arbi- • We have proposed a method to provide declarative ex- trary language keeping not only syntactic correctness [27] planations using PRIDE about the classification pro- but also semantic properties [28]. Algorithms expressed cess made by an algorithm automatically learnt by a in any language are declarative versions of the concepts neural architecture in a typical machine learning sce- learnt what makes evolutionary automatic programming al- nario. Our approach guarantees the logical equivalence gorithms machine learners with good explainability. between the explanations and the algorithm with re- spect to the data used to feed PRIDE. It does not de- Declarative programming paradigms (functional, logi- pend on any particular characteristic of the domain, so cal) are as old as computer science and are implemented it could be applied to any problem. in multiple ways, e.g.: LISP [13], Prolog [5], Datalog [11], Haskell [41], and Answer Set Programs (ASP) [10]. • We have checked the expressive power of these ex- Of particular interest for us within declarative paradigms planations by experimenting in a multimodal machine is logic programming, and in particular first order logic pro- learning testbed around automatic recruitment includ- gramming, which is based on the Robinson’s resolution in- ing different biases (by gender and ethnicity). ference rule that automates the reasoning process of de- The rest of the paper is structured as follows: Sec. 2 sum- ducing new clauses from a first order theory [17]. Intro- marizes the related relevant literature. Sec. 3 describes our ducing examples and counter examples and combining this methodology including LFIT, GULA, and PRIDE. Sec. 4 scheme with the ability of extending the initial theory with presents the experimental framework including the datasets new clauses it is possible to automatically induce a new and experiments conducted. Sec. 5 presents our results. Fi- theory that (logically) entails all of the positive examples nally Secs. 6, 7 and 8 respectively discuss our work and but none of the negative examples. The underlying theory describe our conclusions and further research lines. from which the new one emerges is considered background knowledge. This is the hypothesis of Inductive Logic Pro- 2. Related Works: Inductive Programming for gramming (ILP, [21, 6]) that has received a great research XAI effort in the last two decades. Recently, these approaches have been extended to make them noise tolerant (in order Some meta-heuristics approaches (as genetic algorithms) to overcome one of the main drawbacks of ILP vs statisti- have been used to automatically generate programs. Ge- cal/numerical approaches when facing bad-labeled or noisy netic programming (GP) was introduced by Koza [15] for examples [23]). Other declarative paradigms are also compatible with ILP, e.g., MagicHaskeller (that implements [14]) with the functional programming language Haskell, and Inductive Learning of Answer Set Programs (ILASP) [16]. It has been previously mentioned that ILP implies some kind of search in spaces that can become huge. This search can be eased by hybridising with other techniques, e.g., [26] introduces GA-Progol that applies evolutive techniques. Within ILP methods we have identified LFIT as spe- cially relevant for explainable AI (XAI). In the next sec- tion we will describe the fundamentals of LFIT and its PRIDE implementation, which will be tested experimen- tally for XAI in the experiments that will follow. 2.1. Learning From Interpretation Transition (LFIT) Learning From Interpretation Transition (LFIT) [12] has been proposed to automatically construct a model of the dy- namics of a system from the observation of its state transi- tions. Given some raw data, like time-series data of gene expression, a discretization of those data in the form of state Figure 2: Experimental framework: PRIDE is fed with all transitions is assumed. From those state transitions, accord- the data available (train + test) for increasing the accuracy ing to the semantics of the system dynamics, several infer- of the equivalence. In our experiments we consider the clas- ence algorithms modeling the system as a logic program sifier (see [29] for details) as a black box to perform regres- have been proposed. The semantics of a system’s dynamics sion from input resume attributes (atts.) to output labels